2024-05-10T10:23:24.630 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T10:23:24.630 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_dparmar@teuthology --name dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --verbose --archive /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909 -- /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/orig.config.yaml 2024-05-10T10:23:24.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T10:23:24.778 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909 branch: wip-63896-from-wip-rishabh-testing-20240501.193033 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: null first_in_suite: false job_id: '7700909' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-63896-from-wip-rishabh-testing-20240501.193033 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: d39fa08867836ac070c47cb66c782fa559fd480b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d39fa08867836ac070c47cb66c782fa559fd480b kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-63896-from-wip-rishabh-testing-20240501.193033 sha1: d39fa08867836ac070c47cb66c782fa559fd480b owner: scheduled_dparmar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1698 sha1: d39fa08867836ac070c47cb66c782fa559fd480b sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-63896-from-wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d39fa08867836ac070c47cb66c782fa559fd480b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK170pW49litN+5EbfmOAYDBcgpt4N1Jca05xmku3ldzuDjm5Yj2KxZb/DnlrBojCr2amhUs1bsPFiBxvhTOFfI= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPLvx9cv2KhJqM1O6dcHDgOYHlOVLnnfhQIv2H3uj3O9SvjWai9Cj26e45Cofe6oBhrPFOKX32Bn3RR8OcmV5Js= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_07:35:43 tube: smithi user: dparmar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T10:23:24.778 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa; will attempt to use it 2024-05-10T10:23:24.778 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa/tasks 2024-05-10T10:23:24.779 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T10:23:24.780 INFO:teuthology.task.internal:Checking packages... 2024-05-10T10:23:24.802 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd39fa08867836ac070c47cb66c782fa559fd480b' 2024-05-10T10:23:24.802 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T10:23:24.802 INFO:teuthology.packaging:ref: None 2024-05-10T10:23:24.802 INFO:teuthology.packaging:tag: None 2024-05-10T10:23:24.802 INFO:teuthology.packaging:branch: wip-63896-from-wip-rishabh-testing-20240501.193033 2024-05-10T10:23:24.802 INFO:teuthology.packaging:sha1: d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:23:24.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-63896-from-wip-rishabh-testing-20240501.193033 2024-05-10T10:23:25.016 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3449.gd39fa088 2024-05-10T10:23:25.017 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T10:23:25.026 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T10:23:25.026 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T10:23:25.040 INFO:teuthology.task.internal:Saving configuration 2024-05-10T10:23:25.057 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T10:23:25.067 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:23:25.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:18:22.699854', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK170pW49litN+5EbfmOAYDBcgpt4N1Jca05xmku3ldzuDjm5Yj2KxZb/DnlrBojCr2amhUs1bsPFiBxvhTOFfI='} 2024-05-10T10:23:25.114 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:18:22.698109', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPLvx9cv2KhJqM1O6dcHDgOYHlOVLnnfhQIv2H3uj3O9SvjWai9Cj26e45Cofe6oBhrPFOKX32Bn3RR8OcmV5Js='} 2024-05-10T10:23:25.114 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T10:23:25.125 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-10T10:23:25.125 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-10T10:23:25.125 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T10:23:25.206 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7c72751f70>, signals=[15]) 2024-05-10T10:23:25.206 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T10:23:25.217 INFO:teuthology.task.internal:Opening connections... 2024-05-10T10:23:25.217 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:23:25.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:23:25.300 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:23:25.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:23:25.382 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T10:23:25.390 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-05-10T10:23:25.408 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-05-10T10:23:25.408 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-05-10T10:23:25.466 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:23:25.467 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:23:25.468 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-05-10T10:23:25.490 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-05-10T10:23:25.508 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-05-10T10:23:25.509 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T10:23:25.566 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-05-10T10:23:25.567 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-05-10T10:23:25.567 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T10:23:25.567 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-05-10T10:23:25.567 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T10:23:25.567 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T10:23:25.567 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T10:23:25.567 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-05-10T10:23:25.590 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T10:23:25.600 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T10:23:25.607 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T10:23:25.607 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:23:25.610 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-05-10T10:23:25.626 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T10:23:25.634 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T10:23:25.634 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:23:25.668 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-05-10T10:23:25.757 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T10:23:25.809 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T10:23:25.824 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-10T10:23:25.825 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-10T10:23:25.825 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-10T10:23:25.825 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:23:25.825 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T10:23:25.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:23:25.842 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-05-10T10:23:25.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:23:25.843 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-05-10T10:23:25.899 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:23:25.899 INFO:teuthology.task.kernel:Running kernel on smithi135: 5.14.0-437.el9.x86_64 2024-05-10T10:23:25.900 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2024-05-10T10:23:25.900 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-437.el9.x86_64 2024-05-10T10:23:25.901 INFO:teuthology.task.kernel:Running kernel on smithi053: 5.14.0-437.el9.x86_64 2024-05-10T10:23:25.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2024-05-10T10:23:27.993 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-10T10:23:28.339 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-10T10:23:30.407 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-05-10T10:23:31.544 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-05-10T10:23:34.925 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-05-10T10:23:36.187 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.2 MB 00:00 2024-05-10T10:23:36.701 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-10T10:23:37.729 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-05-10T10:23:38.369 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-10T10:23:38.872 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-10T10:23:43.544 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-10T10:23:44.288 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-10T10:23:45.526 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:23:45.526 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:23:45.618 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:23:45.631 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout:Install 4 Packages 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 78 M 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 123 M 2024-05-10T10:23:45.632 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-05-10T10:23:45.846 INFO:teuthology.orchestra.run.smithi135.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.7 MB/s | 484 kB 00:00 2024-05-10T10:23:46.403 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-10T10:23:46.403 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-10T10:23:46.503 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-10T10:23:46.507 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-10T10:23:46.507 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-05-10T10:23:46.507 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-10T10:23:46.507 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-05-10T10:23:46.507 INFO:teuthology.orchestra.run.smithi053.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-10T10:23:46.507 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-05-10T10:23:46.508 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:23:46.509 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 78 M 2024-05-10T10:23:46.509 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 123 M 2024-05-10T10:23:46.509 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-05-10T10:23:46.748 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.7 MB/s | 484 kB 00:00 2024-05-10T10:23:46.863 INFO:teuthology.orchestra.run.smithi135.stdout:(2/4): kernel-modules-core-5.14.0-444.el9.x86_6 28 MB/s | 28 MB 00:01 2024-05-10T10:23:47.039 INFO:teuthology.orchestra.run.smithi135.stdout:(3/4): kernel-core-5.14.0-444.el9.x86_64.rpm 11 MB/s | 15 MB 00:01 2024-05-10T10:23:47.132 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 28 MB/s | 15 MB 00:00 2024-05-10T10:23:47.514 INFO:teuthology.orchestra.run.smithi135.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 18 MB/s | 34 MB 00:01 2024-05-10T10:23:47.515 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:23:47.515 INFO:teuthology.orchestra.run.smithi135.stdout:Total 41 MB/s | 78 MB 00:01 2024-05-10T10:23:47.648 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 31 MB/s | 28 MB 00:00 2024-05-10T10:23:48.100 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 22 MB/s | 34 MB 00:01 2024-05-10T10:23:48.100 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:23:48.100 INFO:teuthology.orchestra.run.smithi053.stdout:Total 49 MB/s | 78 MB 00:01 2024-05-10T10:23:48.192 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-05-10T10:23:48.518 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-05-10T10:23:48.518 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-05-10T10:23:48.807 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-05-10T10:23:49.229 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-05-10T10:23:49.229 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-05-10T10:23:49.459 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-05-10T10:23:49.459 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-05-10T10:23:50.174 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-05-10T10:23:50.174 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-05-10T10:23:51.886 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-05-10T10:23:52.519 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:23:52.602 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:23:52.784 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-05-10T10:23:53.448 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:23:53.532 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:23:55.264 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:23:55.416 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:23:56.360 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:23:56.486 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:24:00.164 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:24:00.269 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:24:01.614 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:05.034 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:06.726 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:34.661 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:34.699 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:34.982 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:34.982 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:24:34.982 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:24:34.982 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:24:35.764 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:35.764 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:24:35.764 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-05-10T10:24:35.764 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:24:35.765 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:24:35.765 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:24:35.765 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:24:35.765 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:24:35.765 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-05-10T10:24:36.130 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:24:36.213 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:24:36.213 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:24:36.214 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:24:36.214 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:24:36.214 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:24:36.214 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-10T10:24:36.214 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi135.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:24:36.214 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:24:36.264 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:24:36.265 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:24:36.265 DEBUG:teuthology.orchestra.run.smithi135:> sudo rpm -qi grub2-tools 2024-05-10T10:24:36.295 INFO:teuthology.orchestra.run.smithi135.stdout:Name : grub2-tools 2024-05-10T10:24:36.295 INFO:teuthology.orchestra.run.smithi135.stdout:Epoch : 1 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Version : 2.06 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Release : 68.el9 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Architecture: x86_64 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Group : Unspecified 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Size : 8264254 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:License : GPLv3+ 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:24:36.296 INFO:teuthology.orchestra.run.smithi135.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:Packager : builder@centos.org 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:Vendor : CentOS 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:Summary : Support tools for GRUB. 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:Description : 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout:hardware devices. 2024-05-10T10:24:36.297 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:24:36.298 INFO:teuthology.orchestra.run.smithi135.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:24:36.298 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:24:36.299 INFO:teuthology.task.kernel:Updating grub on smithi135 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:24:36.299 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:24:36.919 INFO:teuthology.orchestra.run.smithi135.stderr:Generating grub configuration file ... 2024-05-10T10:24:37.727 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:37.768 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:38.093 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:38.093 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-10T10:24:38.094 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-10T10:24:38.094 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-10T10:24:38.370 INFO:teuthology.orchestra.run.smithi135.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:24:38.406 INFO:teuthology.orchestra.run.smithi135.stderr:done 2024-05-10T10:24:38.408 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-05-10T10:24:38.424 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.QoYQ8mbIZl 2024-05-10T10:24:38.424 DEBUG:teuthology.orchestra.run.smithi135:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QoYQ8mbIZl 2024-05-10T10:24:38.499 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.QoYQ8mbIZl 2024-05-10T10:24:38.662 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.QoYQ8mbIZl is 6KB 2024-05-10T10:24:38.689 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.QoYQ8mbIZl 2024-05-10T10:24:38.716 DEBUG:teuthology.orchestra.run.smithi135:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:24:38.771 INFO:teuthology.orchestra.run.smithi135.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:24:38.771 INFO:teuthology.orchestra.run.smithi135.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:24:38.771 INFO:teuthology.orchestra.run.smithi135.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:24:38.771 INFO:teuthology.orchestra.run.smithi135.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:24:38.773 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:24:38.871 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-10T10:24:38.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo shutdown -r now 2024-05-10T10:24:39.229 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:24:39.306 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:24:39.307 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:24:39.307 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-10T10:24:39.307 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-10T10:24:39.307 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-10T10:24:39.307 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-10T10:24:39.307 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-05-10T10:24:39.307 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T10:24:39.357 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-10T10:24:39.358 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-10T10:24:39.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 68.el9 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2024-05-10T10:24:39.396 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8264254 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-05-10T10:24:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-05-10T10:24:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:24:39.398 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T10:24:39.398 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T10:24:39.398 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T10:24:39.398 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-05-10T10:24:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:24:39.398 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T10:24:39.399 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T10:24:39.399 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 5.14.0-444.el9.x86_64 2024-05-10T10:24:39.399 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T10:24:40.016 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-05-10T10:24:41.369 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T10:24:41.395 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-05-10T10:24:41.397 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-05-10T10:24:41.413 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.flZzQYLMli 2024-05-10T10:24:41.413 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.flZzQYLMli 2024-05-10T10:24:41.480 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.flZzQYLMli 2024-05-10T10:24:41.623 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.flZzQYLMli is 6KB 2024-05-10T10:24:41.671 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.flZzQYLMli 2024-05-10T10:24:41.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T10:24:41.749 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T10:24:41.749 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T10:24:41.749 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T10:24:41.749 INFO:teuthology.orchestra.run.smithi053.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-10T10:24:41.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-10T10:24:41.910 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-05-10T10:25:08.936 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-10T10:25:08.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-05-10T10:25:08.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:25:11.916 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-10T10:25:11.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-10T10:25:11.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:26:08.995 DEBUG:teuthology.orchestra.remote:timed out 2024-05-10T10:26:11.921 DEBUG:teuthology.orchestra.remote:timed out 2024-05-10T10:26:18.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-05-10T10:26:18.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:26:18.433 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-05-10T10:26:18.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-05-10T10:26:18.864 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:26:18.864 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-05-10T10:26:18.919 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:26:18.920 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:26:18.920 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:26:18.920 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:26:19.921 DEBUG:teuthology.parallel:result is None 2024-05-10T10:26:20.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-10T10:26:20.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:26:21.304 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-05-10T10:26:21.668 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-10T10:26:21.668 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-10T10:26:21.668 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-05-10T10:26:21.685 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-444.el9.x86_64 2024-05-10T10:26:21.685 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-10T10:26:21.685 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T10:26:21.685 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T10:26:22.686 DEBUG:teuthology.parallel:result is None 2024-05-10T10:26:22.687 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T10:26:22.694 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T10:26:22.694 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:26:22.697 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T10:26:22.713 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T10:26:22.720 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T10:26:22.730 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T10:26:22.730 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:26:22.759 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T10:26:22.788 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T10:26:22.795 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T10:26:22.795 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:26:22.828 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T10:26:22.902 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:26:22.920 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:26:22.954 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:26:22.969 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T10:26:22.971 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T10:26:22.979 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T10:26:22.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:26:22.981 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T10:26:23.007 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T10:26:23.050 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T10:26:23.051 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:26:23.053 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T10:26:23.068 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:26:23.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:26:23.206 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:26:23.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:26:23.329 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:26:23.330 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:26:23.395 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:26:23.439 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T10:26:23.488 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:26:23.569 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T10:26:23.609 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:26:23.609 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T10:26:23.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-05-10T10:26:23.676 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-05-10T10:26:23.711 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:26:23.765 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:26:24.238 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T10:26:24.247 INFO:teuthology.task.internal:Starting timer... 2024-05-10T10:26:24.247 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T10:26:24.257 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T10:26:24.265 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-10T10:26:24.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-05-10T10:26:24.353 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-05-10T10:26:24.355 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-05-10T10:26:24.446 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-05-10T10:26:24.452 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T10:26:24.452 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-05-10T10:26:24.482 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-05-10T10:26:24.483 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-05-10T10:26:24.518 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-05-10T10:26:24.518 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2024-05-10T10:26:24.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:26:24.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:26:24.551 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:26:24.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:26:24.605 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T10:26:24.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:26:24.633 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-05-10T10:26:25.005 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T10:26:25.015 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-10T10:26:25.021 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-10T10:26:25.164 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T10:26:25.179 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T10:26:25.180 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T10:30:31.970 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2024-05-10T10:30:31.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-10T10:30:31.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:30:32.056 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-05-10T10:30:32.137 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-10T10:30:32.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-05-10T10:30:32.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T10:30:32.222 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-05-10T10:30:32.257 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-05-10T10:30:32.257 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T10:30:32.268 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T10:30:32.268 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T10:30:32.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:30:32.271 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T10:30:32.271 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:30:32.305 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T10:30:32.321 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T10:30:32.326 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T10:30:32.340 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T10:30:32.348 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-05-10T10:30:32.360 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-05-10T10:30:32.364 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-05-10T10:30:32.375 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-05-10T10:30:32.376 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T10:30:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T10:30:32.391 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T10:30:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T10:30:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:30:32.456 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:30:32.456 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-10T10:30:32.456 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.456 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.456 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.456 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.460 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:30:32.463 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:30:32.463 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-05-10T10:30:32.463 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.463 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.463 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.463 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:30:32.464 INFO:teuthology.run_tasks:Running task install... 2024-05-10T10:30:32.473 DEBUG:teuthology.task.install:project ceph 2024-05-10T10:30:32.473 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'}} 2024-05-10T10:30:32.473 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'} 2024-05-10T10:30:32.473 INFO:teuthology.task.install:Using flavor: default 2024-05-10T10:30:32.479 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T10:30:32.479 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T10:30:32.479 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T10:30:32.480 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:30:32.482 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'tag': None, 'wait_for_package': False} 2024-05-10T10:30:32.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:30:32.628 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T10:30:32.629 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T10:30:32.708 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/ 2024-05-10T10:30:32.708 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3449.gd39fa088 2024-05-10T10:30:32.745 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T10:30:32.745 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:30:32.745 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T10:30:32.773 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T10:30:32.773 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T10:30:32.821 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-63896-from-wip-rishabh-testing-20240501.193033/d39fa08867836ac070c47cb66c782fa559fd480b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T10:30:32.821 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:30:32.821 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T10:30:32.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T10:30:32.847 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T10:30:32.847 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d39fa08867836ac070c47cb66c782fa559fd480b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T10:30:32.913 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T10:30:32.919 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T10:30:32.985 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-05-10T10:30:32.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-05-10T10:30:32.988 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T10:30:33.052 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-05-10T10:30:33.054 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-05-10T10:30:33.232 INFO:teuthology.orchestra.run.smithi053.stdout:54 files removed 2024-05-10T10:30:33.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T10:30:33.284 INFO:teuthology.orchestra.run.smithi135.stdout:54 files removed 2024-05-10T10:30:33.314 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T10:30:33.980 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 213 kB/s | 84 kB 00:00 2024-05-10T10:30:34.026 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 175 kB/s | 84 kB 00:00 2024-05-10T10:30:34.372 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-10T10:30:34.460 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-10T10:30:34.732 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-10T10:30:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-10T10:30:36.865 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-10T10:30:39.603 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-10T10:30:42.328 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-10T10:30:44.158 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:06 2024-05-10T10:30:47.088 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-05-10T10:30:48.192 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-05-10T10:30:48.647 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-05-10T10:30:49.610 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 16 kB 00:00 2024-05-10T10:30:49.959 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-10T10:30:51.432 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-10T10:30:55.299 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-05-10T10:30:56.546 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-05-10T10:30:57.684 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:30:57.685 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:30:57.696 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T10:30:57.696 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T10:30:57.697 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T10:30:57.698 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T10:30:57.700 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T10:30:57.702 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T10:30:57.703 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T10:30:57.767 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-10T10:30:57.787 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-05-10T10:30:57.787 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T10:30:57.788 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T10:30:57.789 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T10:30:57.790 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-05-10T10:30:57.791 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T10:30:57.792 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:30:57.793 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T10:30:57.794 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T10:30:57.795 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T10:30:57.796 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T10:30:57.797 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T10:30:57.798 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T10:30:57.799 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:30:57.800 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-10T10:30:57.800 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-05-10T10:30:57.800 INFO:teuthology.orchestra.run.smithi053.stdout:Install 148 Packages 2024-05-10T10:30:57.800 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 8 Packages 2024-05-10T10:30:57.800 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:30:57.801 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 226 M 2024-05-10T10:30:57.801 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-05-10T10:30:58.684 INFO:teuthology.orchestra.run.smithi053.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-05-10T10:30:58.865 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:30:58.866 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T10:30:58.875 INFO:teuthology.orchestra.run.smithi053.stdout:(2/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 4.5 MB/s | 890 kB 00:00 2024-05-10T10:30:58.876 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T10:30:58.877 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T10:30:58.878 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T10:30:58.879 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T10:30:58.880 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T10:30:58.883 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T10:30:58.883 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T10:30:58.934 INFO:teuthology.orchestra.run.smithi053.stdout:(3/156): ceph-immutable-object-cache-19.0.0-344 2.4 MB/s | 145 kB 00:00 2024-05-10T10:30:58.946 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-05-10T10:30:58.960 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-05-10T10:30:58.960 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 6.6 k 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 890 k 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 145 k 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 1.5 M 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 142 k 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 3.5 M 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 7.4 M 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 49 k 2024-05-10T10:30:58.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 11 M 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 49 M 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 273 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 763 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 225 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 27 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 18 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 32 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 746 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 127 k 2024-05-10T10:30:58.962 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 161 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 321 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 298 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 99 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 85 k 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:30:58.963 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 170 k 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.4 M 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 3.1 M 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-10T10:30:58.964 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 21 M 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 34 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 2.4 M 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 274 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 4.7 M 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 17 M 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3449.gd39fa088.el9 ceph-noarch 16 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 25 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T10:30:58.965 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 163 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 500 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 5.4 M 2024-05-10T10:30:58.966 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T10:30:58.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 45 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.0.0-3449.gd39fa088.el9 ceph 135 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T10:30:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T10:30:58.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T10:30:58.970 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T10:30:58.971 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T10:30:58.972 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-10T10:30:58.973 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:30:58.974 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-05-10T10:30:58.974 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-05-10T10:30:58.974 INFO:teuthology.orchestra.run.smithi135.stdout:Install 148 Packages 2024-05-10T10:30:58.974 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 8 Packages 2024-05-10T10:30:58.974 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:30:58.975 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 226 M 2024-05-10T10:30:58.975 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-05-10T10:30:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:(4/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 7.6 MB/s | 5.4 MB 00:00 2024-05-10T10:30:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:(5/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 6.9 MB/s | 2.4 MB 00:00 2024-05-10T10:30:59.393 INFO:teuthology.orchestra.run.smithi053.stdout:(6/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-05-10T10:30:59.877 INFO:teuthology.orchestra.run.smithi053.stdout:(7/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 7.8 MB/s | 4.7 MB 00:00 2024-05-10T10:31:00.031 INFO:teuthology.orchestra.run.smithi135.stdout:(1/156): ceph-19.0.0-3449.gd39fa088.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-10T10:31:01.019 INFO:teuthology.orchestra.run.smithi053.stdout:(8/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x86 10 MB/s | 17 MB 00:01 2024-05-10T10:31:01.032 INFO:teuthology.orchestra.run.smithi135.stdout:(2/156): ceph-fuse-19.0.0-3449.gd39fa088.el9.x8 890 kB/s | 890 kB 00:00 2024-05-10T10:31:01.086 INFO:teuthology.orchestra.run.smithi053.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 379 kB/s | 25 kB 00:00 2024-05-10T10:31:01.174 INFO:teuthology.orchestra.run.smithi135.stdout:(3/156): ceph-immutable-object-cache-19.0.0-344 1.0 MB/s | 145 kB 00:00 2024-05-10T10:31:01.203 INFO:teuthology.orchestra.run.smithi053.stdout:(10/156): ceph-radosgw-19.0.0-3449.gd39fa088.el 8.0 MB/s | 11 MB 00:01 2024-05-10T10:31:01.286 INFO:teuthology.orchestra.run.smithi053.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 2.6 MB/s | 225 kB 00:00 2024-05-10T10:31:01.495 INFO:teuthology.orchestra.run.smithi053.stdout:(12/156): ceph-common-19.0.0-3449.gd39fa088.el9 7.1 MB/s | 21 MB 00:02 2024-05-10T10:31:01.520 INFO:teuthology.orchestra.run.smithi053.stdout:(13/156): cephfs-shell-19.0.0-3449.gd39fa088.el 115 kB/s | 27 kB 00:00 2024-05-10T10:31:01.554 INFO:teuthology.orchestra.run.smithi053.stdout:(14/156): libcephfs-devel-19.0.0-3449.gd39fa088 542 kB/s | 32 kB 00:00 2024-05-10T10:31:01.629 INFO:teuthology.orchestra.run.smithi053.stdout:(15/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 6.7 MB/s | 746 kB 00:00 2024-05-10T10:31:01.655 INFO:teuthology.orchestra.run.smithi053.stdout:(16/156): libcephsqlite-19.0.0-3449.gd39fa088.e 1.6 MB/s | 163 kB 00:00 2024-05-10T10:31:01.697 INFO:teuthology.orchestra.run.smithi053.stdout:(17/156): librados-devel-19.0.0-3449.gd39fa088. 1.8 MB/s | 127 kB 00:00 2024-05-10T10:31:01.822 INFO:teuthology.orchestra.run.smithi053.stdout:(18/156): libradosstriper1-19.0.0-3449.gd39fa08 2.9 MB/s | 500 kB 00:00 2024-05-10T10:31:01.922 INFO:teuthology.orchestra.run.smithi053.stdout:(19/156): python3-ceph-argparse-19.0.0-3449.gd3 452 kB/s | 45 kB 00:00 2024-05-10T10:31:01.999 INFO:teuthology.orchestra.run.smithi135.stdout:(4/156): ceph-mds-19.0.0-3449.gd39fa088.el9.x86 2.9 MB/s | 2.4 MB 00:00 2024-05-10T10:31:02.022 INFO:teuthology.orchestra.run.smithi053.stdout:(20/156): python3-ceph-common-19.0.0-3449.gd39f 1.3 MB/s | 135 kB 00:00 2024-05-10T10:31:02.139 INFO:teuthology.orchestra.run.smithi053.stdout:(21/156): python3-cephfs-19.0.0-3449.gd39fa088. 1.4 MB/s | 161 kB 00:00 2024-05-10T10:31:02.248 INFO:teuthology.orchestra.run.smithi053.stdout:(22/156): python3-rados-19.0.0-3449.gd39fa088.e 2.9 MB/s | 321 kB 00:00 2024-05-10T10:31:02.349 INFO:teuthology.orchestra.run.smithi053.stdout:(23/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 8.3 MB/s | 5.4 MB 00:00 2024-05-10T10:31:02.374 INFO:teuthology.orchestra.run.smithi053.stdout:(24/156): python3-rbd-19.0.0-3449.gd39fa088.el9 2.3 MB/s | 298 kB 00:00 2024-05-10T10:31:02.416 INFO:teuthology.orchestra.run.smithi053.stdout:(25/156): python3-rgw-19.0.0-3449.gd39fa088.el9 1.5 MB/s | 99 kB 00:00 2024-05-10T10:31:02.433 INFO:teuthology.orchestra.run.smithi135.stdout:(5/156): ceph-mgr-19.0.0-3449.gd39fa088.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-05-10T10:31:02.457 INFO:teuthology.orchestra.run.smithi053.stdout:(26/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-10T10:31:02.574 INFO:teuthology.orchestra.run.smithi053.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 1.4 MB/s | 170 kB 00:00 2024-05-10T10:31:02.667 INFO:teuthology.orchestra.run.smithi053.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 373 kB/s | 34 kB 00:00 2024-05-10T10:31:02.775 INFO:teuthology.orchestra.run.smithi053.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 1.3 MB/s | 142 kB 00:00 2024-05-10T10:31:02.834 INFO:teuthology.orchestra.run.smithi053.stdout:(30/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 7.4 MB/s | 3.1 MB 00:00 2024-05-10T10:31:03.342 INFO:teuthology.orchestra.run.smithi053.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 6.2 MB/s | 3.5 MB 00:00 2024-05-10T10:31:03.443 INFO:teuthology.orchestra.run.smithi053.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 2.7 MB/s | 274 kB 00:00 2024-05-10T10:31:03.535 INFO:teuthology.orchestra.run.smithi053.stdout:(33/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 538 kB/s | 49 kB 00:00 2024-05-10T10:31:03.618 INFO:teuthology.orchestra.run.smithi053.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3449.gd 198 kB/s | 16 kB 00:00 2024-05-10T10:31:03.710 INFO:teuthology.orchestra.run.smithi053.stdout:(35/156): ceph-volume-19.0.0-3449.gd39fa088.el9 2.9 MB/s | 273 kB 00:00 2024-05-10T10:31:03.819 INFO:teuthology.orchestra.run.smithi053.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-05-10T10:31:03.850 INFO:teuthology.orchestra.run.smithi135.stdout:(6/156): ceph-mon-19.0.0-3449.gd39fa088.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-05-10T10:31:03.861 INFO:teuthology.orchestra.run.smithi053.stdout:(37/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 5.0 MB/s | 763 kB 00:00 2024-05-10T10:31:03.886 INFO:teuthology.orchestra.run.smithi053.stdout:(38/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 274 kB/s | 18 kB 00:00 2024-05-10T10:31:04.028 INFO:teuthology.orchestra.run.smithi053.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-05-10T10:31:04.112 INFO:teuthology.orchestra.run.smithi053.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-10T10:31:04.154 INFO:teuthology.orchestra.run.smithi053.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-05-10T10:31:04.179 INFO:teuthology.orchestra.run.smithi053.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-10T10:31:04.230 INFO:teuthology.orchestra.run.smithi053.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-10T10:31:04.288 INFO:teuthology.orchestra.run.smithi053.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-10T10:31:04.350 INFO:teuthology.orchestra.run.smithi135.stdout:(7/156): ceph-base-19.0.0-3449.gd39fa088.el9.x8 1.1 MB/s | 5.4 MB 00:04 2024-05-10T10:31:04.480 INFO:teuthology.orchestra.run.smithi053.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 651 kB/s | 125 kB 00:00 2024-05-10T10:31:04.956 INFO:teuthology.orchestra.run.smithi053.stdout:(46/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 13 MB/s | 49 MB 00:03 2024-05-10T10:31:04.997 INFO:teuthology.orchestra.run.smithi053.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 1.5 MB/s | 1.2 MB 00:00 2024-05-10T10:31:05.023 INFO:teuthology.orchestra.run.smithi053.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 396 kB/s | 215 kB 00:00 2024-05-10T10:31:05.106 INFO:teuthology.orchestra.run.smithi053.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-10T10:31:05.156 INFO:teuthology.orchestra.run.smithi053.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-05-10T10:31:05.224 INFO:teuthology.orchestra.run.smithi053.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-10T10:31:05.249 INFO:teuthology.orchestra.run.smithi053.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-05-10T10:31:05.291 INFO:teuthology.orchestra.run.smithi053.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-05-10T10:31:05.308 INFO:teuthology.orchestra.run.smithi053.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-05-10T10:31:05.333 INFO:teuthology.orchestra.run.smithi053.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-05-10T10:31:05.350 INFO:teuthology.orchestra.run.smithi053.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-05-10T10:31:05.384 INFO:teuthology.orchestra.run.smithi053.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-10T10:31:05.410 INFO:teuthology.orchestra.run.smithi053.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-05-10T10:31:05.526 INFO:teuthology.orchestra.run.smithi053.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-05-10T10:31:05.551 INFO:teuthology.orchestra.run.smithi053.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-05-10T10:31:05.576 INFO:teuthology.orchestra.run.smithi053.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-05-10T10:31:05.601 INFO:teuthology.orchestra.run.smithi053.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-10T10:31:05.660 INFO:teuthology.orchestra.run.smithi053.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-10T10:31:05.694 INFO:teuthology.orchestra.run.smithi053.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-10T10:31:05.719 INFO:teuthology.orchestra.run.smithi053.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-05-10T10:31:05.769 INFO:teuthology.orchestra.run.smithi053.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-05-10T10:31:05.992 INFO:teuthology.orchestra.run.smithi135.stdout:(8/156): ceph-osd-19.0.0-3449.gd39fa088.el9.x86 7.9 MB/s | 17 MB 00:02 2024-05-10T10:31:06.050 INFO:teuthology.orchestra.run.smithi135.stdout:(9/156): ceph-selinux-19.0.0-3449.gd39fa088.el9 433 kB/s | 25 kB 00:00 2024-05-10T10:31:06.070 INFO:teuthology.orchestra.run.smithi053.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-05-10T10:31:06.262 INFO:teuthology.orchestra.run.smithi053.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-05-10T10:31:06.320 INFO:teuthology.orchestra.run.smithi053.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-10T10:31:06.421 INFO:teuthology.orchestra.run.smithi053.stdout:(70/156): openblas-openmp-0.3.21-2.el9.x86_64.r 6.5 MB/s | 4.7 MB 00:00 2024-05-10T10:31:06.509 INFO:teuthology.orchestra.run.smithi135.stdout:(10/156): ceph-radosgw-19.0.0-3449.gd39fa088.el 4.9 MB/s | 11 MB 00:02 2024-05-10T10:31:06.546 INFO:teuthology.orchestra.run.smithi053.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-05-10T10:31:06.576 INFO:teuthology.orchestra.run.smithi135.stdout:(11/156): cephfs-mirror-19.0.0-3449.gd39fa088.e 3.3 MB/s | 225 kB 00:00 2024-05-10T10:31:06.618 INFO:teuthology.orchestra.run.smithi135.stdout:(12/156): cephfs-shell-19.0.0-3449.gd39fa088.el 650 kB/s | 27 kB 00:00 2024-05-10T10:31:06.638 INFO:teuthology.orchestra.run.smithi053.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-05-10T10:31:06.663 INFO:teuthology.orchestra.run.smithi053.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 749 kB/s | 87 kB 00:00 2024-05-10T10:31:06.669 INFO:teuthology.orchestra.run.smithi135.stdout:(13/156): libcephfs-devel-19.0.0-3449.gd39fa088 634 kB/s | 32 kB 00:00 2024-05-10T10:31:06.747 INFO:teuthology.orchestra.run.smithi053.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-10T10:31:06.811 INFO:teuthology.orchestra.run.smithi135.stdout:(14/156): libcephfs2-19.0.0-3449.gd39fa088.el9. 5.1 MB/s | 746 kB 00:00 2024-05-10T10:31:06.830 INFO:teuthology.orchestra.run.smithi053.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-10T10:31:06.881 INFO:teuthology.orchestra.run.smithi053.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-10T10:31:06.886 INFO:teuthology.orchestra.run.smithi135.stdout:(15/156): libcephsqlite-19.0.0-3449.gd39fa088.e 2.1 MB/s | 163 kB 00:00 2024-05-10T10:31:06.931 INFO:teuthology.orchestra.run.smithi053.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 385 kB/s | 19 kB 00:00 2024-05-10T10:31:06.945 INFO:teuthology.orchestra.run.smithi135.stdout:(16/156): librados-devel-19.0.0-3449.gd39fa088. 2.1 MB/s | 127 kB 00:00 2024-05-10T10:31:07.015 INFO:teuthology.orchestra.run.smithi053.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-10T10:31:07.062 INFO:teuthology.orchestra.run.smithi135.stdout:(17/156): libradosstriper1-19.0.0-3449.gd39fa08 4.2 MB/s | 500 kB 00:00 2024-05-10T10:31:07.073 INFO:teuthology.orchestra.run.smithi053.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-05-10T10:31:07.357 INFO:teuthology.orchestra.run.smithi053.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 5.6 MB/s | 5.8 MB 00:01 2024-05-10T10:31:07.466 INFO:teuthology.orchestra.run.smithi053.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 7.2 MB/s | 6.0 MB 00:00 2024-05-10T10:31:07.491 INFO:teuthology.orchestra.run.smithi053.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-05-10T10:31:07.524 INFO:teuthology.orchestra.run.smithi053.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-10T10:31:07.550 INFO:teuthology.orchestra.run.smithi053.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-10T10:31:07.575 INFO:teuthology.orchestra.run.smithi053.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-05-10T10:31:07.634 INFO:teuthology.orchestra.run.smithi053.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-10T10:31:07.659 INFO:teuthology.orchestra.run.smithi053.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-10T10:31:07.701 INFO:teuthology.orchestra.run.smithi053.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-10T10:31:07.718 INFO:teuthology.orchestra.run.smithi053.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-05-10T10:31:07.743 INFO:teuthology.orchestra.run.smithi053.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-05-10T10:31:07.785 INFO:teuthology.orchestra.run.smithi053.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 996 kB/s | 42 kB 00:00 2024-05-10T10:31:07.812 INFO:teuthology.orchestra.run.smithi135.stdout:(18/156): librgw2-19.0.0-3449.gd39fa088.el9.x86 7.2 MB/s | 5.4 MB 00:00 2024-05-10T10:31:07.835 INFO:teuthology.orchestra.run.smithi053.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-05-10T10:31:07.855 INFO:teuthology.orchestra.run.smithi135.stdout:(19/156): python3-ceph-argparse-19.0.0-3449.gd3 1.1 MB/s | 45 kB 00:00 2024-05-10T10:31:07.894 INFO:teuthology.orchestra.run.smithi053.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-10T10:31:07.913 INFO:teuthology.orchestra.run.smithi135.stdout:(20/156): python3-ceph-common-19.0.0-3449.gd39f 2.3 MB/s | 135 kB 00:00 2024-05-10T10:31:07.953 INFO:teuthology.orchestra.run.smithi053.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-05-10T10:31:07.972 INFO:teuthology.orchestra.run.smithi135.stdout:(21/156): python3-cephfs-19.0.0-3449.gd39fa088. 2.7 MB/s | 161 kB 00:00 2024-05-10T10:31:08.003 INFO:teuthology.orchestra.run.smithi053.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-10T10:31:08.073 INFO:teuthology.orchestra.run.smithi135.stdout:(22/156): python3-rados-19.0.0-3449.gd39fa088.e 3.1 MB/s | 321 kB 00:00 2024-05-10T10:31:08.104 INFO:teuthology.orchestra.run.smithi053.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-10T10:31:08.156 INFO:teuthology.orchestra.run.smithi135.stdout:(23/156): python3-rbd-19.0.0-3449.gd39fa088.el9 3.5 MB/s | 298 kB 00:00 2024-05-10T10:31:08.195 INFO:teuthology.orchestra.run.smithi053.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-05-10T10:31:08.221 INFO:teuthology.orchestra.run.smithi053.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 495 kB/s | 58 kB 00:00 2024-05-10T10:31:08.240 INFO:teuthology.orchestra.run.smithi135.stdout:(24/156): python3-rgw-19.0.0-3449.gd39fa088.el9 1.2 MB/s | 99 kB 00:00 2024-05-10T10:31:08.246 INFO:teuthology.orchestra.run.smithi053.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-10T10:31:08.280 INFO:teuthology.orchestra.run.smithi053.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-10T10:31:08.290 INFO:teuthology.orchestra.run.smithi135.stdout:(25/156): rbd-fuse-19.0.0-3449.gd39fa088.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-10T10:31:08.305 INFO:teuthology.orchestra.run.smithi053.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-05-10T10:31:08.430 INFO:teuthology.orchestra.run.smithi053.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-05-10T10:31:08.582 INFO:teuthology.orchestra.run.smithi135.stdout:(26/156): ceph-common-19.0.0-3449.gd39fa088.el9 2.4 MB/s | 21 MB 00:08 2024-05-10T10:31:08.630 INFO:teuthology.orchestra.run.smithi053.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-05-10T10:31:08.649 INFO:teuthology.orchestra.run.smithi135.stdout:(27/156): rbd-nbd-19.0.0-3449.gd39fa088.el9.x86 2.5 MB/s | 170 kB 00:00 2024-05-10T10:31:08.708 INFO:teuthology.orchestra.run.smithi135.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3449.g 588 kB/s | 34 kB 00:00 2024-05-10T10:31:08.772 INFO:teuthology.orchestra.run.smithi053.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-10T10:31:08.783 INFO:teuthology.orchestra.run.smithi135.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3449.gd39fa08 1.9 MB/s | 142 kB 00:00 2024-05-10T10:31:08.839 INFO:teuthology.orchestra.run.smithi053.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-05-10T10:31:08.865 INFO:teuthology.orchestra.run.smithi053.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-05-10T10:31:08.900 INFO:teuthology.orchestra.run.smithi135.stdout:(30/156): rbd-mirror-19.0.0-3449.gd39fa088.el9. 5.1 MB/s | 3.1 MB 00:00 2024-05-10T10:31:08.923 INFO:teuthology.orchestra.run.smithi053.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-05-10T10:31:08.982 INFO:teuthology.orchestra.run.smithi053.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-05-10T10:31:09.066 INFO:teuthology.orchestra.run.smithi053.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-10T10:31:09.093 INFO:teuthology.orchestra.run.smithi053.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-05-10T10:31:09.133 INFO:teuthology.orchestra.run.smithi053.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-10T10:31:09.200 INFO:teuthology.orchestra.run.smithi053.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-05-10T10:31:09.234 INFO:teuthology.orchestra.run.smithi053.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-05-10T10:31:09.259 INFO:teuthology.orchestra.run.smithi053.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-05-10T10:31:09.292 INFO:teuthology.orchestra.run.smithi053.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-05-10T10:31:09.318 INFO:teuthology.orchestra.run.smithi053.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-05-10T10:31:09.326 INFO:teuthology.orchestra.run.smithi135.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3449.gd39fa 6.5 MB/s | 3.5 MB 00:00 2024-05-10T10:31:09.360 INFO:teuthology.orchestra.run.smithi053.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-10T10:31:09.385 INFO:teuthology.orchestra.run.smithi053.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-10T10:31:09.393 INFO:teuthology.orchestra.run.smithi135.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3449.gd3 4.0 MB/s | 274 kB 00:00 2024-05-10T10:31:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-10T10:31:09.452 INFO:teuthology.orchestra.run.smithi135.stdout:(33/156): ceph-mgr-rook-19.0.0-3449.gd39fa088.e 847 kB/s | 49 kB 00:00 2024-05-10T10:31:09.452 INFO:teuthology.orchestra.run.smithi053.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-05-10T10:31:09.494 INFO:teuthology.orchestra.run.smithi053.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-10T10:31:09.510 INFO:teuthology.orchestra.run.smithi135.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3449.gd 282 kB/s | 16 kB 00:00 2024-05-10T10:31:09.586 INFO:teuthology.orchestra.run.smithi053.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-05-10T10:31:09.594 INFO:teuthology.orchestra.run.smithi135.stdout:(35/156): ceph-volume-19.0.0-3449.gd39fa088.el9 3.2 MB/s | 273 kB 00:00 2024-05-10T10:31:09.611 INFO:teuthology.orchestra.run.smithi053.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-10T10:31:09.645 INFO:teuthology.orchestra.run.smithi053.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-05-10T10:31:09.670 INFO:teuthology.orchestra.run.smithi053.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-05-10T10:31:09.706 INFO:teuthology.orchestra.run.smithi053.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-05-10T10:31:09.730 INFO:teuthology.orchestra.run.smithi053.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-05-10T10:31:09.744 INFO:teuthology.orchestra.run.smithi135.stdout:(36/156): cephadm-19.0.0-3449.gd39fa088.el9.noa 5.0 MB/s | 763 kB 00:00 2024-05-10T10:31:09.763 INFO:teuthology.orchestra.run.smithi053.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-05-10T10:31:09.795 INFO:teuthology.orchestra.run.smithi135.stdout:(37/156): cephfs-top-19.0.0-3449.gd39fa088.el9. 365 kB/s | 18 kB 00:00 2024-05-10T10:31:09.903 INFO:teuthology.orchestra.run.smithi135.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 7.4 MB/s | 7.4 MB 00:01 2024-05-10T10:31:09.956 INFO:teuthology.orchestra.run.smithi053.stdout:(129/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.3 MB/s | 16 MB 00:02 2024-05-10T10:31:09.972 INFO:teuthology.orchestra.run.smithi053.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2024-05-10T10:31:10.004 INFO:teuthology.orchestra.run.smithi135.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 384 kB/s | 80 kB 00:00 2024-05-10T10:31:10.022 INFO:teuthology.orchestra.run.smithi053.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-10T10:31:10.071 INFO:teuthology.orchestra.run.smithi135.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-10T10:31:10.089 INFO:teuthology.orchestra.run.smithi053.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-10T10:31:10.096 INFO:teuthology.orchestra.run.smithi135.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-05-10T10:31:10.122 INFO:teuthology.orchestra.run.smithi135.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-10T10:31:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 2.4 MB/s | 1.0 MB 00:00 2024-05-10T10:31:10.165 INFO:teuthology.orchestra.run.smithi053.stdout:(134/156): python3-more-itertools-8.12.0-2.el9. 377 kB/s | 79 kB 00:00 2024-05-10T10:31:10.172 INFO:teuthology.orchestra.run.smithi135.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-10T10:31:10.182 INFO:teuthology.orchestra.run.smithi053.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-05-10T10:31:10.207 INFO:teuthology.orchestra.run.smithi053.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-10T10:31:10.232 INFO:teuthology.orchestra.run.smithi053.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-05-10T10:31:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-10T10:31:10.283 INFO:teuthology.orchestra.run.smithi053.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-10T10:31:10.289 INFO:teuthology.orchestra.run.smithi135.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-10T10:31:10.308 INFO:teuthology.orchestra.run.smithi053.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-10T10:31:10.333 INFO:teuthology.orchestra.run.smithi053.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-10T10:31:10.339 INFO:teuthology.orchestra.run.smithi135.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-10T10:31:10.359 INFO:teuthology.orchestra.run.smithi053.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-10T10:31:10.390 INFO:teuthology.orchestra.run.smithi135.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-10T10:31:10.392 INFO:teuthology.orchestra.run.smithi053.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-10T10:31:10.426 INFO:teuthology.orchestra.run.smithi053.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-05-10T10:31:10.451 INFO:teuthology.orchestra.run.smithi053.stdout:(145/156): python3-websocket-client-1.2.3-2.el9 764 kB/s | 90 kB 00:00 2024-05-10T10:31:10.476 INFO:teuthology.orchestra.run.smithi053.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-10T10:31:10.502 INFO:teuthology.orchestra.run.smithi053.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-10T10:31:10.652 INFO:teuthology.orchestra.run.smithi053.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-05-10T10:31:10.911 INFO:teuthology.orchestra.run.smithi053.stdout:(149/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-05-10T10:31:10.969 INFO:teuthology.orchestra.run.smithi053.stdout:(150/156): librados2-19.0.0-3449.gd39fa088.el9. 6.9 MB/s | 3.4 MB 00:00 2024-05-10T10:31:11.045 INFO:teuthology.orchestra.run.smithi053.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.9 MB/s | 2.3 MB 00:00 2024-05-10T10:31:11.057 INFO:teuthology.orchestra.run.smithi135.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-05-10T10:31:11.070 INFO:teuthology.orchestra.run.smithi053.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 417 kB/s | 42 kB 00:00 2024-05-10T10:31:11.112 INFO:teuthology.orchestra.run.smithi053.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-05-10T10:31:11.115 INFO:teuthology.orchestra.run.smithi135.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-10T10:31:11.320 INFO:teuthology.orchestra.run.smithi053.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 1.3 MB 00:00 2024-05-10T10:31:11.324 INFO:teuthology.orchestra.run.smithi135.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-05-10T10:31:11.362 INFO:teuthology.orchestra.run.smithi053.stdout:(155/156): NetworkManager-libnm-1.47.5-1.el9.x8 4.0 MB/s | 1.8 MB 00:00 2024-05-10T10:31:11.463 INFO:teuthology.orchestra.run.smithi053.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-05-10T10:31:11.466 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:31:11.466 INFO:teuthology.orchestra.run.smithi053.stdout:Total 17 MB/s | 226 MB 00:13 2024-05-10T10:31:11.791 INFO:teuthology.orchestra.run.smithi135.stdout:(50/156): ceph-test-19.0.0-3449.gd39fa088.el9.x 8.6 MB/s | 49 MB 00:05 2024-05-10T10:31:11.817 INFO:teuthology.orchestra.run.smithi135.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 44 kB/s | 21 kB 00:00 2024-05-10T10:31:11.842 INFO:teuthology.orchestra.run.smithi135.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 147 kB/s | 107 kB 00:00 2024-05-10T10:31:11.917 INFO:teuthology.orchestra.run.smithi135.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 324 kB/s | 33 kB 00:00 2024-05-10T10:31:12.026 INFO:teuthology.orchestra.run.smithi135.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-10T10:31:12.127 INFO:teuthology.orchestra.run.smithi135.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-10T10:31:12.227 INFO:teuthology.orchestra.run.smithi135.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-05-10T10:31:12.336 INFO:teuthology.orchestra.run.smithi135.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-05-10T10:31:12.386 INFO:teuthology.orchestra.run.smithi135.stdout:(58/156): flex-2.6.4-9.el9.x86_64.rpm 528 kB/s | 313 kB 00:00 2024-05-10T10:31:12.578 INFO:teuthology.orchestra.run.smithi135.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-05-10T10:31:12.603 INFO:teuthology.orchestra.run.smithi135.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-05-10T10:31:12.654 INFO:teuthology.orchestra.run.smithi135.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-05-10T10:31:12.670 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-05-10T10:31:12.679 INFO:teuthology.orchestra.run.smithi135.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-05-10T10:31:12.705 INFO:teuthology.orchestra.run.smithi135.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-05-10T10:31:12.763 INFO:teuthology.orchestra.run.smithi135.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-10T10:31:12.788 INFO:teuthology.orchestra.run.smithi135.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-10T10:31:12.805 INFO:teuthology.orchestra.run.smithi135.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 349 kB/s | 35 kB 00:00 2024-05-10T10:31:12.837 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-05-10T10:31:12.837 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-05-10T10:31:13.089 INFO:teuthology.orchestra.run.smithi135.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-05-10T10:31:13.223 INFO:teuthology.orchestra.run.smithi135.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-05-10T10:31:13.348 INFO:teuthology.orchestra.run.smithi135.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-10T10:31:14.807 INFO:teuthology.orchestra.run.smithi135.stdout:(70/156): openblas-openmp-0.3.21-2.el9.x86_64.r 2.3 MB/s | 4.7 MB 00:02 2024-05-10T10:31:15.088 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-05-10T10:31:15.088 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-05-10T10:31:16.307 INFO:teuthology.orchestra.run.smithi135.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 689 kB/s | 1.0 MB 00:01 2024-05-10T10:31:16.374 INFO:teuthology.orchestra.run.smithi135.stdout:(72/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-05-10T10:31:16.508 INFO:teuthology.orchestra.run.smithi135.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 1.8 MB/s | 5.8 MB 00:03 2024-05-10T10:31:16.533 INFO:teuthology.orchestra.run.smithi135.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 387 kB/s | 87 kB 00:00 2024-05-10T10:31:16.859 INFO:teuthology.orchestra.run.smithi135.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 696 kB/s | 244 kB 00:00 2024-05-10T10:31:16.885 INFO:teuthology.orchestra.run.smithi135.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 708 kB/s | 248 kB 00:00 2024-05-10T10:31:16.959 INFO:teuthology.orchestra.run.smithi135.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-05-10T10:31:16.984 INFO:teuthology.orchestra.run.smithi135.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-05-10T10:31:17.093 INFO:teuthology.orchestra.run.smithi135.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-10T10:31:17.219 INFO:teuthology.orchestra.run.smithi135.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 664 kB/s | 172 kB 00:00 2024-05-10T10:31:17.569 INFO:teuthology.orchestra.run.smithi135.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-05-10T10:31:17.928 INFO:teuthology.orchestra.run.smithi135.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 620 kB/s | 222 kB 00:00 2024-05-10T10:31:18.112 INFO:teuthology.orchestra.run.smithi135.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-05-10T10:31:18.212 INFO:teuthology.orchestra.run.smithi135.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-05-10T10:31:18.637 INFO:teuthology.orchestra.run.smithi135.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 634 kB/s | 269 kB 00:00 2024-05-10T10:31:18.904 INFO:teuthology.orchestra.run.smithi135.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-10T10:31:19.322 INFO:teuthology.orchestra.run.smithi135.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 669 kB/s | 279 kB 00:00 2024-05-10T10:31:19.430 INFO:teuthology.orchestra.run.smithi135.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-05-10T10:31:19.530 INFO:teuthology.orchestra.run.smithi135.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-05-10T10:31:21.471 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-05-10T10:31:21.543 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T10:31:21.585 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T10:31:21.661 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T10:31:21.702 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T10:31:21.735 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T10:31:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T10:31:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T10:31:22.244 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:31:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:31:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:31:22.480 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:31:22.509 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:31:22.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T10:31:22.772 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T10:31:22.791 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:31:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:31:22.917 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:31:22.972 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:31:22.993 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:31:23.018 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:31:23.213 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:31:23.393 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:31:23.507 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:31:23.568 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:31:23.683 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:31:23.778 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:31:23.980 INFO:teuthology.orchestra.run.smithi135.stdout:(90/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 758 kB/s | 5.1 MB 00:06 2024-05-10T10:31:23.997 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:31:24.081 INFO:teuthology.orchestra.run.smithi135.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2024-05-10T10:31:24.105 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T10:31:24.173 INFO:teuthology.orchestra.run.smithi135.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 482 kB/s | 44 kB 00:00 2024-05-10T10:31:24.202 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T10:31:24.245 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T10:31:24.320 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T10:31:24.360 INFO:teuthology.orchestra.run.smithi135.stdout:(93/156): python3-babel-2.9.1-2.el9.noarch.rpm 765 kB/s | 6.0 MB 00:07 2024-05-10T10:31:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T10:31:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T10:31:24.456 INFO:teuthology.orchestra.run.smithi135.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 374 kB/s | 36 kB 00:00 2024-05-10T10:31:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T10:31:24.519 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T10:31:24.546 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T10:31:24.566 INFO:teuthology.orchestra.run.smithi135.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 562 kB/s | 221 kB 00:00 2024-05-10T10:31:24.582 INFO:teuthology.orchestra.run.smithi135.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 377 kB/s | 47 kB 00:00 2024-05-10T10:31:24.618 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T10:31:24.679 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T10:31:24.683 INFO:teuthology.orchestra.run.smithi135.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-05-10T10:31:24.708 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T10:31:24.752 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T10:31:24.851 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T10:31:24.858 INFO:teuthology.orchestra.run.smithi135.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 535 kB/s | 94 kB 00:00 2024-05-10T10:31:24.887 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T10:31:24.931 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:31:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T10:31:25.035 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T10:31:25.042 INFO:teuthology.orchestra.run.smithi135.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 664 kB/s | 305 kB 00:00 2024-05-10T10:31:25.076 INFO:teuthology.orchestra.run.smithi135.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 293 kB/s | 64 kB 00:00 2024-05-10T10:31:25.082 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T10:31:25.159 INFO:teuthology.orchestra.run.smithi135.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 382 kB/s | 45 kB 00:00 2024-05-10T10:31:25.161 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T10:31:25.201 INFO:teuthology.orchestra.run.smithi135.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-05-10T10:31:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T10:31:25.362 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T10:31:25.427 INFO:teuthology.orchestra.run.smithi135.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-05-10T10:31:25.519 INFO:teuthology.orchestra.run.smithi135.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-05-10T10:31:25.561 INFO:teuthology.orchestra.run.smithi135.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-05-10T10:31:25.933 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T10:31:26.018 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T10:31:26.051 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T10:31:26.094 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T10:31:26.132 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T10:31:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T10:31:26.294 INFO:teuthology.orchestra.run.smithi135.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 761 kB/s | 863 kB 00:01 2024-05-10T10:31:26.350 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T10:31:26.410 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T10:31:26.411 INFO:teuthology.orchestra.run.smithi135.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-05-10T10:31:26.437 INFO:teuthology.orchestra.run.smithi135.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-05-10T10:31:26.462 INFO:teuthology.orchestra.run.smithi135.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 965 kB/s | 49 kB 00:00 2024-05-10T10:31:26.467 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T10:31:26.504 INFO:teuthology.orchestra.run.smithi135.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-05-10T10:31:26.530 INFO:teuthology.orchestra.run.smithi135.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 729 kB/s | 67 kB 00:00 2024-05-10T10:31:26.541 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T10:31:26.577 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T10:31:26.596 INFO:teuthology.orchestra.run.smithi135.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-05-10T10:31:26.614 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T10:31:26.639 INFO:teuthology.orchestra.run.smithi135.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-10T10:31:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T10:31:26.680 INFO:teuthology.orchestra.run.smithi135.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 771 kB/s | 32 kB 00:00 2024-05-10T10:31:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T10:31:26.706 INFO:teuthology.orchestra.run.smithi135.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-05-10T10:31:26.731 INFO:teuthology.orchestra.run.smithi135.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-10T10:31:26.740 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T10:31:26.765 INFO:teuthology.orchestra.run.smithi135.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-10T10:31:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T10:31:26.807 INFO:teuthology.orchestra.run.smithi135.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-05-10T10:31:26.833 INFO:teuthology.orchestra.run.smithi135.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-10T10:31:26.850 INFO:teuthology.orchestra.run.smithi135.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 840 kB/s | 35 kB 00:00 2024-05-10T10:31:26.925 INFO:teuthology.orchestra.run.smithi135.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-10T10:31:26.976 INFO:teuthology.orchestra.run.smithi135.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-05-10T10:31:27.001 INFO:teuthology.orchestra.run.smithi135.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-10T10:31:27.026 INFO:teuthology.orchestra.run.smithi135.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-05-10T10:31:27.043 INFO:teuthology.orchestra.run.smithi135.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-05-10T10:31:27.045 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T10:31:27.077 INFO:teuthology.orchestra.run.smithi135.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-10T10:31:27.102 INFO:teuthology.orchestra.run.smithi135.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 330 kB/s | 19 kB 00:00 2024-05-10T10:31:27.127 INFO:teuthology.orchestra.run.smithi135.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-05-10T10:31:27.186 INFO:teuthology.orchestra.run.smithi135.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-10T10:31:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T10:31:27.211 INFO:teuthology.orchestra.run.smithi135.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-05-10T10:31:27.228 INFO:teuthology.orchestra.run.smithi135.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-05-10T10:31:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T10:31:27.254 INFO:teuthology.orchestra.run.smithi135.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-05-10T10:31:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T10:31:27.288 INFO:teuthology.orchestra.run.smithi135.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-05-10T10:31:27.313 INFO:teuthology.orchestra.run.smithi135.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-05-10T10:31:27.338 INFO:teuthology.orchestra.run.smithi135.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-05-10T10:31:27.364 INFO:teuthology.orchestra.run.smithi135.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-05-10T10:31:27.386 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T10:31:27.389 INFO:teuthology.orchestra.run.smithi135.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-05-10T10:31:27.426 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T10:31:27.448 INFO:teuthology.orchestra.run.smithi135.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-10T10:31:27.469 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T10:31:27.473 INFO:teuthology.orchestra.run.smithi135.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-10T10:31:27.499 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T10:31:27.524 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T10:31:27.565 INFO:teuthology.orchestra.run.smithi135.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-05-10T10:31:27.607 INFO:teuthology.orchestra.run.smithi135.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 381 kB/s | 51 kB 00:00 2024-05-10T10:31:27.626 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T10:31:27.633 INFO:teuthology.orchestra.run.smithi135.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-05-10T10:31:27.666 INFO:teuthology.orchestra.run.smithi135.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-10T10:31:27.725 INFO:teuthology.orchestra.run.smithi135.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-05-10T10:31:27.751 INFO:teuthology.orchestra.run.smithi135.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-10T10:31:27.766 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T10:31:27.776 INFO:teuthology.orchestra.run.smithi135.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-10T10:31:27.798 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T10:31:27.831 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T10:31:27.861 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T10:31:27.900 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T10:31:27.940 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T10:31:27.970 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T10:31:28.004 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T10:31:28.037 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T10:31:28.060 INFO:teuthology.orchestra.run.smithi135.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-05-10T10:31:28.079 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T10:31:28.139 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T10:31:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T10:31:28.278 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T10:31:28.325 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T10:31:28.352 INFO:teuthology.orchestra.run.smithi135.stdout:(148/156): librados2-19.0.0-3449.gd39fa088.el9. 5.9 MB/s | 3.4 MB 00:00 2024-05-10T10:31:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T10:31:28.493 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T10:31:28.527 INFO:teuthology.orchestra.run.smithi135.stdout:(149/156): librbd1-19.0.0-3449.gd39fa088.el9.x8 6.7 MB/s | 3.1 MB 00:00 2024-05-10T10:31:28.545 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T10:31:28.602 INFO:teuthology.orchestra.run.smithi135.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 9.2 MB/s | 2.3 MB 00:00 2024-05-10T10:31:28.694 INFO:teuthology.orchestra.run.smithi135.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 455 kB/s | 42 kB 00:00 2024-05-10T10:31:28.736 INFO:teuthology.orchestra.run.smithi135.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 8.7 MB/s | 1.8 MB 00:00 2024-05-10T10:31:28.762 INFO:teuthology.orchestra.run.smithi135.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-05-10T10:31:28.812 INFO:teuthology.orchestra.run.smithi135.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-05-10T10:31:28.888 INFO:teuthology.orchestra.run.smithi135.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-05-10T10:31:29.331 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T10:31:29.382 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T10:31:29.416 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T10:31:29.523 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T10:31:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T10:31:29.667 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T10:31:29.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T10:31:29.887 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T10:31:29.915 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T10:31:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T10:31:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T10:31:30.960 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T10:31:32.318 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T10:31:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T10:31:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T10:31:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T10:31:32.800 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T10:31:32.849 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:31:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:31:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T10:31:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T10:31:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T10:31:33.544 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:31:33.616 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:31:33.645 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T10:31:36.179 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:31:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:31:36.798 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:31:36.837 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:31:36.922 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:31:36.922 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T10:31:36.922 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T10:31:36.922 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:31:41.121 INFO:teuthology.orchestra.run.smithi135.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 771 kB/s | 16 MB 00:21 2024-05-10T10:31:41.124 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-05-10T10:31:41.124 INFO:teuthology.orchestra.run.smithi135.stdout:Total 5.4 MB/s | 226 MB 00:42 2024-05-10T10:31:42.294 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-05-10T10:31:42.445 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-05-10T10:31:42.445 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-05-10T10:31:44.625 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-05-10T10:31:44.625 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-05-10T10:31:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-05-10T10:31:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:49.263 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:31:49.279 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:31:49.279 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:49.279 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T10:31:49.279 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:50.747 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-05-10T10:31:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:31:50.826 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-10T10:31:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:31:50.845 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:50.845 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T10:31:50.845 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:31:50.845 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:31:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:50.864 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-10T10:31:50.925 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-10T10:31:50.960 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-10T10:31:50.986 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-10T10:31:51.035 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-10T10:31:51.084 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:31:51.109 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:31:51.109 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:51.109 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T10:31:51.110 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:31:51.110 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:31:51.110 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:51.375 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-10T10:31:51.487 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:31:51.576 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:31:51.601 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:31:51.601 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:31:51.601 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:51.601 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T10:31:51.601 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:31:51.602 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:31:51.602 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:51.639 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:31:51.716 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T10:31:51.736 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:31:51.769 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:31:51.774 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T10:31:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T10:31:51.946 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-10T10:31:52.008 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-10T10:31:52.035 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:31:52.036 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T10:31:52.079 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T10:31:52.139 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:31:52.161 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T10:31:52.162 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:31:52.224 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 14/164 2024-05-10T10:31:52.246 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:31:52.250 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T10:31:52.269 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-10T10:31:52.403 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T10:31:52.456 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:31:52.462 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T10:31:52.494 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T10:31:52.520 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T10:31:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-10T10:31:52.803 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:31:52.872 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-10T10:31:52.920 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:31:52.945 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:31:52.978 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T10:31:52.986 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:31:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T10:31:53.057 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:31:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:31:53.266 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-10T10:31:53.371 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-10T10:31:53.466 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-10T10:31:53.506 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-10T10:31:53.540 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-10T10:31:53.605 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-10T10:31:53.640 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-10T10:31:53.683 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-10T10:31:53.717 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-10T10:31:53.745 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-10T10:31:53.813 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-10T10:31:53.867 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-10T10:31:53.906 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-10T10:31:53.942 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-10T10:31:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-10T10:31:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 33/164 2024-05-10T10:31:54.096 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:31:54.125 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 35/164 2024-05-10T10:31:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:31:54.196 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-10T10:31:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:31:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-10T10:31:54.315 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-10T10:31:54.450 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-10T10:31:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-10T10:31:55.046 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-10T10:31:55.131 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-10T10:31:55.169 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-10T10:31:55.212 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-10T10:31:55.229 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:31:55.250 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 45/164 2024-05-10T10:31:55.253 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:31:55.293 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-10T10:31:55.373 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:31:55.449 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-10T10:31:55.495 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-10T10:31:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T10:31:55.550 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-10T10:31:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:31:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:31:55.597 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:55.597 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T10:31:55.597 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:31:55.597 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:31:55.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:55.623 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-10T10:31:55.648 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:31:55.655 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-10T10:31:55.660 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:31:55.683 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-10T10:31:55.730 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-10T10:31:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T10:31:55.766 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-10T10:31:55.809 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-10T10:31:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T10:31:55.851 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-10T10:31:55.987 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-10T10:31:56.041 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-10T10:31:56.072 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-10T10:31:56.113 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-10T10:31:56.208 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-10T10:31:56.247 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-10T10:31:56.289 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-10T10:31:56.321 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-10T10:31:56.346 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-10T10:31:56.433 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-10T10:31:56.564 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-10T10:31:56.630 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-10T10:31:56.668 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-10T10:31:56.701 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-10T10:31:56.740 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-10T10:31:56.771 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-10T10:31:56.798 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T10:31:56.801 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-10T10:31:56.825 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:31:56.836 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-10T10:31:56.850 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:31:56.850 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:56.850 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T10:31:56.850 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:31:56.850 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:31:56.850 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:56.870 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-10T10:31:56.893 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:31:56.911 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-10T10:31:56.916 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:31:56.916 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:56.916 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T10:31:56.916 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:56.946 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-10T10:31:56.968 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:31:56.993 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:31:56.993 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:56.993 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T10:31:56.993 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:31:56.993 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:31:56.993 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:57.034 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-10T10:31:57.085 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-10T10:31:57.116 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-10T10:31:57.164 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-10T10:31:57.261 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-10T10:31:57.310 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:31:57.318 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-10T10:31:57.335 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:31:57.335 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:31:57.335 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T10:31:57.336 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:31:57.336 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:31:57.336 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:31:58.042 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-10T10:31:58.083 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-10T10:31:58.117 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-10T10:31:58.219 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-10T10:31:58.275 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-10T10:31:58.364 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-10T10:31:58.404 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-10T10:31:58.579 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-10T10:31:58.610 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-10T10:31:58.688 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-10T10:31:59.341 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-10T10:31:59.459 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-10T10:32:00.725 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-10T10:32:00.819 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-10T10:32:00.846 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-10T10:32:00.880 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-10T10:32:01.192 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-10T10:32:01.235 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:32:01.325 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 101/164 2024-05-10T10:32:01.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 102/164 2024-05-10T10:32:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-10T10:32:01.869 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-10T10:32:01.905 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:32:01.976 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 105/164 2024-05-10T10:32:01.999 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 106/164 2024-05-10T10:32:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T10:32:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T10:32:02.314 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T10:32:02.345 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T10:32:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T10:32:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T10:32:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T10:32:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T10:32:02.696 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T10:32:02.750 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T10:32:03.365 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:32:03.398 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:32:03.420 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:32:04.263 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:32:04.315 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:32:04.340 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T10:32:04.420 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T10:32:04.469 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T10:32:04.561 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T10:32:04.598 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T10:32:04.624 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T10:32:04.624 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:32:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:32:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T10:32:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:32:04.826 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:32:04.851 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T10:32:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:32:04.857 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:32:04.858 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 107/164 2024-05-10T10:32:04.894 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:32:04.985 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 108/164 2024-05-10T10:32:04.985 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T10:32:04.985 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T10:32:04.985 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:05.021 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:32:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:32:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:32:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:32:05.214 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T10:32:10.684 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T10:32:10.685 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:32:10.686 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T10:32:10.687 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T10:32:10.688 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T10:32:10.689 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T10:32:10.690 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T10:32:10.691 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T10:32:10.692 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T10:32:10.693 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T10:32:10.694 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T10:32:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T10:32:10.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T10:32:10.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T10:32:10.696 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T10:32:13.516 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:32:13.516 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:13.516 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T10:32:13.517 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T10:32:13.519 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.520 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T10:32:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T10:32:13.522 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T10:32:13.523 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T10:32:13.524 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T10:32:13.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T10:32:13.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T10:32:13.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T10:32:13.528 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T10:32:13.529 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T10:32:13.529 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:13.529 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-10T10:32:13.932 DEBUG:teuthology.parallel:result is None 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 109/164 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-05-10T10:32:16.839 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-05-10T10:32:16.840 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:32:17.034 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 110/164 2024-05-10T10:32:17.034 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:17.034 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T10:32:17.034 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:18.511 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:32:18.538 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 111/164 2024-05-10T10:32:18.538 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:18.538 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T10:32:18.538 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:32:18.538 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T10:32:18.538 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:18.763 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:32:18.786 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 112/164 2024-05-10T10:32:18.787 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:18.787 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T10:32:18.787 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:32:18.787 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T10:32:18.787 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:19.248 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:32:19.271 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 113/164 2024-05-10T10:32:19.271 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:19.272 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T10:32:19.272 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:32:19.272 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T10:32:19.272 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:19.378 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-10T10:32:19.427 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-10T10:32:19.482 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-10T10:32:19.668 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-10T10:32:19.723 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-10T10:32:19.802 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-10T10:32:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-10T10:32:20.030 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-10T10:32:20.082 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-10T10:32:20.114 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-10T10:32:20.141 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-10T10:32:20.578 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:32:20.606 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 125/164 2024-05-10T10:32:20.638 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 126/164 2024-05-10T10:32:20.710 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 127/164 2024-05-10T10:32:20.738 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:32:21.785 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 128/164 2024-05-10T10:32:21.823 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:32:22.787 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 129/164 2024-05-10T10:32:22.814 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:32:22.926 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 130/164 2024-05-10T10:32:23.094 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 131/164 2024-05-10T10:32:23.125 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:32:23.149 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 132/164 2024-05-10T10:32:23.149 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:23.149 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T10:32:23.149 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:32:23.149 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T10:32:23.149 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:23.193 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:32:23.204 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 133/164 2024-05-10T10:32:23.301 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 134/164 2024-05-10T10:32:23.352 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 135/164 2024-05-10T10:32:24.280 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-10T10:32:24.311 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:32:24.335 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 137/164 2024-05-10T10:32:24.335 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:24.335 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T10:32:24.335 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:32:24.335 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T10:32:24.336 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:24.379 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:32:24.401 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 138/164 2024-05-10T10:32:24.402 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:24.402 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T10:32:24.402 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:24.446 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:32:24.470 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 139/164 2024-05-10T10:32:24.470 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:24.470 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T10:32:24.470 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:32:24.470 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T10:32:24.470 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:24.772 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:32:24.795 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 140/164 2024-05-10T10:32:24.795 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T10:32:24.795 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T10:32:24.796 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:32:24.796 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T10:32:24.796 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:29.459 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 141/164 2024-05-10T10:32:29.500 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 142/164 2024-05-10T10:32:29.541 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 143/164 2024-05-10T10:32:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-10T10:32:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-10T10:32:29.794 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 146/164 2024-05-10T10:32:29.838 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-10T10:32:29.868 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 148/164 2024-05-10T10:32:29.898 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-10T10:32:29.944 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-10T10:32:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:32:30.616 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:32:30.636 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-10T10:32:31.467 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 152/164 2024-05-10T10:32:31.546 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-10T10:32:31.590 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-10T10:32:31.687 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-10T10:32:31.724 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-10T10:32:31.749 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-10T10:32:31.749 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:32:31.826 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-10T10:32:31.859 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-10T10:32:31.859 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:32:31.943 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-10T10:32:31.968 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-10T10:32:31.973 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:32:31.973 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:32:32.162 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-10T10:32:32.162 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:32:32.247 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-10T10:32:32.319 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 1/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 3/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 4/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3449.gd39fa 5/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 6/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 7/164 2024-05-10T10:32:38.171 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 8/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 9/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 10/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 11/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 12/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 13/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 14/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_ 15/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 16/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 17/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_6 18/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86 19/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 20/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el 21/164 2024-05-10T10:32:38.172 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.0.0-3449.gd39fa088.el9. 22/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_6 23/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 24/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 25/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 26/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 27/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 28/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 29/164 2024-05-10T10:32:38.173 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088. 30/164 2024-05-10T10:32:38.186 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noa 31/164 2024-05-10T10:32:38.186 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.n 32/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39 33/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el 34/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 35/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.e 36/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 37/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 38/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 39/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-10T10:32:38.187 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-10T10:32:38.188 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-10T10:32:38.189 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-10T10:32:38.190 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-10T10:32:38.191 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-10T10:32:38.192 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-10T10:32:38.193 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-10T10:32:38.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-10T10:32:38.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 149/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 151/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-10T10:32:38.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-10T10:32:38.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-10T10:32:38.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-10T10:32:38.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-10T10:32:38.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-10T10:32:38.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-10T10:32:40.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-10T10:32:40.741 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:40.741 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-05-10T10:32:40.741 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:40.741 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:40.741 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:40.741 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.742 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.743 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.0.0-3449.gd39fa088.el9.noarch 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T10:32:40.744 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.745 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T10:32:40.746 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T10:32:40.747 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T10:32:40.748 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T10:32:40.749 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T10:32:40.750 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T10:32:40.751 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T10:32:40.752 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T10:32:40.753 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T10:32:40.753 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T10:32:40.753 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T10:32:40.753 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T10:32:40.753 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-10T10:32:40.754 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T10:32:40.754 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T10:32:40.754 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T10:32:40.754 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.0.0-3449.gd39fa088.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T10:32:40.755 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T10:32:40.756 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T10:32:40.756 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:40.756 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-05-10T10:32:41.216 DEBUG:teuthology.parallel:result is None 2024-05-10T10:32:41.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:32:41.359 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T10:32:41.386 INFO:teuthology.orchestra.run.smithi053.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T10:32:41.387 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T10:32:41.387 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T10:32:41.389 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d39fa08867836ac070c47cb66c782fa559fd480b 2024-05-10T10:32:41.530 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T10:32:41.553 INFO:teuthology.orchestra.run.smithi135.stdout:19.0.0-3449.gd39fa088.el9 2024-05-10T10:32:41.554 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3449.gd39fa088.el9 2024-05-10T10:32:41.554 INFO:teuthology.task.install:The correct ceph version 19.0.0-3449.gd39fa088 is installed. 2024-05-10T10:32:41.556 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T10:32:41.556 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:41.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T10:32:41.590 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:41.590 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T10:32:41.621 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T10:32:41.621 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:41.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T10:32:41.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T10:32:41.728 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:41.728 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T10:32:41.753 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T10:32:41.818 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T10:32:41.818 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:41.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T10:32:41.847 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T10:32:41.914 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:41.914 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T10:32:41.939 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T10:32:42.003 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T10:32:42.004 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:42.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T10:32:42.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T10:32:42.096 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:42.096 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T10:32:42.120 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T10:32:42.185 INFO:teuthology.run_tasks:Running task ceph... 2024-05-10T10:32:42.282 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-10T10:32:42.283 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2024-05-10T10:32:42.285 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /var/log/ceph 2024-05-10T10:32:42.310 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-10T10:32:42.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-10T10:32:42.352 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-10T10:32:42.376 INFO:tasks.ceph:Creating extra log directories... 2024-05-10T10:32:42.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-10T10:32:42.418 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-10T10:32:42.450 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-10T10:32:42.450 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-10T10:32:42.450 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-10T10:32:42.451 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:42.451 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-10T10:32:42.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-10T10:32:42.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-10T10:32:43.289 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:43.290 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-10T10:32:43.309 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-10T10:32:43.424 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-10T10:32:43.461 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-10T10:32:43.461 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-10T10:32:43.461 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909', 'branch': 'wip-63896-from-wip-rishabh-testing-20240501.193033', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7700909', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-63896-from-wip-rishabh-testing-20240501.193033'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-63896-from-wip-rishabh-testing-20240501.193033', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'}}, 'owner': 'scheduled_dparmar@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1698, 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'sleep_before_teardown': 0, 'subset': '434/512', 'suite': 'fs', 'suite_branch': 'wip-63896-from-wip-rishabh-testing-20240501.193033', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d39fa08867836ac070c47cb66c782fa559fd480b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK170pW49litN+5EbfmOAYDBcgpt4N1Jca05xmku3ldzuDjm5Yj2KxZb/DnlrBojCr2amhUs1bsPFiBxvhTOFfI=', 'smithi135.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPLvx9cv2KhJqM1O6dcHDgOYHlOVLnnfhQIv2H3uj3O9SvjWai9Cj26e45Cofe6oBhrPFOKX32Bn3RR8OcmV5Js='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd39fa08867836ac070c47cb66c782fa559fd480b', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-10_07:35:43', 'tube': 'smithi', 'user': 'dparmar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-10T10:32:43.462 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-10T10:32:43.465 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-10T10:32:43.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2024-05-10T10:32:43.522 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/run/ceph 2024-05-10T10:32:43.578 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:43.578 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T10:32:43.602 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T10:32:43.602 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-05-10T10:32:43.658 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T10:32:43.658 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:43.658 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-10T10:32:43.658 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:43.659 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:43.659 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-10 10:32:05.612615477 +0000 2024-05-10T10:32:43.659 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-10 10:30:11.170357030 +0000 2024-05-10T10:32:43.659 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-10 10:30:11.170357030 +0000 2024-05-10T10:32:43.659 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-10 10:30:11.170357030 +0000 2024-05-10T10:32:43.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T10:32:43.722 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-10T10:32:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-10T10:32:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000103918 s, 4.9 MB/s 2024-05-10T10:32:43.724 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T10:32:43.781 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-05-10T10:32:43.837 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-10 10:32:05.612615477 +0000 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-10 10:30:11.509350611 +0000 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-10 10:30:11.509350611 +0000 2024-05-10T10:32:43.838 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-10 10:30:11.509350611 +0000 2024-05-10T10:32:43.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T10:32:43.902 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-10T10:32:43.903 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-10T10:32:43.903 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000100059 s, 5.1 MB/s 2024-05-10T10:32:43.904 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T10:32:43.961 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-10 10:32:05.613615457 +0000 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-10 10:30:11.849344174 +0000 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-10 10:30:11.849344174 +0000 2024-05-10T10:32:44.017 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-10 10:30:11.849344174 +0000 2024-05-10T10:32:44.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T10:32:44.080 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-10T10:32:44.081 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-10T10:32:44.081 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000101316 s, 5.1 MB/s 2024-05-10T10:32:44.082 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T10:32:44.139 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-05-10T10:32:44.197 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T10:32:44.197 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:44.198 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-10T10:32:44.198 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:44.198 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:44.198 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-10 10:32:05.613615457 +0000 2024-05-10T10:32:44.198 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-10 10:30:12.192337681 +0000 2024-05-10T10:32:44.198 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-10 10:30:12.192337681 +0000 2024-05-10T10:32:44.198 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-10 10:30:12.192337681 +0000 2024-05-10T10:32:44.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T10:32:44.261 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-10T10:32:44.262 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-10T10:32:44.262 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000100575 s, 5.1 MB/s 2024-05-10T10:32:44.263 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T10:32:44.319 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T10:32:44.320 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:44.320 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T10:32:44.339 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T10:32:44.339 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-05-10T10:32:44.398 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T10:32:44.398 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:44.398 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-10T10:32:44.398 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:44.398 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:44.398 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-05-10 10:32:32.661845484 +0000 2024-05-10T10:32:44.398 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-05-10 10:30:11.178752855 +0000 2024-05-10T10:32:44.399 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-05-10 10:30:11.178752855 +0000 2024-05-10T10:32:44.399 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-05-10 10:30:11.178752855 +0000 2024-05-10T10:32:44.399 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T10:32:44.467 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-05-10T10:32:44.467 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-05-10T10:32:44.467 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000291446 s, 1.8 MB/s 2024-05-10T10:32:44.468 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T10:32:44.528 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-05-10T10:32:44.587 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T10:32:44.587 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:44.587 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-10T10:32:44.588 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:44.588 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:44.588 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-05-10 10:32:32.662845463 +0000 2024-05-10T10:32:44.588 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-05-10 10:30:11.524745694 +0000 2024-05-10T10:32:44.588 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-05-10 10:30:11.524745694 +0000 2024-05-10T10:32:44.588 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-05-10 10:30:11.524745694 +0000 2024-05-10T10:32:44.588 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T10:32:44.655 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-05-10T10:32:44.655 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-05-10T10:32:44.655 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000201518 s, 2.5 MB/s 2024-05-10T10:32:44.657 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T10:32:44.715 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-05-10T10:32:44.772 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-05-10 10:32:32.662845463 +0000 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-05-10 10:30:11.882738284 +0000 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-05-10 10:30:11.882738284 +0000 2024-05-10T10:32:44.773 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-05-10 10:30:11.882738284 +0000 2024-05-10T10:32:44.773 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T10:32:44.838 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-05-10T10:32:44.838 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-05-10T10:32:44.838 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000216108 s, 2.4 MB/s 2024-05-10T10:32:44.839 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T10:32:44.897 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-05-10 10:32:32.663845443 +0000 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-05-10 10:30:12.228731123 +0000 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-05-10 10:30:12.228731123 +0000 2024-05-10T10:32:44.954 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-05-10 10:30:12.228731123 +0000 2024-05-10T10:32:44.955 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T10:32:45.019 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-05-10T10:32:45.019 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-05-10T10:32:45.019 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000205245 s, 2.5 MB/s 2024-05-10T10:32:45.020 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T10:32:45.078 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T10:32:45.078 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-10T10:32:45.078 INFO:tasks.ceph:Generating config... 2024-05-10T10:32:45.080 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-10T10:32:45.080 INFO:tasks.ceph:[client] debug client = 20 2024-05-10T10:32:45.080 INFO:tasks.ceph:[client] debug ms = 1 2024-05-10T10:32:45.080 INFO:tasks.ceph:[client] pid file = 2024-05-10T10:32:45.080 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-10T10:32:45.080 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-10T10:32:45.080 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-10T10:32:45.080 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-10T10:32:45.080 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-10T10:32:45.080 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-10T10:32:45.080 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-10T10:32:45.080 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-10T10:32:45.081 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-10T10:32:45.081 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-10T10:32:45.081 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-10T10:32:45.082 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-10T10:32:45.082 INFO:tasks.ceph:Setting up mon.a... 2024-05-10T10:32:45.083 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-10T10:32:45.136 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.keyring 2024-05-10T10:32:45.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-10T10:32:45.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-10T10:32:45.213 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.53'), ('mon.b', '172.21.15.135'), ('mon.c', '[v2:172.21.15.135:3301,v1:172.21.15.135:6790]')] 2024-05-10T10:32:45.213 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-10T10:32:45.214 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:45.214 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-10T10:32:45.270 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.53 --add b 172.21.15.135 --addv c '[v2:172.21.15.135:3301,v1:172.21.15.135:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: generated fsid 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = quincy 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 0 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:fsid 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:last_changed 2024-05-10T10:32:45.359416+0000 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-05-10T10:32:45.359416+0000 2024-05-10T10:32:45.359 INFO:teuthology.orchestra.run.smithi053.stdout:min_mon_release 17 (quincy) 2024-05-10T10:32:45.360 INFO:teuthology.orchestra.run.smithi053.stdout:election_strategy: 1 2024-05-10T10:32:45.360 INFO:teuthology.orchestra.run.smithi053.stdout:0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-05-10T10:32:45.360 INFO:teuthology.orchestra.run.smithi053.stdout:1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-05-10T10:32:45.360 INFO:teuthology.orchestra.run.smithi053.stdout:2: [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] mon.c 2024-05-10T10:32:45.360 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-10T10:32:45.361 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-10T10:32:45.377 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 543b513e-90c7-47ff-9cdd-f7fa4d978734... 2024-05-10T10:32:45.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:32:45.420 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:32:45.463 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-05-10T10:32:45.463 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-05-10T10:32:45.463 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:32:45.463 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-05-10T10:32:45.463 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:32:45.464 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:32:45.465 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-05-10T10:32:45.466 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.467 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-05-10T10:32:45.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2024-05-10T10:32:45.469 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:32:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-05-10T10:32:45.471 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:32:45.472 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: mon down mkfs grace = 300 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2024-05-10T10:32:45.473 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2024-05-10T10:32:45.474 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:32:45.475 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-05-10T10:32:45.476 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-05-10T10:32:45.477 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:32:45.478 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.479 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug trim objects = true 2024-05-10T10:32:45.480 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.481 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 20 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 20 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 10 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-05-10T10:32:45.482 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:32:45.483 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.484 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: mon down mkfs grace = 300 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-05-10T10:32:45.485 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = "" 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-05-10T10:32:45.486 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = True 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = True 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:32:45.487 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:32:45.489 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-10T10:32:45.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-10T10:32:45.584 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-10T10:32:45.584 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:45.584 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-10T10:32:45.600 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:45.600 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-10T10:32:45.656 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:32:45.656 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:45.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-10T10:32:45.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-10T10:32:45.732 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:32:45.732 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:45.788 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:32:45.789 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:45.789 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-10T10:32:45.789 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-10T10:32:45.827 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:32:45.828 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:45.884 INFO:tasks.ceph:Setting up mon nodes... 2024-05-10T10:32:45.884 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-10T10:32:45.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-10T10:32:45.938 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-10T10:32:45.940 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-10T10:32:46.001 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-10T10:32:46.004 INFO:tasks.ceph:Setting up mds nodes... 2024-05-10T10:32:46.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-10T10:32:46.057 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-10T10:32:46.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-10T10:32:46.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-10T10:32:46.178 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-10T10:32:46.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-10T10:32:46.206 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-10T10:32:46.261 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-10T10:32:46.263 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-10T10:32:46.290 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-10T10:32:46.380 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-10T10:32:46.382 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-10T10:32:46.406 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-10T10:32:46.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T10:32:46.450 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-10T10:32:46.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-10T10:32:46.546 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-10T10:32:46.558 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-10T10:32:46.599 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-10T10:32:46.612 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-10T10:32:46.693 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-10T10:32:46.705 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-10T10:32:46.705 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-10T10:32:46.706 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-10T10:32:46.732 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T10:32:46.732 INFO:tasks.ceph:role: osd.0 2024-05-10T10:32:46.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:32:46.732 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:32:46.798 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:32:46.830 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-05-10T10:32:46.832 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-05-10T10:32:46.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-10T10:32:46.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-10T10:32:46.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-10T10:32:47.036 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T10:32:47.036 INFO:tasks.ceph:role: osd.1 2024-05-10T10:32:47.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:32:47.037 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-10T10:32:47.101 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:32:47.102 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:32:47.134 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-05-10T10:32:47.135 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-05-10T10:32:47.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-10T10:32:47.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-10T10:32:47.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-10T10:32:47.342 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T10:32:47.342 INFO:tasks.ceph:role: osd.2 2024-05-10T10:32:47.342 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:32:47.342 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:32:47.408 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:32:47.440 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-05-10T10:32:47.442 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-05-10T10:32:47.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-10T10:32:47.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-10T10:32:47.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-10T10:32:47.647 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T10:32:47.647 INFO:tasks.ceph:role: osd.3 2024-05-10T10:32:47.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:32:47.647 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-10T10:32:47.713 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:32:47.714 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:32:47.747 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-05-10T10:32:47.748 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-05-10T10:32:47.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-10T10:32:47.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-10T10:32:47.887 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:47.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:47.994+0000 7fa7e9c15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-10T10:32:47.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:47.995+0000 7fa7e9c15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-10T10:32:47.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:47.995+0000 7fa7e9c15640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-10T10:32:47.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:47.995+0000 7fa7e9c15640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-10T10:32:47.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:47.995+0000 7fa7e9c15640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-10T10:32:47.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:47.995+0000 7fa7e9c15640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-10T10:32:50.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-10T10:32:50.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:50.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:50.978+0000 7fc8b49d4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-10T10:32:50.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:50.979+0000 7fc8b49d4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-10T10:32:50.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:50.979+0000 7fc8b49d4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-10T10:32:50.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:50.979+0000 7fc8b49d4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-10T10:32:50.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:50.979+0000 7fc8b49d4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-10T10:32:50.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:50.979+0000 7fc8b49d4640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-10T10:32:53.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-10T10:32:53.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:53.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:53.959+0000 7f56b0dc4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-10T10:32:53.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:53.959+0000 7f56b0dc4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-10T10:32:53.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:53.959+0000 7f56b0dc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-10T10:32:53.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:53.959+0000 7f56b0dc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-10T10:32:53.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:53.959+0000 7f56b0dc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-10T10:32:53.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:53.960+0000 7f56b0dc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-10T10:32:56.803 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-10T10:32:56.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:32:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:56.925+0000 7f6cea6ad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-10T10:32:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:56.926+0000 7f6cea6ad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-10T10:32:56.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:56.926+0000 7f6cea6ad640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-10T10:32:56.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:56.926+0000 7f6cea6ad640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-10T10:32:56.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:56.926+0000 7f6cea6ad640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-10T10:32:56.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:32:56.926+0000 7f6cea6ad640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-10T10:32:59.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-10T10:32:59.807 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-10T10:32:59.838 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T10:32:59.838 INFO:tasks.ceph:role: osd.4 2024-05-10T10:32:59.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:32:59.838 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-10T10:32:59.915 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:32:59.915 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:32:59.915 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:32:59.915 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:32:59.915 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:32:59.915 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:32:59.916 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:32:59.916 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:32:59.916 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:32:59.916 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:33:00.415 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-05-10T10:33:00.417 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-05-10T10:33:00.417 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-10T10:33:00.538 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-10T10:33:00.574 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-10T10:33:00.640 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T10:33:00.640 INFO:tasks.ceph:role: osd.5 2024-05-10T10:33:00.641 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:33:00.641 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-10T10:33:00.714 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:33:00.714 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:33:00.714 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:33:00.714 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:33:00.714 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:33:00.715 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:33:00.715 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:33:00.715 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:33:00.715 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:33:00.715 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:33:01.176 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-05-10T10:33:01.179 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-05-10T10:33:01.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-10T10:33:01.271 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-10T10:33:01.301 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-10T10:33:01.367 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T10:33:01.367 INFO:tasks.ceph:role: osd.6 2024-05-10T10:33:01.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:33:01.367 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-10T10:33:01.440 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:33:01.440 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:33:01.440 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:33:01.440 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:33:01.440 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:33:01.440 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:33:01.440 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:33:01.441 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:33:01.441 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:33:01.441 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:33:01.920 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-05-10T10:33:01.922 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-05-10T10:33:01.922 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-10T10:33:02.052 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-10T10:33:02.084 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-10T10:33:02.150 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T10:33:02.150 INFO:tasks.ceph:role: osd.7 2024-05-10T10:33:02.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:33:02.151 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-10T10:33:02.223 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T10:33:02.223 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T10:33:02.223 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T10:33:02.223 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T10:33:02.223 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T10:33:02.223 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-05-10T10:33:02.224 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T10:33:02.224 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T10:33:02.224 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T10:33:02.224 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T10:33:02.718 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-05-10T10:33:02.720 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-05-10T10:33:02.721 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-10T10:33:02.838 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-10T10:33:02.869 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:33:02.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:02.966+0000 7f7567b44640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-10T10:33:02.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:02.967+0000 7f7567b44640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-10T10:33:02.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:02.967+0000 7f7567b44640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-10T10:33:02.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:02.967+0000 7f7567b44640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-10T10:33:02.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:02.967+0000 7f7567b44640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-10T10:33:02.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:02.967+0000 7f7567b44640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-10T10:33:05.841 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-10T10:33:05.870 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:33:05.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:05.960+0000 7f192111d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-10T10:33:05.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:05.960+0000 7f192111d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-10T10:33:05.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:05.960+0000 7f192111d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-10T10:33:05.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:05.960+0000 7f192111d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-10T10:33:05.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:05.960+0000 7f192111d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-10T10:33:05.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:05.961+0000 7f192111d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-10T10:33:08.810 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-10T10:33:08.852 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:33:08.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:08.950+0000 7fd3590af640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-10T10:33:08.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:08.951+0000 7fd3590af640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-10T10:33:08.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:08.951+0000 7fd3590af640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-10T10:33:08.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:08.951+0000 7fd3590af640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-10T10:33:08.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:08.951+0000 7fd3590af640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-10T10:33:08.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:08.951+0000 7fd3590af640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-10T10:33:11.812 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-10T10:33:11.842 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:33:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:11.931+0000 7f3f07232640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-10T10:33:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:11.931+0000 7f3f07232640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-10T10:33:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:11.931+0000 7f3f07232640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-10T10:33:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:11.931+0000 7f3f07232640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-10T10:33:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:11.931+0000 7f3f07232640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-10T10:33:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:11.932+0000 7f3f07232640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-10T10:33:13.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:33:13.469 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:33:14.786 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-10T10:33:14.814 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-10T10:33:14.815 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:14.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-10T10:33:14.846 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:14.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-10T10:33:14.916 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:14.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-10T10:33:14.985 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:14.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-10T10:33:15.054 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:15.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-10T10:33:15.123 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:15.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-10T10:33:15.190 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:15.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-10T10:33:15.257 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.257 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-10T10:33:15.287 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.287 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-10T10:33:15.355 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.356 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-10T10:33:15.423 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.423 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-10T10:33:15.489 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.489 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-10T10:33:15.555 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.555 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-10T10:33:15.621 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.621 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-10T10:33:15.686 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:15.687 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-10T10:33:15.704 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:15.704 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-10T10:33:15.761 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.762 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-10T10:33:15.778 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:15.778 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-10T10:33:15.834 INFO:tasks.ceph:Adding keys to all mons... 2024-05-10T10:33:15.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-10T10:33:15.837 DEBUG:teuthology.orchestra.run.smithi135:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-10T10:33:15.877 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.y] 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ== 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg== 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA== 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA== 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA== 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw== 2024-05-10T10:33:15.878 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA== 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw== 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ== 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew== 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g== 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A== 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.6] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA== 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.7] 2024-05-10T10:33:15.879 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg== 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout:[client.2] 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw== 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout:[client.1] 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g== 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout:[client.3] 2024-05-10T10:33:15.880 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw== 2024-05-10T10:33:15.898 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.y] 2024-05-10T10:33:15.898 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ== 2024-05-10T10:33:15.898 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-05-10T10:33:15.898 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg== 2024-05-10T10:33:15.898 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-05-10T10:33:15.898 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA== 2024-05-10T10:33:15.898 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.0] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA== 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.1] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA== 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.2] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw== 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.3] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA== 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.x] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw== 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ== 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew== 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.4] 2024-05-10T10:33:15.899 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g== 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.5] 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A== 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.6] 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA== 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.7] 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg== 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout:[client.0] 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout:[client.2] 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw== 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout:[client.1] 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g== 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout:[client.3] 2024-05-10T10:33:15.900 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw== 2024-05-10T10:33:15.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T10:33:15.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T10:33:15.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:15.994 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:16.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:16.050 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:16.104 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.106 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.158 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.160 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.213 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.266 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T10:33:16.318 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T10:33:16.369 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:16.372 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:16.426 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:16.429 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T10:33:16.482 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.485 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.538 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.591 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.645 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T10:33:16.696 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.698 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.755 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.811 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.866 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T10:33:16.920 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-10T10:33:16.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-10T10:33:16.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-10T10:33:17.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-10T10:33:17.070 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-10T10:33:17.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-10T10:33:17.191 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-10T10:33:17.215 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-10T10:33:17.280 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-10T10:33:17.379 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-10T10:33:17.404 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:33:17.406 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-10T10:33:17.459 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-10T10:33:17.460 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-10T10:33:17.460 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-10T10:33:17.462 INFO:tasks.ceph.mon.a:Started 2024-05-10T10:33:17.463 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-10T10:33:17.463 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-10T10:33:17.502 INFO:tasks.ceph.mon.b:Started 2024-05-10T10:33:17.502 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-10T10:33:17.502 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-10T10:33:17.504 INFO:tasks.ceph.mon.c:Started 2024-05-10T10:33:17.504 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-10T10:33:17.504 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-10T10:33:17.505 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-10T10:33:17.507 INFO:tasks.ceph.mgr.y:Started 2024-05-10T10:33:17.507 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-10T10:33:17.507 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-10T10:33:17.509 INFO:tasks.ceph.mgr.x:Started 2024-05-10T10:33:17.509 INFO:tasks.ceph:Setting crush tunables to default 2024-05-10T10:33:17.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd crush tunables default 2024-05-10T10:33:17.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.613+0000 7fae55d19640 1 Processor -- start 2024-05-10T10:33:17.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.613+0000 7fae55d19640 1 -- start start 2024-05-10T10:33:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae55d19640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 0x7fae500b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae55d19640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae5015f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae55d19640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae50163a20 con 0x7fae5015f190 2024-05-10T10:33:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae55d19640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fae500b11f0 con 0x7fae5006c9d0 2024-05-10T10:33:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae55d19640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fae500b6b80 con 0x7fae500b1390 2024-05-10T10:33:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fae5015f190 0x7fae5006c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51662/0 (socket says 172.21.15.53:51662) 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4f7fe640 1 -- 172.21.15.53:0/126863048 learned_addr learned my addr 172.21.15.53:0/126863048 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4effd640 1 --2- 172.21.15.53:0/126863048 >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 0x7fae500b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4ffff640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae5015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4effd640 1 -- 172.21.15.53:0/126863048 >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 msgr2=0x7fae500b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4effd640 1 -- 172.21.15.53:0/126863048 >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 msgr2=0x7fae500b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4effd640 1 --2- 172.21.15.53:0/126863048 >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 0x7fae500b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4effd640 1 --2- 172.21.15.53:0/126863048 >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 0x7fae500b0cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4ffff640 1 -- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 msgr2=0x7fae5015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4ffff640 1 -- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 msgr2=0x7fae5015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4ffff640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae5015f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T10:33:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.614+0000 7fae4ffff640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae5015f570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.768+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 253978877 0 0) 0x7fae50163a20 con 0x7fae5015f190 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.768+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae38003520 con 0x7fae5015f190 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1935430893 0 0) 0x7fae38003520 con 0x7fae5015f190 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 msgr2=0x7fae5015f570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae5015f570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 msgr2=0x7fae500b0cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 --2- 172.21.15.53:0/126863048 >> v2:172.21.15.53:3300/0 conn(0x7fae5006c9d0 0x7fae500b0cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae500b6e60 con 0x7fae5015f190 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fae40001670 con 0x7fae5015f190 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 >> v1:172.21.15.53:6789/0 conn(0x7fae5015f190 legacy=0x7fae5006c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38003fe0 0x7fae38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae38004d40 0x7fae380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fae500b11f0 con 0x7fae38003fe0 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae500b6b80 con 0x7fae38004d40 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fae3800bac0 con 0x7fae500b1390 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4effd640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38003fe0 0x7fae38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4f7fe640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4ffff640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae38004d40 0x7fae380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4effd640 1 -- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 msgr2=0x7fae380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4effd640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae500b1390 0x7fae380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4effd640 1 -- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae38004d40 msgr2=0x7fae380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4effd640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae38004d40 0x7fae380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.769+0000 7fae4effd640 1 -- 172.21.15.53:0/126863048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38003520 con 0x7fae38003fe0 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.770+0000 7fae4effd640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38003fe0 0x7fae38004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae4400b9e0 tx=0x7fae38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bcccbba759d4d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.770+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44030070 con 0x7fae38003fe0 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.770+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4402adb0 con 0x7fae38003fe0 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.770+0000 7fae4e7fc640 1 -- 172.21.15.53:0/126863048 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4400b5e0 con 0x7fae38003fe0 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.771+0000 7fae55d19640 1 -- 172.21.15.53:0/126863048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38003fe0 msgr2=0x7fae38004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.771+0000 7fae55d19640 1 --2- 172.21.15.53:0/126863048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38003fe0 0x7fae38004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae4400b9e0 tx=0x7fae38005830 comp rx=0 tx=0).stop 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.771+0000 7fae4ffff640 1 -- 172.21.15.53:0/126863048 reap_dead start 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.771+0000 7fae55d19640 1 -- 172.21.15.53:0/126863048 shutdown_connections 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.771+0000 7fae55d19640 1 -- 172.21.15.53:0/126863048 >> 172.21.15.53:0/126863048 conn(0x7fae500d1ed0 msgr2=0x7fae500d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.771+0000 7fae55d19640 1 -- 172.21.15.53:0/126863048 shutdown_connections 2024-05-10T10:33:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.771+0000 7fae55d19640 1 -- 172.21.15.53:0/126863048 wait complete. 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.772+0000 7fae55d19640 1 Processor -- start 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.772+0000 7fae55d19640 1 -- start start 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae55d19640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae38003fe0 0x7fae501498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae55d19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae501564b0 0x7fae50154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae55d19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae500cfa70 0x7fae50158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae55d19640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fae500b52e0 con 0x7fae501564b0 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae55d19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae500b5160 con 0x7fae500cfa70 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae55d19640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fae500b5460 con 0x7fae38003fe0 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae501564b0 0x7fae50154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae38003fe0 0x7fae501498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae501564b0 0x7fae50154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41384/0 (socket says 172.21.15.53:41384) 2024-05-10T10:33:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 -- 172.21.15.53:0/1460738692 learned_addr learned my addr 172.21.15.53:0/1460738692 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4ffff640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae500cfa70 0x7fae50158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 -- 172.21.15.53:0/1460738692 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae38003fe0 msgr2=0x7fae501498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae38003fe0 0x7fae501498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 -- 172.21.15.53:0/1460738692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae500cfa70 msgr2=0x7fae50158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae500cfa70 0x7fae50158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.773+0000 7fae4effd640 1 -- 172.21.15.53:0/1460738692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae50154dc0 con 0x7fae501564b0 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae4effd640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae501564b0 0x7fae50154570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae4400bb00 tx=0x7fae44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae37fff640 1 -- 172.21.15.53:0/1460738692 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44043020 con 0x7fae501564b0 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae37fff640 1 -- 172.21.15.53:0/1460738692 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4403f680 con 0x7fae501564b0 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae501550a0 con 0x7fae501564b0 2024-05-10T10:33:17.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae37fff640 1 -- 172.21.15.53:0/1460738692 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4400b5e0 con 0x7fae501564b0 2024-05-10T10:33:17.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae50149e30 con 0x7fae501564b0 2024-05-10T10:33:17.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae37fff640 1 -- 172.21.15.53:0/1460738692 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fae44046020 con 0x7fae501564b0 2024-05-10T10:33:17.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.774+0000 7fae37fff640 1 -- 172.21.15.53:0/1460738692 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fae44006430 con 0x7fae501564b0 2024-05-10T10:33:17.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.775+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae500693f0 con 0x7fae501564b0 2024-05-10T10:33:17.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.777+0000 7fae37fff640 1 -- 172.21.15.53:0/1460738692 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fae4404d030 con 0x7fae501564b0 2024-05-10T10:33:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.844+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fae500b0bf0 con 0x7fae501564b0 2024-05-10T10:33:17.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.846+0000 7fae37fff640 1 -- 172.21.15.53:0/1460738692 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fae44006680 con 0x7fae501564b0 2024-05-10T10:33:17.846 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae501564b0 msgr2=0x7fae50154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae501564b0 0x7fae50154570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae4400bb00 tx=0x7fae44004730 comp rx=0 tx=0).stop 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 shutdown_connections 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae500cfa70 0x7fae50158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae501564b0 0x7fae50154570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 --2- 172.21.15.53:0/1460738692 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae38003fe0 0x7fae501498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 >> 172.21.15.53:0/1460738692 conn(0x7fae500d1ed0 msgr2=0x7fae500af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 shutdown_connections 2024-05-10T10:33:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:17.847+0000 7fae55d19640 1 -- 172.21.15.53:0/1460738692 wait complete. 2024-05-10T10:33:17.859 INFO:tasks.ceph:check_enable_crimson: False 2024-05-10T10:33:17.859 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-10T10:33:17.859 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:17.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-10T10:33:17.878 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:17.877+0000 7ff54df1c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T10:33:17.885 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:17.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-10T10:33:17.898 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:17.898+0000 7f086c5ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T10:33:17.923 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:17.924 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-10T10:33:17.989 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:17.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-10T10:33:18.038 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:18.037+0000 7ff54df1c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:33:18.055 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:18.055 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-10T10:33:18.061 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:18.061+0000 7f086c5ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:33:18.080 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:18.080 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-10T10:33:18.094 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:18.093+0000 7ff54df1c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T10:33:18.118 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:18.118 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-10T10:33:18.119 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:18.118+0000 7f086c5ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T10:33:18.183 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:33:18.183 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-10T10:33:18.249 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new f48a726a-57e3-4859-87d0-db3e2bf78036 0 2024-05-10T10:33:18.366 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:18.365+0000 7ff54df1c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T10:33:18.385 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:18.384+0000 7f086c5ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T10:33:18.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.386+0000 7f0863f86640 1 Processor -- start 2024-05-10T10:33:18.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.386+0000 7f0863f86640 1 -- start start 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0863f86640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f085c15bf20 0x7f085c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0863f86640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0863f86640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085c0d0390 con 0x7f085c071470 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0863f86640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f085c0afba0 con 0x7f085c15bf20 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0863f86640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f085c0d0950 con 0x7f085c073810 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0862523640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0862523640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56158/0 (socket says 172.21.15.135:56158) 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0862523640 1 -- 172.21.15.135:0/1816358436 learned_addr learned my addr 172.21.15.135:0/1816358436 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.387+0000 7f0861d22640 1 --2- 172.21.15.135:0/1816358436 >> v2:172.21.15.53:3300/0 conn(0x7f085c15bf20 0x7f085c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0862523640 1 -- 172.21.15.135:0/1816358436 >> v2:172.21.15.53:3300/0 conn(0x7f085c15bf20 msgr2=0x7f085c0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0862523640 1 --2- 172.21.15.135:0/1816358436 >> v2:172.21.15.53:3300/0 conn(0x7f085c15bf20 0x7f085c0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0862523640 1 -- 172.21.15.135:0/1816358436 >> v1:172.21.15.53:6789/0 conn(0x7f085c071470 legacy=0x7f085c073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0862523640 1 -- 172.21.15.135:0/1816358436 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c0d0670 con 0x7f085c073810 2024-05-10T10:33:18.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0860d20640 1 -- 172.21.15.135:0/1816358436 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4080875491 0 0) 0x7f085c0d0390 con 0x7f085c071470 2024-05-10T10:33:18.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0862523640 1 --2- 172.21.15.135:0/1816358436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15fbc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f085c0d0ad0 tx=0x7f085402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29db5549560bb055 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0860d20640 1 -- 172.21.15.135:0/1816358436 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854036070 con 0x7f085c073810 2024-05-10T10:33:18.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0860d20640 1 -- 172.21.15.135:0/1816358436 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0854031600 con 0x7f085c073810 2024-05-10T10:33:18.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.388+0000 7f0860d20640 1 -- 172.21.15.135:0/1816358436 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540319d0 con 0x7f085c073810 2024-05-10T10:33:18.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 -- 172.21.15.135:0/1816358436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 msgr2=0x7f085c15fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 --2- 172.21.15.135:0/1816358436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15fbc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f085c0d0ad0 tx=0x7f085402a7a0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 -- 172.21.15.135:0/1816358436 shutdown_connections 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 --2- 172.21.15.135:0/1816358436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15fbc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 --2- 172.21.15.135:0/1816358436 >> v2:172.21.15.53:3300/0 conn(0x7f085c15bf20 0x7f085c0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 -- 172.21.15.135:0/1816358436 >> 172.21.15.135:0/1816358436 conn(0x7f085c0b3fb0 msgr2=0x7f085c0d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 -- 172.21.15.135:0/1816358436 shutdown_connections 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.389+0000 7f0863f86640 1 -- 172.21.15.135:0/1816358436 wait complete. 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.390+0000 7f0863f86640 1 Processor -- start 2024-05-10T10:33:18.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.390+0000 7f0863f86640 1 -- start start 2024-05-10T10:33:18.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0863f86640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085c071470 0x7f085c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0863f86640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0863f86640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085c15bf20 0x7f085c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0863f86640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f085c0cff60 con 0x7f085c15bf20 2024-05-10T10:33:18.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0863f86640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f085c0b00b0 con 0x7f085c071470 2024-05-10T10:33:18.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0863f86640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f085c0d00e0 con 0x7f085c073810 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0861d22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085c071470 0x7f085c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0861521640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0861521640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56170/0 (socket says 172.21.15.135:56170) 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0861d22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085c071470 0x7f085c0b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60500/0 (socket says 172.21.15.135:60500) 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0861521640 1 -- 172.21.15.135:0/2074918862 learned_addr learned my addr 172.21.15.135:0/2074918862 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.391+0000 7f0862523640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085c15bf20 0x7f085c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0861d22640 1 -- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 msgr2=0x7f085c15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0861d22640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0861d22640 1 -- 172.21.15.135:0/2074918862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085c15bf20 msgr2=0x7f085c151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0861d22640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085c15bf20 0x7f085c151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0861d22640 1 -- 172.21.15.135:0/2074918862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c15a390 con 0x7f085c071470 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0861521640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0861d22640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085c071470 0x7f085c0b2db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e870 tx=0x7f084c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f08527fc640 1 -- 172.21.15.135:0/2074918862 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c017070 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f08527fc640 1 -- 172.21.15.135:0/2074918862 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c00ee60 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f08527fc640 1 -- 172.21.15.135:0/2074918862 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c00cb60 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c15a670 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c15ac30 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f08527fc640 1 -- 172.21.15.135:0/2074918862 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f084c01a070 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.392+0000 7f08527fc640 1 -- 172.21.15.135:0/2074918862 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f084c01c790 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.393+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085c0693f0 con 0x7f085c071470 2024-05-10T10:33:18.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.395+0000 7f08527fc640 1 -- 172.21.15.135:0/2074918862 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f084c00cda0 con 0x7f085c071470 2024-05-10T10:33:18.415 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:18.413+0000 7ff54df1c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T10:33:18.432 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:18.432+0000 7f086c5ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T10:33:18.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.467+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f48a726a-57e3-4859-87d0-db3e2bf78036", "id": 0} v 0) -- 0x7f085c06d040 con 0x7f085c071470 2024-05-10T10:33:18.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.471+0000 7f08527fc640 1 -- 172.21.15.135:0/2074918862 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f48a726a-57e3-4859-87d0-db3e2bf78036", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f084c01ca60 con 0x7f085c071470 2024-05-10T10:33:18.473 INFO:teuthology.orchestra.run.smithi135.stdout:0 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085c071470 msgr2=0x7f085c0b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085c071470 0x7f085c0b2db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e870 tx=0x7f084c00ed30 comp rx=0 tx=0).stop 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 shutdown_connections 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085c15bf20 0x7f085c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085c073810 0x7f085c15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 --2- 172.21.15.135:0/2074918862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085c071470 0x7f085c0b2db0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 >> 172.21.15.135:0/2074918862 conn(0x7f085c0b3fb0 msgr2=0x7f085c0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 shutdown_connections 2024-05-10T10:33:18.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.472+0000 7f0863f86640 1 -- 172.21.15.135:0/2074918862 wait complete. 2024-05-10T10:33:18.484 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 59347929-0d1d-4f6b-b017-74aeb6cdea40 1 2024-05-10T10:33:18.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.583+0000 7fa4d9e93640 1 Processor -- start 2024-05-10T10:33:18.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.583+0000 7fa4d9e93640 1 -- start start 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.583+0000 7fa4d9e93640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa4d40d1bd0 0x7fa4d40b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.583+0000 7fa4d9e93640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4d40b1ab0 con 0x7fa4d415d180 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.583+0000 7fa4d9e93640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4d40b25d0 con 0x7fa4d415d560 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.583+0000 7fa4d9e93640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa4d40b2380 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4cbfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa4d415d560 0x7fa4d40b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:34302/0 (socket says 172.21.15.135:34302) 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4cbfff640 1 -- 172.21.15.135:0/700577261 learned_addr learned my addr 172.21.15.135:0/700577261 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4d3fff640 1 --2- 172.21.15.135:0/700577261 >> v2:172.21.15.53:3300/0 conn(0x7fa4d40d1bd0 0x7fa4d40b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4d2ffd640 1 -- 172.21.15.135:0/700577261 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2778650080 0 0) 0x7fa4d40b25d0 con 0x7fa4d415d560 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4d2ffd640 1 -- 172.21.15.135:0/700577261 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4c0003520 con 0x7fa4d415d560 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4d3fff640 1 -- 172.21.15.135:0/700577261 >> v1:172.21.15.135:6789/0 conn(0x7fa4d415d560 legacy=0x7fa4d40b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4d3fff640 1 -- 172.21.15.135:0/700577261 >> v1:172.21.15.53:6789/0 conn(0x7fa4d415d180 legacy=0x7fa4d40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.584+0000 7fa4d3fff640 1 -- 172.21.15.135:0/700577261 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d40b28b0 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.585+0000 7fa4d3fff640 1 --2- 172.21.15.135:0/700577261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d40b1430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009d90 tx=0x7fa4cc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214fba4d6d62ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.585+0000 7fa4d2ffd640 1 -- 172.21.15.135:0/700577261 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc007740 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.585+0000 7fa4d2ffd640 1 -- 172.21.15.135:0/700577261 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc0078e0 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.585+0000 7fa4d2ffd640 1 -- 172.21.15.135:0/700577261 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc007cb0 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.586+0000 7fa4d9e93640 1 -- 172.21.15.135:0/700577261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d40d1bd0 msgr2=0x7fa4d40b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.586+0000 7fa4d9e93640 1 --2- 172.21.15.135:0/700577261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d40b1430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009d90 tx=0x7fa4cc02a6f0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.586+0000 7fa4d9e93640 1 -- 172.21.15.135:0/700577261 shutdown_connections 2024-05-10T10:33:18.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.586+0000 7fa4d9e93640 1 --2- 172.21.15.135:0/700577261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d40b1430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.586+0000 7fa4d9e93640 1 -- 172.21.15.135:0/700577261 >> 172.21.15.135:0/700577261 conn(0x7fa4d409ba20 msgr2=0x7fa4d409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:18.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.586+0000 7fa4d9e93640 1 -- 172.21.15.135:0/700577261 shutdown_connections 2024-05-10T10:33:18.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.586+0000 7fa4d9e93640 1 -- 172.21.15.135:0/700577261 wait complete. 2024-05-10T10:33:18.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.587+0000 7fa4d9e93640 1 Processor -- start 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.587+0000 7fa4d9e93640 1 -- start start 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.587+0000 7fa4d9e93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d415ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.587+0000 7fa4d9e93640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4d415d180 0x7fa4d40cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d9e93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d415e360 0x7fa4d40cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d9e93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d40b2c20 con 0x7fa4d415e360 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d9e93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d40b4350 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d9e93640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d4167e90 con 0x7fa4d415d180 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d415ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d415ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60502/0 (socket says 172.21.15.135:60502) 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 -- 172.21.15.135:0/1045367027 learned_addr learned my addr 172.21.15.135:0/1045367027 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4cbfff640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4d415d180 0x7fa4d40cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d3fff640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d415e360 0x7fa4d40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 -- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4d415d180 msgr2=0x7fa4d40cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4d415d180 0x7fa4d40cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 -- 172.21.15.135:0/1045367027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d415e360 msgr2=0x7fa4d40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d415e360 0x7fa4d40cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 -- 172.21.15.135:0/1045367027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d4148b80 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4cbfff640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4d415d180 0x7fa4d40cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d37fe640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d415ea90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c400 tx=0x7fa4c400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d0ff9640 1 -- 172.21.15.135:0/1045367027 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40052d0 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d0ff9640 1 -- 172.21.15.135:0/1045367027 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c4005490 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d40b0420 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d0ff9640 1 -- 172.21.15.135:0/1045367027 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4007260 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.588+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d40cca50 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.589+0000 7fa4d0ff9640 1 -- 172.21.15.135:0/1045367027 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa4c4007400 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.589+0000 7fa4d0ff9640 1 -- 172.21.15.135:0/1045367027 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fa4c4003b20 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.589+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d40693f0 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.591+0000 7fa4d0ff9640 1 -- 172.21.15.135:0/1045367027 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa4c40035f0 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.664+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd new", "uuid": "59347929-0d1d-4f6b-b017-74aeb6cdea40", "id": 1} v 0) -- 0x7fa4d40afc90 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.667+0000 7fa4d0ff9640 1 -- 172.21.15.135:0/1045367027 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "59347929-0d1d-4f6b-b017-74aeb6cdea40", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fa4c4003d70 con 0x7fa4d40d1bd0 2024-05-10T10:33:18.669 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-05-10T10:33:18.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4d40d1bd0 msgr2=0x7fa4d415ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d415ea90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c400 tx=0x7fa4c400c8c0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 shutdown_connections 2024-05-10T10:33:18.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d415e360 0x7fa4d40cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4d415d180 0x7fa4d40cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 --2- 172.21.15.135:0/1045367027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4d40d1bd0 0x7fa4d415ea90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 >> 172.21.15.135:0/1045367027 conn(0x7fa4d409ba20 msgr2=0x7fa4d409be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:18.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 shutdown_connections 2024-05-10T10:33:18.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.668+0000 7fa4d9e93640 1 -- 172.21.15.135:0/1045367027 wait complete. 2024-05-10T10:33:18.681 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 66b03867-514f-45f6-bb2b-312e9595f5c7 2 2024-05-10T10:33:18.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.780+0000 7f83b1951640 1 Processor -- start 2024-05-10T10:33:18.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.780+0000 7f83b1951640 1 -- start start 2024-05-10T10:33:18.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83b1951640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83ac15f410 0x7f83ac0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83b1951640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0d0f10 0x7f83ac0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83b1951640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ac0d2800 con 0x7f83ac0b54d0 2024-05-10T10:33:18.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83b1951640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f83ac0d2ae0 con 0x7f83ac15f410 2024-05-10T10:33:18.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83b1951640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83ac0d01d0 con 0x7f83ac0d0f10 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83ac15f410 0x7f83ac0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aa7fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0d0f10 0x7f83ac0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83ac15f410 0x7f83ac0d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60516/0 (socket says 172.21.15.135:60516) 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aa7fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0d0f10 0x7f83ac0b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56192/0 (socket says 172.21.15.135:56192) 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aaffd640 1 -- 172.21.15.135:0/2765948170 learned_addr learned my addr 172.21.15.135:0/2765948170 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aaffd640 1 -- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0d0f10 msgr2=0x7f83ac0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aaffd640 1 --2- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0d0f10 0x7f83ac0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aaffd640 1 -- 172.21.15.135:0/2765948170 >> v1:172.21.15.135:6789/0 conn(0x7f83ac0b54d0 legacy=0x7f83ac0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aaffd640 1 -- 172.21.15.135:0/2765948170 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac0d04b0 con 0x7f83ac15f410 2024-05-10T10:33:18.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.781+0000 7f83aa7fc640 1 --2- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0d0f10 0x7f83ac0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.782+0000 7f83aaffd640 1 --2- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac15f410 0x7f83ac0d09d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f839c009870 tx=0x7f839c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f4a02532ab3a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.782+0000 7f83a9ffb640 1 -- 172.21.15.135:0/2765948170 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c037070 con 0x7f83ac15f410 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.782+0000 7f83a9ffb640 1 -- 172.21.15.135:0/2765948170 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c032440 con 0x7f83ac15f410 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.782+0000 7f83a9ffb640 1 -- 172.21.15.135:0/2765948170 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c032810 con 0x7f83ac15f410 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 -- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac15f410 msgr2=0x7f83ac0d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 --2- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac15f410 0x7f83ac0d09d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f839c009870 tx=0x7f839c02a1c0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 -- 172.21.15.135:0/2765948170 shutdown_connections 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 --2- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0d0f10 0x7f83ac0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 --2- 172.21.15.135:0/2765948170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac15f410 0x7f83ac0d09d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 -- 172.21.15.135:0/2765948170 >> 172.21.15.135:0/2765948170 conn(0x7f83ac08ed30 msgr2=0x7f83ac09e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:18.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 -- 172.21.15.135:0/2765948170 shutdown_connections 2024-05-10T10:33:18.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.783+0000 7f83b1951640 1 -- 172.21.15.135:0/2765948170 wait complete. 2024-05-10T10:33:18.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.784+0000 7f83b1951640 1 Processor -- start 2024-05-10T10:33:18.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.784+0000 7f83b1951640 1 -- start start 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83b1951640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0b54d0 0x7f83ac0b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83b1951640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac0d0f10 0x7f83ac0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83b1951640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83ac15f410 0x7f83ac0a2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83b1951640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac0cffb0 con 0x7f83ac15f410 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83b1951640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac0cfe30 con 0x7f83ac0d0f10 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83b1951640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83ac0d3de0 con 0x7f83ac0b54d0 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0b54d0 0x7f83ac0b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aa7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac0d0f10 0x7f83ac0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0b54d0 0x7f83ac0b8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56194/0 (socket says 172.21.15.135:56194) 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 -- 172.21.15.135:0/1265164284 learned_addr learned my addr 172.21.15.135:0/1265164284 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:18.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83ab7fe640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83ac15f410 0x7f83ac0a2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 -- 172.21.15.135:0/1265164284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac0d0f10 msgr2=0x7f83ac0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac0d0f10 0x7f83ac0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 -- 172.21.15.135:0/1265164284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83ac15f410 msgr2=0x7f83ac0a2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83ac15f410 0x7f83ac0a2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83aaffd640 1 -- 172.21.15.135:0/1265164284 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac0a2d80 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.785+0000 7f83ab7fe640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83ac15f410 0x7f83ac0a2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83aaffd640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0b54d0 0x7f83ac0b8f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f839c000c00 tx=0x7f839c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83a3fff640 1 -- 172.21.15.135:0/1265164284 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c037040 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83a3fff640 1 -- 172.21.15.135:0/1265164284 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c042070 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83a3fff640 1 -- 172.21.15.135:0/1265164284 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c030880 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac0b9600 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac0b9bc0 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83a3fff640 1 -- 172.21.15.135:0/1265164284 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f839c02ad10 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.786+0000 7f83a3fff640 1 -- 172.21.15.135:0/1265164284 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f839c047870 con 0x7f83ac0b54d0 2024-05-10T10:33:18.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.787+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ac06a220 con 0x7f83ac0b54d0 2024-05-10T10:33:18.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.789+0000 7f83a3fff640 1 -- 172.21.15.135:0/1265164284 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f839c047460 con 0x7f83ac0b54d0 2024-05-10T10:33:18.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.862+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "66b03867-514f-45f6-bb2b-312e9595f5c7", "id": 2} v 0) -- 0x7f83ac075ce0 con 0x7f83ac0b54d0 2024-05-10T10:33:18.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.865+0000 7f83a3fff640 1 -- 172.21.15.135:0/1265164284 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "66b03867-514f-45f6-bb2b-312e9595f5c7", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f839c047b90 con 0x7f83ac0b54d0 2024-05-10T10:33:18.867 INFO:teuthology.orchestra.run.smithi135.stdout:2 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0b54d0 msgr2=0x7f83ac0b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0b54d0 0x7f83ac0b8f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f839c000c00 tx=0x7f839c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 shutdown_connections 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83ac15f410 0x7f83ac0a2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac0d0f10 0x7f83ac0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 --2- 172.21.15.135:0/1265164284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83ac0b54d0 0x7f83ac0b8f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 >> 172.21.15.135:0/1265164284 conn(0x7f83ac08ed30 msgr2=0x7f83ac06d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:18.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 shutdown_connections 2024-05-10T10:33:18.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.866+0000 7f83b1951640 1 -- 172.21.15.135:0/1265164284 wait complete. 2024-05-10T10:33:18.878 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 72678bd2-e32b-4271-9545-28b19dce7a58 3 2024-05-10T10:33:18.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.976+0000 7f787ef9e640 1 Processor -- start 2024-05-10T10:33:18.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.976+0000 7f787ef9e640 1 -- start start 2024-05-10T10:33:18.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787ef9e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f78780cd3f0 0x7f787806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787ef9e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f787806d6f0 0x7f78780b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787ef9e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787806db80 con 0x7f787806c8a0 2024-05-10T10:33:18.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787ef9e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f78780cd7d0 con 0x7f787806d6f0 2024-05-10T10:33:18.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787ef9e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f78780ce650 con 0x7f78780cd3f0 2024-05-10T10:33:18.979 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787e79d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f787806d6f0 0x7f78780b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787df9c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f78780cd3f0 0x7f787806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787df9c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f78780cd3f0 0x7f787806d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56202/0 (socket says 172.21.15.135:56202) 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787e79d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f787806d6f0 0x7f78780b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60532/0 (socket says 172.21.15.135:60532) 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787df9c640 1 -- 172.21.15.135:0/3299086176 learned_addr learned my addr 172.21.15.135:0/3299086176 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787df9c640 1 -- 172.21.15.135:0/3299086176 >> v2:172.21.15.135:3300/0 conn(0x7f787806d6f0 msgr2=0x7f78780b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787df9c640 1 --2- 172.21.15.135:0/3299086176 >> v2:172.21.15.135:3300/0 conn(0x7f787806d6f0 0x7f78780b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787df9c640 1 -- 172.21.15.135:0/3299086176 >> v1:172.21.15.135:6789/0 conn(0x7f787806c8a0 legacy=0x7f78780b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787df9c640 1 -- 172.21.15.135:0/3299086176 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78780b2500 con 0x7f78780cd3f0 2024-05-10T10:33:18.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.977+0000 7f787e79d640 1 --2- 172.21.15.135:0/3299086176 >> v2:172.21.15.135:3300/0 conn(0x7f787806d6f0 0x7f78780b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.978+0000 7f787df9c640 1 --2- 172.21.15.135:0/3299086176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f78780cd3f0 0x7f787806d1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7874009870 tx=0x7f787402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b932b07d78fa07ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.978+0000 7f787cf9a640 1 -- 172.21.15.135:0/3299086176 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874037070 con 0x7f78780cd3f0 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.978+0000 7f787cf9a640 1 -- 172.21.15.135:0/3299086176 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787402ae50 con 0x7f78780cd3f0 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.978+0000 7f787cf9a640 1 -- 172.21.15.135:0/3299086176 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874032860 con 0x7f78780cd3f0 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 -- 172.21.15.135:0/3299086176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f78780cd3f0 msgr2=0x7f787806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 --2- 172.21.15.135:0/3299086176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f78780cd3f0 0x7f787806d1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7874009870 tx=0x7f787402a300 comp rx=0 tx=0).stop 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 -- 172.21.15.135:0/3299086176 shutdown_connections 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 --2- 172.21.15.135:0/3299086176 >> v2:172.21.15.135:3300/0 conn(0x7f787806d6f0 0x7f78780b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 --2- 172.21.15.135:0/3299086176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f78780cd3f0 0x7f787806d1b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 -- 172.21.15.135:0/3299086176 >> 172.21.15.135:0/3299086176 conn(0x7f78780739c0 msgr2=0x7f7878073dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:18.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 -- 172.21.15.135:0/3299086176 shutdown_connections 2024-05-10T10:33:18.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.979+0000 7f787ef9e640 1 -- 172.21.15.135:0/3299086176 wait complete. 2024-05-10T10:33:18.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.980+0000 7f787ef9e640 1 Processor -- start 2024-05-10T10:33:18.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.980+0000 7f787ef9e640 1 -- start start 2024-05-10T10:33:18.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787ef9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787806c8a0 0x7f7878157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787ef9e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f787806d6f0 0x7f78780c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787ef9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78780cd3f0 0x7f787806ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:18.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787ef9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f787806c330 con 0x7f787806c8a0 2024-05-10T10:33:18.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787ef9e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78780ce8f0 con 0x7f78780cd3f0 2024-05-10T10:33:18.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787ef9e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f78780cebd0 con 0x7f787806d6f0 2024-05-10T10:33:18.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787d79b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f787806d6f0 0x7f78780c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787e79d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78780cd3f0 0x7f787806ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787d79b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f787806d6f0 0x7f78780c3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56216/0 (socket says 172.21.15.135:56216) 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787d79b640 1 -- 172.21.15.135:0/879624070 learned_addr learned my addr 172.21.15.135:0/879624070 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787df9c640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787806c8a0 0x7f7878157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787e79d640 1 -- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f787806d6f0 msgr2=0x7f78780c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787e79d640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f787806d6f0 0x7f78780c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787e79d640 1 -- 172.21.15.135:0/879624070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787806c8a0 msgr2=0x7f7878157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787e79d640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787806c8a0 0x7f7878157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787e79d640 1 -- 172.21.15.135:0/879624070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78780d1950 con 0x7f78780cd3f0 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787d79b640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f787806d6f0 0x7f78780c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.981+0000 7f787e79d640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78780cd3f0 0x7f787806ee10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f787000efc0 tx=0x7f787000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.982+0000 7f786e7fc640 1 -- 172.21.15.135:0/879624070 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870018030 con 0x7f78780cd3f0 2024-05-10T10:33:18.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.982+0000 7f786e7fc640 1 -- 172.21.15.135:0/879624070 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787000cd30 con 0x7f78780cd3f0 2024-05-10T10:33:18.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.982+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78780d1c30 con 0x7f78780cd3f0 2024-05-10T10:33:18.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.982+0000 7f786e7fc640 1 -- 172.21.15.135:0/879624070 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870003bb0 con 0x7f78780cd3f0 2024-05-10T10:33:18.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.982+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787815f590 con 0x7f78780cd3f0 2024-05-10T10:33:18.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.982+0000 7f786e7fc640 1 -- 172.21.15.135:0/879624070 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f787001b070 con 0x7f78780cd3f0 2024-05-10T10:33:18.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.982+0000 7f786e7fc640 1 -- 172.21.15.135:0/879624070 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f787001d980 con 0x7f78780cd3f0 2024-05-10T10:33:18.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.983+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787815f790 con 0x7f78780cd3f0 2024-05-10T10:33:18.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:18.985+0000 7f786e7fc640 1 -- 172.21.15.135:0/879624070 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7870010040 con 0x7f78780cd3f0 2024-05-10T10:33:19.059 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.057+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72678bd2-e32b-4271-9545-28b19dce7a58", "id": 3} v 0) -- 0x7f7878076680 con 0x7f78780cd3f0 2024-05-10T10:33:19.062 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.060+0000 7f786e7fc640 1 -- 172.21.15.135:0/879624070 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72678bd2-e32b-4271-9545-28b19dce7a58", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f787001d4b0 con 0x7f78780cd3f0 2024-05-10T10:33:19.062 INFO:teuthology.orchestra.run.smithi135.stdout:3 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78780cd3f0 msgr2=0x7f787806ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78780cd3f0 0x7f787806ee10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f787000efc0 tx=0x7f787000c490 comp rx=0 tx=0).stop 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 shutdown_connections 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78780cd3f0 0x7f787806ee10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f787806d6f0 0x7f78780c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 --2- 172.21.15.135:0/879624070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787806c8a0 0x7f7878157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 >> 172.21.15.135:0/879624070 conn(0x7f78780739c0 msgr2=0x7f78780ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 shutdown_connections 2024-05-10T10:33:19.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.061+0000 7f787ef9e640 1 -- 172.21.15.135:0/879624070 wait complete. 2024-05-10T10:33:19.073 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new d9463593-fb16-49dd-8f5d-c1324a27b3d5 4 2024-05-10T10:33:19.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.171+0000 7f46e575f640 1 Processor -- start 2024-05-10T10:33:19.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.171+0000 7f46e575f640 1 -- start start 2024-05-10T10:33:19.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46e575f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f46e00b08f0 0x7f46e006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46e575f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46e006f650 0x7f46e0073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46e575f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e00b0f40 con 0x7f46e015d200 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46e575f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f46e015e5f0 con 0x7f46e00b08f0 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46e575f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f46e00af4e0 con 0x7f46e006f650 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46df7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46e006f650 0x7f46e0073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46df7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46e006f650 0x7f46e0073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60554/0 (socket says 172.21.15.135:60554) 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46df7fe640 1 -- 172.21.15.135:0/1928730517 learned_addr learned my addr 172.21.15.135:0/1928730517 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.172+0000 7f46de7fc640 1 --2- 172.21.15.135:0/1928730517 >> v2:172.21.15.53:3300/0 conn(0x7f46e00b08f0 0x7f46e006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46df7fe640 1 -- 172.21.15.135:0/1928730517 >> v2:172.21.15.53:3300/0 conn(0x7f46e00b08f0 msgr2=0x7f46e006f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46df7fe640 1 --2- 172.21.15.135:0/1928730517 >> v2:172.21.15.53:3300/0 conn(0x7f46e00b08f0 0x7f46e006f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46df7fe640 1 -- 172.21.15.135:0/1928730517 >> v1:172.21.15.135:6789/0 conn(0x7f46e015d200 legacy=0x7f46e00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46df7fe640 1 -- 172.21.15.135:0/1928730517 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e00b1220 con 0x7f46e006f650 2024-05-10T10:33:19.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46df7fe640 1 --2- 172.21.15.135:0/1928730517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e006f650 0x7f46e0073700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e50 tx=0x7f46d802a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8063551e714743e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46ddffb640 1 -- 172.21.15.135:0/1928730517 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8036070 con 0x7f46e006f650 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46ddffb640 1 -- 172.21.15.135:0/1928730517 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8031600 con 0x7f46e006f650 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.173+0000 7f46ddffb640 1 -- 172.21.15.135:0/1928730517 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d80319d0 con 0x7f46e006f650 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 -- 172.21.15.135:0/1928730517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e006f650 msgr2=0x7f46e0073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 --2- 172.21.15.135:0/1928730517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e006f650 0x7f46e0073700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e50 tx=0x7f46d802a640 comp rx=0 tx=0).stop 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 -- 172.21.15.135:0/1928730517 shutdown_connections 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 --2- 172.21.15.135:0/1928730517 >> v2:172.21.15.53:3300/0 conn(0x7f46e00b08f0 0x7f46e006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 --2- 172.21.15.135:0/1928730517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e006f650 0x7f46e0073700 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 -- 172.21.15.135:0/1928730517 >> 172.21.15.135:0/1928730517 conn(0x7f46e00d1ed0 msgr2=0x7f46e00d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 -- 172.21.15.135:0/1928730517 shutdown_connections 2024-05-10T10:33:19.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.174+0000 7f46e575f640 1 -- 172.21.15.135:0/1928730517 wait complete. 2024-05-10T10:33:19.178 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:19.174+0000 7ff54df1c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T10:33:19.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.175+0000 7f46e575f640 1 Processor -- start 2024-05-10T10:33:19.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.175+0000 7f46e575f640 1 -- start start 2024-05-10T10:33:19.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46e575f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006f650 0x7f46e014ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46e575f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f46e00b08f0 0x7f46e01f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46e575f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e015d200 0x7f46e015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46e575f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f46e00731e0 con 0x7f46e006f650 2024-05-10T10:33:19.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46e575f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46e00b17e0 con 0x7f46e015d200 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46e575f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f46e0073c40 con 0x7f46e00b08f0 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46df7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e015d200 0x7f46e015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46df7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e015d200 0x7f46e015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60560/0 (socket says 172.21.15.135:60560) 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46de7fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f46e00b08f0 0x7f46e01f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46df7fe640 1 -- 172.21.15.135:0/3131808657 learned_addr learned my addr 172.21.15.135:0/3131808657 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.176+0000 7f46deffd640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006f650 0x7f46e014ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46df7fe640 1 -- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f46e00b08f0 msgr2=0x7f46e01f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46df7fe640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f46e00b08f0 0x7f46e01f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46df7fe640 1 -- 172.21.15.135:0/3131808657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006f650 msgr2=0x7f46e014ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46df7fe640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006f650 0x7f46e014ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46df7fe640 1 -- 172.21.15.135:0/3131808657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e01f30f0 con 0x7f46e015d200 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46de7fc640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f46e00b08f0 0x7f46e01f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46df7fe640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e015d200 0x7f46e015c590 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e50 tx=0x7f46d8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46d77fe640 1 -- 172.21.15.135:0/3131808657 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8036050 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46d77fe640 1 -- 172.21.15.135:0/3131808657 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8042070 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e01f33d0 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46d77fe640 1 -- 172.21.15.135:0/3131808657 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8007bd0 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e01f3990 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46d77fe640 1 -- 172.21.15.135:0/3131808657 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46d8047460 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.177+0000 7f46d77fe640 1 -- 172.21.15.135:0/3131808657 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f46d8007d70 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.178+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e0155470 con 0x7f46e015d200 2024-05-10T10:33:19.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.180+0000 7f46d77fe640 1 -- 172.21.15.135:0/3131808657 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f46d802ae20 con 0x7f46e015d200 2024-05-10T10:33:19.198 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:19.198+0000 7f086c5ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T10:33:19.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.275+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9463593-fb16-49dd-8f5d-c1324a27b3d5", "id": 4} v 0) -- 0x7f46e00693f0 con 0x7f46e015d200 2024-05-10T10:33:19.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.278+0000 7f46d77fe640 1 -- 172.21.15.135:0/3131808657 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9463593-fb16-49dd-8f5d-c1324a27b3d5", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f46d8047890 con 0x7f46e015d200 2024-05-10T10:33:19.280 INFO:teuthology.orchestra.run.smithi135.stdout:4 2024-05-10T10:33:19.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e015d200 msgr2=0x7f46e015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e015d200 0x7f46e015c590 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e50 tx=0x7f46d8004a40 comp rx=0 tx=0).stop 2024-05-10T10:33:19.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 shutdown_connections 2024-05-10T10:33:19.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46e015d200 0x7f46e015c590 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f46e00b08f0 0x7f46e01f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 --2- 172.21.15.135:0/3131808657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006f650 0x7f46e014ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 >> 172.21.15.135:0/3131808657 conn(0x7f46e00d1ed0 msgr2=0x7f46e015fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 shutdown_connections 2024-05-10T10:33:19.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.279+0000 7f46e575f640 1 -- 172.21.15.135:0/3131808657 wait complete. 2024-05-10T10:33:19.286 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:19.285+0000 7ff54df1c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T10:33:19.292 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 2c3f1892-fe80-41af-8bb4-cd35fec9c32d 5 2024-05-10T10:33:19.292 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:19.291+0000 7f086c5ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T10:33:19.333 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:19.331+0000 7ff54df1c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T10:33:19.336 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:19.336+0000 7f086c5ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T10:33:19.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.390+0000 7fccc0c8c640 1 Processor -- start 2024-05-10T10:33:19.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.390+0000 7fccc0c8c640 1 -- start start 2024-05-10T10:33:19.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.391+0000 7fccc0c8c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fccbc15bf20 0x7fccbc0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.391+0000 7fccc0c8c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc073810 0x7fccbc15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.391+0000 7fccc0c8c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccbc0d0390 con 0x7fccbc071470 2024-05-10T10:33:19.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.391+0000 7fccc0c8c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fccbc0afba0 con 0x7fccbc15bf20 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.391+0000 7fccc0c8c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fccbc0d0950 con 0x7fccbc073810 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccba59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fccbc15bf20 0x7fccbc0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc073810 0x7fccbc15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc073810 0x7fccbc15fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56226/0 (socket says 172.21.15.135:56226) 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccba59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fccbc15bf20 0x7fccbc0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60568/0 (socket says 172.21.15.135:60568) 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccbad9d640 1 -- 172.21.15.135:0/1179201064 learned_addr learned my addr 172.21.15.135:0/1179201064 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccbad9d640 1 -- 172.21.15.135:0/1179201064 >> v2:172.21.15.135:3300/0 conn(0x7fccbc15bf20 msgr2=0x7fccbc0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccbad9d640 1 --2- 172.21.15.135:0/1179201064 >> v2:172.21.15.135:3300/0 conn(0x7fccbc15bf20 0x7fccbc0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccbad9d640 1 -- 172.21.15.135:0/1179201064 >> v1:172.21.15.53:6789/0 conn(0x7fccbc071470 legacy=0x7fccbc073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccbad9d640 1 -- 172.21.15.135:0/1179201064 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc0d0670 con 0x7fccbc073810 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.392+0000 7fccba59c640 1 --2- 172.21.15.135:0/1179201064 >> v2:172.21.15.135:3300/0 conn(0x7fccbc15bf20 0x7fccbc0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccbad9d640 1 --2- 172.21.15.135:0/1179201064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc073810 0x7fccbc15fbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0d0ad0 tx=0x7fccb402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad009db35319ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccb959a640 1 -- 172.21.15.135:0/1179201064 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4036070 con 0x7fccbc073810 2024-05-10T10:33:19.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccb959a640 1 -- 172.21.15.135:0/1179201064 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb4031600 con 0x7fccbc073810 2024-05-10T10:33:19.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccb959a640 1 -- 172.21.15.135:0/1179201064 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb40319d0 con 0x7fccbc073810 2024-05-10T10:33:19.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccc0c8c640 1 -- 172.21.15.135:0/1179201064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc073810 msgr2=0x7fccbc15fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccc0c8c640 1 --2- 172.21.15.135:0/1179201064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc073810 0x7fccbc15fbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0d0ad0 tx=0x7fccb402a7a0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccc0c8c640 1 -- 172.21.15.135:0/1179201064 shutdown_connections 2024-05-10T10:33:19.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccc0c8c640 1 --2- 172.21.15.135:0/1179201064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc073810 0x7fccbc15fbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccc0c8c640 1 --2- 172.21.15.135:0/1179201064 >> v2:172.21.15.135:3300/0 conn(0x7fccbc15bf20 0x7fccbc0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccc0c8c640 1 -- 172.21.15.135:0/1179201064 >> 172.21.15.135:0/1179201064 conn(0x7fccbc0b3fb0 msgr2=0x7fccbc0d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.393+0000 7fccc0c8c640 1 -- 172.21.15.135:0/1179201064 shutdown_connections 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.394+0000 7fccc0c8c640 1 -- 172.21.15.135:0/1179201064 wait complete. 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.395+0000 7fccc0c8c640 1 Processor -- start 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.395+0000 7fccc0c8c640 1 -- start start 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.395+0000 7fccc0c8c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccbc071470 0x7fccbc159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.395+0000 7fccc0c8c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccbc073810 0x7fccbc156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccc0c8c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc15bf20 0x7fccbc15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccc0c8c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc0cff60 con 0x7fccbc071470 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccc0c8c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc0b00b0 con 0x7fccbc073810 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccc0c8c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fccbc0d00e0 con 0x7fccbc15bf20 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc15bf20 0x7fccbc15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccb9d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccbc073810 0x7fccbc156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc15bf20 0x7fccbc15af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56228/0 (socket says 172.21.15.135:56228) 2024-05-10T10:33:19.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 -- 172.21.15.135:0/914539024 learned_addr learned my addr 172.21.15.135:0/914539024 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccba59c640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccbc071470 0x7fccbc159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 -- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccbc073810 msgr2=0x7fccbc156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccbc073810 0x7fccbc156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 -- 172.21.15.135:0/914539024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccbc071470 msgr2=0x7fccbc159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccbc071470 0x7fccbc159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 -- 172.21.15.135:0/914539024 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc159660 con 0x7fccbc15bf20 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccb9d9b640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccbc073810 0x7fccbc156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccbad9d640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc15bf20 0x7fccbc15af00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fccb402ad90 tx=0x7fccb4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccb2ffd640 1 -- 172.21.15.135:0/914539024 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4036050 con 0x7fccbc15bf20 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccb2ffd640 1 -- 172.21.15.135:0/914539024 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb4042070 con 0x7fccbc15bf20 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccbc159e60 con 0x7fccbc15bf20 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccb2ffd640 1 -- 172.21.15.135:0/914539024 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4009510 con 0x7fccbc15bf20 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.396+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fccbc15a420 con 0x7fccbc15bf20 2024-05-10T10:33:19.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.397+0000 7fccb2ffd640 1 -- 172.21.15.135:0/914539024 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fccb4007bf0 con 0x7fccbc15bf20 2024-05-10T10:33:19.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.397+0000 7fccb2ffd640 1 -- 172.21.15.135:0/914539024 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fccb403d450 con 0x7fccbc15bf20 2024-05-10T10:33:19.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.397+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccbc0693f0 con 0x7fccbc15bf20 2024-05-10T10:33:19.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.399+0000 7fccb2ffd640 1 -- 172.21.15.135:0/914539024 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fccb4047460 con 0x7fccbc15bf20 2024-05-10T10:33:19.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.471+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2c3f1892-fe80-41af-8bb4-cd35fec9c32d", "id": 5} v 0) -- 0x7fccbc0d2cb0 con 0x7fccbc15bf20 2024-05-10T10:33:19.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.474+0000 7fccb2ffd640 1 -- 172.21.15.135:0/914539024 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2c3f1892-fe80-41af-8bb4-cd35fec9c32d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fccb4047640 con 0x7fccbc15bf20 2024-05-10T10:33:19.476 INFO:teuthology.orchestra.run.smithi135.stdout:5 2024-05-10T10:33:19.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc15bf20 msgr2=0x7fccbc15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc15bf20 0x7fccbc15af00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fccb402ad90 tx=0x7fccb4004a40 comp rx=0 tx=0).stop 2024-05-10T10:33:19.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 shutdown_connections 2024-05-10T10:33:19.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fccbc15bf20 0x7fccbc15af00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccbc073810 0x7fccbc156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 --2- 172.21.15.135:0/914539024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccbc071470 0x7fccbc159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 >> 172.21.15.135:0/914539024 conn(0x7fccbc0b3fb0 msgr2=0x7fccbc0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 shutdown_connections 2024-05-10T10:33:19.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.475+0000 7fccc0c8c640 1 -- 172.21.15.135:0/914539024 wait complete. 2024-05-10T10:33:19.487 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 128a5eac-c0c4-4f21-8531-d626e5edf1bc 6 2024-05-10T10:33:19.511 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:19.511+0000 7f086c5ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T10:33:19.512 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:19.510+0000 7ff54df1c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T10:33:19.561 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:19.561+0000 7f086c5ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T10:33:19.570 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:19.568+0000 7ff54df1c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T10:33:19.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.611+0000 7f0e09b9d640 1 Processor -- start 2024-05-10T10:33:19.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.611+0000 7f0e09b9d640 1 -- start start 2024-05-10T10:33:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e09b9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0e040cc4a0 0x7f0e0415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e09b9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0e04071540 0x7f0e040ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e09b9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cff30 0x7f0e040cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e09b9d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0e040d03c0 con 0x7f0e040cc4a0 2024-05-10T10:33:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e09b9d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0e040ccdc0 con 0x7f0e04071540 2024-05-10T10:33:19.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e09b9d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0415fc20 con 0x7f0e040cff30 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e03fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cff30 0x7f0e040cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e03fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cff30 0x7f0e040cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56230/0 (socket says 172.21.15.135:56230) 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e03fff640 1 -- 172.21.15.135:0/2069460369 learned_addr learned my addr 172.21.15.135:0/2069460369 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0dfbfff640 1 --2- 172.21.15.135:0/2069460369 >> v2:172.21.15.135:3300/0 conn(0x7f0e04071540 0x7f0e040ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.612+0000 7f0e037fe640 1 --2- 172.21.15.135:0/2069460369 >> v2:172.21.15.53:3300/0 conn(0x7f0e040cc4a0 0x7f0e0415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e03fff640 1 -- 172.21.15.135:0/2069460369 >> v2:172.21.15.135:3300/0 conn(0x7f0e04071540 msgr2=0x7f0e040ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e03fff640 1 --2- 172.21.15.135:0/2069460369 >> v2:172.21.15.135:3300/0 conn(0x7f0e04071540 0x7f0e040ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e03fff640 1 -- 172.21.15.135:0/2069460369 >> v2:172.21.15.53:3300/0 conn(0x7f0e040cc4a0 msgr2=0x7f0e0415f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e03fff640 1 --2- 172.21.15.135:0/2069460369 >> v2:172.21.15.53:3300/0 conn(0x7f0e040cc4a0 0x7f0e0415f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e03fff640 1 -- 172.21.15.135:0/2069460369 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0415ff00 con 0x7f0e040cff30 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e037fe640 1 --2- 172.21.15.135:0/2069460369 >> v2:172.21.15.53:3300/0 conn(0x7f0e040cc4a0 0x7f0e0415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e03fff640 1 --2- 172.21.15.135:0/2069460369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cff30 0x7f0e040cc880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e040cd500 tx=0x7f0dfc02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1da2154e81e9322 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.613+0000 7f0e02ffd640 1 -- 172.21.15.135:0/2069460369 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc036070 con 0x7f0e040cff30 2024-05-10T10:33:19.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e02ffd640 1 -- 172.21.15.135:0/2069460369 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dfc031600 con 0x7f0e040cff30 2024-05-10T10:33:19.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e02ffd640 1 -- 172.21.15.135:0/2069460369 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc0319d0 con 0x7f0e040cff30 2024-05-10T10:33:19.617 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:19.616+0000 7f086c5ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T10:33:19.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 -- 172.21.15.135:0/2069460369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cff30 msgr2=0x7f0e040cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/2069460369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cff30 0x7f0e040cc880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e040cd500 tx=0x7f0dfc02a6a0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 -- 172.21.15.135:0/2069460369 shutdown_connections 2024-05-10T10:33:19.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/2069460369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cff30 0x7f0e040cc880 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/2069460369 >> v2:172.21.15.135:3300/0 conn(0x7f0e04071540 0x7f0e040ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/2069460369 >> v2:172.21.15.53:3300/0 conn(0x7f0e040cc4a0 0x7f0e0415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 -- 172.21.15.135:0/2069460369 >> 172.21.15.135:0/2069460369 conn(0x7f0e040b3fb0 msgr2=0x7f0e040d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 -- 172.21.15.135:0/2069460369 shutdown_connections 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.614+0000 7f0e09b9d640 1 -- 172.21.15.135:0/2069460369 wait complete. 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.615+0000 7f0e09b9d640 1 Processor -- start 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.615+0000 7f0e09b9d640 1 -- start start 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e09b9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e04071540 0x7f0e0415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e09b9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cc4a0 0x7f0e041625f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e09b9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e040cff30 0x7f0e0415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e09b9d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041607a0 con 0x7f0e040cff30 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e09b9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041604c0 con 0x7f0e04071540 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e09b9d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0e04163120 con 0x7f0e040cc4a0 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cc4a0 0x7f0e041625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e037fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e04071540 0x7f0e0415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cc4a0 0x7f0e041625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56246/0 (socket says 172.21.15.135:56246) 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0dfbfff640 1 -- 172.21.15.135:0/1189743573 learned_addr learned my addr 172.21.15.135:0/1189743573 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.616+0000 7f0e03fff640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e040cff30 0x7f0e0415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0dfbfff640 1 -- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e04071540 msgr2=0x7f0e0415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0dfbfff640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e04071540 0x7f0e0415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0dfbfff640 1 -- 172.21.15.135:0/1189743573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e040cff30 msgr2=0x7f0e0415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0dfbfff640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e040cff30 0x7f0e0415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0dfbfff640 1 -- 172.21.15.135:0/1189743573 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e041f9370 con 0x7f0e040cc4a0 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e037fe640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e04071540 0x7f0e0415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0dfbfff640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cc4a0 0x7f0e041625f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e890 tx=0x7f0dec00ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e00ff9640 1 -- 172.21.15.135:0/1189743573 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec017070 con 0x7f0e040cc4a0 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e00ff9640 1 -- 172.21.15.135:0/1189743573 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec00c940 con 0x7f0e040cc4a0 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e04161b10 con 0x7f0e040cc4a0 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e00ff9640 1 -- 172.21.15.135:0/1189743573 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00cd10 con 0x7f0e040cc4a0 2024-05-10T10:33:19.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e041620d0 con 0x7f0e040cc4a0 2024-05-10T10:33:19.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e00ff9640 1 -- 172.21.15.135:0/1189743573 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0dec01a070 con 0x7f0e040cc4a0 2024-05-10T10:33:19.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.617+0000 7f0e00ff9640 1 -- 172.21.15.135:0/1189743573 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f0dec006520 con 0x7f0e040cc4a0 2024-05-10T10:33:19.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.618+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e040693f0 con 0x7f0e040cc4a0 2024-05-10T10:33:19.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.620+0000 7f0e00ff9640 1 -- 172.21.15.135:0/1189743573 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0dec006840 con 0x7f0e040cc4a0 2024-05-10T10:33:19.629 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:19.625+0000 7ff54df1c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T10:33:19.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.692+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "128a5eac-c0c4-4f21-8531-d626e5edf1bc", "id": 6} v 0) -- 0x7f0e040d2950 con 0x7f0e040cc4a0 2024-05-10T10:33:19.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.696+0000 7f0e00ff9640 1 -- 172.21.15.135:0/1189743573 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "128a5eac-c0c4-4f21-8531-d626e5edf1bc", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f0dec005990 con 0x7f0e040cc4a0 2024-05-10T10:33:19.698 INFO:teuthology.orchestra.run.smithi135.stdout:6 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cc4a0 msgr2=0x7f0e041625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cc4a0 0x7f0e041625f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e890 tx=0x7f0dec00ed50 comp rx=0 tx=0).stop 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 shutdown_connections 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e040cff30 0x7f0e0415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0e040cc4a0 0x7f0e041625f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 --2- 172.21.15.135:0/1189743573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e04071540 0x7f0e0415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 >> 172.21.15.135:0/1189743573 conn(0x7f0e040b3fb0 msgr2=0x7f0e040d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 shutdown_connections 2024-05-10T10:33:19.699 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.697+0000 7f0e09b9d640 1 -- 172.21.15.135:0/1189743573 wait complete. 2024-05-10T10:33:19.709 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new aa88b628-2dcd-4305-81c5-b932a094c0cf 7 2024-05-10T10:33:19.736 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:19.735+0000 7f086c5ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:33:19.747 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:19.746+0000 7ff54df1c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T10:33:19.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.809+0000 7f61cc815640 1 Processor -- start 2024-05-10T10:33:19.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.810+0000 7f61cc815640 1 -- start start 2024-05-10T10:33:19.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.810+0000 7f61cc815640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f61c815f4f0 0x7f61c80b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.810+0000 7f61cc815640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f61c80b3bb0 0x7f61c806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.810+0000 7f61cc815640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c80b2b50 con 0x7f61c80b2060 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61cc815640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f61c80b4710 con 0x7f61c80b3bb0 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61cc815640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f61c80d2150 con 0x7f61c815f4f0 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c659c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f61c815f4f0 0x7f61c80b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c659c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f61c815f4f0 0x7f61c80b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60612/0 (socket says 172.21.15.135:60612) 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c659c640 1 -- 172.21.15.135:0/1944715800 learned_addr learned my addr 172.21.15.135:0/1944715800 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c5d9b640 1 --2- 172.21.15.135:0/1944715800 >> v2:172.21.15.53:3300/0 conn(0x7f61c80b3bb0 0x7f61c806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c659c640 1 -- 172.21.15.135:0/1944715800 >> v2:172.21.15.53:3300/0 conn(0x7f61c80b3bb0 msgr2=0x7f61c806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c659c640 1 --2- 172.21.15.135:0/1944715800 >> v2:172.21.15.53:3300/0 conn(0x7f61c80b3bb0 0x7f61c806dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c659c640 1 -- 172.21.15.135:0/1944715800 >> v1:172.21.15.53:6789/0 conn(0x7f61c80b2060 legacy=0x7f61c80cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.811+0000 7f61c659c640 1 -- 172.21.15.135:0/1944715800 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c80b2df0 con 0x7f61c815f4f0 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.812+0000 7f61c659c640 1 --2- 172.21.15.135:0/1944715800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c815f4f0 0x7f61c80b41d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f61c80b5ab0 tx=0x7f61b802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27abf44475b4554b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.812+0000 7f61c559a640 1 -- 172.21.15.135:0/1944715800 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8037070 con 0x7f61c815f4f0 2024-05-10T10:33:19.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.812+0000 7f61c559a640 1 -- 172.21.15.135:0/1944715800 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8032440 con 0x7f61c815f4f0 2024-05-10T10:33:19.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.812+0000 7f61c559a640 1 -- 172.21.15.135:0/1944715800 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8032810 con 0x7f61c815f4f0 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.812+0000 7f61cc815640 1 -- 172.21.15.135:0/1944715800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c815f4f0 msgr2=0x7f61c80b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.812+0000 7f61cc815640 1 --2- 172.21.15.135:0/1944715800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c815f4f0 0x7f61c80b41d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f61c80b5ab0 tx=0x7f61b802a270 comp rx=0 tx=0).stop 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.813+0000 7f61cc815640 1 -- 172.21.15.135:0/1944715800 shutdown_connections 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.813+0000 7f61cc815640 1 --2- 172.21.15.135:0/1944715800 >> v2:172.21.15.53:3300/0 conn(0x7f61c80b3bb0 0x7f61c806dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.813+0000 7f61cc815640 1 --2- 172.21.15.135:0/1944715800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c815f4f0 0x7f61c80b41d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.813+0000 7f61cc815640 1 -- 172.21.15.135:0/1944715800 >> 172.21.15.135:0/1944715800 conn(0x7f61c809ba20 msgr2=0x7f61c809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.813+0000 7f61cc815640 1 -- 172.21.15.135:0/1944715800 shutdown_connections 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.813+0000 7f61cc815640 1 -- 172.21.15.135:0/1944715800 wait complete. 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.814+0000 7f61cc815640 1 Processor -- start 2024-05-10T10:33:19.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.814+0000 7f61cc815640 1 -- start start 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c80b2060 0x7f61c80c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c80b3bb0 0x7f61c80c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61c815f4f0 0x7f61c80c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80b3690 con 0x7f61c80b2060 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80b33b0 con 0x7f61c80b3bb0 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f61c815f8d0 con 0x7f61c815f4f0 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c80b3bb0 0x7f61c80c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61c815f4f0 0x7f61c80c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c80b3bb0 0x7f61c80c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60614/0 (socket says 172.21.15.135:60614) 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61c815f4f0 0x7f61c80c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:56254/0 (socket says 172.21.15.135:56254) 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 -- 172.21.15.135:0/2320825476 learned_addr learned my addr 172.21.15.135:0/2320825476 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-05-10T10:33:19.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c659c640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c80b2060 0x7f61c80c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 -- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61c815f4f0 msgr2=0x7f61c80c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61c815f4f0 0x7f61c80c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 -- 172.21.15.135:0/2320825476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c80b2060 msgr2=0x7f61c80c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c80b2060 0x7f61c80c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 -- 172.21.15.135:0/2320825476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c81f5bc0 con 0x7f61c80b3bb0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c6d9d640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61c815f4f0 0x7f61c80c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61c5d9b640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c80b3bb0 0x7f61c80c8160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00e840 tx=0x7f61bc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61b6ffd640 1 -- 172.21.15.135:0/2320825476 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc017070 con 0x7f61c80b3bb0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61b6ffd640 1 -- 172.21.15.135:0/2320825476 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61bc00c940 con 0x7f61c80b3bb0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c81f5ea0 con 0x7f61c80b3bb0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61b6ffd640 1 -- 172.21.15.135:0/2320825476 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00cd10 con 0x7f61c80b3bb0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.815+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c80c6ff0 con 0x7f61c80b3bb0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.816+0000 7f61b6ffd640 1 -- 172.21.15.135:0/2320825476 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f61bc01a070 con 0x7f61c80b3bb0 2024-05-10T10:33:19.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.816+0000 7f61b6ffd640 1 -- 172.21.15.135:0/2320825476 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f61bc01cab0 con 0x7f61c80b3bb0 2024-05-10T10:33:19.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.816+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c80d3a20 con 0x7f61c80b3bb0 2024-05-10T10:33:19.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.820+0000 7f61b6ffd640 1 -- 172.21.15.135:0/2320825476 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f61c80d3a20 con 0x7f61c80b3bb0 2024-05-10T10:33:19.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.892+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aa88b628-2dcd-4305-81c5-b932a094c0cf", "id": 7} v 0) -- 0x7f61c80693f0 con 0x7f61c80b3bb0 2024-05-10T10:33:19.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.897+0000 7f61b6ffd640 1 -- 172.21.15.135:0/2320825476 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aa88b628-2dcd-4305-81c5-b932a094c0cf", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f61bc01c450 con 0x7f61c80b3bb0 2024-05-10T10:33:19.898 INFO:teuthology.orchestra.run.smithi135.stdout:7 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c80b3bb0 msgr2=0x7f61c80c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c80b3bb0 0x7f61c80c8160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00e840 tx=0x7f61bc00ed00 comp rx=0 tx=0).stop 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 shutdown_connections 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61c815f4f0 0x7f61c80c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c80b3bb0 0x7f61c80c8160 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 --2- 172.21.15.135:0/2320825476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c80b2060 0x7f61c80c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 >> 172.21.15.135:0/2320825476 conn(0x7f61c809ba20 msgr2=0x7f61c80afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 shutdown_connections 2024-05-10T10:33:19.900 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:33:19.898+0000 7f61cc815640 1 -- 172.21.15.135:0/2320825476 wait complete. 2024-05-10T10:33:19.910 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-10T10:33:19.910 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-10T10:33:19.912 INFO:tasks.ceph.osd.0:Started 2024-05-10T10:33:19.912 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-10T10:33:19.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-10T10:33:19.916 INFO:tasks.ceph.osd.1:Started 2024-05-10T10:33:19.916 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-10T10:33:19.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-10T10:33:19.918 INFO:tasks.ceph.osd.2:Started 2024-05-10T10:33:19.918 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-10T10:33:19.918 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-10T10:33:19.921 INFO:tasks.ceph.osd.3:Started 2024-05-10T10:33:19.921 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-10T10:33:19.921 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-10T10:33:19.953 INFO:tasks.ceph.osd.4:Started 2024-05-10T10:33:19.953 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-10T10:33:19.953 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-10T10:33:19.955 INFO:tasks.ceph.osd.5:Started 2024-05-10T10:33:19.956 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-10T10:33:19.956 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-10T10:33:19.958 INFO:tasks.ceph.osd.6:Started 2024-05-10T10:33:19.958 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-10T10:33:19.958 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-10T10:33:19.960 INFO:tasks.ceph.osd.7:Started 2024-05-10T10:33:19.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:20.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.084+0000 7f4c26daf640 1 Processor -- start 2024-05-10T10:33:20.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.084+0000 7f4c26daf640 1 -- start start 2024-05-10T10:33:20.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.084+0000 7f4c26daf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c20155e10 0x7f4c2006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c26daf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20072ef0 0x7f4c200cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c26daf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2006f2a0 con 0x7f4c2015ffd0 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c26daf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4c201561f0 con 0x7f4c20155e10 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c26daf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2015f2a0 con 0x7f4c20072ef0 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c1ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4c2015ffd0 0x7f4c200727e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51682/0 (socket says 172.21.15.53:51682) 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c1ffff640 1 -- 172.21.15.53:0/255707822 learned_addr learned my addr 172.21.15.53:0/255707822 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c24b4b640 1 --2- 172.21.15.53:0/255707822 >> v2:172.21.15.135:3300/0 conn(0x7f4c20155e10 0x7f4c2006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c2534c640 1 --2- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20072ef0 0x7f4c200cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c1f7fe640 1 -- 172.21.15.53:0/255707822 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635274218 0 0) 0x7f4c2006f2a0 con 0x7f4c2015ffd0 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c1f7fe640 1 -- 172.21.15.53:0/255707822 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c10003520 con 0x7f4c2015ffd0 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c24b4b640 1 -- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20072ef0 msgr2=0x7f4c200cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c24b4b640 1 --2- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20072ef0 0x7f4c200cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c24b4b640 1 -- 172.21.15.53:0/255707822 >> v1:172.21.15.53:6789/0 conn(0x7f4c2015ffd0 legacy=0x7f4c200727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c24b4b640 1 -- 172.21.15.53:0/255707822 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2015f4f0 con 0x7f4c20155e10 2024-05-10T10:33:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.085+0000 7f4c2534c640 1 --2- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20072ef0 0x7f4c200cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.086+0000 7f4c24b4b640 1 --2- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20155e10 0x7f4c2006e800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c140097a0 tx=0x7f4c1402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29efa5bd41164df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.086+0000 7f4c1f7fe640 1 -- 172.21.15.53:0/255707822 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14037070 con 0x7f4c20155e10 2024-05-10T10:33:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.086+0000 7f4c1f7fe640 1 -- 172.21.15.53:0/255707822 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c14032440 con 0x7f4c20155e10 2024-05-10T10:33:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.086+0000 7f4c1f7fe640 1 -- 172.21.15.53:0/255707822 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14032810 con 0x7f4c20155e10 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 -- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20155e10 msgr2=0x7f4c2006e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 --2- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20155e10 0x7f4c2006e800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c140097a0 tx=0x7f4c1402a210 comp rx=0 tx=0).stop 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 -- 172.21.15.53:0/255707822 shutdown_connections 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 --2- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20072ef0 0x7f4c200cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 --2- 172.21.15.53:0/255707822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20155e10 0x7f4c2006e800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 -- 172.21.15.53:0/255707822 >> 172.21.15.53:0/255707822 conn(0x7f4c200d14f0 msgr2=0x7f4c200d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 -- 172.21.15.53:0/255707822 shutdown_connections 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.087+0000 7f4c26daf640 1 -- 172.21.15.53:0/255707822 wait complete. 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.088+0000 7f4c26daf640 1 Processor -- start 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.088+0000 7f4c26daf640 1 -- start start 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c26daf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20072ef0 0x7f4c20152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c26daf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20155e10 0x7f4c20153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c26daf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c2015ffd0 0x7f4c20152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c26daf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2015cba0 con 0x7f4c2015ffd0 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c26daf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2015fab0 con 0x7f4c20072ef0 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c26daf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2006ed40 con 0x7f4c20155e10 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20072ef0 0x7f4c20152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20155e10 0x7f4c20153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c2534c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c2015ffd0 0x7f4c20152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20072ef0 0x7f4c20152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:49920/0 (socket says 172.21.15.53:49920) 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 -- 172.21.15.53:0/3022005611 learned_addr learned my addr 172.21.15.53:0/3022005611 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 -- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20155e10 msgr2=0x7f4c20153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20155e10 0x7f4c20153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 -- 172.21.15.53:0/3022005611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c2015ffd0 msgr2=0x7f4c20152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c2015ffd0 0x7f4c20152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c24b4b640 1 -- 172.21.15.53:0/3022005611 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c201540e0 con 0x7f4c20072ef0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.089+0000 7f4c1ffff640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20155e10 0x7f4c20153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c24b4b640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20072ef0 0x7f4c20152e10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c14035040 tx=0x7f4c140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/3022005611 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14041030 con 0x7f4c20072ef0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/3022005611 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1403d420 con 0x7f4c20072ef0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c20154360 con 0x7f4c20072ef0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/3022005611 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14037040 con 0x7f4c20072ef0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c20154920 con 0x7f4c20072ef0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/3022005611 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4c14044070 con 0x7f4c20072ef0 2024-05-10T10:33:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.090+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/3022005611 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4c140465a0 con 0x7f4c20072ef0 2024-05-10T10:33:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.091+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c20155030 con 0x7f4c20072ef0 2024-05-10T10:33:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.093+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/3022005611 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4c201540e0 con 0x7f4c20072ef0 2024-05-10T10:33:20.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.155+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c200693f0 con 0x7f4c20072ef0 2024-05-10T10:33:20.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.156+0000 7f4c1d7fa640 1 -- 172.21.15.53:0/3022005611 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4c1403d830 con 0x7f4c20072ef0 2024-05-10T10:33:20.157 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:20.157 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":10,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:19.895364+0000","last_up_change":"0.000000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20072ef0 msgr2=0x7f4c20152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20072ef0 0x7f4c20152e10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c14035040 tx=0x7f4c140046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 shutdown_connections 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c2015ffd0 0x7f4c20152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c20155e10 0x7f4c20153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 --2- 172.21.15.53:0/3022005611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c20072ef0 0x7f4c20152e10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 >> 172.21.15.53:0/3022005611 conn(0x7f4c200d14f0 msgr2=0x7f4c2015a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:20.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 shutdown_connections 2024-05-10T10:33:20.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:20.157+0000 7f4c26daf640 1 -- 172.21.15.53:0/3022005611 wait complete. 2024-05-10T10:33:20.168 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-10T10:33:20.169 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-10T10:33:20.175 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:20.174+0000 7ff54df1c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T10:33:20.176 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:20.176+0000 7f086c5ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T10:33:20.315 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:20.313+0000 7ff54df1c040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T10:33:20.323 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:20.322+0000 7f086c5ee040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-10T10:33:20.523 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-05-10T10:33:20.522+0000 7f51f2fc4640 -1 Falling back to public interface 2024-05-10T10:33:20.523 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-05-10T10:33:20.523+0000 7fe5efd23640 -1 Falling back to public interface 2024-05-10T10:33:20.524 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-05-10T10:33:20.523+0000 7f51f1f73640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2024-05-10T10:33:20.527 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-05-10T10:33:20.527+0000 7f09b220f640 -1 Falling back to public interface 2024-05-10T10:33:20.544 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-05-10T10:33:20.544+0000 7ff01b230640 -1 Falling back to public interface 2024-05-10T10:33:20.558 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-05-10T10:33:20.556+0000 7f9adce32640 -1 Falling back to public interface 2024-05-10T10:33:20.562 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-05-10T10:33:20.560+0000 7f0f725a3640 -1 Falling back to public interface 2024-05-10T10:33:20.562 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-05-10T10:33:20.560+0000 7fb470c13640 -1 Falling back to public interface 2024-05-10T10:33:20.577 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:20.576+0000 7ff54df1c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T10:33:20.586 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-05-10T10:33:20.584+0000 7ff7d6c14640 -1 Falling back to public interface 2024-05-10T10:33:20.601 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:20.600+0000 7f086c5ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T10:33:20.630 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:20.629+0000 7ff54df1c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T10:33:20.657 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:20.655+0000 7f086c5ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T10:33:20.745 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:20.743+0000 7ff54df1c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T10:33:20.779 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:20.778+0000 7f086c5ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T10:33:20.974 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:21.040 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:21.038+0000 7ff54df1c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T10:33:21.069 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.068+0000 7f56a6779640 1 Processor -- start 2024-05-10T10:33:21.069 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.068+0000 7f56a6779640 1 -- start start 2024-05-10T10:33:21.070 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f56a6779640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56a015d1b0 0x7f56a00d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:21.070 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f56a6779640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a00ce960 con 0x7f56a015d590 2024-05-10T10:33:21.070 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f56a6779640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a00cec00 con 0x7f56a00ce550 2024-05-10T10:33:21.070 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f56a6779640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f56a00c6b40 con 0x7f56a015d1b0 2024-05-10T10:33:21.070 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f569ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56a015d1b0 0x7f56a00d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:21.071 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f569f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f56a015d590 0x7f56a00cde40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51688/0 (socket says 172.21.15.53:51688) 2024-05-10T10:33:21.071 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f569f7fe640 1 -- 172.21.15.53:0/2232681685 learned_addr learned my addr 172.21.15.53:0/2232681685 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:21.071 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f569ffff640 1 -- 172.21.15.53:0/2232681685 >> v1:172.21.15.135:6789/0 conn(0x7f56a00ce550 legacy=0x7f56a00c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:21.071 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f569ffff640 1 -- 172.21.15.53:0/2232681685 >> v1:172.21.15.53:6789/0 conn(0x7f56a015d590 legacy=0x7f56a00cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:21.071 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f569ffff640 1 -- 172.21.15.53:0/2232681685 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a00ceee0 con 0x7f56a015d1b0 2024-05-10T10:33:21.071 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.069+0000 7f569effd640 1 -- 172.21.15.53:0/2232681685 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072278503 0 0) 0x7f56a00ce960 con 0x7f56a015d590 2024-05-10T10:33:21.072 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.070+0000 7f569ffff640 1 --2- 172.21.15.53:0/2232681685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a015d1b0 0x7f56a00d2ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f56900097a0 tx=0x7f569002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3339b74f577d6353 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:21.072 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.070+0000 7f569effd640 1 -- 172.21.15.53:0/2232681685 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690037070 con 0x7f56a015d1b0 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.070+0000 7f569effd640 1 -- 172.21.15.53:0/2232681685 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690032640 con 0x7f56a015d1b0 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.070+0000 7f569effd640 1 -- 172.21.15.53:0/2232681685 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690032a10 con 0x7f56a015d1b0 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.071+0000 7f56a6779640 1 -- 172.21.15.53:0/2232681685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a015d1b0 msgr2=0x7f56a00d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.071+0000 7f56a6779640 1 --2- 172.21.15.53:0/2232681685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a015d1b0 0x7f56a00d2ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f56900097a0 tx=0x7f569002a2c0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.071+0000 7f56a6779640 1 -- 172.21.15.53:0/2232681685 shutdown_connections 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.071+0000 7f56a6779640 1 --2- 172.21.15.53:0/2232681685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a015d1b0 0x7f56a00d2ae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.071+0000 7f56a6779640 1 -- 172.21.15.53:0/2232681685 >> 172.21.15.53:0/2232681685 conn(0x7f56a009ba60 msgr2=0x7f56a009be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.071+0000 7f56a6779640 1 -- 172.21.15.53:0/2232681685 shutdown_connections 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.071+0000 7f56a6779640 1 -- 172.21.15.53:0/2232681685 wait complete. 2024-05-10T10:33:21.073 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.072+0000 7f56a6779640 1 Processor -- start 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a6779640 1 -- start start 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a6779640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a00ce550 0x7f56a0160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a6779640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a015d1b0 0x7f56a0154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a6779640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56a015d590 0x7f56a0155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a6779640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56a00d3020 con 0x7f56a00ce550 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a6779640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56a00cf4a0 con 0x7f56a015d1b0 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a6779640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f56a00d31a0 con 0x7f56a015d590 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f569ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a00ce550 0x7f56a0160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f569ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a00ce550 0x7f56a0160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41434/0 (socket says 172.21.15.53:41434) 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f569ffff640 1 -- 172.21.15.53:0/3525558574 learned_addr learned my addr 172.21.15.53:0/3525558574 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f569f7fe640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a015d1b0 0x7f56a0154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:21.076 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.073+0000 7f56a4d16640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56a015d590 0x7f56a0155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569ffff640 1 -- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56a015d590 msgr2=0x7f56a0155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569ffff640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56a015d590 0x7f56a0155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569ffff640 1 -- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a015d1b0 msgr2=0x7f56a0154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569ffff640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a015d1b0 0x7f56a0154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569ffff640 1 -- 172.21.15.53:0/3525558574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a015f4b0 con 0x7f56a00ce550 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569ffff640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a00ce550 0x7f56a0160550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56900098c0 tx=0x7f56900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569cff9640 1 -- 172.21.15.53:0/3525558574 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690041020 con 0x7f56a00ce550 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569cff9640 1 -- 172.21.15.53:0/3525558574 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690030970 con 0x7f56a00ce550 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a015f790 con 0x7f56a00ce550 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569cff9640 1 -- 172.21.15.53:0/3525558574 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690037040 con 0x7f56a00ce550 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a015fd50 con 0x7f56a00ce550 2024-05-10T10:33:21.077 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569cff9640 1 -- 172.21.15.53:0/3525558574 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5690044020 con 0x7f56a00ce550 2024-05-10T10:33:21.078 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.074+0000 7f569cff9640 1 -- 172.21.15.53:0/3525558574 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f56900325f0 con 0x7f56a00ce550 2024-05-10T10:33:21.078 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.075+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a015aaa0 con 0x7f56a00ce550 2024-05-10T10:33:21.078 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.077+0000 7f569cff9640 1 -- 172.21.15.53:0/3525558574 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f569004b030 con 0x7f56a00ce550 2024-05-10T10:33:21.088 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:21.087+0000 7ff54df1c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T10:33:21.089 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:21.089+0000 7f086c5ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T10:33:21.175 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:21.174+0000 7f086c5ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T10:33:21.207 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.207+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56a015a4e0 con 0x7f56a00ce550 2024-05-10T10:33:21.208 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.207+0000 7f569cff9640 1 -- 172.21.15.53:0/3525558574 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f569003d940 con 0x7f56a00ce550 2024-05-10T10:33:21.208 INFO:teuthology.misc.health.smithi053.stdout: 2024-05-10T10:33:21.208 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":10,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:19.895364+0000","last_up_change":"0.000000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:21.209 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.208+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a00ce550 msgr2=0x7f56a0160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:21.209 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.208+0000 7f56a6779640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a00ce550 0x7f56a0160550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56900098c0 tx=0x7f56900046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.209 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.209+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 shutdown_connections 2024-05-10T10:33:21.209 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.209+0000 7f56a6779640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56a015d590 0x7f56a0155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.209 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.209+0000 7f56a6779640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56a015d1b0 0x7f56a0154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.210 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.209+0000 7f56a6779640 1 --2- 172.21.15.53:0/3525558574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56a00ce550 0x7f56a0160550 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:21.210 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.209+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 >> 172.21.15.53:0/3525558574 conn(0x7f56a009ba60 msgr2=0x7f56a009be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:21.210 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.209+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 shutdown_connections 2024-05-10T10:33:21.210 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:21.209+0000 7f56a6779640 1 -- 172.21.15.53:0/3525558574 wait complete. 2024-05-10T10:33:21.222 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-10T10:33:21.247 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:21.246+0000 7ff54df1c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T10:33:21.358 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:21.357+0000 7f086c5ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T10:33:21.384 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:21.383+0000 7ff54df1c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T10:33:21.435 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:21.434+0000 7ff54df1c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T10:33:21.503 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:21.503+0000 7f086c5ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T10:33:21.504 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:21.503+0000 7ff54df1c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T10:33:21.549 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:21.547+0000 7ff54df1c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T10:33:21.555 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:21.555+0000 7f086c5ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T10:33:21.630 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:21.629+0000 7f086c5ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T10:33:21.678 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:21.677+0000 7f086c5ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T10:33:22.030 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:22.028+0000 7ff54df1c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T10:33:22.145 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:22.143+0000 7ff54df1c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T10:33:22.184 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:22.183+0000 7f086c5ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T10:33:22.306 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-05-10T10:33:22.306+0000 7f086c5ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T10:33:23.161 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:23.159+0000 7ff4eda0f640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-10T10:33:23.161 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-05-10T10:33:23.159+0000 7ff4eda0f640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-10T10:33:23.974 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-05-10T10:33:23.974+0000 7f51f2fc4640 -1 osd.2 0 log_to_monitors true 2024-05-10T10:33:24.003 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-05-10T10:33:24.003+0000 7fe5efd23640 -1 osd.0 0 log_to_monitors true 2024-05-10T10:33:24.034 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-05-10T10:33:24.034+0000 7f09b220f640 -1 osd.3 0 log_to_monitors true 2024-05-10T10:33:24.061 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-05-10T10:33:24.060+0000 7f0f725a3640 -1 osd.6 0 log_to_monitors true 2024-05-10T10:33:24.064 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-05-10T10:33:24.063+0000 7fb470c13640 -1 osd.7 0 log_to_monitors true 2024-05-10T10:33:24.071 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-05-10T10:33:24.069+0000 7f9adce32640 -1 osd.4 0 log_to_monitors true 2024-05-10T10:33:24.077 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-05-10T10:33:24.077+0000 7ff01b230640 -1 osd.1 0 log_to_monitors true 2024-05-10T10:33:24.091 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-05-10T10:33:24.089+0000 7ff7d6c14640 -1 osd.5 0 log_to_monitors true 2024-05-10T10:33:26.540 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-05-10T10:33:26.539+0000 7f51eef5a640 -1 osd.2 0 waiting for initial osdmap 2024-05-10T10:33:26.550 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-05-10T10:33:26.550+0000 7f51e6d7a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:26.590 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-05-10T10:33:26.589+0000 7ff016778640 -1 osd.1 0 waiting for initial osdmap 2024-05-10T10:33:26.602 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-05-10T10:33:26.602+0000 7fe5ebcb9640 -1 osd.0 0 waiting for initial osdmap 2024-05-10T10:33:26.604 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-05-10T10:33:26.604+0000 7ff00e598640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:26.620 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-05-10T10:33:26.620+0000 7f09adb98640 -1 osd.3 0 waiting for initial osdmap 2024-05-10T10:33:26.625 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-05-10T10:33:26.625+0000 7fe5e3ad9640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:26.630 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-05-10T10:33:26.630+0000 7f09a59b8640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:27.280 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-05-10T10:33:27.279+0000 7f9ad8dc8640 -1 osd.4 0 waiting for initial osdmap 2024-05-10T10:33:27.299 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-05-10T10:33:27.297+0000 7fb46c4f3640 -1 osd.7 0 waiting for initial osdmap 2024-05-10T10:33:27.299 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-05-10T10:33:27.298+0000 7f9ad0be8640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:27.315 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-05-10T10:33:27.314+0000 7fb464313640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:27.323 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-05-10T10:33:27.322+0000 7ff7d2659640 -1 osd.5 0 waiting for initial osdmap 2024-05-10T10:33:27.328 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-05-10T10:33:27.326+0000 7f0f6e539640 -1 osd.6 0 waiting for initial osdmap 2024-05-10T10:33:27.333 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-05-10T10:33:27.332+0000 7ff7ca479640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:27.342 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-05-10T10:33:27.341+0000 7f0f66359640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T10:33:28.029 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:28.141 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.140+0000 7f4416c37640 1 Processor -- start 2024-05-10T10:33:28.141 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.140+0000 7f4416c37640 1 -- start start 2024-05-10T10:33:28.142 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f4416c37640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f441015f580 0x7f44100d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.142 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f4416c37640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f44100d3620 0x7f44100d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.142 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f4416c37640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44100d2480 con 0x7f441015f960 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f4416c37640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f44100d04b0 con 0x7f441015f580 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f4416c37640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f44100d3f70 con 0x7f44100d3620 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44149d3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f441015f580 0x7f44100d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44149d3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f441015f580 0x7f44100d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34692/0 (socket says 172.21.15.53:34692) 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44149d3640 1 -- 172.21.15.53:0/1197020200 learned_addr learned my addr 172.21.15.53:0/1197020200 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44151d4640 1 --2- 172.21.15.53:0/1197020200 >> v2:172.21.15.135:3300/0 conn(0x7f44100d3620 0x7f44100d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44149d3640 1 -- 172.21.15.53:0/1197020200 >> v2:172.21.15.135:3300/0 conn(0x7f44100d3620 msgr2=0x7f44100d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44149d3640 1 --2- 172.21.15.53:0/1197020200 >> v2:172.21.15.135:3300/0 conn(0x7f44100d3620 0x7f44100d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44149d3640 1 -- 172.21.15.53:0/1197020200 >> v1:172.21.15.135:6789/0 conn(0x7f441015f960 legacy=0x7f44100d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44149d3640 1 -- 172.21.15.53:0/1197020200 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44100d21a0 con 0x7f441015f580 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.141+0000 7f44151d4640 1 --2- 172.21.15.53:0/1197020200 >> v2:172.21.15.135:3300/0 conn(0x7f44100d3620 0x7f44100d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:28.143 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.142+0000 7f44149d3640 1 --2- 172.21.15.53:0/1197020200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f44100d0940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f441008f220 tx=0x7f440002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6be6c6da4bfb0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.142+0000 7f440f7fe640 1 -- 172.21.15.53:0/1197020200 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400037070 con 0x7f441015f580 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.142+0000 7f440f7fe640 1 -- 172.21.15.53:0/1197020200 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4400032440 con 0x7f441015f580 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.142+0000 7f440f7fe640 1 -- 172.21.15.53:0/1197020200 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400032810 con 0x7f441015f580 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 -- 172.21.15.53:0/1197020200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 msgr2=0x7f44100d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 --2- 172.21.15.53:0/1197020200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f44100d0940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f441008f220 tx=0x7f440002a270 comp rx=0 tx=0).stop 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 -- 172.21.15.53:0/1197020200 shutdown_connections 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 --2- 172.21.15.53:0/1197020200 >> v2:172.21.15.135:3300/0 conn(0x7f44100d3620 0x7f44100d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 --2- 172.21.15.53:0/1197020200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f44100d0940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 -- 172.21.15.53:0/1197020200 >> 172.21.15.53:0/1197020200 conn(0x7f441009e390 msgr2=0x7f44100d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 -- 172.21.15.53:0/1197020200 shutdown_connections 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.143+0000 7f4416c37640 1 -- 172.21.15.53:0/1197020200 wait complete. 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.144+0000 7f4416c37640 1 Processor -- start 2024-05-10T10:33:28.145 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.144+0000 7f4416c37640 1 -- start start 2024-05-10T10:33:28.147 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f4416c37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44100d3620 0x7f441015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f4416c37640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f441009fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f4416c37640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f441015f960 0x7f44100a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f4416c37640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44100b63b0 con 0x7f441015f580 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f4416c37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44100d0140 con 0x7f44100d3620 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f4416c37640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f44100ccae0 con 0x7f441015f960 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f440ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f441009fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f440ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f441009fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34698/0 (socket says 172.21.15.53:34698) 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f440ffff640 1 -- 172.21.15.53:0/727803450 learned_addr learned my addr 172.21.15.53:0/727803450 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.145+0000 7f44149d3640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44100d3620 0x7f441015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f44151d4640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f441015f960 0x7f44100a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440ffff640 1 -- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f441015f960 msgr2=0x7f44100a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.148 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440ffff640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f441015f960 0x7f44100a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440ffff640 1 -- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44100d3620 msgr2=0x7f441015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440ffff640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44100d3620 0x7f441015a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440ffff640 1 -- 172.21.15.53:0/727803450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44100cc1b0 con 0x7f441015f580 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f44151d4640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f441015f960 0x7f44100a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440ffff640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f441009fac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f440400c400 tx=0x7f440400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440d7fa640 1 -- 172.21.15.53:0/727803450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404017070 con 0x7f441015f580 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440d7fa640 1 -- 172.21.15.53:0/727803450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44040052d0 con 0x7f441015f580 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44100cc490 con 0x7f441015f580 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f440d7fa640 1 -- 172.21.15.53:0/727803450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404003830 con 0x7f441015f580 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.146+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44100cefd0 con 0x7f441015f580 2024-05-10T10:33:28.149 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.147+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4410069d20 con 0x7f441015f580 2024-05-10T10:33:28.150 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.148+0000 7f440d7fa640 1 -- 172.21.15.53:0/727803450 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f440401a070 con 0x7f441015f580 2024-05-10T10:33:28.150 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.148+0000 7f440d7fa640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43fc076240 0x7f43fc078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.150 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.148+0000 7f440d7fa640 1 -- 172.21.15.53:0/727803450 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f44040963a0 con 0x7f441015f580 2024-05-10T10:33:28.150 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.149+0000 7f44149d3640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43fc076240 0x7f43fc078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.150 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.149+0000 7f44149d3640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43fc076240 0x7f43fc078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4400034000 tx=0x7f440002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:28.153 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.153+0000 7f440d7fa640 1 -- 172.21.15.53:0/727803450 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f440405fbc0 con 0x7f441015f580 2024-05-10T10:33:28.346 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.346+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44100d3400 con 0x7f441015f580 2024-05-10T10:33:28.347 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.346+0000 7f440d7fa640 1 -- 172.21.15.53:0/727803450 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10909 (secure 0 0 0) 0x7f4404068020 con 0x7f441015f580 2024-05-10T10:33:28.347 INFO:teuthology.misc.health.smithi053.stdout: 2024-05-10T10:33:28.347 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":13,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:27.436094+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43fc076240 msgr2=0x7f43fc078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43fc076240 0x7f43fc078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4400034000 tx=0x7f440002e000 comp rx=0 tx=0).stop 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 msgr2=0x7f441009fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f441009fac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f440400c400 tx=0x7f440400c8c0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 shutdown_connections 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43fc076240 0x7f43fc078700 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f441015f960 0x7f44100a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.349 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f441015f580 0x7f441009fac0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.350 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.348+0000 7f4416c37640 1 --2- 172.21.15.53:0/727803450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44100d3620 0x7f441015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.350 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.349+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 >> 172.21.15.53:0/727803450 conn(0x7f441009e390 msgr2=0x7f441015fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:28.350 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.349+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 shutdown_connections 2024-05-10T10:33:28.350 INFO:teuthology.misc.health.smithi053.stderr:2024-05-10T10:33:28.349+0000 7f4416c37640 1 -- 172.21.15.53:0/727803450 wait complete. 2024-05-10T10:33:28.359 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-10T10:33:28.359 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-10T10:33:28.359 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:33:28.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:33:28.362 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:33:28.362 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:33:28.362 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:33:28.364 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:33:28.364 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:33:28.364 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:33:28.367 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:33:28.367 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:33:28.367 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:33:28.369 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:33:28.369 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-10T10:33:28.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:28.439 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:33:28.443 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:33:28.454 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:33:28.454 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:33:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.481+0000 7f236442a640 1 Processor -- start 2024-05-10T10:33:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.481+0000 7f236442a640 1 -- start start 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f236442a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f235c0b5520 0x7f235c0d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f236442a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f235c0d41f0 con 0x7f235c0b5900 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f236442a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f235c0d3d70 con 0x7f235c0b9180 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f236442a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f235c0d5c60 con 0x7f235c0b5520 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23619c5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f235c0b5900 0x7f235c0d2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38732/0 (socket says 172.21.15.53:38732) 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23619c5640 1 -- 172.21.15.53:0/3586335892 learned_addr learned my addr 172.21.15.53:0/3586335892 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23621c6640 1 --2- 172.21.15.53:0/3586335892 >> v2:172.21.15.135:3300/0 conn(0x7f235c0b5520 0x7f235c0d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23611c4640 1 -- 172.21.15.53:0/3586335892 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466410551 0 0) 0x7f235c0d41f0 con 0x7f235c0b5900 2024-05-10T10:33:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23611c4640 1 -- 172.21.15.53:0/3586335892 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2348003520 con 0x7f235c0b5900 2024-05-10T10:33:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23611c4640 1 -- 172.21.15.53:0/3586335892 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009408391 0 0) 0x7f235c0d3d70 con 0x7f235c0b9180 2024-05-10T10:33:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23611c4640 1 -- 172.21.15.53:0/3586335892 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f235c0d41f0 con 0x7f235c0b9180 2024-05-10T10:33:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23621c6640 1 -- 172.21.15.53:0/3586335892 >> v1:172.21.15.135:6789/0 conn(0x7f235c0b9180 legacy=0x7f235c0d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23621c6640 1 -- 172.21.15.53:0/3586335892 >> v1:172.21.15.53:6789/0 conn(0x7f235c0b5900 legacy=0x7f235c0d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.482+0000 7f23621c6640 1 -- 172.21.15.53:0/3586335892 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c0d5eb0 con 0x7f235c0b5520 2024-05-10T10:33:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.483+0000 7f23621c6640 1 --2- 172.21.15.53:0/3586335892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 0x7f235c0d18a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23580097a0 tx=0x7f235802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c46332938af0e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.483+0000 7f23611c4640 1 -- 172.21.15.53:0/3586335892 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358037070 con 0x7f235c0b5520 2024-05-10T10:33:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.483+0000 7f23611c4640 1 -- 172.21.15.53:0/3586335892 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2358032440 con 0x7f235c0b5520 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.483+0000 7f23611c4640 1 -- 172.21.15.53:0/3586335892 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23580327f0 con 0x7f235c0b5520 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.484+0000 7f236442a640 1 -- 172.21.15.53:0/3586335892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 msgr2=0x7f235c0d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.484+0000 7f236442a640 1 --2- 172.21.15.53:0/3586335892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 0x7f235c0d18a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23580097a0 tx=0x7f235802a230 comp rx=0 tx=0).stop 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.484+0000 7f236442a640 1 -- 172.21.15.53:0/3586335892 shutdown_connections 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.484+0000 7f236442a640 1 --2- 172.21.15.53:0/3586335892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 0x7f235c0d18a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.484+0000 7f236442a640 1 -- 172.21.15.53:0/3586335892 >> 172.21.15.53:0/3586335892 conn(0x7f235c0d5170 msgr2=0x7f235c0d5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.484+0000 7f236442a640 1 -- 172.21.15.53:0/3586335892 shutdown_connections 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.484+0000 7f236442a640 1 -- 172.21.15.53:0/3586335892 wait complete. 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.485+0000 7f236442a640 1 Processor -- start 2024-05-10T10:33:28.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.485+0000 7f236442a640 1 -- start start 2024-05-10T10:33:28.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f236442a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 0x7f235c1ea920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f236442a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f235c0b5900 0x7f235c1e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f236442a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f235c0b9180 0x7f235c0d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f236442a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f235c0b62e0 con 0x7f235c0b5900 2024-05-10T10:33:28.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f236442a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f235c0d0d10 con 0x7f235c0b5520 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f236442a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f235c0b6460 con 0x7f235c0b9180 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f235c0b5900 0x7f235c1e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f235c0b5900 0x7f235c1e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34748/0 (socket says 172.21.15.53:34748) 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 -- 172.21.15.53:0/3251811913 learned_addr learned my addr 172.21.15.53:0/3251811913 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23621c6640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 0x7f235c1ea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23629c7640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f235c0b9180 0x7f235c0d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 -- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f235c0b9180 msgr2=0x7f235c0d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f235c0b9180 0x7f235c0d33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 -- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 msgr2=0x7f235c1ea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 0x7f235c1ea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 -- 172.21.15.53:0/3251811913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c1e9380 con 0x7f235c0b5900 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.486+0000 7f23619c5640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f235c0b5900 0x7f235c1e8e40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c500 tx=0x7f234c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.487+0000 7f2352ffd640 1 -- 172.21.15.53:0/3251811913 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c017070 con 0x7f235c0b5900 2024-05-10T10:33:28.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.487+0000 7f2352ffd640 1 -- 172.21.15.53:0/3251811913 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c005ce0 con 0x7f235c0b5900 2024-05-10T10:33:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.487+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c1e9620 con 0x7f235c0b5900 2024-05-10T10:33:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.487+0000 7f2352ffd640 1 -- 172.21.15.53:0/3251811913 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c0072f0 con 0x7f235c0b5900 2024-05-10T10:33:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.487+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c1e9be0 con 0x7f235c0b5900 2024-05-10T10:33:28.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.488+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c0693f0 con 0x7f235c0b5900 2024-05-10T10:33:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.488+0000 7f2352ffd640 1 -- 172.21.15.53:0/3251811913 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f234c01a070 con 0x7f235c0b5900 2024-05-10T10:33:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.489+0000 7f2352ffd640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2340076240 0x7f2340078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.489+0000 7f2352ffd640 1 -- 172.21.15.53:0/3251811913 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4846+0+0 (secure 0 0 0) 0x7f234c096440 con 0x7f235c0b5900 2024-05-10T10:33:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.489+0000 7f23621c6640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2340076240 0x7f2340078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.489+0000 7f23621c6640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2340076240 0x7f2340078700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2358034000 tx=0x7f235802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.493+0000 7f2352ffd640 1 -- 172.21.15.53:0/3251811913 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f234c05fc20 con 0x7f235c0b5900 2024-05-10T10:33:28.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.722+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f235c162420 con 0x7f235c0b5900 2024-05-10T10:33:28.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.722+0000 7f2352ffd640 1 -- 172.21.15.53:0/3251811913 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f234c063f20 con 0x7f235c0b5900 2024-05-10T10:33:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:28.723 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-10T10:33:28.723 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 2 2024-05-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2340076240 msgr2=0x7f2340078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2340076240 0x7f2340078700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2358034000 tx=0x7f235802e000 comp rx=0 tx=0).stop 2024-05-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f235c0b5900 msgr2=0x7f235c1e8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f235c0b5900 0x7f235c1e8e40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c500 tx=0x7f234c00c9c0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 shutdown_connections 2024-05-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2340076240 0x7f2340078700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f235c0b9180 0x7f235c0d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f235c0b5900 0x7f235c1e8e40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 --2- 172.21.15.53:0/3251811913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f235c0b5520 0x7f235c1ea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 >> 172.21.15.53:0/3251811913 conn(0x7f235c0d5170 msgr2=0x7f235c0ba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 shutdown_connections 2024-05-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:28.724+0000 7f236442a640 1 -- 172.21.15.53:0/3251811913 wait complete. 2024-05-10T10:33:30.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:30.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.854+0000 7f3b0d146640 1 Processor -- start 2024-05-10T10:33:30.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.854+0000 7f3b0d146640 1 -- start start 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.854+0000 7f3b0d146640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b0806c9d0 0x7f3b080aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.854+0000 7f3b0d146640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b080af4b0 0x7f3b080b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0d146640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b08163bd0 con 0x7f3b081626d0 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0d146640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3b08162ab0 con 0x7f3b0806c9d0 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0d146640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b08162e10 con 0x7f3b080af4b0 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b0806c9d0 0x7f3b080aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b0806c9d0 0x7f3b080aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34762/0 (socket says 172.21.15.53:34762) 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0659c640 1 -- 172.21.15.53:0/1841393055 learned_addr learned my addr 172.21.15.53:0/1841393055 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0759e640 1 --2- 172.21.15.53:0/1841393055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b080af4b0 0x7f3b080b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0659c640 1 -- 172.21.15.53:0/1841393055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b080af4b0 msgr2=0x7f3b080b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0659c640 1 --2- 172.21.15.53:0/1841393055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b080af4b0 0x7f3b080b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0659c640 1 -- 172.21.15.53:0/1841393055 >> v1:172.21.15.135:6789/0 conn(0x7f3b081626d0 legacy=0x7f3b0806c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:30.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.855+0000 7f3b0659c640 1 -- 172.21.15.53:0/1841393055 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b081630f0 con 0x7f3b0806c9d0 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b0659c640 1 --2- 172.21.15.53:0/1841393055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b080aef70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009870 tx=0x7f3afc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe445cc0fb3d69bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b05d9b640 1 -- 172.21.15.53:0/1841393055 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc037070 con 0x7f3b0806c9d0 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b05d9b640 1 -- 172.21.15.53:0/1841393055 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3afc032440 con 0x7f3b0806c9d0 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b05d9b640 1 -- 172.21.15.53:0/1841393055 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc032810 con 0x7f3b0806c9d0 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b0d146640 1 -- 172.21.15.53:0/1841393055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 msgr2=0x7f3b080aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b0d146640 1 --2- 172.21.15.53:0/1841393055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b080aef70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009870 tx=0x7f3afc02a180 comp rx=0 tx=0).stop 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b0d146640 1 -- 172.21.15.53:0/1841393055 shutdown_connections 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b0d146640 1 --2- 172.21.15.53:0/1841393055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b080af4b0 0x7f3b080b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:30.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b0d146640 1 --2- 172.21.15.53:0/1841393055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b080aef70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:30.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.856+0000 7f3b0d146640 1 -- 172.21.15.53:0/1841393055 >> 172.21.15.53:0/1841393055 conn(0x7f3b080d1d70 msgr2=0x7f3b080d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:30.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.857+0000 7f3b0d146640 1 -- 172.21.15.53:0/1841393055 shutdown_connections 2024-05-10T10:33:30.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.857+0000 7f3b0d146640 1 -- 172.21.15.53:0/1841393055 wait complete. 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.858+0000 7f3b0d146640 1 Processor -- start 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.858+0000 7f3b0d146640 1 -- start start 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0d146640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b08149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0d146640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080af4b0 0x7f3b0815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0d146640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b081626d0 0x7f3b081eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0d146640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080af8f0 con 0x7f3b0806c9d0 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0d146640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b081636b0 con 0x7f3b080af4b0 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0d146640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b080afc10 con 0x7f3b081626d0 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b08149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b08149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34778/0 (socket says 172.21.15.53:34778) 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 -- 172.21.15.53:0/2838402773 learned_addr learned my addr 172.21.15.53:0/2838402773 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0659c640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080af4b0 0x7f3b0815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b0759e640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b081626d0 0x7f3b081eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 -- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b081626d0 msgr2=0x7f3b081eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b081626d0 0x7f3b081eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 -- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080af4b0 msgr2=0x7f3b0815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080af4b0 0x7f3b0815b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.859+0000 7f3b06d9d640 1 -- 172.21.15.53:0/2838402773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0815ab50 con 0x7f3b0806c9d0 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.860+0000 7f3b06d9d640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b08149900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3af800b420 tx=0x7f3af800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.860+0000 7f3aef7fe640 1 -- 172.21.15.53:0/2838402773 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8011830 con 0x7f3b0806c9d0 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.860+0000 7f3aef7fe640 1 -- 172.21.15.53:0/2838402773 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af8016070 con 0x7f3b0806c9d0 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.860+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b08149e40 con 0x7f3b0806c9d0 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.860+0000 7f3aef7fe640 1 -- 172.21.15.53:0/2838402773 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8011c10 con 0x7f3b0806c9d0 2024-05-10T10:33:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.860+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0815adf0 con 0x7f3b0806c9d0 2024-05-10T10:33:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.861+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b080693f0 con 0x7f3b0806c9d0 2024-05-10T10:33:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.861+0000 7f3aef7fe640 1 -- 172.21.15.53:0/2838402773 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3af801b460 con 0x7f3b0806c9d0 2024-05-10T10:33:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.862+0000 7f3aef7fe640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af40762d0 0x7f3af4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.862+0000 7f3aef7fe640 1 -- 172.21.15.53:0/2838402773 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3af8096bf0 con 0x7f3b0806c9d0 2024-05-10T10:33:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.862+0000 7f3b0659c640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af40762d0 0x7f3af4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:30.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.862+0000 7f3b0659c640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af40762d0 0x7f3af4078790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3afc034000 tx=0x7f3afc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:30.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:30.867+0000 7f3aef7fe640 1 -- 172.21.15.53:0/2838402773 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3af8020020 con 0x7f3b0806c9d0 2024-05-10T10:33:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.098+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b0815be10 con 0x7f3b0806c9d0 2024-05-10T10:33:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.098+0000 7f3aef7fe640 1 -- 172.21.15.53:0/2838402773 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7f3af8060250 con 0x7f3b0806c9d0 2024-05-10T10:33:31.103 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:31.104 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-10T10:33:31.104 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 3 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af40762d0 msgr2=0x7f3af4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af40762d0 0x7f3af4078790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3afc034000 tx=0x7f3afc02e000 comp rx=0 tx=0).stop 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 msgr2=0x7f3b08149900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b08149900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3af800b420 tx=0x7f3af800b8e0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 shutdown_connections 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af40762d0 0x7f3af4078790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b081626d0 0x7f3b081eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080af4b0 0x7f3b0815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 --2- 172.21.15.53:0/2838402773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0806c9d0 0x7f3b08149900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 >> 172.21.15.53:0/2838402773 conn(0x7f3b080d1d70 msgr2=0x7f3b0806dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 shutdown_connections 2024-05-10T10:33:31.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.100+0000 7f3b0d146640 1 -- 172.21.15.53:0/2838402773 wait complete. 2024-05-10T10:33:31.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:33:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.216+0000 7f26fc946640 1 Processor -- start 2024-05-10T10:33:31.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.216+0000 7f26fc946640 1 -- start start 2024-05-10T10:33:31.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26fc946640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f26f815f3b0 0x7f26f80d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26fc946640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f815f790 0x7f26f80d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26fc946640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26f80d5d30 0x7f26f80c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26fc946640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80d30b0 con 0x7f26f80d5d30 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26fc946640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80d3390 con 0x7f26f815f3b0 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26fc946640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f26f80c8c80 con 0x7f26f815f790 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26f80d5d30 0x7f26f80c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26f80d5d30 0x7f26f80c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34786/0 (socket says 172.21.15.53:34786) 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 -- 172.21.15.53:0/2931127033 learned_addr learned my addr 172.21.15.53:0/2931127033 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f659c640 1 --2- 172.21.15.53:0/2931127033 >> v2:172.21.15.135:3300/0 conn(0x7f26f815f3b0 0x7f26f80d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26effff640 1 --2- 172.21.15.53:0/2931127033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f815f790 0x7f26f80d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 -- 172.21.15.53:0/2931127033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f815f790 msgr2=0x7f26f80d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 --2- 172.21.15.53:0/2931127033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f815f790 0x7f26f80d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 -- 172.21.15.53:0/2931127033 >> v2:172.21.15.135:3300/0 conn(0x7f26f815f3b0 msgr2=0x7f26f80d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 --2- 172.21.15.53:0/2931127033 >> v2:172.21.15.135:3300/0 conn(0x7f26f815f3b0 0x7f26f80d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26f6d9d640 1 -- 172.21.15.53:0/2931127033 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f80c8f60 con 0x7f26f80d5d30 2024-05-10T10:33:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.217+0000 7f26effff640 1 --2- 172.21.15.53:0/2931127033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f815f790 0x7f26f80d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.218+0000 7f26f6d9d640 1 --2- 172.21.15.53:0/2931127033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f80d5d30 0x7f26f80c9760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f26f808e820 tx=0x7f26f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cda958f31b90fd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.218+0000 7f26f5d9b640 1 -- 172.21.15.53:0/2931127033 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0036070 con 0x7f26f80d5d30 2024-05-10T10:33:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.218+0000 7f26f5d9b640 1 -- 172.21.15.53:0/2931127033 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f0031600 con 0x7f26f80d5d30 2024-05-10T10:33:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.218+0000 7f26f5d9b640 1 -- 172.21.15.53:0/2931127033 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00319d0 con 0x7f26f80d5d30 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 -- 172.21.15.53:0/2931127033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f80d5d30 msgr2=0x7f26f80c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 --2- 172.21.15.53:0/2931127033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f80d5d30 0x7f26f80c9760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f26f808e820 tx=0x7f26f002a6a0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 -- 172.21.15.53:0/2931127033 shutdown_connections 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 --2- 172.21.15.53:0/2931127033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f80d5d30 0x7f26f80c9760 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 --2- 172.21.15.53:0/2931127033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f815f790 0x7f26f80d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 --2- 172.21.15.53:0/2931127033 >> v2:172.21.15.135:3300/0 conn(0x7f26f815f3b0 0x7f26f80d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 -- 172.21.15.53:0/2931127033 >> 172.21.15.53:0/2931127033 conn(0x7f26f80d5130 msgr2=0x7f26f80d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 -- 172.21.15.53:0/2931127033 shutdown_connections 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.219+0000 7f26fc946640 1 -- 172.21.15.53:0/2931127033 wait complete. 2024-05-10T10:33:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.220+0000 7f26fc946640 1 Processor -- start 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.220+0000 7f26fc946640 1 -- start start 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26fc946640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f80d5d30 0x7f26f80b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26fc946640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f815f3b0 0x7f26f80ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26fc946640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f80ce910 0x7f26f814feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26fc946640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80c9e20 con 0x7f26f815f3b0 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26fc946640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80c9ca0 con 0x7f26f80ce910 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26fc946640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f26f80c9fa0 con 0x7f26f80d5d30 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f815f3b0 0x7f26f80ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f815f3b0 0x7f26f80ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34800/0 (socket says 172.21.15.53:34800) 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 -- 172.21.15.53:0/595360751 learned_addr learned my addr 172.21.15.53:0/595360751 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26f6d9d640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f80ce910 0x7f26f814feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26f659c640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f80d5d30 0x7f26f80b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 -- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f80d5d30 msgr2=0x7f26f80b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f80d5d30 0x7f26f80b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 -- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f80ce910 msgr2=0x7f26f814feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f80ce910 0x7f26f814feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26effff640 1 -- 172.21.15.53:0/595360751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f8150df0 con 0x7f26f815f3b0 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.221+0000 7f26f659c640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f80d5d30 0x7f26f80b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.222+0000 7f26effff640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f815f3b0 0x7f26f80ce3d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c7f0 tx=0x7f26e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.222+0000 7f26eeffd640 1 -- 172.21.15.53:0/595360751 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0015070 con 0x7f26f815f3b0 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.222+0000 7f26eeffd640 1 -- 172.21.15.53:0/595360751 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0007820 con 0x7f26f815f3b0 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.222+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f80a0620 con 0x7f26f815f3b0 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.222+0000 7f26eeffd640 1 -- 172.21.15.53:0/595360751 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0007c10 con 0x7f26f815f3b0 2024-05-10T10:33:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.222+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f80a0be0 con 0x7f26f815f3b0 2024-05-10T10:33:31.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.223+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f806a220 con 0x7f26f815f3b0 2024-05-10T10:33:31.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.223+0000 7f26eeffd640 1 -- 172.21.15.53:0/595360751 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f26e0018070 con 0x7f26f815f3b0 2024-05-10T10:33:31.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.224+0000 7f26eeffd640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26d80762d0 0x7f26d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.224+0000 7f26eeffd640 1 -- 172.21.15.53:0/595360751 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f26e0097370 con 0x7f26f815f3b0 2024-05-10T10:33:31.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.224+0000 7f26f659c640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26d80762d0 0x7f26d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.225+0000 7f26f659c640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26d80762d0 0x7f26d8078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26e8006fd0 tx=0x7f26e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:31.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.228+0000 7f26eeffd640 1 -- 172.21.15.53:0/595360751 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f26e0060930 con 0x7f26f815f3b0 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.421+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f26f8075a70 con 0x7f26f815f3b0 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.421+0000 7f26eeffd640 1 -- 172.21.15.53:0/595360751 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f26e0064c30 con 0x7f26f815f3b0 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:33:31.423 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:33:31.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26d80762d0 msgr2=0x7f26d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26d80762d0 0x7f26d8078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26e8006fd0 tx=0x7f26e8008040 comp rx=0 tx=0).stop 2024-05-10T10:33:31.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f815f3b0 msgr2=0x7f26f80ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f815f3b0 0x7f26f80ce3d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c7f0 tx=0x7f26e000ccb0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 shutdown_connections 2024-05-10T10:33:31.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26d80762d0 0x7f26d8078790 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f80ce910 0x7f26f814feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f815f3b0 0x7f26f80ce3d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 --2- 172.21.15.53:0/595360751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26f80d5d30 0x7f26f80b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 >> 172.21.15.53:0/595360751 conn(0x7f26f80d5130 msgr2=0x7f26f8161e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:31.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 shutdown_connections 2024-05-10T10:33:31.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.423+0000 7f26fc946640 1 -- 172.21.15.53:0/595360751 wait complete. 2024-05-10T10:33:31.436 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:33:31.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:33:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.542+0000 7f4a02eda640 1 Processor -- start 2024-05-10T10:33:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.542+0000 7f4a02eda640 1 -- start start 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f4a02eda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc15f490 0x7f49fc0d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f4a02eda640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49fc0d5ca0 0x7f49fc0d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f4a02eda640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49fc0d1080 con 0x7f49fc0d3b80 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f4a02eda640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f49fc0b4e70 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f4a02eda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc0b4820 con 0x7f49fc15f490 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f49fbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49fc0d5ca0 0x7f49fc0d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f4a01477640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f49fc0d3b80 0x7f49fc0cfc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38744/0 (socket says 172.21.15.53:38744) 2024-05-10T10:33:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.543+0000 7f4a01477640 1 -- 172.21.15.53:0/835328019 learned_addr learned my addr 172.21.15.53:0/835328019 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.544+0000 7f4a00c76640 1 --2- 172.21.15.53:0/835328019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc15f490 0x7f49fc0d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.544+0000 7f49fbfff640 1 -- 172.21.15.53:0/835328019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc15f490 msgr2=0x7f49fc0d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.544+0000 7f49fbfff640 1 --2- 172.21.15.53:0/835328019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc15f490 0x7f49fc0d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.544+0000 7f49fbfff640 1 -- 172.21.15.53:0/835328019 >> v1:172.21.15.53:6789/0 conn(0x7f49fc0d3b80 legacy=0x7f49fc0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.544+0000 7f49fbfff640 1 -- 172.21.15.53:0/835328019 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc0b4b00 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.544+0000 7f4a00c76640 1 --2- 172.21.15.53:0/835328019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc15f490 0x7f49fc0d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.544+0000 7f49fbfff640 1 --2- 172.21.15.53:0/835328019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc0d3640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009d90 tx=0x7f49e402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528770bb29d92c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f49fb7fe640 1 -- 172.21.15.53:0/835328019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4031600 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f49fb7fe640 1 -- 172.21.15.53:0/835328019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e40317a0 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f49fb7fe640 1 -- 172.21.15.53:0/835328019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4031b70 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 -- 172.21.15.53:0/835328019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 msgr2=0x7f49fc0d3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 --2- 172.21.15.53:0/835328019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc0d3640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009d90 tx=0x7f49e402a5d0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 -- 172.21.15.53:0/835328019 shutdown_connections 2024-05-10T10:33:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 --2- 172.21.15.53:0/835328019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc0d3640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 --2- 172.21.15.53:0/835328019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc15f490 0x7f49fc0d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 -- 172.21.15.53:0/835328019 >> 172.21.15.53:0/835328019 conn(0x7f49fc0d50f0 msgr2=0x7f49fc0d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 -- 172.21.15.53:0/835328019 shutdown_connections 2024-05-10T10:33:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.545+0000 7f4a02eda640 1 -- 172.21.15.53:0/835328019 wait complete. 2024-05-10T10:33:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.546+0000 7f4a02eda640 1 Processor -- start 2024-05-10T10:33:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f4a02eda640 1 -- start start 2024-05-10T10:33:31.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f4a02eda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0d3b80 0x7f49fc0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f4a02eda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f4a02eda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc15f490 0x7f49fc0ba660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f4a02eda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc0b6310 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f4a02eda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc15fa80 con 0x7f49fc15f490 2024-05-10T10:33:31.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f4a02eda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc0b6490 con 0x7f49fc0d3b80 2024-05-10T10:33:31.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.547+0000 7f49fbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc14e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34814/0 (socket says 172.21.15.53:34814) 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 -- 172.21.15.53:0/3009487680 learned_addr learned my addr 172.21.15.53:0/3009487680 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f4a01477640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc15f490 0x7f49fc0ba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f4a00c76640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0d3b80 0x7f49fc0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 -- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0d3b80 msgr2=0x7f49fc0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0d3b80 0x7f49fc0caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 -- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc15f490 msgr2=0x7f49fc0ba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc15f490 0x7f49fc0ba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 -- 172.21.15.53:0/3009487680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc0baba0 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49fbfff640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc14e8d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009d90 tx=0x7f49e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49f97fa640 1 -- 172.21.15.53:0/3009487680 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4040070 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49f97fa640 1 -- 172.21.15.53:0/3009487680 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e4030be0 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f49f97fa640 1 -- 172.21.15.53:0/3009487680 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4045460 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc0b95c0 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.548+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc0b9b80 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.549+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc06a220 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.550+0000 7f49f97fa640 1 -- 172.21.15.53:0/3009487680 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f49e4039040 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.550+0000 7f49f97fa640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49dc0762d0 0x7f49dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.550+0000 7f49f97fa640 1 -- 172.21.15.53:0/3009487680 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f49e40b89c0 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.551+0000 7f4a00c76640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49dc0762d0 0x7f49dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:31.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.551+0000 7f4a00c76640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49dc0762d0 0x7f49dc078790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00aac0 tx=0x7f49ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:31.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.555+0000 7f49f97fa640 1 -- 172.21.15.53:0/3009487680 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f49e40820d0 con 0x7f49fc0d5ca0 2024-05-10T10:33:31.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:31.749+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f49fc0c87c0 con 0x7f49fc0d5ca0 2024-05-10T10:33:32.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.470+0000 7f49f97fa640 1 -- 172.21.15.53:0/3009487680 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f49e40863d0 con 0x7f49fc0d5ca0 2024-05-10T10:33:32.472 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49dc0762d0 msgr2=0x7f49dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49dc0762d0 0x7f49dc078790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00aac0 tx=0x7f49ec008040 comp rx=0 tx=0).stop 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 msgr2=0x7f49fc14e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc14e8d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009d90 tx=0x7f49e4004a40 comp rx=0 tx=0).stop 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 shutdown_connections 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49dc0762d0 0x7f49dc078790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc15f490 0x7f49fc0ba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc0d5ca0 0x7f49fc14e8d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 --2- 172.21.15.53:0/3009487680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0d3b80 0x7f49fc0caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 >> 172.21.15.53:0/3009487680 conn(0x7f49fc0d50f0 msgr2=0x7f49fc0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:32.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 shutdown_connections 2024-05-10T10:33:32.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.474+0000 7f4a02eda640 1 -- 172.21.15.53:0/3009487680 wait complete. 2024-05-10T10:33:32.493 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-10T10:33:32.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.608+0000 7f9491c45640 1 Processor -- start 2024-05-10T10:33:32.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.608+0000 7f9491c45640 1 -- start start 2024-05-10T10:33:32.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.608+0000 7f9491c45640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c15d190 0x7f948c0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:32.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.608+0000 7f9491c45640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f948c0d1b90 0x7f948c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.608+0000 7f9491c45640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948c0cd9b0 con 0x7f948c0b3a80 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.608+0000 7f9491c45640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f948c0afe60 con 0x7f948c0d1b90 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.608+0000 7f9491c45640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f948c0b0140 con 0x7f948c15d190 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f948c0d1b90 0x7f948c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948b7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c15d190 0x7f948c0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f948c0d1b90 0x7f948c0b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:35870/0 (socket says 172.21.15.53:35870) 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948bfff640 1 -- 172.21.15.53:0/360148716 learned_addr learned my addr 172.21.15.53:0/360148716 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948bfff640 1 -- 172.21.15.53:0/360148716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c15d190 msgr2=0x7f948c0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948bfff640 1 --2- 172.21.15.53:0/360148716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c15d190 0x7f948c0b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948bfff640 1 -- 172.21.15.53:0/360148716 >> v1:172.21.15.135:6789/0 conn(0x7f948c0b3a80 legacy=0x7f948c0d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.609+0000 7f948bfff640 1 -- 172.21.15.53:0/360148716 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c0b0420 con 0x7f948c0d1b90 2024-05-10T10:33:32.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.610+0000 7f948bfff640 1 --2- 172.21.15.53:0/360148716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0d1b90 0x7f948c0b0f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9484009870 tx=0x7f948402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f58fc4b78fc249 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.610+0000 7f948a7fc640 1 -- 172.21.15.53:0/360148716 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484037070 con 0x7f948c0d1b90 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.610+0000 7f948a7fc640 1 -- 172.21.15.53:0/360148716 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9484032440 con 0x7f948c0d1b90 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.610+0000 7f948a7fc640 1 -- 172.21.15.53:0/360148716 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484032810 con 0x7f948c0d1b90 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 -- 172.21.15.53:0/360148716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0d1b90 msgr2=0x7f948c0b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 --2- 172.21.15.53:0/360148716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0d1b90 0x7f948c0b0f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9484009870 tx=0x7f948402a270 comp rx=0 tx=0).stop 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 -- 172.21.15.53:0/360148716 shutdown_connections 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 --2- 172.21.15.53:0/360148716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0d1b90 0x7f948c0b0f00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 --2- 172.21.15.53:0/360148716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c15d190 0x7f948c0b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 -- 172.21.15.53:0/360148716 >> 172.21.15.53:0/360148716 conn(0x7f948c09ba20 msgr2=0x7f948c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 -- 172.21.15.53:0/360148716 shutdown_connections 2024-05-10T10:33:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.611+0000 7f9491c45640 1 -- 172.21.15.53:0/360148716 wait complete. 2024-05-10T10:33:32.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.612+0000 7f9491c45640 1 Processor -- start 2024-05-10T10:33:32.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.612+0000 7f9491c45640 1 -- start start 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f9491c45640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0b3a80 0x7f948c160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f9491c45640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c0d1b90 0x7f948c1fa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f9491c45640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f948c15d190 0x7f948c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f9491c45640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f948c0afaf0 con 0x7f948c15d190 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f9491c45640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f948c0b09e0 con 0x7f948c0b3a80 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f9491c45640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f948c15d570 con 0x7f948c0d1b90 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f948c15d190 0x7f948c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f948c15d190 0x7f948c15a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34828/0 (socket says 172.21.15.53:34828) 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 -- 172.21.15.53:0/1794656805 learned_addr learned my addr 172.21.15.53:0/1794656805 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948b7fe640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0b3a80 0x7f948c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948affd640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c0d1b90 0x7f948c1fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 -- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c0d1b90 msgr2=0x7f948c1fa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c0d1b90 0x7f948c1fa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 -- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0b3a80 msgr2=0x7f948c160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0b3a80 0x7f948c160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.613+0000 7f948bfff640 1 -- 172.21.15.53:0/1794656805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c155280 con 0x7f948c15d190 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.614+0000 7f948bfff640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f948c15d190 0x7f948c15a9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9484009870 tx=0x7f94840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.614+0000 7f9473fff640 1 -- 172.21.15.53:0/1794656805 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484041020 con 0x7f948c15d190 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.614+0000 7f9473fff640 1 -- 172.21.15.53:0/1794656805 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948403d640 con 0x7f948c15d190 2024-05-10T10:33:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.614+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c155560 con 0x7f948c15d190 2024-05-10T10:33:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.614+0000 7f9473fff640 1 -- 172.21.15.53:0/1794656805 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484037040 con 0x7f948c15d190 2024-05-10T10:33:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.614+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c155b20 con 0x7f948c15d190 2024-05-10T10:33:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.615+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948c0693f0 con 0x7f948c15d190 2024-05-10T10:33:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.615+0000 7f9473fff640 1 -- 172.21.15.53:0/1794656805 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9484044020 con 0x7f948c15d190 2024-05-10T10:33:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.616+0000 7f9473fff640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9478076300 0x7f94780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.616+0000 7f9473fff640 1 -- 172.21.15.53:0/1794656805 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f94840b7910 con 0x7f948c15d190 2024-05-10T10:33:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.616+0000 7f948b7fe640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9478076300 0x7f94780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.617+0000 7f948b7fe640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9478076300 0x7f94780787c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f948c0bd420 tx=0x7f947c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.621+0000 7f9473fff640 1 -- 172.21.15.53:0/1794656805 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f948404b030 con 0x7f948c15d190 2024-05-10T10:33:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:32.815+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f948c0709b0 con 0x7f948c15d190 2024-05-10T10:33:33.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.473+0000 7f9473fff640 1 -- 172.21.15.53:0/1794656805 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f9484089430 con 0x7f948c15d190 2024-05-10T10:33:33.475 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:33:33.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9478076300 msgr2=0x7f94780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:33.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9478076300 0x7f94780787c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f948c0bd420 tx=0x7f947c00b040 comp rx=0 tx=0).stop 2024-05-10T10:33:33.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f948c15d190 msgr2=0x7f948c15a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:33.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f948c15d190 0x7f948c15a9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9484009870 tx=0x7f94840046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 shutdown_connections 2024-05-10T10:33:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9478076300 0x7f94780787c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f948c15d190 0x7f948c15a9d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f948c0d1b90 0x7f948c1fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 --2- 172.21.15.53:0/1794656805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948c0b3a80 0x7f948c160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.477+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 >> 172.21.15.53:0/1794656805 conn(0x7f948c09ba20 msgr2=0x7f948c158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:33.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.482+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 shutdown_connections 2024-05-10T10:33:33.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.488+0000 7f9491c45640 1 -- 172.21.15.53:0/1794656805 wait complete. 2024-05-10T10:33:33.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:33:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.620+0000 7f7326ec8640 1 Processor -- start 2024-05-10T10:33:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.620+0000 7f7326ec8640 1 -- start start 2024-05-10T10:33:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f7326ec8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f732015f410 0x7f73200d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f7326ec8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73200d2800 con 0x7f73200b56a0 2024-05-10T10:33:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f7326ec8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73200d2ae0 con 0x7f73200d0f10 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f7326ec8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f73200d01d0 con 0x7f732015f410 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f7325465640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f73200b56a0 0x7f73200d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46800/0 (socket says 172.21.15.53:46800) 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f7325465640 1 -- 172.21.15.53:0/520828140 learned_addr learned my addr 172.21.15.53:0/520828140 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f7324c64640 1 --2- 172.21.15.53:0/520828140 >> v2:172.21.15.135:3300/0 conn(0x7f732015f410 0x7f73200d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652032294 0 0) 0x7f73200d2800 con 0x7f73200b56a0 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7310003520 con 0x7f73200b56a0 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 920554905 0 0) 0x7f7310003520 con 0x7f73200b56a0 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 >> v2:172.21.15.135:3300/0 conn(0x7f732015f410 msgr2=0x7f73200d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.621+0000 7f731f7fe640 1 --2- 172.21.15.53:0/520828140 >> v2:172.21.15.135:3300/0 conn(0x7f732015f410 0x7f73200d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 >> v1:172.21.15.135:6789/0 conn(0x7f73200d0f10 legacy=0x7f73200b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200d04b0 con 0x7f73200b56a0 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f73180019f0 con 0x7f73200b56a0 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 >> v1:172.21.15.53:6789/0 conn(0x7f73200b56a0 legacy=0x7f73200d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f7310003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7310004800 0x7f7310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73100051c0 0x7f731000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73200d01d0 con 0x7f7310004800 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f731000c160 con 0x7f73100039b0 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f731000c360 con 0x7f73100051c0 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731ffff640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7310004800 0x7f7310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7324c64640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f7310003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7325465640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73100051c0 0x7f731000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7324c64640 1 -- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73100051c0 msgr2=0x7f731000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7324c64640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73100051c0 0x7f731000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7324c64640 1 -- 172.21.15.53:0/520828140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7310004800 msgr2=0x7f7310004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7324c64640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7310004800 0x7f7310004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7324c64640 1 -- 172.21.15.53:0/520828140 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7310003520 con 0x7f73100039b0 2024-05-10T10:33:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f7325465640 1 -- 172.21.15.53:0/520828140 reap_dead start 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.622+0000 7f731ffff640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7310004800 0x7f7310004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.623+0000 7f7324c64640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f7310003dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7314009ac0 tx=0x7f7310005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dffa2289be15248 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.623+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314036070 con 0x7f73100039b0 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.623+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7314031400 con 0x7f73100039b0 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.623+0000 7f731f7fe640 1 -- 172.21.15.53:0/520828140 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314009690 con 0x7f73100039b0 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.623+0000 7f7326ec8640 1 -- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 msgr2=0x7f7310003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.623+0000 7f7326ec8640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f7310003dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7314009ac0 tx=0x7f7310005830 comp rx=0 tx=0).stop 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.624+0000 7f7326ec8640 1 -- 172.21.15.53:0/520828140 shutdown_connections 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.624+0000 7f7326ec8640 1 --2- 172.21.15.53:0/520828140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f7310003dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.624+0000 7f7326ec8640 1 -- 172.21.15.53:0/520828140 >> 172.21.15.53:0/520828140 conn(0x7f732008ed30 msgr2=0x7f732009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.624+0000 7f7326ec8640 1 -- 172.21.15.53:0/520828140 shutdown_connections 2024-05-10T10:33:33.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.624+0000 7f7326ec8640 1 -- 172.21.15.53:0/520828140 wait complete. 2024-05-10T10:33:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.625+0000 7f7326ec8640 1 Processor -- start 2024-05-10T10:33:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.625+0000 7f7326ec8640 1 -- start start 2024-05-10T10:33:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7326ec8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f73200ba020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7326ec8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73200b7f40 0x7f732015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7326ec8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73201f63c0 0x7f73200d5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7326ec8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73200cffb0 con 0x7f73200b7f40 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7326ec8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73200cfe30 con 0x7f73100039b0 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7326ec8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f73200d09d0 con 0x7f73201f63c0 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73200b7f40 0x7f732015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73200b7f40 0x7f732015e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53144/0 (socket says 172.21.15.53:53144) 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 -- 172.21.15.53:0/1115431723 learned_addr learned my addr 172.21.15.53:0/1115431723 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7324c64640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f73200ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f7325465640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73201f63c0 0x7f73200d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 -- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73201f63c0 msgr2=0x7f73200d5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73201f63c0 0x7f73200d5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 -- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 msgr2=0x7f73200ba020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f73200ba020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 -- 172.21.15.53:0/1115431723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200b9c60 con 0x7f73200b7f40 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731ffff640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73200b7f40 0x7f732015e5b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f730c00f840 tx=0x7f730c00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731d7fa640 1 -- 172.21.15.53:0/1115431723 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c01a070 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731d7fa640 1 -- 172.21.15.53:0/1115431723 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f730c00fe90 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.626+0000 7f731d7fa640 1 -- 172.21.15.53:0/1115431723 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c001370 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.627+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73200b84b0 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.627+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73200b8a70 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.627+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732006a220 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.628+0000 7f731d7fa640 1 -- 172.21.15.53:0/1115431723 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f730c01d070 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.629+0000 7f731d7fa640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f73040762d0 0x7f7304078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.629+0000 7f731d7fa640 1 -- 172.21.15.53:0/1115431723 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f730c096c00 con 0x7f73200b7f40 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.629+0000 7f7324c64640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f73040762d0 0x7f7304078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.630+0000 7f7324c64640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f73040762d0 0x7f7304078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7314033000 tx=0x7f7314008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.633+0000 7f731d7fa640 1 -- 172.21.15.53:0/1115431723 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f730c09d080 con 0x7f73200b7f40 2024-05-10T10:33:33.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:33.863+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f73200b9760 con 0x7f73200b7f40 2024-05-10T10:33:34.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.498+0000 7f731d7fa640 1 -- 172.21.15.53:0/1115431723 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f730c05ff70 con 0x7f73200b7f40 2024-05-10T10:33:34.501 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.507+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f73040762d0 msgr2=0x7f7304078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f73040762d0 0x7f7304078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7314033000 tx=0x7f7314008030 comp rx=0 tx=0).stop 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73200b7f40 msgr2=0x7f732015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73200b7f40 0x7f732015e5b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f730c00f840 tx=0x7f730c00fc40 comp rx=0 tx=0).stop 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 shutdown_connections 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f73040762d0 0x7f7304078790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73201f63c0 0x7f73200d5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73200b7f40 0x7f732015e5b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 --2- 172.21.15.53:0/1115431723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73100039b0 0x7f73200ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 >> 172.21.15.53:0/1115431723 conn(0x7f732008ed30 msgr2=0x7f732006d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:34.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 shutdown_connections 2024-05-10T10:33:34.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.508+0000 7f7326ec8640 1 -- 172.21.15.53:0/1115431723 wait complete. 2024-05-10T10:33:34.530 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-10T10:33:34.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-10T10:33:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.642+0000 7fbed17e4640 1 Processor -- start 2024-05-10T10:33:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.642+0000 7fbed17e4640 1 -- start start 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbed17e4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc15f600 0x7fbecc0d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbed17e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbecc0d3d20 0x7fbecc0b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbed17e4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbecc0b46b0 con 0x7fbecc0d0b90 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbed17e4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbecc0b4990 con 0x7fbecc0d3d20 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbed17e4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc0b4c70 con 0x7fbecc15f600 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbecb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbecc0d3d20 0x7fbecc0b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbecb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbecc0d3d20 0x7fbecc0b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53146/0 (socket says 172.21.15.53:53146) 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbecb7fe640 1 -- 172.21.15.53:0/3395600223 learned_addr learned my addr 172.21.15.53:0/3395600223 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbecaffd640 1 --2- 172.21.15.53:0/3395600223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc15f600 0x7fbecc0d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbecb7fe640 1 -- 172.21.15.53:0/3395600223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc15f600 msgr2=0x7fbecc0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbecb7fe640 1 --2- 172.21.15.53:0/3395600223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc15f600 0x7fbecc0d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.643+0000 7fbecb7fe640 1 -- 172.21.15.53:0/3395600223 >> v1:172.21.15.135:6789/0 conn(0x7fbecc0d0b90 legacy=0x7fbecc0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.644+0000 7fbecb7fe640 1 -- 172.21.15.53:0/3395600223 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc0b4f50 con 0x7fbecc0d3d20 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.644+0000 7fbecaffd640 1 --2- 172.21.15.53:0/3395600223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc15f600 0x7fbecc0d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.644+0000 7fbecb7fe640 1 --2- 172.21.15.53:0/3395600223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc0d3d20 0x7fbecc0b5750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009870 tx=0x7fbebc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e543d61bfcd76a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.644+0000 7fbec9ffb640 1 -- 172.21.15.53:0/3395600223 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc037070 con 0x7fbecc0d3d20 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.644+0000 7fbec9ffb640 1 -- 172.21.15.53:0/3395600223 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbebc032440 con 0x7fbecc0d3d20 2024-05-10T10:33:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.644+0000 7fbec9ffb640 1 -- 172.21.15.53:0/3395600223 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc032810 con 0x7fbecc0d3d20 2024-05-10T10:33:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 -- 172.21.15.53:0/3395600223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc0d3d20 msgr2=0x7fbecc0b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 --2- 172.21.15.53:0/3395600223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc0d3d20 0x7fbecc0b5750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009870 tx=0x7fbebc02a1c0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 -- 172.21.15.53:0/3395600223 shutdown_connections 2024-05-10T10:33:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 --2- 172.21.15.53:0/3395600223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc0d3d20 0x7fbecc0b5750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 --2- 172.21.15.53:0/3395600223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc15f600 0x7fbecc0d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 -- 172.21.15.53:0/3395600223 >> 172.21.15.53:0/3395600223 conn(0x7fbecc09e390 msgr2=0x7fbecc0d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 -- 172.21.15.53:0/3395600223 shutdown_connections 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.645+0000 7fbed17e4640 1 -- 172.21.15.53:0/3395600223 wait complete. 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.646+0000 7fbed17e4640 1 Processor -- start 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.646+0000 7fbed17e4640 1 -- start start 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbed17e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbecc0d0b90 0x7fbecc0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbed17e4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc0d3d20 0x7fbecc1eb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbed17e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc15f600 0x7fbecc15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbed17e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc0b5e10 con 0x7fbecc15f600 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbed17e4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc0b5c90 con 0x7fbecc0d0b90 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbed17e4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc0b5f90 con 0x7fbecc0d3d20 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc15f600 0x7fbecc15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc15f600 0x7fbecc15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53162/0 (socket says 172.21.15.53:53162) 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbecb7fe640 1 -- 172.21.15.53:0/1026879032 learned_addr learned my addr 172.21.15.53:0/1026879032 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbecaffd640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbecc0d0b90 0x7fbecc0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.647+0000 7fbeca7fc640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc0d3d20 0x7fbecc1eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbecb7fe640 1 -- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc0d3d20 msgr2=0x7fbecc1eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbecb7fe640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc0d3d20 0x7fbecc1eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbecb7fe640 1 -- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbecc0d0b90 msgr2=0x7fbecc0cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbecb7fe640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbecc0d0b90 0x7fbecc0cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbecb7fe640 1 -- 172.21.15.53:0/1026879032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc0b9f90 con 0x7fbecc15f600 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbeca7fc640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc0d3d20 0x7fbecc1eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbecb7fe640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc15f600 0x7fbecc15dc00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009870 tx=0x7fbebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbec37fe640 1 -- 172.21.15.53:0/1026879032 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc037040 con 0x7fbecc15f600 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbec37fe640 1 -- 172.21.15.53:0/1026879032 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbebc042070 con 0x7fbecc15f600 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc0ba270 con 0x7fbecc15f600 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbec37fe640 1 -- 172.21.15.53:0/1026879032 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc030bd0 con 0x7fbecc15f600 2024-05-10T10:33:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.648+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc0ba830 con 0x7fbecc15f600 2024-05-10T10:33:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.649+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc06a220 con 0x7fbecc15f600 2024-05-10T10:33:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.650+0000 7fbec37fe640 1 -- 172.21.15.53:0/1026879032 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbebc02ad10 con 0x7fbecc15f600 2024-05-10T10:33:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.650+0000 7fbec37fe640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbeac0762d0 0x7fbeac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.650+0000 7fbec37fe640 1 -- 172.21.15.53:0/1026879032 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fbebc0b7f00 con 0x7fbecc15f600 2024-05-10T10:33:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.650+0000 7fbecaffd640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbeac0762d0 0x7fbeac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.651+0000 7fbecaffd640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbeac0762d0 0x7fbeac078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbec400f500 tx=0x7fbec4007770 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:34.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.655+0000 7fbec37fe640 1 -- 172.21.15.53:0/1026879032 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbebc081860 con 0x7fbecc15f600 2024-05-10T10:33:34.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:34.854+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fbecc0d4e10 con 0x7fbecc15f600 2024-05-10T10:33:35.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.503+0000 7fbec37fe640 1 -- 172.21.15.53:0/1026879032 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7fbebc081160 con 0x7fbecc15f600 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbeac0762d0 msgr2=0x7fbeac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbeac0762d0 0x7fbeac078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbec400f500 tx=0x7fbec4007770 comp rx=0 tx=0).stop 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc15f600 msgr2=0x7fbecc15dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc15f600 0x7fbecc15dc00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009870 tx=0x7fbebc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 shutdown_connections 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbeac0762d0 0x7fbeac078790 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbecc15f600 0x7fbecc15dc00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbecc0d3d20 0x7fbecc1eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 --2- 172.21.15.53:0/1026879032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbecc0d0b90 0x7fbecc0cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.506+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 >> 172.21.15.53:0/1026879032 conn(0x7fbecc09e390 msgr2=0x7fbecc06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.507+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 shutdown_connections 2024-05-10T10:33:35.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.507+0000 7fbed17e4640 1 -- 172.21.15.53:0/1026879032 wait complete. 2024-05-10T10:33:35.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:33:35.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.638+0000 7fa112d5f640 1 Processor -- start 2024-05-10T10:33:35.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.638+0000 7fa112d5f640 1 -- start start 2024-05-10T10:33:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa112d5f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c0d2050 0x7fa10c0d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa112d5f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa10c0b5ea0 0x7fa10c0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa112d5f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10c0d25c0 con 0x7fa10c162770 2024-05-10T10:33:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa112d5f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa10c0d2c40 con 0x7fa10c0b5ea0 2024-05-10T10:33:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa112d5f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa10c0d0910 con 0x7fa10c0d2050 2024-05-10T10:33:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa110afb640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa10c162770 0x7fa10c0d1940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46812/0 (socket says 172.21.15.53:46812) 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa110afb640 1 -- 172.21.15.53:0/3197608173 learned_addr learned my addr 172.21.15.53:0/3197608173 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa10bfff640 1 --2- 172.21.15.53:0/3197608173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c0d2050 0x7fa10c0d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa1112fc640 1 --2- 172.21.15.53:0/3197608173 >> v2:172.21.15.135:3300/0 conn(0x7fa10c0b5ea0 0x7fa10c0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa10b7fe640 1 -- 172.21.15.53:0/3197608173 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2453631377 0 0) 0x7fa10c0d25c0 con 0x7fa10c162770 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.639+0000 7fa10b7fe640 1 -- 172.21.15.53:0/3197608173 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0f4003520 con 0x7fa10c162770 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.640+0000 7fa10b7fe640 1 -- 172.21.15.53:0/3197608173 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 312500017 0 0) 0x7fa0f4003520 con 0x7fa10c162770 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.640+0000 7fa10bfff640 1 -- 172.21.15.53:0/3197608173 >> v2:172.21.15.135:3300/0 conn(0x7fa10c0b5ea0 msgr2=0x7fa10c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.640+0000 7fa10bfff640 1 --2- 172.21.15.53:0/3197608173 >> v2:172.21.15.135:3300/0 conn(0x7fa10c0b5ea0 0x7fa10c0d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.640+0000 7fa10bfff640 1 -- 172.21.15.53:0/3197608173 >> v1:172.21.15.53:6789/0 conn(0x7fa10c162770 legacy=0x7fa10c0d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.640+0000 7fa10bfff640 1 -- 172.21.15.53:0/3197608173 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c0d0bf0 con 0x7fa10c0d2050 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.640+0000 7fa10bfff640 1 --2- 172.21.15.53:0/3197608173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c0d2050 0x7fa10c0d30d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa10002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a406d6af69d85106 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa10b7fe640 1 -- 172.21.15.53:0/3197608173 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100037070 con 0x7fa10c0d2050 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa10b7fe640 1 -- 172.21.15.53:0/3197608173 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10002aba0 con 0x7fa10c0d2050 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa10b7fe640 1 -- 172.21.15.53:0/3197608173 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1040019f0 con 0x7fa10c0d2050 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 -- 172.21.15.53:0/3197608173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c0d2050 msgr2=0x7fa10c0d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 --2- 172.21.15.53:0/3197608173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c0d2050 0x7fa10c0d30d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa10002a110 comp rx=0 tx=0).stop 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 -- 172.21.15.53:0/3197608173 shutdown_connections 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 --2- 172.21.15.53:0/3197608173 >> v2:172.21.15.135:3300/0 conn(0x7fa10c0b5ea0 0x7fa10c0d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 --2- 172.21.15.53:0/3197608173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c0d2050 0x7fa10c0d30d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 -- 172.21.15.53:0/3197608173 >> 172.21.15.53:0/3197608173 conn(0x7fa10c09e3e0 msgr2=0x7fa10c0d5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 -- 172.21.15.53:0/3197608173 shutdown_connections 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.641+0000 7fa112d5f640 1 -- 172.21.15.53:0/3197608173 wait complete. 2024-05-10T10:33:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.642+0000 7fa112d5f640 1 Processor -- start 2024-05-10T10:33:35.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.642+0000 7fa112d5f640 1 -- start start 2024-05-10T10:33:35.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa112d5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa10c0b5ea0 0x7fa10c1f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:35.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa112d5f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c0d2050 0x7fa10c1e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa112d5f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c162770 0x7fa10c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa112d5f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c0b8040 con 0x7fa10c0d2050 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa112d5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c0b7ec0 con 0x7fa10c0b5ea0 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa112d5f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa10c162b50 con 0x7fa10c162770 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c0d2050 0x7fa10c1e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c0d2050 0x7fa10c1e8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53170/0 (socket says 172.21.15.53:53170) 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.643+0000 7fa10bfff640 1 -- 172.21.15.53:0/2795864596 learned_addr learned my addr 172.21.15.53:0/2795864596 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa110afb640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa10c0b5ea0 0x7fa10c1f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa1112fc640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c162770 0x7fa10c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa10bfff640 1 -- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c162770 msgr2=0x7fa10c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa10bfff640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c162770 0x7fa10c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa10bfff640 1 -- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa10c0b5ea0 msgr2=0x7fa10c1f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa10bfff640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa10c0b5ea0 0x7fa10c1f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa10bfff640 1 -- 172.21.15.53:0/2795864596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c0b8e50 con 0x7fa10c0d2050 2024-05-10T10:33:35.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa10bfff640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c0d2050 0x7fa10c1e8d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa1000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:35.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa1097fa640 1 -- 172.21.15.53:0/2795864596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100037050 con 0x7fa10c0d2050 2024-05-10T10:33:35.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa1097fa640 1 -- 172.21.15.53:0/2795864596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa100043070 con 0x7fa10c0d2050 2024-05-10T10:33:35.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c0b9130 con 0x7fa10c0d2050 2024-05-10T10:33:35.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa1097fa640 1 -- 172.21.15.53:0/2795864596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100006e10 con 0x7fa10c0d2050 2024-05-10T10:33:35.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.644+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c0b96f0 con 0x7fa10c0d2050 2024-05-10T10:33:35.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.645+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10c06a290 con 0x7fa10c0d2050 2024-05-10T10:33:35.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.646+0000 7fa1097fa640 1 -- 172.21.15.53:0/2795864596 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa10003ca10 con 0x7fa10c0d2050 2024-05-10T10:33:35.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.646+0000 7fa1097fa640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0f80762d0 0x7fa0f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:35.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.646+0000 7fa1097fa640 1 -- 172.21.15.53:0/2795864596 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7fa1000b8290 con 0x7fa10c0d2050 2024-05-10T10:33:35.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.647+0000 7fa110afb640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0f80762d0 0x7fa0f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:35.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.647+0000 7fa110afb640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0f80762d0 0x7fa0f8078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa104001560 tx=0x7fa104009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:35.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.651+0000 7fa1097fa640 1 -- 172.21.15.53:0/2795864596 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa100081a20 con 0x7fa10c0d2050 2024-05-10T10:33:35.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:35.843+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa10c0cf400 con 0x7fa10c0d2050 2024-05-10T10:33:37.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.510+0000 7fa1097fa640 1 -- 172.21.15.53:0/2795864596 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7fa100081320 con 0x7fa10c0d2050 2024-05-10T10:33:37.512 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data_ec' created 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0f80762d0 msgr2=0x7fa0f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0f80762d0 0x7fa0f8078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa104001560 tx=0x7fa104009040 comp rx=0 tx=0).stop 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c0d2050 msgr2=0x7fa10c1e8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c0d2050 0x7fa10c1e8d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa1000046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 shutdown_connections 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0f80762d0 0x7fa0f8078790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa10c162770 0x7fa10c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c0d2050 0x7fa10c1e8d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 --2- 172.21.15.53:0/2795864596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa10c0b5ea0 0x7fa10c1f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 >> 172.21.15.53:0/2795864596 conn(0x7fa10c09e3e0 msgr2=0x7fa10c0d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 shutdown_connections 2024-05-10T10:33:37.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.513+0000 7fa112d5f640 1 -- 172.21.15.53:0/2795864596 wait complete. 2024-05-10T10:33:37.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-10T10:33:37.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.646+0000 7fae5c6a0640 1 Processor -- start 2024-05-10T10:33:37.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.646+0000 7fae5c6a0640 1 -- start start 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5c6a0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae5415f1e0 0x7fae5406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5c6a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae540b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5c6a0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae540d0790 con 0x7fae5406c800 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5c6a0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fae540cc680 con 0x7fae5415f1e0 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5c6a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fae5406f580 con 0x7fae540d02c0 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5ac3d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae540b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5a43c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fae5415f1e0 0x7fae5406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:37.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5ac3d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae540b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:59210/0 (socket says 172.21.15.53:59210) 2024-05-10T10:33:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae5ac3d640 1 -- 172.21.15.53:0/1561563230 learned_addr learned my addr 172.21.15.53:0/1561563230 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.647+0000 7fae59c3b640 1 --1- 172.21.15.53:0/1561563230 >> v1:172.21.15.135:6789/0 conn(0x7fae5406c800 0x7fae540cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:58890/0 (socket says 172.21.15.53:58890) 2024-05-10T10:33:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.648+0000 7fae5ac3d640 1 -- 172.21.15.53:0/1561563230 >> v2:172.21.15.135:3300/0 conn(0x7fae5415f1e0 msgr2=0x7fae5406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.648+0000 7fae5ac3d640 1 --2- 172.21.15.53:0/1561563230 >> v2:172.21.15.135:3300/0 conn(0x7fae5415f1e0 0x7fae5406c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.648+0000 7fae5ac3d640 1 -- 172.21.15.53:0/1561563230 >> v1:172.21.15.135:6789/0 conn(0x7fae5406c800 legacy=0x7fae540cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.648+0000 7fae5ac3d640 1 -- 172.21.15.53:0/1561563230 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae540d0a70 con 0x7fae540d02c0 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.648+0000 7fae5ac3d640 1 --2- 172.21.15.53:0/1561563230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae540b08c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009e50 tx=0x7fae4c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a66a9cc0a98fa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.648+0000 7fae5943a640 1 -- 172.21.15.53:0/1561563230 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c036070 con 0x7fae540d02c0 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5943a640 1 -- 172.21.15.53:0/1561563230 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4c031600 con 0x7fae540d02c0 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5943a640 1 -- 172.21.15.53:0/1561563230 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0319d0 con 0x7fae540d02c0 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 -- 172.21.15.53:0/1561563230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 msgr2=0x7fae540b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/1561563230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae540b08c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009e50 tx=0x7fae4c02a710 comp rx=0 tx=0).stop 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 -- 172.21.15.53:0/1561563230 shutdown_connections 2024-05-10T10:33:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/1561563230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae540b08c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/1561563230 >> v2:172.21.15.135:3300/0 conn(0x7fae5415f1e0 0x7fae5406c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 -- 172.21.15.53:0/1561563230 >> 172.21.15.53:0/1561563230 conn(0x7fae540d1ce0 msgr2=0x7fae540d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 -- 172.21.15.53:0/1561563230 shutdown_connections 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.649+0000 7fae5c6a0640 1 -- 172.21.15.53:0/1561563230 wait complete. 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.650+0000 7fae5c6a0640 1 Processor -- start 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.650+0000 7fae5c6a0640 1 -- start start 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5c6a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae5406c800 0x7fae540c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5c6a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae5414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5c6a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae5415f1e0 0x7fae54149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5c6a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fae540b0e00 con 0x7fae5415f1e0 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5c6a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae5415f5c0 con 0x7fae5406c800 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5c6a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fae540b0f80 con 0x7fae540d02c0 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5ac3d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae5415f1e0 0x7fae54149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5ac3d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae5415f1e0 0x7fae54149810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53182/0 (socket says 172.21.15.53:53182) 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae5ac3d640 1 -- 172.21.15.53:0/3366686643 learned_addr learned my addr 172.21.15.53:0/3366686643 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.651+0000 7fae59c3b640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae5414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5a43c640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae5406c800 0x7fae540c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5ac3d640 1 -- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 msgr2=0x7fae5414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5ac3d640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae5414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5ac3d640 1 -- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae5406c800 msgr2=0x7fae540c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5ac3d640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae5406c800 0x7fae540c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5ac3d640 1 -- 172.21.15.53:0/3366686643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5414d190 con 0x7fae5415f1e0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5ac3d640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae5415f1e0 0x7fae54149810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fae4c034040 tx=0x7fae4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae4affd640 1 -- 172.21.15.53:0/3366686643 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c03f020 con 0x7fae5415f1e0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae4affd640 1 -- 172.21.15.53:0/3366686643 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4c037040 con 0x7fae5415f1e0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae540c89a0 con 0x7fae5415f1e0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae4affd640 1 -- 172.21.15.53:0/3366686643 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c036070 con 0x7fae5415f1e0 2024-05-10T10:33:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.652+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae540c7be0 con 0x7fae5415f1e0 2024-05-10T10:33:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.653+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae5406a220 con 0x7fae5415f1e0 2024-05-10T10:33:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.654+0000 7fae4affd640 1 -- 172.21.15.53:0/3366686643 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fae4c042020 con 0x7fae5415f1e0 2024-05-10T10:33:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.654+0000 7fae4affd640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fae380762d0 0x7fae38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.654+0000 7fae4affd640 1 -- 172.21.15.53:0/3366686643 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fae4c0b8190 con 0x7fae5415f1e0 2024-05-10T10:33:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.655+0000 7fae5a43c640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fae380762d0 0x7fae38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.655+0000 7fae5a43c640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fae380762d0 0x7fae38078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae50009a30 tx=0x7fae50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.658+0000 7fae4affd640 1 -- 172.21.15.53:0/3366686643 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fae4c049030 con 0x7fae5415f1e0 2024-05-10T10:33:37.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:37.846+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fae54075280 con 0x7fae5415f1e0 2024-05-10T10:33:38.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.512+0000 7fae4affd640 1 -- 172.21.15.53:0/3366686643 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fae5414d190 con 0x7fae5415f1e0 2024-05-10T10:33:38.514 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 4 allow_ec_overwrites to true 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fae380762d0 msgr2=0x7fae38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fae380762d0 0x7fae38078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae50009a30 tx=0x7fae50008040 comp rx=0 tx=0).stop 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae5415f1e0 msgr2=0x7fae54149810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae5415f1e0 0x7fae54149810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fae4c034040 tx=0x7fae4c004a40 comp rx=0 tx=0).stop 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 shutdown_connections 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fae380762d0 0x7fae38078790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae5415f1e0 0x7fae54149810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae540d02c0 0x7fae5414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 --2- 172.21.15.53:0/3366686643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae5406c800 0x7fae540c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 >> 172.21.15.53:0/3366686643 conn(0x7fae540d1ce0 msgr2=0x7fae540d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 shutdown_connections 2024-05-10T10:33:38.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.515+0000 7fae5c6a0640 1 -- 172.21.15.53:0/3366686643 wait complete. 2024-05-10T10:33:38.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-10T10:33:38.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.656+0000 7f5697e6d640 1 Processor -- start 2024-05-10T10:33:38.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.656+0000 7f5697e6d640 1 -- start start 2024-05-10T10:33:38.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5697e6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:38.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5697e6d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56900d4190 con 0x7f5690162900 2024-05-10T10:33:38.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5697e6d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56900cfc40 con 0x7f5690162ce0 2024-05-10T10:33:38.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5697e6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f56900d3eb0 con 0x7f56900d09c0 2024-05-10T10:33:38.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5695c09640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5690162900 0x7f56900d3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46824/0 (socket says 172.21.15.53:46824) 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5695c09640 1 -- 172.21.15.53:0/2355317379 learned_addr learned my addr 172.21.15.53:0/2355317379 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f569640a640 1 --2- 172.21.15.53:0/2355317379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 744653111 0 0) 0x7f568c001110 con 0x7f5690162900 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f567c003520 con 0x7f5690162900 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4048957669 0 0) 0x7f56900cfc40 con 0x7f5690162ce0 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.657+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f568c001110 con 0x7f5690162ce0 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.658+0000 7f569640a640 1 -- 172.21.15.53:0/2355317379 >> v1:172.21.15.135:6789/0 conn(0x7f5690162ce0 legacy=0x7f56900b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.658+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3867055950 0 0) 0x7f567c003520 con 0x7f5690162900 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.658+0000 7f569640a640 1 -- 172.21.15.53:0/2355317379 >> v1:172.21.15.53:6789/0 conn(0x7f5690162900 legacy=0x7f56900d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:38.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.658+0000 7f569640a640 1 -- 172.21.15.53:0/2355317379 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56900cfee0 con 0x7f56900d09c0 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.658+0000 7f569640a640 1 --2- 172.21.15.53:0/2355317379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900d0e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5688009dc0 tx=0x7f568802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd21df2980ff8db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.658+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c001840 con 0x7f56900d09c0 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5688007540 con 0x7f56900d09c0 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5694c07640 1 -- 172.21.15.53:0/2355317379 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688007910 con 0x7f56900d09c0 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5697e6d640 1 -- 172.21.15.53:0/2355317379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 msgr2=0x7f56900d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5697e6d640 1 --2- 172.21.15.53:0/2355317379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900d0e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5688009dc0 tx=0x7f568802a5d0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5697e6d640 1 -- 172.21.15.53:0/2355317379 shutdown_connections 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5697e6d640 1 --2- 172.21.15.53:0/2355317379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900d0e70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5697e6d640 1 -- 172.21.15.53:0/2355317379 >> 172.21.15.53:0/2355317379 conn(0x7f569009e430 msgr2=0x7f56900d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5697e6d640 1 -- 172.21.15.53:0/2355317379 shutdown_connections 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.659+0000 7f5697e6d640 1 -- 172.21.15.53:0/2355317379 wait complete. 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.660+0000 7f5697e6d640 1 Processor -- start 2024-05-10T10:33:38.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.660+0000 7f5697e6d640 1 -- start start 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5697e6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5697e6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5690162900 0x7f56900ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5697e6d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5690162ce0 0x7f56900b9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5697e6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56900b4180 con 0x7f5690162900 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5697e6d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56900d3b20 con 0x7f5690162ce0 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5697e6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f56900b4300 con 0x7f56900d09c0 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5695408640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5690162900 0x7f56900ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5695408640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5690162900 0x7f56900ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53188/0 (socket says 172.21.15.53:53188) 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5695408640 1 -- 172.21.15.53:0/3788247542 learned_addr learned my addr 172.21.15.53:0/3788247542 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:38.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f569640a640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5690162ce0 0x7f56900b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.661+0000 7f5695c09640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5695408640 1 -- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 msgr2=0x7f56900cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5695408640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5695408640 1 -- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5690162ce0 msgr2=0x7f56900b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5695408640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5690162ce0 0x7f56900b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5695408640 1 -- 172.21.15.53:0/3788247542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56900ceed0 con 0x7f5690162900 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5695408640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5690162900 0x7f56900ce990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f568000c460 tx=0x7f568000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f56867fc640 1 -- 172.21.15.53:0/3788247542 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680017070 con 0x7f5690162900 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f56867fc640 1 -- 172.21.15.53:0/3788247542 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56800052f0 con 0x7f5690162900 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56900ce3e0 con 0x7f5690162900 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f56867fc640 1 -- 172.21.15.53:0/3788247542 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680003830 con 0x7f5690162900 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.662+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56900ccfd0 con 0x7f5690162900 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.663+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569006a220 con 0x7f5690162900 2024-05-10T10:33:38.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.663+0000 7f56867fc640 1 -- 172.21.15.53:0/3788247542 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f568001a070 con 0x7f5690162900 2024-05-10T10:33:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.664+0000 7f56867fc640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56740762d0 0x7f5674078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.664+0000 7f56867fc640 1 -- 172.21.15.53:0/3788247542 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f5680096c20 con 0x7f5690162900 2024-05-10T10:33:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.664+0000 7f5695c09640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56740762d0 0x7f5674078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:38.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.665+0000 7f5695c09640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56740762d0 0x7f5674078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f568c0013b0 tx=0x7f568c001df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:38.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.668+0000 7f56867fc640 1 -- 172.21.15.53:0/3788247542 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f568005fc10 con 0x7f5690162900 2024-05-10T10:33:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:38.902+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f569009e840 con 0x7f5690162900 2024-05-10T10:33:39.454 INFO:tasks.ceph.mds.b.smithi135.stderr:2024-05-10T10:33:39.453+0000 7fbc79f60640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:33:39.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.548+0000 7f56867fc640 1 -- 172.21.15.53:0/3788247542 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f5680068250 con 0x7f5690162900 2024-05-10T10:33:39.551 INFO:teuthology.orchestra.run.smithi053.stderr:added data pool 4 to fsmap 2024-05-10T10:33:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56740762d0 msgr2=0x7f5674078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56740762d0 0x7f5674078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f568c0013b0 tx=0x7f568c001df0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5690162900 msgr2=0x7f56900ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5690162900 0x7f56900ce990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f568000c460 tx=0x7f568000c920 comp rx=0 tx=0).stop 2024-05-10T10:33:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 shutdown_connections 2024-05-10T10:33:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56740762d0 0x7f5674078790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5690162ce0 0x7f56900b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5690162900 0x7f56900ce990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 --2- 172.21.15.53:0/3788247542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56900d09c0 0x7f56900cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 >> 172.21.15.53:0/3788247542 conn(0x7f569009e430 msgr2=0x7f5690161b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 shutdown_connections 2024-05-10T10:33:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.552+0000 7f5697e6d640 1 -- 172.21.15.53:0/3788247542 wait complete. 2024-05-10T10:33:39.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:39.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.676+0000 7f71f71b6640 1 Processor -- start 2024-05-10T10:33:39.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.676+0000 7f71f71b6640 1 -- start start 2024-05-10T10:33:39.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.677+0000 7f71f71b6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f0162750 0x7f71f00d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:39.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.677+0000 7f71f71b6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71f0162b30 0x7f71f00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:39.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.677+0000 7f71f71b6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f00b45f0 con 0x7f71f00d5cb0 2024-05-10T10:33:39.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.677+0000 7f71f71b6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f71f00b4f20 con 0x7f71f0162b30 2024-05-10T10:33:39.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.677+0000 7f71f71b6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f71f00d0fe0 con 0x7f71f0162750 2024-05-10T10:33:39.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71e7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f71f00d5cb0 0x7f71f00d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46826/0 (socket says 172.21.15.53:46826) 2024-05-10T10:33:39.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71e7fff640 1 -- 172.21.15.53:0/733097228 learned_addr learned my addr 172.21.15.53:0/733097228 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:39.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71f4f52640 1 --2- 172.21.15.53:0/733097228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f0162750 0x7f71f00d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:39.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71f5753640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71f0162b30 0x7f71f00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:39.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71e77fe640 1 -- 172.21.15.53:0/733097228 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515927230 0 0) 0x7f71f00b45f0 con 0x7f71f00d5cb0 2024-05-10T10:33:39.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71e77fe640 1 -- 172.21.15.53:0/733097228 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71d8003520 con 0x7f71f00d5cb0 2024-05-10T10:33:39.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71f4f52640 1 -- 172.21.15.53:0/733097228 >> v2:172.21.15.53:3300/0 conn(0x7f71f0162b30 msgr2=0x7f71f00b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:39.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71f4f52640 1 --2- 172.21.15.53:0/733097228 >> v2:172.21.15.53:3300/0 conn(0x7f71f0162b30 0x7f71f00b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71f4f52640 1 -- 172.21.15.53:0/733097228 >> v1:172.21.15.53:6789/0 conn(0x7f71f00d5cb0 legacy=0x7f71f00d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.678+0000 7f71f4f52640 1 -- 172.21.15.53:0/733097228 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f00b48d0 con 0x7f71f0162750 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.679+0000 7f71f4f52640 1 --2- 172.21.15.53:0/733097228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f0162750 0x7f71f00d0670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0097a0 tx=0x7f71ec02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf601b741fd7b0bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.679+0000 7f71e77fe640 1 -- 172.21.15.53:0/733097228 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec037070 con 0x7f71f0162750 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.679+0000 7f71e77fe640 1 -- 172.21.15.53:0/733097228 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ec02ae50 con 0x7f71f0162750 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.679+0000 7f71e77fe640 1 -- 172.21.15.53:0/733097228 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec032660 con 0x7f71f0162750 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 -- 172.21.15.53:0/733097228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f0162750 msgr2=0x7f71f00d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 --2- 172.21.15.53:0/733097228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f0162750 0x7f71f00d0670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0097a0 tx=0x7f71ec02a100 comp rx=0 tx=0).stop 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 -- 172.21.15.53:0/733097228 shutdown_connections 2024-05-10T10:33:39.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 --2- 172.21.15.53:0/733097228 >> v2:172.21.15.53:3300/0 conn(0x7f71f0162b30 0x7f71f00b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 --2- 172.21.15.53:0/733097228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f0162750 0x7f71f00d0670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 -- 172.21.15.53:0/733097228 >> 172.21.15.53:0/733097228 conn(0x7f71f009e3b0 msgr2=0x7f71f00d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:39.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 -- 172.21.15.53:0/733097228 shutdown_connections 2024-05-10T10:33:39.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.680+0000 7f71f71b6640 1 -- 172.21.15.53:0/733097228 wait complete. 2024-05-10T10:33:39.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.681+0000 7f71f71b6640 1 Processor -- start 2024-05-10T10:33:39.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.681+0000 7f71f71b6640 1 -- start start 2024-05-10T10:33:39.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71f71b6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f0162750 0x7f71f0156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:39.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71f71b6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71f0162b30 0x7f71f00cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71f71b6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f00c85e0 0x7f71f00b9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71f71b6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f71f00d0d30 con 0x7f71f0162b30 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71f71b6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f71f00d0bb0 con 0x7f71f0162750 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71f71b6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f71f00d3d40 con 0x7f71f00c85e0 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71f0162b30 0x7f71f00cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71f0162b30 0x7f71f00cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53212/0 (socket says 172.21.15.53:53212) 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.682+0000 7f71e7fff640 1 -- 172.21.15.53:0/982125507 learned_addr learned my addr 172.21.15.53:0/982125507 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71f5753640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f00c85e0 0x7f71f00b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71f4f52640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f0162750 0x7f71f0156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e7fff640 1 -- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f00c85e0 msgr2=0x7f71f00b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e7fff640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f00c85e0 0x7f71f00b9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e7fff640 1 -- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f0162750 msgr2=0x7f71f0156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e7fff640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f0162750 0x7f71f0156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e7fff640 1 -- 172.21.15.53:0/982125507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f00cc950 con 0x7f71f0162b30 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71f4f52640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f0162750 0x7f71f0156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e7fff640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71f0162b30 0x7f71f00cc120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c460 tx=0x7f71e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e57fa640 1 -- 172.21.15.53:0/982125507 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0017070 con 0x7f71f0162b30 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e57fa640 1 -- 172.21.15.53:0/982125507 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e0005ce0 con 0x7f71f0162b30 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f00ccc30 con 0x7f71f0162b30 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71e57fa640 1 -- 172.21.15.53:0/982125507 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0007290 con 0x7f71f0162b30 2024-05-10T10:33:39.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.683+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f00cc660 con 0x7f71f0162b30 2024-05-10T10:33:39.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.684+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f006a220 con 0x7f71f0162b30 2024-05-10T10:33:39.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.685+0000 7f71e57fa640 1 -- 172.21.15.53:0/982125507 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f71e001a070 con 0x7f71f0162b30 2024-05-10T10:33:39.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.685+0000 7f71e57fa640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f71dc0762d0 0x7f71dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:39.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.685+0000 7f71e57fa640 1 -- 172.21.15.53:0/982125507 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f71e0096c50 con 0x7f71f0162b30 2024-05-10T10:33:39.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.686+0000 7f71f4f52640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f71dc0762d0 0x7f71dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:39.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.686+0000 7f71f4f52640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f71dc0762d0 0x7f71dc078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71ec034000 tx=0x7f71ec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:39.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.690+0000 7f71e57fa640 1 -- 172.21.15.53:0/982125507 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f71e0060260 con 0x7f71f0162b30 2024-05-10T10:33:39.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.924+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71f00b5830 con 0x7f71f0162b30 2024-05-10T10:33:39.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.925+0000 7f71e57fa640 1 -- 172.21.15.53:0/982125507 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5177 (secure 0 0 0) 0x7f71e0068250 con 0x7f71f0162b30 2024-05-10T10:33:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:39.927 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:38.904372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:33:39.928 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 7 2024-05-10T10:33:39.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f71dc0762d0 msgr2=0x7f71dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f71dc0762d0 0x7f71dc078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71ec034000 tx=0x7f71ec02e000 comp rx=0 tx=0).stop 2024-05-10T10:33:39.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71f0162b30 msgr2=0x7f71f00cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:39.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71f0162b30 0x7f71f00cc120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c460 tx=0x7f71e000c920 comp rx=0 tx=0).stop 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 shutdown_connections 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f71dc0762d0 0x7f71dc078790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f71f00c85e0 0x7f71f00b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71f0162b30 0x7f71f00cc120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 --2- 172.21.15.53:0/982125507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f0162750 0x7f71f0156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 >> 172.21.15.53:0/982125507 conn(0x7f71f009e3b0 msgr2=0x7f71f00b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 shutdown_connections 2024-05-10T10:33:39.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:39.927+0000 7f71f71b6640 1 -- 172.21.15.53:0/982125507 wait complete. 2024-05-10T10:33:39.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:40.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.042+0000 7f287c7a5640 1 Processor -- start 2024-05-10T10:33:40.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.042+0000 7f287c7a5640 1 -- start start 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.043+0000 7f287c7a5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f28741598c0 0x7f287406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.043+0000 7f287c7a5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287406f1c0 con 0x7f287406ec40 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.043+0000 7f287c7a5640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287406f4a0 con 0x7f28740c7390 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.043+0000 7f287c7a5640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f287406f780 con 0x7f28741598c0 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f2879d40640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f287406ec40 0x7f28740c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46840/0 (socket says 172.21.15.53:46840) 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f2879d40640 1 -- 172.21.15.53:0/1555557775 learned_addr learned my addr 172.21.15.53:0/1555557775 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287a541640 1 --2- 172.21.15.53:0/1555557775 >> v2:172.21.15.135:3300/0 conn(0x7f28741598c0 0x7f287406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769070207 0 0) 0x7f287406f1c0 con 0x7f287406ec40 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2860003520 con 0x7f287406ec40 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2354201595 0 0) 0x7f2860003520 con 0x7f287406ec40 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 >> v2:172.21.15.135:3300/0 conn(0x7f28741598c0 msgr2=0x7f287406e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 --2- 172.21.15.53:0/1555557775 >> v2:172.21.15.135:3300/0 conn(0x7f28741598c0 0x7f287406e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 >> v1:172.21.15.135:6789/0 conn(0x7f28740c7390 legacy=0x7f28740cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287406fa60 con 0x7f287406ec40 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f2864001670 con 0x7f287406ec40 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 >> v1:172.21.15.53:6789/0 conn(0x7f287406ec40 legacy=0x7f28740c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600039b0 0x7f2860003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2860004800 0x7f2860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28600051c0 0x7f286000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f287406f780 con 0x7f28600051c0 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f286000c060 con 0x7f28600039b0 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f286000c260 con 0x7f2860004800 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287ad42640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28600051c0 0x7f286000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f287a541640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600039b0 0x7f2860003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.044+0000 7f2879d40640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2860004800 0x7f2860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287ad42640 1 -- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2860004800 msgr2=0x7f2860004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287ad42640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2860004800 0x7f2860004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287ad42640 1 -- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600039b0 msgr2=0x7f2860003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287ad42640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600039b0 0x7f2860003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287ad42640 1 -- 172.21.15.53:0/1555557775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2860003520 con 0x7f28600051c0 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287ad42640 1 -- 172.21.15.53:0/1555557775 reap_dead start 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287ad42640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28600051c0 0x7f286000b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f286c001e20 tx=0x7f2860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4be911a8359ebdf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c001a70 con 0x7f28600051c0 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286c02ae40 con 0x7f28600051c0 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.045+0000 7f287953f640 1 -- 172.21.15.53:0/1555557775 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c003bb0 con 0x7f28600051c0 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.046+0000 7f287c7a5640 1 -- 172.21.15.53:0/1555557775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28600051c0 msgr2=0x7f286000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.046+0000 7f287c7a5640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28600051c0 0x7f286000b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f286c001e20 tx=0x7f2860005830 comp rx=0 tx=0).stop 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.046+0000 7f287c7a5640 1 -- 172.21.15.53:0/1555557775 shutdown_connections 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.046+0000 7f287c7a5640 1 --2- 172.21.15.53:0/1555557775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28600051c0 0x7f286000b9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.046+0000 7f287c7a5640 1 -- 172.21.15.53:0/1555557775 >> 172.21.15.53:0/1555557775 conn(0x7f28740b2900 msgr2=0x7f28740b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:40.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.046+0000 7f287c7a5640 1 -- 172.21.15.53:0/1555557775 shutdown_connections 2024-05-10T10:33:40.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.046+0000 7f287c7a5640 1 -- 172.21.15.53:0/1555557775 wait complete. 2024-05-10T10:33:40.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.047+0000 7f287c7a5640 1 Processor -- start 2024-05-10T10:33:40.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.047+0000 7f287c7a5640 1 -- start start 2024-05-10T10:33:40.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f287c7a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600051c0 0x7f28740cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f287c7a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28741f59b0 0x7f28741f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f287c7a5640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28741f8a20 0x7f28740c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f287c7a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2874159ca0 con 0x7f28741f59b0 2024-05-10T10:33:40.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f287c7a5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f287415c6e0 con 0x7f28600051c0 2024-05-10T10:33:40.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f287c7a5640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2874159e20 con 0x7f28741f8a20 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28741f59b0 0x7f28741f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28741f59b0 0x7f28741f84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53228/0 (socket says 172.21.15.53:53228) 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 -- 172.21.15.53:0/988283707 learned_addr learned my addr 172.21.15.53:0/988283707 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 -- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28741f8a20 msgr2=0x7f28740c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28741f8a20 0x7f28740c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f287a541640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600051c0 0x7f28740cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 -- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600051c0 msgr2=0x7f28740cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600051c0 0x7f28740cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.048+0000 7f2879d40640 1 -- 172.21.15.53:0/988283707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28741f4ef0 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.049+0000 7f2879d40640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28741f59b0 0x7f28741f84e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28640072f0 tx=0x7f28640077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.049+0000 7f286affd640 1 -- 172.21.15.53:0/988283707 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286401a070 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.049+0000 7f286affd640 1 -- 172.21.15.53:0/988283707 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286400f960 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.049+0000 7f286affd640 1 -- 172.21.15.53:0/988283707 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286400fd30 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.049+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28741f8f60 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.049+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28741f51b0 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.050+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287406a220 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.050+0000 7f286affd640 1 -- 172.21.15.53:0/988283707 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f286401d070 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.051+0000 7f286affd640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28580762d0 0x7f2858078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.051+0000 7f286affd640 1 -- 172.21.15.53:0/988283707 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2864097270 con 0x7f28741f59b0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.051+0000 7f287a541640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28580762d0 0x7f2858078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.052+0000 7f287a541640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28580762d0 0x7f2858078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f287415ec40 tx=0x7f2870008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.055+0000 7f286affd640 1 -- 172.21.15.53:0/988283707 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2864060920 con 0x7f28741f59b0 2024-05-10T10:33:40.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.247+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f287406e4e0 con 0x7f28741f59b0 2024-05-10T10:33:40.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.248+0000 7f286affd640 1 -- 172.21.15.53:0/988283707 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f2864060220 con 0x7f28741f59b0 2024-05-10T10:33:40.250 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:40.250 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28580762d0 msgr2=0x7f2858078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28580762d0 0x7f2858078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f287415ec40 tx=0x7f2870008040 comp rx=0 tx=0).stop 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28741f59b0 msgr2=0x7f28741f84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28741f59b0 0x7f28741f84e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28640072f0 tx=0x7f28640077b0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 shutdown_connections 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28580762d0 0x7f2858078790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28741f8a20 0x7f28740c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28741f59b0 0x7f28741f84e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 --2- 172.21.15.53:0/988283707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600051c0 0x7f28740cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 >> 172.21.15.53:0/988283707 conn(0x7f28740b2900 msgr2=0x7f2874158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:40.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 shutdown_connections 2024-05-10T10:33:40.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.250+0000 7f287c7a5640 1 -- 172.21.15.53:0/988283707 wait complete. 2024-05-10T10:33:40.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:40.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.365+0000 7fc4d8edc640 1 Processor -- start 2024-05-10T10:33:40.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.365+0000 7fc4d8edc640 1 -- start start 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d8edc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4d415f3b0 0x7fc4d40d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d8edc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d415f790 0x7fc4d40d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d8edc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d40d30b0 con 0x7fc4d40d5d30 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d8edc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d40d3390 con 0x7fc4d415f3b0 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d8edc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40c8c80 con 0x7fc4d415f790 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4d415f3b0 0x7fc4d40d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4d415f3b0 0x7fc4d40d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53232/0 (socket says 172.21.15.53:53232) 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d259c640 1 -- 172.21.15.53:0/1998118350 learned_addr learned my addr 172.21.15.53:0/1998118350 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4c9d9b640 1 --2- 172.21.15.53:0/1998118350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d415f790 0x7fc4d40d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d259c640 1 -- 172.21.15.53:0/1998118350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d415f790 msgr2=0x7fc4d40d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d259c640 1 --2- 172.21.15.53:0/1998118350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d415f790 0x7fc4d40d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d259c640 1 -- 172.21.15.53:0/1998118350 >> v1:172.21.15.135:6789/0 conn(0x7fc4d40d5d30 legacy=0x7fc4d40c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.366+0000 7fc4d259c640 1 -- 172.21.15.53:0/1998118350 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d40c8f60 con 0x7fc4d415f3b0 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.367+0000 7fc4d259c640 1 --2- 172.21.15.53:0/1998118350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d415f3b0 0x7fc4d40d36f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009870 tx=0x7fc4c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1f2942dd1f95e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.367+0000 7fc4d1d9b640 1 -- 172.21.15.53:0/1998118350 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4037070 con 0x7fc4d415f3b0 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.367+0000 7fc4d1d9b640 1 -- 172.21.15.53:0/1998118350 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c4032440 con 0x7fc4d415f3b0 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.367+0000 7fc4d1d9b640 1 -- 172.21.15.53:0/1998118350 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4032810 con 0x7fc4d415f3b0 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.367+0000 7fc4d8edc640 1 -- 172.21.15.53:0/1998118350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d415f3b0 msgr2=0x7fc4d40d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.367+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/1998118350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d415f3b0 0x7fc4d40d36f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009870 tx=0x7fc4c402a270 comp rx=0 tx=0).stop 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.368+0000 7fc4d8edc640 1 -- 172.21.15.53:0/1998118350 shutdown_connections 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.368+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/1998118350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d415f790 0x7fc4d40d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.368+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/1998118350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d415f3b0 0x7fc4d40d36f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.368+0000 7fc4d8edc640 1 -- 172.21.15.53:0/1998118350 >> 172.21.15.53:0/1998118350 conn(0x7fc4d40d5130 msgr2=0x7fc4d40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:40.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.368+0000 7fc4d8edc640 1 -- 172.21.15.53:0/1998118350 shutdown_connections 2024-05-10T10:33:40.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.368+0000 7fc4d8edc640 1 -- 172.21.15.53:0/1998118350 wait complete. 2024-05-10T10:33:40.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.369+0000 7fc4d8edc640 1 Processor -- start 2024-05-10T10:33:40.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.369+0000 7fc4d8edc640 1 -- start start 2024-05-10T10:33:40.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d8edc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d40d5d30 0x7fc4d41549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d8edc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4d415f3b0 0x7fc4d4156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d8edc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d41551a0 0x7fc4d4161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d8edc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40d0e60 con 0x7fc4d40d5d30 2024-05-10T10:33:40.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d8edc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40d0ce0 con 0x7fc4d415f3b0 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d8edc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40d0fe0 con 0x7fc4d41551a0 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d40d5d30 0x7fc4d41549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d40d5d30 0x7fc4d41549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53234/0 (socket says 172.21.15.53:53234) 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 -- 172.21.15.53:0/2843052800 learned_addr learned my addr 172.21.15.53:0/2843052800 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4c9d9b640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4d415f3b0 0x7fc4d4156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d2d9d640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d41551a0 0x7fc4d4161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 -- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d41551a0 msgr2=0x7fc4d4161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d41551a0 0x7fc4d4161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 -- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4d415f3b0 msgr2=0x7fc4d4156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4d415f3b0 0x7fc4d4156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 -- 172.21.15.53:0/2843052800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4154f20 con 0x7fc4d40d5d30 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.370+0000 7fc4d259c640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d40d5d30 0x7fc4d41549e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c402a2b0 tx=0x7fc4c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.371+0000 7fc4cb7fe640 1 -- 172.21.15.53:0/2843052800 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4041020 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.371+0000 7fc4cb7fe640 1 -- 172.21.15.53:0/2843052800 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c40306c0 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.371+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d4155bf0 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.371+0000 7fc4cb7fe640 1 -- 172.21.15.53:0/2843052800 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4037040 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.371+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d4156490 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.372+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d406a220 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.372+0000 7fc4cb7fe640 1 -- 172.21.15.53:0/2843052800 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4c4044020 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.373+0000 7fc4cb7fe640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4b4076300 0x7fc4b40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.373+0000 7fc4cb7fe640 1 -- 172.21.15.53:0/2843052800 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc4c40b7cf0 con 0x7fc4d40d5d30 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.373+0000 7fc4c9d9b640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4b4076300 0x7fc4b40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.374+0000 7fc4c9d9b640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4b4076300 0x7fc4b40787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc4d415bc50 tx=0x7fc4bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.377+0000 7fc4cb7fe640 1 -- 172.21.15.53:0/2843052800 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc4c40bc070 con 0x7fc4d40d5d30 2024-05-10T10:33:40.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.570+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4d40cf460 con 0x7fc4d40d5d30 2024-05-10T10:33:40.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.571+0000 7fc4cb7fe640 1 -- 172.21.15.53:0/2843052800 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fc4c404b070 con 0x7fc4d40d5d30 2024-05-10T10:33:40.573 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:40.573 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:40.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4b4076300 msgr2=0x7fc4b40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4b4076300 0x7fc4b40787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc4d415bc50 tx=0x7fc4bc008040 comp rx=0 tx=0).stop 2024-05-10T10:33:40.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d40d5d30 msgr2=0x7fc4d41549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d40d5d30 0x7fc4d41549e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c402a2b0 tx=0x7fc4c40046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 shutdown_connections 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4b4076300 0x7fc4b40787c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4d41551a0 0x7fc4d4161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4d415f3b0 0x7fc4d4156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 --2- 172.21.15.53:0/2843052800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4d40d5d30 0x7fc4d41549e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 >> 172.21.15.53:0/2843052800 conn(0x7fc4d40d5130 msgr2=0x7fc4d406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 shutdown_connections 2024-05-10T10:33:40.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.573+0000 7fc4d8edc640 1 -- 172.21.15.53:0/2843052800 wait complete. 2024-05-10T10:33:40.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.704+0000 7f65e0baf640 1 Processor -- start 2024-05-10T10:33:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.704+0000 7f65e0baf640 1 -- start start 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.704+0000 7f65e0baf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f65dc15f290 0x7f65dc0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.704+0000 7f65e0baf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65dc06f3c0 con 0x7f65dc0d3450 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.704+0000 7f65e0baf640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65dc0d16f0 con 0x7f65dc0702e0 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.704+0000 7f65e0baf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f65dc0d14c0 con 0x7f65dc15f290 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65dad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f65dc0d3450 0x7f65dc06d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46854/0 (socket says 172.21.15.53:46854) 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65dad9d640 1 -- 172.21.15.53:0/4212985280 learned_addr learned my addr 172.21.15.53:0/4212985280 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65da59c640 1 --2- 172.21.15.53:0/4212985280 >> v2:172.21.15.135:3300/0 conn(0x7f65dc15f290 0x7f65dc0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307908630 0 0) 0x7f65dc06f3c0 con 0x7f65dc0d3450 2024-05-10T10:33:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65c8003520 con 0x7f65dc0d3450 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 453554072 0 0) 0x7f65c8003520 con 0x7f65dc0d3450 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 >> v2:172.21.15.135:3300/0 conn(0x7f65dc15f290 msgr2=0x7f65dc0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 --2- 172.21.15.53:0/4212985280 >> v2:172.21.15.135:3300/0 conn(0x7f65dc15f290 0x7f65dc0d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 >> v1:172.21.15.135:6789/0 conn(0x7f65dc0702e0 legacy=0x7f65dc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc0d19d0 con 0x7f65dc0d3450 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f65d40019f0 con 0x7f65dc0d3450 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 >> v1:172.21.15.53:6789/0 conn(0x7f65dc0d3450 legacy=0x7f65dc06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65c80039b0 0x7f65c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c8004800 0x7f65c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80051c0 0x7f65c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f65c800c040 con 0x7f65c80039b0 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65c800c1f0 con 0x7f65c8004800 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f65c800c3f0 con 0x7f65c80051c0 2024-05-10T10:33:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65da59c640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65c80039b0 0x7f65c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65dad9d640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80051c0 0x7f65c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65d3fff640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c8004800 0x7f65c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65da59c640 1 -- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80051c0 msgr2=0x7f65c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65da59c640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80051c0 0x7f65c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65da59c640 1 -- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c8004800 msgr2=0x7f65c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65da59c640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c8004800 0x7f65c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65da59c640 1 -- 172.21.15.53:0/4212985280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c8003520 con 0x7f65c80039b0 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.705+0000 7f65dad9d640 1 -- 172.21.15.53:0/4212985280 reap_dead start 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.706+0000 7f65da59c640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65c80039b0 0x7f65c8003dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009ac0 tx=0x7f65c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d5a5b3258dc9e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.706+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc036070 con 0x7f65c80039b0 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.706+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65cc031400 con 0x7f65c80039b0 2024-05-10T10:33:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.706+0000 7f65d9d9b640 1 -- 172.21.15.53:0/4212985280 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc009690 con 0x7f65c80039b0 2024-05-10T10:33:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.707+0000 7f65e0baf640 1 -- 172.21.15.53:0/4212985280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65c80039b0 msgr2=0x7f65c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.707+0000 7f65e0baf640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65c80039b0 0x7f65c8003dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009ac0 tx=0x7f65c8005830 comp rx=0 tx=0).stop 2024-05-10T10:33:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.707+0000 7f65e0baf640 1 -- 172.21.15.53:0/4212985280 shutdown_connections 2024-05-10T10:33:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.707+0000 7f65e0baf640 1 --2- 172.21.15.53:0/4212985280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65c80039b0 0x7f65c8003dd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.707+0000 7f65e0baf640 1 -- 172.21.15.53:0/4212985280 >> 172.21.15.53:0/4212985280 conn(0x7f65dc09ba20 msgr2=0x7f65dc09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.707+0000 7f65e0baf640 1 -- 172.21.15.53:0/4212985280 shutdown_connections 2024-05-10T10:33:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.707+0000 7f65e0baf640 1 -- 172.21.15.53:0/4212985280 wait complete. 2024-05-10T10:33:40.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.708+0000 7f65e0baf640 1 Processor -- start 2024-05-10T10:33:40.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.708+0000 7f65e0baf640 1 -- start start 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65e0baf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80039b0 0x7f65dc152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65e0baf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65dc09d060 0x7f65dc09d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65e0baf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0b5bf0 0x7f65dc0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65e0baf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0d1f90 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65e0baf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0d2300 con 0x7f65dc09d060 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65e0baf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f65dc15f670 con 0x7f65c80039b0 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65dad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0b5bf0 0x7f65dc0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65dad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0b5bf0 0x7f65dc0b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53256/0 (socket says 172.21.15.53:53256) 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65dad9d640 1 -- 172.21.15.53:0/3405208411 learned_addr learned my addr 172.21.15.53:0/3405208411 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65d3fff640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65dc09d060 0x7f65dc09d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.709+0000 7f65da59c640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80039b0 0x7f65dc152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65dad9d640 1 -- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80039b0 msgr2=0x7f65dc152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65dad9d640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80039b0 0x7f65dc152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65dad9d640 1 -- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65dc09d060 msgr2=0x7f65dc09d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65dad9d640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65dc09d060 0x7f65dc09d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65dad9d640 1 -- 172.21.15.53:0/3405208411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc09f1c0 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65da59c640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80039b0 0x7f65dc152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65dad9d640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0b5bf0 0x7f65dc0b6070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f65d4000f80 tx=0x7f65d4005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65d2ffd640 1 -- 172.21.15.53:0/3405208411 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d400eaa0 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65d2ffd640 1 -- 172.21.15.53:0/3405208411 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d4018070 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc09dbd0 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65d2ffd640 1 -- 172.21.15.53:0/3405208411 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d400ee60 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.710+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc09e190 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.711+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc0693f0 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.712+0000 7f65d2ffd640 1 -- 172.21.15.53:0/3405208411 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f65d401d460 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.712+0000 7f65d2ffd640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f65bc076300 0x7f65bc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.712+0000 7f65d2ffd640 1 -- 172.21.15.53:0/3405208411 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f65d4097b60 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.712+0000 7f65da59c640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f65bc076300 0x7f65bc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.713+0000 7f65da59c640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f65bc076300 0x7f65bc0787c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:40.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.716+0000 7f65d2ffd640 1 -- 172.21.15.53:0/3405208411 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f65d4061180 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.944+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65dc0d2c00 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.945+0000 7f65d2ffd640 1 -- 172.21.15.53:0/3405208411 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5177 (secure 0 0 0) 0x7f65d4069170 con 0x7f65dc0b5bf0 2024-05-10T10:33:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:40.947 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:38.904372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:33:40.947 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 7 2024-05-10T10:33:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f65bc076300 msgr2=0x7f65bc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f65bc076300 0x7f65bc0787c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc008030 comp rx=0 tx=0).stop 2024-05-10T10:33:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0b5bf0 msgr2=0x7f65dc0b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0b5bf0 0x7f65dc0b6070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f65d4000f80 tx=0x7f65d4005f70 comp rx=0 tx=0).stop 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 shutdown_connections 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f65bc076300 0x7f65bc0787c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc0b5bf0 0x7f65dc0b6070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.946+0000 7f65e0baf640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65dc09d060 0x7f65dc09d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.947+0000 7f65e0baf640 1 --2- 172.21.15.53:0/3405208411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65c80039b0 0x7f65dc152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.947+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 >> 172.21.15.53:0/3405208411 conn(0x7f65dc09ba20 msgr2=0x7f65dc161900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.947+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 shutdown_connections 2024-05-10T10:33:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:40.947+0000 7f65e0baf640 1 -- 172.21.15.53:0/3405208411 wait complete. 2024-05-10T10:33:40.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:41.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.061+0000 7f4163f19640 1 Processor -- start 2024-05-10T10:33:41.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.062+0000 7f4163f19640 1 -- start start 2024-05-10T10:33:41.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.062+0000 7f4163f19640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f415c15be70 0x7f415c0b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.062+0000 7f4163f19640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415c0b5cc0 con 0x7f415c0aed60 2024-05-10T10:33:41.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.062+0000 7f4163f19640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415c15c250 con 0x7f415c071360 2024-05-10T10:33:41.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.062+0000 7f4163f19640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f415c06fb70 con 0x7f415c15be70 2024-05-10T10:33:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.062+0000 7f4161cb5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f415c15be70 0x7f415c0b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.063+0000 7f41624b6640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f415c0aed60 0x7f415c073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46864/0 (socket says 172.21.15.53:46864) 2024-05-10T10:33:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.063+0000 7f41624b6640 1 -- 172.21.15.53:0/3952426153 learned_addr learned my addr 172.21.15.53:0/3952426153 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.063+0000 7f4161cb5640 1 -- 172.21.15.53:0/3952426153 >> v1:172.21.15.135:6789/0 conn(0x7f415c071360 legacy=0x7f415c070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.063+0000 7f4161cb5640 1 -- 172.21.15.53:0/3952426153 >> v1:172.21.15.53:6789/0 conn(0x7f415c0aed60 legacy=0x7f415c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.063+0000 7f4160cb3640 1 -- 172.21.15.53:0/3952426153 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3070630368 0 0) 0x7f415c0b5cc0 con 0x7f415c0aed60 2024-05-10T10:33:41.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.063+0000 7f4161cb5640 1 -- 172.21.15.53:0/3952426153 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c0b59e0 con 0x7f415c15be70 2024-05-10T10:33:41.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.063+0000 7f4161cb5640 1 --2- 172.21.15.53:0/3952426153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c15be70 0x7f415c0b69f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41540097a0 tx=0x7f415402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a344d516eaf2d0aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4160cb3640 1 -- 172.21.15.53:0/3952426153 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154037070 con 0x7f415c15be70 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4160cb3640 1 -- 172.21.15.53:0/3952426153 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4154032440 con 0x7f415c15be70 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4160cb3640 1 -- 172.21.15.53:0/3952426153 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154032810 con 0x7f415c15be70 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4163f19640 1 -- 172.21.15.53:0/3952426153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c15be70 msgr2=0x7f415c0b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4163f19640 1 --2- 172.21.15.53:0/3952426153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c15be70 0x7f415c0b69f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41540097a0 tx=0x7f415402a230 comp rx=0 tx=0).stop 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4163f19640 1 -- 172.21.15.53:0/3952426153 shutdown_connections 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4163f19640 1 --2- 172.21.15.53:0/3952426153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c15be70 0x7f415c0b69f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4163f19640 1 -- 172.21.15.53:0/3952426153 >> 172.21.15.53:0/3952426153 conn(0x7f415c0d1d70 msgr2=0x7f415c0d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4163f19640 1 -- 172.21.15.53:0/3952426153 shutdown_connections 2024-05-10T10:33:41.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.064+0000 7f4163f19640 1 -- 172.21.15.53:0/3952426153 wait complete. 2024-05-10T10:33:41.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.065+0000 7f4163f19640 1 Processor -- start 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4163f19640 1 -- start start 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4163f19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c071360 0x7f415c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4163f19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f415c0aed60 0x7f415c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4163f19640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f415c15be70 0x7f415c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4163f19640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f415c070080 con 0x7f415c071360 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4163f19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f415c0703f0 con 0x7f415c0aed60 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4163f19640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f415c0b5690 con 0x7f415c15be70 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4161cb5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c071360 0x7f415c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4161cb5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c071360 0x7f415c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53272/0 (socket says 172.21.15.53:53272) 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f4161cb5640 1 -- 172.21.15.53:0/978309505 learned_addr learned my addr 172.21.15.53:0/978309505 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.066+0000 7f41614b4640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f415c0aed60 0x7f415c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4161cb5640 1 -- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f415c15be70 msgr2=0x7f415c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4161cb5640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f415c15be70 0x7f415c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4161cb5640 1 -- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f415c0aed60 msgr2=0x7f415c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4161cb5640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f415c0aed60 0x7f415c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4161cb5640 1 -- 172.21.15.53:0/978309505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c15c9b0 con 0x7f415c071360 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f41614b4640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f415c0aed60 0x7f415c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4161cb5640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c071360 0x7f415c0cd740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f41540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f415a7fc640 1 -- 172.21.15.53:0/978309505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154037040 con 0x7f415c071360 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f415a7fc640 1 -- 172.21.15.53:0/978309505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4154042070 con 0x7f415c071360 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c15cc30 con 0x7f415c071360 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f415a7fc640 1 -- 172.21.15.53:0/978309505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154030a70 con 0x7f415c071360 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.067+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c15d1f0 con 0x7f415c071360 2024-05-10T10:33:41.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.068+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415c0693f0 con 0x7f415c071360 2024-05-10T10:33:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.069+0000 7f415a7fc640 1 -- 172.21.15.53:0/978309505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4154030c10 con 0x7f415c071360 2024-05-10T10:33:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.069+0000 7f415a7fc640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4148076300 0x7f41480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.069+0000 7f415a7fc640 1 -- 172.21.15.53:0/978309505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f41540b8370 con 0x7f415c071360 2024-05-10T10:33:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.070+0000 7f41614b4640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4148076300 0x7f41480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.070+0000 7f41614b4640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4148076300 0x7f41480787c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f415c160770 tx=0x7f4150009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.073+0000 7f415a7fc640 1 -- 172.21.15.53:0/978309505 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f415c15c9b0 con 0x7f415c071360 2024-05-10T10:33:41.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.269+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f415c0b0880 con 0x7f415c071360 2024-05-10T10:33:41.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.271+0000 7f415a7fc640 1 -- 172.21.15.53:0/978309505 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f4154081990 con 0x7f415c071360 2024-05-10T10:33:41.273 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:41.273 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4148076300 msgr2=0x7f41480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4148076300 0x7f41480787c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f415c160770 tx=0x7f4150009040 comp rx=0 tx=0).stop 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c071360 msgr2=0x7f415c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c071360 0x7f415c0cd740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f41540046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 shutdown_connections 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4148076300 0x7f41480787c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f415c15be70 0x7f415c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f415c0aed60 0x7f415c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 --2- 172.21.15.53:0/978309505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f415c071360 0x7f415c0cd740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 >> 172.21.15.53:0/978309505 conn(0x7f415c0d1d70 msgr2=0x7f415c0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 shutdown_connections 2024-05-10T10:33:41.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.273+0000 7f4163f19640 1 -- 172.21.15.53:0/978309505 wait complete. 2024-05-10T10:33:41.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:33:33.864840+0000', 'modified': '2024-05-10T10:33:38.904372+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6836/4124737039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 4124737039}, {'type': 'v1', 'addr': '172.21.15.135:6836', 'nonce': 4124737039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4278, 'qdb_cluster': [4278]} 2024-05-10T10:33:41.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:33:41.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T10:33:41.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.796+0000 7f5f794b4640 1 Processor -- start 2024-05-10T10:33:41.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.796+0000 7f5f794b4640 1 -- start start 2024-05-10T10:33:41.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f794b4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5f7415f790 0x7f5f740cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f794b4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f740d30b0 con 0x7f5f7415f3b0 2024-05-10T10:33:41.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f794b4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f740d3390 con 0x7f5f740d5d30 2024-05-10T10:33:41.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f794b4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5f740c8c80 con 0x7f5f7415f790 2024-05-10T10:33:41.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f72ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5f7415f3b0 0x7f5f740d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46870/0 (socket says 172.21.15.53:46870) 2024-05-10T10:33:41.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f72ffd640 1 -- 172.21.15.53:0/1274021968 learned_addr learned my addr 172.21.15.53:0/1274021968 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:41.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1663562327 0 0) 0x7f5f740d30b0 con 0x7f5f7415f3b0 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f60003520 con 0x7f5f7415f3b0 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/1274021968 >> v2:172.21.15.135:3300/0 conn(0x7f5f7415f790 0x7f5f740cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2580215379 0 0) 0x7f5f60003520 con 0x7f5f7415f3b0 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 >> v2:172.21.15.135:3300/0 conn(0x7f5f7415f790 msgr2=0x7f5f740cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 --2- 172.21.15.53:0/1274021968 >> v2:172.21.15.135:3300/0 conn(0x7f5f7415f790 0x7f5f740cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 >> v1:172.21.15.135:6789/0 conn(0x7f5f740d5d30 legacy=0x7f5f740d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f740c8f60 con 0x7f5f7415f3b0 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f5f64001670 con 0x7f5f7415f3b0 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 >> v1:172.21.15.53:6789/0 conn(0x7f5f7415f3b0 legacy=0x7f5f740d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f600039b0 0x7f5f60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.797+0000 7f5f727fc640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f600051c0 0x7f5f6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f740c8c80 con 0x7f5f60004800 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6000c060 con 0x7f5f600051c0 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6000c260 con 0x7f5f600039b0 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f600039b0 0x7f5f60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f737fe640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f600051c0 0x7f5f6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f6a7fc640 1 -- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f600039b0 msgr2=0x7f5f60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f600039b0 0x7f5f60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f6a7fc640 1 -- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f600051c0 msgr2=0x7f5f6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f600051c0 0x7f5f6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f6a7fc640 1 -- 172.21.15.53:0/1274021968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f60003520 con 0x7f5f60004800 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f737fe640 1 -- 172.21.15.53:0/1274021968 reap_dead start 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.798+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f600039b0 0x7f5f60003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f60004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009a90 tx=0x7f5f60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74ed821d567afa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c036070 con 0x7f5f60004800 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5c031400 con 0x7f5f60004800 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f727fc640 1 -- 172.21.15.53:0/1274021968 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c009690 con 0x7f5f60004800 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f794b4640 1 -- 172.21.15.53:0/1274021968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 msgr2=0x7f5f60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f794b4640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f60004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009a90 tx=0x7f5f60005830 comp rx=0 tx=0).stop 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f794b4640 1 -- 172.21.15.53:0/1274021968 shutdown_connections 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f794b4640 1 --2- 172.21.15.53:0/1274021968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f60004c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f794b4640 1 -- 172.21.15.53:0/1274021968 >> 172.21.15.53:0/1274021968 conn(0x7f5f740d5130 msgr2=0x7f5f740d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f794b4640 1 -- 172.21.15.53:0/1274021968 shutdown_connections 2024-05-10T10:33:41.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.799+0000 7f5f794b4640 1 -- 172.21.15.53:0/1274021968 wait complete. 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.801+0000 7f5f794b4640 1 Processor -- start 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.801+0000 7f5f794b4640 1 -- start start 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.801+0000 7f5f794b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f74160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.801+0000 7f5f794b4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f741622e0 0x7f5f740cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.801+0000 7f5f794b4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f740cc740 0x7f5f740ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f794b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f740d3870 con 0x7f5f60004800 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f794b4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f740d36f0 con 0x7f5f740cc740 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f794b4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5f740d39f0 con 0x7f5f741622e0 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f74160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f74160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53292/0 (socket says 172.21.15.53:53292) 2024-05-10T10:33:41.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 -- 172.21.15.53:0/221147166 learned_addr learned my addr 172.21.15.53:0/221147166 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f737fe640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f740cc740 0x7f5f740ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f741622e0 0x7f5f740cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 -- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f741622e0 msgr2=0x7f5f740cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f741622e0 0x7f5f740cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 -- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f740cc740 msgr2=0x7f5f740ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f740cc740 0x7f5f740ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 -- 172.21.15.53:0/221147166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f74161240 con 0x7f5f60004800 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f74160bd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400e7f0 tx=0x7f5f6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f6bfff640 1 -- 172.21.15.53:0/221147166 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400c870 con 0x7f5f60004800 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f6bfff640 1 -- 172.21.15.53:0/221147166 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f64018070 con 0x7f5f60004800 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f74161520 con 0x7f5f60004800 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f6bfff640 1 -- 172.21.15.53:0/221147166 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400cc10 con 0x7f5f60004800 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.802+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f74161ae0 con 0x7f5f60004800 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.804+0000 7f5f6bfff640 1 -- 172.21.15.53:0/221147166 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5f64013420 con 0x7f5f60004800 2024-05-10T10:33:41.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.804+0000 7f5f6bfff640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f54076300 0x7f5f540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.805+0000 7f5f6bfff640 1 -- 172.21.15.53:0/221147166 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f64097470 con 0x7f5f60004800 2024-05-10T10:33:41.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.805+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f54076300 0x7f5f540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.805+0000 7f5f7c1ce740 1 Processor -- start 2024-05-10T10:33:41.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.805+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f54076300 0x7f5f540787c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c033000 tx=0x7f5f5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.805+0000 7f5f7c1ce740 1 -- start start 2024-05-10T10:33:41.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288c43760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c41330 0x559288c43ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c13bb0 0x559288c173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559288b7d8d0 con 0x559288c40f50 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559288b36c10 con 0x559288c41330 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x559288b79890 con 0x559288c13bb0 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288c43760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288c43760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53304/0 (socket says 172.21.15.53:53304) 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 -- 172.21.15.53:0/945816214 learned_addr learned my addr 172.21.15.53:0/945816214 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f737fe640 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c13bb0 0x559288c173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c41330 0x559288c43ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 -- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c13bb0 msgr2=0x559288c173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c13bb0 0x559288c173c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 -- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c41330 msgr2=0x559288c43ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c41330 0x559288c43ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.806+0000 7f5f72ffd640 1 -- 172.21.15.53:0/945816214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559288b3e5d0 con 0x559288c40f50 2024-05-10T10:33:41.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.807+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288c43760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f64001860 tx=0x7f5f6405faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841520987dcbc89c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.807+0000 7f5f68ff9640 1 -- 172.21.15.53:0/945816214 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64060380 con 0x559288c40f50 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.807+0000 7f5f68ff9640 1 -- 172.21.15.53:0/945816214 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f64064730 con 0x559288c40f50 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.807+0000 7f5f68ff9640 1 -- 172.21.15.53:0/945816214 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64069150 con 0x559288c40f50 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.807+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/945816214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 msgr2=0x559288c43760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.807+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288c43760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f64001860 tx=0x7f5f6405faf0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/945816214 shutdown_connections 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c13bb0 0x559288c173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c41330 0x559288c43ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/945816214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288c43760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/945816214 >> 172.21.15.53:0/945816214 conn(0x559288c15e20 msgr2=0x559288c5e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/945816214 shutdown_connections 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/945816214 wait complete. 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 -- start start 2024-05-10T10:33:41.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.808+0000 7f5f7c1ce740 1 Processor -- start 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 0x559288c36e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c41330 0x559288c5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c61ed0 0x559288c62350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559288c18fd0 con 0x559288c41330 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559288b3a760 con 0x559288c40f50 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x559288c192b0 con 0x559288c61ed0 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f7c1ce740 10 client.0 _ll_register_callbacks cb 0x559288b73cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c41330 0x559288c5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c41330 0x559288c5ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53316/0 (socket says 172.21.15.53:53316) 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 -- 172.21.15.53:0/787874332 learned_addr learned my addr 172.21.15.53:0/787874332 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f737fe640 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c61ed0 0x559288c62350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 0x559288c36e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 -- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c61ed0 msgr2=0x559288c62350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c61ed0 0x559288c62350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 -- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 msgr2=0x559288c36e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 0x559288c36e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.809+0000 7f5f6a7fc640 1 -- 172.21.15.53:0/787874332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928888f2d0 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c41330 0x559288c5ae10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009bb0 tx=0x7f5f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0dd4296591f47e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f51ffb640 1 -- 172.21.15.53:0/787874332 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c04d060 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f51ffb640 1 -- 172.21.15.53:0/787874332 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f5c03c670 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f64064730 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f51ffb640 1 -- 172.21.15.53:0/787874332 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f5f5c036040 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 10 client.4395 fetch_fsmap learned FSMap version 7 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5592889596b0 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f51ffb640 1 -- 172.21.15.53:0/787874332 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f64064730 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f51ffb640 1 -- 172.21.15.53:0/787874332 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f5f5c056080 con 0x559288c41330 2024-05-10T10:33:41.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 10 client.4395 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 20 client.4395 populate_metadata read hostname 'smithi053' 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c60d80 0x559288c61160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 4 client.4395 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 --> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559288b01760 con 0x559288c60d80 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.810+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c60d80 0x559288c61160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.811+0000 7f5f51ffb640 10 client.4395 ms_handle_connect on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.811+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c60d80 0x559288c61160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214499fd38a19f8e server_cookie=db1e24f543a3c0b7 in_seq=0 out_seq=0 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.811+0000 7f5f51ffb640 1 -- 172.21.15.53:0/787874332 <== mds.0 v2:172.21.15.135:6834/4124737039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559288b01760 con 0x559288c60d80 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.811+0000 7f5f51ffb640 10 client.4395 handle_command_reply: tid=0 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 client.4395 shutdown 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c41330 msgr2=0x559288c5ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c41330 0x559288c5ae10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009bb0 tx=0x7f5f5c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 shutdown_connections 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c60d80 0x559288c61160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c61ed0 0x559288c62350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f51ffb640 0 client.4395 ms_handle_reset on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c41330 0x559288c5ae10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/787874332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 0x559288c36e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 >> 172.21.15.53:0/787874332 conn(0x559288c13bb0 msgr2=0x559288c14860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 shutdown_connections 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/787874332 wait complete. 2024-05-10T10:33:41.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.812+0000 7f5f7c1ce740 20 client.4395 trim_cache size 0 max 16384 2024-05-10T10:33:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.831+0000 7f5f7c1ce740 1 Processor -- start 2024-05-10T10:33:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.831+0000 7f5f7c1ce740 1 -- start start 2024-05-10T10:33:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 0x559288c14170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 0x559288c476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c60d80 0x559288c173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74203410 con 0x559288c60d80 2024-05-10T10:33:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74202430 con 0x559288c40f50 2024-05-10T10:33:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5f74201790 con 0x559288c41330 2024-05-10T10:33:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c60d80 0x559288c173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c60d80 0x559288c173c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53330/0 (socket says 172.21.15.53:53330) 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 -- 172.21.15.53:0/2080580812 learned_addr learned my addr 172.21.15.53:0/2080580812 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 -- 172.21.15.53:0/2080580812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 msgr2=0x559288c476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 --2- 172.21.15.53:0/2080580812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 0x559288c476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 -- 172.21.15.53:0/2080580812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 msgr2=0x559288c14170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 --2- 172.21.15.53:0/2080580812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 0x559288c14170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 -- 172.21.15.53:0/2080580812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559288c15960 con 0x559288c60d80 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f737fe640 1 --2- 172.21.15.53:0/2080580812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c60d80 0x559288c173c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f740d36a0 tx=0x7f5f6c001ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d839daf81f22a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f53fff640 1 -- 172.21.15.53:0/2080580812 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003c60 con 0x559288c60d80 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f53fff640 1 -- 172.21.15.53:0/2080580812 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6c003e00 con 0x559288c60d80 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.832+0000 7f5f53fff640 1 -- 172.21.15.53:0/2080580812 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c006b50 con 0x559288c60d80 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/2080580812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c60d80 msgr2=0x559288c173c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/2080580812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c60d80 0x559288c173c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f740d36a0 tx=0x7f5f6c001ce0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/2080580812 shutdown_connections 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/2080580812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c60d80 0x559288c173c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/2080580812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 0x559288c476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/2080580812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c40f50 0x559288c14170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/2080580812 >> 172.21.15.53:0/2080580812 conn(0x559288c13bb0 msgr2=0x559288c36c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/2080580812 shutdown_connections 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/2080580812 wait complete. 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.833+0000 7f5f7c1ce740 1 -- start start 2024-05-10T10:33:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 1 Processor -- start 2024-05-10T10:33:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288cc2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 0x559288c5aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c60d80 0x559288c12a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559288c602b0 con 0x559288c40f50 2024-05-10T10:33:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74201de0 con 0x559288c60d80 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x559288c5b350 con 0x559288c41330 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f7c1ce740 10 client.0 _ll_register_callbacks cb 0x7f5f5c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288cc2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f72ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288cc2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53340/0 (socket says 172.21.15.53:53340) 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.834+0000 7f5f72ffd640 1 -- 172.21.15.53:0/484449435 learned_addr learned my addr 172.21.15.53:0/484449435 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f737fe640 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c60d80 0x559288c12a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 0x559288c5aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f72ffd640 1 -- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 msgr2=0x559288c5aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 0x559288c5aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f72ffd640 1 -- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c60d80 msgr2=0x559288c12a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c60d80 0x559288c12a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f72ffd640 1 -- 172.21.15.53:0/484449435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559288c5a270 con 0x559288c40f50 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f72ffd640 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288cc2c30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f64060030 tx=0x7f5f640017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564656cc6ec11bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f51ffb640 1 -- 172.21.15.53:0/484449435 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64060380 con 0x559288c40f50 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c003e00 con 0x559288c40f50 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f51ffb640 1 -- 172.21.15.53:0/484449435 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f6401de00 con 0x559288c40f50 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f51ffb640 1 -- 172.21.15.53:0/484449435 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f5f64064730 con 0x559288c40f50 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 10 client.4401 fetch_fsmap learned FSMap version 7 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x559288c82a80 con 0x559288c40f50 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f51ffb640 1 -- 172.21.15.53:0/484449435 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6c003e00 con 0x559288c40f50 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f51ffb640 1 -- 172.21.15.53:0/484449435 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f5f6405c610 con 0x559288c40f50 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 10 client.4401 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 20 client.4401 populate_metadata read hostname 'smithi053' 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c61ed0 0x559288c46280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 4 client.4401 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:33:41.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.835+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 --> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] -- command(tid 0: {"prefix": "status"}) -- 0x559288b8b4e0 con 0x559288c61ed0 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c61ed0 0x559288c46280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f6a7fc640 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c61ed0 0x559288c46280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba870d1e86c332d6 server_cookie=2082f1046730e5c in_seq=0 out_seq=0 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f51ffb640 10 client.4401 ms_handle_connect on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f51ffb640 1 -- 172.21.15.53:0/484449435 <== mds.0 v2:172.21.15.135:6834/4124737039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x559288b8b4e0 con 0x559288c61ed0 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f51ffb640 10 client.4401 handle_command_reply: tid=0 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 client.4401 shutdown 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 msgr2=0x559288cc2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288cc2c30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f64060030 tx=0x7f5f640017b0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 shutdown_connections 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x559288c61ed0 0x559288c46280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559288c60d80 0x559288c12a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f51ffb640 0 client.4401 ms_handle_reset on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559288c41330 0x559288c5aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 --2- 172.21.15.53:0/484449435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559288c40f50 0x559288cc2c30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 >> 172.21.15.53:0/484449435 conn(0x559288c13bb0 msgr2=0x559288c61160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 shutdown_connections 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.836+0000 7f5f7c1ce740 1 -- 172.21.15.53:0/484449435 wait complete. 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f7c1ce740 20 client.4401 trim_cache size 0 max 16384 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 4278, 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:33:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:33:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:33:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 7.3369922230000002, 2024-05-10T10:33:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 7, 2024-05-10T10:33:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 24, 2024-05-10T10:33:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 23, 2024-05-10T10:33:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 13.381703004 2024-05-10T10:33:41.846 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f54076300 msgr2=0x7f5f540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f54076300 0x7f5f540787c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c033000 tx=0x7f5f5c008030 comp rx=0 tx=0).stop 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 msgr2=0x7f5f74160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f74160bd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400e7f0 tx=0x7f5f6400ecb0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 shutdown_connections 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f54076300 0x7f5f540787c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f740cc740 0x7f5f740ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f741622e0 0x7f5f740cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 --2- 172.21.15.53:0/221147166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f60004800 0x7f5f74160bd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 >> 172.21.15.53:0/221147166 conn(0x7f5f740d5130 msgr2=0x7f5f740cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 shutdown_connections 2024-05-10T10:33:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.837+0000 7f5f794b4640 1 -- 172.21.15.53:0/221147166 wait complete. 2024-05-10T10:33:41.849 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T10:33:41.849 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:41.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:33:41.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.951+0000 7f6d06112640 1 Processor -- start 2024-05-10T10:33:41.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.951+0000 7f6d06112640 1 -- start start 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6d06112640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6d00162750 0x7f6d000d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6d06112640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6d00162b30 0x7f6d000b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6d06112640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d000b45f0 con 0x7f6d000d5cb0 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6d06112640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6d000b4f20 con 0x7f6d00162750 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6d06112640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6d000d0fe0 con 0x7f6d00162b30 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6cff7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6d00162750 0x7f6d000d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6cff7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6d00162750 0x7f6d000d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53354/0 (socket says 172.21.15.53:53354) 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6cff7fe640 1 -- 172.21.15.53:0/479168522 learned_addr learned my addr 172.21.15.53:0/479168522 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.952+0000 7f6cfffff640 1 --2- 172.21.15.53:0/479168522 >> v2:172.21.15.135:3300/0 conn(0x7f6d00162b30 0x7f6d000b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.953+0000 7f6cff7fe640 1 -- 172.21.15.53:0/479168522 >> v2:172.21.15.135:3300/0 conn(0x7f6d00162b30 msgr2=0x7f6d000b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.953+0000 7f6cff7fe640 1 --2- 172.21.15.53:0/479168522 >> v2:172.21.15.135:3300/0 conn(0x7f6d00162b30 0x7f6d000b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.953+0000 7f6cff7fe640 1 -- 172.21.15.53:0/479168522 >> v1:172.21.15.135:6789/0 conn(0x7f6d000d5cb0 legacy=0x7f6d000d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.953+0000 7f6cff7fe640 1 -- 172.21.15.53:0/479168522 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d000b48d0 con 0x7f6d00162750 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.953+0000 7f6cff7fe640 1 --2- 172.21.15.53:0/479168522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000d0670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d000d32c0 tx=0x7f6cf002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ea4c29e8b4ec26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6cfe7fc640 1 -- 172.21.15.53:0/479168522 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0037070 con 0x7f6d00162750 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6cfe7fc640 1 -- 172.21.15.53:0/479168522 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf0032440 con 0x7f6d00162750 2024-05-10T10:33:41.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6cfe7fc640 1 -- 172.21.15.53:0/479168522 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0032810 con 0x7f6d00162750 2024-05-10T10:33:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 -- 172.21.15.53:0/479168522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 msgr2=0x7f6d000d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 --2- 172.21.15.53:0/479168522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000d0670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d000d32c0 tx=0x7f6cf002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 -- 172.21.15.53:0/479168522 shutdown_connections 2024-05-10T10:33:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 --2- 172.21.15.53:0/479168522 >> v2:172.21.15.135:3300/0 conn(0x7f6d00162b30 0x7f6d000b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 --2- 172.21.15.53:0/479168522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000d0670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 -- 172.21.15.53:0/479168522 >> 172.21.15.53:0/479168522 conn(0x7f6d0009e3b0 msgr2=0x7f6d000d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 -- 172.21.15.53:0/479168522 shutdown_connections 2024-05-10T10:33:41.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.954+0000 7f6d06112640 1 -- 172.21.15.53:0/479168522 wait complete. 2024-05-10T10:33:41.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.955+0000 7f6d06112640 1 Processor -- start 2024-05-10T10:33:41.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6d06112640 1 -- start start 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6d06112640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6d06112640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d00162b30 0x7f6d00160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6d06112640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d000cc120 0x7f6d000cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6d06112640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d000d0d30 con 0x7f6d00162750 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6d06112640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d000d0bb0 con 0x7f6d000cc120 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6d06112640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6d000d3d40 con 0x7f6d00162b30 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6cff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6cff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000b98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53366/0 (socket says 172.21.15.53:53366) 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.956+0000 7f6cff7fe640 1 -- 172.21.15.53:0/3273213492 learned_addr learned my addr 172.21.15.53:0/3273213492 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cfffff640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d000cc120 0x7f6d000cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cfeffd640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d00162b30 0x7f6d00160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cff7fe640 1 -- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d00162b30 msgr2=0x7f6d00160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cff7fe640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d00162b30 0x7f6d00160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cff7fe640 1 -- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d000cc120 msgr2=0x7f6d000cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cff7fe640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d000cc120 0x7f6d000cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cff7fe640 1 -- 172.21.15.53:0/3273213492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d000b9df0 con 0x7f6d00162750 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cfffff640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d000cc120 0x7f6d000cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6cff7fe640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000b98b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6ce7fff640 1 -- 172.21.15.53:0/3273213492 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0041020 con 0x7f6d00162750 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6ce7fff640 1 -- 172.21.15.53:0/3273213492 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf00306c0 con 0x7f6d00162750 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d000cc6e0 con 0x7f6d00162750 2024-05-10T10:33:41.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6ce7fff640 1 -- 172.21.15.53:0/3273213492 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0037040 con 0x7f6d00162750 2024-05-10T10:33:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.957+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d000ccca0 con 0x7f6d00162750 2024-05-10T10:33:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.958+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0006a220 con 0x7f6d00162750 2024-05-10T10:33:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.959+0000 7f6ce7fff640 1 -- 172.21.15.53:0/3273213492 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6cf0044020 con 0x7f6d00162750 2024-05-10T10:33:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.959+0000 7f6ce7fff640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cec076300 0x7f6cec0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:41.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.959+0000 7f6ce7fff640 1 -- 172.21.15.53:0/3273213492 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6cf00b80a0 con 0x7f6d00162750 2024-05-10T10:33:41.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.959+0000 7f6cfeffd640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cec076300 0x7f6cec0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:41.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.960+0000 7f6cfeffd640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cec076300 0x7f6cec0787c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d000d2350 tx=0x7f6cf4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:41.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:41.964+0000 7f6ce7fff640 1 -- 172.21.15.53:0/3273213492 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6cf004b030 con 0x7f6d00162750 2024-05-10T10:33:42.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.191+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d000cf480 con 0x7f6d00162750 2024-05-10T10:33:42.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.192+0000 7f6ce7fff640 1 -- 172.21.15.53:0/3273213492 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f6cf0081750 con 0x7f6d00162750 2024-05-10T10:33:42.193 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T10:33:42.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cec076300 msgr2=0x7f6cec0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:42.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cec076300 0x7f6cec0787c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d000d2350 tx=0x7f6cf4009040 comp rx=0 tx=0).stop 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 msgr2=0x7f6d000b98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000b98b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 shutdown_connections 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cec076300 0x7f6cec0787c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d000cc120 0x7f6d000cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d00162b30 0x7f6d00160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 --2- 172.21.15.53:0/3273213492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d00162750 0x7f6d000b98b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 >> 172.21.15.53:0/3273213492 conn(0x7f6d0009e3b0 msgr2=0x7f6d000d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 shutdown_connections 2024-05-10T10:33:42.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42.194+0000 7f6d06112640 1 -- 172.21.15.53:0/3273213492 wait complete. 2024-05-10T10:33:42.206 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:33:42.206 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:33:42.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:33:42.207 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-10T10:33:42.207 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:42.207 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:33:42.207 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T10:33:42.207 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:33:42.207 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T10:33:42.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:42.207 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:33:42.224 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:33:42.224 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:33:42.224 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:33:42.224 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:42.224 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:33:42.224 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:42.224 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42746sec preferred_lft 42746sec 2024-05-10T10:33:42.225 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:33:42.226 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:42.226 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:42.226 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:42.226 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:33:42.226 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T10:33:42.227 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:42.227 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:42.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2024-05-10T10:33:42.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2024-05-10T10:33:42.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2024-05-10T10:33:42.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T10:33:42.227 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:42.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:42.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:42.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:42.413 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:33:42.438 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:33:42.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:42.440 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:33:42.507 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:33:42.508 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:33:42.508 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:33:42.508 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:33:42.508 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:33:42.508 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:42.508 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:42.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:33:42.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:33:42.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:33:42.509 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:42.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:42.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:42.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:42.839 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:33:42.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:42.856 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-05-10T10:33:42.912 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:42.912 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:42.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:33:42.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-10T10:33:42.912 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:43.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:43.033 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-10T10:33:43.033 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:43.033 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:43.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-10T10:33:43.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T10:33:43.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-10T10:33:43.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-10T10:33:43.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-10T10:33:43.034 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:43.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:43.168 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:43.168 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:43.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:33:43.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T10:33:43.168 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:43.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:43.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:43.247 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-10T10:33:43.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:33:43.248 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:43.264 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T10:33:43.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:33:43.264 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:43.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2024-05-10T10:33:43.387 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T10:33:43.444 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2024-05-10T10:33:43.444 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/bpf 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2024-05-10T10:33:43.445 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-0 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-1 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-2 2024-05-10T10:33:43.446 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-3 2024-05-10T10:33:43.447 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2024-05-10T10:33:43.447 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:33:43.447 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:33:43.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:43.447 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2024-05-10T10:33:43.500 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T10:33:43.500 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-10T10:33:43.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2024-05-10T10:33:43.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:33:43.546 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:33:43.570 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2024-05-10T10:33:43.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:2024-05-10T10:33:43.640+0000 7f908f823480 -1 init, newargv = 0x564f436c7fa0 newargc=15 2024-05-10T10:33:43.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:ceph-fuse[41888]: starting ceph client 2024-05-10T10:33:43.648 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2024-05-10T10:33:43.648 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2024-05-10T10:33:43.648 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/bpf 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2024-05-10T10:33:43.649 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-0 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-1 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-2 2024-05-10T10:33:43.650 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-3 2024-05-10T10:33:43.651 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2024-05-10T10:33:43.651 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:33:43.651 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:33:43.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:43.652 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2024-05-10T10:33:43.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:ceph-fuse[41888]: starting fuse 2024-05-10T10:33:43.708 INFO:teuthology.orchestra.run.smithi053.stdout:40 2024-05-10T10:33:43.708 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-10T10:33:43.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo stdin-killer -- python3 -c ' 2024-05-10T10:33:43.708 DEBUG:teuthology.orchestra.run.smithi053:> import glob 2024-05-10T10:33:43.708 DEBUG:teuthology.orchestra.run.smithi053:> import re 2024-05-10T10:33:43.708 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:33:43.708 DEBUG:teuthology.orchestra.run.smithi053:> import subprocess 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> def _find_admin_socket(client_name): 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> files = glob.glob(asok_path) 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> # Given a non-glob path, it better be there 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> if "*" not in asok_path: 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> assert(len(files) == 1) 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> return files[0] 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> for f in files: 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T10:33:43.709 DEBUG:teuthology.orchestra.run.smithi053:> contents = proc_f.read() 2024-05-10T10:33:43.710 DEBUG:teuthology.orchestra.run.smithi053:> if mountpoint in contents: 2024-05-10T10:33:43.710 DEBUG:teuthology.orchestra.run.smithi053:> return f 2024-05-10T10:33:43.710 DEBUG:teuthology.orchestra.run.smithi053:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T10:33:43.710 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:33:43.710 DEBUG:teuthology.orchestra.run.smithi053:> print(_find_admin_socket("client.admin")) 2024-05-10T10:33:43.710 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:33:43.830 INFO:teuthology.orchestra.run.smithi053.stdout:/var/run/ceph/ceph-client.admin.41888.asok 2024-05-10T10:33:43.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:43.840 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41888.asok 2024-05-10T10:33:43.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:43.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41888.asok status 2024-05-10T10:33:43.979 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:33:43.979 INFO:teuthology.orchestra.run.smithi053.stdout: "metadata": { 2024-05-10T10:33:43.979 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_sha1": "d39fa08867836ac070c47cb66c782fa559fd480b", 2024-05-10T10:33:43.979 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.0.0-3449-gd39fa088 (d39fa08867836ac070c47cb66c782fa559fd480b) squid (dev)", 2024-05-10T10:33:43.979 INFO:teuthology.orchestra.run.smithi053.stdout: "entity_id": "admin", 2024-05-10T10:33:43.979 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": "smithi053", 2024-05-10T10:33:43.979 INFO:teuthology.orchestra.run.smithi053.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "pid": "41888", 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "root": "/" 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_count": 0, 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_pinned_count": 0, 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 4416, 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "inst": { 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "name": { 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "client", 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "num": 4416 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2024-05-10T10:33:43.980 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1929532405 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1929532405 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "inst_str": "client.4416 192.168.0.1:0/1929532405", 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "addr_str": "192.168.0.1:0/1929532405", 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "inode_count": 1, 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_epoch": 7, 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch": 24, 2024-05-10T10:33:43.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch_barrier": 0, 2024-05-10T10:33:43.982 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklisted": false, 2024-05-10T10:33:43.982 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs" 2024-05-10T10:33:43.982 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:33:43.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:43.987 DEBUG:teuthology.orchestra.run.smithi053:> stat --file-system '--printf=%T 2024-05-10T10:33:43.987 DEBUG:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:44.004 INFO:teuthology.orchestra.run.smithi053.stdout:fuseblk 2024-05-10T10:33:44.004 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:44.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:44.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:44.071 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-10T10:33:44.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:33:44.182 INFO:teuthology.orchestra.run.smithi053.stdout:# file: . 2024-05-10T10:33:44.182 INFO:teuthology.orchestra.run.smithi053.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-10T10:33:44.182 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:44.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:44.188 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:44.189 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:33:44.205 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:33:44.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.349+0000 7f5962a93640 1 Processor -- start 2024-05-10T10:33:44.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.349+0000 7f5962a93640 1 -- start start 2024-05-10T10:33:44.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f5962a93640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 0x7f595c0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f5962a93640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f595c0d30b0 con 0x7f595c15f790 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f5962a93640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f595c0d3390 con 0x7f595c0d5d30 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f5962a93640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f595c0c8c80 con 0x7f595c15f3b0 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f595c15f790 0x7f595c0c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39634/0 (socket says 172.21.15.53:39634) 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595bfff640 1 -- 172.21.15.53:0/929948238 learned_addr learned my addr 172.21.15.53:0/929948238 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f596082f640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 0x7f595c0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1067439818 0 0) 0x7f595c0d30b0 con 0x7f595c15f790 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f594c003520 con 0x7f595c15f790 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1722782022 0 0) 0x7f595c0d3390 con 0x7f595c0d5d30 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f595c0d30b0 con 0x7f595c0d5d30 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3495103404 0 0) 0x7f594c003520 con 0x7f595c15f790 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 msgr2=0x7f595c0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 0x7f595c0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 >> v1:172.21.15.135:6789/0 conn(0x7f595c0d5d30 legacy=0x7f595c0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c0c8f60 con 0x7f595c15f790 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f5948001670 con 0x7f595c15f790 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 >> v1:172.21.15.53:6789/0 conn(0x7f595c15f790 legacy=0x7f595c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f594c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 0x7f594c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.350+0000 7f595b7fe640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f594c004d40 0x7f594c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f595c0c8c80 con 0x7f594c003b70 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f594c00ba70 con 0x7f594c004d40 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f594c00bc20 con 0x7f595c15f3b0 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f596082f640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f594c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f595bfff640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 0x7f594c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f5961030640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f594c004d40 0x7f594c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f596082f640 1 -- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 msgr2=0x7f594c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:44.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f596082f640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 0x7f594c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f596082f640 1 -- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f594c004d40 msgr2=0x7f594c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f596082f640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f594c004d40 0x7f594c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f596082f640 1 -- 172.21.15.53:0/929948238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c003520 con 0x7f594c003b70 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f595bfff640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c15f3b0 0x7f594c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.351+0000 7f5961030640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f594c004d40 0x7f594c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f596082f640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f594c003f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5950009ac0 tx=0x7f594c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b1623128888173 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950036070 con 0x7f594c003b70 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5950031400 con 0x7f594c003b70 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f595b7fe640 1 -- 172.21.15.53:0/929948238 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950009690 con 0x7f594c003b70 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f5962a93640 1 -- 172.21.15.53:0/929948238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 msgr2=0x7f594c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f5962a93640 1 --2- 172.21.15.53:0/929948238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f594c003f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5950009ac0 tx=0x7f594c005830 comp rx=0 tx=0).stop 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f5961030640 1 -- 172.21.15.53:0/929948238 reap_dead start 2024-05-10T10:33:44.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f5962a93640 1 -- 172.21.15.53:0/929948238 shutdown_connections 2024-05-10T10:33:44.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f5962a93640 1 -- 172.21.15.53:0/929948238 >> 172.21.15.53:0/929948238 conn(0x7f595c0d5130 msgr2=0x7f595c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:44.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f5962a93640 1 -- 172.21.15.53:0/929948238 shutdown_connections 2024-05-10T10:33:44.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.352+0000 7f5962a93640 1 -- 172.21.15.53:0/929948238 wait complete. 2024-05-10T10:33:44.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 Processor -- start 2024-05-10T10:33:44.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 -- start start 2024-05-10T10:33:44.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f595c158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c0cb100 0x7f595c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f595c0b9580 0x7f595c0b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f595c0d3870 con 0x7f594c003b70 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f595c0d36f0 con 0x7f595c0b9580 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.354+0000 7f5962a93640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f595c0d39f0 con 0x7f595c0cb100 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f595c158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f595c158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43156/0 (socket says 172.21.15.53:43156) 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 -- 172.21.15.53:0/902413298 learned_addr learned my addr 172.21.15.53:0/902413298 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 -- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c0cb100 msgr2=0x7f595c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c0cb100 0x7f595c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 -- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f595c0b9580 msgr2=0x7f595c0b9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f5961030640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f595c0b9580 0x7f595c0b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f595c0b9580 0x7f595c0b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 -- 172.21.15.53:0/902413298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c1589c0 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f596082f640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f595c158480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5950034040 tx=0x7f5950004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f59597fa640 1 -- 172.21.15.53:0/902413298 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950040020 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f59597fa640 1 -- 172.21.15.53:0/902413298 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5950003bc0 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c0ba2f0 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f59597fa640 1 -- 172.21.15.53:0/902413298 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950036040 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.355+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c0ba8b0 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.356+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c06a220 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.357+0000 7f59597fa640 1 -- 172.21.15.53:0/902413298 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5950043020 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.357+0000 7f59597fa640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5940076300 0x7f59400787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.357+0000 7f59597fa640 1 -- 172.21.15.53:0/902413298 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f59500b7de0 con 0x7f594c003b70 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.358+0000 7f595bfff640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5940076300 0x7f59400787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.358+0000 7f595bfff640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5940076300 0x7f59400787c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f595c0cf070 tx=0x7f5948001e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:44.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.362+0000 7f59597fa640 1 -- 172.21.15.53:0/902413298 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f595004a030 con 0x7f594c003b70 2024-05-10T10:33:44.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.565+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f595c0b7a40 con 0x7f594c003b70 2024-05-10T10:33:44.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.565+0000 7f59597fa640 1 -- 172.21.15.53:0/902413298 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f59500044e0 con 0x7f594c003b70 2024-05-10T10:33:44.567 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:33:44.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.567+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5940076300 msgr2=0x7f59400787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.567+0000 7f5962a93640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5940076300 0x7f59400787c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f595c0cf070 tx=0x7f5948001e70 comp rx=0 tx=0).stop 2024-05-10T10:33:44.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.567+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 msgr2=0x7f595c158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.567+0000 7f5962a93640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f595c158480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5950034040 tx=0x7f5950004730 comp rx=0 tx=0).stop 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 shutdown_connections 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5940076300 0x7f59400787c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f595c0b9580 0x7f595c0b9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f595c0cb100 0x7f595c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 --2- 172.21.15.53:0/902413298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f594c003b70 0x7f595c158480 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 >> 172.21.15.53:0/902413298 conn(0x7f595c0d5130 msgr2=0x7f595c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 shutdown_connections 2024-05-10T10:33:44.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.568+0000 7f5962a93640 1 -- 172.21.15.53:0/902413298 wait complete. 2024-05-10T10:33:44.581 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:44.581 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:33:44.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:33:44.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.741+0000 7f8c6b324640 1 Processor -- start 2024-05-10T10:33:44.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.741+0000 7f8c6b324640 1 -- start start 2024-05-10T10:33:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c6b324640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c640cf170 0x7f8c6415f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c6b324640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c640cf5d0 0x7f8c6415f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c6b324640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c640c74c0 con 0x7f8c6415b6d0 2024-05-10T10:33:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c6b324640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8c640cc380 con 0x7f8c640cf170 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c6b324640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640c5d10 con 0x7f8c640cf5d0 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c690c0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8c6415b6d0 0x7f8c640c6db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39646/0 (socket says 172.21.15.53:39646) 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c690c0640 1 -- 172.21.15.53:0/987698663 learned_addr learned my addr 172.21.15.53:0/987698663 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c688bf640 1 --2- 172.21.15.53:0/987698663 >> v2:172.21.15.135:3300/0 conn(0x7f8c640cf170 0x7f8c6415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c698c1640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c640cf5d0 0x7f8c6415f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123076564 0 0) 0x7f8c640c74c0 con 0x7f8c6415b6d0 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.742+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c50005730 con 0x7f8c6415b6d0 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2402848788 0 0) 0x7f8c50005730 con 0x7f8c6415b6d0 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c640cf5d0 msgr2=0x7f8c6415f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c640cf5d0 0x7f8c6415f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 >> v2:172.21.15.135:3300/0 conn(0x7f8c640cf170 msgr2=0x7f8c6415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/987698663 >> v2:172.21.15.135:3300/0 conn(0x7f8c640cf170 0x7f8c6415f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c640c5ff0 con 0x7f8c6415b6d0 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f8c60001670 con 0x7f8c6415b6d0 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c688bf640 1 --2- 172.21.15.53:0/987698663 >> v2:172.21.15.135:3300/0 conn(0x7f8c640cf170 0x7f8c6415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 >> v1:172.21.15.53:6789/0 conn(0x7f8c6415b6d0 legacy=0x7f8c640c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c50005bc0 0x7f8c50005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c500065b0 0x7f8c50006a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c50007100 0x7f8c5000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640c5d10 con 0x7f8c50007100 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5000e110 con 0x7f8c500065b0 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5000e290 con 0x7f8c50005bc0 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c698c1640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c640cf5d0 0x7f8c6415f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c698c1640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c50007100 0x7f8c5000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c690c0640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c50005bc0 0x7f8c50005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c688bf640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c500065b0 0x7f8c50006a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c698c1640 1 -- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c50005bc0 msgr2=0x7f8c50005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c698c1640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c50005bc0 0x7f8c50005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c698c1640 1 -- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c500065b0 msgr2=0x7f8c50006a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c698c1640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c500065b0 0x7f8c50006a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.743+0000 7f8c698c1640 1 -- 172.21.15.53:0/987698663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c50005730 con 0x7f8c50007100 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.744+0000 7f8c698c1640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c50007100 0x7f8c5000dbd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00e220 tx=0x7f8c50007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3b12052d4e72ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.744+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c009d40 con 0x7f8c50007100 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.744+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c035070 con 0x7f8c50007100 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.744+0000 7f8c5bfff640 1 -- 172.21.15.53:0/987698663 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c02c040 con 0x7f8c50007100 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.745+0000 7f8c6b324640 1 -- 172.21.15.53:0/987698663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c50007100 msgr2=0x7f8c5000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.745+0000 7f8c6b324640 1 --2- 172.21.15.53:0/987698663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c50007100 0x7f8c5000dbd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00e220 tx=0x7f8c50007a40 comp rx=0 tx=0).stop 2024-05-10T10:33:44.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.745+0000 7f8c698c1640 1 -- 172.21.15.53:0/987698663 reap_dead start 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.745+0000 7f8c6b324640 1 -- 172.21.15.53:0/987698663 shutdown_connections 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.745+0000 7f8c6b324640 1 -- 172.21.15.53:0/987698663 >> 172.21.15.53:0/987698663 conn(0x7f8c640b2900 msgr2=0x7f8c640b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.745+0000 7f8c6b324640 1 -- 172.21.15.53:0/987698663 shutdown_connections 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.745+0000 7f8c6b324640 1 -- 172.21.15.53:0/987698663 wait complete. 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.746+0000 7f8c6b324640 1 Processor -- start 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.746+0000 7f8c6b324640 1 -- start start 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c6b324640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c50007100 0x7f8c640c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c6b324640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c64148db0 0x7f8c641592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c6b324640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c64146060 0x7f8c641464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c6b324640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640c6890 con 0x7f8c64146060 2024-05-10T10:33:44.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c6b324640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640c65b0 con 0x7f8c50007100 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c6b324640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640cd170 con 0x7f8c64148db0 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c64146060 0x7f8c641464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c64146060 0x7f8c641464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43166/0 (socket says 172.21.15.53:43166) 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 -- 172.21.15.53:0/1572201178 learned_addr learned my addr 172.21.15.53:0/1572201178 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c688bf640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c64148db0 0x7f8c641592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c690c0640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c50007100 0x7f8c640c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 -- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c64148db0 msgr2=0x7f8c641592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c64148db0 0x7f8c641592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 -- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c50007100 msgr2=0x7f8c640c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c50007100 0x7f8c640c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 -- 172.21.15.53:0/1572201178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c640c8200 con 0x7f8c64146060 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.747+0000 7f8c698c1640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c64146060 0x7f8c641464e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c032040 tx=0x7f8c5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.748+0000 7f8c59ffb640 1 -- 172.21.15.53:0/1572201178 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c009d40 con 0x7f8c64146060 2024-05-10T10:33:44.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.748+0000 7f8c59ffb640 1 -- 172.21.15.53:0/1572201178 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c004580 con 0x7f8c64146060 2024-05-10T10:33:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.748+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c64146a20 con 0x7f8c64146060 2024-05-10T10:33:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.748+0000 7f8c59ffb640 1 -- 172.21.15.53:0/1572201178 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c030440 con 0x7f8c64146060 2024-05-10T10:33:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.748+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c64149590 con 0x7f8c64146060 2024-05-10T10:33:44.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.749+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6406a220 con 0x7f8c64146060 2024-05-10T10:33:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.749+0000 7f8c59ffb640 1 -- 172.21.15.53:0/1572201178 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8c5c005d10 con 0x7f8c64146060 2024-05-10T10:33:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.750+0000 7f8c59ffb640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c48076300 0x7f8c480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.750+0000 7f8c59ffb640 1 -- 172.21.15.53:0/1572201178 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8c5c0b8970 con 0x7f8c64146060 2024-05-10T10:33:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.750+0000 7f8c690c0640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c48076300 0x7f8c480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.751+0000 7f8c690c0640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c48076300 0x7f8c480787c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8c60000c00 tx=0x7f8c60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:44.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.754+0000 7f8c59ffb640 1 -- 172.21.15.53:0/1572201178 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8c640c8200 con 0x7f8c64146060 2024-05-10T10:33:44.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.956+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c640c7b40 con 0x7f8c64146060 2024-05-10T10:33:44.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.956+0000 7f8c59ffb640 1 -- 172.21.15.53:0/1572201178 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f8c5c081e60 con 0x7f8c64146060 2024-05-10T10:33:44.958 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:33:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c48076300 msgr2=0x7f8c480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c48076300 0x7f8c480787c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8c60000c00 tx=0x7f8c60009040 comp rx=0 tx=0).stop 2024-05-10T10:33:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c64146060 msgr2=0x7f8c641464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c64146060 0x7f8c641464e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c032040 tx=0x7f8c5c009750 comp rx=0 tx=0).stop 2024-05-10T10:33:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 shutdown_connections 2024-05-10T10:33:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c48076300 0x7f8c480787c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c64146060 0x7f8c641464e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c64148db0 0x7f8c641592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 --2- 172.21.15.53:0/1572201178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c50007100 0x7f8c640c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:44.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 >> 172.21.15.53:0/1572201178 conn(0x7f8c640b2900 msgr2=0x7f8c640b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:44.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 shutdown_connections 2024-05-10T10:33:44.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:44.958+0000 7f8c6b324640 1 -- 172.21.15.53:0/1572201178 wait complete. 2024-05-10T10:33:44.971 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi053.front.sepia.ceph.com... 2024-05-10T10:33:44.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:44.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-10T10:33:44.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:ceph-fuse[41888]: fuse finished with error 0 and tester_r 0 2024-05-10T10:33:44.997 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:33:45.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:33:45.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:45.029 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-10T10:33:45.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:45.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.241+0000 7fc30b4da640 1 Processor -- start 2024-05-10T10:33:45.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.242+0000 7fc30b4da640 1 -- start start 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.242+0000 7fc30b4da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc304071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.242+0000 7fc30b4da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc3040d3510 0x7fc3040ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.242+0000 7fc30b4da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc304072330 con 0x7fc304071880 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.242+0000 7fc30b4da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc304071d70 con 0x7fc3040d3510 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.242+0000 7fc30b4da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc304072050 con 0x7fc30415d290 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc308a75640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc304071880 0x7fc3040d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39648/0 (socket says 172.21.15.53:39648) 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc308a75640 1 -- 172.21.15.53:0/296563684 learned_addr learned my addr 172.21.15.53:0/296563684 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc309a77640 1 --2- 172.21.15.53:0/296563684 >> v2:172.21.15.135:3300/0 conn(0x7fc3040d3510 0x7fc3040ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc309276640 1 --2- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc304071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc2fbfff640 1 -- 172.21.15.53:0/296563684 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590642186 0 0) 0x7fc304072330 con 0x7fc304071880 2024-05-10T10:33:45.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc2fbfff640 1 -- 172.21.15.53:0/296563684 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2f0003520 con 0x7fc304071880 2024-05-10T10:33:45.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc309a77640 1 -- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 msgr2=0x7fc304071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:45.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc309a77640 1 --2- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc304071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc309a77640 1 -- 172.21.15.53:0/296563684 >> v1:172.21.15.53:6789/0 conn(0x7fc304071880 legacy=0x7fc3040d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc309a77640 1 -- 172.21.15.53:0/296563684 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3040cda70 con 0x7fc3040d3510 2024-05-10T10:33:45.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.243+0000 7fc309276640 1 --2- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc304071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:45.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.244+0000 7fc309a77640 1 --2- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 0x7fc3040ce270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009d90 tx=0x7fc2fc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6962d47e094f9ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.244+0000 7fc2fbfff640 1 -- 172.21.15.53:0/296563684 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc007740 con 0x7fc3040d3510 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.244+0000 7fc2fbfff640 1 -- 172.21.15.53:0/296563684 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2fc0078e0 con 0x7fc3040d3510 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.244+0000 7fc2fbfff640 1 -- 172.21.15.53:0/296563684 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc007c90 con 0x7fc3040d3510 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 -- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 msgr2=0x7fc3040ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 --2- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 0x7fc3040ce270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009d90 tx=0x7fc2fc02a660 comp rx=0 tx=0).stop 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 -- 172.21.15.53:0/296563684 shutdown_connections 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 --2- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 0x7fc3040ce270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 --2- 172.21.15.53:0/296563684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc304071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 -- 172.21.15.53:0/296563684 >> 172.21.15.53:0/296563684 conn(0x7fc30409ba20 msgr2=0x7fc30409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 -- 172.21.15.53:0/296563684 shutdown_connections 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.245+0000 7fc30b4da640 1 -- 172.21.15.53:0/296563684 wait complete. 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.246+0000 7fc30b4da640 1 Processor -- start 2024-05-10T10:33:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.246+0000 7fc30b4da640 1 -- start start 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc30b4da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc304071880 0x7fc304159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc30b4da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 0x7fc3041e6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc30b4da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc30415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc30b4da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc3040d1c20 con 0x7fc304071880 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc30b4da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc3040d1aa0 con 0x7fc3040d3510 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc30b4da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc3040cb2e0 con 0x7fc30415d290 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc304071880 0x7fc304159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc304071880 0x7fc304159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43178/0 (socket says 172.21.15.53:43178) 2024-05-10T10:33:45.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 -- 172.21.15.53:0/2624524402 learned_addr learned my addr 172.21.15.53:0/2624524402 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc308a75640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 0x7fc3041e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309a77640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc30415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 -- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 msgr2=0x7fc30415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc30415aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 -- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 msgr2=0x7fc3041e6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 0x7fc3041e6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 -- 172.21.15.53:0/2624524402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30406e070 con 0x7fc304071880 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309a77640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc30415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc309276640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc304071880 0x7fc304159d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc30000e960 tx=0x7fc30000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.247+0000 7fc2f9ffb640 1 -- 172.21.15.53:0/2624524402 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300017070 con 0x7fc304071880 2024-05-10T10:33:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.248+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30406e350 con 0x7fc304071880 2024-05-10T10:33:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.248+0000 7fc2f9ffb640 1 -- 172.21.15.53:0/2624524402 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30000c9a0 con 0x7fc304071880 2024-05-10T10:33:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.248+0000 7fc2f9ffb640 1 -- 172.21.15.53:0/2624524402 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30000cd70 con 0x7fc304071880 2024-05-10T10:33:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.248+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3040cb7f0 con 0x7fc304071880 2024-05-10T10:33:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.249+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3040693f0 con 0x7fc304071880 2024-05-10T10:33:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.249+0000 7fc2f9ffb640 1 -- 172.21.15.53:0/2624524402 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc30001a070 con 0x7fc304071880 2024-05-10T10:33:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.250+0000 7fc2f9ffb640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2e80762d0 0x7fc2e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.250+0000 7fc2f9ffb640 1 -- 172.21.15.53:0/2624524402 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc3000977e0 con 0x7fc304071880 2024-05-10T10:33:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.250+0000 7fc308a75640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2e80762d0 0x7fc2e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.251+0000 7fc308a75640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2e80762d0 0x7fc2e8078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b40c0 tx=0x7fc2f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.254+0000 7fc2f9ffb640 1 -- 172.21.15.53:0/2624524402 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc300060720 con 0x7fc304071880 2024-05-10T10:33:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.446+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3040b8db0 con 0x7fc304071880 2024-05-10T10:33:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.447+0000 7fc2f9ffb640 1 -- 172.21.15.53:0/2624524402 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fc300064a20 con 0x7fc304071880 2024-05-10T10:33:45.449 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:45.449 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2e80762d0 msgr2=0x7fc2e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2e80762d0 0x7fc2e8078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b40c0 tx=0x7fc2f4009040 comp rx=0 tx=0).stop 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc304071880 msgr2=0x7fc304159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc304071880 0x7fc304159d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc30000e960 tx=0x7fc30000ee20 comp rx=0 tx=0).stop 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 shutdown_connections 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2e80762d0 0x7fc2e8078790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc30415d290 0x7fc30415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3040d3510 0x7fc3041e6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 --2- 172.21.15.53:0/2624524402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc304071880 0x7fc304159d60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 >> 172.21.15.53:0/2624524402 conn(0x7fc30409ba20 msgr2=0x7fc3040b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 shutdown_connections 2024-05-10T10:33:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.449+0000 7fc30b4da640 1 -- 172.21.15.53:0/2624524402 wait complete. 2024-05-10T10:33:45.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:45.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.563+0000 7f54075a4640 1 Processor -- start 2024-05-10T10:33:45.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.563+0000 7f54075a4640 1 -- start start 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f54075a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5400162940 0x7f54000b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f54075a4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f54000d59d0 0x7f54000d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f54075a4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54000d2d20 con 0x7f54000b47a0 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f54075a4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5400162d20 con 0x7f54000d59d0 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f54075a4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f54000d3000 con 0x7f5400162940 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5404b3f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f54000d59d0 0x7f54000d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5405b41640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f54000b47a0 0x7f5400162f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36362/0 (socket says 172.21.15.53:36362) 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5405340640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5400162940 0x7f54000b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5405b41640 1 -- 172.21.15.53:0/3784111179 learned_addr learned my addr 172.21.15.53:0/3784111179 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:45.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5404b3f640 1 -- 172.21.15.53:0/3784111179 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5400162940 msgr2=0x7f54000b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:45.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5404b3f640 1 --2- 172.21.15.53:0/3784111179 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5400162940 0x7f54000b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5404b3f640 1 -- 172.21.15.53:0/3784111179 >> v1:172.21.15.135:6789/0 conn(0x7f54000b47a0 legacy=0x7f5400162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.564+0000 7f5404b3f640 1 -- 172.21.15.53:0/3784111179 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54000d32e0 con 0x7f54000d59d0 2024-05-10T10:33:45.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.565+0000 7f5404b3f640 1 --2- 172.21.15.53:0/3784111179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54000d59d0 0x7f54000d3670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097a0 tx=0x7f53f002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1cb20a5d6702c24 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.565+0000 7f53f7fff640 1 -- 172.21.15.53:0/3784111179 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0037070 con 0x7f54000d59d0 2024-05-10T10:33:45.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.565+0000 7f53f7fff640 1 -- 172.21.15.53:0/3784111179 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f002adb0 con 0x7f54000d59d0 2024-05-10T10:33:45.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.565+0000 7f53f7fff640 1 -- 172.21.15.53:0/3784111179 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0032660 con 0x7f54000d59d0 2024-05-10T10:33:45.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 -- 172.21.15.53:0/3784111179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54000d59d0 msgr2=0x7f54000d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 --2- 172.21.15.53:0/3784111179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54000d59d0 0x7f54000d3670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097a0 tx=0x7f53f002a060 comp rx=0 tx=0).stop 2024-05-10T10:33:45.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 -- 172.21.15.53:0/3784111179 shutdown_connections 2024-05-10T10:33:45.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 --2- 172.21.15.53:0/3784111179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54000d59d0 0x7f54000d3670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 --2- 172.21.15.53:0/3784111179 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5400162940 0x7f54000b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 -- 172.21.15.53:0/3784111179 >> 172.21.15.53:0/3784111179 conn(0x7f540009e380 msgr2=0x7f54000d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:45.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 -- 172.21.15.53:0/3784111179 shutdown_connections 2024-05-10T10:33:45.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.566+0000 7f54075a4640 1 -- 172.21.15.53:0/3784111179 wait complete. 2024-05-10T10:33:45.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.567+0000 7f54075a4640 1 Processor -- start 2024-05-10T10:33:45.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.567+0000 7f54075a4640 1 -- start start 2024-05-10T10:33:45.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f54075a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54000b47a0 0x7f5400152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f54075a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000d59d0 0x7f54000ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f54075a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5400162940 0x7f54000cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f54075a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54000d2390 con 0x7f54000d59d0 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f54075a4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54000cfc50 con 0x7f5400162940 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f54075a4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f54000d2510 con 0x7f54000b47a0 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000d59d0 0x7f54000ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000d59d0 0x7f54000ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43182/0 (socket says 172.21.15.53:43182) 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 -- 172.21.15.53:0/780193886 learned_addr learned my addr 172.21.15.53:0/780193886 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5405b41640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5400162940 0x7f54000cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5405340640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54000b47a0 0x7f5400152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 -- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54000b47a0 msgr2=0x7f5400152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54000b47a0 0x7f5400152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 -- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5400162940 msgr2=0x7f54000cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5400162940 0x7f54000cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.568+0000 7f5404b3f640 1 -- 172.21.15.53:0/780193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54001528c0 con 0x7f54000d59d0 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.569+0000 7f5404b3f640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000d59d0 0x7f54000ce730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53f0035040 tx=0x7f53f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.569+0000 7f53f5ffb640 1 -- 172.21.15.53:0/780193886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0041070 con 0x7f54000d59d0 2024-05-10T10:33:45.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.569+0000 7f53f5ffb640 1 -- 172.21.15.53:0/780193886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f003c460 con 0x7f54000d59d0 2024-05-10T10:33:45.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.569+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54000ba380 con 0x7f54000d59d0 2024-05-10T10:33:45.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.569+0000 7f53f5ffb640 1 -- 172.21.15.53:0/780193886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0037040 con 0x7f54000d59d0 2024-05-10T10:33:45.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.569+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54000ba940 con 0x7f54000d59d0 2024-05-10T10:33:45.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.570+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540006a220 con 0x7f54000d59d0 2024-05-10T10:33:45.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.571+0000 7f53f5ffb640 1 -- 172.21.15.53:0/780193886 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f53f003a070 con 0x7f54000d59d0 2024-05-10T10:33:45.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.571+0000 7f53f5ffb640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f53e40762d0 0x7f53e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.571+0000 7f53f5ffb640 1 -- 172.21.15.53:0/780193886 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f53f00b7b40 con 0x7f54000d59d0 2024-05-10T10:33:45.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.571+0000 7f5405340640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f53e40762d0 0x7f53e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.572+0000 7f5405340640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f53e40762d0 0x7f53e4078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0074a0 tx=0x7f53fc00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.575+0000 7f53f5ffb640 1 -- 172.21.15.53:0/780193886 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f53f00bb030 con 0x7f54000d59d0 2024-05-10T10:33:45.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.769+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54000d2730 con 0x7f54000d59d0 2024-05-10T10:33:45.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.770+0000 7f53f5ffb640 1 -- 172.21.15.53:0/780193886 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f53f0089160 con 0x7f54000d59d0 2024-05-10T10:33:45.772 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:45.772 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f53e40762d0 msgr2=0x7f53e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f53e40762d0 0x7f53e4078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0074a0 tx=0x7f53fc00b040 comp rx=0 tx=0).stop 2024-05-10T10:33:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000d59d0 msgr2=0x7f54000ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000d59d0 0x7f54000ce730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53f0035040 tx=0x7f53f00046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 shutdown_connections 2024-05-10T10:33:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f53e40762d0 0x7f53e4078790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5400162940 0x7f54000cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54000d59d0 0x7f54000ce730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 --2- 172.21.15.53:0/780193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54000b47a0 0x7f5400152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 >> 172.21.15.53:0/780193886 conn(0x7f540009e380 msgr2=0x7f540006e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 shutdown_connections 2024-05-10T10:33:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.772+0000 7f54075a4640 1 -- 172.21.15.53:0/780193886 wait complete. 2024-05-10T10:33:45.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:33:45.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.890+0000 7fb599a75640 1 Processor -- start 2024-05-10T10:33:45.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.890+0000 7fb599a75640 1 -- start start 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb599a75640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb5940cffd0 0x7fb5940b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb599a75640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59406e720 con 0x7fb5940cf3e0 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb599a75640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59406de80 con 0x7fb5940cd500 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb599a75640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb59406e160 con 0x7fb5940cffd0 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5937fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb5940cf3e0 0x7fb5940cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39664/0 (socket says 172.21.15.53:39664) 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5937fe640 1 -- 172.21.15.53:0/4094254167 learned_addr learned my addr 172.21.15.53:0/4094254167 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb592ffd640 1 --2- 172.21.15.53:0/4094254167 >> v2:172.21.15.135:3300/0 conn(0x7fb5940cffd0 0x7fb5940b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3433272856 0 0) 0x7fb59406e720 con 0x7fb5940cf3e0 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb57c003520 con 0x7fb5940cf3e0 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 550681665 0 0) 0x7fb57c003520 con 0x7fb5940cf3e0 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 >> v2:172.21.15.135:3300/0 conn(0x7fb5940cffd0 msgr2=0x7fb5940b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 --2- 172.21.15.53:0/4094254167 >> v2:172.21.15.135:3300/0 conn(0x7fb5940cffd0 0x7fb5940b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 >> v1:172.21.15.135:6789/0 conn(0x7fb5940cd500 legacy=0x7fb5940d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59406e440 con 0x7fb5940cf3e0 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fb584001670 con 0x7fb5940cf3e0 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 >> v1:172.21.15.53:6789/0 conn(0x7fb5940cf3e0 legacy=0x7fb5940cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.891+0000 7fb5927fc640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0039b0 0x7fb57c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb5927fc640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb57c004800 0x7fb57c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb5927fc640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb57c0051c0 0x7fb57c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb59406e160 con 0x7fb57c0051c0 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c00c060 con 0x7fb57c004800 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c00c260 con 0x7fb57c0039b0 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb593fff640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb57c0051c0 0x7fb57c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb592ffd640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb57c004800 0x7fb57c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb5937fe640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0039b0 0x7fb57c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb593fff640 1 -- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0039b0 msgr2=0x7fb57c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb593fff640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0039b0 0x7fb57c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb593fff640 1 -- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb57c004800 msgr2=0x7fb57c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb593fff640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb57c004800 0x7fb57c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.892+0000 7fb593fff640 1 -- 172.21.15.53:0/4094254167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c003520 con 0x7fb57c0051c0 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb593fff640 1 -- 172.21.15.53:0/4094254167 reap_dead start 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb593fff640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb57c0051c0 0x7fb57c00b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb58c001e20 tx=0x7fb57c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde2ee7203a012eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c001a70 con 0x7fb57c0051c0 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c02ae40 con 0x7fb57c0051c0 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb5927fc640 1 -- 172.21.15.53:0/4094254167 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c003bb0 con 0x7fb57c0051c0 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb599a75640 1 -- 172.21.15.53:0/4094254167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb57c0051c0 msgr2=0x7fb57c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb599a75640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb57c0051c0 0x7fb57c00b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb58c001e20 tx=0x7fb57c005830 comp rx=0 tx=0).stop 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb599a75640 1 -- 172.21.15.53:0/4094254167 shutdown_connections 2024-05-10T10:33:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb599a75640 1 --2- 172.21.15.53:0/4094254167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb57c0051c0 0x7fb57c00b9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb599a75640 1 -- 172.21.15.53:0/4094254167 >> 172.21.15.53:0/4094254167 conn(0x7fb5940d2eb0 msgr2=0x7fb5940d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:45.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb599a75640 1 -- 172.21.15.53:0/4094254167 shutdown_connections 2024-05-10T10:33:45.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.893+0000 7fb599a75640 1 -- 172.21.15.53:0/4094254167 wait complete. 2024-05-10T10:33:45.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.895+0000 7fb599a75640 1 Processor -- start 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.895+0000 7fb599a75640 1 -- start start 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb599a75640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0051c0 0x7fb5940c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb599a75640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5940b0f50 0x7fb59409cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb599a75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5940b13b0 0x7fb59409d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb599a75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb5940cf8c0 con 0x7fb5940b13b0 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb599a75640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb5940b4510 con 0x7fb5940b0f50 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb599a75640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb5940cfa40 con 0x7fb57c0051c0 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5940b13b0 0x7fb59409d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5940b13b0 0x7fb59409d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43194/0 (socket says 172.21.15.53:43194) 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 -- 172.21.15.53:0/4159951813 learned_addr learned my addr 172.21.15.53:0/4159951813 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb5937fe640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0051c0 0x7fb5940c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb592ffd640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5940b0f50 0x7fb59409cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 -- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0051c0 msgr2=0x7fb5940c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0051c0 0x7fb5940c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 -- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5940b0f50 msgr2=0x7fb59409cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:45.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5940b0f50 0x7fb59409cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:45.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 -- 172.21.15.53:0/4159951813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5940c8850 con 0x7fb5940b13b0 2024-05-10T10:33:45.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb593fff640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5940b13b0 0x7fb59409d1b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb58c001f40 tx=0x7fb58c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb57bfff640 1 -- 172.21.15.53:0/4159951813 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c007db0 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb57bfff640 1 -- 172.21.15.53:0/4159951813 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c006d60 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.896+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59409d6f0 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.897+0000 7fb57bfff640 1 -- 172.21.15.53:0/4159951813 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0056a0 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.897+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5940c8af0 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.898+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb59406a220 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.898+0000 7fb57bfff640 1 -- 172.21.15.53:0/4159951813 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb58c007030 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.899+0000 7fb57bfff640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb5800762d0 0x7fb580078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.899+0000 7fb57bfff640 1 -- 172.21.15.53:0/4159951813 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb58c0b6d30 con 0x7fb5940b13b0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.899+0000 7fb5937fe640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb5800762d0 0x7fb580078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.899+0000 7fb5937fe640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb5800762d0 0x7fb580078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5840017e0 tx=0x7fb5840015f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:45.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:45.903+0000 7fb57bfff640 1 -- 172.21.15.53:0/4159951813 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb58c043060 con 0x7fb5940b13b0 2024-05-10T10:33:46.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.133+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb594071680 con 0x7fb5940b13b0 2024-05-10T10:33:46.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.581+0000 7fb57bfff640 1 -- 172.21.15.53:0/4159951813 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fb58c039070 con 0x7fb5940b13b0 2024-05-10T10:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb5800762d0 msgr2=0x7fb580078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb5800762d0 0x7fb580078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5840017e0 tx=0x7fb5840015f0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5940b13b0 msgr2=0x7fb59409d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5940b13b0 0x7fb59409d1b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb58c001f40 tx=0x7fb58c0014a0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 shutdown_connections 2024-05-10T10:33:46.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb5800762d0 0x7fb580078790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5940b13b0 0x7fb59409d1b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5940b0f50 0x7fb59409cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 --2- 172.21.15.53:0/4159951813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb57c0051c0 0x7fb5940c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 >> 172.21.15.53:0/4159951813 conn(0x7fb5940d2eb0 msgr2=0x7fb5940757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 shutdown_connections 2024-05-10T10:33:46.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.584+0000 7fb599a75640 1 -- 172.21.15.53:0/4159951813 wait complete. 2024-05-10T10:33:46.598 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-10T10:33:46.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:33:46.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.705+0000 7f9e34664640 1 Processor -- start 2024-05-10T10:33:46.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.705+0000 7f9e34664640 1 -- start start 2024-05-10T10:33:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e34664640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e2c0d2d10 0x7f9e2c0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e34664640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9e2c0d1f50 0x7f9e2c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e34664640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e2c0d31a0 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e34664640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2c0cfd50 con 0x7f9e2c0d2d10 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e34664640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2c0b5d90 con 0x7f9e2c0d1f50 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e31bff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e2c0d2d10 0x7f9e2c0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e31bff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e2c0d2d10 0x7f9e2c0d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43200/0 (socket says 172.21.15.53:43200) 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.706+0000 7f9e31bff640 1 -- 172.21.15.53:0/2009829615 learned_addr learned my addr 172.21.15.53:0/2009829615 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.707+0000 7f9e32c01640 1 --2- 172.21.15.53:0/2009829615 >> v2:172.21.15.135:3300/0 conn(0x7f9e2c0d1f50 0x7f9e2c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.707+0000 7f9e31bff640 1 -- 172.21.15.53:0/2009829615 >> v2:172.21.15.135:3300/0 conn(0x7f9e2c0d1f50 msgr2=0x7f9e2c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.707+0000 7f9e31bff640 1 --2- 172.21.15.53:0/2009829615 >> v2:172.21.15.135:3300/0 conn(0x7f9e2c0d1f50 0x7f9e2c0d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.707+0000 7f9e31bff640 1 -- 172.21.15.53:0/2009829615 >> v1:172.21.15.135:6789/0 conn(0x7f9e2c0d3df0 legacy=0x7f9e2c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.707+0000 7f9e31bff640 1 -- 172.21.15.53:0/2009829615 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c0d41d0 con 0x7f9e2c0d2d10 2024-05-10T10:33:46.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.707+0000 7f9e31bff640 1 --2- 172.21.15.53:0/2009829615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d2d10 0x7f9e2c0d1a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0d2ca0 tx=0x7f9e2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362ea7d0895ceea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:46.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e313fe640 1 -- 172.21.15.53:0/2009829615 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20037070 con 0x7f9e2c0d2d10 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e313fe640 1 -- 172.21.15.53:0/2009829615 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e20032440 con 0x7f9e2c0d2d10 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e313fe640 1 -- 172.21.15.53:0/2009829615 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20032810 con 0x7f9e2c0d2d10 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 -- 172.21.15.53:0/2009829615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d2d10 msgr2=0x7f9e2c0d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 --2- 172.21.15.53:0/2009829615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d2d10 0x7f9e2c0d1a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0d2ca0 tx=0x7f9e2002a230 comp rx=0 tx=0).stop 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 -- 172.21.15.53:0/2009829615 shutdown_connections 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 --2- 172.21.15.53:0/2009829615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d2d10 0x7f9e2c0d1a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 --2- 172.21.15.53:0/2009829615 >> v2:172.21.15.135:3300/0 conn(0x7f9e2c0d1f50 0x7f9e2c0d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 -- 172.21.15.53:0/2009829615 >> 172.21.15.53:0/2009829615 conn(0x7f9e2c09e450 msgr2=0x7f9e2c0d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 -- 172.21.15.53:0/2009829615 shutdown_connections 2024-05-10T10:33:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.708+0000 7f9e34664640 1 -- 172.21.15.53:0/2009829615 wait complete. 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 Processor -- start 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 -- start start 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e2c0d1f50 0x7f9e2c1f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e2c0d2d10 0x7f9e2c0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d3df0 0x7f9e2c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0b57d0 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0d2380 con 0x7f9e2c0d2d10 2024-05-10T10:33:46.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e34664640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2c0b5ab0 con 0x7f9e2c0d1f50 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e32c01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d3df0 0x7f9e2c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.710+0000 7f9e32c01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d3df0 0x7f9e2c0ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43202/0 (socket says 172.21.15.53:43202) 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32c01640 1 -- 172.21.15.53:0/1701395572 learned_addr learned my addr 172.21.15.53:0/1701395572 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e31bff640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e2c0d2d10 0x7f9e2c0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32400640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e2c0d1f50 0x7f9e2c1f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32c01640 1 -- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e2c0d1f50 msgr2=0x7f9e2c1f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32c01640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e2c0d1f50 0x7f9e2c1f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32c01640 1 -- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e2c0d2d10 msgr2=0x7f9e2c0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32c01640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e2c0d2d10 0x7f9e2c0b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32c01640 1 -- 172.21.15.53:0/1701395572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c15cb20 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e32c01640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d3df0 0x7f9e2c0ca200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400c7f0 tx=0x7f9e2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e1effd640 1 -- 172.21.15.53:0/1701395572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24015070 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e1effd640 1 -- 172.21.15.53:0/1701395572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e24004220 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c15ce00 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e1effd640 1 -- 172.21.15.53:0/1701395572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24007620 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.711+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c15d3c0 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.712+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c09e8c0 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.713+0000 7f9e1effd640 1 -- 172.21.15.53:0/1701395572 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9e240079d0 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.713+0000 7f9e1effd640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e100762d0 0x7f9e10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:46.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.713+0000 7f9e1effd640 1 -- 172.21.15.53:0/1701395572 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9e24097ac0 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.713+0000 7f9e32400640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e100762d0 0x7f9e10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:46.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.714+0000 7f9e32400640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e100762d0 0x7f9e10078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009f20 tx=0x7f9e280094d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:46.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.718+0000 7f9e1effd640 1 -- 172.21.15.53:0/1701395572 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9e2c09e8c0 con 0x7f9e2c0d3df0 2024-05-10T10:33:46.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:46.948+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9e2c0d27c0 con 0x7f9e2c0d3df0 2024-05-10T10:33:47.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.582+0000 7f9e1effd640 1 -- 172.21.15.53:0/1701395572 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f9e24061040 con 0x7f9e2c0d3df0 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e100762d0 msgr2=0x7f9e10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e100762d0 0x7f9e10078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009f20 tx=0x7f9e280094d0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d3df0 msgr2=0x7f9e2c0ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d3df0 0x7f9e2c0ca200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400c7f0 tx=0x7f9e2400ccb0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 shutdown_connections 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e100762d0 0x7f9e10078790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e2c0d3df0 0x7f9e2c0ca200 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e2c0d2d10 0x7f9e2c0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 --2- 172.21.15.53:0/1701395572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e2c0d1f50 0x7f9e2c1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 >> 172.21.15.53:0/1701395572 conn(0x7f9e2c09e450 msgr2=0x7f9e2c06e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:47.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 shutdown_connections 2024-05-10T10:33:47.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.585+0000 7f9e34664640 1 -- 172.21.15.53:0/1701395572 wait complete. 2024-05-10T10:33:47.600 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:33:47.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:47.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.708+0000 7f2e36693640 1 Processor -- start 2024-05-10T10:33:47.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.708+0000 7f2e36693640 1 -- start start 2024-05-10T10:33:47.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e36693640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e30162940 0x7f2e300b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:47.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e36693640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2e300d3d80 0x7f2e300d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:47.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e36693640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e300d2d20 con 0x7f2e300d59d0 2024-05-10T10:33:47.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e36693640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2e30162d20 con 0x7f2e30162940 2024-05-10T10:33:47.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e36693640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2e300d3000 con 0x7f2e300d3d80 2024-05-10T10:33:47.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e2ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e30162940 0x7f2e300b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e27fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2e300d59d0 0x7f2e300d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39680/0 (socket says 172.21.15.53:39680) 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e27fff640 1 -- 172.21.15.53:0/3081149424 learned_addr learned my addr 172.21.15.53:0/3081149424 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.709+0000 7f2e34c30640 1 --2- 172.21.15.53:0/3081149424 >> v2:172.21.15.135:3300/0 conn(0x7f2e300d3d80 0x7f2e300d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.710+0000 7f2e2ffff640 1 -- 172.21.15.53:0/3081149424 >> v2:172.21.15.135:3300/0 conn(0x7f2e300d3d80 msgr2=0x7f2e300d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.710+0000 7f2e2ffff640 1 --2- 172.21.15.53:0/3081149424 >> v2:172.21.15.135:3300/0 conn(0x7f2e300d3d80 0x7f2e300d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.710+0000 7f2e2ffff640 1 -- 172.21.15.53:0/3081149424 >> v1:172.21.15.53:6789/0 conn(0x7f2e300d59d0 legacy=0x7f2e300d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.710+0000 7f2e2f7fe640 1 -- 172.21.15.53:0/3081149424 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 575789013 0 0) 0x7f2e300d2d20 con 0x7f2e300d59d0 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.710+0000 7f2e2ffff640 1 -- 172.21.15.53:0/3081149424 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e300d32e0 con 0x7f2e30162940 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.710+0000 7f2e2ffff640 1 --2- 172.21.15.53:0/3081149424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e300b5e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2e300d5e50 tx=0x7f2e2002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449f0ab238737a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e2f7fe640 1 -- 172.21.15.53:0/3081149424 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20037070 con 0x7f2e30162940 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e2f7fe640 1 -- 172.21.15.53:0/3081149424 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e20032440 con 0x7f2e30162940 2024-05-10T10:33:47.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e2f7fe640 1 -- 172.21.15.53:0/3081149424 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2002ae60 con 0x7f2e30162940 2024-05-10T10:33:47.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 -- 172.21.15.53:0/3081149424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 msgr2=0x7f2e300b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 --2- 172.21.15.53:0/3081149424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e300b5e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2e300d5e50 tx=0x7f2e2002a110 comp rx=0 tx=0).stop 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 -- 172.21.15.53:0/3081149424 shutdown_connections 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 --2- 172.21.15.53:0/3081149424 >> v2:172.21.15.135:3300/0 conn(0x7f2e300d3d80 0x7f2e300d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 --2- 172.21.15.53:0/3081149424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e300b5e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 -- 172.21.15.53:0/3081149424 >> 172.21.15.53:0/3081149424 conn(0x7f2e3009e380 msgr2=0x7f2e300d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 -- 172.21.15.53:0/3081149424 shutdown_connections 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.711+0000 7f2e36693640 1 -- 172.21.15.53:0/3081149424 wait complete. 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.712+0000 7f2e36693640 1 Processor -- start 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e36693640 1 -- start start 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e36693640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e300d3d80 0x7f2e301f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e36693640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e300d59d0 0x7f2e300ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:47.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e36693640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e30161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e36693640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300d2390 con 0x7f2e30162940 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e36693640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300cfc50 con 0x7f2e300d59d0 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e36693640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2e300d2510 con 0x7f2e300d3d80 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e34c30640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e30161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e34c30640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e30161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43220/0 (socket says 172.21.15.53:43220) 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.713+0000 7f2e34c30640 1 -- 172.21.15.53:0/4119907043 learned_addr learned my addr 172.21.15.53:0/4119907043 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e27fff640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e300d59d0 0x7f2e300ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:47.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e2ffff640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e300d3d80 0x7f2e301f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e34c30640 1 -- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e300d3d80 msgr2=0x7f2e301f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e34c30640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e300d3d80 0x7f2e301f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e34c30640 1 -- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e300d59d0 msgr2=0x7f2e300ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e34c30640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e300d59d0 0x7f2e300ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e34c30640 1 -- 172.21.15.53:0/4119907043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e301f6060 con 0x7f2e30162940 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e34c30640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e30161610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e28007e60 tx=0x7f2e2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e2d7fa640 1 -- 172.21.15.53:0/4119907043 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28014420 con 0x7f2e30162940 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e2d7fa640 1 -- 172.21.15.53:0/4119907043 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e28019070 con 0x7f2e30162940 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e300cc180 con 0x7f2e30162940 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e2d7fa640 1 -- 172.21.15.53:0/4119907043 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28014820 con 0x7f2e30162940 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.714+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e300cc700 con 0x7f2e30162940 2024-05-10T10:33:47.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.715+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3006a220 con 0x7f2e30162940 2024-05-10T10:33:47.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.716+0000 7f2e2d7fa640 1 -- 172.21.15.53:0/4119907043 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e280149c0 con 0x7f2e30162940 2024-05-10T10:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.716+0000 7f2e2d7fa640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e100762d0 0x7f2e10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.716+0000 7f2e2d7fa640 1 -- 172.21.15.53:0/4119907043 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2e28097fc0 con 0x7f2e30162940 2024-05-10T10:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.717+0000 7f2e2ffff640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e100762d0 0x7f2e10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:47.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.717+0000 7f2e2ffff640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e100762d0 0x7f2e10078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e20034000 tx=0x7f2e2002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:47.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.721+0000 7f2e2d7fa640 1 -- 172.21.15.53:0/4119907043 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2e301f6060 con 0x7f2e30162940 2024-05-10T10:33:47.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.948+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e300b2330 con 0x7f2e30162940 2024-05-10T10:33:47.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.949+0000 7f2e2d7fa640 1 -- 172.21.15.53:0/4119907043 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f2e28061020 con 0x7f2e30162940 2024-05-10T10:33:47.950 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:47.951 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:33:47.951 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e100762d0 msgr2=0x7f2e10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e100762d0 0x7f2e10078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e20034000 tx=0x7f2e2002e000 comp rx=0 tx=0).stop 2024-05-10T10:33:47.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 msgr2=0x7f2e30161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e30161610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e28007e60 tx=0x7f2e2800e4b0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 shutdown_connections 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e100762d0 0x7f2e10078790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e30162940 0x7f2e30161610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e300d59d0 0x7f2e300ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 --2- 172.21.15.53:0/4119907043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e300d3d80 0x7f2e301f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 >> 172.21.15.53:0/4119907043 conn(0x7f2e3009e380 msgr2=0x7f2e300ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 shutdown_connections 2024-05-10T10:33:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:47.951+0000 7f2e36693640 1 -- 172.21.15.53:0/4119907043 wait complete. 2024-05-10T10:33:47.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:48.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.066+0000 7f221a415640 1 Processor -- start 2024-05-10T10:33:48.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.066+0000 7f221a415640 1 -- start start 2024-05-10T10:33:48.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f221a415640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f221415f3b0 0x7f22140d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f221a415640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22140d30b0 con 0x7f221415f790 2024-05-10T10:33:48.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f221a415640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22140d3390 con 0x7f22140d5d30 2024-05-10T10:33:48.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f221a415640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f22140c8c80 con 0x7f221415f3b0 2024-05-10T10:33:48.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f22137fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f221415f790 0x7f22140c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39694/0 (socket says 172.21.15.53:39694) 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f22137fe640 1 -- 172.21.15.53:0/1802983436 learned_addr learned my addr 172.21.15.53:0/1802983436 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f2213fff640 1 --2- 172.21.15.53:0/1802983436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f221415f3b0 0x7f22140d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f2212ffd640 1 -- 172.21.15.53:0/1802983436 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2311563208 0 0) 0x7f22140d30b0 con 0x7f221415f790 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f2212ffd640 1 -- 172.21.15.53:0/1802983436 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21fc003520 con 0x7f221415f790 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f2212ffd640 1 -- 172.21.15.53:0/1802983436 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18525635 0 0) 0x7f22140d3390 con 0x7f22140d5d30 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.067+0000 7f2212ffd640 1 -- 172.21.15.53:0/1802983436 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22140d30b0 con 0x7f22140d5d30 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.068+0000 7f2213fff640 1 -- 172.21.15.53:0/1802983436 >> v1:172.21.15.135:6789/0 conn(0x7f22140d5d30 legacy=0x7f22140d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.068+0000 7f2213fff640 1 -- 172.21.15.53:0/1802983436 >> v1:172.21.15.53:6789/0 conn(0x7f221415f790 legacy=0x7f22140c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.068+0000 7f2213fff640 1 -- 172.21.15.53:0/1802983436 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22140c8f60 con 0x7f221415f3b0 2024-05-10T10:33:48.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.068+0000 7f2213fff640 1 --2- 172.21.15.53:0/1802983436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f221415f3b0 0x7f22140d36f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22040097d0 tx=0x7f220402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5587ca58b5cd670f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.068+0000 7f2212ffd640 1 -- 172.21.15.53:0/1802983436 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204037070 con 0x7f221415f3b0 2024-05-10T10:33:48.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f2212ffd640 1 -- 172.21.15.53:0/1802983436 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204032440 con 0x7f221415f3b0 2024-05-10T10:33:48.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f2212ffd640 1 -- 172.21.15.53:0/1802983436 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204032810 con 0x7f221415f3b0 2024-05-10T10:33:48.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f221a415640 1 -- 172.21.15.53:0/1802983436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f221415f3b0 msgr2=0x7f22140d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f221a415640 1 --2- 172.21.15.53:0/1802983436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f221415f3b0 0x7f22140d36f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22040097d0 tx=0x7f220402a210 comp rx=0 tx=0).stop 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f221a415640 1 -- 172.21.15.53:0/1802983436 shutdown_connections 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f221a415640 1 --2- 172.21.15.53:0/1802983436 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f221415f3b0 0x7f22140d36f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f221a415640 1 -- 172.21.15.53:0/1802983436 >> 172.21.15.53:0/1802983436 conn(0x7f22140d5130 msgr2=0x7f22140d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f221a415640 1 -- 172.21.15.53:0/1802983436 shutdown_connections 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.069+0000 7f221a415640 1 -- 172.21.15.53:0/1802983436 wait complete. 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.070+0000 7f221a415640 1 Processor -- start 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.070+0000 7f221a415640 1 -- start start 2024-05-10T10:33:48.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f221a415640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22140d5d30 0x7f2214147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f221a415640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f221415f3b0 0x7f22140ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f221a415640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22140b7e70 0x7f22140d1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f221a415640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f22140d3db0 con 0x7f221415f3b0 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f221a415640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22140d3c30 con 0x7f22140d5d30 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f221a415640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f22140d3f30 con 0x7f22140b7e70 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f22137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f221415f3b0 0x7f22140ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f22137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f221415f3b0 0x7f22140ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43222/0 (socket says 172.21.15.53:43222) 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f22137fe640 1 -- 172.21.15.53:0/938839050 learned_addr learned my addr 172.21.15.53:0/938839050 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f2213fff640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22140d5d30 0x7f2214147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.071+0000 7f22189b2640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22140b7e70 0x7f22140d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f22137fe640 1 -- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22140b7e70 msgr2=0x7f22140d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f22137fe640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22140b7e70 0x7f22140d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f22137fe640 1 -- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22140d5d30 msgr2=0x7f2214147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f22137fe640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22140d5d30 0x7f2214147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f22137fe640 1 -- 172.21.15.53:0/938839050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22140b9610 con 0x7f221415f3b0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f22189b2640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22140b7e70 0x7f22140d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f22137fe640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f221415f3b0 0x7f22140ceaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f220800c430 tx=0x7f220800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f2210ff9640 1 -- 172.21.15.53:0/938839050 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208017070 con 0x7f221415f3b0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f2210ff9640 1 -- 172.21.15.53:0/938839050 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208005ce0 con 0x7f221415f3b0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22140b9890 con 0x7f221415f3b0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f2210ff9640 1 -- 172.21.15.53:0/938839050 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080072f0 con 0x7f221415f3b0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.072+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22140b9e50 con 0x7f221415f3b0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.073+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221406a220 con 0x7f221415f3b0 2024-05-10T10:33:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.074+0000 7f2210ff9640 1 -- 172.21.15.53:0/938839050 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f220801a070 con 0x7f221415f3b0 2024-05-10T10:33:48.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.074+0000 7f2210ff9640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22000762d0 0x7f2200078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.074+0000 7f2210ff9640 1 -- 172.21.15.53:0/938839050 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2208096d70 con 0x7f221415f3b0 2024-05-10T10:33:48.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.074+0000 7f2213fff640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22000762d0 0x7f2200078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.075+0000 7f2213fff640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22000762d0 0x7f2200078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2204034000 tx=0x7f220402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.078+0000 7f2210ff9640 1 -- 172.21.15.53:0/938839050 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f22140b9610 con 0x7f221415f3b0 2024-05-10T10:33:48.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.273+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f221415bcd0 con 0x7f221415f3b0 2024-05-10T10:33:48.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.275+0000 7f2210ff9640 1 -- 172.21.15.53:0/938839050 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f22080603a0 con 0x7f221415f3b0 2024-05-10T10:33:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:48.276 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22000762d0 msgr2=0x7f2200078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22000762d0 0x7f2200078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2204034000 tx=0x7f220402e000 comp rx=0 tx=0).stop 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f221415f3b0 msgr2=0x7f22140ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f221415f3b0 0x7f22140ceaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f220800c430 tx=0x7f220800c8f0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 shutdown_connections 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22000762d0 0x7f2200078790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22140b7e70 0x7f22140d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f221415f3b0 0x7f22140ceaa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 --2- 172.21.15.53:0/938839050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22140d5d30 0x7f2214147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 >> 172.21.15.53:0/938839050 conn(0x7f22140d5130 msgr2=0x7f22140cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 shutdown_connections 2024-05-10T10:33:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.277+0000 7f221a415640 1 -- 172.21.15.53:0/938839050 wait complete. 2024-05-10T10:33:48.290 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:33:48.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:33:48.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.392+0000 7f5f69e6b640 1 Processor -- start 2024-05-10T10:33:48.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.392+0000 7f5f69e6b640 1 -- start start 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.392+0000 7f5f69e6b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5f64168220 0x7f5f6416a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f69e6b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f640b0a90 con 0x7f5f6415f8c0 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f69e6b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f640b3540 con 0x7f5f6415f4e0 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f69e6b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5f640b23f0 con 0x7f5f64168220 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f63fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5f64168220 0x7f5f6416a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f62ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5f6415f8c0 0x7f5f6406e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39696/0 (socket says 172.21.15.53:39696) 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f62ffd640 1 -- 172.21.15.53:0/902318305 learned_addr learned my addr 172.21.15.53:0/902318305 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f63fff640 1 -- 172.21.15.53:0/902318305 >> v1:172.21.15.135:6789/0 conn(0x7f5f6415f4e0 legacy=0x7f5f6406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f63fff640 1 -- 172.21.15.53:0/902318305 >> v1:172.21.15.53:6789/0 conn(0x7f5f6415f8c0 legacy=0x7f5f6406e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f63fff640 1 -- 172.21.15.53:0/902318305 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f640b3820 con 0x7f5f64168220 2024-05-10T10:33:48.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.393+0000 7f5f627fc640 1 -- 172.21.15.53:0/902318305 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2076066943 0 0) 0x7f5f640b0a90 con 0x7f5f6415f8c0 2024-05-10T10:33:48.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.394+0000 7f5f63fff640 1 --2- 172.21.15.53:0/902318305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f64168220 0x7f5f6416a610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009d90 tx=0x7f5f5c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578b0f1af6ee8f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.394+0000 7f5f627fc640 1 -- 172.21.15.53:0/902318305 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c007740 con 0x7f5f64168220 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.394+0000 7f5f627fc640 1 -- 172.21.15.53:0/902318305 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5c0078e0 con 0x7f5f64168220 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.394+0000 7f5f627fc640 1 -- 172.21.15.53:0/902318305 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c007cb0 con 0x7f5f64168220 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.394+0000 7f5f69e6b640 1 -- 172.21.15.53:0/902318305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f64168220 msgr2=0x7f5f6416a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.394+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/902318305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f64168220 0x7f5f6416a610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009d90 tx=0x7f5f5c02a660 comp rx=0 tx=0).stop 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.395+0000 7f5f69e6b640 1 -- 172.21.15.53:0/902318305 shutdown_connections 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.395+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/902318305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f64168220 0x7f5f6416a610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.395+0000 7f5f69e6b640 1 -- 172.21.15.53:0/902318305 >> 172.21.15.53:0/902318305 conn(0x7f5f640d2df0 msgr2=0x7f5f640d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.395+0000 7f5f69e6b640 1 -- 172.21.15.53:0/902318305 shutdown_connections 2024-05-10T10:33:48.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.395+0000 7f5f69e6b640 1 -- 172.21.15.53:0/902318305 wait complete. 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.396+0000 7f5f69e6b640 1 Processor -- start 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.396+0000 7f5f69e6b640 1 -- start start 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f69e6b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6415f4e0 0x7f5f64159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f69e6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f6415f8c0 0x7f5f640c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f69e6b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f64168220 0x7f5f640c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f69e6b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f640b3de0 con 0x7f5f6415f8c0 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f69e6b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f640b4150 con 0x7f5f6415f4e0 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f69e6b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6406e7f0 con 0x7f5f64168220 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f6415f8c0 0x7f5f640c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f6415f8c0 0x7f5f640c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43238/0 (socket says 172.21.15.53:43238) 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 -- 172.21.15.53:0/3640644141 learned_addr learned my addr 172.21.15.53:0/3640644141 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:48.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f637fe640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6415f4e0 0x7f5f64159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f63fff640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f64168220 0x7f5f640c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 -- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f64168220 msgr2=0x7f5f640c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f64168220 0x7f5f640c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 -- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6415f4e0 msgr2=0x7f5f64159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6415f4e0 0x7f5f64159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 -- 172.21.15.53:0/3640644141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f640c97d0 con 0x7f5f6415f8c0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f62ffd640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f6415f8c0 0x7f5f640c8c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c430 tx=0x7f5f5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f4bfff640 1 -- 172.21.15.53:0/3640644141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580074e0 con 0x7f5f6415f8c0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.397+0000 7f5f4bfff640 1 -- 172.21.15.53:0/3640644141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f580047e0 con 0x7f5f6415f8c0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.398+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f640c7b80 con 0x7f5f6415f8c0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.398+0000 7f5f4bfff640 1 -- 172.21.15.53:0/3640644141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58004c10 con 0x7f5f6415f8c0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.398+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f640c70a0 con 0x7f5f6415f8c0 2024-05-10T10:33:48.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.399+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f640c7870 con 0x7f5f6415f8c0 2024-05-10T10:33:48.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.399+0000 7f5f4bfff640 1 -- 172.21.15.53:0/3640644141 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5f58011070 con 0x7f5f6415f8c0 2024-05-10T10:33:48.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.400+0000 7f5f4bfff640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f500762d0 0x7f5f50078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.400+0000 7f5f4bfff640 1 -- 172.21.15.53:0/3640644141 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5f58097410 con 0x7f5f6415f8c0 2024-05-10T10:33:48.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.400+0000 7f5f637fe640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f500762d0 0x7f5f50078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.400+0000 7f5f637fe640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f500762d0 0x7f5f50078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f640b4320 tx=0x7f5f54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.404+0000 7f5f4bfff640 1 -- 172.21.15.53:0/3640644141 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5f58060ac0 con 0x7f5f6415f8c0 2024-05-10T10:33:48.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.590+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f6406a220 con 0x7f5f500762d0 2024-05-10T10:33:48.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.600+0000 7f5f4bfff640 1 -- 172.21.15.53:0/3640644141 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332631 (secure 0 0 0) 0x7f5f6406a220 con 0x7f5f500762d0 2024-05-10T10:33:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.603+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f500762d0 msgr2=0x7f5f50078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.603+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f500762d0 0x7f5f50078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f640b4320 tx=0x7f5f54009040 comp rx=0 tx=0).stop 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.603+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f6415f8c0 msgr2=0x7f5f640c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.603+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f6415f8c0 0x7f5f640c8c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c430 tx=0x7f5f5800c8f0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 shutdown_connections 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5f500762d0 0x7f5f50078790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5f64168220 0x7f5f640c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f6415f8c0 0x7f5f640c8c60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 --2- 172.21.15.53:0/3640644141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6415f4e0 0x7f5f64159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 >> 172.21.15.53:0/3640644141 conn(0x7f5f640d2df0 msgr2=0x7f5f64071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 shutdown_connections 2024-05-10T10:33:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.604+0000 7f5f69e6b640 1 -- 172.21.15.53:0/3640644141 wait complete. 2024-05-10T10:33:48.616 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-05-10T10:33:48.159218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":313,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236040,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738680,"statfs":{"total":773094113280,"available":772852408320,"internally_reserved":0,"allocated":3284992,"data_stored":1987604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.644959"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553612+0000","last_change":"2024-05-10T10:33:33.499661+0000","last_active":"2024-05-10T10:33:39.553612+0000","last_peered":"2024-05-10T10:33:39.553612+0000","last_clean":"2024-05-10T10:33:39.553612+0000","last_became_active":"2024-05-10T10:33:33.499370+0000","last_became_peered":"2024-05-10T10:33:33.499370+0000","last_unstale":"2024-05-10T10:33:39.553612+0000","last_undegraded":"2024-05-10T10:33:39.553612+0000","last_fullsized":"2024-05-10T10:33:39.553612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:01:06.424327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641011+0000","last_change":"2024-05-10T10:33:34.515514+0000","last_active":"2024-05-10T10:33:39.641011+0000","last_peered":"2024-05-10T10:33:39.641011+0000","last_clean":"2024-05-10T10:33:39.641011+0000","last_became_active":"2024-05-10T10:33:34.514965+0000","last_became_peered":"2024-05-10T10:33:34.514965+0000","last_unstale":"2024-05-10T10:33:39.641011+0000","last_undegraded":"2024-05-10T10:33:39.641011+0000","last_fullsized":"2024-05-10T10:33:39.641011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640231+0000","last_change":"2024-05-10T10:33:38.535811+0000","last_active":"2024-05-10T10:33:39.640231+0000","last_peered":"2024-05-10T10:33:39.640231+0000","last_clean":"2024-05-10T10:33:39.640231+0000","last_became_active":"2024-05-10T10:33:38.535356+0000","last_became_peered":"2024-05-10T10:33:38.535356+0000","last_unstale":"2024-05-10T10:33:39.640231+0000","last_undegraded":"2024-05-10T10:33:39.640231+0000","last_fullsized":"2024-05-10T10:33:39.640231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641122+0000","last_change":"2024-05-10T10:33:33.494355+0000","last_active":"2024-05-10T10:33:39.641122+0000","last_peered":"2024-05-10T10:33:39.641122+0000","last_clean":"2024-05-10T10:33:39.641122+0000","last_became_active":"2024-05-10T10:33:33.494042+0000","last_became_peered":"2024-05-10T10:33:33.494042+0000","last_unstale":"2024-05-10T10:33:39.641122+0000","last_undegraded":"2024-05-10T10:33:39.641122+0000","last_fullsized":"2024-05-10T10:33:39.641122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553336+0000","last_change":"2024-05-10T10:33:34.514217+0000","last_active":"2024-05-10T10:33:39.553336+0000","last_peered":"2024-05-10T10:33:39.553336+0000","last_clean":"2024-05-10T10:33:39.553336+0000","last_became_active":"2024-05-10T10:33:34.513895+0000","last_became_peered":"2024-05-10T10:33:34.513895+0000","last_unstale":"2024-05-10T10:33:39.553336+0000","last_undegraded":"2024-05-10T10:33:39.553336+0000","last_fullsized":"2024-05-10T10:33:39.553336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872369+0000","last_change":"2024-05-10T10:33:38.542469+0000","last_active":"2024-05-10T10:33:40.872369+0000","last_peered":"2024-05-10T10:33:40.872369+0000","last_clean":"2024-05-10T10:33:40.872369+0000","last_became_active":"2024-05-10T10:33:38.541985+0000","last_became_peered":"2024-05-10T10:33:38.541985+0000","last_unstale":"2024-05-10T10:33:40.872369+0000","last_undegraded":"2024-05-10T10:33:40.872369+0000","last_fullsized":"2024-05-10T10:33:40.872369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:30.304981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554581+0000","last_change":"2024-05-10T10:33:33.500811+0000","last_active":"2024-05-10T10:33:39.554581+0000","last_peered":"2024-05-10T10:33:39.554581+0000","last_clean":"2024-05-10T10:33:39.554581+0000","last_became_active":"2024-05-10T10:33:33.500355+0000","last_became_peered":"2024-05-10T10:33:33.500355+0000","last_unstale":"2024-05-10T10:33:39.554581+0000","last_undegraded":"2024-05-10T10:33:39.554581+0000","last_fullsized":"2024-05-10T10:33:39.554581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871828+0000","last_change":"2024-05-10T10:33:34.502619+0000","last_active":"2024-05-10T10:33:40.871828+0000","last_peered":"2024-05-10T10:33:40.871828+0000","last_clean":"2024-05-10T10:33:40.871828+0000","last_became_active":"2024-05-10T10:33:34.500045+0000","last_became_peered":"2024-05-10T10:33:34.500045+0000","last_unstale":"2024-05-10T10:33:40.871828+0000","last_undegraded":"2024-05-10T10:33:40.871828+0000","last_fullsized":"2024-05-10T10:33:40.871828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872455+0000","last_change":"2024-05-10T10:33:38.543336+0000","last_active":"2024-05-10T10:33:40.872455+0000","last_peered":"2024-05-10T10:33:40.872455+0000","last_clean":"2024-05-10T10:33:40.872455+0000","last_became_active":"2024-05-10T10:33:38.542915+0000","last_became_peered":"2024-05-10T10:33:38.542915+0000","last_unstale":"2024-05-10T10:33:40.872455+0000","last_undegraded":"2024-05-10T10:33:40.872455+0000","last_fullsized":"2024-05-10T10:33:40.872455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:16.959519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553329+0000","last_change":"2024-05-10T10:33:33.499213+0000","last_active":"2024-05-10T10:33:39.553329+0000","last_peered":"2024-05-10T10:33:39.553329+0000","last_clean":"2024-05-10T10:33:39.553329+0000","last_became_active":"2024-05-10T10:33:33.497831+0000","last_became_peered":"2024-05-10T10:33:33.497831+0000","last_unstale":"2024-05-10T10:33:39.553329+0000","last_undegraded":"2024-05-10T10:33:39.553329+0000","last_fullsized":"2024-05-10T10:33:39.553329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:12.309888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871793+0000","last_change":"2024-05-10T10:33:34.508608+0000","last_active":"2024-05-10T10:33:40.871793+0000","last_peered":"2024-05-10T10:33:40.871793+0000","last_clean":"2024-05-10T10:33:40.871793+0000","last_became_active":"2024-05-10T10:33:34.507711+0000","last_became_peered":"2024-05-10T10:33:34.507711+0000","last_unstale":"2024-05-10T10:33:40.871793+0000","last_undegraded":"2024-05-10T10:33:40.871793+0000","last_fullsized":"2024-05-10T10:33:40.871793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872480+0000","last_change":"2024-05-10T10:33:38.534041+0000","last_active":"2024-05-10T10:33:40.872480+0000","last_peered":"2024-05-10T10:33:40.872480+0000","last_clean":"2024-05-10T10:33:40.872480+0000","last_became_active":"2024-05-10T10:33:38.533397+0000","last_became_peered":"2024-05-10T10:33:38.533397+0000","last_unstale":"2024-05-10T10:33:40.872480+0000","last_undegraded":"2024-05-10T10:33:40.872480+0000","last_fullsized":"2024-05-10T10:33:40.872480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:43.886335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553281+0000","last_change":"2024-05-10T10:33:34.504671+0000","last_active":"2024-05-10T10:33:39.553281+0000","last_peered":"2024-05-10T10:33:39.553281+0000","last_clean":"2024-05-10T10:33:39.553281+0000","last_became_active":"2024-05-10T10:33:34.504389+0000","last_became_peered":"2024-05-10T10:33:34.504389+0000","last_unstale":"2024-05-10T10:33:39.553281+0000","last_undegraded":"2024-05-10T10:33:39.553281+0000","last_fullsized":"2024-05-10T10:33:39.553281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553953+0000","last_change":"2024-05-10T10:33:33.499466+0000","last_active":"2024-05-10T10:33:39.553953+0000","last_peered":"2024-05-10T10:33:39.553953+0000","last_clean":"2024-05-10T10:33:39.553953+0000","last_became_active":"2024-05-10T10:33:33.499185+0000","last_became_peered":"2024-05-10T10:33:33.499185+0000","last_unstale":"2024-05-10T10:33:39.553953+0000","last_undegraded":"2024-05-10T10:33:39.553953+0000","last_fullsized":"2024-05-10T10:33:39.553953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:40.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872905+0000","last_change":"2024-05-10T10:33:38.542258+0000","last_active":"2024-05-10T10:33:40.872905+0000","last_peered":"2024-05-10T10:33:40.872905+0000","last_clean":"2024-05-10T10:33:40.872905+0000","last_became_active":"2024-05-10T10:33:38.541808+0000","last_became_peered":"2024-05-10T10:33:38.541808+0000","last_unstale":"2024-05-10T10:33:40.872905+0000","last_undegraded":"2024-05-10T10:33:40.872905+0000","last_fullsized":"2024-05-10T10:33:40.872905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640629+0000","last_change":"2024-05-10T10:33:34.506055+0000","last_active":"2024-05-10T10:33:39.640629+0000","last_peered":"2024-05-10T10:33:39.640629+0000","last_clean":"2024-05-10T10:33:39.640629+0000","last_became_active":"2024-05-10T10:33:34.505770+0000","last_became_peered":"2024-05-10T10:33:34.505770+0000","last_unstale":"2024-05-10T10:33:39.640629+0000","last_undegraded":"2024-05-10T10:33:39.640629+0000","last_fullsized":"2024-05-10T10:33:39.640629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886170+0000","last_change":"2024-05-10T10:33:33.495139+0000","last_active":"2024-05-10T10:33:39.886170+0000","last_peered":"2024-05-10T10:33:39.886170+0000","last_clean":"2024-05-10T10:33:39.886170+0000","last_became_active":"2024-05-10T10:33:33.494836+0000","last_became_peered":"2024-05-10T10:33:33.494836+0000","last_unstale":"2024-05-10T10:33:39.886170+0000","last_undegraded":"2024-05-10T10:33:39.886170+0000","last_fullsized":"2024-05-10T10:33:39.886170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872645+0000","last_change":"2024-05-10T10:33:38.542825+0000","last_active":"2024-05-10T10:33:40.872645+0000","last_peered":"2024-05-10T10:33:40.872645+0000","last_clean":"2024-05-10T10:33:40.872645+0000","last_became_active":"2024-05-10T10:33:38.542378+0000","last_became_peered":"2024-05-10T10:33:38.542378+0000","last_unstale":"2024-05-10T10:33:40.872645+0000","last_undegraded":"2024-05-10T10:33:40.872645+0000","last_fullsized":"2024-05-10T10:33:40.872645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640415+0000","last_change":"2024-05-10T10:33:34.504517+0000","last_active":"2024-05-10T10:33:39.640415+0000","last_peered":"2024-05-10T10:33:39.640415+0000","last_clean":"2024-05-10T10:33:39.640415+0000","last_became_active":"2024-05-10T10:33:34.504168+0000","last_became_peered":"2024-05-10T10:33:34.504168+0000","last_unstale":"2024-05-10T10:33:39.640415+0000","last_undegraded":"2024-05-10T10:33:39.640415+0000","last_fullsized":"2024-05-10T10:33:39.640415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885550+0000","last_change":"2024-05-10T10:33:33.497956+0000","last_active":"2024-05-10T10:33:39.885550+0000","last_peered":"2024-05-10T10:33:39.885550+0000","last_clean":"2024-05-10T10:33:39.885550+0000","last_became_active":"2024-05-10T10:33:33.497234+0000","last_became_peered":"2024-05-10T10:33:33.497234+0000","last_unstale":"2024-05-10T10:33:39.885550+0000","last_undegraded":"2024-05-10T10:33:39.885550+0000","last_fullsized":"2024-05-10T10:33:39.885550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886149+0000","last_change":"2024-05-10T10:33:38.548654+0000","last_active":"2024-05-10T10:33:39.886149+0000","last_peered":"2024-05-10T10:33:39.886149+0000","last_clean":"2024-05-10T10:33:39.886149+0000","last_became_active":"2024-05-10T10:33:38.547914+0000","last_became_peered":"2024-05-10T10:33:38.547914+0000","last_unstale":"2024-05-10T10:33:39.886149+0000","last_undegraded":"2024-05-10T10:33:39.886149+0000","last_fullsized":"2024-05-10T10:33:39.886149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872730+0000","last_change":"2024-05-10T10:33:34.499454+0000","last_active":"2024-05-10T10:33:40.872730+0000","last_peered":"2024-05-10T10:33:40.872730+0000","last_clean":"2024-05-10T10:33:40.872730+0000","last_became_active":"2024-05-10T10:33:34.499154+0000","last_became_peered":"2024-05-10T10:33:34.499154+0000","last_unstale":"2024-05-10T10:33:40.872730+0000","last_undegraded":"2024-05-10T10:33:40.872730+0000","last_fullsized":"2024-05-10T10:33:40.872730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:06.330274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554557+0000","last_change":"2024-05-10T10:33:33.497112+0000","last_active":"2024-05-10T10:33:39.554557+0000","last_peered":"2024-05-10T10:33:39.554557+0000","last_clean":"2024-05-10T10:33:39.554557+0000","last_became_active":"2024-05-10T10:33:33.496737+0000","last_became_peered":"2024-05-10T10:33:33.496737+0000","last_unstale":"2024-05-10T10:33:39.554557+0000","last_undegraded":"2024-05-10T10:33:39.554557+0000","last_fullsized":"2024-05-10T10:33:39.554557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:20.939521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873091+0000","last_change":"2024-05-10T10:33:38.542337+0000","last_active":"2024-05-10T10:33:40.873091+0000","last_peered":"2024-05-10T10:33:40.873091+0000","last_clean":"2024-05-10T10:33:40.873091+0000","last_became_active":"2024-05-10T10:33:38.541897+0000","last_became_peered":"2024-05-10T10:33:38.541897+0000","last_unstale":"2024-05-10T10:33:40.873091+0000","last_undegraded":"2024-05-10T10:33:40.873091+0000","last_fullsized":"2024-05-10T10:33:40.873091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553701+0000","last_change":"2024-05-10T10:33:34.503858+0000","last_active":"2024-05-10T10:33:39.553701+0000","last_peered":"2024-05-10T10:33:39.553701+0000","last_clean":"2024-05-10T10:33:39.553701+0000","last_became_active":"2024-05-10T10:33:34.503581+0000","last_became_peered":"2024-05-10T10:33:34.503581+0000","last_unstale":"2024-05-10T10:33:39.553701+0000","last_undegraded":"2024-05-10T10:33:39.553701+0000","last_fullsized":"2024-05-10T10:33:39.553701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554729+0000","last_change":"2024-05-10T10:33:33.498121+0000","last_active":"2024-05-10T10:33:39.554729+0000","last_peered":"2024-05-10T10:33:39.554729+0000","last_clean":"2024-05-10T10:33:39.554729+0000","last_became_active":"2024-05-10T10:33:33.497834+0000","last_became_peered":"2024-05-10T10:33:33.497834+0000","last_unstale":"2024-05-10T10:33:39.554729+0000","last_undegraded":"2024-05-10T10:33:39.554729+0000","last_fullsized":"2024-05-10T10:33:39.554729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872705+0000","last_change":"2024-05-10T10:33:38.531480+0000","last_active":"2024-05-10T10:33:40.872705+0000","last_peered":"2024-05-10T10:33:40.872705+0000","last_clean":"2024-05-10T10:33:40.872705+0000","last_became_active":"2024-05-10T10:33:38.531023+0000","last_became_peered":"2024-05-10T10:33:38.531023+0000","last_unstale":"2024-05-10T10:33:40.872705+0000","last_undegraded":"2024-05-10T10:33:40.872705+0000","last_fullsized":"2024-05-10T10:33:40.872705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641080+0000","last_change":"2024-05-10T10:33:34.506192+0000","last_active":"2024-05-10T10:33:39.641080+0000","last_peered":"2024-05-10T10:33:39.641080+0000","last_clean":"2024-05-10T10:33:39.641080+0000","last_became_active":"2024-05-10T10:33:34.505914+0000","last_became_peered":"2024-05-10T10:33:34.505914+0000","last_unstale":"2024-05-10T10:33:39.641080+0000","last_undegraded":"2024-05-10T10:33:39.641080+0000","last_fullsized":"2024-05-10T10:33:39.641080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886029+0000","last_change":"2024-05-10T10:33:33.491388+0000","last_active":"2024-05-10T10:33:39.886029+0000","last_peered":"2024-05-10T10:33:39.886029+0000","last_clean":"2024-05-10T10:33:39.886029+0000","last_became_active":"2024-05-10T10:33:33.491061+0000","last_became_peered":"2024-05-10T10:33:33.491061+0000","last_unstale":"2024-05-10T10:33:39.886029+0000","last_undegraded":"2024-05-10T10:33:39.886029+0000","last_fullsized":"2024-05-10T10:33:39.886029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640139+0000","last_change":"2024-05-10T10:33:38.543260+0000","last_active":"2024-05-10T10:33:39.640139+0000","last_peered":"2024-05-10T10:33:39.640139+0000","last_clean":"2024-05-10T10:33:39.640139+0000","last_became_active":"2024-05-10T10:33:38.542885+0000","last_became_peered":"2024-05-10T10:33:38.542885+0000","last_unstale":"2024-05-10T10:33:39.640139+0000","last_undegraded":"2024-05-10T10:33:39.640139+0000","last_fullsized":"2024-05-10T10:33:39.640139+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886114+0000","last_change":"2024-05-10T10:33:34.501069+0000","last_active":"2024-05-10T10:33:39.886114+0000","last_peered":"2024-05-10T10:33:39.886114+0000","last_clean":"2024-05-10T10:33:39.886114+0000","last_became_active":"2024-05-10T10:33:34.500767+0000","last_became_peered":"2024-05-10T10:33:34.500767+0000","last_unstale":"2024-05-10T10:33:39.886114+0000","last_undegraded":"2024-05-10T10:33:39.886114+0000","last_fullsized":"2024-05-10T10:33:39.886114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554556+0000","last_change":"2024-05-10T10:33:33.496126+0000","last_active":"2024-05-10T10:33:39.554556+0000","last_peered":"2024-05-10T10:33:39.554556+0000","last_clean":"2024-05-10T10:33:39.554556+0000","last_became_active":"2024-05-10T10:33:33.495848+0000","last_became_peered":"2024-05-10T10:33:33.495848+0000","last_unstale":"2024-05-10T10:33:39.554556+0000","last_undegraded":"2024-05-10T10:33:39.554556+0000","last_fullsized":"2024-05-10T10:33:39.554556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553188+0000","last_change":"2024-05-10T10:33:38.542612+0000","last_active":"2024-05-10T10:33:39.553188+0000","last_peered":"2024-05-10T10:33:39.553188+0000","last_clean":"2024-05-10T10:33:39.553188+0000","last_became_active":"2024-05-10T10:33:38.542110+0000","last_became_peered":"2024-05-10T10:33:38.542110+0000","last_unstale":"2024-05-10T10:33:39.553188+0000","last_undegraded":"2024-05-10T10:33:39.553188+0000","last_fullsized":"2024-05-10T10:33:39.553188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871909+0000","last_change":"2024-05-10T10:33:34.515283+0000","last_active":"2024-05-10T10:33:40.871909+0000","last_peered":"2024-05-10T10:33:40.871909+0000","last_clean":"2024-05-10T10:33:40.871909+0000","last_became_active":"2024-05-10T10:33:34.515016+0000","last_became_peered":"2024-05-10T10:33:34.515016+0000","last_unstale":"2024-05-10T10:33:40.871909+0000","last_undegraded":"2024-05-10T10:33:40.871909+0000","last_fullsized":"2024-05-10T10:33:40.871909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886059+0000","last_change":"2024-05-10T10:33:33.488825+0000","last_active":"2024-05-10T10:33:39.886059+0000","last_peered":"2024-05-10T10:33:39.886059+0000","last_clean":"2024-05-10T10:33:39.886059+0000","last_became_active":"2024-05-10T10:33:33.488539+0000","last_became_peered":"2024-05-10T10:33:33.488539+0000","last_unstale":"2024-05-10T10:33:39.886059+0000","last_undegraded":"2024-05-10T10:33:39.886059+0000","last_fullsized":"2024-05-10T10:33:39.886059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885597+0000","last_change":"2024-05-10T10:33:38.547163+0000","last_active":"2024-05-10T10:33:39.885597+0000","last_peered":"2024-05-10T10:33:39.885597+0000","last_clean":"2024-05-10T10:33:39.885597+0000","last_became_active":"2024-05-10T10:33:38.546692+0000","last_became_peered":"2024-05-10T10:33:38.546692+0000","last_unstale":"2024-05-10T10:33:39.885597+0000","last_undegraded":"2024-05-10T10:33:39.885597+0000","last_fullsized":"2024-05-10T10:33:39.885597+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872867+0000","last_change":"2024-05-10T10:33:34.505921+0000","last_active":"2024-05-10T10:33:40.872867+0000","last_peered":"2024-05-10T10:33:40.872867+0000","last_clean":"2024-05-10T10:33:40.872867+0000","last_became_active":"2024-05-10T10:33:34.505637+0000","last_became_peered":"2024-05-10T10:33:34.505637+0000","last_unstale":"2024-05-10T10:33:40.872867+0000","last_undegraded":"2024-05-10T10:33:40.872867+0000","last_fullsized":"2024-05-10T10:33:40.872867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871754+0000","last_change":"2024-05-10T10:33:33.485382+0000","last_active":"2024-05-10T10:33:40.871754+0000","last_peered":"2024-05-10T10:33:40.871754+0000","last_clean":"2024-05-10T10:33:40.871754+0000","last_became_active":"2024-05-10T10:33:33.485082+0000","last_became_peered":"2024-05-10T10:33:33.485082+0000","last_unstale":"2024-05-10T10:33:40.871754+0000","last_undegraded":"2024-05-10T10:33:40.871754+0000","last_fullsized":"2024-05-10T10:33:40.871754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885955+0000","last_change":"2024-05-10T10:33:38.541426+0000","last_active":"2024-05-10T10:33:39.885955+0000","last_peered":"2024-05-10T10:33:39.885955+0000","last_clean":"2024-05-10T10:33:39.885955+0000","last_became_active":"2024-05-10T10:33:38.539367+0000","last_became_peered":"2024-05-10T10:33:38.539367+0000","last_unstale":"2024-05-10T10:33:39.885955+0000","last_undegraded":"2024-05-10T10:33:39.885955+0000","last_fullsized":"2024-05-10T10:33:39.885955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885593+0000","last_change":"2024-05-10T10:33:34.501862+0000","last_active":"2024-05-10T10:33:39.885593+0000","last_peered":"2024-05-10T10:33:39.885593+0000","last_clean":"2024-05-10T10:33:39.885593+0000","last_became_active":"2024-05-10T10:33:34.501493+0000","last_became_peered":"2024-05-10T10:33:34.501493+0000","last_unstale":"2024-05-10T10:33:39.885593+0000","last_undegraded":"2024-05-10T10:33:39.885593+0000","last_fullsized":"2024-05-10T10:33:39.885593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679733+0000","last_change":"2024-05-10T10:33:33.495291+0000","last_active":"2024-05-10T10:33:39.679733+0000","last_peered":"2024-05-10T10:33:39.679733+0000","last_clean":"2024-05-10T10:33:39.679733+0000","last_became_active":"2024-05-10T10:33:33.494955+0000","last_became_peered":"2024-05-10T10:33:33.494955+0000","last_unstale":"2024-05-10T10:33:39.679733+0000","last_undegraded":"2024-05-10T10:33:39.679733+0000","last_fullsized":"2024-05-10T10:33:39.679733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886190+0000","last_change":"2024-05-10T10:33:38.539560+0000","last_active":"2024-05-10T10:33:39.886190+0000","last_peered":"2024-05-10T10:33:39.886190+0000","last_clean":"2024-05-10T10:33:39.886190+0000","last_became_active":"2024-05-10T10:33:38.539131+0000","last_became_peered":"2024-05-10T10:33:38.539131+0000","last_unstale":"2024-05-10T10:33:39.886190+0000","last_undegraded":"2024-05-10T10:33:39.886190+0000","last_fullsized":"2024-05-10T10:33:39.886190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552886+0000","last_change":"2024-05-10T10:33:34.510125+0000","last_active":"2024-05-10T10:33:39.552886+0000","last_peered":"2024-05-10T10:33:39.552886+0000","last_clean":"2024-05-10T10:33:39.552886+0000","last_became_active":"2024-05-10T10:33:34.509820+0000","last_became_peered":"2024-05-10T10:33:34.509820+0000","last_unstale":"2024-05-10T10:33:39.552886+0000","last_undegraded":"2024-05-10T10:33:39.552886+0000","last_fullsized":"2024-05-10T10:33:39.552886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554626+0000","last_change":"2024-05-10T10:33:33.499197+0000","last_active":"2024-05-10T10:33:39.554626+0000","last_peered":"2024-05-10T10:33:39.554626+0000","last_clean":"2024-05-10T10:33:39.554626+0000","last_became_active":"2024-05-10T10:33:33.497808+0000","last_became_peered":"2024-05-10T10:33:33.497808+0000","last_unstale":"2024-05-10T10:33:39.554626+0000","last_undegraded":"2024-05-10T10:33:39.554626+0000","last_fullsized":"2024-05-10T10:33:39.554626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:11.901185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553441+0000","last_change":"2024-05-10T10:33:38.544906+0000","last_active":"2024-05-10T10:33:39.553441+0000","last_peered":"2024-05-10T10:33:39.553441+0000","last_clean":"2024-05-10T10:33:39.553441+0000","last_became_active":"2024-05-10T10:33:38.544304+0000","last_became_peered":"2024-05-10T10:33:38.544304+0000","last_unstale":"2024-05-10T10:33:39.553441+0000","last_undegraded":"2024-05-10T10:33:39.553441+0000","last_fullsized":"2024-05-10T10:33:39.553441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640227+0000","last_change":"2024-05-10T10:33:34.513886+0000","last_active":"2024-05-10T10:33:39.640227+0000","last_peered":"2024-05-10T10:33:39.640227+0000","last_clean":"2024-05-10T10:33:39.640227+0000","last_became_active":"2024-05-10T10:33:34.513617+0000","last_became_peered":"2024-05-10T10:33:34.513617+0000","last_unstale":"2024-05-10T10:33:39.640227+0000","last_undegraded":"2024-05-10T10:33:39.640227+0000","last_fullsized":"2024-05-10T10:33:39.640227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871616+0000","last_change":"2024-05-10T10:33:33.489088+0000","last_active":"2024-05-10T10:33:40.871616+0000","last_peered":"2024-05-10T10:33:40.871616+0000","last_clean":"2024-05-10T10:33:40.871616+0000","last_became_active":"2024-05-10T10:33:33.488785+0000","last_became_peered":"2024-05-10T10:33:33.488785+0000","last_unstale":"2024-05-10T10:33:40.871616+0000","last_undegraded":"2024-05-10T10:33:40.871616+0000","last_fullsized":"2024-05-10T10:33:40.871616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:06:14.764873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554687+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:33:39.554687+0000","last_peered":"2024-05-10T10:33:39.554687+0000","last_clean":"2024-05-10T10:33:39.554687+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:33:39.554687+0000","last_undegraded":"2024-05-10T10:33:39.554687+0000","last_fullsized":"2024-05-10T10:33:39.554687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553037+0000","last_change":"2024-05-10T10:33:38.538561+0000","last_active":"2024-05-10T10:33:39.553037+0000","last_peered":"2024-05-10T10:33:39.553037+0000","last_clean":"2024-05-10T10:33:39.553037+0000","last_became_active":"2024-05-10T10:33:38.538159+0000","last_became_peered":"2024-05-10T10:33:38.538159+0000","last_unstale":"2024-05-10T10:33:39.553037+0000","last_undegraded":"2024-05-10T10:33:39.553037+0000","last_fullsized":"2024-05-10T10:33:39.553037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554231+0000","last_change":"2024-05-10T10:33:34.517851+0000","last_active":"2024-05-10T10:33:39.554231+0000","last_peered":"2024-05-10T10:33:39.554231+0000","last_clean":"2024-05-10T10:33:39.554231+0000","last_became_active":"2024-05-10T10:33:34.517556+0000","last_became_peered":"2024-05-10T10:33:34.517556+0000","last_unstale":"2024-05-10T10:33:39.554231+0000","last_undegraded":"2024-05-10T10:33:39.554231+0000","last_fullsized":"2024-05-10T10:33:39.554231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:46.690299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680077+0000","last_change":"2024-05-10T10:33:33.488242+0000","last_active":"2024-05-10T10:33:39.680077+0000","last_peered":"2024-05-10T10:33:39.680077+0000","last_clean":"2024-05-10T10:33:39.680077+0000","last_became_active":"2024-05-10T10:33:33.487932+0000","last_became_peered":"2024-05-10T10:33:33.487932+0000","last_unstale":"2024-05-10T10:33:39.680077+0000","last_undegraded":"2024-05-10T10:33:39.680077+0000","last_fullsized":"2024-05-10T10:33:39.680077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552940+0000","last_change":"2024-05-10T10:33:38.534596+0000","last_active":"2024-05-10T10:33:39.552940+0000","last_peered":"2024-05-10T10:33:39.552940+0000","last_clean":"2024-05-10T10:33:39.552940+0000","last_became_active":"2024-05-10T10:33:38.534112+0000","last_became_peered":"2024-05-10T10:33:38.534112+0000","last_unstale":"2024-05-10T10:33:39.552940+0000","last_undegraded":"2024-05-10T10:33:39.552940+0000","last_fullsized":"2024-05-10T10:33:39.552940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871944+0000","last_change":"2024-05-10T10:33:34.514478+0000","last_active":"2024-05-10T10:33:40.871944+0000","last_peered":"2024-05-10T10:33:40.871944+0000","last_clean":"2024-05-10T10:33:40.871944+0000","last_became_active":"2024-05-10T10:33:34.514160+0000","last_became_peered":"2024-05-10T10:33:34.514160+0000","last_unstale":"2024-05-10T10:33:40.871944+0000","last_undegraded":"2024-05-10T10:33:40.871944+0000","last_fullsized":"2024-05-10T10:33:40.871944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886005+0000","last_change":"2024-05-10T10:33:33.499523+0000","last_active":"2024-05-10T10:33:39.886005+0000","last_peered":"2024-05-10T10:33:39.886005+0000","last_clean":"2024-05-10T10:33:39.886005+0000","last_became_active":"2024-05-10T10:33:33.499250+0000","last_became_peered":"2024-05-10T10:33:33.499250+0000","last_unstale":"2024-05-10T10:33:39.886005+0000","last_undegraded":"2024-05-10T10:33:39.886005+0000","last_fullsized":"2024-05-10T10:33:39.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553375+0000","last_change":"2024-05-10T10:33:38.543483+0000","last_active":"2024-05-10T10:33:39.553375+0000","last_peered":"2024-05-10T10:33:39.553375+0000","last_clean":"2024-05-10T10:33:39.553375+0000","last_became_active":"2024-05-10T10:33:38.543060+0000","last_became_peered":"2024-05-10T10:33:38.543060+0000","last_unstale":"2024-05-10T10:33:39.553375+0000","last_undegraded":"2024-05-10T10:33:39.553375+0000","last_fullsized":"2024-05-10T10:33:39.553375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554354+0000","last_change":"2024-05-10T10:33:34.522046+0000","last_active":"2024-05-10T10:33:39.554354+0000","last_peered":"2024-05-10T10:33:39.554354+0000","last_clean":"2024-05-10T10:33:39.554354+0000","last_became_active":"2024-05-10T10:33:34.521739+0000","last_became_peered":"2024-05-10T10:33:34.521739+0000","last_unstale":"2024-05-10T10:33:39.554354+0000","last_undegraded":"2024-05-10T10:33:39.554354+0000","last_fullsized":"2024-05-10T10:33:39.554354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:15.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680220+0000","last_change":"2024-05-10T10:33:33.495634+0000","last_active":"2024-05-10T10:33:39.680220+0000","last_peered":"2024-05-10T10:33:39.680220+0000","last_clean":"2024-05-10T10:33:39.680220+0000","last_became_active":"2024-05-10T10:33:33.494432+0000","last_became_peered":"2024-05-10T10:33:33.494432+0000","last_unstale":"2024-05-10T10:33:39.680220+0000","last_undegraded":"2024-05-10T10:33:39.680220+0000","last_fullsized":"2024-05-10T10:33:39.680220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871284+0000","last_change":"2024-05-10T10:33:38.528059+0000","last_active":"2024-05-10T10:33:40.871284+0000","last_peered":"2024-05-10T10:33:40.871284+0000","last_clean":"2024-05-10T10:33:40.871284+0000","last_became_active":"2024-05-10T10:33:38.527628+0000","last_became_peered":"2024-05-10T10:33:38.527628+0000","last_unstale":"2024-05-10T10:33:40.871284+0000","last_undegraded":"2024-05-10T10:33:40.871284+0000","last_fullsized":"2024-05-10T10:33:40.871284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554210+0000","last_change":"2024-05-10T10:33:34.519810+0000","last_active":"2024-05-10T10:33:39.554210+0000","last_peered":"2024-05-10T10:33:39.554210+0000","last_clean":"2024-05-10T10:33:39.554210+0000","last_became_active":"2024-05-10T10:33:34.519517+0000","last_became_peered":"2024-05-10T10:33:34.519517+0000","last_unstale":"2024-05-10T10:33:39.554210+0000","last_undegraded":"2024-05-10T10:33:39.554210+0000","last_fullsized":"2024-05-10T10:33:39.554210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873138+0000","last_change":"2024-05-10T10:33:33.494974+0000","last_active":"2024-05-10T10:33:40.873138+0000","last_peered":"2024-05-10T10:33:40.873138+0000","last_clean":"2024-05-10T10:33:40.873138+0000","last_became_active":"2024-05-10T10:33:33.494667+0000","last_became_peered":"2024-05-10T10:33:33.494667+0000","last_unstale":"2024-05-10T10:33:40.873138+0000","last_undegraded":"2024-05-10T10:33:40.873138+0000","last_fullsized":"2024-05-10T10:33:40.873138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:54.706371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872313+0000","last_change":"2024-05-10T10:33:38.542343+0000","last_active":"2024-05-10T10:33:40.872313+0000","last_peered":"2024-05-10T10:33:40.872313+0000","last_clean":"2024-05-10T10:33:40.872313+0000","last_became_active":"2024-05-10T10:33:38.541842+0000","last_became_peered":"2024-05-10T10:33:38.541842+0000","last_unstale":"2024-05-10T10:33:40.872313+0000","last_undegraded":"2024-05-10T10:33:40.872313+0000","last_fullsized":"2024-05-10T10:33:40.872313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:32.462562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553822+0000","last_change":"2024-05-10T10:33:34.520528+0000","last_active":"2024-05-10T10:33:39.553822+0000","last_peered":"2024-05-10T10:33:39.553822+0000","last_clean":"2024-05-10T10:33:39.553822+0000","last_became_active":"2024-05-10T10:33:34.520239+0000","last_became_peered":"2024-05-10T10:33:34.520239+0000","last_unstale":"2024-05-10T10:33:39.553822+0000","last_undegraded":"2024-05-10T10:33:39.553822+0000","last_fullsized":"2024-05-10T10:33:39.553822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679540+0000","last_change":"2024-05-10T10:33:33.492349+0000","last_active":"2024-05-10T10:33:39.679540+0000","last_peered":"2024-05-10T10:33:39.679540+0000","last_clean":"2024-05-10T10:33:39.679540+0000","last_became_active":"2024-05-10T10:33:33.492019+0000","last_became_peered":"2024-05-10T10:33:33.492019+0000","last_unstale":"2024-05-10T10:33:39.679540+0000","last_undegraded":"2024-05-10T10:33:39.679540+0000","last_fullsized":"2024-05-10T10:33:39.679540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553319+0000","last_change":"2024-05-10T10:33:38.538548+0000","last_active":"2024-05-10T10:33:39.553319+0000","last_peered":"2024-05-10T10:33:39.553319+0000","last_clean":"2024-05-10T10:33:39.553319+0000","last_became_active":"2024-05-10T10:33:38.538144+0000","last_became_peered":"2024-05-10T10:33:38.538144+0000","last_unstale":"2024-05-10T10:33:39.553319+0000","last_undegraded":"2024-05-10T10:33:39.553319+0000","last_fullsized":"2024-05-10T10:33:39.553319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553004+0000","last_change":"2024-05-10T10:33:34.510224+0000","last_active":"2024-05-10T10:33:39.553004+0000","last_peered":"2024-05-10T10:33:39.553004+0000","last_clean":"2024-05-10T10:33:39.553004+0000","last_became_active":"2024-05-10T10:33:34.509584+0000","last_became_peered":"2024-05-10T10:33:34.509584+0000","last_unstale":"2024-05-10T10:33:39.553004+0000","last_undegraded":"2024-05-10T10:33:39.553004+0000","last_fullsized":"2024-05-10T10:33:39.553004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553209+0000","last_change":"2024-05-10T10:33:33.496606+0000","last_active":"2024-05-10T10:33:39.553209+0000","last_peered":"2024-05-10T10:33:39.553209+0000","last_clean":"2024-05-10T10:33:39.553209+0000","last_became_active":"2024-05-10T10:33:33.496326+0000","last_became_peered":"2024-05-10T10:33:33.496326+0000","last_unstale":"2024-05-10T10:33:39.553209+0000","last_undegraded":"2024-05-10T10:33:39.553209+0000","last_fullsized":"2024-05-10T10:33:39.553209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641045+0000","last_change":"2024-05-10T10:33:38.539948+0000","last_active":"2024-05-10T10:33:39.641045+0000","last_peered":"2024-05-10T10:33:39.641045+0000","last_clean":"2024-05-10T10:33:39.641045+0000","last_became_active":"2024-05-10T10:33:38.539181+0000","last_became_peered":"2024-05-10T10:33:38.539181+0000","last_unstale":"2024-05-10T10:33:39.641045+0000","last_undegraded":"2024-05-10T10:33:39.641045+0000","last_fullsized":"2024-05-10T10:33:39.641045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:03.969211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554079+0000","last_change":"2024-05-10T10:33:34.519826+0000","last_active":"2024-05-10T10:33:39.554079+0000","last_peered":"2024-05-10T10:33:39.554079+0000","last_clean":"2024-05-10T10:33:39.554079+0000","last_became_active":"2024-05-10T10:33:34.519540+0000","last_became_peered":"2024-05-10T10:33:34.519540+0000","last_unstale":"2024-05-10T10:33:39.554079+0000","last_undegraded":"2024-05-10T10:33:39.554079+0000","last_fullsized":"2024-05-10T10:33:39.554079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871721+0000","last_change":"2024-05-10T10:33:33.494936+0000","last_active":"2024-05-10T10:33:40.871721+0000","last_peered":"2024-05-10T10:33:40.871721+0000","last_clean":"2024-05-10T10:33:40.871721+0000","last_became_active":"2024-05-10T10:33:33.493154+0000","last_became_peered":"2024-05-10T10:33:33.493154+0000","last_unstale":"2024-05-10T10:33:40.871721+0000","last_undegraded":"2024-05-10T10:33:40.871721+0000","last_fullsized":"2024-05-10T10:33:40.871721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640822+0000","last_change":"2024-05-10T10:33:38.544918+0000","last_active":"2024-05-10T10:33:39.640822+0000","last_peered":"2024-05-10T10:33:39.640822+0000","last_clean":"2024-05-10T10:33:39.640822+0000","last_became_active":"2024-05-10T10:33:38.544423+0000","last_became_peered":"2024-05-10T10:33:38.544423+0000","last_unstale":"2024-05-10T10:33:39.640822+0000","last_undegraded":"2024-05-10T10:33:39.640822+0000","last_fullsized":"2024-05-10T10:33:39.640822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:22.017984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553482+0000","last_change":"2024-05-10T10:33:34.506856+0000","last_active":"2024-05-10T10:33:39.553482+0000","last_peered":"2024-05-10T10:33:39.553482+0000","last_clean":"2024-05-10T10:33:39.553482+0000","last_became_active":"2024-05-10T10:33:34.506563+0000","last_became_peered":"2024-05-10T10:33:34.506563+0000","last_unstale":"2024-05-10T10:33:39.553482+0000","last_undegraded":"2024-05-10T10:33:39.553482+0000","last_fullsized":"2024-05-10T10:33:39.553482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:21.944467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553658+0000","last_change":"2024-05-10T10:33:33.496358+0000","last_active":"2024-05-10T10:33:39.553658+0000","last_peered":"2024-05-10T10:33:39.553658+0000","last_clean":"2024-05-10T10:33:39.553658+0000","last_became_active":"2024-05-10T10:33:33.495968+0000","last_became_peered":"2024-05-10T10:33:33.495968+0000","last_unstale":"2024-05-10T10:33:39.553658+0000","last_undegraded":"2024-05-10T10:33:39.553658+0000","last_fullsized":"2024-05-10T10:33:39.553658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:14.817696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871851+0000","last_change":"2024-05-10T10:33:38.534046+0000","last_active":"2024-05-10T10:33:40.871851+0000","last_peered":"2024-05-10T10:33:40.871851+0000","last_clean":"2024-05-10T10:33:40.871851+0000","last_became_active":"2024-05-10T10:33:38.533660+0000","last_became_peered":"2024-05-10T10:33:38.533660+0000","last_unstale":"2024-05-10T10:33:40.871851+0000","last_undegraded":"2024-05-10T10:33:40.871851+0000","last_fullsized":"2024-05-10T10:33:40.871851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554110+0000","last_change":"2024-05-10T10:33:34.516752+0000","last_active":"2024-05-10T10:33:39.554110+0000","last_peered":"2024-05-10T10:33:39.554110+0000","last_clean":"2024-05-10T10:33:39.554110+0000","last_became_active":"2024-05-10T10:33:34.516410+0000","last_became_peered":"2024-05-10T10:33:34.516410+0000","last_unstale":"2024-05-10T10:33:39.554110+0000","last_undegraded":"2024-05-10T10:33:39.554110+0000","last_fullsized":"2024-05-10T10:33:39.554110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553808+0000","last_change":"2024-05-10T10:33:33.498231+0000","last_active":"2024-05-10T10:33:39.553808+0000","last_peered":"2024-05-10T10:33:39.553808+0000","last_clean":"2024-05-10T10:33:39.553808+0000","last_became_active":"2024-05-10T10:33:33.497918+0000","last_became_peered":"2024-05-10T10:33:33.497918+0000","last_unstale":"2024-05-10T10:33:39.553808+0000","last_undegraded":"2024-05-10T10:33:39.553808+0000","last_fullsized":"2024-05-10T10:33:39.553808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885703+0000","last_change":"2024-05-10T10:33:38.537215+0000","last_active":"2024-05-10T10:33:39.885703+0000","last_peered":"2024-05-10T10:33:39.885703+0000","last_clean":"2024-05-10T10:33:39.885703+0000","last_became_active":"2024-05-10T10:33:38.536826+0000","last_became_peered":"2024-05-10T10:33:38.536826+0000","last_unstale":"2024-05-10T10:33:39.885703+0000","last_undegraded":"2024-05-10T10:33:39.885703+0000","last_fullsized":"2024-05-10T10:33:39.885703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872972+0000","last_change":"2024-05-10T10:33:34.512632+0000","last_active":"2024-05-10T10:33:40.872972+0000","last_peered":"2024-05-10T10:33:40.872972+0000","last_clean":"2024-05-10T10:33:40.872972+0000","last_became_active":"2024-05-10T10:33:34.512337+0000","last_became_peered":"2024-05-10T10:33:34.512337+0000","last_unstale":"2024-05-10T10:33:40.872972+0000","last_undegraded":"2024-05-10T10:33:40.872972+0000","last_fullsized":"2024-05-10T10:33:40.872972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:12.906132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885962+0000","last_change":"2024-05-10T10:33:33.485715+0000","last_active":"2024-05-10T10:33:39.885962+0000","last_peered":"2024-05-10T10:33:39.885962+0000","last_clean":"2024-05-10T10:33:39.885962+0000","last_became_active":"2024-05-10T10:33:33.485400+0000","last_became_peered":"2024-05-10T10:33:33.485400+0000","last_unstale":"2024-05-10T10:33:39.885962+0000","last_undegraded":"2024-05-10T10:33:39.885962+0000","last_fullsized":"2024-05-10T10:33:39.885962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872703+0000","last_change":"2024-05-10T10:33:38.542348+0000","last_active":"2024-05-10T10:33:40.872703+0000","last_peered":"2024-05-10T10:33:40.872703+0000","last_clean":"2024-05-10T10:33:40.872703+0000","last_became_active":"2024-05-10T10:33:38.541943+0000","last_became_peered":"2024-05-10T10:33:38.541943+0000","last_unstale":"2024-05-10T10:33:40.872703+0000","last_undegraded":"2024-05-10T10:33:40.872703+0000","last_fullsized":"2024-05-10T10:33:40.872703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553583+0000","last_change":"2024-05-10T10:33:34.514299+0000","last_active":"2024-05-10T10:33:39.553583+0000","last_peered":"2024-05-10T10:33:39.553583+0000","last_clean":"2024-05-10T10:33:39.553583+0000","last_became_active":"2024-05-10T10:33:34.513977+0000","last_became_peered":"2024-05-10T10:33:34.513977+0000","last_unstale":"2024-05-10T10:33:39.553583+0000","last_undegraded":"2024-05-10T10:33:39.553583+0000","last_fullsized":"2024-05-10T10:33:39.553583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872002+0000","last_change":"2024-05-10T10:33:33.491068+0000","last_active":"2024-05-10T10:33:40.872002+0000","last_peered":"2024-05-10T10:33:40.872002+0000","last_clean":"2024-05-10T10:33:40.872002+0000","last_became_active":"2024-05-10T10:33:33.490801+0000","last_became_peered":"2024-05-10T10:33:33.490801+0000","last_unstale":"2024-05-10T10:33:40.872002+0000","last_undegraded":"2024-05-10T10:33:40.872002+0000","last_fullsized":"2024-05-10T10:33:40.872002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640322+0000","last_change":"2024-05-10T10:33:38.538544+0000","last_active":"2024-05-10T10:33:39.640322+0000","last_peered":"2024-05-10T10:33:39.640322+0000","last_clean":"2024-05-10T10:33:39.640322+0000","last_became_active":"2024-05-10T10:33:38.538176+0000","last_became_peered":"2024-05-10T10:33:38.538176+0000","last_unstale":"2024-05-10T10:33:39.640322+0000","last_undegraded":"2024-05-10T10:33:39.640322+0000","last_fullsized":"2024-05-10T10:33:39.640322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:14.049514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871829+0000","last_change":"2024-05-10T10:33:34.513516+0000","last_active":"2024-05-10T10:33:40.871829+0000","last_peered":"2024-05-10T10:33:40.871829+0000","last_clean":"2024-05-10T10:33:40.871829+0000","last_became_active":"2024-05-10T10:33:34.512746+0000","last_became_peered":"2024-05-10T10:33:34.512746+0000","last_unstale":"2024-05-10T10:33:40.871829+0000","last_undegraded":"2024-05-10T10:33:40.871829+0000","last_fullsized":"2024-05-10T10:33:40.871829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554597+0000","last_change":"2024-05-10T10:33:33.498011+0000","last_active":"2024-05-10T10:33:39.554597+0000","last_peered":"2024-05-10T10:33:39.554597+0000","last_clean":"2024-05-10T10:33:39.554597+0000","last_became_active":"2024-05-10T10:33:33.497732+0000","last_became_peered":"2024-05-10T10:33:33.497732+0000","last_unstale":"2024-05-10T10:33:39.554597+0000","last_undegraded":"2024-05-10T10:33:39.554597+0000","last_fullsized":"2024-05-10T10:33:39.554597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640338+0000","last_change":"2024-05-10T10:33:38.537084+0000","last_active":"2024-05-10T10:33:39.640338+0000","last_peered":"2024-05-10T10:33:39.640338+0000","last_clean":"2024-05-10T10:33:39.640338+0000","last_became_active":"2024-05-10T10:33:38.535424+0000","last_became_peered":"2024-05-10T10:33:38.535424+0000","last_unstale":"2024-05-10T10:33:39.640338+0000","last_undegraded":"2024-05-10T10:33:39.640338+0000","last_fullsized":"2024-05-10T10:33:39.640338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:48:24.535051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553257+0000","last_change":"2024-05-10T10:33:38.545099+0000","last_active":"2024-05-10T10:33:39.553257+0000","last_peered":"2024-05-10T10:33:39.553257+0000","last_clean":"2024-05-10T10:33:39.553257+0000","last_became_active":"2024-05-10T10:33:38.544663+0000","last_became_peered":"2024-05-10T10:33:38.544663+0000","last_unstale":"2024-05-10T10:33:39.553257+0000","last_undegraded":"2024-05-10T10:33:39.553257+0000","last_fullsized":"2024-05-10T10:33:39.553257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885419+0000","last_change":"2024-05-10T10:33:34.505691+0000","last_active":"2024-05-10T10:33:39.885419+0000","last_peered":"2024-05-10T10:33:39.885419+0000","last_clean":"2024-05-10T10:33:39.885419+0000","last_became_active":"2024-05-10T10:33:34.505398+0000","last_became_peered":"2024-05-10T10:33:34.505398+0000","last_unstale":"2024-05-10T10:33:39.885419+0000","last_undegraded":"2024-05-10T10:33:39.885419+0000","last_fullsized":"2024-05-10T10:33:39.885419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871432+0000","last_change":"2024-05-10T10:33:33.494574+0000","last_active":"2024-05-10T10:33:40.871432+0000","last_peered":"2024-05-10T10:33:40.871432+0000","last_clean":"2024-05-10T10:33:40.871432+0000","last_became_active":"2024-05-10T10:33:33.494311+0000","last_became_peered":"2024-05-10T10:33:33.494311+0000","last_unstale":"2024-05-10T10:33:40.871432+0000","last_undegraded":"2024-05-10T10:33:40.871432+0000","last_fullsized":"2024-05-10T10:33:40.871432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554526+0000","last_change":"2024-05-10T10:33:38.537291+0000","last_active":"2024-05-10T10:33:39.554526+0000","last_peered":"2024-05-10T10:33:39.554526+0000","last_clean":"2024-05-10T10:33:39.554526+0000","last_became_active":"2024-05-10T10:33:38.536902+0000","last_became_peered":"2024-05-10T10:33:38.536902+0000","last_unstale":"2024-05-10T10:33:39.554526+0000","last_undegraded":"2024-05-10T10:33:39.554526+0000","last_fullsized":"2024-05-10T10:33:39.554526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640318+0000","last_change":"2024-05-10T10:33:34.505242+0000","last_active":"2024-05-10T10:33:39.640318+0000","last_peered":"2024-05-10T10:33:39.640318+0000","last_clean":"2024-05-10T10:33:39.640318+0000","last_became_active":"2024-05-10T10:33:34.504914+0000","last_became_peered":"2024-05-10T10:33:34.504914+0000","last_unstale":"2024-05-10T10:33:39.640318+0000","last_undegraded":"2024-05-10T10:33:39.640318+0000","last_fullsized":"2024-05-10T10:33:39.640318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640485+0000","last_change":"2024-05-10T10:33:33.498662+0000","last_active":"2024-05-10T10:33:39.640485+0000","last_peered":"2024-05-10T10:33:39.640485+0000","last_clean":"2024-05-10T10:33:39.640485+0000","last_became_active":"2024-05-10T10:33:33.498294+0000","last_became_peered":"2024-05-10T10:33:33.498294+0000","last_unstale":"2024-05-10T10:33:39.640485+0000","last_undegraded":"2024-05-10T10:33:39.640485+0000","last_fullsized":"2024-05-10T10:33:39.640485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553501+0000","last_change":"2024-05-10T10:33:33.495684+0000","last_active":"2024-05-10T10:33:39.553501+0000","last_peered":"2024-05-10T10:33:39.553501+0000","last_clean":"2024-05-10T10:33:39.553501+0000","last_became_active":"2024-05-10T10:33:33.495347+0000","last_became_peered":"2024-05-10T10:33:33.495347+0000","last_unstale":"2024-05-10T10:33:39.553501+0000","last_undegraded":"2024-05-10T10:33:39.553501+0000","last_fullsized":"2024-05-10T10:33:39.553501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553543+0000","last_change":"2024-05-10T10:33:34.502038+0000","last_active":"2024-05-10T10:33:39.553543+0000","last_peered":"2024-05-10T10:33:39.553543+0000","last_clean":"2024-05-10T10:33:39.553543+0000","last_became_active":"2024-05-10T10:33:34.501709+0000","last_became_peered":"2024-05-10T10:33:34.501709+0000","last_unstale":"2024-05-10T10:33:39.553543+0000","last_undegraded":"2024-05-10T10:33:39.553543+0000","last_fullsized":"2024-05-10T10:33:39.553543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679952+0000","last_change":"2024-05-10T10:33:38.536172+0000","last_active":"2024-05-10T10:33:39.679952+0000","last_peered":"2024-05-10T10:33:39.679952+0000","last_clean":"2024-05-10T10:33:39.679952+0000","last_became_active":"2024-05-10T10:33:38.535590+0000","last_became_peered":"2024-05-10T10:33:38.535590+0000","last_unstale":"2024-05-10T10:33:39.679952+0000","last_undegraded":"2024-05-10T10:33:39.679952+0000","last_fullsized":"2024-05-10T10:33:39.679952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:07.728619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640673+0000","last_change":"2024-05-10T10:33:33.497950+0000","last_active":"2024-05-10T10:33:39.640673+0000","last_peered":"2024-05-10T10:33:39.640673+0000","last_clean":"2024-05-10T10:33:39.640673+0000","last_became_active":"2024-05-10T10:33:33.497590+0000","last_became_peered":"2024-05-10T10:33:33.497590+0000","last_unstale":"2024-05-10T10:33:39.640673+0000","last_undegraded":"2024-05-10T10:33:39.640673+0000","last_fullsized":"2024-05-10T10:33:39.640673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872524+0000","last_change":"2024-05-10T10:33:34.508520+0000","last_active":"2024-05-10T10:33:40.872524+0000","last_peered":"2024-05-10T10:33:40.872524+0000","last_clean":"2024-05-10T10:33:40.872524+0000","last_became_active":"2024-05-10T10:33:34.508262+0000","last_became_peered":"2024-05-10T10:33:34.508262+0000","last_unstale":"2024-05-10T10:33:40.872524+0000","last_undegraded":"2024-05-10T10:33:40.872524+0000","last_fullsized":"2024-05-10T10:33:40.872524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:54.281454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640675+0000","last_change":"2024-05-10T10:33:38.542608+0000","last_active":"2024-05-10T10:33:39.640675+0000","last_peered":"2024-05-10T10:33:39.640675+0000","last_clean":"2024-05-10T10:33:39.640675+0000","last_became_active":"2024-05-10T10:33:38.540900+0000","last_became_peered":"2024-05-10T10:33:38.540900+0000","last_unstale":"2024-05-10T10:33:39.640675+0000","last_undegraded":"2024-05-10T10:33:39.640675+0000","last_fullsized":"2024-05-10T10:33:39.640675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:41.505822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552881+0000","last_change":"2024-05-10T10:33:34.510874+0000","last_active":"2024-05-10T10:33:39.552881+0000","last_peered":"2024-05-10T10:33:39.552881+0000","last_clean":"2024-05-10T10:33:39.552881+0000","last_became_active":"2024-05-10T10:33:34.510623+0000","last_became_peered":"2024-05-10T10:33:34.510623+0000","last_unstale":"2024-05-10T10:33:39.552881+0000","last_undegraded":"2024-05-10T10:33:39.552881+0000","last_fullsized":"2024-05-10T10:33:39.552881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679646+0000","last_change":"2024-05-10T10:33:34.503232+0000","last_active":"2024-05-10T10:33:39.679646+0000","last_peered":"2024-05-10T10:33:39.679646+0000","last_clean":"2024-05-10T10:33:39.679646+0000","last_became_active":"2024-05-10T10:33:34.502932+0000","last_became_peered":"2024-05-10T10:33:34.502932+0000","last_unstale":"2024-05-10T10:33:39.679646+0000","last_undegraded":"2024-05-10T10:33:39.679646+0000","last_fullsized":"2024-05-10T10:33:39.679646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871608+0000","last_change":"2024-05-10T10:33:34.516049+0000","last_active":"2024-05-10T10:33:40.871608+0000","last_peered":"2024-05-10T10:33:40.871608+0000","last_clean":"2024-05-10T10:33:40.871608+0000","last_became_active":"2024-05-10T10:33:34.515617+0000","last_became_peered":"2024-05-10T10:33:34.515617+0000","last_unstale":"2024-05-10T10:33:40.871608+0000","last_undegraded":"2024-05-10T10:33:40.871608+0000","last_fullsized":"2024-05-10T10:33:40.871608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553405+0000","last_change":"2024-05-10T10:33:34.508355+0000","last_active":"2024-05-10T10:33:39.553405+0000","last_peered":"2024-05-10T10:33:39.553405+0000","last_clean":"2024-05-10T10:33:39.553405+0000","last_became_active":"2024-05-10T10:33:34.508064+0000","last_became_peered":"2024-05-10T10:33:34.508064+0000","last_unstale":"2024-05-10T10:33:39.553405+0000","last_undegraded":"2024-05-10T10:33:39.553405+0000","last_fullsized":"2024-05-10T10:33:39.553405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:29.644124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640952+0000","last_change":"2024-05-10T10:33:34.508376+0000","last_active":"2024-05-10T10:33:39.640952+0000","last_peered":"2024-05-10T10:33:39.640952+0000","last_clean":"2024-05-10T10:33:39.640952+0000","last_became_active":"2024-05-10T10:33:34.507993+0000","last_became_peered":"2024-05-10T10:33:34.507993+0000","last_unstale":"2024-05-10T10:33:39.640952+0000","last_undegraded":"2024-05-10T10:33:39.640952+0000","last_fullsized":"2024-05-10T10:33:39.640952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885895+0000","last_change":"2024-05-10T10:33:34.508840+0000","last_active":"2024-05-10T10:33:39.885895+0000","last_peered":"2024-05-10T10:33:39.885895+0000","last_clean":"2024-05-10T10:33:39.885895+0000","last_became_active":"2024-05-10T10:33:34.508524+0000","last_became_peered":"2024-05-10T10:33:34.508524+0000","last_unstale":"2024-05-10T10:33:39.885895+0000","last_undegraded":"2024-05-10T10:33:39.885895+0000","last_fullsized":"2024-05-10T10:33:39.885895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554603+0000","last_change":"2024-05-10T10:33:34.513208+0000","last_active":"2024-05-10T10:33:39.554603+0000","last_peered":"2024-05-10T10:33:39.554603+0000","last_clean":"2024-05-10T10:33:39.554603+0000","last_became_active":"2024-05-10T10:33:34.512924+0000","last_became_peered":"2024-05-10T10:33:34.512924+0000","last_unstale":"2024-05-10T10:33:39.554603+0000","last_undegraded":"2024-05-10T10:33:39.554603+0000","last_fullsized":"2024-05-10T10:33:39.554603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641017+0000","last_change":"2024-05-10T10:33:34.504050+0000","last_active":"2024-05-10T10:33:39.641017+0000","last_peered":"2024-05-10T10:33:39.641017+0000","last_clean":"2024-05-10T10:33:39.641017+0000","last_became_active":"2024-05-10T10:33:34.503749+0000","last_became_peered":"2024-05-10T10:33:34.503749+0000","last_unstale":"2024-05-10T10:33:39.641017+0000","last_undegraded":"2024-05-10T10:33:39.641017+0000","last_fullsized":"2024-05-10T10:33:39.641017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871417+0000","last_change":"2024-05-10T10:33:34.508259+0000","last_active":"2024-05-10T10:33:40.871417+0000","last_peered":"2024-05-10T10:33:40.871417+0000","last_clean":"2024-05-10T10:33:40.871417+0000","last_became_active":"2024-05-10T10:33:34.507953+0000","last_became_peered":"2024-05-10T10:33:34.507953+0000","last_unstale":"2024-05-10T10:33:40.871417+0000","last_undegraded":"2024-05-10T10:33:40.871417+0000","last_fullsized":"2024-05-10T10:33:40.871417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640457+0000","last_change":"2024-05-10T10:33:34.505284+0000","last_active":"2024-05-10T10:33:39.640457+0000","last_peered":"2024-05-10T10:33:39.640457+0000","last_clean":"2024-05-10T10:33:39.640457+0000","last_became_active":"2024-05-10T10:33:34.505010+0000","last_became_peered":"2024-05-10T10:33:34.505010+0000","last_unstale":"2024-05-10T10:33:39.640457+0000","last_undegraded":"2024-05-10T10:33:39.640457+0000","last_fullsized":"2024-05-10T10:33:39.640457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871555+0000","last_change":"2024-05-10T10:33:34.511213+0000","last_active":"2024-05-10T10:33:40.871555+0000","last_peered":"2024-05-10T10:33:40.871555+0000","last_clean":"2024-05-10T10:33:40.871555+0000","last_became_active":"2024-05-10T10:33:34.509309+0000","last_became_peered":"2024-05-10T10:33:34.509309+0000","last_unstale":"2024-05-10T10:33:40.871555+0000","last_undegraded":"2024-05-10T10:33:40.871555+0000","last_fullsized":"2024-05-10T10:33:40.871555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553124+0000","last_change":"2024-05-10T10:33:34.508899+0000","last_active":"2024-05-10T10:33:39.553124+0000","last_peered":"2024-05-10T10:33:39.553124+0000","last_clean":"2024-05-10T10:33:39.553124+0000","last_became_active":"2024-05-10T10:33:34.508611+0000","last_became_peered":"2024-05-10T10:33:34.508611+0000","last_unstale":"2024-05-10T10:33:39.553124+0000","last_undegraded":"2024-05-10T10:33:39.553124+0000","last_fullsized":"2024-05-10T10:33:39.553124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:03.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553705+0000","last_change":"2024-05-10T10:33:34.520178+0000","last_active":"2024-05-10T10:33:39.553705+0000","last_peered":"2024-05-10T10:33:39.553705+0000","last_clean":"2024-05-10T10:33:39.553705+0000","last_became_active":"2024-05-10T10:33:34.519912+0000","last_became_peered":"2024-05-10T10:33:34.519912+0000","last_unstale":"2024-05-10T10:33:39.553705+0000","last_undegraded":"2024-05-10T10:33:39.553705+0000","last_fullsized":"2024-05-10T10:33:39.553705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554455+0000","last_change":"2024-05-10T10:33:34.515897+0000","last_active":"2024-05-10T10:33:39.554455+0000","last_peered":"2024-05-10T10:33:39.554455+0000","last_clean":"2024-05-10T10:33:39.554455+0000","last_became_active":"2024-05-10T10:33:34.514938+0000","last_became_peered":"2024-05-10T10:33:34.514938+0000","last_unstale":"2024-05-10T10:33:39.554455+0000","last_undegraded":"2024-05-10T10:33:39.554455+0000","last_fullsized":"2024-05-10T10:33:39.554455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:28:07.142173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680141+0000","last_change":"2024-05-10T10:33:34.523364+0000","last_active":"2024-05-10T10:33:39.680141+0000","last_peered":"2024-05-10T10:33:39.680141+0000","last_clean":"2024-05-10T10:33:39.680141+0000","last_became_active":"2024-05-10T10:33:34.522972+0000","last_became_peered":"2024-05-10T10:33:34.522972+0000","last_unstale":"2024-05-10T10:33:39.680141+0000","last_undegraded":"2024-05-10T10:33:39.680141+0000","last_fullsized":"2024-05-10T10:33:39.680141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872257+0000","last_change":"2024-05-10T10:33:34.508555+0000","last_active":"2024-05-10T10:33:40.872257+0000","last_peered":"2024-05-10T10:33:40.872257+0000","last_clean":"2024-05-10T10:33:40.872257+0000","last_became_active":"2024-05-10T10:33:34.508218+0000","last_became_peered":"2024-05-10T10:33:34.508218+0000","last_unstale":"2024-05-10T10:33:40.872257+0000","last_undegraded":"2024-05-10T10:33:40.872257+0000","last_fullsized":"2024-05-10T10:33:40.872257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871300+0000","last_change":"2024-05-10T10:33:34.512706+0000","last_active":"2024-05-10T10:33:40.871300+0000","last_peered":"2024-05-10T10:33:40.871300+0000","last_clean":"2024-05-10T10:33:40.871300+0000","last_became_active":"2024-05-10T10:33:34.511868+0000","last_became_peered":"2024-05-10T10:33:34.511868+0000","last_unstale":"2024-05-10T10:33:40.871300+0000","last_undegraded":"2024-05-10T10:33:40.871300+0000","last_fullsized":"2024-05-10T10:33:40.871300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885490+0000","last_change":"2024-05-10T10:33:34.504724+0000","last_active":"2024-05-10T10:33:39.885490+0000","last_peered":"2024-05-10T10:33:39.885490+0000","last_clean":"2024-05-10T10:33:39.885490+0000","last_became_active":"2024-05-10T10:33:34.504407+0000","last_became_peered":"2024-05-10T10:33:34.504407+0000","last_unstale":"2024-05-10T10:33:39.885490+0000","last_undegraded":"2024-05-10T10:33:39.885490+0000","last_fullsized":"2024-05-10T10:33:39.885490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553396+0000","last_change":"2024-05-10T10:33:34.507001+0000","last_active":"2024-05-10T10:33:39.553396+0000","last_peered":"2024-05-10T10:33:39.553396+0000","last_clean":"2024-05-10T10:33:39.553396+0000","last_became_active":"2024-05-10T10:33:34.506678+0000","last_became_peered":"2024-05-10T10:33:34.506678+0000","last_unstale":"2024-05-10T10:33:39.553396+0000","last_undegraded":"2024-05-10T10:33:39.553396+0000","last_fullsized":"2024-05-10T10:33:39.553396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640530+0000","last_change":"2024-05-10T10:33:34.516536+0000","last_active":"2024-05-10T10:33:39.640530+0000","last_peered":"2024-05-10T10:33:39.640530+0000","last_clean":"2024-05-10T10:33:39.640530+0000","last_became_active":"2024-05-10T10:33:34.515018+0000","last_became_peered":"2024-05-10T10:33:34.515018+0000","last_unstale":"2024-05-10T10:33:39.640530+0000","last_undegraded":"2024-05-10T10:33:39.640530+0000","last_fullsized":"2024-05-10T10:33:39.640530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872943+0000","last_change":"2024-05-10T10:33:34.517149+0000","last_active":"2024-05-10T10:33:40.872943+0000","last_peered":"2024-05-10T10:33:40.872943+0000","last_clean":"2024-05-10T10:33:40.872943+0000","last_became_active":"2024-05-10T10:33:34.516875+0000","last_became_peered":"2024-05-10T10:33:34.516875+0000","last_unstale":"2024-05-10T10:33:40.872943+0000","last_undegraded":"2024-05-10T10:33:40.872943+0000","last_fullsized":"2024-05-10T10:33:40.872943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553216+0000","last_change":"2024-05-10T10:33:34.518781+0000","last_active":"2024-05-10T10:33:39.553216+0000","last_peered":"2024-05-10T10:33:39.553216+0000","last_clean":"2024-05-10T10:33:39.553216+0000","last_became_active":"2024-05-10T10:33:34.518457+0000","last_became_peered":"2024-05-10T10:33:34.518457+0000","last_unstale":"2024-05-10T10:33:39.553216+0000","last_undegraded":"2024-05-10T10:33:39.553216+0000","last_fullsized":"2024-05-10T10:33:39.553216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554503+0000","last_change":"2024-05-10T10:33:34.518753+0000","last_active":"2024-05-10T10:33:39.554503+0000","last_peered":"2024-05-10T10:33:39.554503+0000","last_clean":"2024-05-10T10:33:39.554503+0000","last_became_active":"2024-05-10T10:33:34.518443+0000","last_became_peered":"2024-05-10T10:33:34.518443+0000","last_unstale":"2024-05-10T10:33:39.554503+0000","last_undegraded":"2024-05-10T10:33:39.554503+0000","last_fullsized":"2024-05-10T10:33:39.554503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554243+0000","last_change":"2024-05-10T10:33:34.517952+0000","last_active":"2024-05-10T10:33:39.554243+0000","last_peered":"2024-05-10T10:33:39.554243+0000","last_clean":"2024-05-10T10:33:39.554243+0000","last_became_active":"2024-05-10T10:33:34.517670+0000","last_became_peered":"2024-05-10T10:33:34.517670+0000","last_unstale":"2024-05-10T10:33:39.554243+0000","last_undegraded":"2024-05-10T10:33:39.554243+0000","last_fullsized":"2024-05-10T10:33:39.554243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:57.405468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640217+0000","last_change":"2024-05-10T10:33:34.508563+0000","last_active":"2024-05-10T10:33:39.640217+0000","last_peered":"2024-05-10T10:33:39.640217+0000","last_clean":"2024-05-10T10:33:39.640217+0000","last_became_active":"2024-05-10T10:33:34.508082+0000","last_became_peered":"2024-05-10T10:33:34.508082+0000","last_unstale":"2024-05-10T10:33:39.640217+0000","last_undegraded":"2024-05-10T10:33:39.640217+0000","last_fullsized":"2024-05-10T10:33:39.640217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553714+0000","last_change":"2024-05-10T10:33:34.513986+0000","last_active":"2024-05-10T10:33:39.553714+0000","last_peered":"2024-05-10T10:33:39.553714+0000","last_clean":"2024-05-10T10:33:39.553714+0000","last_became_active":"2024-05-10T10:33:34.513734+0000","last_became_peered":"2024-05-10T10:33:34.513734+0000","last_unstale":"2024-05-10T10:33:39.553714+0000","last_undegraded":"2024-05-10T10:33:39.553714+0000","last_fullsized":"2024-05-10T10:33:39.553714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871457+0000","last_change":"2024-05-10T10:33:34.509577+0000","last_active":"2024-05-10T10:33:40.871457+0000","last_peered":"2024-05-10T10:33:40.871457+0000","last_clean":"2024-05-10T10:33:40.871457+0000","last_became_active":"2024-05-10T10:33:34.508968+0000","last_became_peered":"2024-05-10T10:33:34.508968+0000","last_unstale":"2024-05-10T10:33:40.871457+0000","last_undegraded":"2024-05-10T10:33:40.871457+0000","last_fullsized":"2024-05-10T10:33:40.871457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553509+0000","last_change":"2024-05-10T10:33:34.515157+0000","last_active":"2024-05-10T10:33:39.553509+0000","last_peered":"2024-05-10T10:33:39.553509+0000","last_clean":"2024-05-10T10:33:39.553509+0000","last_became_active":"2024-05-10T10:33:34.514884+0000","last_became_peered":"2024-05-10T10:33:34.514884+0000","last_unstale":"2024-05-10T10:33:39.553509+0000","last_undegraded":"2024-05-10T10:33:39.553509+0000","last_fullsized":"2024-05-10T10:33:39.553509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871783+0000","last_change":"2024-05-10T10:33:34.513846+0000","last_active":"2024-05-10T10:33:40.871783+0000","last_peered":"2024-05-10T10:33:40.871783+0000","last_clean":"2024-05-10T10:33:40.871783+0000","last_became_active":"2024-05-10T10:33:34.513538+0000","last_became_peered":"2024-05-10T10:33:34.513538+0000","last_unstale":"2024-05-10T10:33:40.871783+0000","last_undegraded":"2024-05-10T10:33:40.871783+0000","last_fullsized":"2024-05-10T10:33:40.871783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553166+0000","last_change":"2024-05-10T10:33:34.503972+0000","last_active":"2024-05-10T10:33:39.553166+0000","last_peered":"2024-05-10T10:33:39.553166+0000","last_clean":"2024-05-10T10:33:39.553166+0000","last_became_active":"2024-05-10T10:33:34.501490+0000","last_became_peered":"2024-05-10T10:33:34.501490+0000","last_unstale":"2024-05-10T10:33:39.553166+0000","last_undegraded":"2024-05-10T10:33:39.553166+0000","last_fullsized":"2024-05-10T10:33:39.553166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885974+0000","last_change":"2024-05-10T10:33:34.502166+0000","last_active":"2024-05-10T10:33:39.885974+0000","last_peered":"2024-05-10T10:33:39.885974+0000","last_clean":"2024-05-10T10:33:39.885974+0000","last_became_active":"2024-05-10T10:33:34.501852+0000","last_became_peered":"2024-05-10T10:33:34.501852+0000","last_unstale":"2024-05-10T10:33:39.885974+0000","last_undegraded":"2024-05-10T10:33:39.885974+0000","last_fullsized":"2024-05-10T10:33:39.885974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554036+0000","last_change":"2024-05-10T10:33:34.522785+0000","last_active":"2024-05-10T10:33:39.554036+0000","last_peered":"2024-05-10T10:33:39.554036+0000","last_clean":"2024-05-10T10:33:39.554036+0000","last_became_active":"2024-05-10T10:33:34.522413+0000","last_became_peered":"2024-05-10T10:33:34.522413+0000","last_unstale":"2024-05-10T10:33:39.554036+0000","last_undegraded":"2024-05-10T10:33:39.554036+0000","last_fullsized":"2024-05-10T10:33:39.554036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":101868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":101868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:33:48.622 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:33:48.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.759+0000 7f41e066d640 1 Processor -- start 2024-05-10T10:33:48.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.759+0000 7f41e066d640 1 -- start start 2024-05-10T10:33:48.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41e066d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f41d80d3980 0x7f41d81626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41e066d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f41d80d2270 0x7f41d8162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41e066d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80b6160 0x7f41d80b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41e066d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80b5c30 con 0x7f41d80d3980 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41e066d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80b5e80 con 0x7f41d80d2270 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41e066d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80d5c50 con 0x7f41d80b6160 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f41d80d3980 0x7f41d81626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f41d80d3980 0x7f41d81626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43246/0 (socket says 172.21.15.53:43246) 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 -- 172.21.15.53:0/3558677572 learned_addr learned my addr 172.21.15.53:0/3558677572 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41ddc08640 1 --2- 172.21.15.53:0/3558677572 >> v2:172.21.15.135:3300/0 conn(0x7f41d80d2270 0x7f41d8162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41dec0a640 1 --2- 172.21.15.53:0/3558677572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80b6160 0x7f41d80b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 -- 172.21.15.53:0/3558677572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80b6160 msgr2=0x7f41d80b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 --2- 172.21.15.53:0/3558677572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80b6160 0x7f41d80b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 -- 172.21.15.53:0/3558677572 >> v2:172.21.15.135:3300/0 conn(0x7f41d80d2270 msgr2=0x7f41d8162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 --2- 172.21.15.53:0/3558677572 >> v2:172.21.15.135:3300/0 conn(0x7f41d80d2270 0x7f41d8162c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41de409640 1 -- 172.21.15.53:0/3558677572 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80d5f30 con 0x7f41d80d3980 2024-05-10T10:33:48.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41ddc08640 1 --2- 172.21.15.53:0/3558677572 >> v2:172.21.15.135:3300/0 conn(0x7f41d80d2270 0x7f41d8162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:48.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.760+0000 7f41dec0a640 1 --2- 172.21.15.53:0/3558677572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80b6160 0x7f41d80b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:48.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.761+0000 7f41de409640 1 --2- 172.21.15.53:0/3558677572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d3980 0x7f41d81626c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41d808d060 tx=0x7f41d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376ebdc0311d174e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.761+0000 7f41dd407640 1 -- 172.21.15.53:0/3558677572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4037070 con 0x7f41d80d3980 2024-05-10T10:33:48.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.761+0000 7f41dd407640 1 -- 172.21.15.53:0/3558677572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d4032440 con 0x7f41d80d3980 2024-05-10T10:33:48.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.761+0000 7f41dd407640 1 -- 172.21.15.53:0/3558677572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4032810 con 0x7f41d80d3980 2024-05-10T10:33:48.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.761+0000 7f41e066d640 1 -- 172.21.15.53:0/3558677572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d3980 msgr2=0x7f41d81626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.761+0000 7f41e066d640 1 --2- 172.21.15.53:0/3558677572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d3980 0x7f41d81626c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41d808d060 tx=0x7f41d402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.762+0000 7f41e066d640 1 -- 172.21.15.53:0/3558677572 shutdown_connections 2024-05-10T10:33:48.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.762+0000 7f41e066d640 1 --2- 172.21.15.53:0/3558677572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80b6160 0x7f41d80b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.762+0000 7f41e066d640 1 --2- 172.21.15.53:0/3558677572 >> v2:172.21.15.135:3300/0 conn(0x7f41d80d2270 0x7f41d8162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.762+0000 7f41e066d640 1 --2- 172.21.15.53:0/3558677572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d3980 0x7f41d81626c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.762+0000 7f41e066d640 1 -- 172.21.15.53:0/3558677572 >> 172.21.15.53:0/3558677572 conn(0x7f41d80d5130 msgr2=0x7f41d80d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.762+0000 7f41e066d640 1 -- 172.21.15.53:0/3558677572 shutdown_connections 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.762+0000 7f41e066d640 1 -- 172.21.15.53:0/3558677572 wait complete. 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.763+0000 7f41e066d640 1 Processor -- start 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.763+0000 7f41e066d640 1 -- start start 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41e066d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d80b6160 0x7f41d80b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41e066d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d2270 0x7f41d80ba880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41e066d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80d3980 0x7f41d80cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41e066d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80b6890 con 0x7f41d80d2270 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41e066d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80d31f0 con 0x7f41d80b6160 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41e066d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80d3d60 con 0x7f41d80d3980 2024-05-10T10:33:48.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d2270 0x7f41d80ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d2270 0x7f41d80ba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43258/0 (socket says 172.21.15.53:43258) 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 -- 172.21.15.53:0/2389038170 learned_addr learned my addr 172.21.15.53:0/2389038170 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41de409640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d80b6160 0x7f41d80b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41dec0a640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80d3980 0x7f41d80cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 -- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80d3980 msgr2=0x7f41d80cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80d3980 0x7f41d80cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 -- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d80b6160 msgr2=0x7f41d80b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d80b6160 0x7f41d80b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 -- 172.21.15.53:0/2389038170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80b97e0 con 0x7f41d80d2270 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ddc08640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d2270 0x7f41d80ba880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e810 tx=0x7f41c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ceffd640 1 -- 172.21.15.53:0/2389038170 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c800f040 con 0x7f41d80d2270 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ceffd640 1 -- 172.21.15.53:0/2389038170 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c8004030 con 0x7f41d80d2270 2024-05-10T10:33:48.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d80b9ac0 con 0x7f41d80d2270 2024-05-10T10:33:48.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.764+0000 7f41ceffd640 1 -- 172.21.15.53:0/2389038170 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8004400 con 0x7f41d80d2270 2024-05-10T10:33:48.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.765+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d80ba080 con 0x7f41d80d2270 2024-05-10T10:33:48.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.765+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d806a220 con 0x7f41d80d2270 2024-05-10T10:33:48.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.766+0000 7f41ceffd640 1 -- 172.21.15.53:0/2389038170 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41c800c6d0 con 0x7f41d80d2270 2024-05-10T10:33:48.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.767+0000 7f41ceffd640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41bc0762d0 0x7f41bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:48.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.767+0000 7f41ceffd640 1 -- 172.21.15.53:0/2389038170 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f41c80974f0 con 0x7f41d80d2270 2024-05-10T10:33:48.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.767+0000 7f41de409640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41bc0762d0 0x7f41bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:48.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.767+0000 7f41de409640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41bc0762d0 0x7f41bc078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f41d4034000 tx=0x7f41d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:48.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.771+0000 7f41ceffd640 1 -- 172.21.15.53:0/2389038170 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f41c8017020 con 0x7f41d80d2270 2024-05-10T10:33:48.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.956+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41d80badc0 con 0x7f41bc0762d0 2024-05-10T10:33:48.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.965+0000 7f41ceffd640 1 -- 172.21.15.53:0/2389038170 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332631 (secure 0 0 0) 0x7f41d4032610 con 0x7f41bc0762d0 2024-05-10T10:33:48.969 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:48.971 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41bc0762d0 msgr2=0x7f41bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41bc0762d0 0x7f41bc078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f41d4034000 tx=0x7f41d402e000 comp rx=0 tx=0).stop 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d2270 msgr2=0x7f41d80ba880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d2270 0x7f41d80ba880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e810 tx=0x7f41c800ecd0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 shutdown_connections 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41bc0762d0 0x7f41bc078790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d80d3980 0x7f41d80cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80d2270 0x7f41d80ba880 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 --2- 172.21.15.53:0/2389038170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d80b6160 0x7f41d80b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 >> 172.21.15.53:0/2389038170 conn(0x7f41d80d5130 msgr2=0x7f41d8160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:48.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 shutdown_connections 2024-05-10T10:33:48.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:48.968+0000 7f41e066d640 1 -- 172.21.15.53:0/2389038170 wait complete. 2024-05-10T10:33:48.980 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-05-10T10:33:48.159218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":313,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236040,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738680,"statfs":{"total":773094113280,"available":772852408320,"internally_reserved":0,"allocated":3284992,"data_stored":1987604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.644959"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553612+0000","last_change":"2024-05-10T10:33:33.499661+0000","last_active":"2024-05-10T10:33:39.553612+0000","last_peered":"2024-05-10T10:33:39.553612+0000","last_clean":"2024-05-10T10:33:39.553612+0000","last_became_active":"2024-05-10T10:33:33.499370+0000","last_became_peered":"2024-05-10T10:33:33.499370+0000","last_unstale":"2024-05-10T10:33:39.553612+0000","last_undegraded":"2024-05-10T10:33:39.553612+0000","last_fullsized":"2024-05-10T10:33:39.553612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:01:06.424327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641011+0000","last_change":"2024-05-10T10:33:34.515514+0000","last_active":"2024-05-10T10:33:39.641011+0000","last_peered":"2024-05-10T10:33:39.641011+0000","last_clean":"2024-05-10T10:33:39.641011+0000","last_became_active":"2024-05-10T10:33:34.514965+0000","last_became_peered":"2024-05-10T10:33:34.514965+0000","last_unstale":"2024-05-10T10:33:39.641011+0000","last_undegraded":"2024-05-10T10:33:39.641011+0000","last_fullsized":"2024-05-10T10:33:39.641011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640231+0000","last_change":"2024-05-10T10:33:38.535811+0000","last_active":"2024-05-10T10:33:39.640231+0000","last_peered":"2024-05-10T10:33:39.640231+0000","last_clean":"2024-05-10T10:33:39.640231+0000","last_became_active":"2024-05-10T10:33:38.535356+0000","last_became_peered":"2024-05-10T10:33:38.535356+0000","last_unstale":"2024-05-10T10:33:39.640231+0000","last_undegraded":"2024-05-10T10:33:39.640231+0000","last_fullsized":"2024-05-10T10:33:39.640231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641122+0000","last_change":"2024-05-10T10:33:33.494355+0000","last_active":"2024-05-10T10:33:39.641122+0000","last_peered":"2024-05-10T10:33:39.641122+0000","last_clean":"2024-05-10T10:33:39.641122+0000","last_became_active":"2024-05-10T10:33:33.494042+0000","last_became_peered":"2024-05-10T10:33:33.494042+0000","last_unstale":"2024-05-10T10:33:39.641122+0000","last_undegraded":"2024-05-10T10:33:39.641122+0000","last_fullsized":"2024-05-10T10:33:39.641122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553336+0000","last_change":"2024-05-10T10:33:34.514217+0000","last_active":"2024-05-10T10:33:39.553336+0000","last_peered":"2024-05-10T10:33:39.553336+0000","last_clean":"2024-05-10T10:33:39.553336+0000","last_became_active":"2024-05-10T10:33:34.513895+0000","last_became_peered":"2024-05-10T10:33:34.513895+0000","last_unstale":"2024-05-10T10:33:39.553336+0000","last_undegraded":"2024-05-10T10:33:39.553336+0000","last_fullsized":"2024-05-10T10:33:39.553336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872369+0000","last_change":"2024-05-10T10:33:38.542469+0000","last_active":"2024-05-10T10:33:40.872369+0000","last_peered":"2024-05-10T10:33:40.872369+0000","last_clean":"2024-05-10T10:33:40.872369+0000","last_became_active":"2024-05-10T10:33:38.541985+0000","last_became_peered":"2024-05-10T10:33:38.541985+0000","last_unstale":"2024-05-10T10:33:40.872369+0000","last_undegraded":"2024-05-10T10:33:40.872369+0000","last_fullsized":"2024-05-10T10:33:40.872369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:30.304981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554581+0000","last_change":"2024-05-10T10:33:33.500811+0000","last_active":"2024-05-10T10:33:39.554581+0000","last_peered":"2024-05-10T10:33:39.554581+0000","last_clean":"2024-05-10T10:33:39.554581+0000","last_became_active":"2024-05-10T10:33:33.500355+0000","last_became_peered":"2024-05-10T10:33:33.500355+0000","last_unstale":"2024-05-10T10:33:39.554581+0000","last_undegraded":"2024-05-10T10:33:39.554581+0000","last_fullsized":"2024-05-10T10:33:39.554581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871828+0000","last_change":"2024-05-10T10:33:34.502619+0000","last_active":"2024-05-10T10:33:40.871828+0000","last_peered":"2024-05-10T10:33:40.871828+0000","last_clean":"2024-05-10T10:33:40.871828+0000","last_became_active":"2024-05-10T10:33:34.500045+0000","last_became_peered":"2024-05-10T10:33:34.500045+0000","last_unstale":"2024-05-10T10:33:40.871828+0000","last_undegraded":"2024-05-10T10:33:40.871828+0000","last_fullsized":"2024-05-10T10:33:40.871828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872455+0000","last_change":"2024-05-10T10:33:38.543336+0000","last_active":"2024-05-10T10:33:40.872455+0000","last_peered":"2024-05-10T10:33:40.872455+0000","last_clean":"2024-05-10T10:33:40.872455+0000","last_became_active":"2024-05-10T10:33:38.542915+0000","last_became_peered":"2024-05-10T10:33:38.542915+0000","last_unstale":"2024-05-10T10:33:40.872455+0000","last_undegraded":"2024-05-10T10:33:40.872455+0000","last_fullsized":"2024-05-10T10:33:40.872455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:16.959519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553329+0000","last_change":"2024-05-10T10:33:33.499213+0000","last_active":"2024-05-10T10:33:39.553329+0000","last_peered":"2024-05-10T10:33:39.553329+0000","last_clean":"2024-05-10T10:33:39.553329+0000","last_became_active":"2024-05-10T10:33:33.497831+0000","last_became_peered":"2024-05-10T10:33:33.497831+0000","last_unstale":"2024-05-10T10:33:39.553329+0000","last_undegraded":"2024-05-10T10:33:39.553329+0000","last_fullsized":"2024-05-10T10:33:39.553329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:12.309888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871793+0000","last_change":"2024-05-10T10:33:34.508608+0000","last_active":"2024-05-10T10:33:40.871793+0000","last_peered":"2024-05-10T10:33:40.871793+0000","last_clean":"2024-05-10T10:33:40.871793+0000","last_became_active":"2024-05-10T10:33:34.507711+0000","last_became_peered":"2024-05-10T10:33:34.507711+0000","last_unstale":"2024-05-10T10:33:40.871793+0000","last_undegraded":"2024-05-10T10:33:40.871793+0000","last_fullsized":"2024-05-10T10:33:40.871793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872480+0000","last_change":"2024-05-10T10:33:38.534041+0000","last_active":"2024-05-10T10:33:40.872480+0000","last_peered":"2024-05-10T10:33:40.872480+0000","last_clean":"2024-05-10T10:33:40.872480+0000","last_became_active":"2024-05-10T10:33:38.533397+0000","last_became_peered":"2024-05-10T10:33:38.533397+0000","last_unstale":"2024-05-10T10:33:40.872480+0000","last_undegraded":"2024-05-10T10:33:40.872480+0000","last_fullsized":"2024-05-10T10:33:40.872480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:43.886335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553281+0000","last_change":"2024-05-10T10:33:34.504671+0000","last_active":"2024-05-10T10:33:39.553281+0000","last_peered":"2024-05-10T10:33:39.553281+0000","last_clean":"2024-05-10T10:33:39.553281+0000","last_became_active":"2024-05-10T10:33:34.504389+0000","last_became_peered":"2024-05-10T10:33:34.504389+0000","last_unstale":"2024-05-10T10:33:39.553281+0000","last_undegraded":"2024-05-10T10:33:39.553281+0000","last_fullsized":"2024-05-10T10:33:39.553281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553953+0000","last_change":"2024-05-10T10:33:33.499466+0000","last_active":"2024-05-10T10:33:39.553953+0000","last_peered":"2024-05-10T10:33:39.553953+0000","last_clean":"2024-05-10T10:33:39.553953+0000","last_became_active":"2024-05-10T10:33:33.499185+0000","last_became_peered":"2024-05-10T10:33:33.499185+0000","last_unstale":"2024-05-10T10:33:39.553953+0000","last_undegraded":"2024-05-10T10:33:39.553953+0000","last_fullsized":"2024-05-10T10:33:39.553953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:40.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872905+0000","last_change":"2024-05-10T10:33:38.542258+0000","last_active":"2024-05-10T10:33:40.872905+0000","last_peered":"2024-05-10T10:33:40.872905+0000","last_clean":"2024-05-10T10:33:40.872905+0000","last_became_active":"2024-05-10T10:33:38.541808+0000","last_became_peered":"2024-05-10T10:33:38.541808+0000","last_unstale":"2024-05-10T10:33:40.872905+0000","last_undegraded":"2024-05-10T10:33:40.872905+0000","last_fullsized":"2024-05-10T10:33:40.872905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640629+0000","last_change":"2024-05-10T10:33:34.506055+0000","last_active":"2024-05-10T10:33:39.640629+0000","last_peered":"2024-05-10T10:33:39.640629+0000","last_clean":"2024-05-10T10:33:39.640629+0000","last_became_active":"2024-05-10T10:33:34.505770+0000","last_became_peered":"2024-05-10T10:33:34.505770+0000","last_unstale":"2024-05-10T10:33:39.640629+0000","last_undegraded":"2024-05-10T10:33:39.640629+0000","last_fullsized":"2024-05-10T10:33:39.640629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886170+0000","last_change":"2024-05-10T10:33:33.495139+0000","last_active":"2024-05-10T10:33:39.886170+0000","last_peered":"2024-05-10T10:33:39.886170+0000","last_clean":"2024-05-10T10:33:39.886170+0000","last_became_active":"2024-05-10T10:33:33.494836+0000","last_became_peered":"2024-05-10T10:33:33.494836+0000","last_unstale":"2024-05-10T10:33:39.886170+0000","last_undegraded":"2024-05-10T10:33:39.886170+0000","last_fullsized":"2024-05-10T10:33:39.886170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872645+0000","last_change":"2024-05-10T10:33:38.542825+0000","last_active":"2024-05-10T10:33:40.872645+0000","last_peered":"2024-05-10T10:33:40.872645+0000","last_clean":"2024-05-10T10:33:40.872645+0000","last_became_active":"2024-05-10T10:33:38.542378+0000","last_became_peered":"2024-05-10T10:33:38.542378+0000","last_unstale":"2024-05-10T10:33:40.872645+0000","last_undegraded":"2024-05-10T10:33:40.872645+0000","last_fullsized":"2024-05-10T10:33:40.872645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640415+0000","last_change":"2024-05-10T10:33:34.504517+0000","last_active":"2024-05-10T10:33:39.640415+0000","last_peered":"2024-05-10T10:33:39.640415+0000","last_clean":"2024-05-10T10:33:39.640415+0000","last_became_active":"2024-05-10T10:33:34.504168+0000","last_became_peered":"2024-05-10T10:33:34.504168+0000","last_unstale":"2024-05-10T10:33:39.640415+0000","last_undegraded":"2024-05-10T10:33:39.640415+0000","last_fullsized":"2024-05-10T10:33:39.640415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885550+0000","last_change":"2024-05-10T10:33:33.497956+0000","last_active":"2024-05-10T10:33:39.885550+0000","last_peered":"2024-05-10T10:33:39.885550+0000","last_clean":"2024-05-10T10:33:39.885550+0000","last_became_active":"2024-05-10T10:33:33.497234+0000","last_became_peered":"2024-05-10T10:33:33.497234+0000","last_unstale":"2024-05-10T10:33:39.885550+0000","last_undegraded":"2024-05-10T10:33:39.885550+0000","last_fullsized":"2024-05-10T10:33:39.885550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886149+0000","last_change":"2024-05-10T10:33:38.548654+0000","last_active":"2024-05-10T10:33:39.886149+0000","last_peered":"2024-05-10T10:33:39.886149+0000","last_clean":"2024-05-10T10:33:39.886149+0000","last_became_active":"2024-05-10T10:33:38.547914+0000","last_became_peered":"2024-05-10T10:33:38.547914+0000","last_unstale":"2024-05-10T10:33:39.886149+0000","last_undegraded":"2024-05-10T10:33:39.886149+0000","last_fullsized":"2024-05-10T10:33:39.886149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872730+0000","last_change":"2024-05-10T10:33:34.499454+0000","last_active":"2024-05-10T10:33:40.872730+0000","last_peered":"2024-05-10T10:33:40.872730+0000","last_clean":"2024-05-10T10:33:40.872730+0000","last_became_active":"2024-05-10T10:33:34.499154+0000","last_became_peered":"2024-05-10T10:33:34.499154+0000","last_unstale":"2024-05-10T10:33:40.872730+0000","last_undegraded":"2024-05-10T10:33:40.872730+0000","last_fullsized":"2024-05-10T10:33:40.872730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:06.330274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554557+0000","last_change":"2024-05-10T10:33:33.497112+0000","last_active":"2024-05-10T10:33:39.554557+0000","last_peered":"2024-05-10T10:33:39.554557+0000","last_clean":"2024-05-10T10:33:39.554557+0000","last_became_active":"2024-05-10T10:33:33.496737+0000","last_became_peered":"2024-05-10T10:33:33.496737+0000","last_unstale":"2024-05-10T10:33:39.554557+0000","last_undegraded":"2024-05-10T10:33:39.554557+0000","last_fullsized":"2024-05-10T10:33:39.554557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:20.939521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873091+0000","last_change":"2024-05-10T10:33:38.542337+0000","last_active":"2024-05-10T10:33:40.873091+0000","last_peered":"2024-05-10T10:33:40.873091+0000","last_clean":"2024-05-10T10:33:40.873091+0000","last_became_active":"2024-05-10T10:33:38.541897+0000","last_became_peered":"2024-05-10T10:33:38.541897+0000","last_unstale":"2024-05-10T10:33:40.873091+0000","last_undegraded":"2024-05-10T10:33:40.873091+0000","last_fullsized":"2024-05-10T10:33:40.873091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553701+0000","last_change":"2024-05-10T10:33:34.503858+0000","last_active":"2024-05-10T10:33:39.553701+0000","last_peered":"2024-05-10T10:33:39.553701+0000","last_clean":"2024-05-10T10:33:39.553701+0000","last_became_active":"2024-05-10T10:33:34.503581+0000","last_became_peered":"2024-05-10T10:33:34.503581+0000","last_unstale":"2024-05-10T10:33:39.553701+0000","last_undegraded":"2024-05-10T10:33:39.553701+0000","last_fullsized":"2024-05-10T10:33:39.553701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554729+0000","last_change":"2024-05-10T10:33:33.498121+0000","last_active":"2024-05-10T10:33:39.554729+0000","last_peered":"2024-05-10T10:33:39.554729+0000","last_clean":"2024-05-10T10:33:39.554729+0000","last_became_active":"2024-05-10T10:33:33.497834+0000","last_became_peered":"2024-05-10T10:33:33.497834+0000","last_unstale":"2024-05-10T10:33:39.554729+0000","last_undegraded":"2024-05-10T10:33:39.554729+0000","last_fullsized":"2024-05-10T10:33:39.554729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872705+0000","last_change":"2024-05-10T10:33:38.531480+0000","last_active":"2024-05-10T10:33:40.872705+0000","last_peered":"2024-05-10T10:33:40.872705+0000","last_clean":"2024-05-10T10:33:40.872705+0000","last_became_active":"2024-05-10T10:33:38.531023+0000","last_became_peered":"2024-05-10T10:33:38.531023+0000","last_unstale":"2024-05-10T10:33:40.872705+0000","last_undegraded":"2024-05-10T10:33:40.872705+0000","last_fullsized":"2024-05-10T10:33:40.872705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641080+0000","last_change":"2024-05-10T10:33:34.506192+0000","last_active":"2024-05-10T10:33:39.641080+0000","last_peered":"2024-05-10T10:33:39.641080+0000","last_clean":"2024-05-10T10:33:39.641080+0000","last_became_active":"2024-05-10T10:33:34.505914+0000","last_became_peered":"2024-05-10T10:33:34.505914+0000","last_unstale":"2024-05-10T10:33:39.641080+0000","last_undegraded":"2024-05-10T10:33:39.641080+0000","last_fullsized":"2024-05-10T10:33:39.641080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886029+0000","last_change":"2024-05-10T10:33:33.491388+0000","last_active":"2024-05-10T10:33:39.886029+0000","last_peered":"2024-05-10T10:33:39.886029+0000","last_clean":"2024-05-10T10:33:39.886029+0000","last_became_active":"2024-05-10T10:33:33.491061+0000","last_became_peered":"2024-05-10T10:33:33.491061+0000","last_unstale":"2024-05-10T10:33:39.886029+0000","last_undegraded":"2024-05-10T10:33:39.886029+0000","last_fullsized":"2024-05-10T10:33:39.886029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640139+0000","last_change":"2024-05-10T10:33:38.543260+0000","last_active":"2024-05-10T10:33:39.640139+0000","last_peered":"2024-05-10T10:33:39.640139+0000","last_clean":"2024-05-10T10:33:39.640139+0000","last_became_active":"2024-05-10T10:33:38.542885+0000","last_became_peered":"2024-05-10T10:33:38.542885+0000","last_unstale":"2024-05-10T10:33:39.640139+0000","last_undegraded":"2024-05-10T10:33:39.640139+0000","last_fullsized":"2024-05-10T10:33:39.640139+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886114+0000","last_change":"2024-05-10T10:33:34.501069+0000","last_active":"2024-05-10T10:33:39.886114+0000","last_peered":"2024-05-10T10:33:39.886114+0000","last_clean":"2024-05-10T10:33:39.886114+0000","last_became_active":"2024-05-10T10:33:34.500767+0000","last_became_peered":"2024-05-10T10:33:34.500767+0000","last_unstale":"2024-05-10T10:33:39.886114+0000","last_undegraded":"2024-05-10T10:33:39.886114+0000","last_fullsized":"2024-05-10T10:33:39.886114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554556+0000","last_change":"2024-05-10T10:33:33.496126+0000","last_active":"2024-05-10T10:33:39.554556+0000","last_peered":"2024-05-10T10:33:39.554556+0000","last_clean":"2024-05-10T10:33:39.554556+0000","last_became_active":"2024-05-10T10:33:33.495848+0000","last_became_peered":"2024-05-10T10:33:33.495848+0000","last_unstale":"2024-05-10T10:33:39.554556+0000","last_undegraded":"2024-05-10T10:33:39.554556+0000","last_fullsized":"2024-05-10T10:33:39.554556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553188+0000","last_change":"2024-05-10T10:33:38.542612+0000","last_active":"2024-05-10T10:33:39.553188+0000","last_peered":"2024-05-10T10:33:39.553188+0000","last_clean":"2024-05-10T10:33:39.553188+0000","last_became_active":"2024-05-10T10:33:38.542110+0000","last_became_peered":"2024-05-10T10:33:38.542110+0000","last_unstale":"2024-05-10T10:33:39.553188+0000","last_undegraded":"2024-05-10T10:33:39.553188+0000","last_fullsized":"2024-05-10T10:33:39.553188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871909+0000","last_change":"2024-05-10T10:33:34.515283+0000","last_active":"2024-05-10T10:33:40.871909+0000","last_peered":"2024-05-10T10:33:40.871909+0000","last_clean":"2024-05-10T10:33:40.871909+0000","last_became_active":"2024-05-10T10:33:34.515016+0000","last_became_peered":"2024-05-10T10:33:34.515016+0000","last_unstale":"2024-05-10T10:33:40.871909+0000","last_undegraded":"2024-05-10T10:33:40.871909+0000","last_fullsized":"2024-05-10T10:33:40.871909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886059+0000","last_change":"2024-05-10T10:33:33.488825+0000","last_active":"2024-05-10T10:33:39.886059+0000","last_peered":"2024-05-10T10:33:39.886059+0000","last_clean":"2024-05-10T10:33:39.886059+0000","last_became_active":"2024-05-10T10:33:33.488539+0000","last_became_peered":"2024-05-10T10:33:33.488539+0000","last_unstale":"2024-05-10T10:33:39.886059+0000","last_undegraded":"2024-05-10T10:33:39.886059+0000","last_fullsized":"2024-05-10T10:33:39.886059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885597+0000","last_change":"2024-05-10T10:33:38.547163+0000","last_active":"2024-05-10T10:33:39.885597+0000","last_peered":"2024-05-10T10:33:39.885597+0000","last_clean":"2024-05-10T10:33:39.885597+0000","last_became_active":"2024-05-10T10:33:38.546692+0000","last_became_peered":"2024-05-10T10:33:38.546692+0000","last_unstale":"2024-05-10T10:33:39.885597+0000","last_undegraded":"2024-05-10T10:33:39.885597+0000","last_fullsized":"2024-05-10T10:33:39.885597+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872867+0000","last_change":"2024-05-10T10:33:34.505921+0000","last_active":"2024-05-10T10:33:40.872867+0000","last_peered":"2024-05-10T10:33:40.872867+0000","last_clean":"2024-05-10T10:33:40.872867+0000","last_became_active":"2024-05-10T10:33:34.505637+0000","last_became_peered":"2024-05-10T10:33:34.505637+0000","last_unstale":"2024-05-10T10:33:40.872867+0000","last_undegraded":"2024-05-10T10:33:40.872867+0000","last_fullsized":"2024-05-10T10:33:40.872867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871754+0000","last_change":"2024-05-10T10:33:33.485382+0000","last_active":"2024-05-10T10:33:40.871754+0000","last_peered":"2024-05-10T10:33:40.871754+0000","last_clean":"2024-05-10T10:33:40.871754+0000","last_became_active":"2024-05-10T10:33:33.485082+0000","last_became_peered":"2024-05-10T10:33:33.485082+0000","last_unstale":"2024-05-10T10:33:40.871754+0000","last_undegraded":"2024-05-10T10:33:40.871754+0000","last_fullsized":"2024-05-10T10:33:40.871754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885955+0000","last_change":"2024-05-10T10:33:38.541426+0000","last_active":"2024-05-10T10:33:39.885955+0000","last_peered":"2024-05-10T10:33:39.885955+0000","last_clean":"2024-05-10T10:33:39.885955+0000","last_became_active":"2024-05-10T10:33:38.539367+0000","last_became_peered":"2024-05-10T10:33:38.539367+0000","last_unstale":"2024-05-10T10:33:39.885955+0000","last_undegraded":"2024-05-10T10:33:39.885955+0000","last_fullsized":"2024-05-10T10:33:39.885955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885593+0000","last_change":"2024-05-10T10:33:34.501862+0000","last_active":"2024-05-10T10:33:39.885593+0000","last_peered":"2024-05-10T10:33:39.885593+0000","last_clean":"2024-05-10T10:33:39.885593+0000","last_became_active":"2024-05-10T10:33:34.501493+0000","last_became_peered":"2024-05-10T10:33:34.501493+0000","last_unstale":"2024-05-10T10:33:39.885593+0000","last_undegraded":"2024-05-10T10:33:39.885593+0000","last_fullsized":"2024-05-10T10:33:39.885593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679733+0000","last_change":"2024-05-10T10:33:33.495291+0000","last_active":"2024-05-10T10:33:39.679733+0000","last_peered":"2024-05-10T10:33:39.679733+0000","last_clean":"2024-05-10T10:33:39.679733+0000","last_became_active":"2024-05-10T10:33:33.494955+0000","last_became_peered":"2024-05-10T10:33:33.494955+0000","last_unstale":"2024-05-10T10:33:39.679733+0000","last_undegraded":"2024-05-10T10:33:39.679733+0000","last_fullsized":"2024-05-10T10:33:39.679733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886190+0000","last_change":"2024-05-10T10:33:38.539560+0000","last_active":"2024-05-10T10:33:39.886190+0000","last_peered":"2024-05-10T10:33:39.886190+0000","last_clean":"2024-05-10T10:33:39.886190+0000","last_became_active":"2024-05-10T10:33:38.539131+0000","last_became_peered":"2024-05-10T10:33:38.539131+0000","last_unstale":"2024-05-10T10:33:39.886190+0000","last_undegraded":"2024-05-10T10:33:39.886190+0000","last_fullsized":"2024-05-10T10:33:39.886190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552886+0000","last_change":"2024-05-10T10:33:34.510125+0000","last_active":"2024-05-10T10:33:39.552886+0000","last_peered":"2024-05-10T10:33:39.552886+0000","last_clean":"2024-05-10T10:33:39.552886+0000","last_became_active":"2024-05-10T10:33:34.509820+0000","last_became_peered":"2024-05-10T10:33:34.509820+0000","last_unstale":"2024-05-10T10:33:39.552886+0000","last_undegraded":"2024-05-10T10:33:39.552886+0000","last_fullsized":"2024-05-10T10:33:39.552886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554626+0000","last_change":"2024-05-10T10:33:33.499197+0000","last_active":"2024-05-10T10:33:39.554626+0000","last_peered":"2024-05-10T10:33:39.554626+0000","last_clean":"2024-05-10T10:33:39.554626+0000","last_became_active":"2024-05-10T10:33:33.497808+0000","last_became_peered":"2024-05-10T10:33:33.497808+0000","last_unstale":"2024-05-10T10:33:39.554626+0000","last_undegraded":"2024-05-10T10:33:39.554626+0000","last_fullsized":"2024-05-10T10:33:39.554626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:11.901185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553441+0000","last_change":"2024-05-10T10:33:38.544906+0000","last_active":"2024-05-10T10:33:39.553441+0000","last_peered":"2024-05-10T10:33:39.553441+0000","last_clean":"2024-05-10T10:33:39.553441+0000","last_became_active":"2024-05-10T10:33:38.544304+0000","last_became_peered":"2024-05-10T10:33:38.544304+0000","last_unstale":"2024-05-10T10:33:39.553441+0000","last_undegraded":"2024-05-10T10:33:39.553441+0000","last_fullsized":"2024-05-10T10:33:39.553441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640227+0000","last_change":"2024-05-10T10:33:34.513886+0000","last_active":"2024-05-10T10:33:39.640227+0000","last_peered":"2024-05-10T10:33:39.640227+0000","last_clean":"2024-05-10T10:33:39.640227+0000","last_became_active":"2024-05-10T10:33:34.513617+0000","last_became_peered":"2024-05-10T10:33:34.513617+0000","last_unstale":"2024-05-10T10:33:39.640227+0000","last_undegraded":"2024-05-10T10:33:39.640227+0000","last_fullsized":"2024-05-10T10:33:39.640227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871616+0000","last_change":"2024-05-10T10:33:33.489088+0000","last_active":"2024-05-10T10:33:40.871616+0000","last_peered":"2024-05-10T10:33:40.871616+0000","last_clean":"2024-05-10T10:33:40.871616+0000","last_became_active":"2024-05-10T10:33:33.488785+0000","last_became_peered":"2024-05-10T10:33:33.488785+0000","last_unstale":"2024-05-10T10:33:40.871616+0000","last_undegraded":"2024-05-10T10:33:40.871616+0000","last_fullsized":"2024-05-10T10:33:40.871616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:06:14.764873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554687+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:33:39.554687+0000","last_peered":"2024-05-10T10:33:39.554687+0000","last_clean":"2024-05-10T10:33:39.554687+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:33:39.554687+0000","last_undegraded":"2024-05-10T10:33:39.554687+0000","last_fullsized":"2024-05-10T10:33:39.554687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553037+0000","last_change":"2024-05-10T10:33:38.538561+0000","last_active":"2024-05-10T10:33:39.553037+0000","last_peered":"2024-05-10T10:33:39.553037+0000","last_clean":"2024-05-10T10:33:39.553037+0000","last_became_active":"2024-05-10T10:33:38.538159+0000","last_became_peered":"2024-05-10T10:33:38.538159+0000","last_unstale":"2024-05-10T10:33:39.553037+0000","last_undegraded":"2024-05-10T10:33:39.553037+0000","last_fullsized":"2024-05-10T10:33:39.553037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554231+0000","last_change":"2024-05-10T10:33:34.517851+0000","last_active":"2024-05-10T10:33:39.554231+0000","last_peered":"2024-05-10T10:33:39.554231+0000","last_clean":"2024-05-10T10:33:39.554231+0000","last_became_active":"2024-05-10T10:33:34.517556+0000","last_became_peered":"2024-05-10T10:33:34.517556+0000","last_unstale":"2024-05-10T10:33:39.554231+0000","last_undegraded":"2024-05-10T10:33:39.554231+0000","last_fullsized":"2024-05-10T10:33:39.554231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:46.690299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680077+0000","last_change":"2024-05-10T10:33:33.488242+0000","last_active":"2024-05-10T10:33:39.680077+0000","last_peered":"2024-05-10T10:33:39.680077+0000","last_clean":"2024-05-10T10:33:39.680077+0000","last_became_active":"2024-05-10T10:33:33.487932+0000","last_became_peered":"2024-05-10T10:33:33.487932+0000","last_unstale":"2024-05-10T10:33:39.680077+0000","last_undegraded":"2024-05-10T10:33:39.680077+0000","last_fullsized":"2024-05-10T10:33:39.680077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552940+0000","last_change":"2024-05-10T10:33:38.534596+0000","last_active":"2024-05-10T10:33:39.552940+0000","last_peered":"2024-05-10T10:33:39.552940+0000","last_clean":"2024-05-10T10:33:39.552940+0000","last_became_active":"2024-05-10T10:33:38.534112+0000","last_became_peered":"2024-05-10T10:33:38.534112+0000","last_unstale":"2024-05-10T10:33:39.552940+0000","last_undegraded":"2024-05-10T10:33:39.552940+0000","last_fullsized":"2024-05-10T10:33:39.552940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871944+0000","last_change":"2024-05-10T10:33:34.514478+0000","last_active":"2024-05-10T10:33:40.871944+0000","last_peered":"2024-05-10T10:33:40.871944+0000","last_clean":"2024-05-10T10:33:40.871944+0000","last_became_active":"2024-05-10T10:33:34.514160+0000","last_became_peered":"2024-05-10T10:33:34.514160+0000","last_unstale":"2024-05-10T10:33:40.871944+0000","last_undegraded":"2024-05-10T10:33:40.871944+0000","last_fullsized":"2024-05-10T10:33:40.871944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886005+0000","last_change":"2024-05-10T10:33:33.499523+0000","last_active":"2024-05-10T10:33:39.886005+0000","last_peered":"2024-05-10T10:33:39.886005+0000","last_clean":"2024-05-10T10:33:39.886005+0000","last_became_active":"2024-05-10T10:33:33.499250+0000","last_became_peered":"2024-05-10T10:33:33.499250+0000","last_unstale":"2024-05-10T10:33:39.886005+0000","last_undegraded":"2024-05-10T10:33:39.886005+0000","last_fullsized":"2024-05-10T10:33:39.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553375+0000","last_change":"2024-05-10T10:33:38.543483+0000","last_active":"2024-05-10T10:33:39.553375+0000","last_peered":"2024-05-10T10:33:39.553375+0000","last_clean":"2024-05-10T10:33:39.553375+0000","last_became_active":"2024-05-10T10:33:38.543060+0000","last_became_peered":"2024-05-10T10:33:38.543060+0000","last_unstale":"2024-05-10T10:33:39.553375+0000","last_undegraded":"2024-05-10T10:33:39.553375+0000","last_fullsized":"2024-05-10T10:33:39.553375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554354+0000","last_change":"2024-05-10T10:33:34.522046+0000","last_active":"2024-05-10T10:33:39.554354+0000","last_peered":"2024-05-10T10:33:39.554354+0000","last_clean":"2024-05-10T10:33:39.554354+0000","last_became_active":"2024-05-10T10:33:34.521739+0000","last_became_peered":"2024-05-10T10:33:34.521739+0000","last_unstale":"2024-05-10T10:33:39.554354+0000","last_undegraded":"2024-05-10T10:33:39.554354+0000","last_fullsized":"2024-05-10T10:33:39.554354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:15.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680220+0000","last_change":"2024-05-10T10:33:33.495634+0000","last_active":"2024-05-10T10:33:39.680220+0000","last_peered":"2024-05-10T10:33:39.680220+0000","last_clean":"2024-05-10T10:33:39.680220+0000","last_became_active":"2024-05-10T10:33:33.494432+0000","last_became_peered":"2024-05-10T10:33:33.494432+0000","last_unstale":"2024-05-10T10:33:39.680220+0000","last_undegraded":"2024-05-10T10:33:39.680220+0000","last_fullsized":"2024-05-10T10:33:39.680220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871284+0000","last_change":"2024-05-10T10:33:38.528059+0000","last_active":"2024-05-10T10:33:40.871284+0000","last_peered":"2024-05-10T10:33:40.871284+0000","last_clean":"2024-05-10T10:33:40.871284+0000","last_became_active":"2024-05-10T10:33:38.527628+0000","last_became_peered":"2024-05-10T10:33:38.527628+0000","last_unstale":"2024-05-10T10:33:40.871284+0000","last_undegraded":"2024-05-10T10:33:40.871284+0000","last_fullsized":"2024-05-10T10:33:40.871284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554210+0000","last_change":"2024-05-10T10:33:34.519810+0000","last_active":"2024-05-10T10:33:39.554210+0000","last_peered":"2024-05-10T10:33:39.554210+0000","last_clean":"2024-05-10T10:33:39.554210+0000","last_became_active":"2024-05-10T10:33:34.519517+0000","last_became_peered":"2024-05-10T10:33:34.519517+0000","last_unstale":"2024-05-10T10:33:39.554210+0000","last_undegraded":"2024-05-10T10:33:39.554210+0000","last_fullsized":"2024-05-10T10:33:39.554210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873138+0000","last_change":"2024-05-10T10:33:33.494974+0000","last_active":"2024-05-10T10:33:40.873138+0000","last_peered":"2024-05-10T10:33:40.873138+0000","last_clean":"2024-05-10T10:33:40.873138+0000","last_became_active":"2024-05-10T10:33:33.494667+0000","last_became_peered":"2024-05-10T10:33:33.494667+0000","last_unstale":"2024-05-10T10:33:40.873138+0000","last_undegraded":"2024-05-10T10:33:40.873138+0000","last_fullsized":"2024-05-10T10:33:40.873138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:54.706371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872313+0000","last_change":"2024-05-10T10:33:38.542343+0000","last_active":"2024-05-10T10:33:40.872313+0000","last_peered":"2024-05-10T10:33:40.872313+0000","last_clean":"2024-05-10T10:33:40.872313+0000","last_became_active":"2024-05-10T10:33:38.541842+0000","last_became_peered":"2024-05-10T10:33:38.541842+0000","last_unstale":"2024-05-10T10:33:40.872313+0000","last_undegraded":"2024-05-10T10:33:40.872313+0000","last_fullsized":"2024-05-10T10:33:40.872313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:32.462562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553822+0000","last_change":"2024-05-10T10:33:34.520528+0000","last_active":"2024-05-10T10:33:39.553822+0000","last_peered":"2024-05-10T10:33:39.553822+0000","last_clean":"2024-05-10T10:33:39.553822+0000","last_became_active":"2024-05-10T10:33:34.520239+0000","last_became_peered":"2024-05-10T10:33:34.520239+0000","last_unstale":"2024-05-10T10:33:39.553822+0000","last_undegraded":"2024-05-10T10:33:39.553822+0000","last_fullsized":"2024-05-10T10:33:39.553822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679540+0000","last_change":"2024-05-10T10:33:33.492349+0000","last_active":"2024-05-10T10:33:39.679540+0000","last_peered":"2024-05-10T10:33:39.679540+0000","last_clean":"2024-05-10T10:33:39.679540+0000","last_became_active":"2024-05-10T10:33:33.492019+0000","last_became_peered":"2024-05-10T10:33:33.492019+0000","last_unstale":"2024-05-10T10:33:39.679540+0000","last_undegraded":"2024-05-10T10:33:39.679540+0000","last_fullsized":"2024-05-10T10:33:39.679540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553319+0000","last_change":"2024-05-10T10:33:38.538548+0000","last_active":"2024-05-10T10:33:39.553319+0000","last_peered":"2024-05-10T10:33:39.553319+0000","last_clean":"2024-05-10T10:33:39.553319+0000","last_became_active":"2024-05-10T10:33:38.538144+0000","last_became_peered":"2024-05-10T10:33:38.538144+0000","last_unstale":"2024-05-10T10:33:39.553319+0000","last_undegraded":"2024-05-10T10:33:39.553319+0000","last_fullsized":"2024-05-10T10:33:39.553319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553004+0000","last_change":"2024-05-10T10:33:34.510224+0000","last_active":"2024-05-10T10:33:39.553004+0000","last_peered":"2024-05-10T10:33:39.553004+0000","last_clean":"2024-05-10T10:33:39.553004+0000","last_became_active":"2024-05-10T10:33:34.509584+0000","last_became_peered":"2024-05-10T10:33:34.509584+0000","last_unstale":"2024-05-10T10:33:39.553004+0000","last_undegraded":"2024-05-10T10:33:39.553004+0000","last_fullsized":"2024-05-10T10:33:39.553004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553209+0000","last_change":"2024-05-10T10:33:33.496606+0000","last_active":"2024-05-10T10:33:39.553209+0000","last_peered":"2024-05-10T10:33:39.553209+0000","last_clean":"2024-05-10T10:33:39.553209+0000","last_became_active":"2024-05-10T10:33:33.496326+0000","last_became_peered":"2024-05-10T10:33:33.496326+0000","last_unstale":"2024-05-10T10:33:39.553209+0000","last_undegraded":"2024-05-10T10:33:39.553209+0000","last_fullsized":"2024-05-10T10:33:39.553209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641045+0000","last_change":"2024-05-10T10:33:38.539948+0000","last_active":"2024-05-10T10:33:39.641045+0000","last_peered":"2024-05-10T10:33:39.641045+0000","last_clean":"2024-05-10T10:33:39.641045+0000","last_became_active":"2024-05-10T10:33:38.539181+0000","last_became_peered":"2024-05-10T10:33:38.539181+0000","last_unstale":"2024-05-10T10:33:39.641045+0000","last_undegraded":"2024-05-10T10:33:39.641045+0000","last_fullsized":"2024-05-10T10:33:39.641045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:03.969211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554079+0000","last_change":"2024-05-10T10:33:34.519826+0000","last_active":"2024-05-10T10:33:39.554079+0000","last_peered":"2024-05-10T10:33:39.554079+0000","last_clean":"2024-05-10T10:33:39.554079+0000","last_became_active":"2024-05-10T10:33:34.519540+0000","last_became_peered":"2024-05-10T10:33:34.519540+0000","last_unstale":"2024-05-10T10:33:39.554079+0000","last_undegraded":"2024-05-10T10:33:39.554079+0000","last_fullsized":"2024-05-10T10:33:39.554079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871721+0000","last_change":"2024-05-10T10:33:33.494936+0000","last_active":"2024-05-10T10:33:40.871721+0000","last_peered":"2024-05-10T10:33:40.871721+0000","last_clean":"2024-05-10T10:33:40.871721+0000","last_became_active":"2024-05-10T10:33:33.493154+0000","last_became_peered":"2024-05-10T10:33:33.493154+0000","last_unstale":"2024-05-10T10:33:40.871721+0000","last_undegraded":"2024-05-10T10:33:40.871721+0000","last_fullsized":"2024-05-10T10:33:40.871721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640822+0000","last_change":"2024-05-10T10:33:38.544918+0000","last_active":"2024-05-10T10:33:39.640822+0000","last_peered":"2024-05-10T10:33:39.640822+0000","last_clean":"2024-05-10T10:33:39.640822+0000","last_became_active":"2024-05-10T10:33:38.544423+0000","last_became_peered":"2024-05-10T10:33:38.544423+0000","last_unstale":"2024-05-10T10:33:39.640822+0000","last_undegraded":"2024-05-10T10:33:39.640822+0000","last_fullsized":"2024-05-10T10:33:39.640822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:22.017984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553482+0000","last_change":"2024-05-10T10:33:34.506856+0000","last_active":"2024-05-10T10:33:39.553482+0000","last_peered":"2024-05-10T10:33:39.553482+0000","last_clean":"2024-05-10T10:33:39.553482+0000","last_became_active":"2024-05-10T10:33:34.506563+0000","last_became_peered":"2024-05-10T10:33:34.506563+0000","last_unstale":"2024-05-10T10:33:39.553482+0000","last_undegraded":"2024-05-10T10:33:39.553482+0000","last_fullsized":"2024-05-10T10:33:39.553482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:21.944467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553658+0000","last_change":"2024-05-10T10:33:33.496358+0000","last_active":"2024-05-10T10:33:39.553658+0000","last_peered":"2024-05-10T10:33:39.553658+0000","last_clean":"2024-05-10T10:33:39.553658+0000","last_became_active":"2024-05-10T10:33:33.495968+0000","last_became_peered":"2024-05-10T10:33:33.495968+0000","last_unstale":"2024-05-10T10:33:39.553658+0000","last_undegraded":"2024-05-10T10:33:39.553658+0000","last_fullsized":"2024-05-10T10:33:39.553658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:14.817696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871851+0000","last_change":"2024-05-10T10:33:38.534046+0000","last_active":"2024-05-10T10:33:40.871851+0000","last_peered":"2024-05-10T10:33:40.871851+0000","last_clean":"2024-05-10T10:33:40.871851+0000","last_became_active":"2024-05-10T10:33:38.533660+0000","last_became_peered":"2024-05-10T10:33:38.533660+0000","last_unstale":"2024-05-10T10:33:40.871851+0000","last_undegraded":"2024-05-10T10:33:40.871851+0000","last_fullsized":"2024-05-10T10:33:40.871851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554110+0000","last_change":"2024-05-10T10:33:34.516752+0000","last_active":"2024-05-10T10:33:39.554110+0000","last_peered":"2024-05-10T10:33:39.554110+0000","last_clean":"2024-05-10T10:33:39.554110+0000","last_became_active":"2024-05-10T10:33:34.516410+0000","last_became_peered":"2024-05-10T10:33:34.516410+0000","last_unstale":"2024-05-10T10:33:39.554110+0000","last_undegraded":"2024-05-10T10:33:39.554110+0000","last_fullsized":"2024-05-10T10:33:39.554110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553808+0000","last_change":"2024-05-10T10:33:33.498231+0000","last_active":"2024-05-10T10:33:39.553808+0000","last_peered":"2024-05-10T10:33:39.553808+0000","last_clean":"2024-05-10T10:33:39.553808+0000","last_became_active":"2024-05-10T10:33:33.497918+0000","last_became_peered":"2024-05-10T10:33:33.497918+0000","last_unstale":"2024-05-10T10:33:39.553808+0000","last_undegraded":"2024-05-10T10:33:39.553808+0000","last_fullsized":"2024-05-10T10:33:39.553808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885703+0000","last_change":"2024-05-10T10:33:38.537215+0000","last_active":"2024-05-10T10:33:39.885703+0000","last_peered":"2024-05-10T10:33:39.885703+0000","last_clean":"2024-05-10T10:33:39.885703+0000","last_became_active":"2024-05-10T10:33:38.536826+0000","last_became_peered":"2024-05-10T10:33:38.536826+0000","last_unstale":"2024-05-10T10:33:39.885703+0000","last_undegraded":"2024-05-10T10:33:39.885703+0000","last_fullsized":"2024-05-10T10:33:39.885703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872972+0000","last_change":"2024-05-10T10:33:34.512632+0000","last_active":"2024-05-10T10:33:40.872972+0000","last_peered":"2024-05-10T10:33:40.872972+0000","last_clean":"2024-05-10T10:33:40.872972+0000","last_became_active":"2024-05-10T10:33:34.512337+0000","last_became_peered":"2024-05-10T10:33:34.512337+0000","last_unstale":"2024-05-10T10:33:40.872972+0000","last_undegraded":"2024-05-10T10:33:40.872972+0000","last_fullsized":"2024-05-10T10:33:40.872972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:12.906132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885962+0000","last_change":"2024-05-10T10:33:33.485715+0000","last_active":"2024-05-10T10:33:39.885962+0000","last_peered":"2024-05-10T10:33:39.885962+0000","last_clean":"2024-05-10T10:33:39.885962+0000","last_became_active":"2024-05-10T10:33:33.485400+0000","last_became_peered":"2024-05-10T10:33:33.485400+0000","last_unstale":"2024-05-10T10:33:39.885962+0000","last_undegraded":"2024-05-10T10:33:39.885962+0000","last_fullsized":"2024-05-10T10:33:39.885962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872703+0000","last_change":"2024-05-10T10:33:38.542348+0000","last_active":"2024-05-10T10:33:40.872703+0000","last_peered":"2024-05-10T10:33:40.872703+0000","last_clean":"2024-05-10T10:33:40.872703+0000","last_became_active":"2024-05-10T10:33:38.541943+0000","last_became_peered":"2024-05-10T10:33:38.541943+0000","last_unstale":"2024-05-10T10:33:40.872703+0000","last_undegraded":"2024-05-10T10:33:40.872703+0000","last_fullsized":"2024-05-10T10:33:40.872703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553583+0000","last_change":"2024-05-10T10:33:34.514299+0000","last_active":"2024-05-10T10:33:39.553583+0000","last_peered":"2024-05-10T10:33:39.553583+0000","last_clean":"2024-05-10T10:33:39.553583+0000","last_became_active":"2024-05-10T10:33:34.513977+0000","last_became_peered":"2024-05-10T10:33:34.513977+0000","last_unstale":"2024-05-10T10:33:39.553583+0000","last_undegraded":"2024-05-10T10:33:39.553583+0000","last_fullsized":"2024-05-10T10:33:39.553583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872002+0000","last_change":"2024-05-10T10:33:33.491068+0000","last_active":"2024-05-10T10:33:40.872002+0000","last_peered":"2024-05-10T10:33:40.872002+0000","last_clean":"2024-05-10T10:33:40.872002+0000","last_became_active":"2024-05-10T10:33:33.490801+0000","last_became_peered":"2024-05-10T10:33:33.490801+0000","last_unstale":"2024-05-10T10:33:40.872002+0000","last_undegraded":"2024-05-10T10:33:40.872002+0000","last_fullsized":"2024-05-10T10:33:40.872002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640322+0000","last_change":"2024-05-10T10:33:38.538544+0000","last_active":"2024-05-10T10:33:39.640322+0000","last_peered":"2024-05-10T10:33:39.640322+0000","last_clean":"2024-05-10T10:33:39.640322+0000","last_became_active":"2024-05-10T10:33:38.538176+0000","last_became_peered":"2024-05-10T10:33:38.538176+0000","last_unstale":"2024-05-10T10:33:39.640322+0000","last_undegraded":"2024-05-10T10:33:39.640322+0000","last_fullsized":"2024-05-10T10:33:39.640322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:14.049514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871829+0000","last_change":"2024-05-10T10:33:34.513516+0000","last_active":"2024-05-10T10:33:40.871829+0000","last_peered":"2024-05-10T10:33:40.871829+0000","last_clean":"2024-05-10T10:33:40.871829+0000","last_became_active":"2024-05-10T10:33:34.512746+0000","last_became_peered":"2024-05-10T10:33:34.512746+0000","last_unstale":"2024-05-10T10:33:40.871829+0000","last_undegraded":"2024-05-10T10:33:40.871829+0000","last_fullsized":"2024-05-10T10:33:40.871829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554597+0000","last_change":"2024-05-10T10:33:33.498011+0000","last_active":"2024-05-10T10:33:39.554597+0000","last_peered":"2024-05-10T10:33:39.554597+0000","last_clean":"2024-05-10T10:33:39.554597+0000","last_became_active":"2024-05-10T10:33:33.497732+0000","last_became_peered":"2024-05-10T10:33:33.497732+0000","last_unstale":"2024-05-10T10:33:39.554597+0000","last_undegraded":"2024-05-10T10:33:39.554597+0000","last_fullsized":"2024-05-10T10:33:39.554597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640338+0000","last_change":"2024-05-10T10:33:38.537084+0000","last_active":"2024-05-10T10:33:39.640338+0000","last_peered":"2024-05-10T10:33:39.640338+0000","last_clean":"2024-05-10T10:33:39.640338+0000","last_became_active":"2024-05-10T10:33:38.535424+0000","last_became_peered":"2024-05-10T10:33:38.535424+0000","last_unstale":"2024-05-10T10:33:39.640338+0000","last_undegraded":"2024-05-10T10:33:39.640338+0000","last_fullsized":"2024-05-10T10:33:39.640338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:48:24.535051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553257+0000","last_change":"2024-05-10T10:33:38.545099+0000","last_active":"2024-05-10T10:33:39.553257+0000","last_peered":"2024-05-10T10:33:39.553257+0000","last_clean":"2024-05-10T10:33:39.553257+0000","last_became_active":"2024-05-10T10:33:38.544663+0000","last_became_peered":"2024-05-10T10:33:38.544663+0000","last_unstale":"2024-05-10T10:33:39.553257+0000","last_undegraded":"2024-05-10T10:33:39.553257+0000","last_fullsized":"2024-05-10T10:33:39.553257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885419+0000","last_change":"2024-05-10T10:33:34.505691+0000","last_active":"2024-05-10T10:33:39.885419+0000","last_peered":"2024-05-10T10:33:39.885419+0000","last_clean":"2024-05-10T10:33:39.885419+0000","last_became_active":"2024-05-10T10:33:34.505398+0000","last_became_peered":"2024-05-10T10:33:34.505398+0000","last_unstale":"2024-05-10T10:33:39.885419+0000","last_undegraded":"2024-05-10T10:33:39.885419+0000","last_fullsized":"2024-05-10T10:33:39.885419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871432+0000","last_change":"2024-05-10T10:33:33.494574+0000","last_active":"2024-05-10T10:33:40.871432+0000","last_peered":"2024-05-10T10:33:40.871432+0000","last_clean":"2024-05-10T10:33:40.871432+0000","last_became_active":"2024-05-10T10:33:33.494311+0000","last_became_peered":"2024-05-10T10:33:33.494311+0000","last_unstale":"2024-05-10T10:33:40.871432+0000","last_undegraded":"2024-05-10T10:33:40.871432+0000","last_fullsized":"2024-05-10T10:33:40.871432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554526+0000","last_change":"2024-05-10T10:33:38.537291+0000","last_active":"2024-05-10T10:33:39.554526+0000","last_peered":"2024-05-10T10:33:39.554526+0000","last_clean":"2024-05-10T10:33:39.554526+0000","last_became_active":"2024-05-10T10:33:38.536902+0000","last_became_peered":"2024-05-10T10:33:38.536902+0000","last_unstale":"2024-05-10T10:33:39.554526+0000","last_undegraded":"2024-05-10T10:33:39.554526+0000","last_fullsized":"2024-05-10T10:33:39.554526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640318+0000","last_change":"2024-05-10T10:33:34.505242+0000","last_active":"2024-05-10T10:33:39.640318+0000","last_peered":"2024-05-10T10:33:39.640318+0000","last_clean":"2024-05-10T10:33:39.640318+0000","last_became_active":"2024-05-10T10:33:34.504914+0000","last_became_peered":"2024-05-10T10:33:34.504914+0000","last_unstale":"2024-05-10T10:33:39.640318+0000","last_undegraded":"2024-05-10T10:33:39.640318+0000","last_fullsized":"2024-05-10T10:33:39.640318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640485+0000","last_change":"2024-05-10T10:33:33.498662+0000","last_active":"2024-05-10T10:33:39.640485+0000","last_peered":"2024-05-10T10:33:39.640485+0000","last_clean":"2024-05-10T10:33:39.640485+0000","last_became_active":"2024-05-10T10:33:33.498294+0000","last_became_peered":"2024-05-10T10:33:33.498294+0000","last_unstale":"2024-05-10T10:33:39.640485+0000","last_undegraded":"2024-05-10T10:33:39.640485+0000","last_fullsized":"2024-05-10T10:33:39.640485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553501+0000","last_change":"2024-05-10T10:33:33.495684+0000","last_active":"2024-05-10T10:33:39.553501+0000","last_peered":"2024-05-10T10:33:39.553501+0000","last_clean":"2024-05-10T10:33:39.553501+0000","last_became_active":"2024-05-10T10:33:33.495347+0000","last_became_peered":"2024-05-10T10:33:33.495347+0000","last_unstale":"2024-05-10T10:33:39.553501+0000","last_undegraded":"2024-05-10T10:33:39.553501+0000","last_fullsized":"2024-05-10T10:33:39.553501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553543+0000","last_change":"2024-05-10T10:33:34.502038+0000","last_active":"2024-05-10T10:33:39.553543+0000","last_peered":"2024-05-10T10:33:39.553543+0000","last_clean":"2024-05-10T10:33:39.553543+0000","last_became_active":"2024-05-10T10:33:34.501709+0000","last_became_peered":"2024-05-10T10:33:34.501709+0000","last_unstale":"2024-05-10T10:33:39.553543+0000","last_undegraded":"2024-05-10T10:33:39.553543+0000","last_fullsized":"2024-05-10T10:33:39.553543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679952+0000","last_change":"2024-05-10T10:33:38.536172+0000","last_active":"2024-05-10T10:33:39.679952+0000","last_peered":"2024-05-10T10:33:39.679952+0000","last_clean":"2024-05-10T10:33:39.679952+0000","last_became_active":"2024-05-10T10:33:38.535590+0000","last_became_peered":"2024-05-10T10:33:38.535590+0000","last_unstale":"2024-05-10T10:33:39.679952+0000","last_undegraded":"2024-05-10T10:33:39.679952+0000","last_fullsized":"2024-05-10T10:33:39.679952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:07.728619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640673+0000","last_change":"2024-05-10T10:33:33.497950+0000","last_active":"2024-05-10T10:33:39.640673+0000","last_peered":"2024-05-10T10:33:39.640673+0000","last_clean":"2024-05-10T10:33:39.640673+0000","last_became_active":"2024-05-10T10:33:33.497590+0000","last_became_peered":"2024-05-10T10:33:33.497590+0000","last_unstale":"2024-05-10T10:33:39.640673+0000","last_undegraded":"2024-05-10T10:33:39.640673+0000","last_fullsized":"2024-05-10T10:33:39.640673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872524+0000","last_change":"2024-05-10T10:33:34.508520+0000","last_active":"2024-05-10T10:33:40.872524+0000","last_peered":"2024-05-10T10:33:40.872524+0000","last_clean":"2024-05-10T10:33:40.872524+0000","last_became_active":"2024-05-10T10:33:34.508262+0000","last_became_peered":"2024-05-10T10:33:34.508262+0000","last_unstale":"2024-05-10T10:33:40.872524+0000","last_undegraded":"2024-05-10T10:33:40.872524+0000","last_fullsized":"2024-05-10T10:33:40.872524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:54.281454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640675+0000","last_change":"2024-05-10T10:33:38.542608+0000","last_active":"2024-05-10T10:33:39.640675+0000","last_peered":"2024-05-10T10:33:39.640675+0000","last_clean":"2024-05-10T10:33:39.640675+0000","last_became_active":"2024-05-10T10:33:38.540900+0000","last_became_peered":"2024-05-10T10:33:38.540900+0000","last_unstale":"2024-05-10T10:33:39.640675+0000","last_undegraded":"2024-05-10T10:33:39.640675+0000","last_fullsized":"2024-05-10T10:33:39.640675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:41.505822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552881+0000","last_change":"2024-05-10T10:33:34.510874+0000","last_active":"2024-05-10T10:33:39.552881+0000","last_peered":"2024-05-10T10:33:39.552881+0000","last_clean":"2024-05-10T10:33:39.552881+0000","last_became_active":"2024-05-10T10:33:34.510623+0000","last_became_peered":"2024-05-10T10:33:34.510623+0000","last_unstale":"2024-05-10T10:33:39.552881+0000","last_undegraded":"2024-05-10T10:33:39.552881+0000","last_fullsized":"2024-05-10T10:33:39.552881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679646+0000","last_change":"2024-05-10T10:33:34.503232+0000","last_active":"2024-05-10T10:33:39.679646+0000","last_peered":"2024-05-10T10:33:39.679646+0000","last_clean":"2024-05-10T10:33:39.679646+0000","last_became_active":"2024-05-10T10:33:34.502932+0000","last_became_peered":"2024-05-10T10:33:34.502932+0000","last_unstale":"2024-05-10T10:33:39.679646+0000","last_undegraded":"2024-05-10T10:33:39.679646+0000","last_fullsized":"2024-05-10T10:33:39.679646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871608+0000","last_change":"2024-05-10T10:33:34.516049+0000","last_active":"2024-05-10T10:33:40.871608+0000","last_peered":"2024-05-10T10:33:40.871608+0000","last_clean":"2024-05-10T10:33:40.871608+0000","last_became_active":"2024-05-10T10:33:34.515617+0000","last_became_peered":"2024-05-10T10:33:34.515617+0000","last_unstale":"2024-05-10T10:33:40.871608+0000","last_undegraded":"2024-05-10T10:33:40.871608+0000","last_fullsized":"2024-05-10T10:33:40.871608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553405+0000","last_change":"2024-05-10T10:33:34.508355+0000","last_active":"2024-05-10T10:33:39.553405+0000","last_peered":"2024-05-10T10:33:39.553405+0000","last_clean":"2024-05-10T10:33:39.553405+0000","last_became_active":"2024-05-10T10:33:34.508064+0000","last_became_peered":"2024-05-10T10:33:34.508064+0000","last_unstale":"2024-05-10T10:33:39.553405+0000","last_undegraded":"2024-05-10T10:33:39.553405+0000","last_fullsized":"2024-05-10T10:33:39.553405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:29.644124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640952+0000","last_change":"2024-05-10T10:33:34.508376+0000","last_active":"2024-05-10T10:33:39.640952+0000","last_peered":"2024-05-10T10:33:39.640952+0000","last_clean":"2024-05-10T10:33:39.640952+0000","last_became_active":"2024-05-10T10:33:34.507993+0000","last_became_peered":"2024-05-10T10:33:34.507993+0000","last_unstale":"2024-05-10T10:33:39.640952+0000","last_undegraded":"2024-05-10T10:33:39.640952+0000","last_fullsized":"2024-05-10T10:33:39.640952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885895+0000","last_change":"2024-05-10T10:33:34.508840+0000","last_active":"2024-05-10T10:33:39.885895+0000","last_peered":"2024-05-10T10:33:39.885895+0000","last_clean":"2024-05-10T10:33:39.885895+0000","last_became_active":"2024-05-10T10:33:34.508524+0000","last_became_peered":"2024-05-10T10:33:34.508524+0000","last_unstale":"2024-05-10T10:33:39.885895+0000","last_undegraded":"2024-05-10T10:33:39.885895+0000","last_fullsized":"2024-05-10T10:33:39.885895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554603+0000","last_change":"2024-05-10T10:33:34.513208+0000","last_active":"2024-05-10T10:33:39.554603+0000","last_peered":"2024-05-10T10:33:39.554603+0000","last_clean":"2024-05-10T10:33:39.554603+0000","last_became_active":"2024-05-10T10:33:34.512924+0000","last_became_peered":"2024-05-10T10:33:34.512924+0000","last_unstale":"2024-05-10T10:33:39.554603+0000","last_undegraded":"2024-05-10T10:33:39.554603+0000","last_fullsized":"2024-05-10T10:33:39.554603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641017+0000","last_change":"2024-05-10T10:33:34.504050+0000","last_active":"2024-05-10T10:33:39.641017+0000","last_peered":"2024-05-10T10:33:39.641017+0000","last_clean":"2024-05-10T10:33:39.641017+0000","last_became_active":"2024-05-10T10:33:34.503749+0000","last_became_peered":"2024-05-10T10:33:34.503749+0000","last_unstale":"2024-05-10T10:33:39.641017+0000","last_undegraded":"2024-05-10T10:33:39.641017+0000","last_fullsized":"2024-05-10T10:33:39.641017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871417+0000","last_change":"2024-05-10T10:33:34.508259+0000","last_active":"2024-05-10T10:33:40.871417+0000","last_peered":"2024-05-10T10:33:40.871417+0000","last_clean":"2024-05-10T10:33:40.871417+0000","last_became_active":"2024-05-10T10:33:34.507953+0000","last_became_peered":"2024-05-10T10:33:34.507953+0000","last_unstale":"2024-05-10T10:33:40.871417+0000","last_undegraded":"2024-05-10T10:33:40.871417+0000","last_fullsized":"2024-05-10T10:33:40.871417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640457+0000","last_change":"2024-05-10T10:33:34.505284+0000","last_active":"2024-05-10T10:33:39.640457+0000","last_peered":"2024-05-10T10:33:39.640457+0000","last_clean":"2024-05-10T10:33:39.640457+0000","last_became_active":"2024-05-10T10:33:34.505010+0000","last_became_peered":"2024-05-10T10:33:34.505010+0000","last_unstale":"2024-05-10T10:33:39.640457+0000","last_undegraded":"2024-05-10T10:33:39.640457+0000","last_fullsized":"2024-05-10T10:33:39.640457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871555+0000","last_change":"2024-05-10T10:33:34.511213+0000","last_active":"2024-05-10T10:33:40.871555+0000","last_peered":"2024-05-10T10:33:40.871555+0000","last_clean":"2024-05-10T10:33:40.871555+0000","last_became_active":"2024-05-10T10:33:34.509309+0000","last_became_peered":"2024-05-10T10:33:34.509309+0000","last_unstale":"2024-05-10T10:33:40.871555+0000","last_undegraded":"2024-05-10T10:33:40.871555+0000","last_fullsized":"2024-05-10T10:33:40.871555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553124+0000","last_change":"2024-05-10T10:33:34.508899+0000","last_active":"2024-05-10T10:33:39.553124+0000","last_peered":"2024-05-10T10:33:39.553124+0000","last_clean":"2024-05-10T10:33:39.553124+0000","last_became_active":"2024-05-10T10:33:34.508611+0000","last_became_peered":"2024-05-10T10:33:34.508611+0000","last_unstale":"2024-05-10T10:33:39.553124+0000","last_undegraded":"2024-05-10T10:33:39.553124+0000","last_fullsized":"2024-05-10T10:33:39.553124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:03.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553705+0000","last_change":"2024-05-10T10:33:34.520178+0000","last_active":"2024-05-10T10:33:39.553705+0000","last_peered":"2024-05-10T10:33:39.553705+0000","last_clean":"2024-05-10T10:33:39.553705+0000","last_became_active":"2024-05-10T10:33:34.519912+0000","last_became_peered":"2024-05-10T10:33:34.519912+0000","last_unstale":"2024-05-10T10:33:39.553705+0000","last_undegraded":"2024-05-10T10:33:39.553705+0000","last_fullsized":"2024-05-10T10:33:39.553705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554455+0000","last_change":"2024-05-10T10:33:34.515897+0000","last_active":"2024-05-10T10:33:39.554455+0000","last_peered":"2024-05-10T10:33:39.554455+0000","last_clean":"2024-05-10T10:33:39.554455+0000","last_became_active":"2024-05-10T10:33:34.514938+0000","last_became_peered":"2024-05-10T10:33:34.514938+0000","last_unstale":"2024-05-10T10:33:39.554455+0000","last_undegraded":"2024-05-10T10:33:39.554455+0000","last_fullsized":"2024-05-10T10:33:39.554455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:28:07.142173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680141+0000","last_change":"2024-05-10T10:33:34.523364+0000","last_active":"2024-05-10T10:33:39.680141+0000","last_peered":"2024-05-10T10:33:39.680141+0000","last_clean":"2024-05-10T10:33:39.680141+0000","last_became_active":"2024-05-10T10:33:34.522972+0000","last_became_peered":"2024-05-10T10:33:34.522972+0000","last_unstale":"2024-05-10T10:33:39.680141+0000","last_undegraded":"2024-05-10T10:33:39.680141+0000","last_fullsized":"2024-05-10T10:33:39.680141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872257+0000","last_change":"2024-05-10T10:33:34.508555+0000","last_active":"2024-05-10T10:33:40.872257+0000","last_peered":"2024-05-10T10:33:40.872257+0000","last_clean":"2024-05-10T10:33:40.872257+0000","last_became_active":"2024-05-10T10:33:34.508218+0000","last_became_peered":"2024-05-10T10:33:34.508218+0000","last_unstale":"2024-05-10T10:33:40.872257+0000","last_undegraded":"2024-05-10T10:33:40.872257+0000","last_fullsized":"2024-05-10T10:33:40.872257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871300+0000","last_change":"2024-05-10T10:33:34.512706+0000","last_active":"2024-05-10T10:33:40.871300+0000","last_peered":"2024-05-10T10:33:40.871300+0000","last_clean":"2024-05-10T10:33:40.871300+0000","last_became_active":"2024-05-10T10:33:34.511868+0000","last_became_peered":"2024-05-10T10:33:34.511868+0000","last_unstale":"2024-05-10T10:33:40.871300+0000","last_undegraded":"2024-05-10T10:33:40.871300+0000","last_fullsized":"2024-05-10T10:33:40.871300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885490+0000","last_change":"2024-05-10T10:33:34.504724+0000","last_active":"2024-05-10T10:33:39.885490+0000","last_peered":"2024-05-10T10:33:39.885490+0000","last_clean":"2024-05-10T10:33:39.885490+0000","last_became_active":"2024-05-10T10:33:34.504407+0000","last_became_peered":"2024-05-10T10:33:34.504407+0000","last_unstale":"2024-05-10T10:33:39.885490+0000","last_undegraded":"2024-05-10T10:33:39.885490+0000","last_fullsized":"2024-05-10T10:33:39.885490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553396+0000","last_change":"2024-05-10T10:33:34.507001+0000","last_active":"2024-05-10T10:33:39.553396+0000","last_peered":"2024-05-10T10:33:39.553396+0000","last_clean":"2024-05-10T10:33:39.553396+0000","last_became_active":"2024-05-10T10:33:34.506678+0000","last_became_peered":"2024-05-10T10:33:34.506678+0000","last_unstale":"2024-05-10T10:33:39.553396+0000","last_undegraded":"2024-05-10T10:33:39.553396+0000","last_fullsized":"2024-05-10T10:33:39.553396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640530+0000","last_change":"2024-05-10T10:33:34.516536+0000","last_active":"2024-05-10T10:33:39.640530+0000","last_peered":"2024-05-10T10:33:39.640530+0000","last_clean":"2024-05-10T10:33:39.640530+0000","last_became_active":"2024-05-10T10:33:34.515018+0000","last_became_peered":"2024-05-10T10:33:34.515018+0000","last_unstale":"2024-05-10T10:33:39.640530+0000","last_undegraded":"2024-05-10T10:33:39.640530+0000","last_fullsized":"2024-05-10T10:33:39.640530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872943+0000","last_change":"2024-05-10T10:33:34.517149+0000","last_active":"2024-05-10T10:33:40.872943+0000","last_peered":"2024-05-10T10:33:40.872943+0000","last_clean":"2024-05-10T10:33:40.872943+0000","last_became_active":"2024-05-10T10:33:34.516875+0000","last_became_peered":"2024-05-10T10:33:34.516875+0000","last_unstale":"2024-05-10T10:33:40.872943+0000","last_undegraded":"2024-05-10T10:33:40.872943+0000","last_fullsized":"2024-05-10T10:33:40.872943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553216+0000","last_change":"2024-05-10T10:33:34.518781+0000","last_active":"2024-05-10T10:33:39.553216+0000","last_peered":"2024-05-10T10:33:39.553216+0000","last_clean":"2024-05-10T10:33:39.553216+0000","last_became_active":"2024-05-10T10:33:34.518457+0000","last_became_peered":"2024-05-10T10:33:34.518457+0000","last_unstale":"2024-05-10T10:33:39.553216+0000","last_undegraded":"2024-05-10T10:33:39.553216+0000","last_fullsized":"2024-05-10T10:33:39.553216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554503+0000","last_change":"2024-05-10T10:33:34.518753+0000","last_active":"2024-05-10T10:33:39.554503+0000","last_peered":"2024-05-10T10:33:39.554503+0000","last_clean":"2024-05-10T10:33:39.554503+0000","last_became_active":"2024-05-10T10:33:34.518443+0000","last_became_peered":"2024-05-10T10:33:34.518443+0000","last_unstale":"2024-05-10T10:33:39.554503+0000","last_undegraded":"2024-05-10T10:33:39.554503+0000","last_fullsized":"2024-05-10T10:33:39.554503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554243+0000","last_change":"2024-05-10T10:33:34.517952+0000","last_active":"2024-05-10T10:33:39.554243+0000","last_peered":"2024-05-10T10:33:39.554243+0000","last_clean":"2024-05-10T10:33:39.554243+0000","last_became_active":"2024-05-10T10:33:34.517670+0000","last_became_peered":"2024-05-10T10:33:34.517670+0000","last_unstale":"2024-05-10T10:33:39.554243+0000","last_undegraded":"2024-05-10T10:33:39.554243+0000","last_fullsized":"2024-05-10T10:33:39.554243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:57.405468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640217+0000","last_change":"2024-05-10T10:33:34.508563+0000","last_active":"2024-05-10T10:33:39.640217+0000","last_peered":"2024-05-10T10:33:39.640217+0000","last_clean":"2024-05-10T10:33:39.640217+0000","last_became_active":"2024-05-10T10:33:34.508082+0000","last_became_peered":"2024-05-10T10:33:34.508082+0000","last_unstale":"2024-05-10T10:33:39.640217+0000","last_undegraded":"2024-05-10T10:33:39.640217+0000","last_fullsized":"2024-05-10T10:33:39.640217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553714+0000","last_change":"2024-05-10T10:33:34.513986+0000","last_active":"2024-05-10T10:33:39.553714+0000","last_peered":"2024-05-10T10:33:39.553714+0000","last_clean":"2024-05-10T10:33:39.553714+0000","last_became_active":"2024-05-10T10:33:34.513734+0000","last_became_peered":"2024-05-10T10:33:34.513734+0000","last_unstale":"2024-05-10T10:33:39.553714+0000","last_undegraded":"2024-05-10T10:33:39.553714+0000","last_fullsized":"2024-05-10T10:33:39.553714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871457+0000","last_change":"2024-05-10T10:33:34.509577+0000","last_active":"2024-05-10T10:33:40.871457+0000","last_peered":"2024-05-10T10:33:40.871457+0000","last_clean":"2024-05-10T10:33:40.871457+0000","last_became_active":"2024-05-10T10:33:34.508968+0000","last_became_peered":"2024-05-10T10:33:34.508968+0000","last_unstale":"2024-05-10T10:33:40.871457+0000","last_undegraded":"2024-05-10T10:33:40.871457+0000","last_fullsized":"2024-05-10T10:33:40.871457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553509+0000","last_change":"2024-05-10T10:33:34.515157+0000","last_active":"2024-05-10T10:33:39.553509+0000","last_peered":"2024-05-10T10:33:39.553509+0000","last_clean":"2024-05-10T10:33:39.553509+0000","last_became_active":"2024-05-10T10:33:34.514884+0000","last_became_peered":"2024-05-10T10:33:34.514884+0000","last_unstale":"2024-05-10T10:33:39.553509+0000","last_undegraded":"2024-05-10T10:33:39.553509+0000","last_fullsized":"2024-05-10T10:33:39.553509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871783+0000","last_change":"2024-05-10T10:33:34.513846+0000","last_active":"2024-05-10T10:33:40.871783+0000","last_peered":"2024-05-10T10:33:40.871783+0000","last_clean":"2024-05-10T10:33:40.871783+0000","last_became_active":"2024-05-10T10:33:34.513538+0000","last_became_peered":"2024-05-10T10:33:34.513538+0000","last_unstale":"2024-05-10T10:33:40.871783+0000","last_undegraded":"2024-05-10T10:33:40.871783+0000","last_fullsized":"2024-05-10T10:33:40.871783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553166+0000","last_change":"2024-05-10T10:33:34.503972+0000","last_active":"2024-05-10T10:33:39.553166+0000","last_peered":"2024-05-10T10:33:39.553166+0000","last_clean":"2024-05-10T10:33:39.553166+0000","last_became_active":"2024-05-10T10:33:34.501490+0000","last_became_peered":"2024-05-10T10:33:34.501490+0000","last_unstale":"2024-05-10T10:33:39.553166+0000","last_undegraded":"2024-05-10T10:33:39.553166+0000","last_fullsized":"2024-05-10T10:33:39.553166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885974+0000","last_change":"2024-05-10T10:33:34.502166+0000","last_active":"2024-05-10T10:33:39.885974+0000","last_peered":"2024-05-10T10:33:39.885974+0000","last_clean":"2024-05-10T10:33:39.885974+0000","last_became_active":"2024-05-10T10:33:34.501852+0000","last_became_peered":"2024-05-10T10:33:34.501852+0000","last_unstale":"2024-05-10T10:33:39.885974+0000","last_undegraded":"2024-05-10T10:33:39.885974+0000","last_fullsized":"2024-05-10T10:33:39.885974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554036+0000","last_change":"2024-05-10T10:33:34.522785+0000","last_active":"2024-05-10T10:33:39.554036+0000","last_peered":"2024-05-10T10:33:39.554036+0000","last_clean":"2024-05-10T10:33:39.554036+0000","last_became_active":"2024-05-10T10:33:34.522413+0000","last_became_peered":"2024-05-10T10:33:34.522413+0000","last_unstale":"2024-05-10T10:33:39.554036+0000","last_undegraded":"2024-05-10T10:33:39.554036+0000","last_fullsized":"2024-05-10T10:33:39.554036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":101868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":266240,"data_stored":101868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:33:48.985 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:33:48.985 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T10:33:48.986 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-10T10:33:48.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-10T10:33:48.986 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-10T10:33:49.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.122+0000 7f3c388c8640 1 Processor -- start 2024-05-10T10:33:49.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.122+0000 7f3c388c8640 1 -- start start 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.123+0000 7f3c388c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c3406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.123+0000 7f3c388c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3c3406c800 0x7f3c340aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.123+0000 7f3c388c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c34163bd0 con 0x7f3c340af4b0 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.123+0000 7f3c388c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3c3406cc60 con 0x7f3c3406c800 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.123+0000 7f3c388c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3c34162ab0 con 0x7f3c341626d0 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c31d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3c3406c800 0x7f3c340aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c3259c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c3406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c32d9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3c340af4b0 0x7f3c34073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36424/0 (socket says 172.21.15.53:36424) 2024-05-10T10:33:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c32d9d640 1 -- 172.21.15.53:0/1426683255 learned_addr learned my addr 172.21.15.53:0/1426683255 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:49.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c31d9b640 1 -- 172.21.15.53:0/1426683255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 msgr2=0x7f3c3406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:49.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c31d9b640 1 --2- 172.21.15.53:0/1426683255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c3406c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c31d9b640 1 -- 172.21.15.53:0/1426683255 >> v1:172.21.15.135:6789/0 conn(0x7f3c340af4b0 legacy=0x7f3c34073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.124+0000 7f3c31d9b640 1 -- 172.21.15.53:0/1426683255 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c340af940 con 0x7f3c3406c800 2024-05-10T10:33:49.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.125+0000 7f3c31d9b640 1 --2- 172.21.15.53:0/1426683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340aef70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0097d0 tx=0x7f3c1c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8c8fa0ac236727 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.125+0000 7f3c3159a640 1 -- 172.21.15.53:0/1426683255 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c037070 con 0x7f3c3406c800 2024-05-10T10:33:49.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.125+0000 7f3c3159a640 1 -- 172.21.15.53:0/1426683255 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c02ae40 con 0x7f3c3406c800 2024-05-10T10:33:49.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.125+0000 7f3c3159a640 1 -- 172.21.15.53:0/1426683255 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c032660 con 0x7f3c3406c800 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 -- 172.21.15.53:0/1426683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 msgr2=0x7f3c340aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 --2- 172.21.15.53:0/1426683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340aef70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0097d0 tx=0x7f3c1c02a130 comp rx=0 tx=0).stop 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 -- 172.21.15.53:0/1426683255 shutdown_connections 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 --2- 172.21.15.53:0/1426683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340aef70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 --2- 172.21.15.53:0/1426683255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c3406c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 -- 172.21.15.53:0/1426683255 >> 172.21.15.53:0/1426683255 conn(0x7f3c340d1d70 msgr2=0x7f3c340d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 -- 172.21.15.53:0/1426683255 shutdown_connections 2024-05-10T10:33:49.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.126+0000 7f3c388c8640 1 -- 172.21.15.53:0/1426683255 wait complete. 2024-05-10T10:33:49.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.127+0000 7f3c388c8640 1 Processor -- start 2024-05-10T10:33:49.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.127+0000 7f3c388c8640 1 -- start start 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c388c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c388c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c340af4b0 0x7f3c34150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c388c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c341e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c388c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c341633d0 con 0x7f3c340af4b0 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c388c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3c341630f0 con 0x7f3c3406c800 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c388c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3c341636b0 con 0x7f3c341626d0 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c340af4b0 0x7f3c34150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c340af4b0 0x7f3c34150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43268/0 (socket says 172.21.15.53:43268) 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 -- 172.21.15.53:0/66361158 learned_addr learned my addr 172.21.15.53:0/66361158 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c3259c640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c32d9d640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c341e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 -- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 msgr2=0x7f3c341e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c341e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 -- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 msgr2=0x7f3c340c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c31d9b640 1 -- 172.21.15.53:0/66361158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c34150db0 con 0x7f3c340af4b0 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.128+0000 7f3c3259c640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.129+0000 7f3c31d9b640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c340af4b0 0x7f3c34150870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c035040 tx=0x7f3c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.129+0000 7f3c2affd640 1 -- 172.21.15.53:0/66361158 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c041070 con 0x7f3c340af4b0 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.129+0000 7f3c2affd640 1 -- 172.21.15.53:0/66361158 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c03c460 con 0x7f3c340af4b0 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.129+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c341f5030 con 0x7f3c340af4b0 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.129+0000 7f3c2affd640 1 -- 172.21.15.53:0/66361158 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c037040 con 0x7f3c340af4b0 2024-05-10T10:33:49.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.129+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c341e9cc0 con 0x7f3c340af4b0 2024-05-10T10:33:49.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.130+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c340693f0 con 0x7f3c340af4b0 2024-05-10T10:33:49.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.130+0000 7f3c2affd640 1 -- 172.21.15.53:0/66361158 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3c1c044070 con 0x7f3c340af4b0 2024-05-10T10:33:49.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.131+0000 7f3c2affd640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c140762d0 0x7f3c14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.131+0000 7f3c2affd640 1 -- 172.21.15.53:0/66361158 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3c1c0b7ea0 con 0x7f3c340af4b0 2024-05-10T10:33:49.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.131+0000 7f3c3259c640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c140762d0 0x7f3c14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.132+0000 7f3c3259c640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c140762d0 0x7f3c14078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c340c8400 tx=0x7f3c2400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.135+0000 7f3c2affd640 1 -- 172.21.15.53:0/66361158 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3c34150db0 con 0x7f3c340af4b0 2024-05-10T10:33:49.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.382+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3c34075290 con 0x7f3c340af4b0 2024-05-10T10:33:49.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.385+0000 7f3c2affd640 1 -- 172.21.15.53:0/66361158 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189526 (secure 0 0 0) 0x7f3c1c080e70 con 0x7f3c340af4b0 2024-05-10T10:33:49.389 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c140762d0 msgr2=0x7f3c14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c140762d0 0x7f3c14078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c340c8400 tx=0x7f3c2400b040 comp rx=0 tx=0).stop 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c340af4b0 msgr2=0x7f3c34150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c340af4b0 0x7f3c34150870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c035040 tx=0x7f3c1c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 shutdown_connections 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c140762d0 0x7f3c14078790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c341626d0 0x7f3c341e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c340af4b0 0x7f3c34150870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 --2- 172.21.15.53:0/66361158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c3406c800 0x7f3c340c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 >> 172.21.15.53:0/66361158 conn(0x7f3c340d1d70 msgr2=0x7f3c340b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 shutdown_connections 2024-05-10T10:33:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.388+0000 7f3c388c8640 1 -- 172.21.15.53:0/66361158 wait complete. 2024-05-10T10:33:49.401 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1385963166},{"type":"v1","addr":"172.21.15.135:6833","nonce":1385963166}]},"active_addr":"172.21.15.135:6833/1385963166","active_change":"2024-05-10T10:33:22.146886+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":254301089}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1705765367}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":831157869}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1573451865}]}]} 2024-05-10T10:33:49.405 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-10T10:33:49.405 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-10T10:33:49.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:49.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.508+0000 7f4d0ca2a640 1 Processor -- start 2024-05-10T10:33:49.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.508+0000 7f4d0ca2a640 1 -- start start 2024-05-10T10:33:49.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d0ca2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d0ca2a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d080d2270 0x7f4d080d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d0ca2a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d080b5e80 con 0x7f4d080d3980 2024-05-10T10:33:49.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d0ca2a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4d080b5c30 con 0x7f4d080d2270 2024-05-10T10:33:49.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d0ca2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4d080d5c50 con 0x7f4d08162dd0 2024-05-10T10:33:49.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d06d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d080d2270 0x7f4d080d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d0659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4d080d3980 0x7f4d081626c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39708/0 (socket says 172.21.15.53:39708) 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.509+0000 7f4d0659c640 1 -- 172.21.15.53:0/849394177 learned_addr learned my addr 172.21.15.53:0/849394177 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d06d9d640 1 -- 172.21.15.53:0/849394177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 msgr2=0x7f4d080cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d05d9b640 1 --2- 172.21.15.53:0/849394177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d06d9d640 1 --2- 172.21.15.53:0/849394177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d06d9d640 1 -- 172.21.15.53:0/849394177 >> v1:172.21.15.53:6789/0 conn(0x7f4d080d3980 legacy=0x7f4d081626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d0559a640 1 -- 172.21.15.53:0/849394177 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225264996 0 0) 0x7f4d080b5e80 con 0x7f4d080d3980 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d06d9d640 1 -- 172.21.15.53:0/849394177 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d080d5f30 con 0x7f4d080d2270 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d05d9b640 1 --2- 172.21.15.53:0/849394177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.510+0000 7f4d06d9d640 1 --2- 172.21.15.53:0/849394177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d2270 0x7f4d080d03f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d080b5db0 tx=0x7f4d0002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbff4a869b04788 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0559a640 1 -- 172.21.15.53:0/849394177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0002ae60 con 0x7f4d080d2270 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0559a640 1 -- 172.21.15.53:0/849394177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d00007740 con 0x7f4d080d2270 2024-05-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0559a640 1 -- 172.21.15.53:0/849394177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00007b10 con 0x7f4d080d2270 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/849394177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d2270 msgr2=0x7f4d080d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/849394177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d2270 0x7f4d080d03f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d080b5db0 tx=0x7f4d0002a640 comp rx=0 tx=0).stop 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/849394177 shutdown_connections 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/849394177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d2270 0x7f4d080d03f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/849394177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/849394177 >> 172.21.15.53:0/849394177 conn(0x7f4d080d5130 msgr2=0x7f4d080d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/849394177 shutdown_connections 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.511+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/849394177 wait complete. 2024-05-10T10:33:49.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 Processor -- start 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 -- start start 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d080d2270 0x7f4d081548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d3980 0x7f4d080b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4d080b6890 con 0x7f4d080d3980 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d080d31f0 con 0x7f4d080d2270 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.513+0000 7f4d0ca2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4d080b6160 con 0x7f4d08162dd0 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d3980 0x7f4d080b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d3980 0x7f4d080b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43282/0 (socket says 172.21.15.53:43282) 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 -- 172.21.15.53:0/3158973223 learned_addr learned my addr 172.21.15.53:0/3158973223 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d06d9d640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d0659c640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d080d2270 0x7f4d081548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 -- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 msgr2=0x7f4d080b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 -- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d080d2270 msgr2=0x7f4d081548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d080d2270 0x7f4d081548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 -- 172.21.15.53:0/3158973223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d080b8750 con 0x7f4d080d3980 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d0659c640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d080d2270 0x7f4d081548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d06d9d640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d05d9b640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d3980 0x7f4d080b7f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c7c0 tx=0x7f4cf000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4cfeffd640 1 -- 172.21.15.53:0/3158973223 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0015070 con 0x7f4d080d3980 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4cfeffd640 1 -- 172.21.15.53:0/3158973223 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf0007820 con 0x7f4d080d3980 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4cfeffd640 1 -- 172.21.15.53:0/3158973223 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0007bf0 con 0x7f4d080d3980 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d080b8a30 con 0x7f4d080d3980 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.514+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d080cd5f0 con 0x7f4d080d3980 2024-05-10T10:33:49.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.515+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0806a220 con 0x7f4d080d3980 2024-05-10T10:33:49.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.516+0000 7f4cfeffd640 1 -- 172.21.15.53:0/3158973223 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4cf0018070 con 0x7f4d080d3980 2024-05-10T10:33:49.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.516+0000 7f4cfeffd640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ce80762d0 0x7f4ce8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.517+0000 7f4cfeffd640 1 -- 172.21.15.53:0/3158973223 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4cf00978d0 con 0x7f4d080d3980 2024-05-10T10:33:49.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.517+0000 7f4d0659c640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ce80762d0 0x7f4ce8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.517+0000 7f4d0659c640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ce80762d0 0x7f4ce8078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8000f60 tx=0x7f4cf8007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.521+0000 7f4cfeffd640 1 -- 172.21.15.53:0/3158973223 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4cf0060e50 con 0x7f4d080d3980 2024-05-10T10:33:49.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.713+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d080b4410 con 0x7f4d080d3980 2024-05-10T10:33:49.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.714+0000 7f4cfeffd640 1 -- 172.21.15.53:0/3158973223 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f4cf0060750 con 0x7f4d080d3980 2024-05-10T10:33:49.716 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:49.716 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ce80762d0 msgr2=0x7f4ce8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ce80762d0 0x7f4ce8078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8000f60 tx=0x7f4cf8007c10 comp rx=0 tx=0).stop 2024-05-10T10:33:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d3980 msgr2=0x7f4d080b7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d3980 0x7f4d080b7f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c7c0 tx=0x7f4cf000cc80 comp rx=0 tx=0).stop 2024-05-10T10:33:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 shutdown_connections 2024-05-10T10:33:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ce80762d0 0x7f4ce8078790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d08162dd0 0x7f4d080b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d080d3980 0x7f4d080b7f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 --2- 172.21.15.53:0/3158973223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d080d2270 0x7f4d081548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 >> 172.21.15.53:0/3158973223 conn(0x7f4d080d5130 msgr2=0x7f4d08161c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 shutdown_connections 2024-05-10T10:33:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.716+0000 7f4d0ca2a640 1 -- 172.21.15.53:0/3158973223 wait complete. 2024-05-10T10:33:49.729 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-10T10:33:49.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:49.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.831+0000 7fb8b1103640 1 Processor -- start 2024-05-10T10:33:49.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.831+0000 7fb8b1103640 1 -- start start 2024-05-10T10:33:49.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8b1103640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8ac15f390 0x7fb8ac0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8b1103640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8b1103640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8ac0cfd10 con 0x7fb8ac0d5190 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8b1103640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb8ac0cff90 con 0x7fb8ac15f390 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8b1103640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb8ac163d00 con 0x7fb8ac0b4c00 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8aa59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb8ac0d5190 0x7fb8ac0d55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39714/0 (socket says 172.21.15.53:39714) 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8aa59c640 1 -- 172.21.15.53:0/2100670771 learned_addr learned my addr 172.21.15.53:0/2100670771 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8aad9d640 1 --2- 172.21.15.53:0/2100670771 >> v2:172.21.15.135:3300/0 conn(0x7fb8ac15f390 0x7fb8ac0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.832+0000 7fb8ab59e640 1 --2- 172.21.15.53:0/2100670771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8a9d9b640 1 -- 172.21.15.53:0/2100670771 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1074139861 0 0) 0x7fb894001110 con 0x7fb8ac0d5190 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8a9d9b640 1 -- 172.21.15.53:0/2100670771 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb898003520 con 0x7fb8ac0d5190 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8ab59e640 1 -- 172.21.15.53:0/2100670771 >> v2:172.21.15.135:3300/0 conn(0x7fb8ac15f390 msgr2=0x7fb8ac0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8ab59e640 1 --2- 172.21.15.53:0/2100670771 >> v2:172.21.15.135:3300/0 conn(0x7fb8ac15f390 0x7fb8ac0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8ab59e640 1 -- 172.21.15.53:0/2100670771 >> v1:172.21.15.53:6789/0 conn(0x7fb8ac0d5190 legacy=0x7fb8ac0d55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8a9d9b640 1 -- 172.21.15.53:0/2100670771 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3849382594 0 0) 0x7fb898003520 con 0x7fb8ac0d5190 2024-05-10T10:33:49.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8ab59e640 1 -- 172.21.15.53:0/2100670771 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac0d0250 con 0x7fb8ac0b4c00 2024-05-10T10:33:49.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.833+0000 7fb8aad9d640 1 --2- 172.21.15.53:0/2100670771 >> v2:172.21.15.135:3300/0 conn(0x7fb8ac15f390 0x7fb8ac0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:49.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8ab59e640 1 --2- 172.21.15.53:0/2100670771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0b5070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009d90 tx=0x7fb89c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a545dc7c887062 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8a9d9b640 1 -- 172.21.15.53:0/2100670771 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894001840 con 0x7fb8ac0b4c00 2024-05-10T10:33:49.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8a9d9b640 1 -- 172.21.15.53:0/2100670771 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89c007540 con 0x7fb8ac0b4c00 2024-05-10T10:33:49.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8a9d9b640 1 -- 172.21.15.53:0/2100670771 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c007910 con 0x7fb8ac0b4c00 2024-05-10T10:33:49.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 -- 172.21.15.53:0/2100670771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 msgr2=0x7fb8ac0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 --2- 172.21.15.53:0/2100670771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0b5070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009d90 tx=0x7fb89c02a5f0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 -- 172.21.15.53:0/2100670771 shutdown_connections 2024-05-10T10:33:49.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 --2- 172.21.15.53:0/2100670771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0b5070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 --2- 172.21.15.53:0/2100670771 >> v2:172.21.15.135:3300/0 conn(0x7fb8ac15f390 0x7fb8ac0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 -- 172.21.15.53:0/2100670771 >> 172.21.15.53:0/2100670771 conn(0x7fb8ac08ecb0 msgr2=0x7fb8ac09e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:49.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 -- 172.21.15.53:0/2100670771 shutdown_connections 2024-05-10T10:33:49.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.834+0000 7fb8b1103640 1 -- 172.21.15.53:0/2100670771 wait complete. 2024-05-10T10:33:49.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.836+0000 7fb8b1103640 1 Processor -- start 2024-05-10T10:33:49.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.836+0000 7fb8b1103640 1 -- start start 2024-05-10T10:33:49.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8ac0d5190 0x7fb8ac161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8ac15f390 0x7fb8ac159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac0d0f90 con 0x7fb8ac0d5190 2024-05-10T10:33:49.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac0b55b0 con 0x7fb8ac15f390 2024-05-10T10:33:49.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb8ac0d1110 con 0x7fb8ac0b4c00 2024-05-10T10:33:49.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8ac0d5190 0x7fb8ac161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8ac0d5190 0x7fb8ac161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43284/0 (socket says 172.21.15.53:43284) 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 -- 172.21.15.53:0/3082681255 learned_addr learned my addr 172.21.15.53:0/3082681255 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8ab59e640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8ac15f390 0x7fb8ac159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aad9d640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 -- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 msgr2=0x7fb8ac0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 -- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8ac15f390 msgr2=0x7fb8ac159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8ac15f390 0x7fb8ac159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 -- 172.21.15.53:0/3082681255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac0c9700 con 0x7fb8ac0d5190 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8aa59c640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8ac0d5190 0x7fb8ac161410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb894001eb0 tx=0x7fb89400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8a37fe640 1 -- 172.21.15.53:0/3082681255 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894017070 con 0x7fb8ac0d5190 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8a37fe640 1 -- 172.21.15.53:0/3082681255 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894005ad0 con 0x7fb8ac0d5190 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac0c99e0 con 0x7fb8ac0d5190 2024-05-10T10:33:49.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8a37fe640 1 -- 172.21.15.53:0/3082681255 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894007070 con 0x7fb8ac0d5190 2024-05-10T10:33:49.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.837+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac0c9fa0 con 0x7fb8ac0d5190 2024-05-10T10:33:49.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.838+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ac075f90 con 0x7fb8ac0d5190 2024-05-10T10:33:49.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.839+0000 7fb8a37fe640 1 -- 172.21.15.53:0/3082681255 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb89401a070 con 0x7fb8ac0d5190 2024-05-10T10:33:49.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.840+0000 7fb8a37fe640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb88c0762d0 0x7fb88c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:49.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.840+0000 7fb8a37fe640 1 -- 172.21.15.53:0/3082681255 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb894096de0 con 0x7fb8ac0d5190 2024-05-10T10:33:49.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.840+0000 7fb8aad9d640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb88c0762d0 0x7fb88c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:49.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.840+0000 7fb8aad9d640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb88c0762d0 0x7fb88c078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009560 tx=0x7fb8a4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:49.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:49.844+0000 7fb8a37fe640 1 -- 172.21.15.53:0/3082681255 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb8ac075f90 con 0x7fb8ac0d5190 2024-05-10T10:33:50.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.034+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8ac0d2aa0 con 0x7fb8ac0d5190 2024-05-10T10:33:50.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.035+0000 7fb8a37fe640 1 -- 172.21.15.53:0/3082681255 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fb8ac0cfd10 con 0x7fb8ac0d5190 2024-05-10T10:33:50.037 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:50.038 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb88c0762d0 msgr2=0x7fb88c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb88c0762d0 0x7fb88c078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009560 tx=0x7fb8a4009190 comp rx=0 tx=0).stop 2024-05-10T10:33:50.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8ac0d5190 msgr2=0x7fb8ac161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8ac0d5190 0x7fb8ac161410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb894001eb0 tx=0x7fb89400cb90 comp rx=0 tx=0).stop 2024-05-10T10:33:50.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 shutdown_connections 2024-05-10T10:33:50.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb88c0762d0 0x7fb88c078790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8ac15f390 0x7fb8ac159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8ac0d5190 0x7fb8ac161410 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 --2- 172.21.15.53:0/3082681255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb8ac0b4c00 0x7fb8ac0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 >> 172.21.15.53:0/3082681255 conn(0x7fb8ac08ecb0 msgr2=0x7fb8ac0d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 shutdown_connections 2024-05-10T10:33:50.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.037+0000 7fb8b1103640 1 -- 172.21.15.53:0/3082681255 wait complete. 2024-05-10T10:33:50.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-10T10:33:50.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-10T10:33:50.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-10T10:33:50.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-10T10:33:50.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-10T10:33:50.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-10T10:33:50.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-10T10:33:50.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.230+0000 7fad30ce8640 1 Processor -- start 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.230+0000 7fad30ce8640 1 -- start start 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad30ce8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fad2c152f10 0x7fad2c0d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad30ce8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad2c1602a0 0x7fad2c0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad30ce8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad2c1532f0 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad30ce8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fad2c160710 con 0x7fad2c1602a0 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad30ce8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fad2c0cf040 con 0x7fad2c152f10 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad2ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad2c1602a0 0x7fad2c0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad2ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad2c1602a0 0x7fad2c0ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43292/0 (socket says 172.21.15.53:43292) 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad2ad9d640 1 -- 172.21.15.53:0/740249076 learned_addr learned my addr 172.21.15.53:0/740249076 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.232+0000 7fad2a59c640 1 --2- 172.21.15.53:0/740249076 >> v2:172.21.15.135:3300/0 conn(0x7fad2c152f10 0x7fad2c0d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.233+0000 7fad2ad9d640 1 -- 172.21.15.53:0/740249076 >> v2:172.21.15.135:3300/0 conn(0x7fad2c152f10 msgr2=0x7fad2c0d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.233+0000 7fad2ad9d640 1 --2- 172.21.15.53:0/740249076 >> v2:172.21.15.135:3300/0 conn(0x7fad2c152f10 0x7fad2c0d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.233+0000 7fad2ad9d640 1 -- 172.21.15.53:0/740249076 >> v1:172.21.15.135:6789/0 conn(0x7fad2c0ce6a0 legacy=0x7fad2c15fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.233+0000 7ff76d50a640 1 Processor -- start 2024-05-10T10:33:50.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.233+0000 7fad2ad9d640 1 -- 172.21.15.53:0/740249076 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c0cfc30 con 0x7fad2c1602a0 2024-05-10T10:33:50.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.233+0000 7fad2a59c640 1 --2- 172.21.15.53:0/740249076 >> v2:172.21.15.135:3300/0 conn(0x7fad2c152f10 0x7fad2c0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:50.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.234+0000 7fad2ad9d640 1 --2- 172.21.15.53:0/740249076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c1602a0 0x7fad2c0ceb00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad2c074450 tx=0x7fad2402c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa4c99be6cf74da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.233+0000 7ff76d50a640 1 -- start start 2024-05-10T10:33:50.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff76d50a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7680b00f0 0x7ff768072880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff76d50a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7680b4030 con 0x7ff768158af0 2024-05-10T10:33:50.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff76d50a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7680b4310 con 0x7ff768072440 2024-05-10T10:33:50.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7fad2959a640 1 -- 172.21.15.53:0/740249076 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24038070 con 0x7fad2c1602a0 2024-05-10T10:33:50.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7fad2959a640 1 -- 172.21.15.53:0/740249076 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad24009600 con 0x7fad2c1602a0 2024-05-10T10:33:50.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7fad2959a640 1 -- 172.21.15.53:0/740249076 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240099d0 con 0x7fad2c1602a0 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff76d50a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff7680b45f0 con 0x7ff7680b00f0 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff766ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff768158af0 0x7ff7680b50d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39724/0 (socket says 172.21.15.53:39724) 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff7667fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff768072440 0x7ff7680af9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36442/0 (socket says 172.21.15.53:36442) 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff7667fc640 1 -- 172.21.15.53:0/2433239214 learned_addr learned my addr 172.21.15.53:0/2433239214 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7ff7677fe640 1 --2- 172.21.15.53:0/2433239214 >> v2:172.21.15.135:3300/0 conn(0x7ff7680b00f0 0x7ff768072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7ff7677fe640 1 -- 172.21.15.53:0/2433239214 >> v1:172.21.15.135:6789/0 conn(0x7ff768072440 legacy=0x7ff7680af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7ff7677fe640 1 -- 172.21.15.53:0/2433239214 >> v1:172.21.15.53:6789/0 conn(0x7ff768158af0 legacy=0x7ff7680b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7ff7677fe640 1 -- 172.21.15.53:0/2433239214 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7680b48d0 con 0x7ff7680b00f0 2024-05-10T10:33:50.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7fad30ce8640 1 -- 172.21.15.53:0/740249076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c1602a0 msgr2=0x7fad2c0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7fad30ce8640 1 --2- 172.21.15.53:0/740249076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c1602a0 0x7fad2c0ceb00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad2c074450 tx=0x7fad2402c730 comp rx=0 tx=0).stop 2024-05-10T10:33:50.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7f8ab4b32640 1 Processor -- start 2024-05-10T10:33:50.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.235+0000 7f8ab4b32640 1 -- start start 2024-05-10T10:33:50.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7f8682cc0640 1 Processor -- start 2024-05-10T10:33:50.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8682cc0640 1 -- start start 2024-05-10T10:33:50.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7fad30ce8640 1 -- 172.21.15.53:0/740249076 shutdown_connections 2024-05-10T10:33:50.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7fad30ce8640 1 --2- 172.21.15.53:0/740249076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c1602a0 0x7fad2c0ceb00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7fad30ce8640 1 --2- 172.21.15.53:0/740249076 >> v2:172.21.15.135:3300/0 conn(0x7fad2c152f10 0x7fad2c0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7fad30ce8640 1 -- 172.21.15.53:0/740249076 >> 172.21.15.53:0/740249076 conn(0x7fad2c0b3870 msgr2=0x7fad2c0b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7fad30ce8640 1 -- 172.21.15.53:0/740249076 shutdown_connections 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7f8ab4b32640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8ab0163180 0x7f8ab0163560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7f8ab4b32640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0073c80 0x7f8ab0163aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7f8ab4b32640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8ab0071cb0 0x7f8ab0072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7f8ab4b32640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab006dd30 con 0x7f8ab0071cb0 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7f8ab4b32640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab00b51a0 con 0x7f8ab0163180 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.236+0000 7f8ab4b32640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab006df80 con 0x7f8ab0073c80 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0073c80 0x7f8ab0163aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0073c80 0x7f8ab0163aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36148/0 (socket says 172.21.15.53:36148) 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 -- 172.21.15.53:0/3436610124 learned_addr learned my addr 172.21.15.53:0/3436610124 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aae59c640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.135:3300/0 conn(0x7f8ab0163180 0x7f8ab0163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 -- 172.21.15.53:0/3436610124 >> v2:172.21.15.135:3300/0 conn(0x7f8ab0163180 msgr2=0x7f8ab0163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.135:3300/0 conn(0x7f8ab0163180 0x7f8ab0163560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 -- 172.21.15.53:0/3436610124 >> v2:172.21.15.53:3300/0 conn(0x7f8ab0071cb0 msgr2=0x7f8ab0072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7ff7677fe640 1 --2- 172.21.15.53:0/2433239214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7680b00f0 0x7ff768072880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0097a0 tx=0x7ff75c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aadfc55be6d5d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aaed9d640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.53:3300/0 conn(0x7f8ab0071cb0 0x7f8ab0072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.53:3300/0 conn(0x7f8ab0071cb0 0x7f8ab0072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7f8aadd9b640 1 -- 172.21.15.53:0/3436610124 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab006e260 con 0x7f8ab0073c80 2024-05-10T10:33:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7ff765ffb640 1 -- 172.21.15.53:0/2433239214 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c037070 con 0x7ff7680b00f0 2024-05-10T10:33:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7ff765ffb640 1 -- 172.21.15.53:0/2433239214 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75c032440 con 0x7ff7680b00f0 2024-05-10T10:33:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7ff765ffb640 1 -- 172.21.15.53:0/2433239214 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c032810 con 0x7ff7680b00f0 2024-05-10T10:33:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8aae59c640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.135:3300/0 conn(0x7f8ab0163180 0x7f8ab0163560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8aaed9d640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.53:3300/0 conn(0x7f8ab0071cb0 0x7f8ab0072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8682cc0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f867404fc50 0x7f86740500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8682cc0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8674006ea0 0x7f8674007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8682cc0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86740060e0 con 0x7f86740e0e40 2024-05-10T10:33:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8682cc0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8674005e90 con 0x7f8674006ea0 2024-05-10T10:33:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8682cc0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f86740063c0 con 0x7f867404fc50 2024-05-10T10:33:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f86824bf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8674006ea0 0x7f8674007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.237+0000 7fad30ce8640 1 -- 172.21.15.53:0/740249076 wait complete. 2024-05-10T10:33:50.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7ff76d50a640 1 -- 172.21.15.53:0/2433239214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7680b00f0 msgr2=0x7ff768072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7ff76d50a640 1 --2- 172.21.15.53:0/2433239214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7680b00f0 0x7ff768072880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0097a0 tx=0x7ff75c02a230 comp rx=0 tx=0).stop 2024-05-10T10:33:50.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f8aadd9b640 1 --2- 172.21.15.53:0/3436610124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0073c80 0x7f8ab0163aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009870 tx=0x7f8aa402a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71565d32f4a9c63e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f86824bf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8674006ea0 0x7f8674007330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43308/0 (socket says 172.21.15.53:43308) 2024-05-10T10:33:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.238+0000 7f86824bf640 1 -- 172.21.15.53:0/4282259849 learned_addr learned my addr 172.21.15.53:0/4282259849 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f4a1603e640 1 Processor -- start 2024-05-10T10:33:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f86824bf640 1 -- 172.21.15.53:0/4282259849 >> v2:172.21.15.135:3300/0 conn(0x7f867404fc50 msgr2=0x7f86740500c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:33:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7ff76d50a640 1 -- 172.21.15.53:0/2433239214 shutdown_connections 2024-05-10T10:33:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7ff76d50a640 1 --2- 172.21.15.53:0/2433239214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7680b00f0 0x7ff768072880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7ff76d50a640 1 -- 172.21.15.53:0/2433239214 >> 172.21.15.53:0/2433239214 conn(0x7ff7680b2610 msgr2=0x7ff7680b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f86824bf640 1 --2- 172.21.15.53:0/4282259849 >> v2:172.21.15.135:3300/0 conn(0x7f867404fc50 0x7f86740500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f86824bf640 1 -- 172.21.15.53:0/4282259849 >> v1:172.21.15.135:6789/0 conn(0x7f86740e0e40 legacy=0x7f8674048610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f86824bf640 1 -- 172.21.15.53:0/4282259849 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86740066a0 con 0x7f8674006ea0 2024-05-10T10:33:50.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f4a1603e640 1 -- start start 2024-05-10T10:33:50.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7fe4ef62a640 1 Processor -- start 2024-05-10T10:33:50.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f8aad59a640 1 -- 172.21.15.53:0/3436610124 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4037070 con 0x7f8ab0073c80 2024-05-10T10:33:50.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f8aad59a640 1 -- 172.21.15.53:0/3436610124 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa4032440 con 0x7f8ab0073c80 2024-05-10T10:33:50.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.239+0000 7f8aad59a640 1 -- 172.21.15.53:0/3436610124 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4032850 con 0x7f8ab0073c80 2024-05-10T10:33:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7fe4ef62a640 1 -- start start 2024-05-10T10:33:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f86824bf640 1 --2- 172.21.15.53:0/4282259849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674006ea0 0x7f8674007330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f867c050c60 tx=0x7f867c0a67c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d1cee2f385ab68 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7fa3ca810640 1 Processor -- start 2024-05-10T10:33:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7ff76d50a640 1 -- 172.21.15.53:0/2433239214 shutdown_connections 2024-05-10T10:33:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7ff76d50a640 1 -- 172.21.15.53:0/2433239214 wait complete. 2024-05-10T10:33:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7fa3ca810640 1 -- start start 2024-05-10T10:33:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8680cbc640 1 -- 172.21.15.53:0/4282259849 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0ae610 con 0x7f8674006ea0 2024-05-10T10:33:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8680cbc640 1 -- 172.21.15.53:0/4282259849 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867c0aa070 con 0x7f8674006ea0 2024-05-10T10:33:50.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8680cbc640 1 -- 172.21.15.53:0/4282259849 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0aeab0 con 0x7f8674006ea0 2024-05-10T10:33:50.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7fad30ce8640 1 Processor -- start 2024-05-10T10:33:50.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7fad30ce8640 1 -- start start 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f4a1603e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4a080069e0 0x7f4a08006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f4a1603e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a08053720 con 0x7f4a08005e90 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f4a1603e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a08053a00 con 0x7f4a080ead00 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f4a1603e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4a08054330 con 0x7f4a080069e0 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7f4a1503c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4a080ead00 0x7f4a0804fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36462/0 (socket says 172.21.15.53:36462) 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7f4a1503c640 1 -- 172.21.15.53:0/3033170825 learned_addr learned my addr 172.21.15.53:0/3033170825 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7f4a1583d640 1 --2- 172.21.15.53:0/3033170825 >> v2:172.21.15.53:3300/0 conn(0x7f4a080069e0 0x7f4a08006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2190826786 0 0) 0x7f4a08053a00 con 0x7f4a080ead00 2024-05-10T10:33:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7f8682cc0640 1 -- 172.21.15.53:0/4282259849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674006ea0 msgr2=0x7f8674007330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7f8682cc0640 1 --2- 172.21.15.53:0/4282259849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674006ea0 0x7f8674007330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f867c050c60 tx=0x7f867c0a67c0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49fc003520 con 0x7f4a080ead00 2024-05-10T10:33:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7fe4ef62a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4e80cfdb0 0x7fe4e80d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7fe4ef62a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e8071f10 0x7fe4e80723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7fe4ef62a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4e80ced10 con 0x7fe4e80c4210 2024-05-10T10:33:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7fe4ef62a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e80c45f0 con 0x7fe4e80cfdb0 2024-05-10T10:33:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7fe4ef62a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e80cf920 con 0x7fe4e8071f10 2024-05-10T10:33:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.241+0000 7fad30ce8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad2c0ce6a0 0x7fad2c1efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad30ce8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c152f10 0x7fad2c1f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ecbc5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4e80cfdb0 0x7fe4e80d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 <== mon.1 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3503481994 0 0) 0x7f49fc003520 con 0x7f4a080ead00 2024-05-10T10:33:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 >> v2:172.21.15.53:3300/0 conn(0x7f4a080069e0 msgr2=0x7f4a08006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 --2- 172.21.15.53:0/3033170825 >> v2:172.21.15.53:3300/0 conn(0x7f4a080069e0 0x7f4a08006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 >> v1:172.21.15.53:6789/0 conn(0x7f4a08005e90 legacy=0x7f4a080062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ecbc5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4e80cfdb0 0x7fe4e80d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43320/0 (socket says 172.21.15.53:43320) 2024-05-10T10:33:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ecbc5640 1 -- 172.21.15.53:0/166887063 learned_addr learned my addr 172.21.15.53:0/166887063 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a08053ce0 con 0x7f4a080ead00 2024-05-10T10:33:50.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 <== mon.1 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4a10050f90 con 0x7f4a080ead00 2024-05-10T10:33:50.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a1583d640 1 --2- 172.21.15.53:0/3033170825 >> v2:172.21.15.53:3300/0 conn(0x7f4a080069e0 0x7f4a08006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:50.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 >> v1:172.21.15.135:6789/0 conn(0x7f4a080ead00 legacy=0x7f4a0804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4edbc7640 1 --2- 172.21.15.53:0/166887063 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e8071f10 0x7fe4e80723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ed3c6640 1 --1- 172.21.15.53:0/166887063 >> v1:172.21.15.135:6789/0 conn(0x7fe4e80c4210 0x7fe4e806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36470/0 (socket says 172.21.15.53:36470) 2024-05-10T10:33:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3ca810640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3c415b180 0x7fa3c40cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08054330 con 0x7f49fc004800 2024-05-10T10:33:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc00bfc0 con 0x7f49fc0039b0 2024-05-10T10:33:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc00c1c0 con 0x7f49fc0051c0 2024-05-10T10:33:50.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f2fe1eb2640 1 Processor -- start 2024-05-10T10:33:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3ca810640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c40cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3c980e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3c415b180 0x7fa3c40cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3c980e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3c415b180 0x7fa3c40cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43334/0 (socket says 172.21.15.53:43334) 2024-05-10T10:33:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3ca810640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c4071c20 con 0x7fa3c40cb5b0 2024-05-10T10:33:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3c980e640 1 -- 172.21.15.53:0/3239794717 learned_addr learned my addr 172.21.15.53:0/3239794717 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a1503c640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f4a1483b640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 -- 172.21.15.53:0/3436610124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0073c80 msgr2=0x7f8ab0163aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 --2- 172.21.15.53:0/3436610124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0073c80 0x7f8ab0163aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009870 tx=0x7f8aa402a2e0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 -- 172.21.15.53:0/3436610124 shutdown_connections 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.53:3300/0 conn(0x7f8ab0071cb0 0x7f8ab0072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 --2- 172.21.15.53:0/3436610124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0073c80 0x7f8ab0163aa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 --2- 172.21.15.53:0/3436610124 >> v2:172.21.15.135:3300/0 conn(0x7f8ab0163180 0x7f8ab0163560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 -- 172.21.15.53:0/3436610124 >> 172.21.15.53:0/3436610124 conn(0x7f8ab00b2610 msgr2=0x7f8ab00b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 -- 172.21.15.53:0/3436610124 shutdown_connections 2024-05-10T10:33:50.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.240+0000 7f8ab4b32640 1 -- 172.21.15.53:0/3436610124 wait complete. 2024-05-10T10:33:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f8682cc0640 1 -- 172.21.15.53:0/4282259849 shutdown_connections 2024-05-10T10:33:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f8682cc0640 1 --2- 172.21.15.53:0/4282259849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674006ea0 0x7f8674007330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f8682cc0640 1 --2- 172.21.15.53:0/4282259849 >> v2:172.21.15.135:3300/0 conn(0x7f867404fc50 0x7f86740500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7f8682cc0640 1 -- 172.21.15.53:0/4282259849 >> 172.21.15.53:0/4282259849 conn(0x7f8674009fb0 msgr2=0x7f867400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad30ce8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad2c1602a0 0x7fad2c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad30ce8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c0d0560 con 0x7fad2c152f10 2024-05-10T10:33:50.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad30ce8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c160940 con 0x7fad2c1602a0 2024-05-10T10:33:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad30ce8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0d01f0 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad2a59c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad2c0ce6a0 0x7fad2c1efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad2a59c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad2c0ce6a0 0x7fad2c1efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36182/0 (socket says 172.21.15.53:36182) 2024-05-10T10:33:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fad2a59c640 1 -- 172.21.15.53:0/2762347703 learned_addr learned my addr 172.21.15.53:0/2762347703 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7ff76d50a640 1 Processor -- start 2024-05-10T10:33:50.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f8ab4b32640 1 Processor -- start 2024-05-10T10:33:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7ff76d50a640 1 -- start start 2024-05-10T10:33:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f2fe1eb2640 1 -- start start 2024-05-10T10:33:50.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f8ab4b32640 1 -- start start 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad29d9b640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c152f10 0x7fad2c1f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad2a59c640 1 -- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad2c1602a0 msgr2=0x7fad2c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad2a59c640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad2c1602a0 0x7fad2c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad2a59c640 1 -- 172.21.15.53:0/2762347703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c152f10 msgr2=0x7fad2c1f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad2a59c640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c152f10 0x7fad2c1f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad2a59c640 1 -- 172.21.15.53:0/2762347703 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c1f9a50 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad29d9b640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad2c152f10 0x7fad2c1f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:50.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f4a1483b640 1 -- 172.21.15.53:0/3033170825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0051c0 msgr2=0x7f49fc00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T10:33:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f4a1483b640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f4a1483b640 1 -- 172.21.15.53:0/3033170825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0039b0 msgr2=0x7f49fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f4a1483b640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f4a1483b640 1 -- 172.21.15.53:0/3033170825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a08053ce0 con 0x7f49fc004800 2024-05-10T10:33:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f4a1583d640 1 -- 172.21.15.53:0/3033170825 reap_dead start 2024-05-10T10:33:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7f8682cc0640 1 -- 172.21.15.53:0/4282259849 shutdown_connections 2024-05-10T10:33:50.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3ca810640 1 -- 172.21.15.53:0/3239794717 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c415f450 con 0x7fa3c415b180 2024-05-10T10:33:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fa3ca810640 1 -- 172.21.15.53:0/3239794717 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c415f730 con 0x7fa3c415ec00 2024-05-10T10:33:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fa3c900d640 1 --2- 172.21.15.53:0/3239794717 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c40cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fad2a59c640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad2c0ce6a0 0x7fad2c1efd40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c770 tx=0x7fad2000cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ecbc5640 1 -- 172.21.15.53:0/166887063 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e8071f10 msgr2=0x7fe4e80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ecbc5640 1 --2- 172.21.15.53:0/166887063 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e8071f10 0x7fe4e80723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ecbc5640 1 -- 172.21.15.53:0/166887063 >> v1:172.21.15.135:6789/0 conn(0x7fe4e80c4210 legacy=0x7fe4e806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.242+0000 7fe4ecbc5640 1 -- 172.21.15.53:0/166887063 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e80ceff0 con 0x7fe4e80cfdb0 2024-05-10T10:33:50.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fa3c900d640 1 -- 172.21.15.53:0/3239794717 >> v2:172.21.15.53:3300/0 conn(0x7fa3c415b180 msgr2=0x7fa3c40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fa3c900d640 1 --2- 172.21.15.53:0/3239794717 >> v2:172.21.15.53:3300/0 conn(0x7fa3c415b180 0x7fa3c40cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fa3c900d640 1 -- 172.21.15.53:0/3239794717 >> v1:172.21.15.135:6789/0 conn(0x7fa3c40cb5b0 legacy=0x7fa3c406dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fa3c900d640 1 -- 172.21.15.53:0/3239794717 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c415fa10 con 0x7fa3c415ec00 2024-05-10T10:33:50.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.243+0000 7fa3c980e640 1 --2- 172.21.15.53:0/3239794717 >> v2:172.21.15.53:3300/0 conn(0x7fa3c415b180 0x7fa3c40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:50.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7fe4ecbc5640 1 --2- 172.21.15.53:0/166887063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80cfdb0 0x7fe4e80d0210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009870 tx=0x7fe4d802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60418f715f853e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f4a1483b640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001730 tx=0x7f49fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5012eebdd5a595a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20015070 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f8ab4b32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ab0071cb0 0x7f8ab0157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7fe4dffff640 1 -- 172.21.15.53:0/166887063 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8037070 con 0x7fe4e80cfdb0 2024-05-10T10:33:50.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f8ab4b32640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ab0073c80 0x7f8ab01299f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2c1f9d30 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f8ab4b32640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0163180 0x7f8ab00ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f8ab4b32640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab00b0370 con 0x7f8ab0071cb0 2024-05-10T10:33:50.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f8682cc0640 1 -- 172.21.15.53:0/4282259849 wait complete. 2024-05-10T10:33:50.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7ff76d50a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff768072440 0x7ff768151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f8ab4b32640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab00b01f0 con 0x7f8ab0073c80 2024-05-10T10:33:50.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7fe4dffff640 1 -- 172.21.15.53:0/166887063 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d8032440 con 0x7fe4e80cfdb0 2024-05-10T10:33:50.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7fe4dffff640 1 -- 172.21.15.53:0/166887063 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8032850 con 0x7fe4e80cfdb0 2024-05-10T10:33:50.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f8ab4b32640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab006ea60 con 0x7f8ab0163180 2024-05-10T10:33:50.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2c0c5850 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aaed9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0163180 0x7f8ab00ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ab0071cb0 0x7f8ab0157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ab0071cb0 0x7f8ab0157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43372/0 (socket says 172.21.15.53:43372) 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 -- 172.21.15.53:0/2011052357 learned_addr learned my addr 172.21.15.53:0/2011052357 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f2fe1eb2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2fd4009e30 0x7f2fd4056130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fe1eb2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 0x7f2fd403a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fe1eb2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd4056670 con 0x7f2fd400a210 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fe0eb0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2fd4009e30 0x7f2fd4056130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fd3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2fd400a210 0x7f2fd4051d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39754/0 (socket says 172.21.15.53:39754) 2024-05-10T10:33:50.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fd3fff640 1 -- 172.21.15.53:0/1700346051 learned_addr learned my addr 172.21.15.53:0/1700346051 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:50.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00001350 con 0x7f49fc004800 2024-05-10T10:33:50.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a000052f0 con 0x7f49fc004800 2024-05-10T10:33:50.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.244+0000 7f4a07fff640 1 -- 172.21.15.53:0/3033170825 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0002aaf0 con 0x7f49fc004800 2024-05-10T10:33:50.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aadd9b640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ab0073c80 0x7f8ab01299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/1700346051 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd4055ca0 con 0x7f2fd4009e30 2024-05-10T10:33:50.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/1700346051 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40553b0 con 0x7f2fd403a240 2024-05-10T10:33:50.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7fa3c900d640 1 --2- 172.21.15.53:0/3239794717 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c40cb070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40097a0 tx=0x7fa3b402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830b1c129095e6cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fd37fe640 1 -- 172.21.15.53:0/1700346051 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3466647435 0 0) 0x7f2fd4056670 con 0x7f2fd400a210 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 -- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0163180 msgr2=0x7f8ab00ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0163180 0x7f8ab00ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 -- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ab0073c80 msgr2=0x7f8ab01299f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ab0073c80 0x7f8ab01299f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 -- 172.21.15.53:0/2011052357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab00ac540 con 0x7f8ab0071cb0 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7ff766ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff768072440 0x7ff768151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7ff766ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff768072440 0x7ff768151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43358/0 (socket says 172.21.15.53:43358) 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7ff766ffd640 1 -- 172.21.15.53:0/11260198 learned_addr learned my addr 172.21.15.53:0/11260198 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff76d50a640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7680b00f0 0x7ff768201310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff76d50a640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff768158af0 0x7ff76814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff76d50a640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff768072080 con 0x7ff768072440 2024-05-10T10:33:50.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff76d50a640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff768158ed0 con 0x7ff768158af0 2024-05-10T10:33:50.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff76d50a640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff768072200 con 0x7ff7680b00f0 2024-05-10T10:33:50.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aaed9d640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ab0163180 0x7f8ab00ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:50.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f8aae59c640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ab0071cb0 0x7f8ab0157e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800ce00 tx=0x7f8aa800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800f890 con 0x7f8ab0071cb0 2024-05-10T10:33:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f2fe16b1640 1 --2- 172.21.15.53:0/1700346051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 0x7f2fd403a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f2fd37fe640 1 -- 172.21.15.53:0/1700346051 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fc4007930 con 0x7f2fd400a210 2024-05-10T10:33:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad20007820 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20007bf0 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7fad18ff9640 1 -- 172.21.15.53:0/2762347703 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad10001030 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f4a1603e640 1 -- 172.21.15.53:0/3033170825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 msgr2=0x7f49fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.245+0000 7f4a1603e640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001730 tx=0x7f49fc005830 comp rx=0 tx=0).stop 2024-05-10T10:33:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f4a1603e640 1 -- 172.21.15.53:0/3033170825 shutdown_connections 2024-05-10T10:33:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f4a1603e640 1 --2- 172.21.15.53:0/3033170825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f4a1603e640 1 -- 172.21.15.53:0/3033170825 >> 172.21.15.53:0/3033170825 conn(0x7f4a08009fb0 msgr2=0x7f4a0800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7fe4ef62a640 1 -- 172.21.15.53:0/166887063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80cfdb0 msgr2=0x7fe4e80d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7fe4ef62a640 1 --2- 172.21.15.53:0/166887063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80cfdb0 0x7fe4e80d0210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009870 tx=0x7fe4d802a1f0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff766ffd640 1 -- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7680b00f0 msgr2=0x7ff768201310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:33:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f8682cc0640 1 Processor -- start 2024-05-10T10:33:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7fa3c880c640 1 -- 172.21.15.53:0/3239794717 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4037070 con 0x7fa3c415ec00 2024-05-10T10:33:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7fa3c880c640 1 -- 172.21.15.53:0/3239794717 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4032440 con 0x7fa3c415ec00 2024-05-10T10:33:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7fa3c880c640 1 -- 172.21.15.53:0/3239794717 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4032850 con 0x7fa3c415ec00 2024-05-10T10:33:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f2fe0eb0640 1 -- 172.21.15.53:0/1700346051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 msgr2=0x7f2fd403a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f2fe0eb0640 1 --2- 172.21.15.53:0/1700346051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 0x7f2fd403a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f2fe0eb0640 1 -- 172.21.15.53:0/1700346051 >> v1:172.21.15.53:6789/0 conn(0x7f2fd400a210 legacy=0x7f2fd4051d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f2fe0eb0640 1 -- 172.21.15.53:0/1700346051 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd4055650 con 0x7f2fd4009e30 2024-05-10T10:33:50.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff766ffd640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7680b00f0 0x7ff768201310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff766ffd640 1 -- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff768158af0 msgr2=0x7ff76814e070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:33:50.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7f8682cc0640 1 -- start start 2024-05-10T10:33:50.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7ff766ffd640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff768158af0 0x7ff76814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7ff766ffd640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76815b6a0 con 0x7ff768072440 2024-05-10T10:33:50.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7ff766ffd640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff768072440 0x7ff768151360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c3e0 tx=0x7ff75800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab00ac820 con 0x7f8ab0071cb0 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab00acde0 con 0x7f8ab0071cb0 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.246+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa800fa30 con 0x7f8ab0071cb0 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800fde0 con 0x7f8ab0071cb0 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fa3ca810640 1 -- 172.21.15.53:0/3239794717 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 msgr2=0x7fa3c40cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fa3ca810640 1 --2- 172.21.15.53:0/3239794717 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c40cb070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40097a0 tx=0x7fa3b402a1a0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fa3ca810640 1 -- 172.21.15.53:0/3239794717 shutdown_connections 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fa3ca810640 1 --2- 172.21.15.53:0/3239794717 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c40cb070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fa3ca810640 1 --2- 172.21.15.53:0/3239794717 >> v2:172.21.15.53:3300/0 conn(0x7fa3c415b180 0x7fa3c40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fa3ca810640 1 -- 172.21.15.53:0/3239794717 >> 172.21.15.53:0/3239794717 conn(0x7fa3c40b2530 msgr2=0x7fa3c40b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fe4ef62a640 1 -- 172.21.15.53:0/166887063 shutdown_connections 2024-05-10T10:33:50.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fe4ef62a640 1 --2- 172.21.15.53:0/166887063 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e8071f10 0x7fe4e80723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fe4ef62a640 1 --2- 172.21.15.53:0/166887063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80cfdb0 0x7fe4e80d0210 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fe4ef62a640 1 -- 172.21.15.53:0/166887063 >> 172.21.15.53:0/166887063 conn(0x7fe4e80d0c90 msgr2=0x7fe4e80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7f2fe0eb0640 1 --2- 172.21.15.53:0/1700346051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd4056130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000b60 tx=0x7f2fdc0a6380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8369a5cc8c63e4f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fad20018070 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7fa3ca810640 1 -- 172.21.15.53:0/3239794717 shutdown_connections 2024-05-10T10:33:50.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580074e0 con 0x7ff768072440 2024-05-10T10:33:50.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a94001030 con 0x7f8ab0071cb0 2024-05-10T10:33:50.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.247+0000 7f4a1603e640 1 -- 172.21.15.53:0/3033170825 shutdown_connections 2024-05-10T10:33:50.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f4a1603e640 1 -- 172.21.15.53:0/3033170825 wait complete. 2024-05-10T10:33:50.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff758007680 con 0x7ff768072440 2024-05-10T10:33:50.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7fe4ef62a640 1 -- 172.21.15.53:0/166887063 shutdown_connections 2024-05-10T10:33:50.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580049e0 con 0x7ff768072440 2024-05-10T10:33:50.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7fa3ca810640 1 -- 172.21.15.53:0/3239794717 wait complete. 2024-05-10T10:33:50.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7fad1affd640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fad0c0763a0 0x7fad0c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f8682cc0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8674006ea0 0x7f86740d5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f8682cc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f867404fc50 0x7f8674037bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fad20097e50 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f8682cc0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86740e0e40 0x7f8674038100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f8682cc0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8674050f50 con 0x7f867404fc50 2024-05-10T10:33:50.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f8682cc0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8674048d20 con 0x7f86740e0e40 2024-05-10T10:33:50.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7fad1affd640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7fad0c07fb20 0x7fad0c081f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad0c082630 con 0x7fad0c07fb20 2024-05-10T10:33:50.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fad20098240 con 0x7fad2c0ce6a0 2024-05-10T10:33:50.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f8682cc0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f86740510d0 con 0x7f8674006ea0 2024-05-10T10:33:50.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7fad2ad9d640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7fad0c07fb20 0x7fad0c081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f86824bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86740e0e40 0x7f8674038100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f86824bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86740e0e40 0x7f8674038100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51068/0 (socket says 172.21.15.53:51068) 2024-05-10T10:33:50.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7f86824bf640 1 -- 172.21.15.53:0/3564142899 learned_addr learned my addr 172.21.15.53:0/3564142899 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7ff76d50a640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff768150880 con 0x7ff768072440 2024-05-10T10:33:50.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f86824bf640 1 -- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8674006ea0 msgr2=0x7f86740d5280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T10:33:50.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f86824bf640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8674006ea0 0x7f86740d5280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f86824bf640 1 -- 172.21.15.53:0/3564142899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f867404fc50 msgr2=0x7f8674037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f86824bf640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f867404fc50 0x7f8674037bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f86824bf640 1 -- 172.21.15.53:0/3564142899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86740d5d90 con 0x7f86740e0e40 2024-05-10T10:33:50.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f86824bf640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86740e0e40 0x7f8674038100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f867c074cd0 tx=0x7f867c076270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7ff76d50a640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76814e5b0 con 0x7ff768072440 2024-05-10T10:33:50.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0c8680 con 0x7f86740e0e40 2024-05-10T10:33:50.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7fad2ad9d640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7fad0c07fb20 0x7fad0c081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7fa3ca810640 1 Processor -- start 2024-05-10T10:33:50.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.248+0000 7fe4ef62a640 1 -- 172.21.15.53:0/166887063 wait complete. 2024-05-10T10:33:50.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7fe4ef62a640 1 Processor -- start 2024-05-10T10:33:50.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7fa3ca810640 1 -- start start 2024-05-10T10:33:50.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7fad29d9b640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fad0c0763a0 0x7fad0c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f8682cc0640 1 -- 172.21.15.53:0/3564142899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86740d6070 con 0x7f86740e0e40 2024-05-10T10:33:50.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f8682cc0640 1 -- 172.21.15.53:0/3564142899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86740d57c0 con 0x7f86740e0e40 2024-05-10T10:33:50.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867c0bde50 con 0x7f86740e0e40 2024-05-10T10:33:50.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0c8a20 con 0x7f86740e0e40 2024-05-10T10:33:50.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff768201850 con 0x7ff768072440 2024-05-10T10:33:50.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad0c082630 con 0x7fad0c07fb20 2024-05-10T10:33:50.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8658001030 con 0x7f86740e0e40 2024-05-10T10:33:50.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fad29d9b640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fad0c0763a0 0x7fad0c078860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad1c007fd0 tx=0x7fad1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fa3ca810640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c40cb5b0 0x7fa3c414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fa3ca810640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415b180 0x7fa3c4150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f4a1603e640 1 Processor -- start 2024-05-10T10:33:50.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.249+0000 7f4a1603e640 1 -- start start 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7fe4ef62a640 1 -- start start 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ef62a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4e8071f10 0x7fe4e815f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ef62a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80c4210 0x7fe4e80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ef62a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e80cfdb0 0x7fe4e8165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ef62a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e806e620 con 0x7fe4e80c4210 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ef62a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80cf5b0 con 0x7fe4e8071f10 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f4a1603e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f4a0804dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f4a1603e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0802e060 0x7f4a080e2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ef62a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e80cc0b0 con 0x7fe4e80cfdb0 2024-05-10T10:33:50.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f4a1603e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0802e4c0 0x7f4a080e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f4a1603e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08054940 con 0x7f49fc004800 2024-05-10T10:33:50.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f4a1603e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080547c0 con 0x7f4a0802e4c0 2024-05-10T10:33:50.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f4a1603e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0804f550 con 0x7f4a0802e060 2024-05-10T10:33:50.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.250+0000 7f4a1503c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f4a0804dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1503c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f4a0804dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43394/0 (socket says 172.21.15.53:43394) 2024-05-10T10:33:50.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1503c640 1 -- 172.21.15.53:0/493671655 learned_addr learned my addr 172.21.15.53:0/493671655 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1583d640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0802e4c0 0x7f4a080e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1483b640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0802e060 0x7f4a080e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ed3c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4e8071f10 0x7fe4e815f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fa3c980e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c40cb5b0 0x7fa3c414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fa3c980e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c40cb5b0 0x7fa3c414faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43404/0 (socket says 172.21.15.53:43404) 2024-05-10T10:33:50.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1583d640 1 -- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0802e060 msgr2=0x7f4a080e2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1583d640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0802e060 0x7f4a080e2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1583d640 1 -- 172.21.15.53:0/493671655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 msgr2=0x7f4a0804dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1583d640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f4a0804dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7f4a1583d640 1 -- 172.21.15.53:0/493671655 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0802ea70 con 0x7f4a0802e4c0 2024-05-10T10:33:50.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3ca810640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7f2fd37fe640 1 -- 172.21.15.53:0/1700346051 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0b3070 con 0x7f2fd4009e30 2024-05-10T10:33:50.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7f4a1583d640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0802e4c0 0x7f4a080e28b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800b3a0 tx=0x7f4a0c00e7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7f2fd37fe640 1 -- 172.21.15.53:0/1700346051 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc0ae640 con 0x7f2fd4009e30 2024-05-10T10:33:50.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7f2fd37fe640 1 -- 172.21.15.53:0/1700346051 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0aead0 con 0x7f2fd4009e30 2024-05-10T10:33:50.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3c900d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415b180 0x7fa3c4150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3c900d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415b180 0x7fa3c4150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51098/0 (socket says 172.21.15.53:51098) 2024-05-10T10:33:50.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00f040 con 0x7f4a0802e4c0 2024-05-10T10:33:50.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3ca810640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c415f1e0 con 0x7fa3c40cb5b0 2024-05-10T10:33:50.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3c980e640 1 -- 172.21.15.53:0/1396877260 learned_addr learned my addr 172.21.15.53:0/1396877260 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f867c0ac030 con 0x7f86740e0e40 2024-05-10T10:33:50.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3ca810640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c415f060 con 0x7fa3c415b180 2024-05-10T10:33:50.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3ca00f640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fa3ca810640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c406fd10 con 0x7fa3c415ec00 2024-05-10T10:33:50.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0c004230 con 0x7f4a0802e4c0 2024-05-10T10:33:50.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/1700346051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 msgr2=0x7f2fd4056130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 --2- 172.21.15.53:0/1700346051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd4056130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000b60 tx=0x7f2fdc0a6380 comp rx=0 tx=0).stop 2024-05-10T10:33:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff75801a020 con 0x7ff768072440 2024-05-10T10:33:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fa3c900d640 1 -- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 msgr2=0x7fa3c415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fa3c900d640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c415ec00 0x7fa3c415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fa3c900d640 1 -- 172.21.15.53:0/1396877260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c40cb5b0 msgr2=0x7fa3c414faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fa3c900d640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c40cb5b0 0x7fa3c414faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fa3c900d640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c415cd30 con 0x7fa3c415b180 2024-05-10T10:33:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/1700346051 shutdown_connections 2024-05-10T10:33:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 --2- 172.21.15.53:0/1700346051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 0x7f2fd403a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 --2- 172.21.15.53:0/1700346051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd4056130 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/1700346051 >> 172.21.15.53:0/1700346051 conn(0x7f2fd40537b0 msgr2=0x7f2fd4053b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c003610 con 0x7f4a0802e4c0 2024-05-10T10:33:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f86727fc640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f86600763a0 0x7f8660078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f8681cbe640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f86600763a0 0x7f8660078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f867c13d1f0 con 0x7f86740e0e40 2024-05-10T10:33:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fa3c900d640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415b180 0x7fa3c4150a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4031000 tx=0x7fa3b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/1700346051 shutdown_connections 2024-05-10T10:33:50.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/1700346051 wait complete. 2024-05-10T10:33:50.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0802ed50 con 0x7f4a0802e4c0 2024-05-10T10:33:50.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8aa80213f0 con 0x7f8ab0071cb0 2024-05-10T10:33:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4037050 con 0x7fa3c415b180 2024-05-10T10:33:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4043070 con 0x7fa3c415b180 2024-05-10T10:33:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0802b7f0 con 0x7f4a0802e4c0 2024-05-10T10:33:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7fa3ca810640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c415d010 con 0x7fa3c415b180 2024-05-10T10:33:50.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7fa3ca810640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c415c5c0 con 0x7fa3c415b180 2024-05-10T10:33:50.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ed3c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4e8071f10 0x7fe4e815f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51096/0 (socket says 172.21.15.53:51096) 2024-05-10T10:33:50.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ed3c6640 1 -- 172.21.15.53:0/3692459645 learned_addr learned my addr 172.21.15.53:0/3692459645 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7ff74f7fe640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7540763d0 0x7ff754078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.251+0000 7fe4ecbc5640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80c4210 0x7fe4e80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ed3c6640 1 -- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e80cfdb0 msgr2=0x7fe4e8165aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T10:33:50.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ed3c6640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe4e80cfdb0 0x7fe4e8165aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ed3c6640 1 -- 172.21.15.53:0/3692459645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80c4210 msgr2=0x7fe4e80b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ed3c6640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4e80c4210 0x7fe4e80b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff7580976f0 con 0x7ff768072440 2024-05-10T10:33:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7ff7667fc640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7540763d0 0x7ff754078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f86727fc640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f866007fb20 0x7f8660081f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f867c0aaec0 con 0x7f866007fb20 2024-05-10T10:33:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f867c13d5e0 con 0x7f86740e0e40 2024-05-10T10:33:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7f86814bd640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f866007fb20 0x7f8660081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4030e50 con 0x7fa3c415b180 2024-05-10T10:33:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ed3c6640 1 -- 172.21.15.53:0/3692459645 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e8164e50 con 0x7fe4e8071f10 2024-05-10T10:33:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.255+0000 7f8681cbe640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f86600763a0 0x7f8660078860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f866c000f60 tx=0x7f866c0079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.255+0000 7f86814bd640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f866007fb20 0x7f8660081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.255+0000 7f8a9effd640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8a900763a0 0x7f8a90078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ed3c6640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4e8071f10 0x7fe4e815f110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c4d0 tx=0x7fe4e000c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0018070 con 0x7fe4e8071f10 2024-05-10T10:33:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.255+0000 7f2fe1eb2640 1 Processor -- start 2024-05-10T10:33:50.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.252+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e8164890 con 0x7fe4e8071f10 2024-05-10T10:33:50.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e80b3930 con 0x7fe4e8071f10 2024-05-10T10:33:50.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e0007480 con 0x7fe4e8071f10 2024-05-10T10:33:50.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.253+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00047e0 con 0x7fe4e8071f10 2024-05-10T10:33:50.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.254+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4c8001030 con 0x7fe4e8071f10 2024-05-10T10:33:50.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe4e001b070 con 0x7fe4e8071f10 2024-05-10T10:33:50.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3c40765a0 con 0x7fa3c415b180 2024-05-10T10:33:50.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a08005390 con 0x7f4a0802e4c0 2024-05-10T10:33:50.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7f2fe1eb2640 1 -- start start 2024-05-10T10:33:50.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f867c0aaec0 con 0x7f866007fb20 2024-05-10T10:33:50.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7ff7667fc640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7540763d0 0x7ff754078890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff760004b40 tx=0x7ff760004fe0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7ff74f7fe640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7ff75407fb50 0x7ff754081f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff754082660 con 0x7ff75407fb50 2024-05-10T10:33:50.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff758097ae0 con 0x7ff768072440 2024-05-10T10:33:50.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.256+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a0c0037b0 con 0x7f4a0802e4c0 2024-05-10T10:33:50.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7ff7677fe640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7ff75407fb50 0x7ff754081f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8aa8098a70 con 0x7f8ab0071cb0 2024-05-10T10:33:50.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa3b4048460 con 0x7fa3c415b180 2024-05-10T10:33:50.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7f8a9effd640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f8a9007fb20 0x7f8a90081f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8aa800fa30 con 0x7f8a9007fb20 2024-05-10T10:33:50.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8aa8098e60 con 0x7f8ab0071cb0 2024-05-10T10:33:50.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7f8aadd9b640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8a900763a0 0x7f8a90078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.257+0000 7ff7677fe640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7ff75407fb50 0x7ff754081f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f4a05ffb640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40763a0 0x7f49f4078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe1eb2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd40498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f8aaed9d640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f8a9007fb20 0x7f8a90081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe1eb2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd400a210 0x7f2fd40de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe1eb2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 0x7f2fd404dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7fa3ba7fc640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3a8076570 0x7fa3a8078a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f4a1503c640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40763a0 0x7f49f4078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe1eb2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd400a670 con 0x7f2fd4009e30 2024-05-10T10:33:50.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe1eb2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd40527a0 con 0x7f2fd400a210 2024-05-10T10:33:50.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe1eb2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd4052410 con 0x7f2fd403a240 2024-05-10T10:33:50.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe0eb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd40498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4a0c096fb0 con 0x7f4a0802e4c0 2024-05-10T10:33:50.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe0eb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd40498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43422/0 (socket says 172.21.15.53:43422) 2024-05-10T10:33:50.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fe0eb0640 1 -- 172.21.15.53:0/2427586149 learned_addr learned my addr 172.21.15.53:0/2427586149 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f4a05ffb640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f49f407fb20 0x7f49f4081f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f8aaed9d640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f8a9007fb20 0x7f8a90081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f2fd3fff640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd400a210 0x7f2fd40de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa3b40b87b0 con 0x7fa3c415b180 2024-05-10T10:33:50.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.258+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49f4082630 con 0x7f49f407fb20 2024-05-10T10:33:50.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f4a1483b640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f49f407fb20 0x7f49f4081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fe0eb0640 1 -- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 msgr2=0x7f2fd404dd70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T10:33:50.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7fa3ba7fc640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7fa3a807fcf0 0x7fa3a8082130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fe16b1640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 0x7f2fd404dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3b404be80 con 0x7fa3a807fcf0 2024-05-10T10:33:50.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fe0eb0640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2fd403a240 0x7f2fd404dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3b40bf050 con 0x7fa3c415b180 2024-05-10T10:33:50.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fe0eb0640 1 -- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd400a210 msgr2=0x7f2fd40de5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c099610 con 0x7f4a0802e4c0 2024-05-10T10:33:50.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fe0eb0640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd400a210 0x7f2fd40de5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fe0eb0640 1 -- 172.21.15.53:0/2427586149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd404b820 con 0x7f2fd4009e30 2024-05-10T10:33:50.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7fa3c980e640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3a8076570 0x7fa3a8078a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7fe4ddffb640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe4cc0763d0 0x7fe4cc078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff754082660 con 0x7ff75407fb50 2024-05-10T10:33:50.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fd3fff640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd400a210 0x7f2fd40de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:50.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7fe4ecbc5640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe4cc0763d0 0x7fe4cc078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f2fe0eb0640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd40498f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc069aa0 tx=0x7f2fdc076270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f4a1483b640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f49f407fb20 0x7f49f4081f60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f4a1503c640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40763a0 0x7f49f4078860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a10063f70 tx=0x7f4a100756d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8aa800fa30 con 0x7f8a9007fb20 2024-05-10T10:33:50.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.259+0000 7fa3ca00f640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7fa3a807fcf0 0x7fa3a8082130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe4e0096b60 con 0x7fe4e8071f10 2024-05-10T10:33:50.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7fe4ecbc5640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe4cc0763d0 0x7fe4cc078890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8034000 tx=0x7fe4d802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0bb070 con 0x7f2fd4009e30 2024-05-10T10:33:50.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/2427586149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd40dbf60 con 0x7f2fd4009e30 2024-05-10T10:33:50.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/2427586149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd4050d60 con 0x7f2fd4009e30 2024-05-10T10:33:50.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc0ae640 con 0x7f2fd4009e30 2024-05-10T10:33:50.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0bb070 con 0x7f2fd4009e30 2024-05-10T10:33:50.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7fa3c980e640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3a8076570 0x7fa3a8078a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ac20 tx=0x7fa3c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7fe4ddffb640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7fe4cc07fb50 0x7fe4cc081f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4e0011ec0 con 0x7fe4cc07fb50 2024-05-10T10:33:50.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe4e009b2a0 con 0x7fe4e8071f10 2024-05-10T10:33:50.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7fe4edbc7640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7fe4cc07fb50 0x7fe4cc081f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.261+0000 7fa3ca00f640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7fa3a807fcf0 0x7fa3a8082130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.260+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49f4082630 con 0x7f49f407fb20 2024-05-10T10:33:50.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.261+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/2427586149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fbc001030 con 0x7f2fd4009e30 2024-05-10T10:33:50.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.261+0000 7f8aadd9b640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8a900763a0 0x7f8a90078860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4034000 tx=0x7f8aa402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.261+0000 7fe4edbc7640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7fe4cc07fb50 0x7fe4cc081f90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.261+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3b404be80 con 0x7fa3a807fcf0 2024-05-10T10:33:50.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.262+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e0011ec0 con 0x7fe4cc07fb50 2024-05-10T10:33:50.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.263+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2fdc0be070 con 0x7f2fd4009e30 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.264+0000 7f2fd17fa640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2fc80766d0 0x7f2fc8078b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.264+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2fdc0ae440 con 0x7f2fd4009e30 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.264+0000 7f2fd17fa640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f2fc80809a0 0x7f2fc8082e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.264+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fc8083510 con 0x7f2fc80809a0 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.264+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2fdc0f9150 con 0x7f2fd4009e30 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.265+0000 7f2fd3fff640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2fc80766d0 0x7f2fc8078b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.265+0000 7f2fe16b1640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f2fc80809a0 0x7f2fc8082e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.266+0000 7f2fe16b1640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f2fc80809a0 0x7f2fc8082e40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.267+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fc8083510 con 0x7f2fc80809a0 2024-05-10T10:33:50.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.276+0000 7f2fd3fff640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2fc80766d0 0x7f2fc8078b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4178140 tx=0x7f2fcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.287+0000 7fad18ff9640 1 -- 172.21.15.53:0/2762347703 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad10002dd0 con 0x7fad0c07fb20 2024-05-10T10:33:50.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.288+0000 7fad1affd640 1 -- 172.21.15.53:0/2762347703 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fad10002dd0 con 0x7fad0c07fb20 2024-05-10T10:33:50.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8658002dd0 con 0x7f866007fb20 2024-05-10T10:33:50.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7fad0c07fb20 msgr2=0x7fad0c081f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7fad30ce8640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7fad0c07fb20 0x7fad0c081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7f86727fc640 1 -- 172.21.15.53:0/3564142899 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8658002dd0 con 0x7f866007fb20 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fad0c0763a0 msgr2=0x7fad0c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7fad30ce8640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fad0c0763a0 0x7fad0c078860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad1c007fd0 tx=0x7fad1c009040 comp rx=0 tx=0).stop 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad2c0ce6a0 msgr2=0x7fad2c1efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7fad30ce8640 1 --2- 172.21.15.53:0/2762347703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad2c0ce6a0 0x7fad2c1efd40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c770 tx=0x7fad2000cb70 comp rx=0 tx=0).stop 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.289+0000 7fad2ad9d640 1 -- 172.21.15.53:0/2762347703 reap_dead start 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 shutdown_connections 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 >> 172.21.15.53:0/2762347703 conn(0x7fad2c0b3870 msgr2=0x7fad2c06f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 shutdown_connections 2024-05-10T10:33:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7fad30ce8640 1 -- 172.21.15.53:0/2762347703 wait complete. 2024-05-10T10:33:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f866007fb20 msgr2=0x7f8660081f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7f865ffff640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f866007fb20 0x7f8660081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f86600763a0 msgr2=0x7f8660078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7f865ffff640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f86600763a0 0x7f8660078860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f866c000f60 tx=0x7f866c0079b0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86740e0e40 msgr2=0x7f8674038100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.290+0000 7f865ffff640 1 --2- 172.21.15.53:0/3564142899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86740e0e40 0x7f8674038100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f867c074cd0 tx=0x7f867c076270 comp rx=0 tx=0).stop 2024-05-10T10:33:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.291+0000 7f86824bf640 1 -- 172.21.15.53:0/3564142899 reap_dead start 2024-05-10T10:33:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.291+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 shutdown_connections 2024-05-10T10:33:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.291+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 >> 172.21.15.53:0/3564142899 conn(0x7f8674009fb0 msgr2=0x7f867400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.291+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff768201850 con 0x7ff75407fb50 2024-05-10T10:33:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.291+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 shutdown_connections 2024-05-10T10:33:50.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.291+0000 7f865ffff640 1 -- 172.21.15.53:0/3564142899 wait complete. 2024-05-10T10:33:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7ff74f7fe640 1 -- 172.21.15.53:0/11260198 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff75c038040 con 0x7ff75407fb50 2024-05-10T10:33:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.291+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a94002dd0 con 0x7f8a9007fb20 2024-05-10T10:33:50.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7ff75407fb50 msgr2=0x7ff754081f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7f8a9effd640 1 -- 172.21.15.53:0/2011052357 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8a94002dd0 con 0x7f8a9007fb20 2024-05-10T10:33:50.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7ff74d7fa640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7ff75407fb50 0x7ff754081f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f8a9007fb20 msgr2=0x7f8a90081f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7540763d0 msgr2=0x7ff754078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7ff74d7fa640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7540763d0 0x7ff754078890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff760004b40 tx=0x7ff760004fe0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff768072440 msgr2=0x7ff768151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.292+0000 7f8ab4b32640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f8a9007fb20 0x7f8a90081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8a900763a0 msgr2=0x7f8a90078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8ab4b32640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8a900763a0 0x7f8a90078860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4034000 tx=0x7f8aa402e000 comp rx=0 tx=0).stop 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ab0071cb0 msgr2=0x7f8ab0157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8ab4b32640 1 --2- 172.21.15.53:0/2011052357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ab0071cb0 0x7f8ab0157e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800ce00 tx=0x7f8aa800a650 comp rx=0 tx=0).stop 2024-05-10T10:33:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7ff74d7fa640 1 --2- 172.21.15.53:0/11260198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff768072440 0x7ff768151360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c3e0 tx=0x7ff75800c8a0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7ff7677fe640 1 -- 172.21.15.53:0/11260198 reap_dead start 2024-05-10T10:33:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8aaed9d640 1 -- 172.21.15.53:0/2011052357 reap_dead start 2024-05-10T10:33:50.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 shutdown_connections 2024-05-10T10:33:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 >> 172.21.15.53:0/11260198 conn(0x7ff7680b2610 msgr2=0x7ff76815cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 shutdown_connections 2024-05-10T10:33:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 >> 172.21.15.53:0/2011052357 conn(0x7f8ab00b2610 msgr2=0x7f8ab00b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 shutdown_connections 2024-05-10T10:33:50.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7ff74d7fa640 1 -- 172.21.15.53:0/11260198 wait complete. 2024-05-10T10:33:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.293+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 shutdown_connections 2024-05-10T10:33:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.294+0000 7f8ab4b32640 1 -- 172.21.15.53:0/2011052357 wait complete. 2024-05-10T10:33:50.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.295+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4c8002dd0 con 0x7fe4cc07fb50 2024-05-10T10:33:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.295+0000 7fe4ddffb640 1 -- 172.21.15.53:0/3692459645 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe4c8002dd0 con 0x7fe4cc07fb50 2024-05-10T10:33:50.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7fe4cc07fb50 msgr2=0x7fe4cc081f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7fe4cc07fb50 0x7fe4cc081f90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe4cc0763d0 msgr2=0x7fe4cc078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe4cc0763d0 0x7fe4cc078890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8034000 tx=0x7fe4d802e000 comp rx=0 tx=0).stop 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4e8071f10 msgr2=0x7fe4e815f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 --2- 172.21.15.53:0/3692459645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4e8071f10 0x7fe4e815f110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000c4d0 tx=0x7fe4e000c990 comp rx=0 tx=0).stop 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4edbc7640 1 -- 172.21.15.53:0/3692459645 reap_dead start 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 shutdown_connections 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 >> 172.21.15.53:0/3692459645 conn(0x7fe4e80d0c90 msgr2=0x7fe4e80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.297+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a08048620 con 0x7f49f407fb20 2024-05-10T10:33:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.296+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3c4152540 con 0x7fa3a807fcf0 2024-05-10T10:33:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.297+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 shutdown_connections 2024-05-10T10:33:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.297+0000 7fe4ef62a640 1 -- 172.21.15.53:0/3692459645 wait complete. 2024-05-10T10:33:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.298+0000 7f4a05ffb640 1 -- 172.21.15.53:0/493671655 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a08048620 con 0x7f49f407fb20 2024-05-10T10:33:50.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.298+0000 7fa3ba7fc640 1 -- 172.21.15.53:0/1396877260 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3c4152540 con 0x7fa3a807fcf0 2024-05-10T10:33:50.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.298+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7fa3a807fcf0 msgr2=0x7fa3a8082130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.298+0000 7fa3a7fff640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7fa3a807fcf0 0x7fa3a8082130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.298+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f49f407fb20 msgr2=0x7f49f4081f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.298+0000 7f4a1603e640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f49f407fb20 0x7f49f4081f60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3a8076570 msgr2=0x7fa3a8078a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7fa3a7fff640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3a8076570 0x7fa3a8078a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ac20 tx=0x7fa3c0008040 comp rx=0 tx=0).stop 2024-05-10T10:33:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415b180 msgr2=0x7fa3c4150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7fa3a7fff640 1 --2- 172.21.15.53:0/1396877260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415b180 0x7fa3c4150a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4031000 tx=0x7fa3b40046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7fa3ca00f640 1 -- 172.21.15.53:0/1396877260 reap_dead start 2024-05-10T10:33:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40763a0 msgr2=0x7f49f4078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7f4a1603e640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40763a0 0x7f49f4078860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a10063f70 tx=0x7f4a100756d0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0802e4c0 msgr2=0x7f4a080e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7f4a1603e640 1 --2- 172.21.15.53:0/493671655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0802e4c0 0x7f4a080e28b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800b3a0 tx=0x7f4a0c00e7e0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7f4a1583d640 1 -- 172.21.15.53:0/493671655 reap_dead start 2024-05-10T10:33:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 shutdown_connections 2024-05-10T10:33:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 >> 172.21.15.53:0/493671655 conn(0x7f4a08009fb0 msgr2=0x7f4a08049870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 shutdown_connections 2024-05-10T10:33:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.299+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 >> 172.21.15.53:0/1396877260 conn(0x7fa3c40b2530 msgr2=0x7fa3c40b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.300+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 shutdown_connections 2024-05-10T10:33:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.300+0000 7f4a1603e640 1 -- 172.21.15.53:0/493671655 wait complete. 2024-05-10T10:33:50.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.300+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 shutdown_connections 2024-05-10T10:33:50.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.300+0000 7fa3a7fff640 1 -- 172.21.15.53:0/1396877260 wait complete. 2024-05-10T10:33:50.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.300+0000 7f2fe1eb2640 1 -- 172.21.15.53:0/2427586149 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fbc002dd0 con 0x7f2fc80809a0 2024-05-10T10:33:50.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.301+0000 7f2fd17fa640 1 -- 172.21.15.53:0/2427586149 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2fbc002dd0 con 0x7f2fc80809a0 2024-05-10T10:33:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.305+0000 7f2fc2ffd640 1 -- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f2fc80809a0 msgr2=0x7f2fc8082e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.305+0000 7f2fc2ffd640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f2fc80809a0 0x7f2fc8082e40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.306+0000 7f2fc2ffd640 1 -- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2fc80766d0 msgr2=0x7f2fc8078b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.306+0000 7f2fc2ffd640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2fc80766d0 0x7f2fc8078b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4178140 tx=0x7f2fcc009040 comp rx=0 tx=0).stop 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.306+0000 7f2fc2ffd640 1 -- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 msgr2=0x7f2fd40498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.306+0000 7f2fc2ffd640 1 --2- 172.21.15.53:0/2427586149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd4009e30 0x7f2fd40498f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc069aa0 tx=0x7f2fdc076270 comp rx=0 tx=0).stop 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.306+0000 7f2fe16b1640 1 -- 172.21.15.53:0/2427586149 reap_dead start 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.308+0000 7f2fc2ffd640 1 -- 172.21.15.53:0/2427586149 shutdown_connections 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.308+0000 7f2fc2ffd640 1 -- 172.21.15.53:0/2427586149 >> 172.21.15.53:0/2427586149 conn(0x7f2fd40537b0 msgr2=0x7f2fd4054870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.308+0000 7f2fc2ffd640 1 -- 172.21.15.53:0/2427586149 shutdown_connections 2024-05-10T10:33:50.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.308+0000 7f2fc2ffd640 1 -- 172.21.15.53:0/2427586149 wait complete. 2024-05-10T10:33:50.361 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:50.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-10T10:33:50.362 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:50.362 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-10T10:33:50.363 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:50.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-10T10:33:50.364 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:50.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-10T10:33:50.365 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:50.366 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-10T10:33:50.366 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:50.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-10T10:33:50.367 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:50.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-10T10:33:50.368 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:50.369 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-10T10:33:50.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.581+0000 7f3b04a44640 1 Processor -- start 2024-05-10T10:33:50.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.581+0000 7f3b04a44640 1 -- start start 2024-05-10T10:33:50.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f3b04a44640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b0006c2c0 0x7f3b0006c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f3b04a44640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b00167950 con 0x7f3b00167390 2024-05-10T10:33:50.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f3b04a44640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b00167c30 con 0x7f3b000c4f20 2024-05-10T10:33:50.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f3b04a44640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3b000cfe40 con 0x7f3b0006c2c0 2024-05-10T10:33:50.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f6508aa8640 1 Processor -- start 2024-05-10T10:33:50.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f3afe59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3b00167390 0x7f3b0006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39766/0 (socket says 172.21.15.53:39766) 2024-05-10T10:33:50.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f3afe59c640 1 -- 172.21.15.53:0/2073081207 learned_addr learned my addr 172.21.15.53:0/2073081207 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:50.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f6508aa8640 1 -- start start 2024-05-10T10:33:50.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f94d0ac0640 1 Processor -- start 2024-05-10T10:33:50.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f7b949c1640 1 Processor -- start 2024-05-10T10:33:50.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f7b949c1640 1 -- start start 2024-05-10T10:33:50.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f94d0ac0640 1 -- start start 2024-05-10T10:33:50.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.583+0000 7f3aff59e640 1 --2- 172.21.15.53:0/2073081207 >> v2:172.21.15.135:3300/0 conn(0x7f3b0006c2c0 0x7f3b0006c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3712116875 0 0) 0x7f3b00167950 con 0x7f3b00167390 2024-05-10T10:33:50.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ae8003520 con 0x7f3b00167390 2024-05-10T10:33:50.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249540588 0 0) 0x7f3b00167c30 con 0x7f3b000c4f20 2024-05-10T10:33:50.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b00167950 con 0x7f3b000c4f20 2024-05-10T10:33:50.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2155838387 0 0) 0x7f3ae8003520 con 0x7f3b00167390 2024-05-10T10:33:50.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f6508aa8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc053e70 0x7f64fc0542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f6508aa8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc050d50 con 0x7f64fc054980 2024-05-10T10:33:50.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f6508aa8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc050ff0 con 0x7f64fc053a90 2024-05-10T10:33:50.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f6508aa8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc009500 con 0x7f64fc053e70 2024-05-10T10:33:50.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 >> v2:172.21.15.135:3300/0 conn(0x7f3b0006c2c0 msgr2=0x7f3b0006c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 --2- 172.21.15.53:0/2073081207 >> v2:172.21.15.135:3300/0 conn(0x7f3b0006c2c0 0x7f3b0006c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.584+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 >> v1:172.21.15.135:6789/0 conn(0x7f3b000c4f20 legacy=0x7f3b000ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f65037fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f64fc053a90 0x7f64fc0502a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36500/0 (socket says 172.21.15.53:36500) 2024-05-10T10:33:50.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b00167f10 con 0x7f3b00167390 2024-05-10T10:33:50.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f6502ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f64fc054980 0x7f64fc008df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39770/0 (socket says 172.21.15.53:39770) 2024-05-10T10:33:50.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f6502ffd640 1 -- 172.21.15.53:0/3414406001 learned_addr learned my addr 172.21.15.53:0/3414406001 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:50.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f3af00015a0 con 0x7f3b00167390 2024-05-10T10:33:50.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 >> v1:172.21.15.53:6789/0 conn(0x7f3b00167390 legacy=0x7f3b0006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae8003be0 0x7f3ae8004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f6503fff640 1 --2- 172.21.15.53:0/3414406001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc053e70 0x7f64fc0542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae8004800 0x7f3ae8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f65027fc640 1 -- 172.21.15.53:0/3414406001 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546863156 0 0) 0x7f64fc050d50 con 0x7f64fc054980 2024-05-10T10:33:50.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94d0ac0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c4005e90 0x7f94c40062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f65027fc640 1 -- 172.21.15.53:0/3414406001 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64f4003520 con 0x7f64fc054980 2024-05-10T10:33:50.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8005330 0x7f3ae800ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000cfe40 con 0x7f3ae8004800 2024-05-10T10:33:50.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae800c2d0 con 0x7f3ae8005330 2024-05-10T10:33:50.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae800c4d0 con 0x7f3ae8003be0 2024-05-10T10:33:50.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7fd685274640 1 Processor -- start 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94d0ac0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f94c4006810 0x7f94c4006ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94d0ac0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c4053720 con 0x7f94c40ead00 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94d0ac0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f94c4054330 con 0x7f94c4006810 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94d0ac0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f94c4053a00 con 0x7f94c4005e90 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94cb7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f94c40ead00 0x7f94c404fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36502/0 (socket says 172.21.15.53:36502) 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94cb7fe640 1 -- 172.21.15.53:0/3050994108 learned_addr learned my addr 172.21.15.53:0/3050994108 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94caffd640 1 --2- 172.21.15.53:0/3050994108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c4005e90 0x7f94c40062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.585+0000 7f94cbfff640 1 --2- 172.21.15.53:0/3050994108 >> v2:172.21.15.53:3300/0 conn(0x7f94c4006810 0x7f94c4006ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3afed9d640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae8003be0 0x7f3ae8004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3afe59c640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae8004800 0x7f3ae8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3afe59c640 1 -- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae8003be0 msgr2=0x7f3ae8004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3afe59c640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae8003be0 0x7f3ae8004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3afe59c640 1 -- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8005330 msgr2=0x7f3ae800ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f6503fff640 1 -- 172.21.15.53:0/3414406001 >> v1:172.21.15.135:6789/0 conn(0x7f64fc053a90 legacy=0x7f64fc0502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f6503fff640 1 -- 172.21.15.53:0/3414406001 >> v1:172.21.15.53:6789/0 conn(0x7f64fc054980 legacy=0x7f64fc008df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7fdafbfff640 1 Processor -- start 2024-05-10T10:33:50.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7fdafbfff640 1 -- start start 2024-05-10T10:33:50.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f6503fff640 1 -- 172.21.15.53:0/3414406001 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0085f0 con 0x7f64fc053e70 2024-05-10T10:33:50.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f94ca7fc640 1 -- 172.21.15.53:0/3050994108 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1115569249 0 0) 0x7f94c4053720 con 0x7f94c40ead00 2024-05-10T10:33:50.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f94ca7fc640 1 -- 172.21.15.53:0/3050994108 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94bc003520 con 0x7f94c40ead00 2024-05-10T10:33:50.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f94caffd640 1 -- 172.21.15.53:0/3050994108 >> v2:172.21.15.53:3300/0 conn(0x7f94c4006810 msgr2=0x7f94c4006ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f94caffd640 1 --2- 172.21.15.53:0/3050994108 >> v2:172.21.15.53:3300/0 conn(0x7f94c4006810 0x7f94c4006ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f94caffd640 1 -- 172.21.15.53:0/3050994108 >> v1:172.21.15.135:6789/0 conn(0x7f94c40ead00 legacy=0x7f94c404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f94caffd640 1 -- 172.21.15.53:0/3050994108 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c4053ce0 con 0x7f94c4005e90 2024-05-10T10:33:50.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3aff59e640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8005330 0x7f3ae800ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3afe59c640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8005330 0x7f3ae800ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f3afe59c640 1 -- 172.21.15.53:0/2073081207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae8003520 con 0x7f3ae8004800 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7fd685274640 1 -- start start 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd685274640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6801608b0 0x7fd6800d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd685274640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd6800d1230 0x7fd6800d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd685274640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd680160c90 con 0x7fd6800b2900 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd685274640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd6800ce5f0 con 0x7fd6800d1230 2024-05-10T10:33:50.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd685274640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd6800ce8d0 con 0x7fd6801608b0 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f7b949c1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b880078d0 0x7f7b880edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f7b949c1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b88058560 con 0x7f7b8803e830 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f7b949c1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b88058840 con 0x7f7b880e30b0 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.586+0000 7f7b949c1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7b88058b20 con 0x7f7b880078d0 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f7b8ed9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b880078d0 0x7f7b880edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f7b8ed9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b880078d0 0x7f7b880edd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36258/0 (socket says 172.21.15.53:36258) 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f7b8ed9d640 1 -- 172.21.15.53:0/4146285777 learned_addr learned my addr 172.21.15.53:0/4146285777 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f6503fff640 1 --2- 172.21.15.53:0/3414406001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc053e70 0x7f64fc0542a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6504000b60 tx=0x7f65040a64d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1904c801adbb31e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f6707719640 1 Processor -- start 2024-05-10T10:33:50.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f3aff59e640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8005330 0x7f3ae800ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:50.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f3aff59e640 1 -- 172.21.15.53:0/2073081207 reap_dead start 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd67effd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6801608b0 0x7fd6800d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd67f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd6800b2900 0x7fd6800b2d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36530/0 (socket says 172.21.15.53:36530) 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd67f7fe640 1 -- 172.21.15.53:0/3379443172 learned_addr learned my addr 172.21.15.53:0/3379443172 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fd67e7fc640 1 --2- 172.21.15.53:0/3379443172 >> v2:172.21.15.135:3300/0 conn(0x7fd6800d1230 0x7fd6800d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fd67dffb640 1 -- 172.21.15.53:0/3379443172 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 863613544 0 0) 0x7fd674001110 con 0x7fd6800b2900 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fd67dffb640 1 -- 172.21.15.53:0/3379443172 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd668003520 con 0x7fd6800b2900 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fd67effd640 1 -- 172.21.15.53:0/3379443172 >> v2:172.21.15.135:3300/0 conn(0x7fd6800d1230 msgr2=0x7fd6800d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fd67effd640 1 --2- 172.21.15.53:0/3379443172 >> v2:172.21.15.135:3300/0 conn(0x7fd6800d1230 0x7fd6800d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fd67effd640 1 -- 172.21.15.53:0/3379443172 >> v1:172.21.15.135:6789/0 conn(0x7fd6800b2900 legacy=0x7fd6800b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fd67effd640 1 -- 172.21.15.53:0/3379443172 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6800cebb0 con 0x7fd6801608b0 2024-05-10T10:33:50.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f7b8ed9d640 1 -- 172.21.15.53:0/4146285777 >> v1:172.21.15.135:6789/0 conn(0x7f7b880e30b0 legacy=0x7f7b88057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f7b8ed9d640 1 -- 172.21.15.53:0/4146285777 >> v1:172.21.15.53:6789/0 conn(0x7f7b8803e830 legacy=0x7f7b88053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f7b8ed9d640 1 -- 172.21.15.53:0/4146285777 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b880063a0 con 0x7f7b880078d0 2024-05-10T10:33:50.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7fdafbfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdafc0c6d70 0x7fdafc0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f65027fc640 1 -- 172.21.15.53:0/3414406001 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040b2070 con 0x7f64fc053e70 2024-05-10T10:33:50.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f65027fc640 1 -- 172.21.15.53:0/3414406001 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65040ab710 con 0x7f64fc053e70 2024-05-10T10:33:50.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdafc06d9f0 0x7fdafc06de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafbfff640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdafc06e340 con 0x7fdafc06e5b0 2024-05-10T10:33:50.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafbfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fdafc160300 con 0x7fdafc06d9f0 2024-05-10T10:33:50.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafbfff640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdafc0ce8e0 con 0x7fdafc0c6d70 2024-05-10T10:33:50.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f65027fc640 1 -- 172.21.15.53:0/3414406001 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040abb40 con 0x7f64fc053e70 2024-05-10T10:33:50.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdafc0c6d70 0x7fdafc0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdafc0c6d70 0x7fdafc0d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51144/0 (socket says 172.21.15.53:51144) 2024-05-10T10:33:50.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafaffd640 1 -- 172.21.15.53:0/450963536 learned_addr learned my addr 172.21.15.53:0/450963536 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafb7fe640 1 --2- 172.21.15.53:0/450963536 >> v2:172.21.15.53:3300/0 conn(0x7fdafc06d9f0 0x7fdafc06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f94caffd640 1 --2- 172.21.15.53:0/3050994108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c4005e90 0x7f94c40062d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f94b80097a0 tx=0x7f94b802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445f778a9dc5ba44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafaffd640 1 -- 172.21.15.53:0/450963536 >> v2:172.21.15.53:3300/0 conn(0x7fdafc06d9f0 msgr2=0x7fdafc06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafaffd640 1 --2- 172.21.15.53:0/450963536 >> v2:172.21.15.53:3300/0 conn(0x7fdafc06d9f0 0x7fdafc06de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafaffd640 1 -- 172.21.15.53:0/450963536 >> v1:172.21.15.135:6789/0 conn(0x7fdafc06e5b0 legacy=0x7fdafc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7fdafaffd640 1 -- 172.21.15.53:0/450963536 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc0cebc0 con 0x7fdafc0c6d70 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.587+0000 7f6707719640 1 -- start start 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f6707719640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f6700167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f6707719640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67000cdd50 0x7f67000ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f6707719640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6700168020 con 0x7f67000c7520 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f6707719640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f67000c7a20 con 0x7f67000cdd50 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f6707719640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f67000ce880 con 0x7f67001634b0 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6705cb6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67000cdd50 0x7f67000ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6705cb6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67000cdd50 0x7f67000ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51160/0 (socket says 172.21.15.53:51160) 2024-05-10T10:33:50.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6705cb6640 1 -- 172.21.15.53:0/132132600 learned_addr learned my addr 172.21.15.53:0/132132600 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f67054b5640 1 --2- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f6700167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6508aa8640 1 -- 172.21.15.53:0/3414406001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc053e70 msgr2=0x7f64fc0542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7fd67effd640 1 --2- 172.21.15.53:0/3379443172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6801608b0 0x7fd6800d0cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd67802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9551d19687e8fdb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f7b8ed9d640 1 --2- 172.21.15.53:0/4146285777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b880078d0 0x7f7b880edd80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b90069aa0 tx=0x7f7b9006a170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9754f0f435dd2b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f3afe59c640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae8004800 0x7f3ae8004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3af0007ed0 tx=0x7f3ae8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f94dcbcce1f44a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0007af0 con 0x7f3ae8004800 2024-05-10T10:33:50.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af0005af0 con 0x7f3ae8004800 2024-05-10T10:33:50.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6508aa8640 1 --2- 172.21.15.53:0/3414406001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc053e70 0x7f64fc0542a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6504000b60 tx=0x7f65040a64d0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6705cb6640 1 -- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 msgr2=0x7f6700167ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6705cb6640 1 --2- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f6700167ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6705cb6640 1 -- 172.21.15.53:0/132132600 >> v1:172.21.15.135:6789/0 conn(0x7f67000c7520 legacy=0x7f67000cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6705cb6640 1 -- 172.21.15.53:0/132132600 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67000ceb60 con 0x7f67000cdd50 2024-05-10T10:33:50.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7fd67dffb640 1 -- 172.21.15.53:0/3379443172 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678007740 con 0x7fd6801608b0 2024-05-10T10:33:50.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6508aa8640 1 -- 172.21.15.53:0/3414406001 shutdown_connections 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6508aa8640 1 --2- 172.21.15.53:0/3414406001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc053e70 0x7f64fc0542a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6508aa8640 1 -- 172.21.15.53:0/3414406001 >> 172.21.15.53:0/3414406001 conn(0x7f64fc055c50 msgr2=0x7f64fc056060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f94ca7fc640 1 -- 172.21.15.53:0/3050994108 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8037070 con 0x7f94c4005e90 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f94ca7fc640 1 -- 172.21.15.53:0/3050994108 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b8032440 con 0x7f94c4005e90 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.588+0000 7f94ca7fc640 1 -- 172.21.15.53:0/3050994108 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8032810 con 0x7f94c4005e90 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3050994108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c4005e90 msgr2=0x7f94c40062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3050994108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c4005e90 0x7f94c40062d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f94b80097a0 tx=0x7f94b802a180 comp rx=0 tx=0).stop 2024-05-10T10:33:50.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7f6508aa8640 1 -- 172.21.15.53:0/3414406001 shutdown_connections 2024-05-10T10:33:50.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.589+0000 7fd67dffb640 1 -- 172.21.15.53:0/3379443172 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6780078e0 con 0x7fd6801608b0 2024-05-10T10:33:50.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7fd67dffb640 1 -- 172.21.15.53:0/3379443172 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678007cb0 con 0x7fd6801608b0 2024-05-10T10:33:50.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f6508aa8640 1 -- 172.21.15.53:0/3414406001 wait complete. 2024-05-10T10:33:50.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3050994108 shutdown_connections 2024-05-10T10:33:50.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3050994108 >> v2:172.21.15.53:3300/0 conn(0x7f94c4006810 0x7f94c4006ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3050994108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c4005e90 0x7f94c40062d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3050994108 >> 172.21.15.53:0/3050994108 conn(0x7f94c4009fb0 msgr2=0x7f94c400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7fdafaffd640 1 --2- 172.21.15.53:0/450963536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc0c6d70 0x7fdafc0d0400 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdafc070180 tx=0x7fdaf402a760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1a4b896044920b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3050994108 shutdown_connections 2024-05-10T10:33:50.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3afdd9b640 1 -- 172.21.15.53:0/2073081207 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0007070 con 0x7f3ae8004800 2024-05-10T10:33:50.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3b04a44640 1 -- 172.21.15.53:0/2073081207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae8004800 msgr2=0x7f3ae8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3b04a44640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae8004800 0x7f3ae8004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3af0007ed0 tx=0x7f3ae8005830 comp rx=0 tx=0).stop 2024-05-10T10:33:50.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3050994108 wait complete. 2024-05-10T10:33:50.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f7b8d59a640 1 -- 172.21.15.53:0/4146285777 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900b2070 con 0x7f7b880078d0 2024-05-10T10:33:50.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f7b8d59a640 1 -- 172.21.15.53:0/4146285777 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b900ad600 con 0x7f7b880078d0 2024-05-10T10:33:50.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f7b8d59a640 1 -- 172.21.15.53:0/4146285777 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900ada30 con 0x7f7b880078d0 2024-05-10T10:33:50.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7fdaf9ffb640 1 -- 172.21.15.53:0/450963536 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4036070 con 0x7fdafc0c6d70 2024-05-10T10:33:50.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7fdaf9ffb640 1 -- 172.21.15.53:0/450963536 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf4031600 con 0x7fdafc0c6d70 2024-05-10T10:33:50.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f6705cb6640 1 --2- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000ae230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009e50 tx=0x7f66f802a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f524e795bf3080f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7fdaf9ffb640 1 -- 172.21.15.53:0/450963536 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4031a10 con 0x7fdafc0c6d70 2024-05-10T10:33:50.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7fd685274640 1 -- 172.21.15.53:0/3379443172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6801608b0 msgr2=0x7fd6800d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7fd685274640 1 --2- 172.21.15.53:0/3379443172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6801608b0 0x7fd6800d0cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd67802a660 comp rx=0 tx=0).stop 2024-05-10T10:33:50.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f66f7fff640 1 -- 172.21.15.53:0/132132600 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8036070 con 0x7f67000cdd50 2024-05-10T10:33:50.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3b04a44640 1 -- 172.21.15.53:0/2073081207 shutdown_connections 2024-05-10T10:33:50.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3b04a44640 1 --2- 172.21.15.53:0/2073081207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae8004800 0x7f3ae8004c80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3b04a44640 1 -- 172.21.15.53:0/2073081207 >> 172.21.15.53:0/2073081207 conn(0x7f3b000d1b80 msgr2=0x7f3b000d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f66f7fff640 1 -- 172.21.15.53:0/132132600 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f8031600 con 0x7f67000cdd50 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f66f7fff640 1 -- 172.21.15.53:0/132132600 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8031a10 con 0x7f67000cdd50 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f6707719640 1 -- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 msgr2=0x7f67000ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f7b949c1640 1 -- 172.21.15.53:0/4146285777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b880078d0 msgr2=0x7f7b880edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f7b949c1640 1 --2- 172.21.15.53:0/4146285777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b880078d0 0x7f7b880edd80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b90069aa0 tx=0x7f7b9006a170 comp rx=0 tx=0).stop 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f7b949c1640 1 -- 172.21.15.53:0/4146285777 shutdown_connections 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f7b949c1640 1 --2- 172.21.15.53:0/4146285777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b880078d0 0x7f7b880edd80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f7b949c1640 1 -- 172.21.15.53:0/4146285777 >> 172.21.15.53:0/4146285777 conn(0x7f7b880559d0 msgr2=0x7f7b88055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f6508aa8640 1 Processor -- start 2024-05-10T10:33:50.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f94d0ac0640 1 Processor -- start 2024-05-10T10:33:50.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3b04a44640 1 -- 172.21.15.53:0/2073081207 shutdown_connections 2024-05-10T10:33:50.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.590+0000 7f3b04a44640 1 -- 172.21.15.53:0/2073081207 wait complete. 2024-05-10T10:33:50.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f7b949c1640 1 -- 172.21.15.53:0/4146285777 shutdown_connections 2024-05-10T10:33:50.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f6707719640 1 --2- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000ae230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009e50 tx=0x7f66f802a6a0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7fd685274640 1 -- 172.21.15.53:0/3379443172 shutdown_connections 2024-05-10T10:33:50.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7f94d0ac0640 1 -- start start 2024-05-10T10:33:50.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fdafbfff640 1 -- 172.21.15.53:0/450963536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc0c6d70 msgr2=0x7fdafc0d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7fd685274640 1 --2- 172.21.15.53:0/3379443172 >> v2:172.21.15.135:3300/0 conn(0x7fd6800d1230 0x7fd6800d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7fd685274640 1 --2- 172.21.15.53:0/3379443172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6801608b0 0x7fd6800d0cf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7fd685274640 1 -- 172.21.15.53:0/3379443172 >> 172.21.15.53:0/3379443172 conn(0x7fd6800d2df0 msgr2=0x7fd6800d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fdafbfff640 1 --2- 172.21.15.53:0/450963536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc0c6d70 0x7fdafc0d0400 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdafc070180 tx=0x7fdaf402a760 comp rx=0 tx=0).stop 2024-05-10T10:33:50.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fdafbfff640 1 -- 172.21.15.53:0/450963536 shutdown_connections 2024-05-10T10:33:50.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fdafbfff640 1 --2- 172.21.15.53:0/450963536 >> v2:172.21.15.53:3300/0 conn(0x7fdafc06d9f0 0x7fdafc06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fdafbfff640 1 --2- 172.21.15.53:0/450963536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc0c6d70 0x7fdafc0d0400 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdafc070180 tx=0x7fdaf402a760 comp rx=0 tx=0).stop 2024-05-10T10:33:50.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fdafbfff640 1 -- 172.21.15.53:0/450963536 >> 172.21.15.53:0/450963536 conn(0x7fdafc0cd460 msgr2=0x7fdafc0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7f6707719640 1 -- 172.21.15.53:0/132132600 shutdown_connections 2024-05-10T10:33:50.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7f6707719640 1 --2- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000ae230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7f6707719640 1 --2- 172.21.15.53:0/132132600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f6700167ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7f6707719640 1 -- 172.21.15.53:0/132132600 >> 172.21.15.53:0/132132600 conn(0x7f67000b2900 msgr2=0x7f67000b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7f3b04a44640 1 Processor -- start 2024-05-10T10:33:50.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.591+0000 7f7b949c1640 1 -- 172.21.15.53:0/4146285777 wait complete. 2024-05-10T10:33:50.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f3b04a44640 1 -- start start 2024-05-10T10:33:50.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7fdd1dc0b640 1 Processor -- start 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7fdd1dc0b640 1 -- start start 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94d0ac0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4005e90 0x7f94c417c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94d0ac0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c4006810 0x7f94c4035e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94d0ac0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c40ead00 0x7f94c4189070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94d0ac0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40547c0 con 0x7f94c4005e90 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94d0ac0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40501a0 con 0x7f94c4006810 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94d0ac0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f94c4054940 con 0x7f94c40ead00 2024-05-10T10:33:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94cbfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c40ead00 0x7f94c4189070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94cbfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c40ead00 0x7f94c4189070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36286/0 (socket says 172.21.15.53:36286) 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94cbfff640 1 -- 172.21.15.53:0/3611345401 learned_addr learned my addr 172.21.15.53:0/3611345401 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94caffd640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c4006810 0x7f94c4035e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f94cb7fe640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4005e90 0x7f94c417c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f94cbfff640 1 -- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c4006810 msgr2=0x7f94c4035e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f94cbfff640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c4006810 0x7f94c4035e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f94cbfff640 1 -- 172.21.15.53:0/3611345401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4005e90 msgr2=0x7f94c417c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f94cbfff640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4005e90 0x7f94c417c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f94cbfff640 1 -- 172.21.15.53:0/3611345401 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c417c9b0 con 0x7f94c40ead00 2024-05-10T10:33:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f94cb7fe640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4005e90 0x7f94c417c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f7b949c1640 1 Processor -- start 2024-05-10T10:33:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f7b949c1640 1 -- start start 2024-05-10T10:33:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f6707719640 1 -- 172.21.15.53:0/132132600 shutdown_connections 2024-05-10T10:33:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f3b04a44640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8004800 0x7f3b000c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f3b04a44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0014e970 0x7f3b000c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f3b04a44640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b0014a960 0x7f3b0014adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f3b04a44640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000c5480 con 0x7f3b0014e970 2024-05-10T10:33:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f94cbfff640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c40ead00 0x7f94c4189070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94cc06a840 tx=0x7f94cc069ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f3b04a44640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000c5300 con 0x7f3ae8004800 2024-05-10T10:33:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f3b04a44640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0006c750 con 0x7f3b0014a960 2024-05-10T10:33:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.593+0000 7f6707719640 1 -- 172.21.15.53:0/132132600 wait complete. 2024-05-10T10:33:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7f6508aa8640 1 -- start start 2024-05-10T10:33:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7f3afed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8004800 0x7f3b000c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f3afed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8004800 0x7f3b000c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51176/0 (socket says 172.21.15.53:51176) 2024-05-10T10:33:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f3afed9d640 1 -- 172.21.15.53:0/1756795038 learned_addr learned my addr 172.21.15.53:0/1756795038 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6707719640 1 Processor -- start 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f3afe59c640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0014e970 0x7f3b000c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f3aff59e640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b0014a960 0x7f3b0014adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7fdd1dc0b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdd1004b3c0 0x7fdd10034020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7fdd1dc0b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004f4d0 0x7fdd1004f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd1dc0b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd10054c10 con 0x7fdd1004fe90 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd1dc0b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdd100062e0 con 0x7fdd1004b3c0 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd1dc0b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdd10053a50 con 0x7fdd1004f4d0 2024-05-10T10:33:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd17fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004f4d0 0x7fdd1004f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd17fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004f4d0 0x7fdd1004f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36294/0 (socket says 172.21.15.53:36294) 2024-05-10T10:33:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd17fff640 1 -- 172.21.15.53:0/479571264 learned_addr learned my addr 172.21.15.53:0/479571264 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd1cc09640 1 --2- 172.21.15.53:0/479571264 >> v2:172.21.15.135:3300/0 conn(0x7fdd1004b3c0 0x7fdd10034020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6707719640 1 -- start start 2024-05-10T10:33:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f7b949c1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b880078d0 0x7f7b8804ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdafbfff640 1 -- 172.21.15.53:0/450963536 shutdown_connections 2024-05-10T10:33:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6508aa8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64fc053a90 0x7f64fc0cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6508aa8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64fc053e70 0x7f64fc0d3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6508aa8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc054980 0x7f64fc04eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f7b949c1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b8803e830 0x7f7b880dde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6508aa8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc038d80 con 0x7f64fc053a90 2024-05-10T10:33:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6508aa8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0512d0 con 0x7f64fc053e70 2024-05-10T10:33:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6508aa8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc038f00 con 0x7f64fc054980 2024-05-10T10:33:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f7b8e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b880078d0 0x7f7b8804ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f7b8e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b880078d0 0x7f7b8804ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51190/0 (socket says 172.21.15.53:51190) 2024-05-10T10:33:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6503fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc054980 0x7f64fc04eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64fc053e70 0x7f64fc0d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f65037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64fc053a90 0x7f64fc0cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f7b8dd9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b8803e830 0x7f7b880dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f7b8dd9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b8803e830 0x7f7b880dde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36308/0 (socket says 172.21.15.53:36308) 2024-05-10T10:33:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f7b8e59c640 1 -- 172.21.15.53:0/233407549 learned_addr learned my addr 172.21.15.53:0/233407549 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f65037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64fc053a90 0x7f64fc0cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43492/0 (socket says 172.21.15.53:43492) 2024-05-10T10:33:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f65037fe640 1 -- 172.21.15.53:0/860227813 learned_addr learned my addr 172.21.15.53:0/860227813 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7f6503fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc054980 0x7f64fc04eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36306/0 (socket says 172.21.15.53:36306) 2024-05-10T10:33:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f7b949c1640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b880e30b0 0x7f7b880de380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fd685274640 1 -- 172.21.15.53:0/3379443172 shutdown_connections 2024-05-10T10:33:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd17fff640 1 -- 172.21.15.53:0/479571264 >> v2:172.21.15.135:3300/0 conn(0x7fdd1004b3c0 msgr2=0x7fdd10034020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd17fff640 1 --2- 172.21.15.53:0/479571264 >> v2:172.21.15.135:3300/0 conn(0x7fdd1004b3c0 0x7fdd10034020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd17fff640 1 -- 172.21.15.53:0/479571264 >> v1:172.21.15.135:6789/0 conn(0x7fdd1004fe90 legacy=0x7fdd100eb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdd17fff640 1 -- 172.21.15.53:0/479571264 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd10053d30 con 0x7fdd1004f4d0 2024-05-10T10:33:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880542f0 con 0x7f7b880e30b0 2024-05-10T10:33:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880e3490 con 0x7f7b880078d0 2024-05-10T10:33:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7b88054470 con 0x7f7b8803e830 2024-05-10T10:33:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.592+0000 7fd685274640 1 -- 172.21.15.53:0/3379443172 wait complete. 2024-05-10T10:33:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f3afed9d640 1 -- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b0014a960 msgr2=0x7f3b0014adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f3afed9d640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b0014a960 0x7f3b0014adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f3afed9d640 1 -- 172.21.15.53:0/1756795038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0014e970 msgr2=0x7f3b000c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f3afed9d640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0014e970 0x7f3b000c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f3afed9d640 1 -- 172.21.15.53:0/1756795038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0014edf0 con 0x7f3ae8004800 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7fd685274640 1 Processor -- start 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.594+0000 7fd685274640 1 -- start start 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fd685274640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6800b2900 0x7fd680156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fd685274640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6800d1230 0x7fd68006e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fd685274640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6801608b0 0x7fd6800cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fd685274640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd6800d1f10 con 0x7fd6800b2900 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fd685274640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6800d1d90 con 0x7fd6801608b0 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fd685274640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd6800d03c0 con 0x7fd6800d1230 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6800b2900 0x7fd680156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6800b2900 0x7fd680156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43478/0 (socket says 172.21.15.53:43478) 2024-05-10T10:33:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 -- 172.21.15.53:0/3236701420 learned_addr learned my addr 172.21.15.53:0/3236701420 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67e7fc640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6800d1230 0x7fd68006e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67f7fe640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6801608b0 0x7fd6800cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f7b8ed9d640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b880e30b0 0x7f7b880de380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f6503fff640 1 -- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64fc053e70 msgr2=0x7f64fc0d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f6503fff640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64fc053e70 0x7f64fc0d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f6503fff640 1 -- 172.21.15.53:0/860227813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64fc053a90 msgr2=0x7f64fc0cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f6503fff640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64fc053a90 0x7f64fc0cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f6503fff640 1 -- 172.21.15.53:0/860227813 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0cf8e0 con 0x7f64fc054980 2024-05-10T10:33:50.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7f3afed9d640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8004800 0x7f3b000c5fd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3af400f810 tx=0x7f3af400fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.595+0000 7fdafbfff640 1 -- 172.21.15.53:0/450963536 wait complete. 2024-05-10T10:33:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 -- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6800d1230 msgr2=0x7fd68006e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6800d1230 0x7fd68006e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 -- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6801608b0 msgr2=0x7fd6800cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6801608b0 0x7fd6800cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.596+0000 7fd67effd640 1 -- 172.21.15.53:0/3236701420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd680155930 con 0x7fd6800b2900 2024-05-10T10:33:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fd67e7fc640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6800d1230 0x7fd68006e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f6503fff640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc054980 0x7f64fc04eaa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6504052040 tx=0x7f6504076270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f7b8e59c640 1 -- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b8803e830 msgr2=0x7f7b880dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f7b8e59c640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b8803e830 0x7f7b880dde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f7b8e59c640 1 -- 172.21.15.53:0/233407549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b880e30b0 msgr2=0x7f7b880de380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f7b8e59c640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b880e30b0 0x7f7b880de380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f7b8e59c640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b880d74b0 con 0x7f7b880078d0 2024-05-10T10:33:50.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fd67effd640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6800b2900 0x7fd680156d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fdd17fff640 1 --2- 172.21.15.53:0/479571264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004f4d0 0x7fdd1004f950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009870 tx=0x7fdd0c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd215cb2be3eb1d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f6707719640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67000c7520 0x7f67000b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fd6677fe640 1 -- 172.21.15.53:0/3236701420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67803f070 con 0x7fd6800b2900 2024-05-10T10:33:50.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f7b8e59c640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b880078d0 0x7f7b8804ea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c430 tx=0x7f7b8400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fd685274640 1 -- 172.21.15.53:0/3236701420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68006e970 con 0x7fd6800b2900 2024-05-10T10:33:50.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fd685274640 1 -- 172.21.15.53:0/3236701420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6801577d0 con 0x7fd6800b2900 2024-05-10T10:33:50.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fd6677fe640 1 -- 172.21.15.53:0/3236701420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd678031bc0 con 0x7fd6800b2900 2024-05-10T10:33:50.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f6707719640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f67054b5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67000c7520 0x7f67000b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f6707719640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f67000b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f6707719640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6700163890 con 0x7f67000c7520 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f6707719640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67000ae770 con 0x7f67000cdd50 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f6707719640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f67000b6830 con 0x7f67001634b0 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f67054b5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67000c7520 0x7f67000b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43506/0 (socket says 172.21.15.53:43506) 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7f67054b5640 1 -- 172.21.15.53:0/2233086923 learned_addr learned my addr 172.21.15.53:0/2233086923 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.597+0000 7fd6677fe640 1 -- 172.21.15.53:0/3236701420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678044460 con 0x7fd6800b2900 2024-05-10T10:33:50.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f6704cb4640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7fdafbfff640 1 Processor -- start 2024-05-10T10:33:50.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f6705cb6640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f67000b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f67054b5640 1 -- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 msgr2=0x7f67000b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f67054b5640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f67000b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f67054b5640 1 -- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 msgr2=0x7f67000b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f67054b5640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f67054b5640 1 -- 172.21.15.53:0/2233086923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6700164a90 con 0x7f67000c7520 2024-05-10T10:33:50.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7fdafbfff640 1 -- start start 2024-05-10T10:33:50.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f6705cb6640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f67000b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:50.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f6704cb4640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:50.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7f67054b5640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67000c7520 0x7f67000b41e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00d5f0 tx=0x7f66fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.598+0000 7fd685274640 1 -- 172.21.15.53:0/3236701420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68006a220 con 0x7fd6800b2900 2024-05-10T10:33:50.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f66f5ffb640 1 -- 172.21.15.53:0/2233086923 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0125f0 con 0x7f67000c7520 2024-05-10T10:33:50.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6700164d70 con 0x7f67000c7520 2024-05-10T10:33:50.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f66f5ffb640 1 -- 172.21.15.53:0/2233086923 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66fc019070 con 0x7f67000c7520 2024-05-10T10:33:50.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f66f5ffb640 1 -- 172.21.15.53:0/2233086923 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0129d0 con 0x7f67000c7520 2024-05-10T10:33:50.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdafc06d9f0 0x7fdafc0b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc06e5b0 0x7fdafc159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdafc0b20f0 0x7fdafc144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc15ff60 con 0x7fdafc06d9f0 2024-05-10T10:33:50.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafbfff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc0ce5b0 con 0x7fdafc06e5b0 2024-05-10T10:33:50.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafbfff640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdafc1600e0 con 0x7fdafc0b20f0 2024-05-10T10:33:50.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdafc06d9f0 0x7fdafc0b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdafc06d9f0 0x7fdafc0b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43508/0 (socket says 172.21.15.53:43508) 2024-05-10T10:33:50.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafaffd640 1 -- 172.21.15.53:0/776182723 learned_addr learned my addr 172.21.15.53:0/776182723 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7fdafb7fe640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdafc0b20f0 0x7fdafc144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6700165610 con 0x7f67000c7520 2024-05-10T10:33:50.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f94b7fff640 1 -- 172.21.15.53:0/3611345401 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0757b0 con 0x7f94c40ead00 2024-05-10T10:33:50.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f64effff640 1 -- 172.21.15.53:0/860227813 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040bd070 con 0x7f64fc054980 2024-05-10T10:33:50.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.599+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c4036840 con 0x7f94c40ead00 2024-05-10T10:33:50.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c4186ce0 con 0x7f94c40ead00 2024-05-10T10:33:50.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc0d38a0 con 0x7f64fc054980 2024-05-10T10:33:50.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc0d9460 con 0x7f64fc054980 2024-05-10T10:33:50.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f94b7fff640 1 -- 172.21.15.53:0/3611345401 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc0759b0 con 0x7f94c40ead00 2024-05-10T10:33:50.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f94b7fff640 1 -- 172.21.15.53:0/3611345401 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc075de0 con 0x7f94c40ead00 2024-05-10T10:33:50.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f64effff640 1 -- 172.21.15.53:0/860227813 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65040ad970 con 0x7f64fc054980 2024-05-10T10:33:50.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f64effff640 1 -- 172.21.15.53:0/860227813 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040b2040 con 0x7f64fc054980 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafa7fc640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc06e5b0 0x7fdafc159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafb7fe640 1 -- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc06e5b0 msgr2=0x7fdafc159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafb7fe640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc06e5b0 0x7fdafc159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafb7fe640 1 -- 172.21.15.53:0/776182723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdafc06d9f0 msgr2=0x7fdafc0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafb7fe640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdafc06d9f0 0x7fdafc0b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafb7fe640 1 -- 172.21.15.53:0/776182723 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc0b19f0 con 0x7fdafc0b20f0 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafa7fc640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc06e5b0 0x7fdafc159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:50.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafaffd640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdafc06d9f0 0x7fdafc0b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:50.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f3ae77fe640 1 -- 172.21.15.53:0/1756795038 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40052e0 con 0x7f3ae8004800 2024-05-10T10:33:50.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f3ae77fe640 1 -- 172.21.15.53:0/1756795038 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af4018070 con 0x7f3ae8004800 2024-05-10T10:33:50.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f3ae77fe640 1 -- 172.21.15.53:0/1756795038 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af400d970 con 0x7f3ae8004800 2024-05-10T10:33:50.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b000b5420 con 0x7f3ae8004800 2024-05-10T10:33:50.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b000b59e0 con 0x7f3ae8004800 2024-05-10T10:33:50.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7fdafb7fe640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdafc0b20f0 0x7fdafc144310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00d560 tx=0x7fdaec00da20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.601+0000 7fdd177fe640 1 -- 172.21.15.53:0/479571264 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c037070 con 0x7fdd1004f4d0 2024-05-10T10:33:50.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.601+0000 7fdd177fe640 1 -- 172.21.15.53:0/479571264 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0c032440 con 0x7fdd1004f4d0 2024-05-10T10:33:50.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.601+0000 7fdd177fe640 1 -- 172.21.15.53:0/479571264 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c032810 con 0x7fdd1004f4d0 2024-05-10T10:33:50.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.601+0000 7fd6677fe640 1 -- 172.21.15.53:0/3236701420 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd678044600 con 0x7fd6800b2900 2024-05-10T10:33:50.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fd6677fe640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd66c0762d0 0x7fd66c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fd6677fe640 1 -- 172.21.15.53:0/3236701420 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd6780b9310 con 0x7fd6800b2900 2024-05-10T10:33:50.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/479571264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004f4d0 msgr2=0x7fdd1004f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fdd1dc0b640 1 --2- 172.21.15.53:0/479571264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004f4d0 0x7fdd1004f950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009870 tx=0x7fdd0c02a210 comp rx=0 tx=0).stop 2024-05-10T10:33:50.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/479571264 shutdown_connections 2024-05-10T10:33:50.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fdd1dc0b640 1 --2- 172.21.15.53:0/479571264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004f4d0 0x7fdd1004f950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fdd1dc0b640 1 --2- 172.21.15.53:0/479571264 >> v2:172.21.15.135:3300/0 conn(0x7fdd1004b3c0 0x7fdd10034020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/479571264 >> 172.21.15.53:0/479571264 conn(0x7fdd1000a400 msgr2=0x7fdd1000a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:50.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.603+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae0005290 con 0x7f3ae8004800 2024-05-10T10:33:50.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7fdae37fe640 1 -- 172.21.15.53:0/776182723 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0125f0 con 0x7fdafc0b20f0 2024-05-10T10:33:50.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.603+0000 7fdae37fe640 1 -- 172.21.15.53:0/776182723 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec019070 con 0x7fdafc0b20f0 2024-05-10T10:33:50.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7fd67e7fc640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd66c0762d0 0x7fd66c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.601+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6700164f40 con 0x7f67000c7520 2024-05-10T10:33:50.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7fdae37fe640 1 -- 172.21.15.53:0/776182723 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0129b0 con 0x7fdafc0b20f0 2024-05-10T10:33:50.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7f66f5ffb640 1 -- 172.21.15.53:0/2233086923 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f66fc012b70 con 0x7f67000c7520 2024-05-10T10:33:50.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.603+0000 7f66f5ffb640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66e40762d0 0x7f66e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.602+0000 7f94b7fff640 1 -- 172.21.15.53:0/3611345401 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f94cc0741c0 con 0x7f94c40ead00 2024-05-10T10:33:50.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.603+0000 7f94b7fff640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94b0076300 0x7f94b00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7f94b7fff640 1 -- 172.21.15.53:0/3611345401 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f94cc1137c0 con 0x7f94c40ead00 2024-05-10T10:33:50.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac005290 con 0x7f94c40ead00 2024-05-10T10:33:50.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/479571264 shutdown_connections 2024-05-10T10:33:50.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7f6704cb4640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66e40762d0 0x7f66e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7fdafbfff640 1 -- 172.21.15.53:0/776182723 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc0b0430 con 0x7fdafc0b20f0 2024-05-10T10:33:50.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7fdafbfff640 1 -- 172.21.15.53:0/776182723 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc0b09d0 con 0x7fdafc0b20f0 2024-05-10T10:33:50.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7f94cb7fe640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94b0076300 0x7f94b00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f7b7effd640 1 -- 172.21.15.53:0/233407549 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011790 con 0x7f7b880078d0 2024-05-10T10:33:50.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f7b7effd640 1 -- 172.21.15.53:0/233407549 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84017070 con 0x7f7b880078d0 2024-05-10T10:33:50.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.600+0000 7f7b7effd640 1 -- 172.21.15.53:0/233407549 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011bb0 con 0x7f7b880078d0 2024-05-10T10:33:50.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.603+0000 7f64effff640 1 -- 172.21.15.53:0/860227813 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f65040b8470 con 0x7f64fc054980 2024-05-10T10:33:50.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.603+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e4005290 con 0x7f64fc054980 2024-05-10T10:33:50.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7f64effff640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64e80762d0 0x7f64e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.601+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b880d7790 con 0x7f7b880078d0 2024-05-10T10:33:50.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.601+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b880e0f30 con 0x7f7b880078d0 2024-05-10T10:33:50.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.603+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b880057d0 con 0x7f7b880078d0 2024-05-10T10:33:50.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7f64effff640 1 -- 172.21.15.53:0/860227813 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6504134200 con 0x7f64fc054980 2024-05-10T10:33:50.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7fd67e7fc640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd66c0762d0 0x7fd66c078790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd670009f30 tx=0x7fd670009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.604+0000 7f65037fe640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64e80762d0 0x7f64e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.605+0000 7f3ae77fe640 1 -- 172.21.15.53:0/1756795038 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3af400db10 con 0x7f3ae8004800 2024-05-10T10:33:50.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.605+0000 7f6704cb4640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66e40762d0 0x7f66e4078790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66f0007fd0 tx=0x7f66f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.605+0000 7f7b7effd640 1 -- 172.21.15.53:0/233407549 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7b8401c460 con 0x7f7b880078d0 2024-05-10T10:33:50.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.605+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/479571264 wait complete. 2024-05-10T10:33:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.605+0000 7f94cb7fe640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94b0076300 0x7f94b00787c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007fd0 tx=0x7f94c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.605+0000 7f65037fe640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64e80762d0 0x7f64e8078790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005ed0 tx=0x7f64f0007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.605+0000 7f3ae77fe640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3aec076320 0x7f3aec0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7f7b7effd640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b700763a0 0x7f7b70078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7f3ae77fe640 1 -- 172.21.15.53:0/1756795038 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3af4098670 con 0x7f3ae8004800 2024-05-10T10:33:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7f3afe59c640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3aec076320 0x7f3aec0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7f7b7effd640 1 -- 172.21.15.53:0/233407549 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7b84097730 con 0x7f7b880078d0 2024-05-10T10:33:50.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7f7b8dd9b640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b700763a0 0x7f7b70078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7fdafbfff640 1 -- 172.21.15.53:0/776182723 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdafc0765a0 con 0x7fdafc0b20f0 2024-05-10T10:33:50.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7f3afe59c640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3aec076320 0x7f3aec0787e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3af0032000 tx=0x7f3af0009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7fdd1dc0b640 1 Processor -- start 2024-05-10T10:33:50.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7f7b8dd9b640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b700763a0 0x7f7b70078860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b880533e0 tx=0x7f7b80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.606+0000 7fdd1dc0b640 1 -- start start 2024-05-10T10:33:50.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.607+0000 7fd6677fe640 1 -- 172.21.15.53:0/3236701420 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd678052080 con 0x7fd6800b2900 2024-05-10T10:33:50.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.607+0000 7fdd1dc0b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004b3c0 0x7fdd100dbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.607+0000 7fdd1dc0b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1004fe90 0x7fdd100ea220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.607+0000 7f66f5ffb640 1 -- 172.21.15.53:0/2233086923 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f66fc012e60 con 0x7f67000c7520 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1dc0b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd100ea760 0x7fdd1004df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1dc0b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10054830 con 0x7fdd100ea760 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1dc0b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1004b7a0 con 0x7fdd1004fe90 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1dc0b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdd100549b0 con 0x7fdd1004b3c0 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd17fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1004fe90 0x7fdd100ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1cc09640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004b3c0 0x7fdd100dbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd17fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1004fe90 0x7fdd100ea220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51254/0 (socket says 172.21.15.53:51254) 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd17fff640 1 -- 172.21.15.53:0/1443479952 learned_addr learned my addr 172.21.15.53:0/1443479952 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1cc09640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004b3c0 0x7fdd100dbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36336/0 (socket says 172.21.15.53:36336) 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1d40a640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd100ea760 0x7fdd1004df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1cc09640 1 -- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1004fe90 msgr2=0x7fdd100ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1cc09640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1004fe90 0x7fdd100ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1cc09640 1 -- 172.21.15.53:0/1443479952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd100ea760 msgr2=0x7fdd1004df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1cc09640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd100ea760 0x7fdd1004df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1cc09640 1 -- 172.21.15.53:0/1443479952 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd10180f80 con 0x7fdd1004b3c0 2024-05-10T10:33:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.608+0000 7fdd1d40a640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd100ea760 0x7fdd1004df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.609+0000 7fdd1cc09640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004b3c0 0x7fdd100dbfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e880 tx=0x7fdd0400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.610+0000 7f3ae77fe640 1 -- 172.21.15.53:0/1756795038 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3af4061c70 con 0x7f3ae8004800 2024-05-10T10:33:50.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.610+0000 7f64effff640 1 -- 172.21.15.53:0/860227813 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f65040fd810 con 0x7f64fc054980 2024-05-10T10:33:50.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.612+0000 7f66f5ffb640 1 -- 172.21.15.53:0/2233086923 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f66fc061390 con 0x7f67000c7520 2024-05-10T10:33:50.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.614+0000 7f94b7fff640 1 -- 172.21.15.53:0/3611345401 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f94cc0dcd80 con 0x7f94c40ead00 2024-05-10T10:33:50.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.614+0000 7f7b7effd640 1 -- 172.21.15.53:0/233407549 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7b84060d30 con 0x7f7b880078d0 2024-05-10T10:33:50.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.619+0000 7fdd157fa640 1 -- 172.21.15.53:0/1443479952 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04017070 con 0x7fdd1004b3c0 2024-05-10T10:33:50.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.619+0000 7fdd157fa640 1 -- 172.21.15.53:0/1443479952 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0400c940 con 0x7fdd1004b3c0 2024-05-10T10:33:50.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.619+0000 7fdd157fa640 1 -- 172.21.15.53:0/1443479952 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04017070 con 0x7fdd1004b3c0 2024-05-10T10:33:50.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.619+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/1443479952 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd10181260 con 0x7fdd1004b3c0 2024-05-10T10:33:50.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.620+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/1443479952 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd10181820 con 0x7fdd1004b3c0 2024-05-10T10:33:50.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.621+0000 7fdae37fe640 1 -- 172.21.15.53:0/776182723 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdaec012b50 con 0x7fdafc0b20f0 2024-05-10T10:33:50.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.622+0000 7fdae37fe640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdae80763a0 0x7fdae8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.622+0000 7fdafaffd640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdae80763a0 0x7fdae8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.622+0000 7fdae37fe640 1 -- 172.21.15.53:0/776182723 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdaec098710 con 0x7fdafc0b20f0 2024-05-10T10:33:50.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.623+0000 7fdafaffd640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdae80763a0 0x7fdae8078860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402a7a0 tx=0x7fdaf4009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.623+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/1443479952 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf8005290 con 0x7fdd1004b3c0 2024-05-10T10:33:50.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.624+0000 7fdae37fe640 1 -- 172.21.15.53:0/776182723 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdaec061d10 con 0x7fdafc0b20f0 2024-05-10T10:33:50.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.626+0000 7fdd157fa640 1 -- 172.21.15.53:0/1443479952 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdd0401a070 con 0x7fdd1004b3c0 2024-05-10T10:33:50.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.627+0000 7fdd157fa640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdcfc0766d0 0x7fdcfc078b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:50.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.627+0000 7fdd157fa640 1 -- 172.21.15.53:0/1443479952 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdd0400f040 con 0x7fdd1004b3c0 2024-05-10T10:33:50.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.627+0000 7fdd17fff640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdcfc0766d0 0x7fdcfc078b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:50.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.628+0000 7fdd17fff640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdcfc0766d0 0x7fdcfc078b90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd100e6fd0 tx=0x7fdd0c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:50.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:50.629+0000 7fdd157fa640 1 -- 172.21.15.53:0/1443479952 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdd04060a00 con 0x7fdd1004b3c0 2024-05-10T10:33:51.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.052+0000 7fd685274640 1 -- 172.21.15.53:0/3236701420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd68006f040 con 0x7fd6800b2900 2024-05-10T10:33:51.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.053+0000 7fd6677fe640 1 -- 172.21.15.53:0/3236701420 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd68006e970 con 0x7fd6800b2900 2024-05-10T10:33:51.054 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:51.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f64e4005580 con 0x7f64fc054980 2024-05-10T10:33:51.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 -- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd66c0762d0 msgr2=0x7fd66c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd66c0762d0 0x7fd66c078790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd670009f30 tx=0x7fd670009190 comp rx=0 tx=0).stop 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 -- 172.21.15.53:0/3236701420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6800b2900 msgr2=0x7fd680156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6800b2900 0x7fd680156d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd678009d90 tx=0x7fd678004a40 comp rx=0 tx=0).stop 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 -- 172.21.15.53:0/3236701420 shutdown_connections 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd66c0762d0 0x7fd66c078790 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6801608b0 0x7fd6800cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6800d1230 0x7fd68006e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.056+0000 7fd6657fa640 1 --2- 172.21.15.53:0/3236701420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6800b2900 0x7fd680156d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.057+0000 7fd6657fa640 1 -- 172.21.15.53:0/3236701420 >> 172.21.15.53:0/3236701420 conn(0x7fd6800d2df0 msgr2=0x7fd680070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.057+0000 7fd6657fa640 1 -- 172.21.15.53:0/3236701420 shutdown_connections 2024-05-10T10:33:51.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.057+0000 7fd6657fa640 1 -- 172.21.15.53:0/3236701420 wait complete. 2024-05-10T10:33:51.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.057+0000 7f64effff640 1 -- 172.21.15.53:0/860227813 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f65040fd110 con 0x7f64fc054980 2024-05-10T10:33:51.060 INFO:teuthology.orchestra.run.smithi053.stdout:55834574853 2024-05-10T10:33:51.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.060+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f670006a220 con 0x7f67000c7520 2024-05-10T10:33:51.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.060+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f94ac005580 con 0x7f94c40ead00 2024-05-10T10:33:51.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.060+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3ae0005850 con 0x7f3ae8004800 2024-05-10T10:33:51.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.060+0000 7f66f5ffb640 1 -- 172.21.15.53:0/2233086923 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66fc061390 con 0x7f67000c7520 2024-05-10T10:33:51.062 INFO:teuthology.orchestra.run.smithi053.stdout:55834574853 2024-05-10T10:33:51.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.061+0000 7f3ae77fe640 1 -- 172.21.15.53:0/1756795038 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3af4061570 con 0x7f3ae8004800 2024-05-10T10:33:51.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.061+0000 7f94b7fff640 1 -- 172.21.15.53:0/3611345401 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f94cc0dc680 con 0x7f94c40ead00 2024-05-10T10:33:51.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.061+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64e80762d0 msgr2=0x7f64e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.061+0000 7f6508aa8640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64e80762d0 0x7f64e8078790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005ed0 tx=0x7f64f0007d50 comp rx=0 tx=0).stop 2024-05-10T10:33:51.064 INFO:teuthology.orchestra.run.smithi053.stdout:55834574853 2024-05-10T10:33:51.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.061+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc054980 msgr2=0x7f64fc04eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.061+0000 7f6508aa8640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc054980 0x7f64fc04eaa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6504052040 tx=0x7f6504076270 comp rx=0 tx=0).stop 2024-05-10T10:33:51.064 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:51.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 shutdown_connections 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64e80762d0 0x7f64e8078790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64fc054980 0x7f64fc04eaa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64fc053e70 0x7f64fc0d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 --2- 172.21.15.53:0/860227813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64fc053a90 0x7f64fc0cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 >> 172.21.15.53:0/860227813 conn(0x7f64fc055c50 msgr2=0x7f64fc036e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 shutdown_connections 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f6508aa8640 1 -- 172.21.15.53:0/860227813 wait complete. 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.062+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7b880d8580 con 0x7f7b880078d0 2024-05-10T10:33:51.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.063+0000 7f7b7effd640 1 -- 172.21.15.53:0/233407549 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b84060630 con 0x7f7b880078d0 2024-05-10T10:33:51.067 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:51.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3aec076320 msgr2=0x7f3aec0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3aec076320 0x7f3aec0787e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3af0032000 tx=0x7f3af0009030 comp rx=0 tx=0).stop 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8004800 msgr2=0x7f3b000c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8004800 0x7f3b000c5fd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3af400f810 tx=0x7f3af400fcd0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 shutdown_connections 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3aec076320 0x7f3aec0787e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b0014a960 0x7f3b0014adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0014e970 0x7f3b000c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 --2- 172.21.15.53:0/1756795038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae8004800 0x7f3b000c5fd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 >> 172.21.15.53:0/1756795038 conn(0x7f3b000d1b80 msgr2=0x7f3b000d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94b0076300 msgr2=0x7f94b00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94b0076300 0x7f94b00787c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007fd0 tx=0x7f94c0009040 comp rx=0 tx=0).stop 2024-05-10T10:33:51.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.064+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c40ead00 msgr2=0x7f94c4189070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c40ead00 0x7f94c4189070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94cc06a840 tx=0x7f94cc069ed0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 shutdown_connections 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 shutdown_connections 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94b0076300 0x7f94b00787c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94c40ead00 0x7f94c4189070 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94c4006810 0x7f94c4035e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 --2- 172.21.15.53:0/3611345401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c4005e90 0x7f94c417c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 >> 172.21.15.53:0/3611345401 conn(0x7f94c4009fb0 msgr2=0x7f94c404ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 shutdown_connections 2024-05-10T10:33:51.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f94d0ac0640 1 -- 172.21.15.53:0/3611345401 wait complete. 2024-05-10T10:33:51.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f3b04a44640 1 -- 172.21.15.53:0/1756795038 wait complete. 2024-05-10T10:33:51.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66e40762d0 msgr2=0x7f66e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f6707719640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66e40762d0 0x7f66e4078790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66f0007fd0 tx=0x7f66f0009040 comp rx=0 tx=0).stop 2024-05-10T10:33:51.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67000c7520 msgr2=0x7f67000b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.065+0000 7f6707719640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67000c7520 0x7f67000b41e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00d5f0 tx=0x7f66fc00dab0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 shutdown_connections 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66e40762d0 0x7f66e4078790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67001634b0 0x7f67000b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67000cdd50 0x7f67000b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 --2- 172.21.15.53:0/2233086923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67000c7520 0x7f67000b41e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 >> 172.21.15.53:0/2233086923 conn(0x7f67000b2900 msgr2=0x7f670006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 shutdown_connections 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b700763a0 msgr2=0x7f7b70078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b700763a0 0x7f7b70078860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b880533e0 tx=0x7f7b80009040 comp rx=0 tx=0).stop 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b880078d0 msgr2=0x7f7b8804ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b880078d0 0x7f7b8804ea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c430 tx=0x7f7b8400c8f0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.066+0000 7f6707719640 1 -- 172.21.15.53:0/2233086923 wait complete. 2024-05-10T10:33:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 shutdown_connections 2024-05-10T10:33:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b700763a0 0x7f7b70078860 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b880e30b0 0x7f7b880de380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b8803e830 0x7f7b880dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 --2- 172.21.15.53:0/233407549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b880078d0 0x7f7b8804ea40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.067+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 >> 172.21.15.53:0/233407549 conn(0x7f7b880559d0 msgr2=0x7f7b880578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.070+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 shutdown_connections 2024-05-10T10:33:51.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.070+0000 7f7b949c1640 1 -- 172.21.15.53:0/233407549 wait complete. 2024-05-10T10:33:51.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.073+0000 7fdd1dc0b640 1 -- 172.21.15.53:0/1443479952 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdcf8005850 con 0x7fdd1004b3c0 2024-05-10T10:33:51.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.073+0000 7fdafbfff640 1 -- 172.21.15.53:0/776182723 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdafc0ce1c0 con 0x7fdafc0b20f0 2024-05-10T10:33:51.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.073+0000 7fdd157fa640 1 -- 172.21.15.53:0/1443479952 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdd04064d00 con 0x7fdd1004b3c0 2024-05-10T10:33:51.076 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:51.076 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:51.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.073+0000 7fdae37fe640 1 -- 172.21.15.53:0/776182723 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdaec061610 con 0x7fdafc0b20f0 2024-05-10T10:33:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.077+0000 7fdd02ffd640 1 -- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdcfc0766d0 msgr2=0x7fdcfc078b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.077+0000 7fdd02ffd640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdcfc0766d0 0x7fdcfc078b90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd100e6fd0 tx=0x7fdd0c02e000 comp rx=0 tx=0).stop 2024-05-10T10:33:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.077+0000 7fdae17fa640 1 -- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdae80763a0 msgr2=0x7fdae8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.077+0000 7fdd02ffd640 1 -- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004b3c0 msgr2=0x7fdd100dbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.077+0000 7fdae17fa640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdae80763a0 0x7fdae8078860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402a7a0 tx=0x7fdaf4009000 comp rx=0 tx=0).stop 2024-05-10T10:33:51.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.077+0000 7fdd02ffd640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004b3c0 0x7fdd100dbfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e880 tx=0x7fdd0400ed40 comp rx=0 tx=0).stop 2024-05-10T10:33:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 -- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdafc0b20f0 msgr2=0x7fdafc144310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdafc0b20f0 0x7fdafc144310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00d560 tx=0x7fdaec00da20 comp rx=0 tx=0).stop 2024-05-10T10:33:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 -- 172.21.15.53:0/776182723 shutdown_connections 2024-05-10T10:33:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdae80763a0 0x7fdae8078860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdafc0b20f0 0x7fdafc144310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdafc06e5b0 0x7fdafc159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 --2- 172.21.15.53:0/776182723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdafc06d9f0 0x7fdafc0b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 -- 172.21.15.53:0/776182723 >> 172.21.15.53:0/776182723 conn(0x7fdafc0cd460 msgr2=0x7fdafc15f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 -- 172.21.15.53:0/1443479952 shutdown_connections 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdcfc0766d0 0x7fdcfc078b90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd100ea760 0x7fdd1004df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1004fe90 0x7fdd100ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 --2- 172.21.15.53:0/1443479952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd1004b3c0 0x7fdd100dbfd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 -- 172.21.15.53:0/1443479952 >> 172.21.15.53:0/1443479952 conn(0x7fdd1000a400 msgr2=0x7fdd100e6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 -- 172.21.15.53:0/1443479952 shutdown_connections 2024-05-10T10:33:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdd02ffd640 1 -- 172.21.15.53:0/1443479952 wait complete. 2024-05-10T10:33:51.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 -- 172.21.15.53:0/776182723 shutdown_connections 2024-05-10T10:33:51.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:51.078+0000 7fdae17fa640 1 -- 172.21.15.53:0/776182723 wait complete. 2024-05-10T10:33:51.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-05-10T10:33:51.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-05-10T10:33:51.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-05-10T10:33:51.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-05-10T10:33:51.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-10T10:33:51.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-05-10T10:33:51.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-05-10T10:33:51.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-05-10T10:33:52.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-10T10:33:52.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-10T10:33:52.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-10T10:33:52.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-10T10:33:52.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-10T10:33:52.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-10T10:33:52.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-10T10:33:52.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-10T10:33:52.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.256+0000 7f68a4d2f640 1 Processor -- start 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.256+0000 7f68a4d2f640 1 -- start start 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.258+0000 7f68a4d2f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f6898051760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.258+0000 7f68a4d2f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68980548a0 0x7f68980f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.258+0000 7f68a4d2f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6898048510 con 0x7f689804fc00 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.258+0000 7f68a4d2f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f68980487b0 con 0x7f68980548a0 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.258+0000 7f68a4d2f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6898048a70 con 0x7f68980081c0 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.258+0000 7f689f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f689804fc00 0x7f689804ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36560/0 (socket says 172.21.15.53:36560) 2024-05-10T10:33:52.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.258+0000 7f689f7fe640 1 -- 172.21.15.53:0/2121956497 learned_addr learned my addr 172.21.15.53:0/2121956497 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:52.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.259+0000 7f689effd640 1 --2- 172.21.15.53:0/2121956497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f6898051760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.259+0000 7f689ffff640 1 --2- 172.21.15.53:0/2121956497 >> v2:172.21.15.53:3300/0 conn(0x7f68980548a0 0x7f68980f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.260+0000 7f689effd640 1 -- 172.21.15.53:0/2121956497 >> v2:172.21.15.53:3300/0 conn(0x7f68980548a0 msgr2=0x7f68980f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.260+0000 7f689effd640 1 --2- 172.21.15.53:0/2121956497 >> v2:172.21.15.53:3300/0 conn(0x7f68980548a0 0x7f68980f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.260+0000 7f689effd640 1 -- 172.21.15.53:0/2121956497 >> v1:172.21.15.135:6789/0 conn(0x7f689804fc00 legacy=0x7f689804ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.260+0000 7f689effd640 1 -- 172.21.15.53:0/2121956497 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6898053090 con 0x7f68980081c0 2024-05-10T10:33:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.261+0000 7f689ffff640 1 --2- 172.21.15.53:0/2121956497 >> v2:172.21.15.53:3300/0 conn(0x7f68980548a0 0x7f68980f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.261+0000 7f689effd640 1 --2- 172.21.15.53:0/2121956497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f6898051760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68940097a0 tx=0x7f689402a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a076d89f33cae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.262+0000 7f689e7fc640 1 -- 172.21.15.53:0/2121956497 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894037070 con 0x7f68980081c0 2024-05-10T10:33:52.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.262+0000 7f689e7fc640 1 -- 172.21.15.53:0/2121956497 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689402ad30 con 0x7f68980081c0 2024-05-10T10:33:52.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.263+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2121956497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 msgr2=0x7f6898051760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.263+0000 7f68a4d2f640 1 --2- 172.21.15.53:0/2121956497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f6898051760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68940097a0 tx=0x7f689402a0a0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.264+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2121956497 shutdown_connections 2024-05-10T10:33:52.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.264+0000 7f68a4d2f640 1 --2- 172.21.15.53:0/2121956497 >> v2:172.21.15.53:3300/0 conn(0x7f68980548a0 0x7f68980f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.264+0000 7f68a4d2f640 1 --2- 172.21.15.53:0/2121956497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f6898051760 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.264+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2121956497 >> 172.21.15.53:0/2121956497 conn(0x7f6898055340 msgr2=0x7f6898055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.264+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2121956497 shutdown_connections 2024-05-10T10:33:52.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.264+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2121956497 wait complete. 2024-05-10T10:33:52.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.266+0000 7f68a4d2f640 1 Processor -- start 2024-05-10T10:33:52.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.267+0000 7f68a4d2f640 1 -- start start 2024-05-10T10:33:52.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f68a4d2f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f68980e9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f68a4d2f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f689804fc00 0x7f68980d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f68a4d2f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980548a0 0x7f68980d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f68a4d2f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f68980f16a0 con 0x7f689804fc00 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f68a4d2f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68980506f0 con 0x7f68980548a0 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f68a4d2f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f68980f1820 con 0x7f68980081c0 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f689effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f689804fc00 0x7f68980d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.268+0000 7f689f7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f68980e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.269+0000 7f689f7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f68980e9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36356/0 (socket says 172.21.15.53:36356) 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.269+0000 7f689f7fe640 1 -- 172.21.15.53:0/2240847295 learned_addr learned my addr 172.21.15.53:0/2240847295 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.270+0000 7f689f7fe640 1 -- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980548a0 msgr2=0x7f68980d4fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.270+0000 7f689f7fe640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980548a0 0x7f68980d4fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.270+0000 7f689f7fe640 1 -- 172.21.15.53:0/2240847295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f689804fc00 msgr2=0x7f68980d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.270+0000 7f689f7fe640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f689804fc00 0x7f68980d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.270+0000 7f689f7fe640 1 -- 172.21.15.53:0/2240847295 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68980cddd0 con 0x7f68980081c0 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.270+0000 7f689effd640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f689804fc00 0x7f68980d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.271+0000 7f689f7fe640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f68980e9340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68a0063f70 tx=0x7f68a0075260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.271+0000 7f688bfff640 1 -- 172.21.15.53:0/2240847295 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0075390 con 0x7f68980081c0 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.271+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2240847295 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68980ce0b0 con 0x7f68980081c0 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.271+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2240847295 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f68980e9880 con 0x7f68980081c0 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.271+0000 7f688bfff640 1 -- 172.21.15.53:0/2240847295 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a0075d60 con 0x7f68980081c0 2024-05-10T10:33:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.271+0000 7f688bfff640 1 -- 172.21.15.53:0/2240847295 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0073460 con 0x7f68980081c0 2024-05-10T10:33:52.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.274+0000 7f688bfff640 1 -- 172.21.15.53:0/2240847295 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68a006d070 con 0x7f68980081c0 2024-05-10T10:33:52.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.275+0000 7f688bfff640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68840762d0 0x7f6884078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.275+0000 7f688bfff640 1 -- 172.21.15.53:0/2240847295 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f68a01144e0 con 0x7f68980081c0 2024-05-10T10:33:52.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.275+0000 7f689effd640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68840762d0 0x7f6884078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.277+0000 7f689effd640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68840762d0 0x7f6884078790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6894034000 tx=0x7f689402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.278+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2240847295 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c005290 con 0x7f68980081c0 2024-05-10T10:33:52.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.283+0000 7f688bfff640 1 -- 172.21.15.53:0/2240847295 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f68a00dd410 con 0x7f68980081c0 2024-05-10T10:33:52.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f577359e640 1 Processor -- start 2024-05-10T10:33:52.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.312+0000 7f755546d640 1 Processor -- start 2024-05-10T10:33:52.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.312+0000 7f755546d640 1 -- start start 2024-05-10T10:33:52.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f755546d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f75500ce880 0x7f75500d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f755546d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75500d0a30 con 0x7f755016f9c0 2024-05-10T10:33:52.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f755546d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f755006e8d0 con 0x7f755006fac0 2024-05-10T10:33:52.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f755546d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f75500ceff0 con 0x7f75500ce880 2024-05-10T10:33:52.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f577359e640 1 -- start start 2024-05-10T10:33:52.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f754effd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f75500ce880 0x7f75500d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7fe672ae4640 1 Processor -- start 2024-05-10T10:33:52.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f754effd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f75500ce880 0x7f75500d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51258/0 (socket says 172.21.15.53:51258) 2024-05-10T10:33:52.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7ff3d4854640 1 Processor -- start 2024-05-10T10:33:52.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7ff3d4854640 1 -- start start 2024-05-10T10:33:52.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.314+0000 7f754effd640 1 -- 172.21.15.53:0/4147873626 learned_addr learned my addr 172.21.15.53:0/4147873626 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7f754effd640 1 -- 172.21.15.53:0/4147873626 >> v1:172.21.15.135:6789/0 conn(0x7f755006fac0 legacy=0x7f75500add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7f754effd640 1 -- 172.21.15.53:0/4147873626 >> v1:172.21.15.53:6789/0 conn(0x7f755016f9c0 legacy=0x7f755006f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7f754effd640 1 -- 172.21.15.53:0/4147873626 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755006ebb0 con 0x7f75500ce880 2024-05-10T10:33:52.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7fda395ca640 1 Processor -- start 2024-05-10T10:33:52.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7fda395ca640 1 -- start start 2024-05-10T10:33:52.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3d4854640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3d015ce10 0x7ff3d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3d4854640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d01616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f577359e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f576c00ab30 0x7f576c009d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f577359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f576c0516d0 0x7f576c00a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f577359e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f576c054d00 con 0x7f576c0e1300 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f577359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f576c055910 con 0x7f576c0516d0 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f577359e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f576c054fe0 con 0x7f576c00ab30 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f577259c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f576c0e1300 0x7f576c03a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36582/0 (socket says 172.21.15.53:36582) 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3d4854640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d00cbb40 con 0x7ff3d00cd830 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3d4854640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d00cbe20 con 0x7ff3d015ce10 2024-05-10T10:33:52.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3d4854640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d00d0e60 con 0x7ff3d015d1f0 2024-05-10T10:33:52.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f754effd640 1 --2- 172.21.15.53:0/4147873626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75500ce880 0x7f75500d04f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7540009870 tx=0x7f754002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7d68dbdceb414c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3ce59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3d015ce10 0x7ff3d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f577259c640 1 -- 172.21.15.53:0/1131520482 learned_addr learned my addr 172.21.15.53:0/1131520482 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:52.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3ce59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3d015ce10 0x7ff3d006d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43572/0 (socket says 172.21.15.53:43572) 2024-05-10T10:33:52.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3ce59c640 1 -- 172.21.15.53:0/3551890384 learned_addr learned my addr 172.21.15.53:0/3551890384 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7ff3cdd9b640 1 --2- 172.21.15.53:0/3551890384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d01616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda395ca640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fda2c0065d0 0x7fda2c006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda395ca640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda2c006fa0 0x7fda2c0073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda33fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fda2c0061f0 0x7fda2c056fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39812/0 (socket says 172.21.15.53:39812) 2024-05-10T10:33:52.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda337fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fda2c0065d0 0x7fda2c006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda38dc9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda2c006fa0 0x7fda2c0073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda38dc9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda2c006fa0 0x7fda2c0073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43588/0 (socket says 172.21.15.53:43588) 2024-05-10T10:33:52.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda337fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fda2c0065d0 0x7fda2c006a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51290/0 (socket says 172.21.15.53:51290) 2024-05-10T10:33:52.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7ff3ce59c640 1 -- 172.21.15.53:0/3551890384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 msgr2=0x7ff3d01616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7fda33fff640 1 -- 172.21.15.53:0/283914563 learned_addr learned my addr 172.21.15.53:0/283914563 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:52.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7ff3ce59c640 1 --2- 172.21.15.53:0/3551890384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d01616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7ff3ce59c640 1 -- 172.21.15.53:0/3551890384 >> v1:172.21.15.53:6789/0 conn(0x7ff3d00cd830 legacy=0x7ff3d00727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7ff3ce59c640 1 -- 172.21.15.53:0/3551890384 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d00d1140 con 0x7ff3d015ce10 2024-05-10T10:33:52.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7f5771d9b640 1 --2- 172.21.15.53:0/1131520482 >> v2:172.21.15.135:3300/0 conn(0x7f576c00ab30 0x7f576c009d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7f5772d9d640 1 --2- 172.21.15.53:0/1131520482 >> v2:172.21.15.53:3300/0 conn(0x7f576c0516d0 0x7f576c00a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f5772d9d640 1 -- 172.21.15.53:0/1131520482 >> v2:172.21.15.135:3300/0 conn(0x7f576c00ab30 msgr2=0x7f576c009d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.315+0000 7fe672ae4640 1 -- start start 2024-05-10T10:33:52.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.316+0000 7fe672ae4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe66c163470 0x7fe66c06f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2c03d010 con 0x7fda2c0061f0 2024-05-10T10:33:52.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fda2c057a60 con 0x7fda2c006fa0 2024-05-10T10:33:52.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fda2c057d40 con 0x7fda2c0065d0 2024-05-10T10:33:52.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7f754dffb640 1 -- 172.21.15.53:0/4147873626 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540037070 con 0x7f75500ce880 2024-05-10T10:33:52.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe672ae4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe66c0b60e0 0x7fe66c163850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.317+0000 7f754dffb640 1 -- 172.21.15.53:0/4147873626 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7540032440 con 0x7f75500ce880 2024-05-10T10:33:52.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe672ae4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c06e070 0x7fe66c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f5772d9d640 1 --2- 172.21.15.53:0/1131520482 >> v2:172.21.15.135:3300/0 conn(0x7f576c00ab30 0x7f576c009d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f5772d9d640 1 -- 172.21.15.53:0/1131520482 >> v1:172.21.15.135:6789/0 conn(0x7f576c0e1300 legacy=0x7f576c03a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f5772d9d640 1 -- 172.21.15.53:0/1131520482 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c0552c0 con 0x7f576c0516d0 2024-05-10T10:33:52.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe671ae2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe66c163470 0x7fe66c06f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f755546d640 1 -- 172.21.15.53:0/4147873626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75500ce880 msgr2=0x7f75500d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe671ae2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe66c163470 0x7fe66c06f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51302/0 (socket says 172.21.15.53:51302) 2024-05-10T10:33:52.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe671ae2640 1 -- 172.21.15.53:0/2856953646 learned_addr learned my addr 172.21.15.53:0/2856953646 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe6712e1640 1 --2- 172.21.15.53:0/2856953646 >> v2:172.21.15.53:3300/0 conn(0x7fe66c0b60e0 0x7fe66c163850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f755546d640 1 --2- 172.21.15.53:0/4147873626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75500ce880 0x7f75500d04f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7540009870 tx=0x7f754002a270 comp rx=0 tx=0).stop 2024-05-10T10:33:52.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe6722e3640 1 --2- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c06e070 0x7fe66c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c0b6570 con 0x7fe66c0b60e0 2024-05-10T10:33:52.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c163d90 con 0x7fe66c163470 2024-05-10T10:33:52.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c06fc20 con 0x7fe66c06e070 2024-05-10T10:33:52.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7ff3ce59c640 1 --2- 172.21.15.53:0/3551890384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d015ce10 0x7ff3d006d4e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5537a5229b97c17b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f04ff69c640 1 Processor -- start 2024-05-10T10:33:52.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f755546d640 1 -- 172.21.15.53:0/4147873626 shutdown_connections 2024-05-10T10:33:52.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f755546d640 1 --2- 172.21.15.53:0/4147873626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75500ce880 0x7f75500d04f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f755546d640 1 -- 172.21.15.53:0/4147873626 >> 172.21.15.53:0/4147873626 conn(0x7f755006cd60 msgr2=0x7f75500b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f755546d640 1 -- 172.21.15.53:0/4147873626 shutdown_connections 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7f755546d640 1 -- 172.21.15.53:0/4147873626 wait complete. 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7f04ff69c640 1 -- start start 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7fe671ae2640 1 -- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c06e070 msgr2=0x7fe66c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7fe671ae2640 1 --2- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c06e070 0x7fe66c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7fe671ae2640 1 -- 172.21.15.53:0/2856953646 >> v2:172.21.15.53:3300/0 conn(0x7fe66c0b60e0 msgr2=0x7fe66c163850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7fe671ae2640 1 --2- 172.21.15.53:0/2856953646 >> v2:172.21.15.53:3300/0 conn(0x7fe66c0b60e0 0x7fe66c163850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7fe671ae2640 1 -- 172.21.15.53:0/2856953646 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c06ff00 con 0x7fe66c163470 2024-05-10T10:33:52.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda32ffd640 1 -- 172.21.15.53:0/283914563 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449815457 0 0) 0x7fda2c03d010 con 0x7fda2c0061f0 2024-05-10T10:33:52.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda32ffd640 1 -- 172.21.15.53:0/283914563 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda24003520 con 0x7fda2c0061f0 2024-05-10T10:33:52.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda38dc9640 1 -- 172.21.15.53:0/283914563 >> v2:172.21.15.135:3300/0 conn(0x7fda2c0065d0 msgr2=0x7fda2c006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda38dc9640 1 --2- 172.21.15.53:0/283914563 >> v2:172.21.15.135:3300/0 conn(0x7fda2c0065d0 0x7fda2c006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda38dc9640 1 -- 172.21.15.53:0/283914563 >> v1:172.21.15.53:6789/0 conn(0x7fda2c0061f0 legacy=0x7fda2c056fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.318+0000 7fda38dc9640 1 -- 172.21.15.53:0/283914563 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c058020 con 0x7fda2c006fa0 2024-05-10T10:33:52.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7ff3cd59a640 1 -- 172.21.15.53:0/3551890384 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0037070 con 0x7ff3d015ce10 2024-05-10T10:33:52.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7ff3cd59a640 1 -- 172.21.15.53:0/3551890384 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c0032440 con 0x7ff3d015ce10 2024-05-10T10:33:52.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7f5772d9d640 1 --2- 172.21.15.53:0/1131520482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0516d0 0x7f576c00a280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5774000b60 tx=0x7f57740a6460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffdb5df13685393 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.319+0000 7fda38dc9640 1 --2- 172.21.15.53:0/283914563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c006fa0 0x7fda2c0073f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda34000b60 tx=0x7fda340a6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e458c17aecad5945 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7ff3cd59a640 1 -- 172.21.15.53:0/3551890384 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0032810 con 0x7ff3d015ce10 2024-05-10T10:33:52.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7fe671ae2640 1 --2- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c163470 0x7fe66c06f5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0b3b00 tx=0x7fe66402a600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbfb81e40a2f0c83 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7fda32ffd640 1 -- 172.21.15.53:0/283914563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340b2070 con 0x7fda2c006fa0 2024-05-10T10:33:52.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7fda32ffd640 1 -- 172.21.15.53:0/283914563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda340ab7d0 con 0x7fda2c006fa0 2024-05-10T10:33:52.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7fda32ffd640 1 -- 172.21.15.53:0/283914563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340abbc0 con 0x7fda2c006fa0 2024-05-10T10:33:52.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f577159a640 1 -- 172.21.15.53:0/1131520482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57740b2070 con 0x7f576c0516d0 2024-05-10T10:33:52.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f577159a640 1 -- 172.21.15.53:0/1131520482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57740ab7d0 con 0x7f576c0516d0 2024-05-10T10:33:52.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f577159a640 1 -- 172.21.15.53:0/1131520482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57740abbc0 con 0x7f576c0516d0 2024-05-10T10:33:52.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f755546d640 1 Processor -- start 2024-05-10T10:33:52.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f755546d640 1 -- start start 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7ff3d4854640 1 -- 172.21.15.53:0/3551890384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d015ce10 msgr2=0x7ff3d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3551890384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d015ce10 0x7ff3d006d4e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c002a210 comp rx=0 tx=0).stop 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f04ff69c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04f806f5d0 0x7f04f8166e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f04ff69c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f80ccde0 con 0x7f04f8159a20 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f04ff69c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f80cd0c0 con 0x7f04f815c470 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7f04ff69c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f04f80ccba0 con 0x7f04f806f5d0 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7ff3d4854640 1 -- 172.21.15.53:0/3551890384 shutdown_connections 2024-05-10T10:33:52.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3551890384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d01616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3551890384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d015ce10 0x7ff3d006d4e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7ff3d4854640 1 -- 172.21.15.53:0/3551890384 >> 172.21.15.53:0/3551890384 conn(0x7ff3d00d14f0 msgr2=0x7ff3d00d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7fc27b009640 1 Processor -- start 2024-05-10T10:33:52.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.320+0000 7fc27b009640 1 -- start start 2024-05-10T10:33:52.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f04fdc39640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04f806f5d0 0x7f04f8166e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fe670ae0640 1 -- 172.21.15.53:0/2856953646 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664036070 con 0x7fe66c163470 2024-05-10T10:33:52.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fe670ae0640 1 -- 172.21.15.53:0/2856953646 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664031600 con 0x7fe66c163470 2024-05-10T10:33:52.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fe670ae0640 1 -- 172.21.15.53:0/2856953646 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6640319d0 con 0x7fe66c163470 2024-05-10T10:33:52.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f04fcc37640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f04f815c470 0x7f04f806eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36588/0 (socket says 172.21.15.53:36588) 2024-05-10T10:33:52.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f04fcc37640 1 -- 172.21.15.53:0/2495620712 learned_addr learned my addr 172.21.15.53:0/2495620712 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:52.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7ff3d4854640 1 -- 172.21.15.53:0/3551890384 shutdown_connections 2024-05-10T10:33:52.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c006fa0 msgr2=0x7fda2c0073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fda395ca640 1 --2- 172.21.15.53:0/283914563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c006fa0 0x7fda2c0073f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda34000b60 tx=0x7fda340a6580 comp rx=0 tx=0).stop 2024-05-10T10:33:52.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 shutdown_connections 2024-05-10T10:33:52.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f577359e640 1 -- 172.21.15.53:0/1131520482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0516d0 msgr2=0x7f576c00a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f577359e640 1 --2- 172.21.15.53:0/1131520482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0516d0 0x7f576c00a280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5774000b60 tx=0x7f57740a6460 comp rx=0 tx=0).stop 2024-05-10T10:33:52.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fda395ca640 1 --2- 172.21.15.53:0/283914563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c006fa0 0x7fda2c0073f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fda395ca640 1 --2- 172.21.15.53:0/283914563 >> v2:172.21.15.135:3300/0 conn(0x7fda2c0065d0 0x7fda2c006a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 >> 172.21.15.53:0/283914563 conn(0x7fda2c058ed0 msgr2=0x7fda2c0592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f577359e640 1 -- 172.21.15.53:0/1131520482 shutdown_connections 2024-05-10T10:33:52.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f577359e640 1 --2- 172.21.15.53:0/1131520482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0516d0 0x7f576c00a280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7ff3d4854640 1 -- 172.21.15.53:0/3551890384 wait complete. 2024-05-10T10:33:52.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f577359e640 1 --2- 172.21.15.53:0/1131520482 >> v2:172.21.15.135:3300/0 conn(0x7f576c00ab30 0x7f576c009d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f577359e640 1 -- 172.21.15.53:0/1131520482 >> 172.21.15.53:0/1131520482 conn(0x7f576c056cc0 msgr2=0x7f576c0570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f755546d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f755006fac0 0x7f7550073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f755546d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f755016f9c0 0x7f755015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 shutdown_connections 2024-05-10T10:33:52.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f755546d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7550161450 0x7f755009dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c163470 msgr2=0x7fe66c06f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f755546d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f75501706b0 con 0x7f755016f9c0 2024-05-10T10:33:52.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fe672ae4640 1 --2- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c163470 0x7fe66c06f5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0b3b00 tx=0x7fe66402a600 comp rx=0 tx=0).stop 2024-05-10T10:33:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3686544637 0 0) 0x7f04f80cd0c0 con 0x7f04f815c470 2024-05-10T10:33:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc27b009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2740cec70 0x7fc27415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc27b009640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2740cd840 con 0x7fc2740cda90 2024-05-10T10:33:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc27b009640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc27415ef90 con 0x7fc27415b550 2024-05-10T10:33:52.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc27b009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc27415f270 con 0x7fc2740cec70 2024-05-10T10:33:52.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 shutdown_connections 2024-05-10T10:33:52.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc278da5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc2740cda90 0x7fc2740cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39834/0 (socket says 172.21.15.53:39834) 2024-05-10T10:33:52.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc273fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc27415b550 0x7fc2740cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36602/0 (socket says 172.21.15.53:36602) 2024-05-10T10:33:52.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fda395ca640 1 -- 172.21.15.53:0/283914563 wait complete. 2024-05-10T10:33:52.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc278da5640 1 -- 172.21.15.53:0/1070722349 learned_addr learned my addr 172.21.15.53:0/1070722349 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:52.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fe672ae4640 1 --2- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c06e070 0x7fe66c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fe672ae4640 1 --2- 172.21.15.53:0/2856953646 >> v2:172.21.15.53:3300/0 conn(0x7fe66c0b60e0 0x7fe66c163850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04e4003520 con 0x7f04f815c470 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2038972557 0 0) 0x7f04f80ccde0 con 0x7f04f8159a20 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.321+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04f80cd0c0 con 0x7f04f8159a20 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 <== mon.1 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3200231511 0 0) 0x7f04e4003520 con 0x7f04f815c470 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 >> v2:172.21.15.53:3300/0 conn(0x7f04f806f5d0 msgr2=0x7f04f8166e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 --2- 172.21.15.53:0/2495620712 >> v2:172.21.15.53:3300/0 conn(0x7f04f806f5d0 0x7f04f8166e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 >> v1:172.21.15.53:6789/0 conn(0x7f04f8159a20 legacy=0x7f04f80cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f80cd3a0 con 0x7f04f815c470 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 <== mon.1 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f04e8001670 con 0x7f04f815c470 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 >> v1:172.21.15.135:6789/0 conn(0x7f04f815c470 legacy=0x7f04f806eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04e4004ba0 0x7f04e4004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e40037a0 0x7f04e4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004160 0x7f04e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80ccba0 con 0x7f04e40037a0 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40148a0 con 0x7f04e4004160 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f04e4014aa0 con 0x7f04e4004ba0 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fd438640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04e4004ba0 0x7f04e4004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7fe672ae4640 1 --2- 172.21.15.53:0/2856953646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c163470 0x7fe66c06f5b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f754f7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7550161450 0x7f755009dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7fc2795a6640 1 --2- 172.21.15.53:0/1070722349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2740cec70 0x7fc27415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fdc39640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004160 0x7f04e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f754f7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7550161450 0x7f755009dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36390/0 (socket says 172.21.15.53:36390) 2024-05-10T10:33:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f754f7fe640 1 -- 172.21.15.53:0/2714270981 learned_addr learned my addr 172.21.15.53:0/2714270981 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f755546d640 1 -- 172.21.15.53:0/2714270981 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f755016c240 con 0x7f755006fac0 2024-05-10T10:33:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f755546d640 1 -- 172.21.15.53:0/2714270981 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7550170830 con 0x7f7550161450 2024-05-10T10:33:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 >> 172.21.15.53:0/2856953646 conn(0x7fe66c0b2cf0 msgr2=0x7fe66c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7fc2737fe640 1 -- 172.21.15.53:0/1070722349 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667209972 0 0) 0x7fc27415ef90 con 0x7fc27415b550 2024-05-10T10:33:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7fc2737fe640 1 -- 172.21.15.53:0/1070722349 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc264003520 con 0x7fc27415b550 2024-05-10T10:33:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7fc2795a6640 1 -- 172.21.15.53:0/1070722349 >> v1:172.21.15.135:6789/0 conn(0x7fc27415b550 legacy=0x7fc2740cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7fc2795a6640 1 -- 172.21.15.53:0/1070722349 >> v1:172.21.15.53:6789/0 conn(0x7fc2740cda90 legacy=0x7fc2740cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7fc2795a6640 1 -- 172.21.15.53:0/1070722349 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27415f550 con 0x7fc2740cec70 2024-05-10T10:33:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fd438640 1 -- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004160 msgr2=0x7f04e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fd438640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004160 0x7f04e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7ff3d4854640 1 Processor -- start 2024-05-10T10:33:52.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fd438640 1 -- 172.21.15.53:0/2495620712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e40037a0 msgr2=0x7f04e4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fd438640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e40037a0 0x7f04e4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fd438640 1 -- 172.21.15.53:0/2495620712 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f80cd3a0 con 0x7f04e4004ba0 2024-05-10T10:33:52.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7ff3d4854640 1 -- start start 2024-05-10T10:33:52.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fdc39640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004160 0x7f04e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:52.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f04fdc39640 1 -- 172.21.15.53:0/2495620712 reap_dead start 2024-05-10T10:33:52.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f577359e640 1 -- 172.21.15.53:0/1131520482 shutdown_connections 2024-05-10T10:33:52.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.322+0000 7f577359e640 1 -- 172.21.15.53:0/1131520482 wait complete. 2024-05-10T10:33:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7fda395ca640 1 Processor -- start 2024-05-10T10:33:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f754effd640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f755006fac0 0x7f7550073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.323+0000 7f754e7fc640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f755016f9c0 0x7f755015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7f754e7fc640 1 -- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7550161450 msgr2=0x7f755009dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7f754e7fc640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7550161450 0x7f755009dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7f754e7fc640 1 -- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f755006fac0 msgr2=0x7f7550073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7f754e7fc640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f755006fac0 0x7f7550073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7f754e7fc640 1 -- 172.21.15.53:0/2714270981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7550160760 con 0x7f755016f9c0 2024-05-10T10:33:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7f577359e640 1 Processor -- start 2024-05-10T10:33:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 shutdown_connections 2024-05-10T10:33:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7fe672ae4640 1 -- 172.21.15.53:0/2856953646 wait complete. 2024-05-10T10:33:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7fc2795a6640 1 --2- 172.21.15.53:0/1070722349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2740cec70 0x7fc27415ba50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00bfd0 tx=0x7fc26c02c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a5033fca5c9dac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7f754e7fc640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f755016f9c0 0x7f755015fea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f754800bf90 tx=0x7f7548009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.324+0000 7fda395ca640 1 -- start start 2024-05-10T10:33:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7f04fd438640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04e4004ba0 0x7f04e4004f80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007e80 tx=0x7f04e4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cfdf9e8b84e92df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3d4854640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d00cd830 0x7ff3d00b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3d4854640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d015ce10 0x7ff3d01f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7f75377fe640 1 -- 172.21.15.53:0/2714270981 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548013600 con 0x7f755016f9c0 2024-05-10T10:33:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4007a50 con 0x7f04e4004ba0 2024-05-10T10:33:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4005350 con 0x7f04e4004ba0 2024-05-10T10:33:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7f04effff640 1 -- 172.21.15.53:0/2495620712 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4003830 con 0x7f04e4004ba0 2024-05-10T10:33:52.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3d4854640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d00b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3d4854640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0073070 con 0x7ff3d00cd830 2024-05-10T10:33:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3d4854640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0072ef0 con 0x7ff3d015ce10 2024-05-10T10:33:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3d4854640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d00cf3f0 con 0x7ff3d015d1f0 2024-05-10T10:33:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7f755546d640 1 -- 172.21.15.53:0/2714270981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7550072ee0 con 0x7f755016f9c0 2024-05-10T10:33:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d015ce10 0x7ff3d01f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3ced9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d00b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d015ce10 0x7ff3d01f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51316/0 (socket says 172.21.15.53:51316) 2024-05-10T10:33:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 -- 172.21.15.53:0/3205225975 learned_addr learned my addr 172.21.15.53:0/3205225975 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7fda395ca640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda2c0061f0 0x7fda2c17c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3ce59c640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d00cd830 0x7ff3d00b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 -- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 msgr2=0x7ff3d00b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f577359e640 1 -- start start 2024-05-10T10:33:52.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d00b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 -- 172.21.15.53:0/3205225975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d00cd830 msgr2=0x7ff3d00b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d00cd830 0x7ff3d00b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7ff3cdd9b640 1 -- 172.21.15.53:0/3205225975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d015d6c0 con 0x7ff3d015ce10 2024-05-10T10:33:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7ff3ce59c640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d00cd830 0x7ff3d00b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7ff3cdd9b640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d015ce10 0x7ff3d01f6420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400d5f0 tx=0x7ff3c400dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7fc2737fe640 1 -- 172.21.15.53:0/1070722349 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c02f070 con 0x7fc2740cec70 2024-05-10T10:33:52.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7fc2737fe640 1 -- 172.21.15.53:0/1070722349 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c033750 con 0x7fc2740cec70 2024-05-10T10:33:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.325+0000 7fc2737fe640 1 -- 172.21.15.53:0/1070722349 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c033b20 con 0x7fc2740cec70 2024-05-10T10:33:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7fc27b009640 1 -- 172.21.15.53:0/1070722349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2740cec70 msgr2=0x7fc27415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7fc27b009640 1 --2- 172.21.15.53:0/1070722349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2740cec70 0x7fc27415ba50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00bfd0 tx=0x7fc26c02c8e0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7fc27b009640 1 -- 172.21.15.53:0/1070722349 shutdown_connections 2024-05-10T10:33:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7fc27b009640 1 --2- 172.21.15.53:0/1070722349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2740cec70 0x7fc27415ba50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7fc27b009640 1 -- 172.21.15.53:0/1070722349 >> 172.21.15.53:0/1070722349 conn(0x7fc2740b2c30 msgr2=0x7fc2740b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f04ff69c640 1 -- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04e4004ba0 msgr2=0x7f04e4004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f04ff69c640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04e4004ba0 0x7f04e4004f80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007e80 tx=0x7f04e4005830 comp rx=0 tx=0).stop 2024-05-10T10:33:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7ff3beffd640 1 -- 172.21.15.53:0/3205225975 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40125f0 con 0x7ff3d015ce10 2024-05-10T10:33:52.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f04ff69c640 1 -- 172.21.15.53:0/2495620712 shutdown_connections 2024-05-10T10:33:52.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f04ff69c640 1 --2- 172.21.15.53:0/2495620712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04e4004ba0 0x7f04e4004f80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f04ff69c640 1 -- 172.21.15.53:0/2495620712 >> 172.21.15.53:0/2495620712 conn(0x7f04f80d14f0 msgr2=0x7f04f80d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7ff3beffd640 1 -- 172.21.15.53:0/3205225975 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c4019070 con 0x7ff3d015ce10 2024-05-10T10:33:52.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7ff3beffd640 1 -- 172.21.15.53:0/3205225975 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40129d0 con 0x7ff3d015ce10 2024-05-10T10:33:52.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f04ff69c640 1 -- 172.21.15.53:0/2495620712 shutdown_connections 2024-05-10T10:33:52.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01f5b20 con 0x7ff3d015ce10 2024-05-10T10:33:52.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f75377fe640 1 -- 172.21.15.53:0/2714270981 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75480137a0 con 0x7f755016f9c0 2024-05-10T10:33:52.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f04ff69c640 1 -- 172.21.15.53:0/2495620712 wait complete. 2024-05-10T10:33:52.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01f60e0 con 0x7ff3d015ce10 2024-05-10T10:33:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.326+0000 7f755546d640 1 -- 172.21.15.53:0/2714270981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75500734a0 con 0x7f755016f9c0 2024-05-10T10:33:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7fe672ae4640 1 Processor -- start 2024-05-10T10:33:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7fe672ae4640 1 -- start start 2024-05-10T10:33:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7f577359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c00ab30 0x7f576c04b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7f577359e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f576c0516d0 0x7f576c04a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7f577359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0e1300 0x7f576c0501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7f577359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f576c054a10 con 0x7f576c0e1300 2024-05-10T10:33:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7f577359e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f576c054890 con 0x7f576c00ab30 2024-05-10T10:33:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7f577359e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f576c055da0 con 0x7f576c0516d0 2024-05-10T10:33:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7fc27b009640 1 -- 172.21.15.53:0/1070722349 shutdown_connections 2024-05-10T10:33:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7fda395ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c0065d0 0x7fda2c03a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.327+0000 7fda33fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda2c0061f0 0x7fda2c17c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda33fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda2c0061f0 0x7fda2c17c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36422/0 (socket says 172.21.15.53:36422) 2024-05-10T10:33:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda33fff640 1 -- 172.21.15.53:0/586268510 learned_addr learned my addr 172.21.15.53:0/586268510 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda395ca640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda2c006fa0 0x7fda2c00c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda337fe640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c0065d0 0x7fda2c03a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c0e3510 con 0x7fda2c0065d0 2024-05-10T10:33:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7f577259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c00ab30 0x7f576c04b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7f577259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c00ab30 0x7f576c04b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51328/0 (socket says 172.21.15.53:51328) 2024-05-10T10:33:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7f577259c640 1 -- 172.21.15.53:0/3851328938 learned_addr learned my addr 172.21.15.53:0/3851328938 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c0585e0 con 0x7fda2c006fa0 2024-05-10T10:33:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c03edc0 con 0x7fda2c0061f0 2024-05-10T10:33:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda38dc9640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda2c006fa0 0x7fda2c00c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7f75377fe640 1 -- 172.21.15.53:0/2714270981 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548013b90 con 0x7f755016f9c0 2024-05-10T10:33:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7f5772d9d640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0e1300 0x7f576c0501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7f04ff69c640 1 Processor -- start 2024-05-10T10:33:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7f577259c640 1 -- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f576c0516d0 msgr2=0x7f576c04a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7f577259c640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f576c0516d0 0x7f576c04a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda33fff640 1 -- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda2c006fa0 msgr2=0x7fda2c00c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda33fff640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda2c006fa0 0x7fda2c00c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda33fff640 1 -- 172.21.15.53:0/586268510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c0065d0 msgr2=0x7fda2c03a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fda33fff640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c0065d0 0x7fda2c03a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7fda33fff640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c03b170 con 0x7fda2c0061f0 2024-05-10T10:33:52.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7f577259c640 1 -- 172.21.15.53:0/3851328938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0e1300 msgr2=0x7f576c0501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7f577259c640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0e1300 0x7f576c0501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7f577259c640 1 -- 172.21.15.53:0/3851328938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c17db00 con 0x7f576c00ab30 2024-05-10T10:33:52.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7ff3beffd640 1 -- 172.21.15.53:0/3205225975 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff3c4012b70 con 0x7ff3d015ce10 2024-05-10T10:33:52.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7fe672ae4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c06e070 0x7fe66c1f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7f577259c640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c00ab30 0x7f576c04b030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f576400b430 tx=0x7f576400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7fe672ae4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c0b60e0 0x7fe66c165050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fe671ae2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c06e070 0x7fe66c1f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7fda33fff640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda2c0061f0 0x7fda2c17c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fda2800c530 tx=0x7fda2800c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7fda30ff9640 1 -- 172.21.15.53:0/586268510 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28017070 con 0x7fda2c0061f0 2024-05-10T10:33:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fda30ff9640 1 -- 172.21.15.53:0/586268510 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda28005ce0 con 0x7fda2c0061f0 2024-05-10T10:33:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fda30ff9640 1 -- 172.21.15.53:0/586268510 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda280072b0 con 0x7fda2c0061f0 2024-05-10T10:33:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c03b450 con 0x7fda2c0061f0 2024-05-10T10:33:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7ff3beffd640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3b00763a0 0x7ff3b0078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fe671ae2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c06e070 0x7fe66c1f9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51338/0 (socket says 172.21.15.53:51338) 2024-05-10T10:33:52.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fe671ae2640 1 -- 172.21.15.53:0/3540068753 learned_addr learned my addr 172.21.15.53:0/3540068753 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d00693f0 con 0x7ff3d015ce10 2024-05-10T10:33:52.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7ff3beffd640 1 -- 172.21.15.53:0/3205225975 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff3c4097cd0 con 0x7ff3d015ce10 2024-05-10T10:33:52.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.329+0000 7f04ff69c640 1 -- start start 2024-05-10T10:33:52.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04ff69c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004ba0 0x7f04f8147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04ff69c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04f80cfb40 0x7f04f80cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04ff69c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04f81e6230 0x7f04f81e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04ff69c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8159e00 con 0x7f04f80cfb40 2024-05-10T10:33:52.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f5762ffd640 1 -- 172.21.15.53:0/3851328938 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764015020 con 0x7f576c00ab30 2024-05-10T10:33:52.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04ff69c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80cd960 con 0x7f04e4004ba0 2024-05-10T10:33:52.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04ff69c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f04f80cc490 con 0x7f04f81e6230 2024-05-10T10:33:52.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7ff3ce59c640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3b00763a0 0x7ff3b0078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f75377fe640 1 -- 172.21.15.53:0/2714270981 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f754801c400 con 0x7f755016f9c0 2024-05-10T10:33:52.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c17dde0 con 0x7f576c00ab30 2024-05-10T10:33:52.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c17e8a0 con 0x7f576c00ab30 2024-05-10T10:33:52.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f5762ffd640 1 -- 172.21.15.53:0/3851328938 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5764011830 con 0x7f576c00ab30 2024-05-10T10:33:52.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c00bf50 con 0x7fda2c0061f0 2024-05-10T10:33:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f5762ffd640 1 -- 172.21.15.53:0/3851328938 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764011c20 con 0x7f576c00ab30 2024-05-10T10:33:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.328+0000 7fc27b009640 1 -- 172.21.15.53:0/1070722349 wait complete. 2024-05-10T10:33:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fc27b009640 1 Processor -- start 2024-05-10T10:33:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7fc27b009640 1 -- start start 2024-05-10T10:33:52.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7528005290 con 0x7f755016f9c0 2024-05-10T10:33:52.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fe672ae4640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe66c163470 0x7fe66c165590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0724c0 con 0x7fe66c163470 2024-05-10T10:33:52.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0704c0 con 0x7fe66c06e070 2024-05-10T10:33:52.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0b4bb0 con 0x7fe66c0b60e0 2024-05-10T10:33:52.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fe6712e1640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c0b60e0 0x7fe66c165050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fe6722e3640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe66c163470 0x7fe66c165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fe671ae2640 1 -- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c0b60e0 msgr2=0x7fe66c165050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fe671ae2640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c0b60e0 0x7fe66c165050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fe671ae2640 1 -- 172.21.15.53:0/3540068753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe66c163470 msgr2=0x7fe66c165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7ff3ce59c640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3b00763a0 0x7ff3b0078860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0034000 tx=0x7ff3c002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7f75377fe640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f753c0763a0 0x7f753c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fc27b009640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2740cda90 0x7fc27414fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fc27b009640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2740cec70 0x7fc27415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fc27b009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27415b550 0x7fc27415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc273fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2740cec70 0x7fc27415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc273fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2740cec70 0x7fc27415d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51358/0 (socket says 172.21.15.53:51358) 2024-05-10T10:33:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc278da5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2740cda90 0x7fc27414fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc278da5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2740cda90 0x7fc27414fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43692/0 (socket says 172.21.15.53:43692) 2024-05-10T10:33:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04fd438640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004ba0 0x7f04f8147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04fd438640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004ba0 0x7f04f8147ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51352/0 (socket says 172.21.15.53:51352) 2024-05-10T10:33:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.330+0000 7f04fd438640 1 -- 172.21.15.53:0/3936935383 learned_addr learned my addr 172.21.15.53:0/3936935383 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7f04fd438640 1 -- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04f81e6230 msgr2=0x7f04f81e66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7f04fd438640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04f81e6230 0x7f04f81e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7f04fd438640 1 -- 172.21.15.53:0/3936935383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04f80cfb40 msgr2=0x7f04f80cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7f04fd438640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04f80cfb40 0x7f04f80cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7f04fd438640 1 -- 172.21.15.53:0/3936935383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81e6bf0 con 0x7f04e4004ba0 2024-05-10T10:33:52.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7f04fd438640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004ba0 0x7f04f8147ef0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007fa0 tx=0x7f04f4001710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.331+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda14005290 con 0x7fda2c0061f0 2024-05-10T10:33:52.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7f75377fe640 1 -- 172.21.15.53:0/2714270981 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7548098450 con 0x7f755016f9c0 2024-05-10T10:33:52.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7f754effd640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f753c0763a0 0x7f753c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc27b009640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc2740cbf80 con 0x7fc2740cda90 2024-05-10T10:33:52.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc27b009640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2740cbe00 con 0x7fc2740cec70 2024-05-10T10:33:52.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc27b009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc27406a220 con 0x7fc27415b550 2024-05-10T10:33:52.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc2795a6640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27415b550 0x7fc27415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7fc273fff640 1 -- 172.21.15.53:0/1973737848 learned_addr learned my addr 172.21.15.53:0/1973737848 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.332+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5758005290 con 0x7f576c00ab30 2024-05-10T10:33:52.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fda30ff9640 1 -- 172.21.15.53:0/586268510 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fda2801a070 con 0x7fda2c0061f0 2024-05-10T10:33:52.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fc273fff640 1 -- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27415b550 msgr2=0x7fc27415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fc273fff640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27415b550 0x7fc27415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fc273fff640 1 -- 172.21.15.53:0/1973737848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2740cda90 msgr2=0x7fc27414fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fc273fff640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2740cda90 0x7fc27414fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fc273fff640 1 -- 172.21.15.53:0/1973737848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27415c7f0 con 0x7fc2740cec70 2024-05-10T10:33:52.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fe671ae2640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe66c163470 0x7fe66c165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fe671ae2640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c1fa020 con 0x7fe66c06e070 2024-05-10T10:33:52.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fe6722e3640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe66c163470 0x7fe66c165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:52.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fe671ae2640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c06e070 0x7fe66c1f9ae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe664000f80 tx=0x7fe664004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7f5762ffd640 1 -- 172.21.15.53:0/3851328938 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5764018020 con 0x7f576c00ab30 2024-05-10T10:33:52.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7f754effd640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f753c0763a0 0x7f753c078860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7540034000 tx=0x7f754002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fc273fff640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2740cec70 0x7fc27415d1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c4e0 tx=0x7fc26000c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fc2795a6640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27415b550 0x7fc27415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:52.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.333+0000 7fda30ff9640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda18076320 0x7fda180787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.334+0000 7f5762ffd640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57540763a0 0x7f5754078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.334+0000 7fda337fe640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda18076320 0x7fda180787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.405 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.334+0000 7fda30ff9640 1 -- 172.21.15.53:0/586268510 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fda28096c50 con 0x7fda2c0061f0 2024-05-10T10:33:52.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.334+0000 7f5762ffd640 1 -- 172.21.15.53:0/3851328938 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f57640977f0 con 0x7f576c00ab30 2024-05-10T10:33:52.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.334+0000 7f5771d9b640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57540763a0 0x7f5754078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.334+0000 7fda337fe640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda18076320 0x7fda180787e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0e21d0 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.334+0000 7f5771d9b640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57540763a0 0x7f5754078860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f576c050c40 tx=0x7f5768008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7fc2717fa640 1 -- 172.21.15.53:0/1973737848 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260011870 con 0x7fc2740cec70 2024-05-10T10:33:52.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7f04edffb640 1 -- 172.21.15.53:0/3936935383 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f403f070 con 0x7f04e4004ba0 2024-05-10T10:33:52.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f81e4760 con 0x7f04e4004ba0 2024-05-10T10:33:52.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27415cad0 con 0x7fc2740cec70 2024-05-10T10:33:52.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2740b3dc0 con 0x7fc2740cec70 2024-05-10T10:33:52.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7ff3beffd640 1 -- 172.21.15.53:0/3205225975 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff3c40612d0 con 0x7ff3d015ce10 2024-05-10T10:33:52.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f81e4d20 con 0x7f04e4004ba0 2024-05-10T10:33:52.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.337+0000 7f75377fe640 1 -- 172.21.15.53:0/2714270981 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7548061400 con 0x7f755016f9c0 2024-05-10T10:33:52.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.338+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f80693f0 con 0x7f04e4004ba0 2024-05-10T10:33:52.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.338+0000 7fda30ff9640 1 -- 172.21.15.53:0/586268510 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fda2805fd50 con 0x7fda2c0061f0 2024-05-10T10:33:52.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.338+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2740693f0 con 0x7fc2740cec70 2024-05-10T10:33:52.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.341+0000 7f04edffb640 1 -- 172.21.15.53:0/3936935383 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4005350 con 0x7f04e4004ba0 2024-05-10T10:33:52.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.341+0000 7fe6627fc640 1 -- 172.21.15.53:0/3540068753 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664041020 con 0x7fe66c06e070 2024-05-10T10:33:52.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.341+0000 7fe6627fc640 1 -- 172.21.15.53:0/3540068753 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664037040 con 0x7fe66c06e070 2024-05-10T10:33:52.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.341+0000 7fe6627fc640 1 -- 172.21.15.53:0/3540068753 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664036070 con 0x7fe66c06e070 2024-05-10T10:33:52.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.341+0000 7f5762ffd640 1 -- 172.21.15.53:0/3851328938 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f576401f020 con 0x7f576c00ab30 2024-05-10T10:33:52.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.342+0000 7f04edffb640 1 -- 172.21.15.53:0/3936935383 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40035f0 con 0x7f04e4004ba0 2024-05-10T10:33:52.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.342+0000 7fc2717fa640 1 -- 172.21.15.53:0/1973737848 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc260011a10 con 0x7fc2740cec70 2024-05-10T10:33:52.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.342+0000 7fc2717fa640 1 -- 172.21.15.53:0/1973737848 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26001e060 con 0x7fc2740cec70 2024-05-10T10:33:52.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.342+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c15fa40 con 0x7fe66c06e070 2024-05-10T10:33:52.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.342+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c160000 con 0x7fe66c06e070 2024-05-10T10:33:52.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.344+0000 7fc2717fa640 1 -- 172.21.15.53:0/1973737848 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc260023480 con 0x7fc2740cec70 2024-05-10T10:33:52.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.344+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c0693f0 con 0x7fe66c06e070 2024-05-10T10:33:52.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.345+0000 7fc2717fa640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2580762d0 0x7fc258078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.345+0000 7fc2717fa640 1 -- 172.21.15.53:0/1973737848 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc260097ff0 con 0x7fc2740cec70 2024-05-10T10:33:52.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.345+0000 7f04edffb640 1 -- 172.21.15.53:0/3936935383 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f04f4042070 con 0x7f04e4004ba0 2024-05-10T10:33:52.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.345+0000 7fc278da5640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2580762d0 0x7fc258078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.346+0000 7fc278da5640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2580762d0 0x7fc258078790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc268000f60 tx=0x7fc268007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.346+0000 7f04edffb640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04dc0762d0 0x7f04dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.346+0000 7f04edffb640 1 -- 172.21.15.53:0/3936935383 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f04f40b7d20 con 0x7f04e4004ba0 2024-05-10T10:33:52.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.346+0000 7f04fcc37640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04dc0762d0 0x7f04dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.347+0000 7f04fcc37640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04dc0762d0 0x7f04dc078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f04f80b26d0 tx=0x7f04e80015a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.347+0000 7f04edffb640 1 -- 172.21.15.53:0/3936935383 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f04f4049070 con 0x7f04e4004ba0 2024-05-10T10:33:52.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.351+0000 7fc2717fa640 1 -- 172.21.15.53:0/1973737848 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc260015020 con 0x7fc2740cec70 2024-05-10T10:33:52.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.354+0000 7fe6627fc640 1 -- 172.21.15.53:0/3540068753 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe664044020 con 0x7fe66c06e070 2024-05-10T10:33:52.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.354+0000 7fe6627fc640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe6500763a0 0x7fe650078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.355+0000 7fe6627fc640 1 -- 172.21.15.53:0/3540068753 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe6640b87d0 con 0x7fe66c06e070 2024-05-10T10:33:52.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.355+0000 7fe6712e1640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe6500763a0 0x7fe650078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.356+0000 7fe6712e1640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe6500763a0 0x7fe650078860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe66c165dd0 tx=0x7fe668008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.356+0000 7fe6627fc640 1 -- 172.21.15.53:0/3540068753 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe66404b030 con 0x7fe66c06e070 2024-05-10T10:33:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.711+0000 7f68a4d2f640 1 -- 172.21.15.53:0/2240847295 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f687c005580 con 0x7f68980081c0 2024-05-10T10:33:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.712+0000 7f688bfff640 1 -- 172.21.15.53:0/2240847295 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f68a00e1710 con 0x7f68980081c0 2024-05-10T10:33:52.714 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:52.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.715+0000 7f6889ffb640 1 -- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68840762d0 msgr2=0x7f6884078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.715+0000 7f6889ffb640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68840762d0 0x7f6884078790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6894034000 tx=0x7f689402e000 comp rx=0 tx=0).stop 2024-05-10T10:33:52.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.715+0000 7f6889ffb640 1 -- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 msgr2=0x7f68980e9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.715+0000 7f6889ffb640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f68980e9340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68a0063f70 tx=0x7f68a0075260 comp rx=0 tx=0).stop 2024-05-10T10:33:52.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 -- 172.21.15.53:0/2240847295 shutdown_connections 2024-05-10T10:33:52.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68840762d0 0x7f6884078790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980548a0 0x7f68980d4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f689804fc00 0x7f68980d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 --2- 172.21.15.53:0/2240847295 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980081c0 0x7f68980e9340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 -- 172.21.15.53:0/2240847295 >> 172.21.15.53:0/2240847295 conn(0x7f6898055340 msgr2=0x7f6898004fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 -- 172.21.15.53:0/2240847295 shutdown_connections 2024-05-10T10:33:52.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.716+0000 7f6889ffb640 1 -- 172.21.15.53:0/2240847295 wait complete. 2024-05-10T10:33:52.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-05-10T10:33:52.736 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.759+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe66c0740a0 con 0x7fe66c06e070 2024-05-10T10:33:52.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.759+0000 7fe6627fc640 1 -- 172.21.15.53:0/3540068753 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe66402ae20 con 0x7fe66c06e070 2024-05-10T10:33:52.761 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe6500763a0 msgr2=0x7fe650078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe6500763a0 0x7fe650078860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe66c165dd0 tx=0x7fe668008040 comp rx=0 tx=0).stop 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c06e070 msgr2=0x7fe66c1f9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c06e070 0x7fe66c1f9ae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe664000f80 tx=0x7fe664004a40 comp rx=0 tx=0).stop 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 shutdown_connections 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe6500763a0 0x7fe650078860 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe66c163470 0x7fe66c165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe66c0b60e0 0x7fe66c165050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 --2- 172.21.15.53:0/3540068753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c06e070 0x7fe66c1f9ae0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 >> 172.21.15.53:0/3540068753 conn(0x7fe66c0b2cf0 msgr2=0x7fe66c0b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 shutdown_connections 2024-05-10T10:33:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.762+0000 7fe672ae4640 1 -- 172.21.15.53:0/3540068753 wait complete. 2024-05-10T10:33:52.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-05-10T10:33:52.775 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.778+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff3d00cb370 con 0x7ff3d015ce10 2024-05-10T10:33:52.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.778+0000 7ff3beffd640 1 -- 172.21.15.53:0/3205225975 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3c4060bd0 con 0x7ff3d015ce10 2024-05-10T10:33:52.780 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:52.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.780+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3b00763a0 msgr2=0x7ff3b0078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.780+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3b00763a0 0x7ff3b0078860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0034000 tx=0x7ff3c002e000 comp rx=0 tx=0).stop 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.780+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d015ce10 msgr2=0x7ff3d01f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.780+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d015ce10 0x7ff3d01f6420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400d5f0 tx=0x7ff3c400dab0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 shutdown_connections 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3b00763a0 0x7ff3b0078860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3d015d1f0 0x7ff3d00b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d015ce10 0x7ff3d01f6420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 --2- 172.21.15.53:0/3205225975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d00cd830 0x7ff3d00b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 >> 172.21.15.53:0/3205225975 conn(0x7ff3d00d14f0 msgr2=0x7ff3d00d18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 shutdown_connections 2024-05-10T10:33:52.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.781+0000 7ff3d4854640 1 -- 172.21.15.53:0/3205225975 wait complete. 2024-05-10T10:33:52.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.783+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fda14005580 con 0x7fda2c0061f0 2024-05-10T10:33:52.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.783+0000 7fda30ff9640 1 -- 172.21.15.53:0/586268510 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fda28064050 con 0x7fda2c0061f0 2024-05-10T10:33:52.785 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:52.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.786+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda18076320 msgr2=0x7fda180787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.786+0000 7fda395ca640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda18076320 0x7fda180787e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0e21d0 tx=0x7fda20008040 comp rx=0 tx=0).stop 2024-05-10T10:33:52.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda2c0061f0 msgr2=0x7fda2c17c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda2c0061f0 0x7fda2c17c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fda2800c530 tx=0x7fda2800c9f0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7528005580 con 0x7f755016f9c0 2024-05-10T10:33:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 shutdown_connections 2024-05-10T10:33:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda18076320 0x7fda180787e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda2c006fa0 0x7fda2c00c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda2c0065d0 0x7fda2c03a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 --2- 172.21.15.53:0/586268510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda2c0061f0 0x7fda2c17c2b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 >> 172.21.15.53:0/586268510 conn(0x7fda2c058ed0 msgr2=0x7fda2c056d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 shutdown_connections 2024-05-10T10:33:52.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7fda395ca640 1 -- 172.21.15.53:0/586268510 wait complete. 2024-05-10T10:33:52.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.787+0000 7f75377fe640 1 -- 172.21.15.53:0/2714270981 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7548065700 con 0x7f755016f9c0 2024-05-10T10:33:52.791 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:52.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.789+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f04f80b0810 con 0x7f04e4004ba0 2024-05-10T10:33:52.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.789+0000 7f04edffb640 1 -- 172.21.15.53:0/3936935383 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f04f4081270 con 0x7f04e4004ba0 2024-05-10T10:33:52.793 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:52.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.790+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f753c0763a0 msgr2=0x7f753c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f753c0763a0 0x7f753c078860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7540034000 tx=0x7f754002e000 comp rx=0 tx=0).stop 2024-05-10T10:33:52.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f755016f9c0 msgr2=0x7f755015fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f755016f9c0 0x7f755015fea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f754800bf90 tx=0x7f7548009f10 comp rx=0 tx=0).stop 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 shutdown_connections 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f753c0763a0 0x7f753c078860 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7550161450 0x7f755009dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f755016f9c0 0x7f755015fea0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 --2- 172.21.15.53:0/2714270981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f755006fac0 0x7f7550073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 >> 172.21.15.53:0/2714270981 conn(0x7f755006cd60 msgr2=0x7f75500af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 shutdown_connections 2024-05-10T10:33:52.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.791+0000 7f75357fa640 1 -- 172.21.15.53:0/2714270981 wait complete. 2024-05-10T10:33:52.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04dc0762d0 msgr2=0x7f04dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04dc0762d0 0x7f04dc078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f04f80b26d0 tx=0x7f04e80015a0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004ba0 msgr2=0x7f04f8147ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004ba0 0x7f04f8147ef0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007fa0 tx=0x7f04f4001710 comp rx=0 tx=0).stop 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 shutdown_connections 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04dc0762d0 0x7f04dc078790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04f81e6230 0x7f04f81e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04f80cfb40 0x7f04f80cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 --2- 172.21.15.53:0/3936935383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04e4004ba0 0x7f04f8147ef0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 >> 172.21.15.53:0/3936935383 conn(0x7f04f80d14f0 msgr2=0x7f04f806edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5758005580 con 0x7f576c00ab30 2024-05-10T10:33:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 shutdown_connections 2024-05-10T10:33:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.793+0000 7f04ff69c640 1 -- 172.21.15.53:0/3936935383 wait complete. 2024-05-10T10:33:52.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.794+0000 7f5762ffd640 1 -- 172.21.15.53:0/3851328938 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5764067e20 con 0x7f576c00ab30 2024-05-10T10:33:52.800 INFO:teuthology.orchestra.run.smithi053.stdout:55834574854 2024-05-10T10:33:52.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-05-10T10:33:52.801 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57540763a0 msgr2=0x7f5754078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57540763a0 0x7f5754078860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f576c050c40 tx=0x7f5768008040 comp rx=0 tx=0).stop 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c00ab30 msgr2=0x7f576c04b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c00ab30 0x7f576c04b030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f576400b430 tx=0x7f576400b8f0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 shutdown_connections 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57540763a0 0x7f5754078860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f576c0e1300 0x7f576c0501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f576c0516d0 0x7f576c04a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 --2- 172.21.15.53:0/3851328938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c00ab30 0x7f576c04b030 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 >> 172.21.15.53:0/3851328938 conn(0x7f576c056cc0 msgr2=0x7f576c009d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 shutdown_connections 2024-05-10T10:33:52.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.797+0000 7f577359e640 1 -- 172.21.15.53:0/3851328938 wait complete. 2024-05-10T10:33:52.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.798+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc2740d0970 con 0x7fc2740cec70 2024-05-10T10:33:52.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.799+0000 7fc2717fa640 1 -- 172.21.15.53:0/1973737848 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc260060fa0 con 0x7fc2740cec70 2024-05-10T10:33:52.803 INFO:teuthology.orchestra.run.smithi053.stdout:55834574855 2024-05-10T10:33:52.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2580762d0 msgr2=0x7fc258078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2580762d0 0x7fc258078790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc268000f60 tx=0x7fc268007e80 comp rx=0 tx=0).stop 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2740cec70 msgr2=0x7fc27415d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2740cec70 0x7fc27415d1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c4e0 tx=0x7fc26000c9a0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 shutdown_connections 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2580762d0 0x7fc258078790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27415b550 0x7fc27415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2740cec70 0x7fc27415d1b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 --2- 172.21.15.53:0/1973737848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2740cda90 0x7fc27414fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 >> 172.21.15.53:0/1973737848 conn(0x7fc2740b2c30 msgr2=0x7fc2740b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 shutdown_connections 2024-05-10T10:33:52.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.802+0000 7fc27b009640 1 -- 172.21.15.53:0/1973737848 wait complete. 2024-05-10T10:33:52.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-05-10T10:33:52.809 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-10T10:33:52.811 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-10T10:33:52.812 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-10T10:33:52.818 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-05-10T10:33:52.819 DEBUG:teuthology.parallel:result is None 2024-05-10T10:33:52.819 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:33:52.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:33:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.961+0000 7f945d189640 1 Processor -- start 2024-05-10T10:33:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.961+0000 7f945d189640 1 -- start start 2024-05-10T10:33:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.962+0000 7f945d189640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94580b8970 0x7f945806ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.962+0000 7f945d189640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94580d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.962+0000 7f945d189640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94580cd4b0 con 0x7f94580d2260 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.962+0000 7f945d189640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f945806e760 con 0x7f94580b8970 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.962+0000 7f945d189640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f94580cf210 con 0x7f945806e300 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f9456d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94580b8970 0x7f945806ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f945659c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94580d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f9456d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94580b8970 0x7f945806ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:45718/0 (socket says 172.21.15.53:45718) 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f9456d9d640 1 -- 172.21.15.53:0/1195164242 learned_addr learned my addr 172.21.15.53:0/1195164242 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f945759e640 1 --1- 172.21.15.53:0/1195164242 >> v1:172.21.15.135:6789/0 conn(0x7f94580d2260 0x7f94580cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:34746/0 (socket says 172.21.15.53:34746) 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f9456d9d640 1 -- 172.21.15.53:0/1195164242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 msgr2=0x7f94580d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f9456d9d640 1 --2- 172.21.15.53:0/1195164242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94580d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f9456d9d640 1 -- 172.21.15.53:0/1195164242 >> v1:172.21.15.135:6789/0 conn(0x7f94580d2260 legacy=0x7f94580cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.963+0000 7f9456d9d640 1 -- 172.21.15.53:0/1195164242 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94580cd720 con 0x7f94580b8970 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f9456d9d640 1 --2- 172.21.15.53:0/1195164242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580b8970 0x7f945806ddc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9450009870 tx=0x7f945002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a602bc55633144c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f9455d9b640 1 -- 172.21.15.53:0/1195164242 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450037070 con 0x7f94580b8970 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f9455d9b640 1 -- 172.21.15.53:0/1195164242 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9450032440 con 0x7f94580b8970 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f9455d9b640 1 -- 172.21.15.53:0/1195164242 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450032810 con 0x7f94580b8970 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 -- 172.21.15.53:0/1195164242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580b8970 msgr2=0x7f945806ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 --2- 172.21.15.53:0/1195164242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580b8970 0x7f945806ddc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9450009870 tx=0x7f945002a250 comp rx=0 tx=0).stop 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 -- 172.21.15.53:0/1195164242 shutdown_connections 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 --2- 172.21.15.53:0/1195164242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94580d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 --2- 172.21.15.53:0/1195164242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580b8970 0x7f945806ddc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 -- 172.21.15.53:0/1195164242 >> 172.21.15.53:0/1195164242 conn(0x7f94580d2dc0 msgr2=0x7f94580d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 -- 172.21.15.53:0/1195164242 shutdown_connections 2024-05-10T10:33:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.964+0000 7f945d189640 1 -- 172.21.15.53:0/1195164242 wait complete. 2024-05-10T10:33:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.966+0000 7f945d189640 1 Processor -- start 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.966+0000 7f945d189640 1 -- start start 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94581578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94580b8970 0x7f9458162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580d2260 0x7f94580ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f94580d26f0 con 0x7f94580b8970 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94580cdce0 con 0x7f94580d2260 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f94580b5c10 con 0x7f945806e300 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94580b8970 0x7f9458162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94580b8970 0x7f9458162290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58706/0 (socket says 172.21.15.53:58706) 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 -- 172.21.15.53:0/707646282 learned_addr learned my addr 172.21.15.53:0/707646282 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f9456d9d640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94581578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945759e640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580d2260 0x7f94580ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 -- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 msgr2=0x7f94581578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94581578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 -- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580d2260 msgr2=0x7f94580ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580d2260 0x7f94580ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 -- 172.21.15.53:0/707646282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9458157e30 con 0x7f94580b8970 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945759e640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580d2260 0x7f94580ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945659c640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94580b8970 0x7f9458162290 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f944000e8a0 tx=0x7f944000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f944f7fe640 1 -- 172.21.15.53:0/707646282 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440017070 con 0x7f94580b8970 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f944f7fe640 1 -- 172.21.15.53:0/707646282 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944000c940 con 0x7f94580b8970 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94580c9fc0 con 0x7f94580b8970 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f944f7fe640 1 -- 172.21.15.53:0/707646282 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944000cd30 con 0x7f94580b8970 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.967+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94580ca580 con 0x7f94580b8970 2024-05-10T10:33:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.968+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94580693f0 con 0x7f94580b8970 2024-05-10T10:33:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.969+0000 7f944f7fe640 1 -- 172.21.15.53:0/707646282 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f944001a070 con 0x7f94580b8970 2024-05-10T10:33:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.970+0000 7f944f7fe640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94380762d0 0x7f9438078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.970+0000 7f944f7fe640 1 -- 172.21.15.53:0/707646282 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f94400976e0 con 0x7f94580b8970 2024-05-10T10:33:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.970+0000 7f9456d9d640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94380762d0 0x7f9438078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.970+0000 7f9456d9d640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94380762d0 0x7f9438078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9450034000 tx=0x7f945002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:52.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:52.974+0000 7f944f7fe640 1 -- 172.21.15.53:0/707646282 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9440060740 con 0x7f94580b8970 2024-05-10T10:33:53.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.160+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f945806ea10 con 0x7f94380762d0 2024-05-10T10:33:53.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.169+0000 7f944f7fe640 1 -- 172.21.15.53:0/707646282 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f945806ea10 con 0x7f94380762d0 2024-05-10T10:33:53.174 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:53.176 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94380762d0 msgr2=0x7f9438078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94380762d0 0x7f9438078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9450034000 tx=0x7f945002e000 comp rx=0 tx=0).stop 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94580b8970 msgr2=0x7f9458162290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94580b8970 0x7f9458162290 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f944000e8a0 tx=0x7f944000ed60 comp rx=0 tx=0).stop 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 shutdown_connections 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94380762d0 0x7f9438078790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94580d2260 0x7f94580ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94580b8970 0x7f9458162290 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.172+0000 7f945d189640 1 --2- 172.21.15.53:0/707646282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f945806e300 0x7f94581578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.173+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 >> 172.21.15.53:0/707646282 conn(0x7f94580d2dc0 msgr2=0x7f9458070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.173+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 shutdown_connections 2024-05-10T10:33:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.173+0000 7f945d189640 1 -- 172.21.15.53:0/707646282 wait complete. 2024-05-10T10:33:53.185 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-05-10T10:33:52.160478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236024,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738696,"statfs":{"total":773094113280,"available":772852424704,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553612+0000","last_change":"2024-05-10T10:33:33.499661+0000","last_active":"2024-05-10T10:33:39.553612+0000","last_peered":"2024-05-10T10:33:39.553612+0000","last_clean":"2024-05-10T10:33:39.553612+0000","last_became_active":"2024-05-10T10:33:33.499370+0000","last_became_peered":"2024-05-10T10:33:33.499370+0000","last_unstale":"2024-05-10T10:33:39.553612+0000","last_undegraded":"2024-05-10T10:33:39.553612+0000","last_fullsized":"2024-05-10T10:33:39.553612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:01:06.424327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641011+0000","last_change":"2024-05-10T10:33:34.515514+0000","last_active":"2024-05-10T10:33:39.641011+0000","last_peered":"2024-05-10T10:33:39.641011+0000","last_clean":"2024-05-10T10:33:39.641011+0000","last_became_active":"2024-05-10T10:33:34.514965+0000","last_became_peered":"2024-05-10T10:33:34.514965+0000","last_unstale":"2024-05-10T10:33:39.641011+0000","last_undegraded":"2024-05-10T10:33:39.641011+0000","last_fullsized":"2024-05-10T10:33:39.641011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640231+0000","last_change":"2024-05-10T10:33:38.535811+0000","last_active":"2024-05-10T10:33:39.640231+0000","last_peered":"2024-05-10T10:33:39.640231+0000","last_clean":"2024-05-10T10:33:39.640231+0000","last_became_active":"2024-05-10T10:33:38.535356+0000","last_became_peered":"2024-05-10T10:33:38.535356+0000","last_unstale":"2024-05-10T10:33:39.640231+0000","last_undegraded":"2024-05-10T10:33:39.640231+0000","last_fullsized":"2024-05-10T10:33:39.640231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641122+0000","last_change":"2024-05-10T10:33:33.494355+0000","last_active":"2024-05-10T10:33:39.641122+0000","last_peered":"2024-05-10T10:33:39.641122+0000","last_clean":"2024-05-10T10:33:39.641122+0000","last_became_active":"2024-05-10T10:33:33.494042+0000","last_became_peered":"2024-05-10T10:33:33.494042+0000","last_unstale":"2024-05-10T10:33:39.641122+0000","last_undegraded":"2024-05-10T10:33:39.641122+0000","last_fullsized":"2024-05-10T10:33:39.641122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553336+0000","last_change":"2024-05-10T10:33:34.514217+0000","last_active":"2024-05-10T10:33:39.553336+0000","last_peered":"2024-05-10T10:33:39.553336+0000","last_clean":"2024-05-10T10:33:39.553336+0000","last_became_active":"2024-05-10T10:33:34.513895+0000","last_became_peered":"2024-05-10T10:33:34.513895+0000","last_unstale":"2024-05-10T10:33:39.553336+0000","last_undegraded":"2024-05-10T10:33:39.553336+0000","last_fullsized":"2024-05-10T10:33:39.553336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872369+0000","last_change":"2024-05-10T10:33:38.542469+0000","last_active":"2024-05-10T10:33:40.872369+0000","last_peered":"2024-05-10T10:33:40.872369+0000","last_clean":"2024-05-10T10:33:40.872369+0000","last_became_active":"2024-05-10T10:33:38.541985+0000","last_became_peered":"2024-05-10T10:33:38.541985+0000","last_unstale":"2024-05-10T10:33:40.872369+0000","last_undegraded":"2024-05-10T10:33:40.872369+0000","last_fullsized":"2024-05-10T10:33:40.872369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:30.304981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554581+0000","last_change":"2024-05-10T10:33:33.500811+0000","last_active":"2024-05-10T10:33:39.554581+0000","last_peered":"2024-05-10T10:33:39.554581+0000","last_clean":"2024-05-10T10:33:39.554581+0000","last_became_active":"2024-05-10T10:33:33.500355+0000","last_became_peered":"2024-05-10T10:33:33.500355+0000","last_unstale":"2024-05-10T10:33:39.554581+0000","last_undegraded":"2024-05-10T10:33:39.554581+0000","last_fullsized":"2024-05-10T10:33:39.554581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871828+0000","last_change":"2024-05-10T10:33:34.502619+0000","last_active":"2024-05-10T10:33:40.871828+0000","last_peered":"2024-05-10T10:33:40.871828+0000","last_clean":"2024-05-10T10:33:40.871828+0000","last_became_active":"2024-05-10T10:33:34.500045+0000","last_became_peered":"2024-05-10T10:33:34.500045+0000","last_unstale":"2024-05-10T10:33:40.871828+0000","last_undegraded":"2024-05-10T10:33:40.871828+0000","last_fullsized":"2024-05-10T10:33:40.871828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872455+0000","last_change":"2024-05-10T10:33:38.543336+0000","last_active":"2024-05-10T10:33:40.872455+0000","last_peered":"2024-05-10T10:33:40.872455+0000","last_clean":"2024-05-10T10:33:40.872455+0000","last_became_active":"2024-05-10T10:33:38.542915+0000","last_became_peered":"2024-05-10T10:33:38.542915+0000","last_unstale":"2024-05-10T10:33:40.872455+0000","last_undegraded":"2024-05-10T10:33:40.872455+0000","last_fullsized":"2024-05-10T10:33:40.872455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:16.959519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553329+0000","last_change":"2024-05-10T10:33:33.499213+0000","last_active":"2024-05-10T10:33:39.553329+0000","last_peered":"2024-05-10T10:33:39.553329+0000","last_clean":"2024-05-10T10:33:39.553329+0000","last_became_active":"2024-05-10T10:33:33.497831+0000","last_became_peered":"2024-05-10T10:33:33.497831+0000","last_unstale":"2024-05-10T10:33:39.553329+0000","last_undegraded":"2024-05-10T10:33:39.553329+0000","last_fullsized":"2024-05-10T10:33:39.553329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:12.309888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871793+0000","last_change":"2024-05-10T10:33:34.508608+0000","last_active":"2024-05-10T10:33:40.871793+0000","last_peered":"2024-05-10T10:33:40.871793+0000","last_clean":"2024-05-10T10:33:40.871793+0000","last_became_active":"2024-05-10T10:33:34.507711+0000","last_became_peered":"2024-05-10T10:33:34.507711+0000","last_unstale":"2024-05-10T10:33:40.871793+0000","last_undegraded":"2024-05-10T10:33:40.871793+0000","last_fullsized":"2024-05-10T10:33:40.871793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872480+0000","last_change":"2024-05-10T10:33:38.534041+0000","last_active":"2024-05-10T10:33:40.872480+0000","last_peered":"2024-05-10T10:33:40.872480+0000","last_clean":"2024-05-10T10:33:40.872480+0000","last_became_active":"2024-05-10T10:33:38.533397+0000","last_became_peered":"2024-05-10T10:33:38.533397+0000","last_unstale":"2024-05-10T10:33:40.872480+0000","last_undegraded":"2024-05-10T10:33:40.872480+0000","last_fullsized":"2024-05-10T10:33:40.872480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:43.886335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553281+0000","last_change":"2024-05-10T10:33:34.504671+0000","last_active":"2024-05-10T10:33:39.553281+0000","last_peered":"2024-05-10T10:33:39.553281+0000","last_clean":"2024-05-10T10:33:39.553281+0000","last_became_active":"2024-05-10T10:33:34.504389+0000","last_became_peered":"2024-05-10T10:33:34.504389+0000","last_unstale":"2024-05-10T10:33:39.553281+0000","last_undegraded":"2024-05-10T10:33:39.553281+0000","last_fullsized":"2024-05-10T10:33:39.553281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553953+0000","last_change":"2024-05-10T10:33:33.499466+0000","last_active":"2024-05-10T10:33:39.553953+0000","last_peered":"2024-05-10T10:33:39.553953+0000","last_clean":"2024-05-10T10:33:39.553953+0000","last_became_active":"2024-05-10T10:33:33.499185+0000","last_became_peered":"2024-05-10T10:33:33.499185+0000","last_unstale":"2024-05-10T10:33:39.553953+0000","last_undegraded":"2024-05-10T10:33:39.553953+0000","last_fullsized":"2024-05-10T10:33:39.553953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:40.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872905+0000","last_change":"2024-05-10T10:33:38.542258+0000","last_active":"2024-05-10T10:33:40.872905+0000","last_peered":"2024-05-10T10:33:40.872905+0000","last_clean":"2024-05-10T10:33:40.872905+0000","last_became_active":"2024-05-10T10:33:38.541808+0000","last_became_peered":"2024-05-10T10:33:38.541808+0000","last_unstale":"2024-05-10T10:33:40.872905+0000","last_undegraded":"2024-05-10T10:33:40.872905+0000","last_fullsized":"2024-05-10T10:33:40.872905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640629+0000","last_change":"2024-05-10T10:33:34.506055+0000","last_active":"2024-05-10T10:33:39.640629+0000","last_peered":"2024-05-10T10:33:39.640629+0000","last_clean":"2024-05-10T10:33:39.640629+0000","last_became_active":"2024-05-10T10:33:34.505770+0000","last_became_peered":"2024-05-10T10:33:34.505770+0000","last_unstale":"2024-05-10T10:33:39.640629+0000","last_undegraded":"2024-05-10T10:33:39.640629+0000","last_fullsized":"2024-05-10T10:33:39.640629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886170+0000","last_change":"2024-05-10T10:33:33.495139+0000","last_active":"2024-05-10T10:33:39.886170+0000","last_peered":"2024-05-10T10:33:39.886170+0000","last_clean":"2024-05-10T10:33:39.886170+0000","last_became_active":"2024-05-10T10:33:33.494836+0000","last_became_peered":"2024-05-10T10:33:33.494836+0000","last_unstale":"2024-05-10T10:33:39.886170+0000","last_undegraded":"2024-05-10T10:33:39.886170+0000","last_fullsized":"2024-05-10T10:33:39.886170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872645+0000","last_change":"2024-05-10T10:33:38.542825+0000","last_active":"2024-05-10T10:33:40.872645+0000","last_peered":"2024-05-10T10:33:40.872645+0000","last_clean":"2024-05-10T10:33:40.872645+0000","last_became_active":"2024-05-10T10:33:38.542378+0000","last_became_peered":"2024-05-10T10:33:38.542378+0000","last_unstale":"2024-05-10T10:33:40.872645+0000","last_undegraded":"2024-05-10T10:33:40.872645+0000","last_fullsized":"2024-05-10T10:33:40.872645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640415+0000","last_change":"2024-05-10T10:33:34.504517+0000","last_active":"2024-05-10T10:33:39.640415+0000","last_peered":"2024-05-10T10:33:39.640415+0000","last_clean":"2024-05-10T10:33:39.640415+0000","last_became_active":"2024-05-10T10:33:34.504168+0000","last_became_peered":"2024-05-10T10:33:34.504168+0000","last_unstale":"2024-05-10T10:33:39.640415+0000","last_undegraded":"2024-05-10T10:33:39.640415+0000","last_fullsized":"2024-05-10T10:33:39.640415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885550+0000","last_change":"2024-05-10T10:33:33.497956+0000","last_active":"2024-05-10T10:33:39.885550+0000","last_peered":"2024-05-10T10:33:39.885550+0000","last_clean":"2024-05-10T10:33:39.885550+0000","last_became_active":"2024-05-10T10:33:33.497234+0000","last_became_peered":"2024-05-10T10:33:33.497234+0000","last_unstale":"2024-05-10T10:33:39.885550+0000","last_undegraded":"2024-05-10T10:33:39.885550+0000","last_fullsized":"2024-05-10T10:33:39.885550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886149+0000","last_change":"2024-05-10T10:33:38.548654+0000","last_active":"2024-05-10T10:33:39.886149+0000","last_peered":"2024-05-10T10:33:39.886149+0000","last_clean":"2024-05-10T10:33:39.886149+0000","last_became_active":"2024-05-10T10:33:38.547914+0000","last_became_peered":"2024-05-10T10:33:38.547914+0000","last_unstale":"2024-05-10T10:33:39.886149+0000","last_undegraded":"2024-05-10T10:33:39.886149+0000","last_fullsized":"2024-05-10T10:33:39.886149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872730+0000","last_change":"2024-05-10T10:33:34.499454+0000","last_active":"2024-05-10T10:33:40.872730+0000","last_peered":"2024-05-10T10:33:40.872730+0000","last_clean":"2024-05-10T10:33:40.872730+0000","last_became_active":"2024-05-10T10:33:34.499154+0000","last_became_peered":"2024-05-10T10:33:34.499154+0000","last_unstale":"2024-05-10T10:33:40.872730+0000","last_undegraded":"2024-05-10T10:33:40.872730+0000","last_fullsized":"2024-05-10T10:33:40.872730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:06.330274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554557+0000","last_change":"2024-05-10T10:33:33.497112+0000","last_active":"2024-05-10T10:33:39.554557+0000","last_peered":"2024-05-10T10:33:39.554557+0000","last_clean":"2024-05-10T10:33:39.554557+0000","last_became_active":"2024-05-10T10:33:33.496737+0000","last_became_peered":"2024-05-10T10:33:33.496737+0000","last_unstale":"2024-05-10T10:33:39.554557+0000","last_undegraded":"2024-05-10T10:33:39.554557+0000","last_fullsized":"2024-05-10T10:33:39.554557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:20.939521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873091+0000","last_change":"2024-05-10T10:33:38.542337+0000","last_active":"2024-05-10T10:33:40.873091+0000","last_peered":"2024-05-10T10:33:40.873091+0000","last_clean":"2024-05-10T10:33:40.873091+0000","last_became_active":"2024-05-10T10:33:38.541897+0000","last_became_peered":"2024-05-10T10:33:38.541897+0000","last_unstale":"2024-05-10T10:33:40.873091+0000","last_undegraded":"2024-05-10T10:33:40.873091+0000","last_fullsized":"2024-05-10T10:33:40.873091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553701+0000","last_change":"2024-05-10T10:33:34.503858+0000","last_active":"2024-05-10T10:33:39.553701+0000","last_peered":"2024-05-10T10:33:39.553701+0000","last_clean":"2024-05-10T10:33:39.553701+0000","last_became_active":"2024-05-10T10:33:34.503581+0000","last_became_peered":"2024-05-10T10:33:34.503581+0000","last_unstale":"2024-05-10T10:33:39.553701+0000","last_undegraded":"2024-05-10T10:33:39.553701+0000","last_fullsized":"2024-05-10T10:33:39.553701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554729+0000","last_change":"2024-05-10T10:33:33.498121+0000","last_active":"2024-05-10T10:33:39.554729+0000","last_peered":"2024-05-10T10:33:39.554729+0000","last_clean":"2024-05-10T10:33:39.554729+0000","last_became_active":"2024-05-10T10:33:33.497834+0000","last_became_peered":"2024-05-10T10:33:33.497834+0000","last_unstale":"2024-05-10T10:33:39.554729+0000","last_undegraded":"2024-05-10T10:33:39.554729+0000","last_fullsized":"2024-05-10T10:33:39.554729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872705+0000","last_change":"2024-05-10T10:33:38.531480+0000","last_active":"2024-05-10T10:33:40.872705+0000","last_peered":"2024-05-10T10:33:40.872705+0000","last_clean":"2024-05-10T10:33:40.872705+0000","last_became_active":"2024-05-10T10:33:38.531023+0000","last_became_peered":"2024-05-10T10:33:38.531023+0000","last_unstale":"2024-05-10T10:33:40.872705+0000","last_undegraded":"2024-05-10T10:33:40.872705+0000","last_fullsized":"2024-05-10T10:33:40.872705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641080+0000","last_change":"2024-05-10T10:33:34.506192+0000","last_active":"2024-05-10T10:33:39.641080+0000","last_peered":"2024-05-10T10:33:39.641080+0000","last_clean":"2024-05-10T10:33:39.641080+0000","last_became_active":"2024-05-10T10:33:34.505914+0000","last_became_peered":"2024-05-10T10:33:34.505914+0000","last_unstale":"2024-05-10T10:33:39.641080+0000","last_undegraded":"2024-05-10T10:33:39.641080+0000","last_fullsized":"2024-05-10T10:33:39.641080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886029+0000","last_change":"2024-05-10T10:33:33.491388+0000","last_active":"2024-05-10T10:33:39.886029+0000","last_peered":"2024-05-10T10:33:39.886029+0000","last_clean":"2024-05-10T10:33:39.886029+0000","last_became_active":"2024-05-10T10:33:33.491061+0000","last_became_peered":"2024-05-10T10:33:33.491061+0000","last_unstale":"2024-05-10T10:33:39.886029+0000","last_undegraded":"2024-05-10T10:33:39.886029+0000","last_fullsized":"2024-05-10T10:33:39.886029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640139+0000","last_change":"2024-05-10T10:33:38.543260+0000","last_active":"2024-05-10T10:33:39.640139+0000","last_peered":"2024-05-10T10:33:39.640139+0000","last_clean":"2024-05-10T10:33:39.640139+0000","last_became_active":"2024-05-10T10:33:38.542885+0000","last_became_peered":"2024-05-10T10:33:38.542885+0000","last_unstale":"2024-05-10T10:33:39.640139+0000","last_undegraded":"2024-05-10T10:33:39.640139+0000","last_fullsized":"2024-05-10T10:33:39.640139+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886114+0000","last_change":"2024-05-10T10:33:34.501069+0000","last_active":"2024-05-10T10:33:39.886114+0000","last_peered":"2024-05-10T10:33:39.886114+0000","last_clean":"2024-05-10T10:33:39.886114+0000","last_became_active":"2024-05-10T10:33:34.500767+0000","last_became_peered":"2024-05-10T10:33:34.500767+0000","last_unstale":"2024-05-10T10:33:39.886114+0000","last_undegraded":"2024-05-10T10:33:39.886114+0000","last_fullsized":"2024-05-10T10:33:39.886114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554556+0000","last_change":"2024-05-10T10:33:33.496126+0000","last_active":"2024-05-10T10:33:39.554556+0000","last_peered":"2024-05-10T10:33:39.554556+0000","last_clean":"2024-05-10T10:33:39.554556+0000","last_became_active":"2024-05-10T10:33:33.495848+0000","last_became_peered":"2024-05-10T10:33:33.495848+0000","last_unstale":"2024-05-10T10:33:39.554556+0000","last_undegraded":"2024-05-10T10:33:39.554556+0000","last_fullsized":"2024-05-10T10:33:39.554556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553188+0000","last_change":"2024-05-10T10:33:38.542612+0000","last_active":"2024-05-10T10:33:39.553188+0000","last_peered":"2024-05-10T10:33:39.553188+0000","last_clean":"2024-05-10T10:33:39.553188+0000","last_became_active":"2024-05-10T10:33:38.542110+0000","last_became_peered":"2024-05-10T10:33:38.542110+0000","last_unstale":"2024-05-10T10:33:39.553188+0000","last_undegraded":"2024-05-10T10:33:39.553188+0000","last_fullsized":"2024-05-10T10:33:39.553188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871909+0000","last_change":"2024-05-10T10:33:34.515283+0000","last_active":"2024-05-10T10:33:40.871909+0000","last_peered":"2024-05-10T10:33:40.871909+0000","last_clean":"2024-05-10T10:33:40.871909+0000","last_became_active":"2024-05-10T10:33:34.515016+0000","last_became_peered":"2024-05-10T10:33:34.515016+0000","last_unstale":"2024-05-10T10:33:40.871909+0000","last_undegraded":"2024-05-10T10:33:40.871909+0000","last_fullsized":"2024-05-10T10:33:40.871909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886059+0000","last_change":"2024-05-10T10:33:33.488825+0000","last_active":"2024-05-10T10:33:39.886059+0000","last_peered":"2024-05-10T10:33:39.886059+0000","last_clean":"2024-05-10T10:33:39.886059+0000","last_became_active":"2024-05-10T10:33:33.488539+0000","last_became_peered":"2024-05-10T10:33:33.488539+0000","last_unstale":"2024-05-10T10:33:39.886059+0000","last_undegraded":"2024-05-10T10:33:39.886059+0000","last_fullsized":"2024-05-10T10:33:39.886059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885597+0000","last_change":"2024-05-10T10:33:38.547163+0000","last_active":"2024-05-10T10:33:39.885597+0000","last_peered":"2024-05-10T10:33:39.885597+0000","last_clean":"2024-05-10T10:33:39.885597+0000","last_became_active":"2024-05-10T10:33:38.546692+0000","last_became_peered":"2024-05-10T10:33:38.546692+0000","last_unstale":"2024-05-10T10:33:39.885597+0000","last_undegraded":"2024-05-10T10:33:39.885597+0000","last_fullsized":"2024-05-10T10:33:39.885597+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872867+0000","last_change":"2024-05-10T10:33:34.505921+0000","last_active":"2024-05-10T10:33:40.872867+0000","last_peered":"2024-05-10T10:33:40.872867+0000","last_clean":"2024-05-10T10:33:40.872867+0000","last_became_active":"2024-05-10T10:33:34.505637+0000","last_became_peered":"2024-05-10T10:33:34.505637+0000","last_unstale":"2024-05-10T10:33:40.872867+0000","last_undegraded":"2024-05-10T10:33:40.872867+0000","last_fullsized":"2024-05-10T10:33:40.872867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871754+0000","last_change":"2024-05-10T10:33:33.485382+0000","last_active":"2024-05-10T10:33:40.871754+0000","last_peered":"2024-05-10T10:33:40.871754+0000","last_clean":"2024-05-10T10:33:40.871754+0000","last_became_active":"2024-05-10T10:33:33.485082+0000","last_became_peered":"2024-05-10T10:33:33.485082+0000","last_unstale":"2024-05-10T10:33:40.871754+0000","last_undegraded":"2024-05-10T10:33:40.871754+0000","last_fullsized":"2024-05-10T10:33:40.871754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885955+0000","last_change":"2024-05-10T10:33:38.541426+0000","last_active":"2024-05-10T10:33:39.885955+0000","last_peered":"2024-05-10T10:33:39.885955+0000","last_clean":"2024-05-10T10:33:39.885955+0000","last_became_active":"2024-05-10T10:33:38.539367+0000","last_became_peered":"2024-05-10T10:33:38.539367+0000","last_unstale":"2024-05-10T10:33:39.885955+0000","last_undegraded":"2024-05-10T10:33:39.885955+0000","last_fullsized":"2024-05-10T10:33:39.885955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885593+0000","last_change":"2024-05-10T10:33:34.501862+0000","last_active":"2024-05-10T10:33:39.885593+0000","last_peered":"2024-05-10T10:33:39.885593+0000","last_clean":"2024-05-10T10:33:39.885593+0000","last_became_active":"2024-05-10T10:33:34.501493+0000","last_became_peered":"2024-05-10T10:33:34.501493+0000","last_unstale":"2024-05-10T10:33:39.885593+0000","last_undegraded":"2024-05-10T10:33:39.885593+0000","last_fullsized":"2024-05-10T10:33:39.885593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679733+0000","last_change":"2024-05-10T10:33:33.495291+0000","last_active":"2024-05-10T10:33:39.679733+0000","last_peered":"2024-05-10T10:33:39.679733+0000","last_clean":"2024-05-10T10:33:39.679733+0000","last_became_active":"2024-05-10T10:33:33.494955+0000","last_became_peered":"2024-05-10T10:33:33.494955+0000","last_unstale":"2024-05-10T10:33:39.679733+0000","last_undegraded":"2024-05-10T10:33:39.679733+0000","last_fullsized":"2024-05-10T10:33:39.679733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886190+0000","last_change":"2024-05-10T10:33:38.539560+0000","last_active":"2024-05-10T10:33:39.886190+0000","last_peered":"2024-05-10T10:33:39.886190+0000","last_clean":"2024-05-10T10:33:39.886190+0000","last_became_active":"2024-05-10T10:33:38.539131+0000","last_became_peered":"2024-05-10T10:33:38.539131+0000","last_unstale":"2024-05-10T10:33:39.886190+0000","last_undegraded":"2024-05-10T10:33:39.886190+0000","last_fullsized":"2024-05-10T10:33:39.886190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552886+0000","last_change":"2024-05-10T10:33:34.510125+0000","last_active":"2024-05-10T10:33:39.552886+0000","last_peered":"2024-05-10T10:33:39.552886+0000","last_clean":"2024-05-10T10:33:39.552886+0000","last_became_active":"2024-05-10T10:33:34.509820+0000","last_became_peered":"2024-05-10T10:33:34.509820+0000","last_unstale":"2024-05-10T10:33:39.552886+0000","last_undegraded":"2024-05-10T10:33:39.552886+0000","last_fullsized":"2024-05-10T10:33:39.552886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554626+0000","last_change":"2024-05-10T10:33:33.499197+0000","last_active":"2024-05-10T10:33:39.554626+0000","last_peered":"2024-05-10T10:33:39.554626+0000","last_clean":"2024-05-10T10:33:39.554626+0000","last_became_active":"2024-05-10T10:33:33.497808+0000","last_became_peered":"2024-05-10T10:33:33.497808+0000","last_unstale":"2024-05-10T10:33:39.554626+0000","last_undegraded":"2024-05-10T10:33:39.554626+0000","last_fullsized":"2024-05-10T10:33:39.554626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:11.901185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553441+0000","last_change":"2024-05-10T10:33:38.544906+0000","last_active":"2024-05-10T10:33:39.553441+0000","last_peered":"2024-05-10T10:33:39.553441+0000","last_clean":"2024-05-10T10:33:39.553441+0000","last_became_active":"2024-05-10T10:33:38.544304+0000","last_became_peered":"2024-05-10T10:33:38.544304+0000","last_unstale":"2024-05-10T10:33:39.553441+0000","last_undegraded":"2024-05-10T10:33:39.553441+0000","last_fullsized":"2024-05-10T10:33:39.553441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640227+0000","last_change":"2024-05-10T10:33:34.513886+0000","last_active":"2024-05-10T10:33:39.640227+0000","last_peered":"2024-05-10T10:33:39.640227+0000","last_clean":"2024-05-10T10:33:39.640227+0000","last_became_active":"2024-05-10T10:33:34.513617+0000","last_became_peered":"2024-05-10T10:33:34.513617+0000","last_unstale":"2024-05-10T10:33:39.640227+0000","last_undegraded":"2024-05-10T10:33:39.640227+0000","last_fullsized":"2024-05-10T10:33:39.640227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871616+0000","last_change":"2024-05-10T10:33:33.489088+0000","last_active":"2024-05-10T10:33:40.871616+0000","last_peered":"2024-05-10T10:33:40.871616+0000","last_clean":"2024-05-10T10:33:40.871616+0000","last_became_active":"2024-05-10T10:33:33.488785+0000","last_became_peered":"2024-05-10T10:33:33.488785+0000","last_unstale":"2024-05-10T10:33:40.871616+0000","last_undegraded":"2024-05-10T10:33:40.871616+0000","last_fullsized":"2024-05-10T10:33:40.871616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:06:14.764873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554687+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:33:39.554687+0000","last_peered":"2024-05-10T10:33:39.554687+0000","last_clean":"2024-05-10T10:33:39.554687+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:33:39.554687+0000","last_undegraded":"2024-05-10T10:33:39.554687+0000","last_fullsized":"2024-05-10T10:33:39.554687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553037+0000","last_change":"2024-05-10T10:33:38.538561+0000","last_active":"2024-05-10T10:33:39.553037+0000","last_peered":"2024-05-10T10:33:39.553037+0000","last_clean":"2024-05-10T10:33:39.553037+0000","last_became_active":"2024-05-10T10:33:38.538159+0000","last_became_peered":"2024-05-10T10:33:38.538159+0000","last_unstale":"2024-05-10T10:33:39.553037+0000","last_undegraded":"2024-05-10T10:33:39.553037+0000","last_fullsized":"2024-05-10T10:33:39.553037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554231+0000","last_change":"2024-05-10T10:33:34.517851+0000","last_active":"2024-05-10T10:33:39.554231+0000","last_peered":"2024-05-10T10:33:39.554231+0000","last_clean":"2024-05-10T10:33:39.554231+0000","last_became_active":"2024-05-10T10:33:34.517556+0000","last_became_peered":"2024-05-10T10:33:34.517556+0000","last_unstale":"2024-05-10T10:33:39.554231+0000","last_undegraded":"2024-05-10T10:33:39.554231+0000","last_fullsized":"2024-05-10T10:33:39.554231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:46.690299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680077+0000","last_change":"2024-05-10T10:33:33.488242+0000","last_active":"2024-05-10T10:33:39.680077+0000","last_peered":"2024-05-10T10:33:39.680077+0000","last_clean":"2024-05-10T10:33:39.680077+0000","last_became_active":"2024-05-10T10:33:33.487932+0000","last_became_peered":"2024-05-10T10:33:33.487932+0000","last_unstale":"2024-05-10T10:33:39.680077+0000","last_undegraded":"2024-05-10T10:33:39.680077+0000","last_fullsized":"2024-05-10T10:33:39.680077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552940+0000","last_change":"2024-05-10T10:33:38.534596+0000","last_active":"2024-05-10T10:33:39.552940+0000","last_peered":"2024-05-10T10:33:39.552940+0000","last_clean":"2024-05-10T10:33:39.552940+0000","last_became_active":"2024-05-10T10:33:38.534112+0000","last_became_peered":"2024-05-10T10:33:38.534112+0000","last_unstale":"2024-05-10T10:33:39.552940+0000","last_undegraded":"2024-05-10T10:33:39.552940+0000","last_fullsized":"2024-05-10T10:33:39.552940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871944+0000","last_change":"2024-05-10T10:33:34.514478+0000","last_active":"2024-05-10T10:33:40.871944+0000","last_peered":"2024-05-10T10:33:40.871944+0000","last_clean":"2024-05-10T10:33:40.871944+0000","last_became_active":"2024-05-10T10:33:34.514160+0000","last_became_peered":"2024-05-10T10:33:34.514160+0000","last_unstale":"2024-05-10T10:33:40.871944+0000","last_undegraded":"2024-05-10T10:33:40.871944+0000","last_fullsized":"2024-05-10T10:33:40.871944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886005+0000","last_change":"2024-05-10T10:33:33.499523+0000","last_active":"2024-05-10T10:33:39.886005+0000","last_peered":"2024-05-10T10:33:39.886005+0000","last_clean":"2024-05-10T10:33:39.886005+0000","last_became_active":"2024-05-10T10:33:33.499250+0000","last_became_peered":"2024-05-10T10:33:33.499250+0000","last_unstale":"2024-05-10T10:33:39.886005+0000","last_undegraded":"2024-05-10T10:33:39.886005+0000","last_fullsized":"2024-05-10T10:33:39.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553375+0000","last_change":"2024-05-10T10:33:38.543483+0000","last_active":"2024-05-10T10:33:39.553375+0000","last_peered":"2024-05-10T10:33:39.553375+0000","last_clean":"2024-05-10T10:33:39.553375+0000","last_became_active":"2024-05-10T10:33:38.543060+0000","last_became_peered":"2024-05-10T10:33:38.543060+0000","last_unstale":"2024-05-10T10:33:39.553375+0000","last_undegraded":"2024-05-10T10:33:39.553375+0000","last_fullsized":"2024-05-10T10:33:39.553375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554354+0000","last_change":"2024-05-10T10:33:34.522046+0000","last_active":"2024-05-10T10:33:39.554354+0000","last_peered":"2024-05-10T10:33:39.554354+0000","last_clean":"2024-05-10T10:33:39.554354+0000","last_became_active":"2024-05-10T10:33:34.521739+0000","last_became_peered":"2024-05-10T10:33:34.521739+0000","last_unstale":"2024-05-10T10:33:39.554354+0000","last_undegraded":"2024-05-10T10:33:39.554354+0000","last_fullsized":"2024-05-10T10:33:39.554354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:15.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680220+0000","last_change":"2024-05-10T10:33:33.495634+0000","last_active":"2024-05-10T10:33:39.680220+0000","last_peered":"2024-05-10T10:33:39.680220+0000","last_clean":"2024-05-10T10:33:39.680220+0000","last_became_active":"2024-05-10T10:33:33.494432+0000","last_became_peered":"2024-05-10T10:33:33.494432+0000","last_unstale":"2024-05-10T10:33:39.680220+0000","last_undegraded":"2024-05-10T10:33:39.680220+0000","last_fullsized":"2024-05-10T10:33:39.680220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871284+0000","last_change":"2024-05-10T10:33:38.528059+0000","last_active":"2024-05-10T10:33:40.871284+0000","last_peered":"2024-05-10T10:33:40.871284+0000","last_clean":"2024-05-10T10:33:40.871284+0000","last_became_active":"2024-05-10T10:33:38.527628+0000","last_became_peered":"2024-05-10T10:33:38.527628+0000","last_unstale":"2024-05-10T10:33:40.871284+0000","last_undegraded":"2024-05-10T10:33:40.871284+0000","last_fullsized":"2024-05-10T10:33:40.871284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554210+0000","last_change":"2024-05-10T10:33:34.519810+0000","last_active":"2024-05-10T10:33:39.554210+0000","last_peered":"2024-05-10T10:33:39.554210+0000","last_clean":"2024-05-10T10:33:39.554210+0000","last_became_active":"2024-05-10T10:33:34.519517+0000","last_became_peered":"2024-05-10T10:33:34.519517+0000","last_unstale":"2024-05-10T10:33:39.554210+0000","last_undegraded":"2024-05-10T10:33:39.554210+0000","last_fullsized":"2024-05-10T10:33:39.554210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873138+0000","last_change":"2024-05-10T10:33:33.494974+0000","last_active":"2024-05-10T10:33:40.873138+0000","last_peered":"2024-05-10T10:33:40.873138+0000","last_clean":"2024-05-10T10:33:40.873138+0000","last_became_active":"2024-05-10T10:33:33.494667+0000","last_became_peered":"2024-05-10T10:33:33.494667+0000","last_unstale":"2024-05-10T10:33:40.873138+0000","last_undegraded":"2024-05-10T10:33:40.873138+0000","last_fullsized":"2024-05-10T10:33:40.873138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:54.706371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872313+0000","last_change":"2024-05-10T10:33:38.542343+0000","last_active":"2024-05-10T10:33:40.872313+0000","last_peered":"2024-05-10T10:33:40.872313+0000","last_clean":"2024-05-10T10:33:40.872313+0000","last_became_active":"2024-05-10T10:33:38.541842+0000","last_became_peered":"2024-05-10T10:33:38.541842+0000","last_unstale":"2024-05-10T10:33:40.872313+0000","last_undegraded":"2024-05-10T10:33:40.872313+0000","last_fullsized":"2024-05-10T10:33:40.872313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:32.462562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553822+0000","last_change":"2024-05-10T10:33:34.520528+0000","last_active":"2024-05-10T10:33:39.553822+0000","last_peered":"2024-05-10T10:33:39.553822+0000","last_clean":"2024-05-10T10:33:39.553822+0000","last_became_active":"2024-05-10T10:33:34.520239+0000","last_became_peered":"2024-05-10T10:33:34.520239+0000","last_unstale":"2024-05-10T10:33:39.553822+0000","last_undegraded":"2024-05-10T10:33:39.553822+0000","last_fullsized":"2024-05-10T10:33:39.553822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679540+0000","last_change":"2024-05-10T10:33:33.492349+0000","last_active":"2024-05-10T10:33:39.679540+0000","last_peered":"2024-05-10T10:33:39.679540+0000","last_clean":"2024-05-10T10:33:39.679540+0000","last_became_active":"2024-05-10T10:33:33.492019+0000","last_became_peered":"2024-05-10T10:33:33.492019+0000","last_unstale":"2024-05-10T10:33:39.679540+0000","last_undegraded":"2024-05-10T10:33:39.679540+0000","last_fullsized":"2024-05-10T10:33:39.679540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553319+0000","last_change":"2024-05-10T10:33:38.538548+0000","last_active":"2024-05-10T10:33:39.553319+0000","last_peered":"2024-05-10T10:33:39.553319+0000","last_clean":"2024-05-10T10:33:39.553319+0000","last_became_active":"2024-05-10T10:33:38.538144+0000","last_became_peered":"2024-05-10T10:33:38.538144+0000","last_unstale":"2024-05-10T10:33:39.553319+0000","last_undegraded":"2024-05-10T10:33:39.553319+0000","last_fullsized":"2024-05-10T10:33:39.553319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553004+0000","last_change":"2024-05-10T10:33:34.510224+0000","last_active":"2024-05-10T10:33:39.553004+0000","last_peered":"2024-05-10T10:33:39.553004+0000","last_clean":"2024-05-10T10:33:39.553004+0000","last_became_active":"2024-05-10T10:33:34.509584+0000","last_became_peered":"2024-05-10T10:33:34.509584+0000","last_unstale":"2024-05-10T10:33:39.553004+0000","last_undegraded":"2024-05-10T10:33:39.553004+0000","last_fullsized":"2024-05-10T10:33:39.553004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553209+0000","last_change":"2024-05-10T10:33:33.496606+0000","last_active":"2024-05-10T10:33:39.553209+0000","last_peered":"2024-05-10T10:33:39.553209+0000","last_clean":"2024-05-10T10:33:39.553209+0000","last_became_active":"2024-05-10T10:33:33.496326+0000","last_became_peered":"2024-05-10T10:33:33.496326+0000","last_unstale":"2024-05-10T10:33:39.553209+0000","last_undegraded":"2024-05-10T10:33:39.553209+0000","last_fullsized":"2024-05-10T10:33:39.553209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641045+0000","last_change":"2024-05-10T10:33:38.539948+0000","last_active":"2024-05-10T10:33:39.641045+0000","last_peered":"2024-05-10T10:33:39.641045+0000","last_clean":"2024-05-10T10:33:39.641045+0000","last_became_active":"2024-05-10T10:33:38.539181+0000","last_became_peered":"2024-05-10T10:33:38.539181+0000","last_unstale":"2024-05-10T10:33:39.641045+0000","last_undegraded":"2024-05-10T10:33:39.641045+0000","last_fullsized":"2024-05-10T10:33:39.641045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:03.969211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554079+0000","last_change":"2024-05-10T10:33:34.519826+0000","last_active":"2024-05-10T10:33:39.554079+0000","last_peered":"2024-05-10T10:33:39.554079+0000","last_clean":"2024-05-10T10:33:39.554079+0000","last_became_active":"2024-05-10T10:33:34.519540+0000","last_became_peered":"2024-05-10T10:33:34.519540+0000","last_unstale":"2024-05-10T10:33:39.554079+0000","last_undegraded":"2024-05-10T10:33:39.554079+0000","last_fullsized":"2024-05-10T10:33:39.554079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871721+0000","last_change":"2024-05-10T10:33:33.494936+0000","last_active":"2024-05-10T10:33:40.871721+0000","last_peered":"2024-05-10T10:33:40.871721+0000","last_clean":"2024-05-10T10:33:40.871721+0000","last_became_active":"2024-05-10T10:33:33.493154+0000","last_became_peered":"2024-05-10T10:33:33.493154+0000","last_unstale":"2024-05-10T10:33:40.871721+0000","last_undegraded":"2024-05-10T10:33:40.871721+0000","last_fullsized":"2024-05-10T10:33:40.871721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640822+0000","last_change":"2024-05-10T10:33:38.544918+0000","last_active":"2024-05-10T10:33:39.640822+0000","last_peered":"2024-05-10T10:33:39.640822+0000","last_clean":"2024-05-10T10:33:39.640822+0000","last_became_active":"2024-05-10T10:33:38.544423+0000","last_became_peered":"2024-05-10T10:33:38.544423+0000","last_unstale":"2024-05-10T10:33:39.640822+0000","last_undegraded":"2024-05-10T10:33:39.640822+0000","last_fullsized":"2024-05-10T10:33:39.640822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:22.017984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553482+0000","last_change":"2024-05-10T10:33:34.506856+0000","last_active":"2024-05-10T10:33:39.553482+0000","last_peered":"2024-05-10T10:33:39.553482+0000","last_clean":"2024-05-10T10:33:39.553482+0000","last_became_active":"2024-05-10T10:33:34.506563+0000","last_became_peered":"2024-05-10T10:33:34.506563+0000","last_unstale":"2024-05-10T10:33:39.553482+0000","last_undegraded":"2024-05-10T10:33:39.553482+0000","last_fullsized":"2024-05-10T10:33:39.553482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:21.944467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553658+0000","last_change":"2024-05-10T10:33:33.496358+0000","last_active":"2024-05-10T10:33:39.553658+0000","last_peered":"2024-05-10T10:33:39.553658+0000","last_clean":"2024-05-10T10:33:39.553658+0000","last_became_active":"2024-05-10T10:33:33.495968+0000","last_became_peered":"2024-05-10T10:33:33.495968+0000","last_unstale":"2024-05-10T10:33:39.553658+0000","last_undegraded":"2024-05-10T10:33:39.553658+0000","last_fullsized":"2024-05-10T10:33:39.553658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:14.817696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871851+0000","last_change":"2024-05-10T10:33:38.534046+0000","last_active":"2024-05-10T10:33:40.871851+0000","last_peered":"2024-05-10T10:33:40.871851+0000","last_clean":"2024-05-10T10:33:40.871851+0000","last_became_active":"2024-05-10T10:33:38.533660+0000","last_became_peered":"2024-05-10T10:33:38.533660+0000","last_unstale":"2024-05-10T10:33:40.871851+0000","last_undegraded":"2024-05-10T10:33:40.871851+0000","last_fullsized":"2024-05-10T10:33:40.871851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554110+0000","last_change":"2024-05-10T10:33:34.516752+0000","last_active":"2024-05-10T10:33:39.554110+0000","last_peered":"2024-05-10T10:33:39.554110+0000","last_clean":"2024-05-10T10:33:39.554110+0000","last_became_active":"2024-05-10T10:33:34.516410+0000","last_became_peered":"2024-05-10T10:33:34.516410+0000","last_unstale":"2024-05-10T10:33:39.554110+0000","last_undegraded":"2024-05-10T10:33:39.554110+0000","last_fullsized":"2024-05-10T10:33:39.554110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:44.179696+0000","last_change":"2024-05-10T10:33:33.498231+0000","last_active":"2024-05-10T10:33:44.179696+0000","last_peered":"2024-05-10T10:33:44.179696+0000","last_clean":"2024-05-10T10:33:44.179696+0000","last_became_active":"2024-05-10T10:33:33.497918+0000","last_became_peered":"2024-05-10T10:33:33.497918+0000","last_unstale":"2024-05-10T10:33:44.179696+0000","last_undegraded":"2024-05-10T10:33:44.179696+0000","last_fullsized":"2024-05-10T10:33:44.179696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885703+0000","last_change":"2024-05-10T10:33:38.537215+0000","last_active":"2024-05-10T10:33:39.885703+0000","last_peered":"2024-05-10T10:33:39.885703+0000","last_clean":"2024-05-10T10:33:39.885703+0000","last_became_active":"2024-05-10T10:33:38.536826+0000","last_became_peered":"2024-05-10T10:33:38.536826+0000","last_unstale":"2024-05-10T10:33:39.885703+0000","last_undegraded":"2024-05-10T10:33:39.885703+0000","last_fullsized":"2024-05-10T10:33:39.885703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872972+0000","last_change":"2024-05-10T10:33:34.512632+0000","last_active":"2024-05-10T10:33:40.872972+0000","last_peered":"2024-05-10T10:33:40.872972+0000","last_clean":"2024-05-10T10:33:40.872972+0000","last_became_active":"2024-05-10T10:33:34.512337+0000","last_became_peered":"2024-05-10T10:33:34.512337+0000","last_unstale":"2024-05-10T10:33:40.872972+0000","last_undegraded":"2024-05-10T10:33:40.872972+0000","last_fullsized":"2024-05-10T10:33:40.872972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:12.906132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885962+0000","last_change":"2024-05-10T10:33:33.485715+0000","last_active":"2024-05-10T10:33:39.885962+0000","last_peered":"2024-05-10T10:33:39.885962+0000","last_clean":"2024-05-10T10:33:39.885962+0000","last_became_active":"2024-05-10T10:33:33.485400+0000","last_became_peered":"2024-05-10T10:33:33.485400+0000","last_unstale":"2024-05-10T10:33:39.885962+0000","last_undegraded":"2024-05-10T10:33:39.885962+0000","last_fullsized":"2024-05-10T10:33:39.885962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872703+0000","last_change":"2024-05-10T10:33:38.542348+0000","last_active":"2024-05-10T10:33:40.872703+0000","last_peered":"2024-05-10T10:33:40.872703+0000","last_clean":"2024-05-10T10:33:40.872703+0000","last_became_active":"2024-05-10T10:33:38.541943+0000","last_became_peered":"2024-05-10T10:33:38.541943+0000","last_unstale":"2024-05-10T10:33:40.872703+0000","last_undegraded":"2024-05-10T10:33:40.872703+0000","last_fullsized":"2024-05-10T10:33:40.872703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553583+0000","last_change":"2024-05-10T10:33:34.514299+0000","last_active":"2024-05-10T10:33:39.553583+0000","last_peered":"2024-05-10T10:33:39.553583+0000","last_clean":"2024-05-10T10:33:39.553583+0000","last_became_active":"2024-05-10T10:33:34.513977+0000","last_became_peered":"2024-05-10T10:33:34.513977+0000","last_unstale":"2024-05-10T10:33:39.553583+0000","last_undegraded":"2024-05-10T10:33:39.553583+0000","last_fullsized":"2024-05-10T10:33:39.553583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872002+0000","last_change":"2024-05-10T10:33:33.491068+0000","last_active":"2024-05-10T10:33:40.872002+0000","last_peered":"2024-05-10T10:33:40.872002+0000","last_clean":"2024-05-10T10:33:40.872002+0000","last_became_active":"2024-05-10T10:33:33.490801+0000","last_became_peered":"2024-05-10T10:33:33.490801+0000","last_unstale":"2024-05-10T10:33:40.872002+0000","last_undegraded":"2024-05-10T10:33:40.872002+0000","last_fullsized":"2024-05-10T10:33:40.872002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640322+0000","last_change":"2024-05-10T10:33:38.538544+0000","last_active":"2024-05-10T10:33:39.640322+0000","last_peered":"2024-05-10T10:33:39.640322+0000","last_clean":"2024-05-10T10:33:39.640322+0000","last_became_active":"2024-05-10T10:33:38.538176+0000","last_became_peered":"2024-05-10T10:33:38.538176+0000","last_unstale":"2024-05-10T10:33:39.640322+0000","last_undegraded":"2024-05-10T10:33:39.640322+0000","last_fullsized":"2024-05-10T10:33:39.640322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:14.049514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871829+0000","last_change":"2024-05-10T10:33:34.513516+0000","last_active":"2024-05-10T10:33:40.871829+0000","last_peered":"2024-05-10T10:33:40.871829+0000","last_clean":"2024-05-10T10:33:40.871829+0000","last_became_active":"2024-05-10T10:33:34.512746+0000","last_became_peered":"2024-05-10T10:33:34.512746+0000","last_unstale":"2024-05-10T10:33:40.871829+0000","last_undegraded":"2024-05-10T10:33:40.871829+0000","last_fullsized":"2024-05-10T10:33:40.871829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554597+0000","last_change":"2024-05-10T10:33:33.498011+0000","last_active":"2024-05-10T10:33:39.554597+0000","last_peered":"2024-05-10T10:33:39.554597+0000","last_clean":"2024-05-10T10:33:39.554597+0000","last_became_active":"2024-05-10T10:33:33.497732+0000","last_became_peered":"2024-05-10T10:33:33.497732+0000","last_unstale":"2024-05-10T10:33:39.554597+0000","last_undegraded":"2024-05-10T10:33:39.554597+0000","last_fullsized":"2024-05-10T10:33:39.554597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640338+0000","last_change":"2024-05-10T10:33:38.537084+0000","last_active":"2024-05-10T10:33:39.640338+0000","last_peered":"2024-05-10T10:33:39.640338+0000","last_clean":"2024-05-10T10:33:39.640338+0000","last_became_active":"2024-05-10T10:33:38.535424+0000","last_became_peered":"2024-05-10T10:33:38.535424+0000","last_unstale":"2024-05-10T10:33:39.640338+0000","last_undegraded":"2024-05-10T10:33:39.640338+0000","last_fullsized":"2024-05-10T10:33:39.640338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:48:24.535051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553257+0000","last_change":"2024-05-10T10:33:38.545099+0000","last_active":"2024-05-10T10:33:39.553257+0000","last_peered":"2024-05-10T10:33:39.553257+0000","last_clean":"2024-05-10T10:33:39.553257+0000","last_became_active":"2024-05-10T10:33:38.544663+0000","last_became_peered":"2024-05-10T10:33:38.544663+0000","last_unstale":"2024-05-10T10:33:39.553257+0000","last_undegraded":"2024-05-10T10:33:39.553257+0000","last_fullsized":"2024-05-10T10:33:39.553257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885419+0000","last_change":"2024-05-10T10:33:34.505691+0000","last_active":"2024-05-10T10:33:39.885419+0000","last_peered":"2024-05-10T10:33:39.885419+0000","last_clean":"2024-05-10T10:33:39.885419+0000","last_became_active":"2024-05-10T10:33:34.505398+0000","last_became_peered":"2024-05-10T10:33:34.505398+0000","last_unstale":"2024-05-10T10:33:39.885419+0000","last_undegraded":"2024-05-10T10:33:39.885419+0000","last_fullsized":"2024-05-10T10:33:39.885419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871432+0000","last_change":"2024-05-10T10:33:33.494574+0000","last_active":"2024-05-10T10:33:40.871432+0000","last_peered":"2024-05-10T10:33:40.871432+0000","last_clean":"2024-05-10T10:33:40.871432+0000","last_became_active":"2024-05-10T10:33:33.494311+0000","last_became_peered":"2024-05-10T10:33:33.494311+0000","last_unstale":"2024-05-10T10:33:40.871432+0000","last_undegraded":"2024-05-10T10:33:40.871432+0000","last_fullsized":"2024-05-10T10:33:40.871432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554526+0000","last_change":"2024-05-10T10:33:38.537291+0000","last_active":"2024-05-10T10:33:39.554526+0000","last_peered":"2024-05-10T10:33:39.554526+0000","last_clean":"2024-05-10T10:33:39.554526+0000","last_became_active":"2024-05-10T10:33:38.536902+0000","last_became_peered":"2024-05-10T10:33:38.536902+0000","last_unstale":"2024-05-10T10:33:39.554526+0000","last_undegraded":"2024-05-10T10:33:39.554526+0000","last_fullsized":"2024-05-10T10:33:39.554526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640318+0000","last_change":"2024-05-10T10:33:34.505242+0000","last_active":"2024-05-10T10:33:39.640318+0000","last_peered":"2024-05-10T10:33:39.640318+0000","last_clean":"2024-05-10T10:33:39.640318+0000","last_became_active":"2024-05-10T10:33:34.504914+0000","last_became_peered":"2024-05-10T10:33:34.504914+0000","last_unstale":"2024-05-10T10:33:39.640318+0000","last_undegraded":"2024-05-10T10:33:39.640318+0000","last_fullsized":"2024-05-10T10:33:39.640318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640485+0000","last_change":"2024-05-10T10:33:33.498662+0000","last_active":"2024-05-10T10:33:39.640485+0000","last_peered":"2024-05-10T10:33:39.640485+0000","last_clean":"2024-05-10T10:33:39.640485+0000","last_became_active":"2024-05-10T10:33:33.498294+0000","last_became_peered":"2024-05-10T10:33:33.498294+0000","last_unstale":"2024-05-10T10:33:39.640485+0000","last_undegraded":"2024-05-10T10:33:39.640485+0000","last_fullsized":"2024-05-10T10:33:39.640485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553501+0000","last_change":"2024-05-10T10:33:33.495684+0000","last_active":"2024-05-10T10:33:39.553501+0000","last_peered":"2024-05-10T10:33:39.553501+0000","last_clean":"2024-05-10T10:33:39.553501+0000","last_became_active":"2024-05-10T10:33:33.495347+0000","last_became_peered":"2024-05-10T10:33:33.495347+0000","last_unstale":"2024-05-10T10:33:39.553501+0000","last_undegraded":"2024-05-10T10:33:39.553501+0000","last_fullsized":"2024-05-10T10:33:39.553501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553543+0000","last_change":"2024-05-10T10:33:34.502038+0000","last_active":"2024-05-10T10:33:39.553543+0000","last_peered":"2024-05-10T10:33:39.553543+0000","last_clean":"2024-05-10T10:33:39.553543+0000","last_became_active":"2024-05-10T10:33:34.501709+0000","last_became_peered":"2024-05-10T10:33:34.501709+0000","last_unstale":"2024-05-10T10:33:39.553543+0000","last_undegraded":"2024-05-10T10:33:39.553543+0000","last_fullsized":"2024-05-10T10:33:39.553543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679952+0000","last_change":"2024-05-10T10:33:38.536172+0000","last_active":"2024-05-10T10:33:39.679952+0000","last_peered":"2024-05-10T10:33:39.679952+0000","last_clean":"2024-05-10T10:33:39.679952+0000","last_became_active":"2024-05-10T10:33:38.535590+0000","last_became_peered":"2024-05-10T10:33:38.535590+0000","last_unstale":"2024-05-10T10:33:39.679952+0000","last_undegraded":"2024-05-10T10:33:39.679952+0000","last_fullsized":"2024-05-10T10:33:39.679952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:07.728619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640673+0000","last_change":"2024-05-10T10:33:33.497950+0000","last_active":"2024-05-10T10:33:39.640673+0000","last_peered":"2024-05-10T10:33:39.640673+0000","last_clean":"2024-05-10T10:33:39.640673+0000","last_became_active":"2024-05-10T10:33:33.497590+0000","last_became_peered":"2024-05-10T10:33:33.497590+0000","last_unstale":"2024-05-10T10:33:39.640673+0000","last_undegraded":"2024-05-10T10:33:39.640673+0000","last_fullsized":"2024-05-10T10:33:39.640673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872524+0000","last_change":"2024-05-10T10:33:34.508520+0000","last_active":"2024-05-10T10:33:40.872524+0000","last_peered":"2024-05-10T10:33:40.872524+0000","last_clean":"2024-05-10T10:33:40.872524+0000","last_became_active":"2024-05-10T10:33:34.508262+0000","last_became_peered":"2024-05-10T10:33:34.508262+0000","last_unstale":"2024-05-10T10:33:40.872524+0000","last_undegraded":"2024-05-10T10:33:40.872524+0000","last_fullsized":"2024-05-10T10:33:40.872524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:54.281454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640675+0000","last_change":"2024-05-10T10:33:38.542608+0000","last_active":"2024-05-10T10:33:39.640675+0000","last_peered":"2024-05-10T10:33:39.640675+0000","last_clean":"2024-05-10T10:33:39.640675+0000","last_became_active":"2024-05-10T10:33:38.540900+0000","last_became_peered":"2024-05-10T10:33:38.540900+0000","last_unstale":"2024-05-10T10:33:39.640675+0000","last_undegraded":"2024-05-10T10:33:39.640675+0000","last_fullsized":"2024-05-10T10:33:39.640675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:41.505822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552881+0000","last_change":"2024-05-10T10:33:34.510874+0000","last_active":"2024-05-10T10:33:39.552881+0000","last_peered":"2024-05-10T10:33:39.552881+0000","last_clean":"2024-05-10T10:33:39.552881+0000","last_became_active":"2024-05-10T10:33:34.510623+0000","last_became_peered":"2024-05-10T10:33:34.510623+0000","last_unstale":"2024-05-10T10:33:39.552881+0000","last_undegraded":"2024-05-10T10:33:39.552881+0000","last_fullsized":"2024-05-10T10:33:39.552881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679646+0000","last_change":"2024-05-10T10:33:34.503232+0000","last_active":"2024-05-10T10:33:39.679646+0000","last_peered":"2024-05-10T10:33:39.679646+0000","last_clean":"2024-05-10T10:33:39.679646+0000","last_became_active":"2024-05-10T10:33:34.502932+0000","last_became_peered":"2024-05-10T10:33:34.502932+0000","last_unstale":"2024-05-10T10:33:39.679646+0000","last_undegraded":"2024-05-10T10:33:39.679646+0000","last_fullsized":"2024-05-10T10:33:39.679646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871608+0000","last_change":"2024-05-10T10:33:34.516049+0000","last_active":"2024-05-10T10:33:40.871608+0000","last_peered":"2024-05-10T10:33:40.871608+0000","last_clean":"2024-05-10T10:33:40.871608+0000","last_became_active":"2024-05-10T10:33:34.515617+0000","last_became_peered":"2024-05-10T10:33:34.515617+0000","last_unstale":"2024-05-10T10:33:40.871608+0000","last_undegraded":"2024-05-10T10:33:40.871608+0000","last_fullsized":"2024-05-10T10:33:40.871608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553405+0000","last_change":"2024-05-10T10:33:34.508355+0000","last_active":"2024-05-10T10:33:39.553405+0000","last_peered":"2024-05-10T10:33:39.553405+0000","last_clean":"2024-05-10T10:33:39.553405+0000","last_became_active":"2024-05-10T10:33:34.508064+0000","last_became_peered":"2024-05-10T10:33:34.508064+0000","last_unstale":"2024-05-10T10:33:39.553405+0000","last_undegraded":"2024-05-10T10:33:39.553405+0000","last_fullsized":"2024-05-10T10:33:39.553405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:29.644124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640952+0000","last_change":"2024-05-10T10:33:34.508376+0000","last_active":"2024-05-10T10:33:39.640952+0000","last_peered":"2024-05-10T10:33:39.640952+0000","last_clean":"2024-05-10T10:33:39.640952+0000","last_became_active":"2024-05-10T10:33:34.507993+0000","last_became_peered":"2024-05-10T10:33:34.507993+0000","last_unstale":"2024-05-10T10:33:39.640952+0000","last_undegraded":"2024-05-10T10:33:39.640952+0000","last_fullsized":"2024-05-10T10:33:39.640952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885895+0000","last_change":"2024-05-10T10:33:34.508840+0000","last_active":"2024-05-10T10:33:39.885895+0000","last_peered":"2024-05-10T10:33:39.885895+0000","last_clean":"2024-05-10T10:33:39.885895+0000","last_became_active":"2024-05-10T10:33:34.508524+0000","last_became_peered":"2024-05-10T10:33:34.508524+0000","last_unstale":"2024-05-10T10:33:39.885895+0000","last_undegraded":"2024-05-10T10:33:39.885895+0000","last_fullsized":"2024-05-10T10:33:39.885895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554603+0000","last_change":"2024-05-10T10:33:34.513208+0000","last_active":"2024-05-10T10:33:39.554603+0000","last_peered":"2024-05-10T10:33:39.554603+0000","last_clean":"2024-05-10T10:33:39.554603+0000","last_became_active":"2024-05-10T10:33:34.512924+0000","last_became_peered":"2024-05-10T10:33:34.512924+0000","last_unstale":"2024-05-10T10:33:39.554603+0000","last_undegraded":"2024-05-10T10:33:39.554603+0000","last_fullsized":"2024-05-10T10:33:39.554603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641017+0000","last_change":"2024-05-10T10:33:34.504050+0000","last_active":"2024-05-10T10:33:39.641017+0000","last_peered":"2024-05-10T10:33:39.641017+0000","last_clean":"2024-05-10T10:33:39.641017+0000","last_became_active":"2024-05-10T10:33:34.503749+0000","last_became_peered":"2024-05-10T10:33:34.503749+0000","last_unstale":"2024-05-10T10:33:39.641017+0000","last_undegraded":"2024-05-10T10:33:39.641017+0000","last_fullsized":"2024-05-10T10:33:39.641017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871417+0000","last_change":"2024-05-10T10:33:34.508259+0000","last_active":"2024-05-10T10:33:40.871417+0000","last_peered":"2024-05-10T10:33:40.871417+0000","last_clean":"2024-05-10T10:33:40.871417+0000","last_became_active":"2024-05-10T10:33:34.507953+0000","last_became_peered":"2024-05-10T10:33:34.507953+0000","last_unstale":"2024-05-10T10:33:40.871417+0000","last_undegraded":"2024-05-10T10:33:40.871417+0000","last_fullsized":"2024-05-10T10:33:40.871417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640457+0000","last_change":"2024-05-10T10:33:34.505284+0000","last_active":"2024-05-10T10:33:39.640457+0000","last_peered":"2024-05-10T10:33:39.640457+0000","last_clean":"2024-05-10T10:33:39.640457+0000","last_became_active":"2024-05-10T10:33:34.505010+0000","last_became_peered":"2024-05-10T10:33:34.505010+0000","last_unstale":"2024-05-10T10:33:39.640457+0000","last_undegraded":"2024-05-10T10:33:39.640457+0000","last_fullsized":"2024-05-10T10:33:39.640457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871555+0000","last_change":"2024-05-10T10:33:34.511213+0000","last_active":"2024-05-10T10:33:40.871555+0000","last_peered":"2024-05-10T10:33:40.871555+0000","last_clean":"2024-05-10T10:33:40.871555+0000","last_became_active":"2024-05-10T10:33:34.509309+0000","last_became_peered":"2024-05-10T10:33:34.509309+0000","last_unstale":"2024-05-10T10:33:40.871555+0000","last_undegraded":"2024-05-10T10:33:40.871555+0000","last_fullsized":"2024-05-10T10:33:40.871555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553124+0000","last_change":"2024-05-10T10:33:34.508899+0000","last_active":"2024-05-10T10:33:39.553124+0000","last_peered":"2024-05-10T10:33:39.553124+0000","last_clean":"2024-05-10T10:33:39.553124+0000","last_became_active":"2024-05-10T10:33:34.508611+0000","last_became_peered":"2024-05-10T10:33:34.508611+0000","last_unstale":"2024-05-10T10:33:39.553124+0000","last_undegraded":"2024-05-10T10:33:39.553124+0000","last_fullsized":"2024-05-10T10:33:39.553124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:03.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553705+0000","last_change":"2024-05-10T10:33:34.520178+0000","last_active":"2024-05-10T10:33:39.553705+0000","last_peered":"2024-05-10T10:33:39.553705+0000","last_clean":"2024-05-10T10:33:39.553705+0000","last_became_active":"2024-05-10T10:33:34.519912+0000","last_became_peered":"2024-05-10T10:33:34.519912+0000","last_unstale":"2024-05-10T10:33:39.553705+0000","last_undegraded":"2024-05-10T10:33:39.553705+0000","last_fullsized":"2024-05-10T10:33:39.553705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554455+0000","last_change":"2024-05-10T10:33:34.515897+0000","last_active":"2024-05-10T10:33:39.554455+0000","last_peered":"2024-05-10T10:33:39.554455+0000","last_clean":"2024-05-10T10:33:39.554455+0000","last_became_active":"2024-05-10T10:33:34.514938+0000","last_became_peered":"2024-05-10T10:33:34.514938+0000","last_unstale":"2024-05-10T10:33:39.554455+0000","last_undegraded":"2024-05-10T10:33:39.554455+0000","last_fullsized":"2024-05-10T10:33:39.554455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:28:07.142173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680141+0000","last_change":"2024-05-10T10:33:34.523364+0000","last_active":"2024-05-10T10:33:39.680141+0000","last_peered":"2024-05-10T10:33:39.680141+0000","last_clean":"2024-05-10T10:33:39.680141+0000","last_became_active":"2024-05-10T10:33:34.522972+0000","last_became_peered":"2024-05-10T10:33:34.522972+0000","last_unstale":"2024-05-10T10:33:39.680141+0000","last_undegraded":"2024-05-10T10:33:39.680141+0000","last_fullsized":"2024-05-10T10:33:39.680141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872257+0000","last_change":"2024-05-10T10:33:34.508555+0000","last_active":"2024-05-10T10:33:40.872257+0000","last_peered":"2024-05-10T10:33:40.872257+0000","last_clean":"2024-05-10T10:33:40.872257+0000","last_became_active":"2024-05-10T10:33:34.508218+0000","last_became_peered":"2024-05-10T10:33:34.508218+0000","last_unstale":"2024-05-10T10:33:40.872257+0000","last_undegraded":"2024-05-10T10:33:40.872257+0000","last_fullsized":"2024-05-10T10:33:40.872257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871300+0000","last_change":"2024-05-10T10:33:34.512706+0000","last_active":"2024-05-10T10:33:40.871300+0000","last_peered":"2024-05-10T10:33:40.871300+0000","last_clean":"2024-05-10T10:33:40.871300+0000","last_became_active":"2024-05-10T10:33:34.511868+0000","last_became_peered":"2024-05-10T10:33:34.511868+0000","last_unstale":"2024-05-10T10:33:40.871300+0000","last_undegraded":"2024-05-10T10:33:40.871300+0000","last_fullsized":"2024-05-10T10:33:40.871300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885490+0000","last_change":"2024-05-10T10:33:34.504724+0000","last_active":"2024-05-10T10:33:39.885490+0000","last_peered":"2024-05-10T10:33:39.885490+0000","last_clean":"2024-05-10T10:33:39.885490+0000","last_became_active":"2024-05-10T10:33:34.504407+0000","last_became_peered":"2024-05-10T10:33:34.504407+0000","last_unstale":"2024-05-10T10:33:39.885490+0000","last_undegraded":"2024-05-10T10:33:39.885490+0000","last_fullsized":"2024-05-10T10:33:39.885490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553396+0000","last_change":"2024-05-10T10:33:34.507001+0000","last_active":"2024-05-10T10:33:39.553396+0000","last_peered":"2024-05-10T10:33:39.553396+0000","last_clean":"2024-05-10T10:33:39.553396+0000","last_became_active":"2024-05-10T10:33:34.506678+0000","last_became_peered":"2024-05-10T10:33:34.506678+0000","last_unstale":"2024-05-10T10:33:39.553396+0000","last_undegraded":"2024-05-10T10:33:39.553396+0000","last_fullsized":"2024-05-10T10:33:39.553396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640530+0000","last_change":"2024-05-10T10:33:34.516536+0000","last_active":"2024-05-10T10:33:39.640530+0000","last_peered":"2024-05-10T10:33:39.640530+0000","last_clean":"2024-05-10T10:33:39.640530+0000","last_became_active":"2024-05-10T10:33:34.515018+0000","last_became_peered":"2024-05-10T10:33:34.515018+0000","last_unstale":"2024-05-10T10:33:39.640530+0000","last_undegraded":"2024-05-10T10:33:39.640530+0000","last_fullsized":"2024-05-10T10:33:39.640530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872943+0000","last_change":"2024-05-10T10:33:34.517149+0000","last_active":"2024-05-10T10:33:40.872943+0000","last_peered":"2024-05-10T10:33:40.872943+0000","last_clean":"2024-05-10T10:33:40.872943+0000","last_became_active":"2024-05-10T10:33:34.516875+0000","last_became_peered":"2024-05-10T10:33:34.516875+0000","last_unstale":"2024-05-10T10:33:40.872943+0000","last_undegraded":"2024-05-10T10:33:40.872943+0000","last_fullsized":"2024-05-10T10:33:40.872943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553216+0000","last_change":"2024-05-10T10:33:34.518781+0000","last_active":"2024-05-10T10:33:39.553216+0000","last_peered":"2024-05-10T10:33:39.553216+0000","last_clean":"2024-05-10T10:33:39.553216+0000","last_became_active":"2024-05-10T10:33:34.518457+0000","last_became_peered":"2024-05-10T10:33:34.518457+0000","last_unstale":"2024-05-10T10:33:39.553216+0000","last_undegraded":"2024-05-10T10:33:39.553216+0000","last_fullsized":"2024-05-10T10:33:39.553216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554503+0000","last_change":"2024-05-10T10:33:34.518753+0000","last_active":"2024-05-10T10:33:39.554503+0000","last_peered":"2024-05-10T10:33:39.554503+0000","last_clean":"2024-05-10T10:33:39.554503+0000","last_became_active":"2024-05-10T10:33:34.518443+0000","last_became_peered":"2024-05-10T10:33:34.518443+0000","last_unstale":"2024-05-10T10:33:39.554503+0000","last_undegraded":"2024-05-10T10:33:39.554503+0000","last_fullsized":"2024-05-10T10:33:39.554503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554243+0000","last_change":"2024-05-10T10:33:34.517952+0000","last_active":"2024-05-10T10:33:39.554243+0000","last_peered":"2024-05-10T10:33:39.554243+0000","last_clean":"2024-05-10T10:33:39.554243+0000","last_became_active":"2024-05-10T10:33:34.517670+0000","last_became_peered":"2024-05-10T10:33:34.517670+0000","last_unstale":"2024-05-10T10:33:39.554243+0000","last_undegraded":"2024-05-10T10:33:39.554243+0000","last_fullsized":"2024-05-10T10:33:39.554243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:57.405468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640217+0000","last_change":"2024-05-10T10:33:34.508563+0000","last_active":"2024-05-10T10:33:39.640217+0000","last_peered":"2024-05-10T10:33:39.640217+0000","last_clean":"2024-05-10T10:33:39.640217+0000","last_became_active":"2024-05-10T10:33:34.508082+0000","last_became_peered":"2024-05-10T10:33:34.508082+0000","last_unstale":"2024-05-10T10:33:39.640217+0000","last_undegraded":"2024-05-10T10:33:39.640217+0000","last_fullsized":"2024-05-10T10:33:39.640217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553714+0000","last_change":"2024-05-10T10:33:34.513986+0000","last_active":"2024-05-10T10:33:39.553714+0000","last_peered":"2024-05-10T10:33:39.553714+0000","last_clean":"2024-05-10T10:33:39.553714+0000","last_became_active":"2024-05-10T10:33:34.513734+0000","last_became_peered":"2024-05-10T10:33:34.513734+0000","last_unstale":"2024-05-10T10:33:39.553714+0000","last_undegraded":"2024-05-10T10:33:39.553714+0000","last_fullsized":"2024-05-10T10:33:39.553714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871457+0000","last_change":"2024-05-10T10:33:34.509577+0000","last_active":"2024-05-10T10:33:40.871457+0000","last_peered":"2024-05-10T10:33:40.871457+0000","last_clean":"2024-05-10T10:33:40.871457+0000","last_became_active":"2024-05-10T10:33:34.508968+0000","last_became_peered":"2024-05-10T10:33:34.508968+0000","last_unstale":"2024-05-10T10:33:40.871457+0000","last_undegraded":"2024-05-10T10:33:40.871457+0000","last_fullsized":"2024-05-10T10:33:40.871457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553509+0000","last_change":"2024-05-10T10:33:34.515157+0000","last_active":"2024-05-10T10:33:39.553509+0000","last_peered":"2024-05-10T10:33:39.553509+0000","last_clean":"2024-05-10T10:33:39.553509+0000","last_became_active":"2024-05-10T10:33:34.514884+0000","last_became_peered":"2024-05-10T10:33:34.514884+0000","last_unstale":"2024-05-10T10:33:39.553509+0000","last_undegraded":"2024-05-10T10:33:39.553509+0000","last_fullsized":"2024-05-10T10:33:39.553509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871783+0000","last_change":"2024-05-10T10:33:34.513846+0000","last_active":"2024-05-10T10:33:40.871783+0000","last_peered":"2024-05-10T10:33:40.871783+0000","last_clean":"2024-05-10T10:33:40.871783+0000","last_became_active":"2024-05-10T10:33:34.513538+0000","last_became_peered":"2024-05-10T10:33:34.513538+0000","last_unstale":"2024-05-10T10:33:40.871783+0000","last_undegraded":"2024-05-10T10:33:40.871783+0000","last_fullsized":"2024-05-10T10:33:40.871783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553166+0000","last_change":"2024-05-10T10:33:34.503972+0000","last_active":"2024-05-10T10:33:39.553166+0000","last_peered":"2024-05-10T10:33:39.553166+0000","last_clean":"2024-05-10T10:33:39.553166+0000","last_became_active":"2024-05-10T10:33:34.501490+0000","last_became_peered":"2024-05-10T10:33:34.501490+0000","last_unstale":"2024-05-10T10:33:39.553166+0000","last_undegraded":"2024-05-10T10:33:39.553166+0000","last_fullsized":"2024-05-10T10:33:39.553166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885974+0000","last_change":"2024-05-10T10:33:34.502166+0000","last_active":"2024-05-10T10:33:39.885974+0000","last_peered":"2024-05-10T10:33:39.885974+0000","last_clean":"2024-05-10T10:33:39.885974+0000","last_became_active":"2024-05-10T10:33:34.501852+0000","last_became_peered":"2024-05-10T10:33:34.501852+0000","last_unstale":"2024-05-10T10:33:39.885974+0000","last_undegraded":"2024-05-10T10:33:39.885974+0000","last_fullsized":"2024-05-10T10:33:39.885974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554036+0000","last_change":"2024-05-10T10:33:34.522785+0000","last_active":"2024-05-10T10:33:39.554036+0000","last_peered":"2024-05-10T10:33:39.554036+0000","last_clean":"2024-05-10T10:33:39.554036+0000","last_became_active":"2024-05-10T10:33:34.522413+0000","last_became_peered":"2024-05-10T10:33:34.522413+0000","last_unstale":"2024-05-10T10:33:39.554036+0000","last_undegraded":"2024-05-10T10:33:39.554036+0000","last_fullsized":"2024-05-10T10:33:39.554036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:33:53.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:33:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.328+0000 7f041a778640 1 Processor -- start 2024-05-10T10:33:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.328+0000 7f041a778640 1 -- start start 2024-05-10T10:33:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.328+0000 7f041a778640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f041415f790 0x7f04140cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.328+0000 7f041a778640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04140d5d30 0x7f04140d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.328+0000 7f041a778640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04140d30b0 con 0x7f041415f3b0 2024-05-10T10:33:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.328+0000 7f041a778640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f04140d3390 con 0x7f04140d5d30 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f041a778640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f04140c8c80 con 0x7f041415f790 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f0418d15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04140d5d30 0x7f04140d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f0418d15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04140d5d30 0x7f04140d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58710/0 (socket says 172.21.15.53:58710) 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f0418d15640 1 -- 172.21.15.53:0/3004133929 learned_addr learned my addr 172.21.15.53:0/3004133929 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f04137fe640 1 --2- 172.21.15.53:0/3004133929 >> v2:172.21.15.135:3300/0 conn(0x7f041415f790 0x7f04140cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f0418d15640 1 -- 172.21.15.53:0/3004133929 >> v2:172.21.15.135:3300/0 conn(0x7f041415f790 msgr2=0x7f04140cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f0418d15640 1 --2- 172.21.15.53:0/3004133929 >> v2:172.21.15.135:3300/0 conn(0x7f041415f790 0x7f04140cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f0418d15640 1 -- 172.21.15.53:0/3004133929 >> v1:172.21.15.135:6789/0 conn(0x7f041415f3b0 legacy=0x7f04140d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f0418d15640 1 -- 172.21.15.53:0/3004133929 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04140c8f60 con 0x7f04140d5d30 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.329+0000 7f04137fe640 1 --2- 172.21.15.53:0/3004133929 >> v2:172.21.15.135:3300/0 conn(0x7f041415f790 0x7f04140cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f0418d15640 1 --2- 172.21.15.53:0/3004133929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f04140d0210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f04140c8c00 tx=0x7f040c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ae9e480b79f53f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f0412ffd640 1 -- 172.21.15.53:0/3004133929 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c036070 con 0x7f04140d5d30 2024-05-10T10:33:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f0412ffd640 1 -- 172.21.15.53:0/3004133929 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c031600 con 0x7f04140d5d30 2024-05-10T10:33:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f0412ffd640 1 -- 172.21.15.53:0/3004133929 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0319d0 con 0x7f04140d5d30 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f041a778640 1 -- 172.21.15.53:0/3004133929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 msgr2=0x7f04140d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f041a778640 1 --2- 172.21.15.53:0/3004133929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f04140d0210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f04140c8c00 tx=0x7f040c02a6f0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f041a778640 1 -- 172.21.15.53:0/3004133929 shutdown_connections 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f041a778640 1 --2- 172.21.15.53:0/3004133929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f04140d0210 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f041a778640 1 --2- 172.21.15.53:0/3004133929 >> v2:172.21.15.135:3300/0 conn(0x7f041415f790 0x7f04140cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.330+0000 7f041a778640 1 -- 172.21.15.53:0/3004133929 >> 172.21.15.53:0/3004133929 conn(0x7f04140d5130 msgr2=0x7f04140d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.331+0000 7f041a778640 1 -- 172.21.15.53:0/3004133929 shutdown_connections 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.331+0000 7f041a778640 1 -- 172.21.15.53:0/3004133929 wait complete. 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.332+0000 7f041a778640 1 Processor -- start 2024-05-10T10:33:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.332+0000 7f041a778640 1 -- start start 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f0414157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f041415f3b0 0x7f04141f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0414154660 0x7f0414161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04140d3f80 con 0x7f04140d5d30 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04140d3e00 con 0x7f041415f3b0 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f04140ca770 con 0x7f0414154660 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f0414157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f0414157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58716/0 (socket says 172.21.15.53:58716) 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 -- 172.21.15.53:0/2206362356 learned_addr learned my addr 172.21.15.53:0/2206362356 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f04137fe640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f041415f3b0 0x7f04141f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0418d15640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0414154660 0x7f0414161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 -- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0414154660 msgr2=0x7f0414161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0414154660 0x7f0414161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 -- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f041415f3b0 msgr2=0x7f04141f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f041415f3b0 0x7f04141f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 -- 172.21.15.53:0/2206362356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414158200 con 0x7f04140d5d30 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0418d15640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0414154660 0x7f0414161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0413fff640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f0414157cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f040400c430 tx=0x7f040400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0410ff9640 1 -- 172.21.15.53:0/2206362356 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404011790 con 0x7f04140d5d30 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0410ff9640 1 -- 172.21.15.53:0/2206362356 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404017070 con 0x7f04140d5d30 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04140cd5c0 con 0x7f04140d5d30 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f0410ff9640 1 -- 172.21.15.53:0/2206362356 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404011bb0 con 0x7f04140d5d30 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.333+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04140cdb80 con 0x7f04140d5d30 2024-05-10T10:33:53.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.334+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041406a220 con 0x7f04140d5d30 2024-05-10T10:33:53.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.335+0000 7f0410ff9640 1 -- 172.21.15.53:0/2206362356 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f040401c460 con 0x7f04140d5d30 2024-05-10T10:33:53.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.336+0000 7f0410ff9640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0400076300 0x7f04000787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.336+0000 7f04137fe640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0400076300 0x7f04000787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.336+0000 7f0410ff9640 1 -- 172.21.15.53:0/2206362356 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f04040977a0 con 0x7f04140d5d30 2024-05-10T10:33:53.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.336+0000 7f04137fe640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0400076300 0x7f04000787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0414155710 tx=0x7f0408008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:53.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.340+0000 7f0410ff9640 1 -- 172.21.15.53:0/2206362356 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0404060dc0 con 0x7f04140d5d30 2024-05-10T10:33:53.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.527+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04140c8a10 con 0x7f0400076300 2024-05-10T10:33:53.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.535+0000 7f0410ff9640 1 -- 172.21.15.53:0/2206362356 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f04140c8a10 con 0x7f0400076300 2024-05-10T10:33:53.540 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:53.543 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:33:53.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.538+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0400076300 msgr2=0x7f04000787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.538+0000 7f041a778640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0400076300 0x7f04000787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0414155710 tx=0x7f0408008040 comp rx=0 tx=0).stop 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.538+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 msgr2=0x7f0414157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.538+0000 7f041a778640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f0414157cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f040400c430 tx=0x7f040400c8f0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 shutdown_connections 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0400076300 0x7f04000787c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0414154660 0x7f0414161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f041415f3b0 0x7f04141f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 --2- 172.21.15.53:0/2206362356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04140d5d30 0x7f0414157cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 >> 172.21.15.53:0/2206362356 conn(0x7f04140d5130 msgr2=0x7f04140cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 shutdown_connections 2024-05-10T10:33:53.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.539+0000 7f041a778640 1 -- 172.21.15.53:0/2206362356 wait complete. 2024-05-10T10:33:53.551 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-05-10T10:33:52.160478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236024,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738696,"statfs":{"total":773094113280,"available":772852424704,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553612+0000","last_change":"2024-05-10T10:33:33.499661+0000","last_active":"2024-05-10T10:33:39.553612+0000","last_peered":"2024-05-10T10:33:39.553612+0000","last_clean":"2024-05-10T10:33:39.553612+0000","last_became_active":"2024-05-10T10:33:33.499370+0000","last_became_peered":"2024-05-10T10:33:33.499370+0000","last_unstale":"2024-05-10T10:33:39.553612+0000","last_undegraded":"2024-05-10T10:33:39.553612+0000","last_fullsized":"2024-05-10T10:33:39.553612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:01:06.424327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641011+0000","last_change":"2024-05-10T10:33:34.515514+0000","last_active":"2024-05-10T10:33:39.641011+0000","last_peered":"2024-05-10T10:33:39.641011+0000","last_clean":"2024-05-10T10:33:39.641011+0000","last_became_active":"2024-05-10T10:33:34.514965+0000","last_became_peered":"2024-05-10T10:33:34.514965+0000","last_unstale":"2024-05-10T10:33:39.641011+0000","last_undegraded":"2024-05-10T10:33:39.641011+0000","last_fullsized":"2024-05-10T10:33:39.641011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640231+0000","last_change":"2024-05-10T10:33:38.535811+0000","last_active":"2024-05-10T10:33:39.640231+0000","last_peered":"2024-05-10T10:33:39.640231+0000","last_clean":"2024-05-10T10:33:39.640231+0000","last_became_active":"2024-05-10T10:33:38.535356+0000","last_became_peered":"2024-05-10T10:33:38.535356+0000","last_unstale":"2024-05-10T10:33:39.640231+0000","last_undegraded":"2024-05-10T10:33:39.640231+0000","last_fullsized":"2024-05-10T10:33:39.640231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641122+0000","last_change":"2024-05-10T10:33:33.494355+0000","last_active":"2024-05-10T10:33:39.641122+0000","last_peered":"2024-05-10T10:33:39.641122+0000","last_clean":"2024-05-10T10:33:39.641122+0000","last_became_active":"2024-05-10T10:33:33.494042+0000","last_became_peered":"2024-05-10T10:33:33.494042+0000","last_unstale":"2024-05-10T10:33:39.641122+0000","last_undegraded":"2024-05-10T10:33:39.641122+0000","last_fullsized":"2024-05-10T10:33:39.641122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553336+0000","last_change":"2024-05-10T10:33:34.514217+0000","last_active":"2024-05-10T10:33:39.553336+0000","last_peered":"2024-05-10T10:33:39.553336+0000","last_clean":"2024-05-10T10:33:39.553336+0000","last_became_active":"2024-05-10T10:33:34.513895+0000","last_became_peered":"2024-05-10T10:33:34.513895+0000","last_unstale":"2024-05-10T10:33:39.553336+0000","last_undegraded":"2024-05-10T10:33:39.553336+0000","last_fullsized":"2024-05-10T10:33:39.553336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872369+0000","last_change":"2024-05-10T10:33:38.542469+0000","last_active":"2024-05-10T10:33:40.872369+0000","last_peered":"2024-05-10T10:33:40.872369+0000","last_clean":"2024-05-10T10:33:40.872369+0000","last_became_active":"2024-05-10T10:33:38.541985+0000","last_became_peered":"2024-05-10T10:33:38.541985+0000","last_unstale":"2024-05-10T10:33:40.872369+0000","last_undegraded":"2024-05-10T10:33:40.872369+0000","last_fullsized":"2024-05-10T10:33:40.872369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:30.304981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554581+0000","last_change":"2024-05-10T10:33:33.500811+0000","last_active":"2024-05-10T10:33:39.554581+0000","last_peered":"2024-05-10T10:33:39.554581+0000","last_clean":"2024-05-10T10:33:39.554581+0000","last_became_active":"2024-05-10T10:33:33.500355+0000","last_became_peered":"2024-05-10T10:33:33.500355+0000","last_unstale":"2024-05-10T10:33:39.554581+0000","last_undegraded":"2024-05-10T10:33:39.554581+0000","last_fullsized":"2024-05-10T10:33:39.554581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871828+0000","last_change":"2024-05-10T10:33:34.502619+0000","last_active":"2024-05-10T10:33:40.871828+0000","last_peered":"2024-05-10T10:33:40.871828+0000","last_clean":"2024-05-10T10:33:40.871828+0000","last_became_active":"2024-05-10T10:33:34.500045+0000","last_became_peered":"2024-05-10T10:33:34.500045+0000","last_unstale":"2024-05-10T10:33:40.871828+0000","last_undegraded":"2024-05-10T10:33:40.871828+0000","last_fullsized":"2024-05-10T10:33:40.871828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872455+0000","last_change":"2024-05-10T10:33:38.543336+0000","last_active":"2024-05-10T10:33:40.872455+0000","last_peered":"2024-05-10T10:33:40.872455+0000","last_clean":"2024-05-10T10:33:40.872455+0000","last_became_active":"2024-05-10T10:33:38.542915+0000","last_became_peered":"2024-05-10T10:33:38.542915+0000","last_unstale":"2024-05-10T10:33:40.872455+0000","last_undegraded":"2024-05-10T10:33:40.872455+0000","last_fullsized":"2024-05-10T10:33:40.872455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:16.959519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553329+0000","last_change":"2024-05-10T10:33:33.499213+0000","last_active":"2024-05-10T10:33:39.553329+0000","last_peered":"2024-05-10T10:33:39.553329+0000","last_clean":"2024-05-10T10:33:39.553329+0000","last_became_active":"2024-05-10T10:33:33.497831+0000","last_became_peered":"2024-05-10T10:33:33.497831+0000","last_unstale":"2024-05-10T10:33:39.553329+0000","last_undegraded":"2024-05-10T10:33:39.553329+0000","last_fullsized":"2024-05-10T10:33:39.553329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:12.309888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871793+0000","last_change":"2024-05-10T10:33:34.508608+0000","last_active":"2024-05-10T10:33:40.871793+0000","last_peered":"2024-05-10T10:33:40.871793+0000","last_clean":"2024-05-10T10:33:40.871793+0000","last_became_active":"2024-05-10T10:33:34.507711+0000","last_became_peered":"2024-05-10T10:33:34.507711+0000","last_unstale":"2024-05-10T10:33:40.871793+0000","last_undegraded":"2024-05-10T10:33:40.871793+0000","last_fullsized":"2024-05-10T10:33:40.871793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872480+0000","last_change":"2024-05-10T10:33:38.534041+0000","last_active":"2024-05-10T10:33:40.872480+0000","last_peered":"2024-05-10T10:33:40.872480+0000","last_clean":"2024-05-10T10:33:40.872480+0000","last_became_active":"2024-05-10T10:33:38.533397+0000","last_became_peered":"2024-05-10T10:33:38.533397+0000","last_unstale":"2024-05-10T10:33:40.872480+0000","last_undegraded":"2024-05-10T10:33:40.872480+0000","last_fullsized":"2024-05-10T10:33:40.872480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:43.886335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553281+0000","last_change":"2024-05-10T10:33:34.504671+0000","last_active":"2024-05-10T10:33:39.553281+0000","last_peered":"2024-05-10T10:33:39.553281+0000","last_clean":"2024-05-10T10:33:39.553281+0000","last_became_active":"2024-05-10T10:33:34.504389+0000","last_became_peered":"2024-05-10T10:33:34.504389+0000","last_unstale":"2024-05-10T10:33:39.553281+0000","last_undegraded":"2024-05-10T10:33:39.553281+0000","last_fullsized":"2024-05-10T10:33:39.553281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553953+0000","last_change":"2024-05-10T10:33:33.499466+0000","last_active":"2024-05-10T10:33:39.553953+0000","last_peered":"2024-05-10T10:33:39.553953+0000","last_clean":"2024-05-10T10:33:39.553953+0000","last_became_active":"2024-05-10T10:33:33.499185+0000","last_became_peered":"2024-05-10T10:33:33.499185+0000","last_unstale":"2024-05-10T10:33:39.553953+0000","last_undegraded":"2024-05-10T10:33:39.553953+0000","last_fullsized":"2024-05-10T10:33:39.553953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:40.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872905+0000","last_change":"2024-05-10T10:33:38.542258+0000","last_active":"2024-05-10T10:33:40.872905+0000","last_peered":"2024-05-10T10:33:40.872905+0000","last_clean":"2024-05-10T10:33:40.872905+0000","last_became_active":"2024-05-10T10:33:38.541808+0000","last_became_peered":"2024-05-10T10:33:38.541808+0000","last_unstale":"2024-05-10T10:33:40.872905+0000","last_undegraded":"2024-05-10T10:33:40.872905+0000","last_fullsized":"2024-05-10T10:33:40.872905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640629+0000","last_change":"2024-05-10T10:33:34.506055+0000","last_active":"2024-05-10T10:33:39.640629+0000","last_peered":"2024-05-10T10:33:39.640629+0000","last_clean":"2024-05-10T10:33:39.640629+0000","last_became_active":"2024-05-10T10:33:34.505770+0000","last_became_peered":"2024-05-10T10:33:34.505770+0000","last_unstale":"2024-05-10T10:33:39.640629+0000","last_undegraded":"2024-05-10T10:33:39.640629+0000","last_fullsized":"2024-05-10T10:33:39.640629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886170+0000","last_change":"2024-05-10T10:33:33.495139+0000","last_active":"2024-05-10T10:33:39.886170+0000","last_peered":"2024-05-10T10:33:39.886170+0000","last_clean":"2024-05-10T10:33:39.886170+0000","last_became_active":"2024-05-10T10:33:33.494836+0000","last_became_peered":"2024-05-10T10:33:33.494836+0000","last_unstale":"2024-05-10T10:33:39.886170+0000","last_undegraded":"2024-05-10T10:33:39.886170+0000","last_fullsized":"2024-05-10T10:33:39.886170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872645+0000","last_change":"2024-05-10T10:33:38.542825+0000","last_active":"2024-05-10T10:33:40.872645+0000","last_peered":"2024-05-10T10:33:40.872645+0000","last_clean":"2024-05-10T10:33:40.872645+0000","last_became_active":"2024-05-10T10:33:38.542378+0000","last_became_peered":"2024-05-10T10:33:38.542378+0000","last_unstale":"2024-05-10T10:33:40.872645+0000","last_undegraded":"2024-05-10T10:33:40.872645+0000","last_fullsized":"2024-05-10T10:33:40.872645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640415+0000","last_change":"2024-05-10T10:33:34.504517+0000","last_active":"2024-05-10T10:33:39.640415+0000","last_peered":"2024-05-10T10:33:39.640415+0000","last_clean":"2024-05-10T10:33:39.640415+0000","last_became_active":"2024-05-10T10:33:34.504168+0000","last_became_peered":"2024-05-10T10:33:34.504168+0000","last_unstale":"2024-05-10T10:33:39.640415+0000","last_undegraded":"2024-05-10T10:33:39.640415+0000","last_fullsized":"2024-05-10T10:33:39.640415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885550+0000","last_change":"2024-05-10T10:33:33.497956+0000","last_active":"2024-05-10T10:33:39.885550+0000","last_peered":"2024-05-10T10:33:39.885550+0000","last_clean":"2024-05-10T10:33:39.885550+0000","last_became_active":"2024-05-10T10:33:33.497234+0000","last_became_peered":"2024-05-10T10:33:33.497234+0000","last_unstale":"2024-05-10T10:33:39.885550+0000","last_undegraded":"2024-05-10T10:33:39.885550+0000","last_fullsized":"2024-05-10T10:33:39.885550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886149+0000","last_change":"2024-05-10T10:33:38.548654+0000","last_active":"2024-05-10T10:33:39.886149+0000","last_peered":"2024-05-10T10:33:39.886149+0000","last_clean":"2024-05-10T10:33:39.886149+0000","last_became_active":"2024-05-10T10:33:38.547914+0000","last_became_peered":"2024-05-10T10:33:38.547914+0000","last_unstale":"2024-05-10T10:33:39.886149+0000","last_undegraded":"2024-05-10T10:33:39.886149+0000","last_fullsized":"2024-05-10T10:33:39.886149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872730+0000","last_change":"2024-05-10T10:33:34.499454+0000","last_active":"2024-05-10T10:33:40.872730+0000","last_peered":"2024-05-10T10:33:40.872730+0000","last_clean":"2024-05-10T10:33:40.872730+0000","last_became_active":"2024-05-10T10:33:34.499154+0000","last_became_peered":"2024-05-10T10:33:34.499154+0000","last_unstale":"2024-05-10T10:33:40.872730+0000","last_undegraded":"2024-05-10T10:33:40.872730+0000","last_fullsized":"2024-05-10T10:33:40.872730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:06.330274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554557+0000","last_change":"2024-05-10T10:33:33.497112+0000","last_active":"2024-05-10T10:33:39.554557+0000","last_peered":"2024-05-10T10:33:39.554557+0000","last_clean":"2024-05-10T10:33:39.554557+0000","last_became_active":"2024-05-10T10:33:33.496737+0000","last_became_peered":"2024-05-10T10:33:33.496737+0000","last_unstale":"2024-05-10T10:33:39.554557+0000","last_undegraded":"2024-05-10T10:33:39.554557+0000","last_fullsized":"2024-05-10T10:33:39.554557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:20.939521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873091+0000","last_change":"2024-05-10T10:33:38.542337+0000","last_active":"2024-05-10T10:33:40.873091+0000","last_peered":"2024-05-10T10:33:40.873091+0000","last_clean":"2024-05-10T10:33:40.873091+0000","last_became_active":"2024-05-10T10:33:38.541897+0000","last_became_peered":"2024-05-10T10:33:38.541897+0000","last_unstale":"2024-05-10T10:33:40.873091+0000","last_undegraded":"2024-05-10T10:33:40.873091+0000","last_fullsized":"2024-05-10T10:33:40.873091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553701+0000","last_change":"2024-05-10T10:33:34.503858+0000","last_active":"2024-05-10T10:33:39.553701+0000","last_peered":"2024-05-10T10:33:39.553701+0000","last_clean":"2024-05-10T10:33:39.553701+0000","last_became_active":"2024-05-10T10:33:34.503581+0000","last_became_peered":"2024-05-10T10:33:34.503581+0000","last_unstale":"2024-05-10T10:33:39.553701+0000","last_undegraded":"2024-05-10T10:33:39.553701+0000","last_fullsized":"2024-05-10T10:33:39.553701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554729+0000","last_change":"2024-05-10T10:33:33.498121+0000","last_active":"2024-05-10T10:33:39.554729+0000","last_peered":"2024-05-10T10:33:39.554729+0000","last_clean":"2024-05-10T10:33:39.554729+0000","last_became_active":"2024-05-10T10:33:33.497834+0000","last_became_peered":"2024-05-10T10:33:33.497834+0000","last_unstale":"2024-05-10T10:33:39.554729+0000","last_undegraded":"2024-05-10T10:33:39.554729+0000","last_fullsized":"2024-05-10T10:33:39.554729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872705+0000","last_change":"2024-05-10T10:33:38.531480+0000","last_active":"2024-05-10T10:33:40.872705+0000","last_peered":"2024-05-10T10:33:40.872705+0000","last_clean":"2024-05-10T10:33:40.872705+0000","last_became_active":"2024-05-10T10:33:38.531023+0000","last_became_peered":"2024-05-10T10:33:38.531023+0000","last_unstale":"2024-05-10T10:33:40.872705+0000","last_undegraded":"2024-05-10T10:33:40.872705+0000","last_fullsized":"2024-05-10T10:33:40.872705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641080+0000","last_change":"2024-05-10T10:33:34.506192+0000","last_active":"2024-05-10T10:33:39.641080+0000","last_peered":"2024-05-10T10:33:39.641080+0000","last_clean":"2024-05-10T10:33:39.641080+0000","last_became_active":"2024-05-10T10:33:34.505914+0000","last_became_peered":"2024-05-10T10:33:34.505914+0000","last_unstale":"2024-05-10T10:33:39.641080+0000","last_undegraded":"2024-05-10T10:33:39.641080+0000","last_fullsized":"2024-05-10T10:33:39.641080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886029+0000","last_change":"2024-05-10T10:33:33.491388+0000","last_active":"2024-05-10T10:33:39.886029+0000","last_peered":"2024-05-10T10:33:39.886029+0000","last_clean":"2024-05-10T10:33:39.886029+0000","last_became_active":"2024-05-10T10:33:33.491061+0000","last_became_peered":"2024-05-10T10:33:33.491061+0000","last_unstale":"2024-05-10T10:33:39.886029+0000","last_undegraded":"2024-05-10T10:33:39.886029+0000","last_fullsized":"2024-05-10T10:33:39.886029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640139+0000","last_change":"2024-05-10T10:33:38.543260+0000","last_active":"2024-05-10T10:33:39.640139+0000","last_peered":"2024-05-10T10:33:39.640139+0000","last_clean":"2024-05-10T10:33:39.640139+0000","last_became_active":"2024-05-10T10:33:38.542885+0000","last_became_peered":"2024-05-10T10:33:38.542885+0000","last_unstale":"2024-05-10T10:33:39.640139+0000","last_undegraded":"2024-05-10T10:33:39.640139+0000","last_fullsized":"2024-05-10T10:33:39.640139+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886114+0000","last_change":"2024-05-10T10:33:34.501069+0000","last_active":"2024-05-10T10:33:39.886114+0000","last_peered":"2024-05-10T10:33:39.886114+0000","last_clean":"2024-05-10T10:33:39.886114+0000","last_became_active":"2024-05-10T10:33:34.500767+0000","last_became_peered":"2024-05-10T10:33:34.500767+0000","last_unstale":"2024-05-10T10:33:39.886114+0000","last_undegraded":"2024-05-10T10:33:39.886114+0000","last_fullsized":"2024-05-10T10:33:39.886114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554556+0000","last_change":"2024-05-10T10:33:33.496126+0000","last_active":"2024-05-10T10:33:39.554556+0000","last_peered":"2024-05-10T10:33:39.554556+0000","last_clean":"2024-05-10T10:33:39.554556+0000","last_became_active":"2024-05-10T10:33:33.495848+0000","last_became_peered":"2024-05-10T10:33:33.495848+0000","last_unstale":"2024-05-10T10:33:39.554556+0000","last_undegraded":"2024-05-10T10:33:39.554556+0000","last_fullsized":"2024-05-10T10:33:39.554556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553188+0000","last_change":"2024-05-10T10:33:38.542612+0000","last_active":"2024-05-10T10:33:39.553188+0000","last_peered":"2024-05-10T10:33:39.553188+0000","last_clean":"2024-05-10T10:33:39.553188+0000","last_became_active":"2024-05-10T10:33:38.542110+0000","last_became_peered":"2024-05-10T10:33:38.542110+0000","last_unstale":"2024-05-10T10:33:39.553188+0000","last_undegraded":"2024-05-10T10:33:39.553188+0000","last_fullsized":"2024-05-10T10:33:39.553188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871909+0000","last_change":"2024-05-10T10:33:34.515283+0000","last_active":"2024-05-10T10:33:40.871909+0000","last_peered":"2024-05-10T10:33:40.871909+0000","last_clean":"2024-05-10T10:33:40.871909+0000","last_became_active":"2024-05-10T10:33:34.515016+0000","last_became_peered":"2024-05-10T10:33:34.515016+0000","last_unstale":"2024-05-10T10:33:40.871909+0000","last_undegraded":"2024-05-10T10:33:40.871909+0000","last_fullsized":"2024-05-10T10:33:40.871909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886059+0000","last_change":"2024-05-10T10:33:33.488825+0000","last_active":"2024-05-10T10:33:39.886059+0000","last_peered":"2024-05-10T10:33:39.886059+0000","last_clean":"2024-05-10T10:33:39.886059+0000","last_became_active":"2024-05-10T10:33:33.488539+0000","last_became_peered":"2024-05-10T10:33:33.488539+0000","last_unstale":"2024-05-10T10:33:39.886059+0000","last_undegraded":"2024-05-10T10:33:39.886059+0000","last_fullsized":"2024-05-10T10:33:39.886059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885597+0000","last_change":"2024-05-10T10:33:38.547163+0000","last_active":"2024-05-10T10:33:39.885597+0000","last_peered":"2024-05-10T10:33:39.885597+0000","last_clean":"2024-05-10T10:33:39.885597+0000","last_became_active":"2024-05-10T10:33:38.546692+0000","last_became_peered":"2024-05-10T10:33:38.546692+0000","last_unstale":"2024-05-10T10:33:39.885597+0000","last_undegraded":"2024-05-10T10:33:39.885597+0000","last_fullsized":"2024-05-10T10:33:39.885597+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872867+0000","last_change":"2024-05-10T10:33:34.505921+0000","last_active":"2024-05-10T10:33:40.872867+0000","last_peered":"2024-05-10T10:33:40.872867+0000","last_clean":"2024-05-10T10:33:40.872867+0000","last_became_active":"2024-05-10T10:33:34.505637+0000","last_became_peered":"2024-05-10T10:33:34.505637+0000","last_unstale":"2024-05-10T10:33:40.872867+0000","last_undegraded":"2024-05-10T10:33:40.872867+0000","last_fullsized":"2024-05-10T10:33:40.872867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871754+0000","last_change":"2024-05-10T10:33:33.485382+0000","last_active":"2024-05-10T10:33:40.871754+0000","last_peered":"2024-05-10T10:33:40.871754+0000","last_clean":"2024-05-10T10:33:40.871754+0000","last_became_active":"2024-05-10T10:33:33.485082+0000","last_became_peered":"2024-05-10T10:33:33.485082+0000","last_unstale":"2024-05-10T10:33:40.871754+0000","last_undegraded":"2024-05-10T10:33:40.871754+0000","last_fullsized":"2024-05-10T10:33:40.871754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885955+0000","last_change":"2024-05-10T10:33:38.541426+0000","last_active":"2024-05-10T10:33:39.885955+0000","last_peered":"2024-05-10T10:33:39.885955+0000","last_clean":"2024-05-10T10:33:39.885955+0000","last_became_active":"2024-05-10T10:33:38.539367+0000","last_became_peered":"2024-05-10T10:33:38.539367+0000","last_unstale":"2024-05-10T10:33:39.885955+0000","last_undegraded":"2024-05-10T10:33:39.885955+0000","last_fullsized":"2024-05-10T10:33:39.885955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885593+0000","last_change":"2024-05-10T10:33:34.501862+0000","last_active":"2024-05-10T10:33:39.885593+0000","last_peered":"2024-05-10T10:33:39.885593+0000","last_clean":"2024-05-10T10:33:39.885593+0000","last_became_active":"2024-05-10T10:33:34.501493+0000","last_became_peered":"2024-05-10T10:33:34.501493+0000","last_unstale":"2024-05-10T10:33:39.885593+0000","last_undegraded":"2024-05-10T10:33:39.885593+0000","last_fullsized":"2024-05-10T10:33:39.885593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679733+0000","last_change":"2024-05-10T10:33:33.495291+0000","last_active":"2024-05-10T10:33:39.679733+0000","last_peered":"2024-05-10T10:33:39.679733+0000","last_clean":"2024-05-10T10:33:39.679733+0000","last_became_active":"2024-05-10T10:33:33.494955+0000","last_became_peered":"2024-05-10T10:33:33.494955+0000","last_unstale":"2024-05-10T10:33:39.679733+0000","last_undegraded":"2024-05-10T10:33:39.679733+0000","last_fullsized":"2024-05-10T10:33:39.679733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886190+0000","last_change":"2024-05-10T10:33:38.539560+0000","last_active":"2024-05-10T10:33:39.886190+0000","last_peered":"2024-05-10T10:33:39.886190+0000","last_clean":"2024-05-10T10:33:39.886190+0000","last_became_active":"2024-05-10T10:33:38.539131+0000","last_became_peered":"2024-05-10T10:33:38.539131+0000","last_unstale":"2024-05-10T10:33:39.886190+0000","last_undegraded":"2024-05-10T10:33:39.886190+0000","last_fullsized":"2024-05-10T10:33:39.886190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552886+0000","last_change":"2024-05-10T10:33:34.510125+0000","last_active":"2024-05-10T10:33:39.552886+0000","last_peered":"2024-05-10T10:33:39.552886+0000","last_clean":"2024-05-10T10:33:39.552886+0000","last_became_active":"2024-05-10T10:33:34.509820+0000","last_became_peered":"2024-05-10T10:33:34.509820+0000","last_unstale":"2024-05-10T10:33:39.552886+0000","last_undegraded":"2024-05-10T10:33:39.552886+0000","last_fullsized":"2024-05-10T10:33:39.552886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554626+0000","last_change":"2024-05-10T10:33:33.499197+0000","last_active":"2024-05-10T10:33:39.554626+0000","last_peered":"2024-05-10T10:33:39.554626+0000","last_clean":"2024-05-10T10:33:39.554626+0000","last_became_active":"2024-05-10T10:33:33.497808+0000","last_became_peered":"2024-05-10T10:33:33.497808+0000","last_unstale":"2024-05-10T10:33:39.554626+0000","last_undegraded":"2024-05-10T10:33:39.554626+0000","last_fullsized":"2024-05-10T10:33:39.554626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:11.901185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553441+0000","last_change":"2024-05-10T10:33:38.544906+0000","last_active":"2024-05-10T10:33:39.553441+0000","last_peered":"2024-05-10T10:33:39.553441+0000","last_clean":"2024-05-10T10:33:39.553441+0000","last_became_active":"2024-05-10T10:33:38.544304+0000","last_became_peered":"2024-05-10T10:33:38.544304+0000","last_unstale":"2024-05-10T10:33:39.553441+0000","last_undegraded":"2024-05-10T10:33:39.553441+0000","last_fullsized":"2024-05-10T10:33:39.553441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:01.443307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640227+0000","last_change":"2024-05-10T10:33:34.513886+0000","last_active":"2024-05-10T10:33:39.640227+0000","last_peered":"2024-05-10T10:33:39.640227+0000","last_clean":"2024-05-10T10:33:39.640227+0000","last_became_active":"2024-05-10T10:33:34.513617+0000","last_became_peered":"2024-05-10T10:33:34.513617+0000","last_unstale":"2024-05-10T10:33:39.640227+0000","last_undegraded":"2024-05-10T10:33:39.640227+0000","last_fullsized":"2024-05-10T10:33:39.640227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871616+0000","last_change":"2024-05-10T10:33:33.489088+0000","last_active":"2024-05-10T10:33:40.871616+0000","last_peered":"2024-05-10T10:33:40.871616+0000","last_clean":"2024-05-10T10:33:40.871616+0000","last_became_active":"2024-05-10T10:33:33.488785+0000","last_became_peered":"2024-05-10T10:33:33.488785+0000","last_unstale":"2024-05-10T10:33:40.871616+0000","last_undegraded":"2024-05-10T10:33:40.871616+0000","last_fullsized":"2024-05-10T10:33:40.871616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:06:14.764873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554687+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:33:39.554687+0000","last_peered":"2024-05-10T10:33:39.554687+0000","last_clean":"2024-05-10T10:33:39.554687+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:33:39.554687+0000","last_undegraded":"2024-05-10T10:33:39.554687+0000","last_fullsized":"2024-05-10T10:33:39.554687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553037+0000","last_change":"2024-05-10T10:33:38.538561+0000","last_active":"2024-05-10T10:33:39.553037+0000","last_peered":"2024-05-10T10:33:39.553037+0000","last_clean":"2024-05-10T10:33:39.553037+0000","last_became_active":"2024-05-10T10:33:38.538159+0000","last_became_peered":"2024-05-10T10:33:38.538159+0000","last_unstale":"2024-05-10T10:33:39.553037+0000","last_undegraded":"2024-05-10T10:33:39.553037+0000","last_fullsized":"2024-05-10T10:33:39.553037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:54.812506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554231+0000","last_change":"2024-05-10T10:33:34.517851+0000","last_active":"2024-05-10T10:33:39.554231+0000","last_peered":"2024-05-10T10:33:39.554231+0000","last_clean":"2024-05-10T10:33:39.554231+0000","last_became_active":"2024-05-10T10:33:34.517556+0000","last_became_peered":"2024-05-10T10:33:34.517556+0000","last_unstale":"2024-05-10T10:33:39.554231+0000","last_undegraded":"2024-05-10T10:33:39.554231+0000","last_fullsized":"2024-05-10T10:33:39.554231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:46.690299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680077+0000","last_change":"2024-05-10T10:33:33.488242+0000","last_active":"2024-05-10T10:33:39.680077+0000","last_peered":"2024-05-10T10:33:39.680077+0000","last_clean":"2024-05-10T10:33:39.680077+0000","last_became_active":"2024-05-10T10:33:33.487932+0000","last_became_peered":"2024-05-10T10:33:33.487932+0000","last_unstale":"2024-05-10T10:33:39.680077+0000","last_undegraded":"2024-05-10T10:33:39.680077+0000","last_fullsized":"2024-05-10T10:33:39.680077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552940+0000","last_change":"2024-05-10T10:33:38.534596+0000","last_active":"2024-05-10T10:33:39.552940+0000","last_peered":"2024-05-10T10:33:39.552940+0000","last_clean":"2024-05-10T10:33:39.552940+0000","last_became_active":"2024-05-10T10:33:38.534112+0000","last_became_peered":"2024-05-10T10:33:38.534112+0000","last_unstale":"2024-05-10T10:33:39.552940+0000","last_undegraded":"2024-05-10T10:33:39.552940+0000","last_fullsized":"2024-05-10T10:33:39.552940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:53.660814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871944+0000","last_change":"2024-05-10T10:33:34.514478+0000","last_active":"2024-05-10T10:33:40.871944+0000","last_peered":"2024-05-10T10:33:40.871944+0000","last_clean":"2024-05-10T10:33:40.871944+0000","last_became_active":"2024-05-10T10:33:34.514160+0000","last_became_peered":"2024-05-10T10:33:34.514160+0000","last_unstale":"2024-05-10T10:33:40.871944+0000","last_undegraded":"2024-05-10T10:33:40.871944+0000","last_fullsized":"2024-05-10T10:33:40.871944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.886005+0000","last_change":"2024-05-10T10:33:33.499523+0000","last_active":"2024-05-10T10:33:39.886005+0000","last_peered":"2024-05-10T10:33:39.886005+0000","last_clean":"2024-05-10T10:33:39.886005+0000","last_became_active":"2024-05-10T10:33:33.499250+0000","last_became_peered":"2024-05-10T10:33:33.499250+0000","last_unstale":"2024-05-10T10:33:39.886005+0000","last_undegraded":"2024-05-10T10:33:39.886005+0000","last_fullsized":"2024-05-10T10:33:39.886005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:05.325327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553375+0000","last_change":"2024-05-10T10:33:38.543483+0000","last_active":"2024-05-10T10:33:39.553375+0000","last_peered":"2024-05-10T10:33:39.553375+0000","last_clean":"2024-05-10T10:33:39.553375+0000","last_became_active":"2024-05-10T10:33:38.543060+0000","last_became_peered":"2024-05-10T10:33:38.543060+0000","last_unstale":"2024-05-10T10:33:39.553375+0000","last_undegraded":"2024-05-10T10:33:39.553375+0000","last_fullsized":"2024-05-10T10:33:39.553375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:37.163177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554354+0000","last_change":"2024-05-10T10:33:34.522046+0000","last_active":"2024-05-10T10:33:39.554354+0000","last_peered":"2024-05-10T10:33:39.554354+0000","last_clean":"2024-05-10T10:33:39.554354+0000","last_became_active":"2024-05-10T10:33:34.521739+0000","last_became_peered":"2024-05-10T10:33:34.521739+0000","last_unstale":"2024-05-10T10:33:39.554354+0000","last_undegraded":"2024-05-10T10:33:39.554354+0000","last_fullsized":"2024-05-10T10:33:39.554354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:15:15.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680220+0000","last_change":"2024-05-10T10:33:33.495634+0000","last_active":"2024-05-10T10:33:39.680220+0000","last_peered":"2024-05-10T10:33:39.680220+0000","last_clean":"2024-05-10T10:33:39.680220+0000","last_became_active":"2024-05-10T10:33:33.494432+0000","last_became_peered":"2024-05-10T10:33:33.494432+0000","last_unstale":"2024-05-10T10:33:39.680220+0000","last_undegraded":"2024-05-10T10:33:39.680220+0000","last_fullsized":"2024-05-10T10:33:39.680220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871284+0000","last_change":"2024-05-10T10:33:38.528059+0000","last_active":"2024-05-10T10:33:40.871284+0000","last_peered":"2024-05-10T10:33:40.871284+0000","last_clean":"2024-05-10T10:33:40.871284+0000","last_became_active":"2024-05-10T10:33:38.527628+0000","last_became_peered":"2024-05-10T10:33:38.527628+0000","last_unstale":"2024-05-10T10:33:40.871284+0000","last_undegraded":"2024-05-10T10:33:40.871284+0000","last_fullsized":"2024-05-10T10:33:40.871284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:51.676729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554210+0000","last_change":"2024-05-10T10:33:34.519810+0000","last_active":"2024-05-10T10:33:39.554210+0000","last_peered":"2024-05-10T10:33:39.554210+0000","last_clean":"2024-05-10T10:33:39.554210+0000","last_became_active":"2024-05-10T10:33:34.519517+0000","last_became_peered":"2024-05-10T10:33:34.519517+0000","last_unstale":"2024-05-10T10:33:39.554210+0000","last_undegraded":"2024-05-10T10:33:39.554210+0000","last_fullsized":"2024-05-10T10:33:39.554210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.873138+0000","last_change":"2024-05-10T10:33:33.494974+0000","last_active":"2024-05-10T10:33:40.873138+0000","last_peered":"2024-05-10T10:33:40.873138+0000","last_clean":"2024-05-10T10:33:40.873138+0000","last_became_active":"2024-05-10T10:33:33.494667+0000","last_became_peered":"2024-05-10T10:33:33.494667+0000","last_unstale":"2024-05-10T10:33:40.873138+0000","last_undegraded":"2024-05-10T10:33:40.873138+0000","last_fullsized":"2024-05-10T10:33:40.873138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:54.706371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872313+0000","last_change":"2024-05-10T10:33:38.542343+0000","last_active":"2024-05-10T10:33:40.872313+0000","last_peered":"2024-05-10T10:33:40.872313+0000","last_clean":"2024-05-10T10:33:40.872313+0000","last_became_active":"2024-05-10T10:33:38.541842+0000","last_became_peered":"2024-05-10T10:33:38.541842+0000","last_unstale":"2024-05-10T10:33:40.872313+0000","last_undegraded":"2024-05-10T10:33:40.872313+0000","last_fullsized":"2024-05-10T10:33:40.872313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:32.462562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553822+0000","last_change":"2024-05-10T10:33:34.520528+0000","last_active":"2024-05-10T10:33:39.553822+0000","last_peered":"2024-05-10T10:33:39.553822+0000","last_clean":"2024-05-10T10:33:39.553822+0000","last_became_active":"2024-05-10T10:33:34.520239+0000","last_became_peered":"2024-05-10T10:33:34.520239+0000","last_unstale":"2024-05-10T10:33:39.553822+0000","last_undegraded":"2024-05-10T10:33:39.553822+0000","last_fullsized":"2024-05-10T10:33:39.553822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679540+0000","last_change":"2024-05-10T10:33:33.492349+0000","last_active":"2024-05-10T10:33:39.679540+0000","last_peered":"2024-05-10T10:33:39.679540+0000","last_clean":"2024-05-10T10:33:39.679540+0000","last_became_active":"2024-05-10T10:33:33.492019+0000","last_became_peered":"2024-05-10T10:33:33.492019+0000","last_unstale":"2024-05-10T10:33:39.679540+0000","last_undegraded":"2024-05-10T10:33:39.679540+0000","last_fullsized":"2024-05-10T10:33:39.679540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553319+0000","last_change":"2024-05-10T10:33:38.538548+0000","last_active":"2024-05-10T10:33:39.553319+0000","last_peered":"2024-05-10T10:33:39.553319+0000","last_clean":"2024-05-10T10:33:39.553319+0000","last_became_active":"2024-05-10T10:33:38.538144+0000","last_became_peered":"2024-05-10T10:33:38.538144+0000","last_unstale":"2024-05-10T10:33:39.553319+0000","last_undegraded":"2024-05-10T10:33:39.553319+0000","last_fullsized":"2024-05-10T10:33:39.553319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:50.728138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553004+0000","last_change":"2024-05-10T10:33:34.510224+0000","last_active":"2024-05-10T10:33:39.553004+0000","last_peered":"2024-05-10T10:33:39.553004+0000","last_clean":"2024-05-10T10:33:39.553004+0000","last_became_active":"2024-05-10T10:33:34.509584+0000","last_became_peered":"2024-05-10T10:33:34.509584+0000","last_unstale":"2024-05-10T10:33:39.553004+0000","last_undegraded":"2024-05-10T10:33:39.553004+0000","last_fullsized":"2024-05-10T10:33:39.553004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553209+0000","last_change":"2024-05-10T10:33:33.496606+0000","last_active":"2024-05-10T10:33:39.553209+0000","last_peered":"2024-05-10T10:33:39.553209+0000","last_clean":"2024-05-10T10:33:39.553209+0000","last_became_active":"2024-05-10T10:33:33.496326+0000","last_became_peered":"2024-05-10T10:33:33.496326+0000","last_unstale":"2024-05-10T10:33:39.553209+0000","last_undegraded":"2024-05-10T10:33:39.553209+0000","last_fullsized":"2024-05-10T10:33:39.553209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641045+0000","last_change":"2024-05-10T10:33:38.539948+0000","last_active":"2024-05-10T10:33:39.641045+0000","last_peered":"2024-05-10T10:33:39.641045+0000","last_clean":"2024-05-10T10:33:39.641045+0000","last_became_active":"2024-05-10T10:33:38.539181+0000","last_became_peered":"2024-05-10T10:33:38.539181+0000","last_unstale":"2024-05-10T10:33:39.641045+0000","last_undegraded":"2024-05-10T10:33:39.641045+0000","last_fullsized":"2024-05-10T10:33:39.641045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:06:03.969211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554079+0000","last_change":"2024-05-10T10:33:34.519826+0000","last_active":"2024-05-10T10:33:39.554079+0000","last_peered":"2024-05-10T10:33:39.554079+0000","last_clean":"2024-05-10T10:33:39.554079+0000","last_became_active":"2024-05-10T10:33:34.519540+0000","last_became_peered":"2024-05-10T10:33:34.519540+0000","last_unstale":"2024-05-10T10:33:39.554079+0000","last_undegraded":"2024-05-10T10:33:39.554079+0000","last_fullsized":"2024-05-10T10:33:39.554079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871721+0000","last_change":"2024-05-10T10:33:33.494936+0000","last_active":"2024-05-10T10:33:40.871721+0000","last_peered":"2024-05-10T10:33:40.871721+0000","last_clean":"2024-05-10T10:33:40.871721+0000","last_became_active":"2024-05-10T10:33:33.493154+0000","last_became_peered":"2024-05-10T10:33:33.493154+0000","last_unstale":"2024-05-10T10:33:40.871721+0000","last_undegraded":"2024-05-10T10:33:40.871721+0000","last_fullsized":"2024-05-10T10:33:40.871721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640822+0000","last_change":"2024-05-10T10:33:38.544918+0000","last_active":"2024-05-10T10:33:39.640822+0000","last_peered":"2024-05-10T10:33:39.640822+0000","last_clean":"2024-05-10T10:33:39.640822+0000","last_became_active":"2024-05-10T10:33:38.544423+0000","last_became_peered":"2024-05-10T10:33:38.544423+0000","last_unstale":"2024-05-10T10:33:39.640822+0000","last_undegraded":"2024-05-10T10:33:39.640822+0000","last_fullsized":"2024-05-10T10:33:39.640822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:22.017984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553482+0000","last_change":"2024-05-10T10:33:34.506856+0000","last_active":"2024-05-10T10:33:39.553482+0000","last_peered":"2024-05-10T10:33:39.553482+0000","last_clean":"2024-05-10T10:33:39.553482+0000","last_became_active":"2024-05-10T10:33:34.506563+0000","last_became_peered":"2024-05-10T10:33:34.506563+0000","last_unstale":"2024-05-10T10:33:39.553482+0000","last_undegraded":"2024-05-10T10:33:39.553482+0000","last_fullsized":"2024-05-10T10:33:39.553482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:21.944467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553658+0000","last_change":"2024-05-10T10:33:33.496358+0000","last_active":"2024-05-10T10:33:39.553658+0000","last_peered":"2024-05-10T10:33:39.553658+0000","last_clean":"2024-05-10T10:33:39.553658+0000","last_became_active":"2024-05-10T10:33:33.495968+0000","last_became_peered":"2024-05-10T10:33:33.495968+0000","last_unstale":"2024-05-10T10:33:39.553658+0000","last_undegraded":"2024-05-10T10:33:39.553658+0000","last_fullsized":"2024-05-10T10:33:39.553658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:14.817696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871851+0000","last_change":"2024-05-10T10:33:38.534046+0000","last_active":"2024-05-10T10:33:40.871851+0000","last_peered":"2024-05-10T10:33:40.871851+0000","last_clean":"2024-05-10T10:33:40.871851+0000","last_became_active":"2024-05-10T10:33:38.533660+0000","last_became_peered":"2024-05-10T10:33:38.533660+0000","last_unstale":"2024-05-10T10:33:40.871851+0000","last_undegraded":"2024-05-10T10:33:40.871851+0000","last_fullsized":"2024-05-10T10:33:40.871851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:01.662958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554110+0000","last_change":"2024-05-10T10:33:34.516752+0000","last_active":"2024-05-10T10:33:39.554110+0000","last_peered":"2024-05-10T10:33:39.554110+0000","last_clean":"2024-05-10T10:33:39.554110+0000","last_became_active":"2024-05-10T10:33:34.516410+0000","last_became_peered":"2024-05-10T10:33:34.516410+0000","last_unstale":"2024-05-10T10:33:39.554110+0000","last_undegraded":"2024-05-10T10:33:39.554110+0000","last_fullsized":"2024-05-10T10:33:39.554110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:44.179696+0000","last_change":"2024-05-10T10:33:33.498231+0000","last_active":"2024-05-10T10:33:44.179696+0000","last_peered":"2024-05-10T10:33:44.179696+0000","last_clean":"2024-05-10T10:33:44.179696+0000","last_became_active":"2024-05-10T10:33:33.497918+0000","last_became_peered":"2024-05-10T10:33:33.497918+0000","last_unstale":"2024-05-10T10:33:44.179696+0000","last_undegraded":"2024-05-10T10:33:44.179696+0000","last_fullsized":"2024-05-10T10:33:44.179696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885703+0000","last_change":"2024-05-10T10:33:38.537215+0000","last_active":"2024-05-10T10:33:39.885703+0000","last_peered":"2024-05-10T10:33:39.885703+0000","last_clean":"2024-05-10T10:33:39.885703+0000","last_became_active":"2024-05-10T10:33:38.536826+0000","last_became_peered":"2024-05-10T10:33:38.536826+0000","last_unstale":"2024-05-10T10:33:39.885703+0000","last_undegraded":"2024-05-10T10:33:39.885703+0000","last_fullsized":"2024-05-10T10:33:39.885703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872972+0000","last_change":"2024-05-10T10:33:34.512632+0000","last_active":"2024-05-10T10:33:40.872972+0000","last_peered":"2024-05-10T10:33:40.872972+0000","last_clean":"2024-05-10T10:33:40.872972+0000","last_became_active":"2024-05-10T10:33:34.512337+0000","last_became_peered":"2024-05-10T10:33:34.512337+0000","last_unstale":"2024-05-10T10:33:40.872972+0000","last_undegraded":"2024-05-10T10:33:40.872972+0000","last_fullsized":"2024-05-10T10:33:40.872972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:12.906132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885962+0000","last_change":"2024-05-10T10:33:33.485715+0000","last_active":"2024-05-10T10:33:39.885962+0000","last_peered":"2024-05-10T10:33:39.885962+0000","last_clean":"2024-05-10T10:33:39.885962+0000","last_became_active":"2024-05-10T10:33:33.485400+0000","last_became_peered":"2024-05-10T10:33:33.485400+0000","last_unstale":"2024-05-10T10:33:39.885962+0000","last_undegraded":"2024-05-10T10:33:39.885962+0000","last_fullsized":"2024-05-10T10:33:39.885962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:53.276507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872703+0000","last_change":"2024-05-10T10:33:38.542348+0000","last_active":"2024-05-10T10:33:40.872703+0000","last_peered":"2024-05-10T10:33:40.872703+0000","last_clean":"2024-05-10T10:33:40.872703+0000","last_became_active":"2024-05-10T10:33:38.541943+0000","last_became_peered":"2024-05-10T10:33:38.541943+0000","last_unstale":"2024-05-10T10:33:40.872703+0000","last_undegraded":"2024-05-10T10:33:40.872703+0000","last_fullsized":"2024-05-10T10:33:40.872703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553583+0000","last_change":"2024-05-10T10:33:34.514299+0000","last_active":"2024-05-10T10:33:39.553583+0000","last_peered":"2024-05-10T10:33:39.553583+0000","last_clean":"2024-05-10T10:33:39.553583+0000","last_became_active":"2024-05-10T10:33:34.513977+0000","last_became_peered":"2024-05-10T10:33:34.513977+0000","last_unstale":"2024-05-10T10:33:39.553583+0000","last_undegraded":"2024-05-10T10:33:39.553583+0000","last_fullsized":"2024-05-10T10:33:39.553583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872002+0000","last_change":"2024-05-10T10:33:33.491068+0000","last_active":"2024-05-10T10:33:40.872002+0000","last_peered":"2024-05-10T10:33:40.872002+0000","last_clean":"2024-05-10T10:33:40.872002+0000","last_became_active":"2024-05-10T10:33:33.490801+0000","last_became_peered":"2024-05-10T10:33:33.490801+0000","last_unstale":"2024-05-10T10:33:40.872002+0000","last_undegraded":"2024-05-10T10:33:40.872002+0000","last_fullsized":"2024-05-10T10:33:40.872002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640322+0000","last_change":"2024-05-10T10:33:38.538544+0000","last_active":"2024-05-10T10:33:39.640322+0000","last_peered":"2024-05-10T10:33:39.640322+0000","last_clean":"2024-05-10T10:33:39.640322+0000","last_became_active":"2024-05-10T10:33:38.538176+0000","last_became_peered":"2024-05-10T10:33:38.538176+0000","last_unstale":"2024-05-10T10:33:39.640322+0000","last_undegraded":"2024-05-10T10:33:39.640322+0000","last_fullsized":"2024-05-10T10:33:39.640322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:14.049514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871829+0000","last_change":"2024-05-10T10:33:34.513516+0000","last_active":"2024-05-10T10:33:40.871829+0000","last_peered":"2024-05-10T10:33:40.871829+0000","last_clean":"2024-05-10T10:33:40.871829+0000","last_became_active":"2024-05-10T10:33:34.512746+0000","last_became_peered":"2024-05-10T10:33:34.512746+0000","last_unstale":"2024-05-10T10:33:40.871829+0000","last_undegraded":"2024-05-10T10:33:40.871829+0000","last_fullsized":"2024-05-10T10:33:40.871829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:24.450973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554597+0000","last_change":"2024-05-10T10:33:33.498011+0000","last_active":"2024-05-10T10:33:39.554597+0000","last_peered":"2024-05-10T10:33:39.554597+0000","last_clean":"2024-05-10T10:33:39.554597+0000","last_became_active":"2024-05-10T10:33:33.497732+0000","last_became_peered":"2024-05-10T10:33:33.497732+0000","last_unstale":"2024-05-10T10:33:39.554597+0000","last_undegraded":"2024-05-10T10:33:39.554597+0000","last_fullsized":"2024-05-10T10:33:39.554597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:14:42.203731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640338+0000","last_change":"2024-05-10T10:33:38.537084+0000","last_active":"2024-05-10T10:33:39.640338+0000","last_peered":"2024-05-10T10:33:39.640338+0000","last_clean":"2024-05-10T10:33:39.640338+0000","last_became_active":"2024-05-10T10:33:38.535424+0000","last_became_peered":"2024-05-10T10:33:38.535424+0000","last_unstale":"2024-05-10T10:33:39.640338+0000","last_undegraded":"2024-05-10T10:33:39.640338+0000","last_fullsized":"2024-05-10T10:33:39.640338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:48:24.535051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553257+0000","last_change":"2024-05-10T10:33:38.545099+0000","last_active":"2024-05-10T10:33:39.553257+0000","last_peered":"2024-05-10T10:33:39.553257+0000","last_clean":"2024-05-10T10:33:39.553257+0000","last_became_active":"2024-05-10T10:33:38.544663+0000","last_became_peered":"2024-05-10T10:33:38.544663+0000","last_unstale":"2024-05-10T10:33:39.553257+0000","last_undegraded":"2024-05-10T10:33:39.553257+0000","last_fullsized":"2024-05-10T10:33:39.553257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:28.488813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885419+0000","last_change":"2024-05-10T10:33:34.505691+0000","last_active":"2024-05-10T10:33:39.885419+0000","last_peered":"2024-05-10T10:33:39.885419+0000","last_clean":"2024-05-10T10:33:39.885419+0000","last_became_active":"2024-05-10T10:33:34.505398+0000","last_became_peered":"2024-05-10T10:33:34.505398+0000","last_unstale":"2024-05-10T10:33:39.885419+0000","last_undegraded":"2024-05-10T10:33:39.885419+0000","last_fullsized":"2024-05-10T10:33:39.885419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:20.064310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871432+0000","last_change":"2024-05-10T10:33:33.494574+0000","last_active":"2024-05-10T10:33:40.871432+0000","last_peered":"2024-05-10T10:33:40.871432+0000","last_clean":"2024-05-10T10:33:40.871432+0000","last_became_active":"2024-05-10T10:33:33.494311+0000","last_became_peered":"2024-05-10T10:33:33.494311+0000","last_unstale":"2024-05-10T10:33:40.871432+0000","last_undegraded":"2024-05-10T10:33:40.871432+0000","last_fullsized":"2024-05-10T10:33:40.871432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:22.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554526+0000","last_change":"2024-05-10T10:33:38.537291+0000","last_active":"2024-05-10T10:33:39.554526+0000","last_peered":"2024-05-10T10:33:39.554526+0000","last_clean":"2024-05-10T10:33:39.554526+0000","last_became_active":"2024-05-10T10:33:38.536902+0000","last_became_peered":"2024-05-10T10:33:38.536902+0000","last_unstale":"2024-05-10T10:33:39.554526+0000","last_undegraded":"2024-05-10T10:33:39.554526+0000","last_fullsized":"2024-05-10T10:33:39.554526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:27.486005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640318+0000","last_change":"2024-05-10T10:33:34.505242+0000","last_active":"2024-05-10T10:33:39.640318+0000","last_peered":"2024-05-10T10:33:39.640318+0000","last_clean":"2024-05-10T10:33:39.640318+0000","last_became_active":"2024-05-10T10:33:34.504914+0000","last_became_peered":"2024-05-10T10:33:34.504914+0000","last_unstale":"2024-05-10T10:33:39.640318+0000","last_undegraded":"2024-05-10T10:33:39.640318+0000","last_fullsized":"2024-05-10T10:33:39.640318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640485+0000","last_change":"2024-05-10T10:33:33.498662+0000","last_active":"2024-05-10T10:33:39.640485+0000","last_peered":"2024-05-10T10:33:39.640485+0000","last_clean":"2024-05-10T10:33:39.640485+0000","last_became_active":"2024-05-10T10:33:33.498294+0000","last_became_peered":"2024-05-10T10:33:33.498294+0000","last_unstale":"2024-05-10T10:33:39.640485+0000","last_undegraded":"2024-05-10T10:33:39.640485+0000","last_fullsized":"2024-05-10T10:33:39.640485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:46:34.978938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553501+0000","last_change":"2024-05-10T10:33:33.495684+0000","last_active":"2024-05-10T10:33:39.553501+0000","last_peered":"2024-05-10T10:33:39.553501+0000","last_clean":"2024-05-10T10:33:39.553501+0000","last_became_active":"2024-05-10T10:33:33.495347+0000","last_became_peered":"2024-05-10T10:33:33.495347+0000","last_unstale":"2024-05-10T10:33:39.553501+0000","last_undegraded":"2024-05-10T10:33:39.553501+0000","last_fullsized":"2024-05-10T10:33:39.553501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553543+0000","last_change":"2024-05-10T10:33:34.502038+0000","last_active":"2024-05-10T10:33:39.553543+0000","last_peered":"2024-05-10T10:33:39.553543+0000","last_clean":"2024-05-10T10:33:39.553543+0000","last_became_active":"2024-05-10T10:33:34.501709+0000","last_became_peered":"2024-05-10T10:33:34.501709+0000","last_unstale":"2024-05-10T10:33:39.553543+0000","last_undegraded":"2024-05-10T10:33:39.553543+0000","last_fullsized":"2024-05-10T10:33:39.553543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679952+0000","last_change":"2024-05-10T10:33:38.536172+0000","last_active":"2024-05-10T10:33:39.679952+0000","last_peered":"2024-05-10T10:33:39.679952+0000","last_clean":"2024-05-10T10:33:39.679952+0000","last_became_active":"2024-05-10T10:33:38.535590+0000","last_became_peered":"2024-05-10T10:33:38.535590+0000","last_unstale":"2024-05-10T10:33:39.679952+0000","last_undegraded":"2024-05-10T10:33:39.679952+0000","last_fullsized":"2024-05-10T10:33:39.679952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:07.728619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640673+0000","last_change":"2024-05-10T10:33:33.497950+0000","last_active":"2024-05-10T10:33:39.640673+0000","last_peered":"2024-05-10T10:33:39.640673+0000","last_clean":"2024-05-10T10:33:39.640673+0000","last_became_active":"2024-05-10T10:33:33.497590+0000","last_became_peered":"2024-05-10T10:33:33.497590+0000","last_unstale":"2024-05-10T10:33:39.640673+0000","last_undegraded":"2024-05-10T10:33:39.640673+0000","last_fullsized":"2024-05-10T10:33:39.640673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:32.467025+0000","last_clean_scrub_stamp":"2024-05-10T10:33:32.467025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:04.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872524+0000","last_change":"2024-05-10T10:33:34.508520+0000","last_active":"2024-05-10T10:33:40.872524+0000","last_peered":"2024-05-10T10:33:40.872524+0000","last_clean":"2024-05-10T10:33:40.872524+0000","last_became_active":"2024-05-10T10:33:34.508262+0000","last_became_peered":"2024-05-10T10:33:34.508262+0000","last_unstale":"2024-05-10T10:33:40.872524+0000","last_undegraded":"2024-05-10T10:33:40.872524+0000","last_fullsized":"2024-05-10T10:33:40.872524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:26:54.281454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640675+0000","last_change":"2024-05-10T10:33:38.542608+0000","last_active":"2024-05-10T10:33:39.640675+0000","last_peered":"2024-05-10T10:33:39.640675+0000","last_clean":"2024-05-10T10:33:39.640675+0000","last_became_active":"2024-05-10T10:33:38.540900+0000","last_became_peered":"2024-05-10T10:33:38.540900+0000","last_unstale":"2024-05-10T10:33:39.640675+0000","last_undegraded":"2024-05-10T10:33:39.640675+0000","last_fullsized":"2024-05-10T10:33:39.640675+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:37.509811+0000","last_clean_scrub_stamp":"2024-05-10T10:33:37.509811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:41.505822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.552881+0000","last_change":"2024-05-10T10:33:34.510874+0000","last_active":"2024-05-10T10:33:39.552881+0000","last_peered":"2024-05-10T10:33:39.552881+0000","last_clean":"2024-05-10T10:33:39.552881+0000","last_became_active":"2024-05-10T10:33:34.510623+0000","last_became_peered":"2024-05-10T10:33:34.510623+0000","last_unstale":"2024-05-10T10:33:39.552881+0000","last_undegraded":"2024-05-10T10:33:39.552881+0000","last_fullsized":"2024-05-10T10:33:39.552881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.679646+0000","last_change":"2024-05-10T10:33:34.503232+0000","last_active":"2024-05-10T10:33:39.679646+0000","last_peered":"2024-05-10T10:33:39.679646+0000","last_clean":"2024-05-10T10:33:39.679646+0000","last_became_active":"2024-05-10T10:33:34.502932+0000","last_became_peered":"2024-05-10T10:33:34.502932+0000","last_unstale":"2024-05-10T10:33:39.679646+0000","last_undegraded":"2024-05-10T10:33:39.679646+0000","last_fullsized":"2024-05-10T10:33:39.679646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:13.314835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871608+0000","last_change":"2024-05-10T10:33:34.516049+0000","last_active":"2024-05-10T10:33:40.871608+0000","last_peered":"2024-05-10T10:33:40.871608+0000","last_clean":"2024-05-10T10:33:40.871608+0000","last_became_active":"2024-05-10T10:33:34.515617+0000","last_became_peered":"2024-05-10T10:33:34.515617+0000","last_unstale":"2024-05-10T10:33:40.871608+0000","last_undegraded":"2024-05-10T10:33:40.871608+0000","last_fullsized":"2024-05-10T10:33:40.871608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553405+0000","last_change":"2024-05-10T10:33:34.508355+0000","last_active":"2024-05-10T10:33:39.553405+0000","last_peered":"2024-05-10T10:33:39.553405+0000","last_clean":"2024-05-10T10:33:39.553405+0000","last_became_active":"2024-05-10T10:33:34.508064+0000","last_became_peered":"2024-05-10T10:33:34.508064+0000","last_unstale":"2024-05-10T10:33:39.553405+0000","last_undegraded":"2024-05-10T10:33:39.553405+0000","last_fullsized":"2024-05-10T10:33:39.553405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:29.644124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640952+0000","last_change":"2024-05-10T10:33:34.508376+0000","last_active":"2024-05-10T10:33:39.640952+0000","last_peered":"2024-05-10T10:33:39.640952+0000","last_clean":"2024-05-10T10:33:39.640952+0000","last_became_active":"2024-05-10T10:33:34.507993+0000","last_became_peered":"2024-05-10T10:33:34.507993+0000","last_unstale":"2024-05-10T10:33:39.640952+0000","last_undegraded":"2024-05-10T10:33:39.640952+0000","last_fullsized":"2024-05-10T10:33:39.640952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885895+0000","last_change":"2024-05-10T10:33:34.508840+0000","last_active":"2024-05-10T10:33:39.885895+0000","last_peered":"2024-05-10T10:33:39.885895+0000","last_clean":"2024-05-10T10:33:39.885895+0000","last_became_active":"2024-05-10T10:33:34.508524+0000","last_became_peered":"2024-05-10T10:33:34.508524+0000","last_unstale":"2024-05-10T10:33:39.885895+0000","last_undegraded":"2024-05-10T10:33:39.885895+0000","last_fullsized":"2024-05-10T10:33:39.885895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554603+0000","last_change":"2024-05-10T10:33:34.513208+0000","last_active":"2024-05-10T10:33:39.554603+0000","last_peered":"2024-05-10T10:33:39.554603+0000","last_clean":"2024-05-10T10:33:39.554603+0000","last_became_active":"2024-05-10T10:33:34.512924+0000","last_became_peered":"2024-05-10T10:33:34.512924+0000","last_unstale":"2024-05-10T10:33:39.554603+0000","last_undegraded":"2024-05-10T10:33:39.554603+0000","last_fullsized":"2024-05-10T10:33:39.554603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.641017+0000","last_change":"2024-05-10T10:33:34.504050+0000","last_active":"2024-05-10T10:33:39.641017+0000","last_peered":"2024-05-10T10:33:39.641017+0000","last_clean":"2024-05-10T10:33:39.641017+0000","last_became_active":"2024-05-10T10:33:34.503749+0000","last_became_peered":"2024-05-10T10:33:34.503749+0000","last_unstale":"2024-05-10T10:33:39.641017+0000","last_undegraded":"2024-05-10T10:33:39.641017+0000","last_fullsized":"2024-05-10T10:33:39.641017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871417+0000","last_change":"2024-05-10T10:33:34.508259+0000","last_active":"2024-05-10T10:33:40.871417+0000","last_peered":"2024-05-10T10:33:40.871417+0000","last_clean":"2024-05-10T10:33:40.871417+0000","last_became_active":"2024-05-10T10:33:34.507953+0000","last_became_peered":"2024-05-10T10:33:34.507953+0000","last_unstale":"2024-05-10T10:33:40.871417+0000","last_undegraded":"2024-05-10T10:33:40.871417+0000","last_fullsized":"2024-05-10T10:33:40.871417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640457+0000","last_change":"2024-05-10T10:33:34.505284+0000","last_active":"2024-05-10T10:33:39.640457+0000","last_peered":"2024-05-10T10:33:39.640457+0000","last_clean":"2024-05-10T10:33:39.640457+0000","last_became_active":"2024-05-10T10:33:34.505010+0000","last_became_peered":"2024-05-10T10:33:34.505010+0000","last_unstale":"2024-05-10T10:33:39.640457+0000","last_undegraded":"2024-05-10T10:33:39.640457+0000","last_fullsized":"2024-05-10T10:33:39.640457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:37:50.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871555+0000","last_change":"2024-05-10T10:33:34.511213+0000","last_active":"2024-05-10T10:33:40.871555+0000","last_peered":"2024-05-10T10:33:40.871555+0000","last_clean":"2024-05-10T10:33:40.871555+0000","last_became_active":"2024-05-10T10:33:34.509309+0000","last_became_peered":"2024-05-10T10:33:34.509309+0000","last_unstale":"2024-05-10T10:33:40.871555+0000","last_undegraded":"2024-05-10T10:33:40.871555+0000","last_fullsized":"2024-05-10T10:33:40.871555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:28.424723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553124+0000","last_change":"2024-05-10T10:33:34.508899+0000","last_active":"2024-05-10T10:33:39.553124+0000","last_peered":"2024-05-10T10:33:39.553124+0000","last_clean":"2024-05-10T10:33:39.553124+0000","last_became_active":"2024-05-10T10:33:34.508611+0000","last_became_peered":"2024-05-10T10:33:34.508611+0000","last_unstale":"2024-05-10T10:33:39.553124+0000","last_undegraded":"2024-05-10T10:33:39.553124+0000","last_fullsized":"2024-05-10T10:33:39.553124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:03.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553705+0000","last_change":"2024-05-10T10:33:34.520178+0000","last_active":"2024-05-10T10:33:39.553705+0000","last_peered":"2024-05-10T10:33:39.553705+0000","last_clean":"2024-05-10T10:33:39.553705+0000","last_became_active":"2024-05-10T10:33:34.519912+0000","last_became_peered":"2024-05-10T10:33:34.519912+0000","last_unstale":"2024-05-10T10:33:39.553705+0000","last_undegraded":"2024-05-10T10:33:39.553705+0000","last_fullsized":"2024-05-10T10:33:39.553705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:14:41.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554455+0000","last_change":"2024-05-10T10:33:34.515897+0000","last_active":"2024-05-10T10:33:39.554455+0000","last_peered":"2024-05-10T10:33:39.554455+0000","last_clean":"2024-05-10T10:33:39.554455+0000","last_became_active":"2024-05-10T10:33:34.514938+0000","last_became_peered":"2024-05-10T10:33:34.514938+0000","last_unstale":"2024-05-10T10:33:39.554455+0000","last_undegraded":"2024-05-10T10:33:39.554455+0000","last_fullsized":"2024-05-10T10:33:39.554455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:28:07.142173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.680141+0000","last_change":"2024-05-10T10:33:34.523364+0000","last_active":"2024-05-10T10:33:39.680141+0000","last_peered":"2024-05-10T10:33:39.680141+0000","last_clean":"2024-05-10T10:33:39.680141+0000","last_became_active":"2024-05-10T10:33:34.522972+0000","last_became_peered":"2024-05-10T10:33:34.522972+0000","last_unstale":"2024-05-10T10:33:39.680141+0000","last_undegraded":"2024-05-10T10:33:39.680141+0000","last_fullsized":"2024-05-10T10:33:39.680141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872257+0000","last_change":"2024-05-10T10:33:34.508555+0000","last_active":"2024-05-10T10:33:40.872257+0000","last_peered":"2024-05-10T10:33:40.872257+0000","last_clean":"2024-05-10T10:33:40.872257+0000","last_became_active":"2024-05-10T10:33:34.508218+0000","last_became_peered":"2024-05-10T10:33:34.508218+0000","last_unstale":"2024-05-10T10:33:40.872257+0000","last_undegraded":"2024-05-10T10:33:40.872257+0000","last_fullsized":"2024-05-10T10:33:40.872257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:42:33.125338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871300+0000","last_change":"2024-05-10T10:33:34.512706+0000","last_active":"2024-05-10T10:33:40.871300+0000","last_peered":"2024-05-10T10:33:40.871300+0000","last_clean":"2024-05-10T10:33:40.871300+0000","last_became_active":"2024-05-10T10:33:34.511868+0000","last_became_peered":"2024-05-10T10:33:34.511868+0000","last_unstale":"2024-05-10T10:33:40.871300+0000","last_undegraded":"2024-05-10T10:33:40.871300+0000","last_fullsized":"2024-05-10T10:33:40.871300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:56.039973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885490+0000","last_change":"2024-05-10T10:33:34.504724+0000","last_active":"2024-05-10T10:33:39.885490+0000","last_peered":"2024-05-10T10:33:39.885490+0000","last_clean":"2024-05-10T10:33:39.885490+0000","last_became_active":"2024-05-10T10:33:34.504407+0000","last_became_peered":"2024-05-10T10:33:34.504407+0000","last_unstale":"2024-05-10T10:33:39.885490+0000","last_undegraded":"2024-05-10T10:33:39.885490+0000","last_fullsized":"2024-05-10T10:33:39.885490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553396+0000","last_change":"2024-05-10T10:33:34.507001+0000","last_active":"2024-05-10T10:33:39.553396+0000","last_peered":"2024-05-10T10:33:39.553396+0000","last_clean":"2024-05-10T10:33:39.553396+0000","last_became_active":"2024-05-10T10:33:34.506678+0000","last_became_peered":"2024-05-10T10:33:34.506678+0000","last_unstale":"2024-05-10T10:33:39.553396+0000","last_undegraded":"2024-05-10T10:33:39.553396+0000","last_fullsized":"2024-05-10T10:33:39.553396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:12.681838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640530+0000","last_change":"2024-05-10T10:33:34.516536+0000","last_active":"2024-05-10T10:33:39.640530+0000","last_peered":"2024-05-10T10:33:39.640530+0000","last_clean":"2024-05-10T10:33:39.640530+0000","last_became_active":"2024-05-10T10:33:34.515018+0000","last_became_peered":"2024-05-10T10:33:34.515018+0000","last_unstale":"2024-05-10T10:33:39.640530+0000","last_undegraded":"2024-05-10T10:33:39.640530+0000","last_fullsized":"2024-05-10T10:33:39.640530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:39.848496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.872943+0000","last_change":"2024-05-10T10:33:34.517149+0000","last_active":"2024-05-10T10:33:40.872943+0000","last_peered":"2024-05-10T10:33:40.872943+0000","last_clean":"2024-05-10T10:33:40.872943+0000","last_became_active":"2024-05-10T10:33:34.516875+0000","last_became_peered":"2024-05-10T10:33:34.516875+0000","last_unstale":"2024-05-10T10:33:40.872943+0000","last_undegraded":"2024-05-10T10:33:40.872943+0000","last_fullsized":"2024-05-10T10:33:40.872943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:20:41.440441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553216+0000","last_change":"2024-05-10T10:33:34.518781+0000","last_active":"2024-05-10T10:33:39.553216+0000","last_peered":"2024-05-10T10:33:39.553216+0000","last_clean":"2024-05-10T10:33:39.553216+0000","last_became_active":"2024-05-10T10:33:34.518457+0000","last_became_peered":"2024-05-10T10:33:34.518457+0000","last_unstale":"2024-05-10T10:33:39.553216+0000","last_undegraded":"2024-05-10T10:33:39.553216+0000","last_fullsized":"2024-05-10T10:33:39.553216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554503+0000","last_change":"2024-05-10T10:33:34.518753+0000","last_active":"2024-05-10T10:33:39.554503+0000","last_peered":"2024-05-10T10:33:39.554503+0000","last_clean":"2024-05-10T10:33:39.554503+0000","last_became_active":"2024-05-10T10:33:34.518443+0000","last_became_peered":"2024-05-10T10:33:34.518443+0000","last_unstale":"2024-05-10T10:33:39.554503+0000","last_undegraded":"2024-05-10T10:33:39.554503+0000","last_fullsized":"2024-05-10T10:33:39.554503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554243+0000","last_change":"2024-05-10T10:33:34.517952+0000","last_active":"2024-05-10T10:33:39.554243+0000","last_peered":"2024-05-10T10:33:39.554243+0000","last_clean":"2024-05-10T10:33:39.554243+0000","last_became_active":"2024-05-10T10:33:34.517670+0000","last_became_peered":"2024-05-10T10:33:34.517670+0000","last_unstale":"2024-05-10T10:33:39.554243+0000","last_undegraded":"2024-05-10T10:33:39.554243+0000","last_fullsized":"2024-05-10T10:33:39.554243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:46:57.405468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.640217+0000","last_change":"2024-05-10T10:33:34.508563+0000","last_active":"2024-05-10T10:33:39.640217+0000","last_peered":"2024-05-10T10:33:39.640217+0000","last_clean":"2024-05-10T10:33:39.640217+0000","last_became_active":"2024-05-10T10:33:34.508082+0000","last_became_peered":"2024-05-10T10:33:34.508082+0000","last_unstale":"2024-05-10T10:33:39.640217+0000","last_undegraded":"2024-05-10T10:33:39.640217+0000","last_fullsized":"2024-05-10T10:33:39.640217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:33.551065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553714+0000","last_change":"2024-05-10T10:33:34.513986+0000","last_active":"2024-05-10T10:33:39.553714+0000","last_peered":"2024-05-10T10:33:39.553714+0000","last_clean":"2024-05-10T10:33:39.553714+0000","last_became_active":"2024-05-10T10:33:34.513734+0000","last_became_peered":"2024-05-10T10:33:34.513734+0000","last_unstale":"2024-05-10T10:33:39.553714+0000","last_undegraded":"2024-05-10T10:33:39.553714+0000","last_fullsized":"2024-05-10T10:33:39.553714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:12.921680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871457+0000","last_change":"2024-05-10T10:33:34.509577+0000","last_active":"2024-05-10T10:33:40.871457+0000","last_peered":"2024-05-10T10:33:40.871457+0000","last_clean":"2024-05-10T10:33:40.871457+0000","last_became_active":"2024-05-10T10:33:34.508968+0000","last_became_peered":"2024-05-10T10:33:34.508968+0000","last_unstale":"2024-05-10T10:33:40.871457+0000","last_undegraded":"2024-05-10T10:33:40.871457+0000","last_fullsized":"2024-05-10T10:33:40.871457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553509+0000","last_change":"2024-05-10T10:33:34.515157+0000","last_active":"2024-05-10T10:33:39.553509+0000","last_peered":"2024-05-10T10:33:39.553509+0000","last_clean":"2024-05-10T10:33:39.553509+0000","last_became_active":"2024-05-10T10:33:34.514884+0000","last_became_peered":"2024-05-10T10:33:34.514884+0000","last_unstale":"2024-05-10T10:33:39.553509+0000","last_undegraded":"2024-05-10T10:33:39.553509+0000","last_fullsized":"2024-05-10T10:33:39.553509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:49.622975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:40.871783+0000","last_change":"2024-05-10T10:33:34.513846+0000","last_active":"2024-05-10T10:33:40.871783+0000","last_peered":"2024-05-10T10:33:40.871783+0000","last_clean":"2024-05-10T10:33:40.871783+0000","last_became_active":"2024-05-10T10:33:34.513538+0000","last_became_peered":"2024-05-10T10:33:34.513538+0000","last_unstale":"2024-05-10T10:33:40.871783+0000","last_undegraded":"2024-05-10T10:33:40.871783+0000","last_fullsized":"2024-05-10T10:33:40.871783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:36:26.267142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.553166+0000","last_change":"2024-05-10T10:33:34.503972+0000","last_active":"2024-05-10T10:33:39.553166+0000","last_peered":"2024-05-10T10:33:39.553166+0000","last_clean":"2024-05-10T10:33:39.553166+0000","last_became_active":"2024-05-10T10:33:34.501490+0000","last_became_peered":"2024-05-10T10:33:34.501490+0000","last_unstale":"2024-05-10T10:33:39.553166+0000","last_undegraded":"2024-05-10T10:33:39.553166+0000","last_fullsized":"2024-05-10T10:33:39.553166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:11.076815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.885974+0000","last_change":"2024-05-10T10:33:34.502166+0000","last_active":"2024-05-10T10:33:39.885974+0000","last_peered":"2024-05-10T10:33:39.885974+0000","last_clean":"2024-05-10T10:33:39.885974+0000","last_became_active":"2024-05-10T10:33:34.501852+0000","last_became_peered":"2024-05-10T10:33:34.501852+0000","last_unstale":"2024-05-10T10:33:39.885974+0000","last_undegraded":"2024-05-10T10:33:39.885974+0000","last_fullsized":"2024-05-10T10:33:39.885974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:14.242686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T10:33:39.554036+0000","last_change":"2024-05-10T10:33:34.522785+0000","last_active":"2024-05-10T10:33:39.554036+0000","last_peered":"2024-05-10T10:33:39.554036+0000","last_clean":"2024-05-10T10:33:39.554036+0000","last_became_active":"2024-05-10T10:33:34.522413+0000","last_became_peered":"2024-05-10T10:33:34.522413+0000","last_unstale":"2024-05-10T10:33:39.554036+0000","last_undegraded":"2024-05-10T10:33:39.554036+0000","last_fullsized":"2024-05-10T10:33:39.554036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:33.471971+0000","last_clean_scrub_stamp":"2024-05-10T10:33:33.471971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:23.448165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:33:53.556 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:33:53.557 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T10:33:53.557 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-10T10:33:53.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T10:33:53.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.693+0000 7f5a49312640 1 Processor -- start 2024-05-10T10:33:53.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.693+0000 7f5a49312640 1 -- start start 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a49312640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5a44162750 0x7f5a440d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a49312640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5a44162b30 0x7f5a440b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a49312640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a440b45f0 con 0x7f5a440d5cb0 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a49312640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5a440b4f20 con 0x7f5a44162750 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a49312640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5a440d0fe0 con 0x7f5a44162b30 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a42ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5a44162750 0x7f5a440d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a427fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5a440d5cb0 0x7f5a440d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55170/0 (socket says 172.21.15.53:55170) 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a427fc640 1 -- 172.21.15.53:0/1304825545 learned_addr learned my addr 172.21.15.53:0/1304825545 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a437fe640 1 --2- 172.21.15.53:0/1304825545 >> v2:172.21.15.135:3300/0 conn(0x7f5a44162b30 0x7f5a440b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a42ffd640 1 -- 172.21.15.53:0/1304825545 >> v2:172.21.15.135:3300/0 conn(0x7f5a44162b30 msgr2=0x7f5a440b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:53.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a42ffd640 1 --2- 172.21.15.53:0/1304825545 >> v2:172.21.15.135:3300/0 conn(0x7f5a44162b30 0x7f5a440b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a42ffd640 1 -- 172.21.15.53:0/1304825545 >> v1:172.21.15.53:6789/0 conn(0x7f5a440d5cb0 legacy=0x7f5a440d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.694+0000 7f5a42ffd640 1 -- 172.21.15.53:0/1304825545 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a440b48d0 con 0x7f5a44162750 2024-05-10T10:33:53.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.695+0000 7f5a42ffd640 1 --2- 172.21.15.53:0/1304825545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440d0670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d32c0 tx=0x7f5a3402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02f6dc5b2e59418 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:53.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.695+0000 7f5a41ffb640 1 -- 172.21.15.53:0/1304825545 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34037070 con 0x7f5a44162750 2024-05-10T10:33:53.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.695+0000 7f5a41ffb640 1 -- 172.21.15.53:0/1304825545 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a34032440 con 0x7f5a44162750 2024-05-10T10:33:53.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.695+0000 7f5a41ffb640 1 -- 172.21.15.53:0/1304825545 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3402ae60 con 0x7f5a44162750 2024-05-10T10:33:53.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 -- 172.21.15.53:0/1304825545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 msgr2=0x7f5a440d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 --2- 172.21.15.53:0/1304825545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440d0670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d32c0 tx=0x7f5a3402a110 comp rx=0 tx=0).stop 2024-05-10T10:33:53.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 -- 172.21.15.53:0/1304825545 shutdown_connections 2024-05-10T10:33:53.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 --2- 172.21.15.53:0/1304825545 >> v2:172.21.15.135:3300/0 conn(0x7f5a44162b30 0x7f5a440b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 --2- 172.21.15.53:0/1304825545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440d0670 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 -- 172.21.15.53:0/1304825545 >> 172.21.15.53:0/1304825545 conn(0x7f5a4409e3b0 msgr2=0x7f5a440d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:53.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 -- 172.21.15.53:0/1304825545 shutdown_connections 2024-05-10T10:33:53.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.696+0000 7f5a49312640 1 -- 172.21.15.53:0/1304825545 wait complete. 2024-05-10T10:33:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.697+0000 7f5a49312640 1 Processor -- start 2024-05-10T10:33:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.697+0000 7f5a49312640 1 -- start start 2024-05-10T10:33:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a49312640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440ba720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a49312640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a44162b30 0x7f5a44156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a49312640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a441f5300 0x7f5a441f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a49312640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a440d0d30 con 0x7f5a44162750 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a49312640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a440d0bb0 con 0x7f5a44162b30 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a49312640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5a440d3d40 con 0x7f5a441f5300 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440ba720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58736/0 (socket says 172.21.15.53:58736) 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 -- 172.21.15.53:0/152941348 learned_addr learned my addr 172.21.15.53:0/152941348 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a427fc640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a44162b30 0x7f5a44156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a437fe640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a441f5300 0x7f5a441f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 -- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a441f5300 msgr2=0x7f5a441f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a441f5300 0x7f5a441f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 -- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a44162b30 msgr2=0x7f5a44156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a44162b30 0x7f5a44156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 -- 172.21.15.53:0/152941348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a440cd470 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a437fe640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a441f5300 0x7f5a441f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a427fc640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a44162b30 0x7f5a44156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a42ffd640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440ba720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a34000c00 tx=0x7f5a340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.698+0000 7f5a2b7fe640 1 -- 172.21.15.53:0/152941348 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34037040 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.699+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a441576a0 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.699+0000 7f5a2b7fe640 1 -- 172.21.15.53:0/152941348 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a34042070 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.699+0000 7f5a2b7fe640 1 -- 172.21.15.53:0/152941348 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34030bd0 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.699+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a440cceb0 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.699+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4406a220 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.700+0000 7f5a2b7fe640 1 -- 172.21.15.53:0/152941348 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5a34047460 con 0x7f5a44162750 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.701+0000 7f5a2b7fe640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5a300762d0 0x7f5a30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.701+0000 7f5a2b7fe640 1 -- 172.21.15.53:0/152941348 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5a340b8580 con 0x7f5a44162750 2024-05-10T10:33:53.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.701+0000 7f5a427fc640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5a300762d0 0x7f5a30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:53.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.701+0000 7f5a427fc640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5a300762d0 0x7f5a30078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d2350 tx=0x7f5a38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:53.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.705+0000 7f5a2b7fe640 1 -- 172.21.15.53:0/152941348 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5a34032e20 con 0x7f5a44162750 2024-05-10T10:33:53.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.934+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5a44157160 con 0x7f5a44162750 2024-05-10T10:33:53.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.935+0000 7f5a2b7fe640 1 -- 172.21.15.53:0/152941348 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5a34081b20 con 0x7f5a44162750 2024-05-10T10:33:53.936 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:53.936 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T10:33:53.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5a300762d0 msgr2=0x7f5a30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5a300762d0 0x7f5a30078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a440d2350 tx=0x7f5a38009040 comp rx=0 tx=0).stop 2024-05-10T10:33:53.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 msgr2=0x7f5a440ba720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:53.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440ba720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a34000c00 tx=0x7f5a340046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 shutdown_connections 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5a300762d0 0x7f5a30078790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a441f5300 0x7f5a441f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a44162b30 0x7f5a44156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 --2- 172.21.15.53:0/152941348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a44162750 0x7f5a440ba720 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.936+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 >> 172.21.15.53:0/152941348 conn(0x7f5a4409e3b0 msgr2=0x7f5a4406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.937+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 shutdown_connections 2024-05-10T10:33:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:53.937+0000 7f5a49312640 1 -- 172.21.15.53:0/152941348 wait complete. 2024-05-10T10:33:53.949 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-10T10:33:53.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.052+0000 7f6d7b511640 1 Processor -- start 2024-05-10T10:33:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.052+0000 7f6d7b511640 1 -- start start 2024-05-10T10:33:54.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.052+0000 7f6d7b511640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d7b511640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d740d30b0 con 0x7f6d740d5d30 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d7b511640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d740d3390 con 0x7f6d7415f790 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d7b511640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6d740c8c80 con 0x7f6d7415f3b0 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d79aae640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6d740d5d30 0x7f6d740d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55176/0 (socket says 172.21.15.53:55176) 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d79aae640 1 -- 172.21.15.53:0/1159473334 learned_addr learned my addr 172.21.15.53:0/1159473334 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d792ad640 1 --2- 172.21.15.53:0/1159473334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956538600 0 0) 0x7f6d740d30b0 con 0x7f6d740d5d30 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d5c003520 con 0x7f6d740d5d30 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842415766 0 0) 0x7f6d740d3390 con 0x7f6d7415f790 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d740d30b0 con 0x7f6d7415f790 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d792ad640 1 -- 172.21.15.53:0/1159473334 >> v1:172.21.15.135:6789/0 conn(0x7f6d7415f790 legacy=0x7f6d740c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d792ad640 1 -- 172.21.15.53:0/1159473334 >> v1:172.21.15.53:6789/0 conn(0x7f6d740d5d30 legacy=0x7f6d740d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 260842949 0 0) 0x7f6d5c003520 con 0x7f6d740d5d30 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.053+0000 7f6d792ad640 1 -- 172.21.15.53:0/1159473334 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d740c8f60 con 0x7f6d7415f3b0 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.054+0000 7f6d792ad640 1 --2- 172.21.15.53:0/1159473334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740d36f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0097d0 tx=0x7f6d6c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2422d3f6474a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.054+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c037070 con 0x7f6d7415f3b0 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.054+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d6c02ac30 con 0x7f6d7415f3b0 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.054+0000 7f6d73fff640 1 -- 172.21.15.53:0/1159473334 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640019f0 con 0x7f6d7415f3b0 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.055+0000 7f6d7b511640 1 -- 172.21.15.53:0/1159473334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 msgr2=0x7f6d740d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.055+0000 7f6d7b511640 1 --2- 172.21.15.53:0/1159473334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740d36f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0097d0 tx=0x7f6d6c02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.055+0000 7f6d7b511640 1 -- 172.21.15.53:0/1159473334 shutdown_connections 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.055+0000 7f6d7b511640 1 --2- 172.21.15.53:0/1159473334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740d36f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.055+0000 7f6d7b511640 1 -- 172.21.15.53:0/1159473334 >> 172.21.15.53:0/1159473334 conn(0x7f6d740d5130 msgr2=0x7f6d740d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.055+0000 7f6d7b511640 1 -- 172.21.15.53:0/1159473334 shutdown_connections 2024-05-10T10:33:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.055+0000 7f6d7b511640 1 -- 172.21.15.53:0/1159473334 wait complete. 2024-05-10T10:33:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.056+0000 7f6d7b511640 1 Processor -- start 2024-05-10T10:33:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.056+0000 7f6d7b511640 1 -- start start 2024-05-10T10:33:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d7b511640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d740d5d30 0x7f6d740cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d7b511640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d7b511640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d74147e60 0x7f6d74147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d7b511640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740d3db0 con 0x7f6d74147e60 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d7b511640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d740d3c30 con 0x7f6d740d5d30 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d7b511640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6d740d3f30 con 0x7f6d7415f3b0 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d74147e60 0x7f6d74147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d792ad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d740d5d30 0x7f6d740cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d74147e60 0x7f6d74147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58742/0 (socket says 172.21.15.53:58742) 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 -- 172.21.15.53:0/720679973 learned_addr learned my addr 172.21.15.53:0/720679973 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d78aac640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 -- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 msgr2=0x7f6d740b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 -- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d740d5d30 msgr2=0x7f6d740cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d740d5d30 0x7f6d740cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.057+0000 7f6d79aae640 1 -- 172.21.15.53:0/720679973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d74160170 con 0x7f6d74147e60 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.058+0000 7f6d79aae640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d74147e60 0x7f6d74147100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005c00 tx=0x7f6d6400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.058+0000 7f6d71ffb640 1 -- 172.21.15.53:0/720679973 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64017070 con 0x7f6d74147e60 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.058+0000 7f6d71ffb640 1 -- 172.21.15.53:0/720679973 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d64012400 con 0x7f6d74147e60 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.058+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d740ba450 con 0x7f6d74147e60 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.058+0000 7f6d71ffb640 1 -- 172.21.15.53:0/720679973 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640127f0 con 0x7f6d74147e60 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.058+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d740baa10 con 0x7f6d74147e60 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.059+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7406a220 con 0x7f6d74147e60 2024-05-10T10:33:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.060+0000 7f6d71ffb640 1 -- 172.21.15.53:0/720679973 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6d6401a070 con 0x7f6d74147e60 2024-05-10T10:33:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.060+0000 7f6d71ffb640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d600762d0 0x7f6d60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.060+0000 7f6d71ffb640 1 -- 172.21.15.53:0/720679973 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d64097780 con 0x7f6d74147e60 2024-05-10T10:33:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.061+0000 7f6d792ad640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d600762d0 0x7f6d60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.061+0000 7f6d792ad640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d600762d0 0x7f6d60078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c034000 tx=0x7f6d6c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.064+0000 7f6d71ffb640 1 -- 172.21.15.53:0/720679973 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6d640607e0 con 0x7f6d74147e60 2024-05-10T10:33:54.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.295+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d74160340 con 0x7f6d74147e60 2024-05-10T10:33:54.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.295+0000 7f6d71ffb640 1 -- 172.21.15.53:0/720679973 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f6d64064ae0 con 0x7f6d74147e60 2024-05-10T10:33:54.297 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:54.297 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:33:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:33:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d600762d0 msgr2=0x7f6d60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d600762d0 0x7f6d60078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c034000 tx=0x7f6d6c02e000 comp rx=0 tx=0).stop 2024-05-10T10:33:54.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d74147e60 msgr2=0x7f6d74147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d74147e60 0x7f6d74147100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005c00 tx=0x7f6d6400a430 comp rx=0 tx=0).stop 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 shutdown_connections 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d600762d0 0x7f6d60078790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d74147e60 0x7f6d74147100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.297+0000 7f6d7b511640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d7415f3b0 0x7f6d740b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.298+0000 7f6d7b511640 1 --2- 172.21.15.53:0/720679973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d740d5d30 0x7f6d740cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.298+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 >> 172.21.15.53:0/720679973 conn(0x7f6d740d5130 msgr2=0x7f6d740d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.298+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 shutdown_connections 2024-05-10T10:33:54.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.298+0000 7f6d7b511640 1 -- 172.21.15.53:0/720679973 wait complete. 2024-05-10T10:33:54.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:54.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.412+0000 7f1de7675640 1 Processor -- start 2024-05-10T10:33:54.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.412+0000 7f1de7675640 1 -- start start 2024-05-10T10:33:54.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1de7675640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de006c990 0x7f1de0163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1de7675640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1de0163ba0 con 0x7f1de0070bd0 2024-05-10T10:33:54.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1de7675640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1de0160190 con 0x7f1de015f620 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1de7675640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1de006cd70 con 0x7f1de006c990 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1de5c12640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1de0070bd0 0x7f1de00cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55188/0 (socket says 172.21.15.53:55188) 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1de5c12640 1 -- 172.21.15.53:0/852588891 learned_addr learned my addr 172.21.15.53:0/852588891 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1de5411640 1 --2- 172.21.15.53:0/852588891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de006c990 0x7f1de0163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1dd7fff640 1 -- 172.21.15.53:0/852588891 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3814981802 0 0) 0x7f1de0163ba0 con 0x7f1de0070bd0 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.413+0000 7f1dd7fff640 1 -- 172.21.15.53:0/852588891 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dcc003520 con 0x7f1de0070bd0 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.414+0000 7f1dd7fff640 1 -- 172.21.15.53:0/852588891 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263211276 0 0) 0x7f1de0160190 con 0x7f1de015f620 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.414+0000 7f1dd7fff640 1 -- 172.21.15.53:0/852588891 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1de0163ba0 con 0x7f1de015f620 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.414+0000 7f1de5411640 1 -- 172.21.15.53:0/852588891 >> v1:172.21.15.135:6789/0 conn(0x7f1de015f620 legacy=0x7f1de00704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.414+0000 7f1de5411640 1 -- 172.21.15.53:0/852588891 >> v1:172.21.15.53:6789/0 conn(0x7f1de0070bd0 legacy=0x7f1de00cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.414+0000 7f1de5411640 1 -- 172.21.15.53:0/852588891 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de0160470 con 0x7f1de006c990 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.414+0000 7f1de5411640 1 --2- 172.21.15.53:0/852588891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de006c990 0x7f1de0163660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0097d0 tx=0x7f1ddc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb594e435ae0fdf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1dd7fff640 1 -- 172.21.15.53:0/852588891 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc037070 con 0x7f1de006c990 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1dd7fff640 1 -- 172.21.15.53:0/852588891 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc032440 con 0x7f1de006c990 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1dd7fff640 1 -- 172.21.15.53:0/852588891 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc032810 con 0x7f1de006c990 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1de7675640 1 -- 172.21.15.53:0/852588891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de006c990 msgr2=0x7f1de0163660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1de7675640 1 --2- 172.21.15.53:0/852588891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de006c990 0x7f1de0163660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0097d0 tx=0x7f1ddc02a210 comp rx=0 tx=0).stop 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1de7675640 1 -- 172.21.15.53:0/852588891 shutdown_connections 2024-05-10T10:33:54.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1de7675640 1 --2- 172.21.15.53:0/852588891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de006c990 0x7f1de0163660 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1de7675640 1 -- 172.21.15.53:0/852588891 >> 172.21.15.53:0/852588891 conn(0x7f1de00b3980 msgr2=0x7f1de00b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:54.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1de7675640 1 -- 172.21.15.53:0/852588891 shutdown_connections 2024-05-10T10:33:54.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.415+0000 7f1de7675640 1 -- 172.21.15.53:0/852588891 wait complete. 2024-05-10T10:33:54.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.416+0000 7f1de7675640 1 Processor -- start 2024-05-10T10:33:54.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de7675640 1 -- start start 2024-05-10T10:33:54.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de7675640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de006c990 0x7f1de01e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de7675640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de0070bd0 0x7f1de0161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de7675640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1de015f620 0x7f1de0161d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de7675640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0073e70 con 0x7f1de006c990 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de7675640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0163140 con 0x7f1de015f620 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de7675640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1de015fa80 con 0x7f1de0070bd0 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de5411640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de006c990 0x7f1de01e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de5411640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de006c990 0x7f1de01e8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58754/0 (socket says 172.21.15.53:58754) 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.417+0000 7f1de5411640 1 -- 172.21.15.53:0/1993628790 learned_addr learned my addr 172.21.15.53:0/1993628790 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de4c10640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de0070bd0 0x7f1de0161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5c12640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1de015f620 0x7f1de0161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5411640 1 -- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de0070bd0 msgr2=0x7f1de0161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5411640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de0070bd0 0x7f1de0161850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5411640 1 -- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1de015f620 msgr2=0x7f1de0161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5411640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1de015f620 0x7f1de0161d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5411640 1 -- 172.21.15.53:0/1993628790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de00b0720 con 0x7f1de006c990 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de4c10640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de0070bd0 0x7f1de0161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5c12640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1de015f620 0x7f1de0161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de5411640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de006c990 0x7f1de01e8ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0098f0 tx=0x7f1ddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1dd5ffb640 1 -- 172.21.15.53:0/1993628790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc041020 con 0x7f1de006c990 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1dd5ffb640 1 -- 172.21.15.53:0/1993628790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc03d6e0 con 0x7f1de006c990 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de015caa0 con 0x7f1de006c990 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1dd5ffb640 1 -- 172.21.15.53:0/1993628790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc037040 con 0x7f1de006c990 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.418+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de00afb60 con 0x7f1de006c990 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.419+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de006a220 con 0x7f1de006c990 2024-05-10T10:33:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.420+0000 7f1dd5ffb640 1 -- 172.21.15.53:0/1993628790 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ddc044020 con 0x7f1de006c990 2024-05-10T10:33:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.421+0000 7f1dd5ffb640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1dc40762d0 0x7f1dc4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.421+0000 7f1dd5ffb640 1 -- 172.21.15.53:0/1993628790 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1ddc0b80d0 con 0x7f1de006c990 2024-05-10T10:33:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.421+0000 7f1de4c10640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1dc40762d0 0x7f1dc4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.421+0000 7f1de4c10640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1dc40762d0 0x7f1dc4078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1de0162880 tx=0x7f1dd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.425+0000 7f1dd5ffb640 1 -- 172.21.15.53:0/1993628790 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1ddc04b030 con 0x7f1de006c990 2024-05-10T10:33:54.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.658+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1de00b08f0 con 0x7f1de006c990 2024-05-10T10:33:54.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.658+0000 7f1dd5ffb640 1 -- 172.21.15.53:0/1993628790 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f1ddc081660 con 0x7f1de006c990 2024-05-10T10:33:54.660 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:54.660 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:33:54.660 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:33:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1dc40762d0 msgr2=0x7f1dc4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1dc40762d0 0x7f1dc4078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1de0162880 tx=0x7f1dd0009040 comp rx=0 tx=0).stop 2024-05-10T10:33:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de006c990 msgr2=0x7f1de01e8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de006c990 0x7f1de01e8ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0098f0 tx=0x7f1ddc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 shutdown_connections 2024-05-10T10:33:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1dc40762d0 0x7f1dc4078790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1de015f620 0x7f1de0161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1de0070bd0 0x7f1de0161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 --2- 172.21.15.53:0/1993628790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de006c990 0x7f1de01e8ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 >> 172.21.15.53:0/1993628790 conn(0x7f1de00b3980 msgr2=0x7f1de00704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:54.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 shutdown_connections 2024-05-10T10:33:54.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.660+0000 7f1de7675640 1 -- 172.21.15.53:0/1993628790 wait complete. 2024-05-10T10:33:54.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:33:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.775+0000 7fbf7df9e640 1 Processor -- start 2024-05-10T10:33:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.775+0000 7fbf7df9e640 1 -- start start 2024-05-10T10:33:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf7df9e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 0x7fbf780d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf7df9e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf780d0b90 con 0x7fbf780d3bd0 2024-05-10T10:33:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf7df9e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf780b4470 con 0x7fbf7815f1f0 2024-05-10T10:33:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf7df9e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf780d2ff0 con 0x7fbf780d0500 2024-05-10T10:33:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbf780d3bd0 0x7fbf780b4f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55202/0 (socket says 172.21.15.53:55202) 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 -- 172.21.15.53:0/2529937566 learned_addr learned my addr 172.21.15.53:0/2529937566 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf7d79d640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 0x7fbf780d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931173632 0 0) 0x7fbf780d0b90 con 0x7fbf780d3bd0 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf68003520 con 0x7fbf780d3bd0 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1381317695 0 0) 0x7fbf68003520 con 0x7fbf780d3bd0 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 msgr2=0x7fbf780d5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 0x7fbf780d5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 >> v1:172.21.15.135:6789/0 conn(0x7fbf7815f1f0 legacy=0x7fbf780d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf780b4750 con 0x7fbf780d3bd0 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fbf64001690 con 0x7fbf780d3bd0 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 >> v1:172.21.15.53:6789/0 conn(0x7fbf780d3bd0 legacy=0x7fbf780b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf680039e0 0x7fbf68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf68004800 0x7fbf68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 0x7fbf680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780d2ff0 con 0x7fbf68004800 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6800baf0 con 0x7fbf680039e0 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6800bcf0 con 0x7fbf780d0500 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf68004800 0x7fbf68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf7cf9c640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf680039e0 0x7fbf68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf7d79d640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 0x7fbf680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 -- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 msgr2=0x7fbf680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf780d0500 0x7fbf680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 -- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf680039e0 msgr2=0x7fbf68003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf680039e0 0x7fbf68003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.777+0000 7fbf77fff640 1 -- 172.21.15.53:0/2529937566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf68003520 con 0x7fbf68004800 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.778+0000 7fbf77fff640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf68004800 0x7fbf68004c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c160 tx=0x7fbf68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8e64a42aa9b25b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.778+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64007c80 con 0x7fbf68004800 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.778+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64005ad0 con 0x7fbf68004800 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.778+0000 7fbf777fe640 1 -- 172.21.15.53:0/2529937566 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64007070 con 0x7fbf68004800 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.779+0000 7fbf7df9e640 1 -- 172.21.15.53:0/2529937566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf68004800 msgr2=0x7fbf68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.779+0000 7fbf7df9e640 1 --2- 172.21.15.53:0/2529937566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf68004800 0x7fbf68004c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c160 tx=0x7fbf68005830 comp rx=0 tx=0).stop 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.779+0000 7fbf7d79d640 1 -- 172.21.15.53:0/2529937566 reap_dead start 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.779+0000 7fbf7df9e640 1 -- 172.21.15.53:0/2529937566 shutdown_connections 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.779+0000 7fbf7df9e640 1 -- 172.21.15.53:0/2529937566 >> 172.21.15.53:0/2529937566 conn(0x7fbf780d4ee0 msgr2=0x7fbf780d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.779+0000 7fbf7df9e640 1 -- 172.21.15.53:0/2529937566 shutdown_connections 2024-05-10T10:33:54.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.779+0000 7fbf7df9e640 1 -- 172.21.15.53:0/2529937566 wait complete. 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.780+0000 7fbf7df9e640 1 Processor -- start 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.780+0000 7fbf7df9e640 1 -- start start 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7df9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf68004800 0x7fbf780cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7df9e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf78162620 0x7fbf781f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7df9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf781f4030 0x7fbf781f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7df9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780d5b80 con 0x7fbf781f4030 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7df9e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf780d5a00 con 0x7fbf68004800 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7df9e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf780d5d00 con 0x7fbf78162620 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf781f4030 0x7fbf781f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf781f4030 0x7fbf781f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58776/0 (socket says 172.21.15.53:58776) 2024-05-10T10:33:54.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 -- 172.21.15.53:0/289255136 learned_addr learned my addr 172.21.15.53:0/289255136 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf77fff640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf78162620 0x7fbf781f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7cf9c640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf68004800 0x7fbf780cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 -- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf78162620 msgr2=0x7fbf781f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf78162620 0x7fbf781f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 -- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf68004800 msgr2=0x7fbf780cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf68004800 0x7fbf780cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.781+0000 7fbf7d79d640 1 -- 172.21.15.53:0/289255136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf780ccd50 con 0x7fbf781f4030 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.782+0000 7fbf7d79d640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf781f4030 0x7fbf781f44b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf70011c80 tx=0x7fbf7000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.782+0000 7fbf757fa640 1 -- 172.21.15.53:0/289255136 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70015430 con 0x7fbf781f4030 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.782+0000 7fbf757fa640 1 -- 172.21.15.53:0/289255136 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7001d070 con 0x7fbf781f4030 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.782+0000 7fbf757fa640 1 -- 172.21.15.53:0/289255136 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70015810 con 0x7fbf781f4030 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.782+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf780cd030 con 0x7fbf781f4030 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.782+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf780cd5f0 con 0x7fbf781f4030 2024-05-10T10:33:54.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.783+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7806a220 con 0x7fbf781f4030 2024-05-10T10:33:54.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.783+0000 7fbf757fa640 1 -- 172.21.15.53:0/289255136 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf700159b0 con 0x7fbf781f4030 2024-05-10T10:33:54.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.784+0000 7fbf757fa640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf5c0762d0 0x7fbf5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:54.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.784+0000 7fbf757fa640 1 -- 172.21.15.53:0/289255136 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbf70098180 con 0x7fbf781f4030 2024-05-10T10:33:54.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.784+0000 7fbf7cf9c640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf5c0762d0 0x7fbf5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:54.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.786+0000 7fbf7cf9c640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf5c0762d0 0x7fbf5c078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c000c00 tx=0x7fbf6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:54.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.788+0000 7fbf757fa640 1 -- 172.21.15.53:0/289255136 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbf70010040 con 0x7fbf781f4030 2024-05-10T10:33:54.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.978+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf780d3710 con 0x7fbf781f4030 2024-05-10T10:33:54.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.979+0000 7fbf757fa640 1 -- 172.21.15.53:0/289255136 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fbf70061150 con 0x7fbf781f4030 2024-05-10T10:33:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:54.981 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:33:54.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.981+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf5c0762d0 msgr2=0x7fbf5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.981+0000 7fbf7df9e640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf5c0762d0 0x7fbf5c078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c000c00 tx=0x7fbf6c009040 comp rx=0 tx=0).stop 2024-05-10T10:33:54.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.981+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf781f4030 msgr2=0x7fbf781f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:54.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.981+0000 7fbf7df9e640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf781f4030 0x7fbf781f44b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf70011c80 tx=0x7fbf7000f580 comp rx=0 tx=0).stop 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.981+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 shutdown_connections 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.981+0000 7fbf7df9e640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf5c0762d0 0x7fbf5c078790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.981+0000 7fbf7df9e640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf781f4030 0x7fbf781f44b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.982+0000 7fbf7df9e640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf78162620 0x7fbf781f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.982+0000 7fbf7df9e640 1 --2- 172.21.15.53:0/289255136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf68004800 0x7fbf780cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.982+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 >> 172.21.15.53:0/289255136 conn(0x7fbf780d4ee0 msgr2=0x7fbf780d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.982+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 shutdown_connections 2024-05-10T10:33:54.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:54.982+0000 7fbf7df9e640 1 -- 172.21.15.53:0/289255136 wait complete. 2024-05-10T10:33:54.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:33:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.096+0000 7fc19bb04640 1 Processor -- start 2024-05-10T10:33:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19bb04640 1 -- start start 2024-05-10T10:33:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19bb04640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc19415f3b0 0x7fc1940d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19bb04640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc19415f790 0x7fc1940d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19bb04640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1940d5d30 0x7fc1940c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19bb04640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc1940d30b0 con 0x7fc19415f790 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19bb04640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc1940d3390 con 0x7fc19415f3b0 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19bb04640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc1940c8c80 con 0x7fc1940d5d30 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.097+0000 7fc19909f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc19415f790 0x7fc1940d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19909f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc19415f790 0x7fc1940d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58786/0 (socket says 172.21.15.53:58786) 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19909f640 1 -- 172.21.15.53:0/2584397631 learned_addr learned my addr 172.21.15.53:0/2584397631 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc1998a0640 1 --2- 172.21.15.53:0/2584397631 >> v2:172.21.15.135:3300/0 conn(0x7fc19415f3b0 0x7fc1940d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19a0a1640 1 --2- 172.21.15.53:0/2584397631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1940d5d30 0x7fc1940c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19909f640 1 -- 172.21.15.53:0/2584397631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1940d5d30 msgr2=0x7fc1940c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19909f640 1 --2- 172.21.15.53:0/2584397631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1940d5d30 0x7fc1940c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19909f640 1 -- 172.21.15.53:0/2584397631 >> v2:172.21.15.135:3300/0 conn(0x7fc19415f3b0 msgr2=0x7fc1940d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19909f640 1 --2- 172.21.15.53:0/2584397631 >> v2:172.21.15.135:3300/0 conn(0x7fc19415f3b0 0x7fc1940d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.098+0000 7fc19909f640 1 -- 172.21.15.53:0/2584397631 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1940c8f60 con 0x7fc19415f790 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19909f640 1 --2- 172.21.15.53:0/2584397631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19415f790 0x7fc1940d3c30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc19408e820 tx=0x7fc18802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c25f8f8b197ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19889e640 1 -- 172.21.15.53:0/2584397631 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188037070 con 0x7fc19415f790 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19889e640 1 -- 172.21.15.53:0/2584397631 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc188032440 con 0x7fc19415f790 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19889e640 1 -- 172.21.15.53:0/2584397631 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188032810 con 0x7fc19415f790 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 -- 172.21.15.53:0/2584397631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19415f790 msgr2=0x7fc1940d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2584397631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19415f790 0x7fc1940d3c30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc19408e820 tx=0x7fc18802a1e0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 -- 172.21.15.53:0/2584397631 shutdown_connections 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2584397631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1940d5d30 0x7fc1940c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2584397631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19415f790 0x7fc1940d3c30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2584397631 >> v2:172.21.15.135:3300/0 conn(0x7fc19415f3b0 0x7fc1940d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 -- 172.21.15.53:0/2584397631 >> 172.21.15.53:0/2584397631 conn(0x7fc1940d5130 msgr2=0x7fc1940d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 -- 172.21.15.53:0/2584397631 shutdown_connections 2024-05-10T10:33:55.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.099+0000 7fc19bb04640 1 -- 172.21.15.53:0/2584397631 wait complete. 2024-05-10T10:33:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 Processor -- start 2024-05-10T10:33:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 -- start start 2024-05-10T10:33:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1940d5d30 0x7fc194154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc19415f3b0 0x7fc194157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc194157ff0 0x7fc194155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1940c9e20 con 0x7fc1940d5d30 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1940c9ca0 con 0x7fc19415f3b0 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc19bb04640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc1940c9fa0 con 0x7fc194157ff0 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.101+0000 7fc1998a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1940d5d30 0x7fc194154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1940d5d30 0x7fc194154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58802/0 (socket says 172.21.15.53:58802) 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 -- 172.21.15.53:0/2926157656 learned_addr learned my addr 172.21.15.53:0/2926157656 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc19a0a1640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc194157ff0 0x7fc194155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc19909f640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc19415f3b0 0x7fc194157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 -- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc194157ff0 msgr2=0x7fc194155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc194157ff0 0x7fc194155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 -- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc19415f3b0 msgr2=0x7fc194157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc19415f3b0 0x7fc194157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 -- 172.21.15.53:0/2926157656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc194154b70 con 0x7fc1940d5d30 2024-05-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc19909f640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc19415f3b0 0x7fc194157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1998a0640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1940d5d30 0x7fc194154630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c7f0 tx=0x7fc18c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1927fc640 1 -- 172.21.15.53:0/2926157656 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c015070 con 0x7fc1940d5d30 2024-05-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1927fc640 1 -- 172.21.15.53:0/2926157656 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c004240 con 0x7fc1940d5d30 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941584b0 con 0x7fc1940d5d30 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc1927fc640 1 -- 172.21.15.53:0/2926157656 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c007620 con 0x7fc1940d5d30 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.102+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941f5f60 con 0x7fc1940d5d30 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.103+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19406a220 con 0x7fc1940d5d30 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.104+0000 7fc1927fc640 1 -- 172.21.15.53:0/2926157656 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc18c0079d0 con 0x7fc1940d5d30 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.106+0000 7fc1927fc640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1800762d0 0x7fc180078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.106+0000 7fc1927fc640 1 -- 172.21.15.53:0/2926157656 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc18c097970 con 0x7fc1940d5d30 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.106+0000 7fc19909f640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1800762d0 0x7fc180078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.106+0000 7fc19909f640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1800762d0 0x7fc180078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc188034000 tx=0x7fc18802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.110+0000 7fc1927fc640 1 -- 172.21.15.53:0/2926157656 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc194154b70 con 0x7fc1940d5d30 2024-05-10T10:33:55.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.346+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1940d25b0 con 0x7fc1940d5d30 2024-05-10T10:33:55.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.346+0000 7fc1927fc640 1 -- 172.21.15.53:0/2926157656 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7fc18c060f90 con 0x7fc1940d5d30 2024-05-10T10:33:55.349 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:33:55.349 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:33:55.350 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:33:55.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1800762d0 msgr2=0x7fc180078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1800762d0 0x7fc180078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc188034000 tx=0x7fc18802e000 comp rx=0 tx=0).stop 2024-05-10T10:33:55.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1940d5d30 msgr2=0x7fc194154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1940d5d30 0x7fc194154630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c7f0 tx=0x7fc18c00ccb0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 shutdown_connections 2024-05-10T10:33:55.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1800762d0 0x7fc180078790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc194157ff0 0x7fc194155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc19415f3b0 0x7fc194157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 --2- 172.21.15.53:0/2926157656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1940d5d30 0x7fc194154630 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 >> 172.21.15.53:0/2926157656 conn(0x7fc1940d5130 msgr2=0x7fc1940cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 shutdown_connections 2024-05-10T10:33:55.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.348+0000 7fc19bb04640 1 -- 172.21.15.53:0/2926157656 wait complete. 2024-05-10T10:33:55.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:33:33.864840+0000', 'modified': '2024-05-10T10:33:46.950725+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6836/4124737039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 4124737039}, {'type': 'v1', 'addr': '172.21.15.135:6836', 'nonce': 4124737039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4278, 'qdb_cluster': [4278]} 2024-05-10T10:33:55.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:33:55.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T10:33:55.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.869+0000 7f55d17dd640 1 Processor -- start 2024-05-10T10:33:55.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.870+0000 7f55d17dd640 1 -- start start 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.870+0000 7f55d17dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d1aa0 0x7f55cc0d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.870+0000 7f55d17dd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55cc0d3340 0x7f55cc0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.870+0000 7f55d17dd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55cc072330 con 0x7f55cc15d290 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.870+0000 7f55d17dd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f55cc0d37d0 con 0x7f55cc0d3340 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.870+0000 7f55d17dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc071d70 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.870+0000 7f55caffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f55cc15d290 0x7f55cc071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55204/0 (socket says 172.21.15.53:55204) 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55caffd640 1 -- 172.21.15.53:0/3855666279 learned_addr learned my addr 172.21.15.53:0/3855666279 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/3855666279 >> v2:172.21.15.135:3300/0 conn(0x7f55cc0d3340 0x7f55cc0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d1aa0 0x7f55cc0d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55c9ffb640 1 -- 172.21.15.53:0/3855666279 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203015990 0 0) 0x7f55cc072330 con 0x7f55cc15d290 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55c9ffb640 1 -- 172.21.15.53:0/3855666279 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55b8003520 con 0x7f55cc15d290 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55c9ffb640 1 -- 172.21.15.53:0/3855666279 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 526397059 0 0) 0x7f55b8003520 con 0x7f55cc15d290 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55cb7fe640 1 -- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d1aa0 msgr2=0x7f55cc0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d1aa0 0x7f55cc0d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55cb7fe640 1 -- 172.21.15.53:0/3855666279 >> v1:172.21.15.53:6789/0 conn(0x7f55cc15d290 legacy=0x7f55cc071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55cb7fe640 1 -- 172.21.15.53:0/3855666279 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc072050 con 0x7f55cc0d3340 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.871+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d1aa0 0x7f55cc0d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.872+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc0d3340 0x7f55cc0ce270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009d90 tx=0x7f55bc02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972aa50604c82c56 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.872+0000 7f55c9ffb640 1 -- 172.21.15.53:0/3855666279 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4001670 con 0x7f55cc0d3340 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.872+0000 7f55c9ffb640 1 -- 172.21.15.53:0/3855666279 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55bc007540 con 0x7f55cc0d3340 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.872+0000 7f55c9ffb640 1 -- 172.21.15.53:0/3855666279 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc007910 con 0x7f55cc0d3340 2024-05-10T10:33:55.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 -- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc0d3340 msgr2=0x7f55cc0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 --2- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc0d3340 0x7f55cc0ce270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009d90 tx=0x7f55bc02a5f0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 -- 172.21.15.53:0/3855666279 shutdown_connections 2024-05-10T10:33:55.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 --2- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc0d3340 0x7f55cc0ce270 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 --2- 172.21.15.53:0/3855666279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d1aa0 0x7f55cc0d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 -- 172.21.15.53:0/3855666279 >> 172.21.15.53:0/3855666279 conn(0x7f55cc09ba20 msgr2=0x7f55cc09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 -- 172.21.15.53:0/3855666279 shutdown_connections 2024-05-10T10:33:55.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.873+0000 7f55d17dd640 1 -- 172.21.15.53:0/3855666279 wait complete. 2024-05-10T10:33:55.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.874+0000 7f55d17dd640 1 Processor -- start 2024-05-10T10:33:55.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.874+0000 7f55d17dd640 1 -- start start 2024-05-10T10:33:55.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55d17dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0d1aa0 0x7f55cc06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55d17dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d3340 0x7f55cc14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55d17dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc15d290 0x7f55cc1e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55d17dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc071a50 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55d17dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc0cdd50 con 0x7f55cc15d290 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55d17dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc0ce7b0 con 0x7f55cc0d3340 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0d1aa0 0x7f55cc06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0d1aa0 0x7f55cc06dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58810/0 (socket says 172.21.15.53:58810) 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55caffd640 1 -- 172.21.15.53:0/1998522850 learned_addr learned my addr 172.21.15.53:0/1998522850 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:55.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc15d290 0x7f55cc1e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.875+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d3340 0x7f55cc14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55caffd640 1 -- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d3340 msgr2=0x7f55cc14e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55caffd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d3340 0x7f55cc14e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55caffd640 1 -- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc15d290 msgr2=0x7f55cc1e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55caffd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc15d290 0x7f55cc1e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55caffd640 1 -- 172.21.15.53:0/1998522850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc0b7210 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55caffd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0d1aa0 0x7f55cc06dde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55c4007fd0 tx=0x7f55c400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55c37fe640 1 -- 172.21.15.53:0/1998522850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4017070 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55c37fe640 1 -- 172.21.15.53:0/1998522850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c4005ce0 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc06e6a0 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55c37fe640 1 -- 172.21.15.53:0/1998522850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4007270 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.876+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc06ec60 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.878+0000 7f55c37fe640 1 -- 172.21.15.53:0/1998522850 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f55c401a070 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.878+0000 7f55c37fe640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55ac076300 0x7f55ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.878+0000 7f55c37fe640 1 -- 172.21.15.53:0/1998522850 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f55c4096c10 con 0x7f55cc0d1aa0 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.878+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55ac076300 0x7f55ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.879+0000 7f55d45d5740 1 Processor -- start 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.879+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55ac076300 0x7f55ac0787c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009560 tx=0x7f55b4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.879+0000 7f55d45d5740 1 -- start start 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d956400760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fe330 0x55d956400ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563d0bb0 0x55d9563d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55d45d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d95633a8d0 con 0x55d9563fdf50 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9562f3c10 con 0x55d9563fe330 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d956336890 con 0x55d9563d0bb0 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d956400760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d956400760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58816/0 (socket says 172.21.15.53:58816) 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 -- 172.21.15.53:0/3804591208 learned_addr learned my addr 172.21.15.53:0/3804591208 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fe330 0x55d956400ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563d0bb0 0x55d9563d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 -- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563d0bb0 msgr2=0x55d9563d43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563d0bb0 0x55d9563d43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 -- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fe330 msgr2=0x55d956400ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fe330 0x55d956400ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 -- 172.21.15.53:0/3804591208 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9562fb5d0 con 0x55d9563fdf50 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.880+0000 7f55caffd640 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d956400760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55c405bc50 tx=0x7f55c405f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ba2439745b4eca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55c0ff9640 1 -- 172.21.15.53:0/3804591208 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4068240 con 0x55d9563fdf50 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55c0ff9640 1 -- 172.21.15.53:0/3804591208 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c4060e50 con 0x55d9563fdf50 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55c0ff9640 1 -- 172.21.15.53:0/3804591208 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4068660 con 0x55d9563fdf50 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 -- 172.21.15.53:0/3804591208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 msgr2=0x55d956400760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d956400760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55c405bc50 tx=0x7f55c405f7a0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 -- 172.21.15.53:0/3804591208 shutdown_connections 2024-05-10T10:33:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563d0bb0 0x55d9563d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fe330 0x55d956400ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 --2- 172.21.15.53:0/3804591208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d956400760 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 -- 172.21.15.53:0/3804591208 >> 172.21.15.53:0/3804591208 conn(0x55d9563d2e20 msgr2=0x55d95641b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 -- 172.21.15.53:0/3804591208 shutdown_connections 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.881+0000 7f55d45d5740 1 -- 172.21.15.53:0/3804591208 wait complete. 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.882+0000 7f55d45d5740 1 -- start start 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.882+0000 7f55d45d5740 1 Processor -- start 2024-05-10T10:33:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d9563fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fe330 0x55d9563d4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fac80 0x55d9563d5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55d45d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d9563d5fd0 con 0x55d9563fdf50 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9562f7760 con 0x55d9563fac80 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d9563d62b0 con 0x55d9563fe330 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55d45d5740 10 client.0 _ll_register_callbacks cb 0x55d956330cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d9563fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d9563fa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58828/0 (socket says 172.21.15.53:58828) 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 -- 172.21.15.53:0/2592336708 learned_addr learned my addr 172.21.15.53:0/2592336708 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fac80 0x55d9563d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fe330 0x55d9563d4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 -- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fe330 msgr2=0x55d9563d4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fe330 0x55d9563d4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 -- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fac80 msgr2=0x55d9563d5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fac80 0x55d9563d5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 -- 172.21.15.53:0/2592336708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95604c2d0 con 0x55d9563fdf50 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fe330 0x55d9563d4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55caffd640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d9563fa070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55c405c390 tx=0x7f55c4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1c985d0e3cf480 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.883+0000 7f55a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:33:55.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55a9ffb640 1 -- 172.21.15.53:0/2592336708 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4068240 con 0x55d9563fdf50 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55a9ffb640 1 -- 172.21.15.53:0/2592336708 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f55c4007410 con 0x55d9563fdf50 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c4060e50 con 0x55d9563fdf50 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55a9ffb640 1 -- 172.21.15.53:0/2592336708 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f55c4081050 con 0x55d9563fdf50 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 10 client.4668 fetch_fsmap learned FSMap version 9 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d9561166b0 con 0x55d9563fdf50 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55a9ffb640 1 -- 172.21.15.53:0/2592336708 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c4060e50 con 0x55d9563fdf50 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55a9ffb640 1 -- 172.21.15.53:0/2592336708 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f55c405b540 con 0x55d9563fdf50 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 10 client.4668 fetch_fsmap finished waiting for FSMap version 9 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 10 client.4668 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:33:55.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 20 client.4668 populate_metadata read hostname 'smithi053' 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d956415f20 0x55d956416300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 4 client.4668 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 --> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9562be760 con 0x55d956415f20 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.884+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d956415f20 0x55d956416300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.885+0000 7f55a9ffb640 10 client.4668 ms_handle_connect on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.885+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d956415f20 0x55d956416300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1016d77bd6df1000 server_cookie=374db80941a4cb46 in_seq=0 out_seq=0 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.885+0000 7f55a9ffb640 1 -- 172.21.15.53:0/2592336708 <== mds.0 v2:172.21.15.135:6834/4124737039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d9562be760 con 0x55d956415f20 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.885+0000 7f55a9ffb640 10 client.4668 handle_command_reply: tid=0 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 client.4668 shutdown 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 msgr2=0x55d9563fa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d9563fa070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55c405c390 tx=0x7f55c4000ed0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 shutdown_connections 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d956415f20 0x55d956416300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fac80 0x55d9563d5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55a9ffb640 0 client.4668 ms_handle_reset on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fe330 0x55d9563d4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 --2- 172.21.15.53:0/2592336708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fdf50 0x55d9563fa070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 >> 172.21.15.53:0/2592336708 conn(0x55d9563d0bb0 msgr2=0x55d9563d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 shutdown_connections 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 1 -- 172.21.15.53:0/2592336708 wait complete. 2024-05-10T10:33:55.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.886+0000 7f55d45d5740 20 client.4668 trim_cache size 0 max 16384 2024-05-10T10:33:55.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 Processor -- start 2024-05-10T10:33:55.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 -- start start 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d9563d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc201b20 con 0x55d9563fe330 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc0b8bb0 con 0x55d9563fdf50 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc2014d0 con 0x55d9563fac80 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.905+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58840/0 (socket says 172.21.15.53:58840) 2024-05-10T10:33:55.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 -- 172.21.15.53:0/1997800929 learned_addr learned my addr 172.21.15.53:0/1997800929 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55caffd640 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d9563d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 -- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 msgr2=0x55d9563d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 -- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 msgr2=0x55d9563d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d9563d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 -- 172.21.15.53:0/1997800929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95641f270 con 0x55d9563fe330 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641d7d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009e80 tx=0x7f55bc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a10b7b2d6d6719a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55abfff640 1 -- 172.21.15.53:0/1997800929 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0077f0 con 0x55d9563fe330 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55abfff640 1 -- 172.21.15.53:0/1997800929 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55bc0079b0 con 0x55d9563fe330 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.906+0000 7f55abfff640 1 -- 172.21.15.53:0/1997800929 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc007da0 con 0x55d9563fe330 2024-05-10T10:33:55.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 -- 172.21.15.53:0/1997800929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 msgr2=0x55d95641d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641d7d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009e80 tx=0x7f55bc02e000 comp rx=0 tx=0).stop 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 -- 172.21.15.53:0/1997800929 shutdown_connections 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641d7d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d9563d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 --2- 172.21.15.53:0/1997800929 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 -- 172.21.15.53:0/1997800929 >> 172.21.15.53:0/1997800929 conn(0x55d9563d0bb0 msgr2=0x55d956415060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 -- 172.21.15.53:0/1997800929 shutdown_connections 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 -- 172.21.15.53:0/1997800929 wait complete. 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 -- start start 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.907+0000 7f55d45d5740 1 Processor -- start 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d95641b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55d45d5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55d45d5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d9563d5fd0 con 0x55d9563fe330 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9562f7760 con 0x55d9563fdf50 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55d45d5740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d95641cd60 con 0x55d9563fac80 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55d45d5740 10 client.0 _ll_register_callbacks cb 0x7f55b40097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58844/0 (socket says 172.21.15.53:58844) 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55cb7fe640 1 -- 172.21.15.53:0/177228572 learned_addr learned my addr 172.21.15.53:0/177228572 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55caffd640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55ca7fc640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d95641b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55cb7fe640 1 -- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 msgr2=0x55d9563d4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.908+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55cb7fe640 1 -- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 msgr2=0x55d95641b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:55.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d95641b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55cb7fe640 1 -- 172.21.15.53:0/177228572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95649c8e0 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55caffd640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55cb7fe640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641be10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009d90 tx=0x7f55bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246d90d618ca0886 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55a9ffb640 1 -- 172.21.15.53:0/177228572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc055030 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55a9ffb640 1 -- 172.21.15.53:0/177228572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f55bc04d040 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55bc0079b0 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55a9ffb640 1 -- 172.21.15.53:0/177228572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f55bc003ba0 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 10 client.4674 fetch_fsmap learned FSMap version 9 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d9563d5190 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55a9ffb640 1 -- 172.21.15.53:0/177228572 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55bc0079b0 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55a9ffb640 1 -- 172.21.15.53:0/177228572 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f55bc046070 con 0x55d9563fe330 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 10 client.4674 fetch_fsmap finished waiting for FSMap version 9 2024-05-10T10:33:55.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 10 client.4674 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 20 client.4674 populate_metadata read hostname 'smithi053' 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d9563fcda0 0x55d9563fd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 4 client.4674 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.909+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 --> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] -- command(tid 0: {"prefix": "status"}) -- 0x55d9563484e0 con 0x55d9563fcda0 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55caffd640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d9563fcda0 0x55d9563fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55caffd640 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d9563fcda0 0x55d9563fd180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868c298c1dc8fc06 server_cookie=45803ba2359670b7 in_seq=0 out_seq=0 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55a9ffb640 10 client.4674 ms_handle_connect on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55a9ffb640 1 -- 172.21.15.53:0/177228572 <== mds.0 v2:172.21.15.135:6834/4124737039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55d9563484e0 con 0x55d9563fcda0 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55a9ffb640 10 client.4674 handle_command_reply: tid=0 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 client.4674 shutdown 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 msgr2=0x55d95641be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641be10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009d90 tx=0x7f55bc004ab0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 shutdown_connections 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:6834/4124737039,v1:172.21.15.135:6836/4124737039] conn(0x55d9563fcda0 0x55d9563fd180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9563fe330 0x55d95641be10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55a9ffb640 0 client.4674 ms_handle_reset on v2:172.21.15.135:6834/4124737039 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9563fdf50 0x55d95641b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 --2- 172.21.15.53:0/177228572 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d9563fac80 0x55d9563d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 >> 172.21.15.53:0/177228572 conn(0x55d9563d0bb0 msgr2=0x55d9563d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 shutdown_connections 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.910+0000 7f55d45d5740 1 -- 172.21.15.53:0/177228572 wait complete. 2024-05-10T10:33:55.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d45d5740 20 client.4674 trim_cache size 0 max 16384 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 4278, 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 21.409949917999999, 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 9, 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 24, 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 24, 2024-05-10T10:33:55.920 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 27.454660204 2024-05-10T10:33:55.921 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:33:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55ac076300 msgr2=0x7f55ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55ac076300 0x7f55ac0787c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009560 tx=0x7f55b4009190 comp rx=0 tx=0).stop 2024-05-10T10:33:55.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0d1aa0 msgr2=0x7f55cc06dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0d1aa0 0x7f55cc06dde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55c4007fd0 tx=0x7f55c400c9b0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 shutdown_connections 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55ac076300 0x7f55ac0787c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55cc15d290 0x7f55cc1e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55cc0d3340 0x7f55cc14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 --2- 172.21.15.53:0/1998522850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0d1aa0 0x7f55cc06dde0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 >> 172.21.15.53:0/1998522850 conn(0x7f55cc09ba20 msgr2=0x7f55cc0b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 shutdown_connections 2024-05-10T10:33:55.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:55.911+0000 7f55d17dd640 1 -- 172.21.15.53:0/1998522850 wait complete. 2024-05-10T10:33:55.923 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T10:33:56.143 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T10:33:56.143 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T10:33:56.144 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T10:33:56.144 INFO:teuthology.run_tasks:Running task kclient... 2024-05-10T10:33:56.175 INFO:tasks.kclient:Mounting kernel clients... 2024-05-10T10:33:56.176 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-10T10:33:56.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.176 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:33:56.193 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-10T10:33:56.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T10:33:56.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.261 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-05-10T10:33:56.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.360 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:33:56.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-05-10T10:33:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-05-10T10:33:56.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:33:56.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.447 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-05-10T10:33:56.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.467 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-05-10T10:33:56.539 INFO:teuthology.orchestra.run.smithi135.stderr:Cannot find device "ceph-brx" 2024-05-10T10:33:56.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:33:56.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.541 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-05-10T10:33:56.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:56.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-05-10T10:33:56.667 INFO:teuthology.orchestra.run.smithi135.stderr:Cannot find device "ceph-brx" 2024-05-10T10:33:56.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:33:56.668 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-10T10:33:56.668 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T10:33:56.668 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:33:56.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:33:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.774+0000 7fe2ca390640 1 Processor -- start 2024-05-10T10:33:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.774+0000 7fe2ca390640 1 -- start start 2024-05-10T10:33:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2ca390640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe2c415ce60 0x7fe2c40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2ca390640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe2c41611e0 0x7fe2c415d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2ca390640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c40c6a60 con 0x7fe2c40c65f0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2ca390640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c40cd090 con 0x7fe2c41611e0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2ca390640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c40cd370 con 0x7fe2c415ce60 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c892d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe2c41611e0 0x7fe2c415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c892d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe2c41611e0 0x7fe2c415d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58854/0 (socket says 172.21.15.53:58854) 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c892d640 1 -- 172.21.15.53:0/258511941 learned_addr learned my addr 172.21.15.53:0/258511941 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c3fff640 1 --2- 172.21.15.53:0/258511941 >> v2:172.21.15.135:3300/0 conn(0x7fe2c415ce60 0x7fe2c40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c892d640 1 -- 172.21.15.53:0/258511941 >> v2:172.21.15.135:3300/0 conn(0x7fe2c415ce60 msgr2=0x7fe2c40cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c892d640 1 --2- 172.21.15.53:0/258511941 >> v2:172.21.15.135:3300/0 conn(0x7fe2c415ce60 0x7fe2c40cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c892d640 1 -- 172.21.15.53:0/258511941 >> v1:172.21.15.135:6789/0 conn(0x7fe2c40c65f0 legacy=0x7fe2c4160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.775+0000 7fe2c892d640 1 -- 172.21.15.53:0/258511941 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c40cd650 con 0x7fe2c41611e0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2c892d640 1 --2- 172.21.15.53:0/258511941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c415d240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4070280 tx=0x7fe2bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533dabce1ebf6f87 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2c2ffd640 1 -- 172.21.15.53:0/258511941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc036070 con 0x7fe2c41611e0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2c2ffd640 1 -- 172.21.15.53:0/258511941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc031600 con 0x7fe2c41611e0 2024-05-10T10:33:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2c2ffd640 1 -- 172.21.15.53:0/258511941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0319d0 con 0x7fe2c41611e0 2024-05-10T10:33:56.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2ca390640 1 -- 172.21.15.53:0/258511941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 msgr2=0x7fe2c415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2ca390640 1 --2- 172.21.15.53:0/258511941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c415d240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4070280 tx=0x7fe2bc02a660 comp rx=0 tx=0).stop 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2ca390640 1 -- 172.21.15.53:0/258511941 shutdown_connections 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2ca390640 1 --2- 172.21.15.53:0/258511941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c415d240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2ca390640 1 --2- 172.21.15.53:0/258511941 >> v2:172.21.15.135:3300/0 conn(0x7fe2c415ce60 0x7fe2c40cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2ca390640 1 -- 172.21.15.53:0/258511941 >> 172.21.15.53:0/258511941 conn(0x7fe2c40b2900 msgr2=0x7fe2c40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.776+0000 7fe2ca390640 1 -- 172.21.15.53:0/258511941 shutdown_connections 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.777+0000 7fe2ca390640 1 -- 172.21.15.53:0/258511941 wait complete. 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.778+0000 7fe2ca390640 1 Processor -- start 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.778+0000 7fe2ca390640 1 -- start start 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2ca390640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c65f0 0x7fe2c414c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2ca390640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe2c415ce60 0x7fe2c40b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2ca390640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c40b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2ca390640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c40cde50 con 0x7fe2c41611e0 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2ca390640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c40ccd60 con 0x7fe2c40c65f0 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2ca390640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c40cdfd0 con 0x7fe2c415ce60 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c40b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c40b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58860/0 (socket says 172.21.15.53:58860) 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 -- 172.21.15.53:0/3972248933 learned_addr learned my addr 172.21.15.53:0/3972248933 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c3fff640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c65f0 0x7fe2c414c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c37fe640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe2c415ce60 0x7fe2c40b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 -- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe2c415ce60 msgr2=0x7fe2c40b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe2c415ce60 0x7fe2c40b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 -- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c65f0 msgr2=0x7fe2c414c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c65f0 0x7fe2c414c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 -- 172.21.15.53:0/3972248933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c40b1fb0 con 0x7fe2c41611e0 2024-05-10T10:33:56.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c892d640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c40b1360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009f40 tx=0x7fe2bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.779+0000 7fe2c0ff9640 1 -- 172.21.15.53:0/3972248933 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc036050 con 0x7fe2c41611e0 2024-05-10T10:33:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.780+0000 7fe2c0ff9640 1 -- 172.21.15.53:0/3972248933 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc042070 con 0x7fe2c41611e0 2024-05-10T10:33:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.780+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c40af7f0 con 0x7fe2c41611e0 2024-05-10T10:33:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.780+0000 7fe2c0ff9640 1 -- 172.21.15.53:0/3972248933 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc009510 con 0x7fe2c41611e0 2024-05-10T10:33:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.780+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c40afa90 con 0x7fe2c41611e0 2024-05-10T10:33:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.780+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c406a220 con 0x7fe2c41611e0 2024-05-10T10:33:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.781+0000 7fe2c0ff9640 1 -- 172.21.15.53:0/3972248933 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe2bc007c30 con 0x7fe2c41611e0 2024-05-10T10:33:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.782+0000 7fe2c0ff9640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe2b00762d0 0x7fe2b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.782+0000 7fe2c0ff9640 1 -- 172.21.15.53:0/3972248933 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe2bc0b8820 con 0x7fe2c41611e0 2024-05-10T10:33:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.782+0000 7fe2c3fff640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe2b00762d0 0x7fe2b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.782+0000 7fe2c3fff640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe2b00762d0 0x7fe2b0078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4006fd0 tx=0x7fe2b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:56.786+0000 7fe2c0ff9640 1 -- 172.21.15.53:0/3972248933 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe2bc081ed0 con 0x7fe2c41611e0 2024-05-10T10:33:57.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.017+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe2c40c7300 con 0x7fe2c41611e0 2024-05-10T10:33:57.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.017+0000 7fe2c0ff9640 1 -- 172.21.15.53:0/3972248933 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fe2bc0817d0 con 0x7fe2c41611e0 2024-05-10T10:33:57.018 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T10:33:57.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe2b00762d0 msgr2=0x7fe2b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:57.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe2b00762d0 0x7fe2b0078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4006fd0 tx=0x7fe2b4008040 comp rx=0 tx=0).stop 2024-05-10T10:33:57.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 msgr2=0x7fe2c40b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:57.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c40b1360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009f40 tx=0x7fe2bc004a40 comp rx=0 tx=0).stop 2024-05-10T10:33:57.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 shutdown_connections 2024-05-10T10:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe2b00762d0 0x7fe2b0078790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2c41611e0 0x7fe2c40b1360 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe2c415ce60 0x7fe2c40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 --2- 172.21.15.53:0/3972248933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2c40c65f0 0x7fe2c414c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.019+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 >> 172.21.15.53:0/3972248933 conn(0x7fe2c40b2900 msgr2=0x7fe2c40d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.020+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 shutdown_connections 2024-05-10T10:33:57.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57.020+0000 7fe2ca390640 1 -- 172.21.15.53:0/3972248933 wait complete. 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:33:57.031 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:33:57.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:57.032 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:33:57.048 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:33:57.048 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:33:57.048 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:33:57.048 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:57.048 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:33:57.048 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42732sec preferred_lft 42732sec 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:33:57.049 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:57.050 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:57.050 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:57.050 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:33:57.050 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T10:33:57.050 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:57.050 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:57.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2024-05-10T10:33:57.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2024-05-10T10:33:57.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2024-05-10T10:33:57.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T10:33:57.051 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:57.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:57.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:57.191 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:33:57.215 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:33:57.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:57.217 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:33:57.274 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:33:57.275 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:33:57.275 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:33:57.275 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:33:57.275 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:33:57.275 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:57.275 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:57.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:33:57.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:33:57.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:33:57.275 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:57.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:57.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:57.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:57.404 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:33:57.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:57.421 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-05-10T10:33:57.476 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:57.477 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:57.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:33:57.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T10:33:57.477 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:57.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:57.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:57.597 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T10:33:57.597 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:57.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:57.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:57.715 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:57.715 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:57.715 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:33:57.715 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T10:33:57.715 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:57.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:57.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:57.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:33:57.795 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:33:57.824 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:33:57.825 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:33:57.825 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:33:57.880 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:33:57.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:57.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:33:57.965 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:33:57.965 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:33:57.965 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:33:58.071 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:33:58.071 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:33:58.072 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:33:58.072 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:33:58.072 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:33:58.081 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:33:58.083 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:33:58.083 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:33:58.083 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:33:58.084 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:33:58.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:58.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:33:58.110 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:33:58.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:33:58.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:58.211 INFO:teuthology.orchestra.run.smithi053.stdout:client4666 2024-05-10T10:33:58.211 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:58.211 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:33:58.228 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:33:58.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:33:58.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:58.326 INFO:teuthology.orchestra.run.smithi053.stdout:client4666 2024-05-10T10:33:58.326 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:33:58.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:33:58.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:58.384 INFO:teuthology.orchestra.run.smithi053.stdout:client4666 2024-05-10T10:33:58.385 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/543b513e-90c7-47ff-9cdd-f7fa4d978734.client4666/status') 2024-05-10T10:33:58.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:58.445 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.4666 (3)172.21.15.53:0/1419806607 2024-05-10T10:33:58.445 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2024-05-10T10:33:58.445 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2024-05-10T10:33:58.445 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2024-05-10T10:33:58.445 INFO:teuthology.orchestra.run.smithi053.stderr:70 bytes copied, 3.0323e-05 s, 2.3 MB/s 2024-05-10T10:33:58.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:58.453 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-10T10:33:58.453 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T10:33:58.453 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-10T10:33:58.453 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:33:58.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.557+0000 7f15baed1640 1 Processor -- start 2024-05-10T10:33:58.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.557+0000 7f15baed1640 1 -- start start 2024-05-10T10:33:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15baed1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 0x7f15b406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15baed1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f15b40cfd80 0x7f15b40ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15baed1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b415fc90 con 0x7f15b40d0230 2024-05-10T10:33:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15baed1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f15b406ee50 con 0x7f15b40cfd80 2024-05-10T10:33:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15baed1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f15b40b8c40 con 0x7f15b40b8860 2024-05-10T10:33:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f15b40d0230 0x7f15b40cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55216/0 (socket says 172.21.15.53:55216) 2024-05-10T10:33:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b3fff640 1 -- 172.21.15.53:0/3449351292 learned_addr learned my addr 172.21.15.53:0/3449351292 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b8c6d640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 0x7f15b406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766870239 0 0) 0x7f15b415fc90 con 0x7f15b40d0230 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b946e640 1 --2- 172.21.15.53:0/3449351292 >> v2:172.21.15.135:3300/0 conn(0x7f15b40cfd80 0x7f15b40ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15a4003520 con 0x7f15b40d0230 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2888734847 0 0) 0x7f15a4003520 con 0x7f15b40d0230 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 msgr2=0x7f15b406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 0x7f15b406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 >> v2:172.21.15.135:3300/0 conn(0x7f15b40cfd80 msgr2=0x7f15b40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 --2- 172.21.15.53:0/3449351292 >> v2:172.21.15.135:3300/0 conn(0x7f15b40cfd80 0x7f15b40ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.558+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b40cd570 con 0x7f15b40d0230 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f15a0001670 con 0x7f15b40d0230 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 >> v1:172.21.15.53:6789/0 conn(0x7f15b40d0230 legacy=0x7f15b40cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:58.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a40039b0 0x7f15a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a4004800 0x7f15a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 0x7f15a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15b406ee50 con 0x7f15a4004800 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40b8c40 con 0x7f15a40039b0 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f15a400b9c0 con 0x7f15b40b8860 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b3fff640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a4004800 0x7f15a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b8c6d640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a40039b0 0x7f15a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b946e640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 0x7f15a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b3fff640 1 -- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 msgr2=0x7f15a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b3fff640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b8860 0x7f15a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b3fff640 1 -- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a40039b0 msgr2=0x7f15a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b3fff640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a40039b0 0x7f15a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.559+0000 7f15b3fff640 1 -- 172.21.15.53:0/3449351292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a4003520 con 0x7f15a4004800 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15b3fff640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a4004800 0x7f15a4004c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15a000c240 tx=0x7f15a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac2f71d60d1e0e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0007d50 con 0x7f15a4004800 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a002e070 con 0x7f15a4004800 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15b37fe640 1 -- 172.21.15.53:0/3449351292 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0005d30 con 0x7f15a4004800 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15baed1640 1 -- 172.21.15.53:0/3449351292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a4004800 msgr2=0x7f15a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15baed1640 1 --2- 172.21.15.53:0/3449351292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a4004800 0x7f15a4004c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15a000c240 tx=0x7f15a4005830 comp rx=0 tx=0).stop 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15b946e640 1 -- 172.21.15.53:0/3449351292 reap_dead start 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15baed1640 1 -- 172.21.15.53:0/3449351292 shutdown_connections 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15baed1640 1 -- 172.21.15.53:0/3449351292 >> 172.21.15.53:0/3449351292 conn(0x7f15b40d2df0 msgr2=0x7f15b40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15baed1640 1 -- 172.21.15.53:0/3449351292 shutdown_connections 2024-05-10T10:33:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.560+0000 7f15baed1640 1 -- 172.21.15.53:0/3449351292 wait complete. 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.562+0000 7f15baed1640 1 Processor -- start 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.562+0000 7f15baed1640 1 -- start start 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15baed1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4004800 0x7f15b40a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15baed1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b00b0 0x7f15b4155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15baed1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40b0970 0x7f15b40b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15baed1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40cde10 con 0x7f15b40b0970 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15baed1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40cdb30 con 0x7f15a4004800 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15baed1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f15b40b5c40 con 0x7f15b40b00b0 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40b0970 0x7f15b40b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40b0970 0x7f15b40b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58886/0 (socket says 172.21.15.53:58886) 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 -- 172.21.15.53:0/2193513482 learned_addr learned my addr 172.21.15.53:0/2193513482 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b8c6d640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4004800 0x7f15b40a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b3fff640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b00b0 0x7f15b4155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 -- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b00b0 msgr2=0x7f15b4155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b00b0 0x7f15b4155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 -- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4004800 msgr2=0x7f15b40a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4004800 0x7f15b40a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 -- 172.21.15.53:0/2193513482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b40a0960 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b946e640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40b0970 0x7f15b40b0df0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f15b40737b0 tx=0x7f15ac00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b17fa640 1 -- 172.21.15.53:0/2193513482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac019070 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b17fa640 1 -- 172.21.15.53:0/2193513482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac00f8d0 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b409f0a0 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.563+0000 7f15b17fa640 1 -- 172.21.15.53:0/2193513482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac003e60 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.564+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b409f660 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.564+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b406a220 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.565+0000 7f15b17fa640 1 -- 172.21.15.53:0/2193513482 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f15ac01c070 con 0x7f15b40b0970 2024-05-10T10:33:58.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.566+0000 7f15b17fa640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15980762d0 0x7f1598078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:33:58.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.566+0000 7f15b17fa640 1 -- 172.21.15.53:0/2193513482 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f15ac0981a0 con 0x7f15b40b0970 2024-05-10T10:33:58.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.566+0000 7f15b8c6d640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15980762d0 0x7f1598078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:33:58.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.567+0000 7f15b8c6d640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15980762d0 0x7f1598078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15b40c9c30 tx=0x7f15a8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:33:58.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.570+0000 7f15b17fa640 1 -- 172.21.15.53:0/2193513482 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f15ac010040 con 0x7f15b40b0970 2024-05-10T10:33:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.802+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f15b40d1e20 con 0x7f15b40b0970 2024-05-10T10:33:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.803+0000 7f15b17fa640 1 -- 172.21.15.53:0/2193513482 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f15ac061160 con 0x7f15b40b0970 2024-05-10T10:33:58.804 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15980762d0 msgr2=0x7f1598078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15980762d0 0x7f1598078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15b40c9c30 tx=0x7f15a8009190 comp rx=0 tx=0).stop 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40b0970 msgr2=0x7f15b40b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40b0970 0x7f15b40b0df0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f15b40737b0 tx=0x7f15ac00f1b0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 shutdown_connections 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15980762d0 0x7f1598078790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40b0970 0x7f15b40b0df0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15b40b00b0 0x7f15b4155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 --2- 172.21.15.53:0/2193513482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4004800 0x7f15b40a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 >> 172.21.15.53:0/2193513482 conn(0x7f15b40d2df0 msgr2=0x7f15b406e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:33:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 shutdown_connections 2024-05-10T10:33:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:58.805+0000 7f15baed1640 1 -- 172.21.15.53:0/2193513482 wait complete. 2024-05-10T10:33:58.816 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:33:58.817 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-10T10:33:58.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:58.817 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42730sec preferred_lft 42730sec 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link tentative 2024-05-10T10:33:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:58.836 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:33:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:33:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link tentative 2024-05-10T10:33:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:58.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:58.836 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:33:58.899 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:33:58.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:58.901 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:33:58.958 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:33:58.958 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:33:58.958 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:33:58.958 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:33:58.958 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:33:58.959 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:58.959 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:58.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:33:58.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:33:58.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:33:58.959 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:59.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:59.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:59.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:59.088 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:33:59.104 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:33:59.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:59.105 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-05-10T10:33:59.160 INFO:teuthology.orchestra.run.smithi053.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-10T10:33:59.161 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:59.161 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:59.161 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:33:59.161 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-10T10:33:59.161 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:59.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:59.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:59.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:59.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 3a:71:58:63:c0:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T10:33:59.306 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.307 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::3871:58ff:fe63:c050/64 scope link 2024-05-10T10:33:59.307 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:59.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 3a:71:58:63:c0:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T10:33:59.374 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.375 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::3871:58ff:fe63:c050/64 scope link 2024-05-10T10:33:59.375 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:33:59.376 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-10T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:59.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:59.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:59.533 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:33:59.533 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:33:59.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.1 up 2024-05-10T10:33:59.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.1 master ceph-brx 2024-05-10T10:33:59.534 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:33:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:33:59.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:59.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:33:59.613 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-10T10:33:59.636 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-10T10:33:59.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:33:59.637 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-10T10:33:59.692 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:33:59.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:59.693 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T10:33:59.777 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T10:33:59.777 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-10T10:33:59.777 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:33:59.786 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T10:33:59.786 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-10T10:33:59.786 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:33:59.786 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:33:59.786 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:33:59.791 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T10:33:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-10T10:33:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:33:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:33:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:33:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:33:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:33:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:33:59.793 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-10T10:33:59.793 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:33:59.793 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:33:59.793 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:33:59.793 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:33:59.794 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:33:59.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:33:59.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-10T10:33:59.820 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:33:59.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:33:59.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:33:59.918 INFO:teuthology.orchestra.run.smithi053.stdout:client4689 2024-05-10T10:33:59.919 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:33:59.919 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:33:59.936 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:00.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:00.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:00.032 INFO:teuthology.orchestra.run.smithi053.stdout:client4689 2024-05-10T10:34:00.033 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:00.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:00.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:00.090 INFO:teuthology.orchestra.run.smithi053.stdout:client4689 2024-05-10T10:34:00.090 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/543b513e-90c7-47ff-9cdd-f7fa4d978734.client4689/status') 2024-05-10T10:34:00.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:00.150 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.4689 (3)192.168.0.2:0/2031889898 2024-05-10T10:34:00.150 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2024-05-10T10:34:00.151 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2024-05-10T10:34:00.151 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2024-05-10T10:34:00.151 INFO:teuthology.orchestra.run.smithi053.stderr:69 bytes copied, 2.8854e-05 s, 2.4 MB/s 2024-05-10T10:34:00.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:00.159 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-10T10:34:00.159 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T10:34:00.159 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T10:34:00.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:34:00.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.261+0000 7f4dba5f0640 1 Processor -- start 2024-05-10T10:34:00.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.262+0000 7f4dba5f0640 1 -- start start 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.262+0000 7f4dba5f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4db415f4e0 0x7f4db406d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.262+0000 7f4dba5f0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db40b0a90 con 0x7f4db415f8c0 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.262+0000 7f4dba5f0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db40b3540 con 0x7f4db406e620 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.262+0000 7f4dba5f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4db40b23f0 con 0x7f4db415f4e0 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4dabfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4db415f8c0 0x7f4db406df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55220/0 (socket says 172.21.15.53:55220) 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4dabfff640 1 -- 172.21.15.53:0/2478220968 learned_addr learned my addr 172.21.15.53:0/2478220968 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777495784 0 0) 0x7f4db40b0a90 con 0x7f4db415f8c0 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4da0003520 con 0x7f4db415f8c0 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db3fff640 1 --2- 172.21.15.53:0/2478220968 >> v2:172.21.15.135:3300/0 conn(0x7f4db415f4e0 0x7f4db406d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3832789836 0 0) 0x7f4da0003520 con 0x7f4db415f8c0 2024-05-10T10:34:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 >> v2:172.21.15.135:3300/0 conn(0x7f4db415f4e0 msgr2=0x7f4db406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 --2- 172.21.15.53:0/2478220968 >> v2:172.21.15.135:3300/0 conn(0x7f4db415f4e0 0x7f4db406d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 >> v1:172.21.15.135:6789/0 conn(0x7f4db406e620 legacy=0x7f4db40d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db40b3820 con 0x7f4db415f8c0 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4d9c001670 con 0x7f4db415f8c0 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 >> v1:172.21.15.53:6789/0 conn(0x7f4db415f8c0 legacy=0x7f4db406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4da00039b0 0x7f4da0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4da0004800 0x7f4da0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00051c0 0x7f4da000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4db40b23f0 con 0x7f4da00051c0 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4da000c160 con 0x7f4da00039b0 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4da000c360 con 0x7f4da0004800 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db3fff640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4da00039b0 0x7f4da0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4db8b8d640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00051c0 0x7f4da000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.263+0000 7f4dabfff640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4da0004800 0x7f4da0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.264+0000 7f4db3fff640 1 -- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4da0004800 msgr2=0x7f4da0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.264+0000 7f4db3fff640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4da0004800 0x7f4da0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.264+0000 7f4db3fff640 1 -- 172.21.15.53:0/2478220968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00051c0 msgr2=0x7f4da000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.264+0000 7f4db3fff640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00051c0 0x7f4da000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.264+0000 7f4db3fff640 1 -- 172.21.15.53:0/2478220968 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da0003520 con 0x7f4da00039b0 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.264+0000 7f4db8b8d640 1 -- 172.21.15.53:0/2478220968 reap_dead start 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.264+0000 7f4db3fff640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4da00039b0 0x7f4da0003dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009ac0 tx=0x7f4da0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d0910b66843186 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4036070 con 0x7f4da00039b0 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da4031400 con 0x7f4da00039b0 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4db37fe640 1 -- 172.21.15.53:0/2478220968 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4009690 con 0x7f4da00039b0 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4dba5f0640 1 -- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4da00039b0 msgr2=0x7f4da0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4da00039b0 0x7f4da0003dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009ac0 tx=0x7f4da0005830 comp rx=0 tx=0).stop 2024-05-10T10:34:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4dba5f0640 1 -- 172.21.15.53:0/2478220968 shutdown_connections 2024-05-10T10:34:00.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/2478220968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4da00039b0 0x7f4da0003dd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4dba5f0640 1 -- 172.21.15.53:0/2478220968 >> 172.21.15.53:0/2478220968 conn(0x7f4db40d2df0 msgr2=0x7f4db40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:00.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4dba5f0640 1 -- 172.21.15.53:0/2478220968 shutdown_connections 2024-05-10T10:34:00.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.265+0000 7f4dba5f0640 1 -- 172.21.15.53:0/2478220968 wait complete. 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.266+0000 7f4dba5f0640 1 Processor -- start 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dba5f0640 1 -- start start 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dba5f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00039b0 0x7f4db4147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dba5f0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4db4152370 0x7f4db415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dba5f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db415fcd0 0x7f4db4160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dba5f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4db40b3de0 con 0x7f4da00039b0 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dba5f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4db40b4150 con 0x7f4db415fcd0 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dba5f0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4db40b45e0 con 0x7f4db4152370 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00039b0 0x7f4db4147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00039b0 0x7f4db4147240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58890/0 (socket says 172.21.15.53:58890) 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4db3fff640 1 -- 172.21.15.53:0/1421735755 learned_addr learned my addr 172.21.15.53:0/1421735755 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4db8b8d640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db415fcd0 0x7f4db4160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.267+0000 7f4dabfff640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4db4152370 0x7f4db415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db3fff640 1 -- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4db4152370 msgr2=0x7f4db415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db3fff640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4db4152370 0x7f4db415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db3fff640 1 -- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db415fcd0 msgr2=0x7f4db4160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db3fff640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db415fcd0 0x7f4db4160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db3fff640 1 -- 172.21.15.53:0/1421735755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db415abe0 con 0x7f4da00039b0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4dabfff640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4db4152370 0x7f4db415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db3fff640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00039b0 0x7f4db4147240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009be0 tx=0x7f4da4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db17fa640 1 -- 172.21.15.53:0/1421735755 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da403f020 con 0x7f4da00039b0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db17fa640 1 -- 172.21.15.53:0/1421735755 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da4037040 con 0x7f4da00039b0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db415aec0 con 0x7f4da00039b0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4db17fa640 1 -- 172.21.15.53:0/1421735755 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4036070 con 0x7f4da00039b0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.268+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db40c70e0 con 0x7f4da00039b0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.269+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db406a220 con 0x7f4da00039b0 2024-05-10T10:34:00.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.270+0000 7f4db17fa640 1 -- 172.21.15.53:0/1421735755 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4da4042020 con 0x7f4da00039b0 2024-05-10T10:34:00.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.270+0000 7f4db17fa640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d940762d0 0x7f4d94078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:00.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.270+0000 7f4db17fa640 1 -- 172.21.15.53:0/1421735755 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4da40b74d0 con 0x7f4da00039b0 2024-05-10T10:34:00.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.270+0000 7f4dabfff640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d940762d0 0x7f4d94078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:00.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.271+0000 7f4dabfff640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d940762d0 0x7f4d94078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c007fd0 tx=0x7f4d9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:00.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.274+0000 7f4db17fa640 1 -- 172.21.15.53:0/1421735755 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4da4049070 con 0x7f4da00039b0 2024-05-10T10:34:00.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.510+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4db40b1a40 con 0x7f4da00039b0 2024-05-10T10:34:00.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.510+0000 7f4db17fa640 1 -- 172.21.15.53:0/1421735755 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f4da4080530 con 0x7f4da00039b0 2024-05-10T10:34:00.511 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d940762d0 msgr2=0x7f4d94078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d940762d0 0x7f4d94078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c007fd0 tx=0x7f4d9c009040 comp rx=0 tx=0).stop 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00039b0 msgr2=0x7f4db4147240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00039b0 0x7f4db4147240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009be0 tx=0x7f4da4004730 comp rx=0 tx=0).stop 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 shutdown_connections 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d940762d0 0x7f4d94078790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db415fcd0 0x7f4db4160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4db4152370 0x7f4db415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 --2- 172.21.15.53:0/1421735755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da00039b0 0x7f4db4147240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 >> 172.21.15.53:0/1421735755 conn(0x7f4db40d2df0 msgr2=0x7f4db415d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 shutdown_connections 2024-05-10T10:34:00.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:00.513+0000 7f4dba5f0640 1 -- 172.21.15.53:0/1421735755 wait complete. 2024-05-10T10:34:00.524 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:34:00.525 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T10:34:00.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:00.525 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f0 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:00.545 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f1 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42736sec preferred_lft 42736sec 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::ec4:7aff:fe88:8083/64 scope link 2024-05-10T10:34:00.546 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:00.546 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T10:34:00.546 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:00.547 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:00.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add name ceph-brx type bridge 2024-05-10T10:34:00.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr flush dev ceph-brx 2024-05-10T10:34:00.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set ceph-brx up 2024-05-10T10:34:00.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T10:34:00.547 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:00.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:00.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:00.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:00.822 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:34:00.847 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-05-10T10:34:00.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:00.848 DEBUG:teuthology.orchestra.run.smithi135:> route 2024-05-10T10:34:00.923 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2024-05-10T10:34:00.923 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:34:00.923 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T10:34:00.923 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T10:34:00.923 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:34:00.923 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:00.923 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:00.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T10:34:00.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T10:34:00.924 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T10:34:00.924 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:01.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:01.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:01.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:01.196 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-05-10T10:34:01.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:01.212 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2024-05-10T10:34:01.267 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:01.267 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:01.268 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T10:34:01.268 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-10T10:34:01.268 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:01.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:01.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:01.388 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-10T10:34:01.389 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:01.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:01.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:01.523 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:01.523 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:01.523 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.0 up 2024-05-10T10:34:01.524 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T10:34:01.524 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:01.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:01.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:01.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:34:01.602 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T10:34:01.617 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T10:34:01.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:34:01.617 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T10:34:01.673 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:34:01.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:01.673 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T10:34:01.768 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T10:34:01.769 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T10:34:01.769 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-05-10T10:34:01.883 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T10:34:01.884 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T10:34:01.884 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-05-10T10:34:01.884 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:34:01.884 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:34:01.888 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T10:34:01.888 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T10:34:01.888 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-05-10T10:34:01.888 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:34:01.889 INFO:teuthology.orchestra.run.smithi135.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:34:01.891 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:34:01.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:01.891 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T10:34:01.916 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:02.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:02.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:02.013 INFO:teuthology.orchestra.run.smithi135.stdout:client4511 2024-05-10T10:34:02.013 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:02.013 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:02.029 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:02.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:02.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:02.125 INFO:teuthology.orchestra.run.smithi135.stdout:client4511 2024-05-10T10:34:02.126 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:02.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:02.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:02.182 INFO:teuthology.orchestra.run.smithi135.stdout:client4511 2024-05-10T10:34:02.182 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/543b513e-90c7-47ff-9cdd-f7fa4d978734.client4511/status') 2024-05-10T10:34:02.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:02.242 INFO:teuthology.orchestra.run.smithi135.stdout:instance: client.4511 (3)192.168.0.1:0/3409751541 2024-05-10T10:34:02.242 INFO:teuthology.orchestra.run.smithi135.stdout:blocklisted: false 2024-05-10T10:34:02.242 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records in 2024-05-10T10:34:02.242 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records out 2024-05-10T10:34:02.242 INFO:teuthology.orchestra.run.smithi135.stderr:69 bytes copied, 2.1929e-05 s, 3.1 MB/s 2024-05-10T10:34:02.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:02.249 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-10T10:34:02.250 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T10:34:02.250 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-10T10:34:02.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:34:02.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.365+0000 7f13a6124640 1 Processor -- start 2024-05-10T10:34:02.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.365+0000 7f13a6124640 1 -- start start 2024-05-10T10:34:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f13a6124640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13a0162750 0x7f13a00d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f13a6124640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0162b30 0x7f13a00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f13a6124640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a00b45f0 con 0x7f13a00d5cb0 2024-05-10T10:34:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f13a6124640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f13a00b4f20 con 0x7f13a0162750 2024-05-10T10:34:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f13a6124640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f13a00d0fe0 con 0x7f13a0162b30 2024-05-10T10:34:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f139effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f13a00d5cb0 0x7f13a00d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55230/0 (socket says 172.21.15.53:55230) 2024-05-10T10:34:02.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f139effd640 1 -- 172.21.15.53:0/3424374765 learned_addr learned my addr 172.21.15.53:0/3424374765 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f139f7fe640 1 --2- 172.21.15.53:0/3424374765 >> v2:172.21.15.135:3300/0 conn(0x7f13a0162750 0x7f13a00d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f139ffff640 1 --2- 172.21.15.53:0/3424374765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0162b30 0x7f13a00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.366+0000 7f139e7fc640 1 -- 172.21.15.53:0/3424374765 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4083246804 0 0) 0x7f13a00b45f0 con 0x7f13a00d5cb0 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.367+0000 7f139e7fc640 1 -- 172.21.15.53:0/3424374765 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f138c003520 con 0x7f13a00d5cb0 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.367+0000 7f139ffff640 1 -- 172.21.15.53:0/3424374765 >> v2:172.21.15.135:3300/0 conn(0x7f13a0162750 msgr2=0x7f13a00d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.367+0000 7f139ffff640 1 --2- 172.21.15.53:0/3424374765 >> v2:172.21.15.135:3300/0 conn(0x7f13a0162750 0x7f13a00d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.367+0000 7f139ffff640 1 -- 172.21.15.53:0/3424374765 >> v1:172.21.15.53:6789/0 conn(0x7f13a00d5cb0 legacy=0x7f13a00d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.367+0000 7f139ffff640 1 -- 172.21.15.53:0/3424374765 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a00b48d0 con 0x7f13a0162b30 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.367+0000 7f139f7fe640 1 --2- 172.21.15.53:0/3424374765 >> v2:172.21.15.135:3300/0 conn(0x7f13a0162750 0x7f13a00d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.367+0000 7f139ffff640 1 --2- 172.21.15.53:0/3424374765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0162b30 0x7f13a00b53b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1398009d90 tx=0x7f139802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a110c98626162a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f139e7fc640 1 -- 172.21.15.53:0/3424374765 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398031600 con 0x7f13a0162b30 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f139e7fc640 1 -- 172.21.15.53:0/3424374765 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13980317a0 con 0x7f13a0162b30 2024-05-10T10:34:02.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f139e7fc640 1 -- 172.21.15.53:0/3424374765 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398031b50 con 0x7f13a0162b30 2024-05-10T10:34:02.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 -- 172.21.15.53:0/3424374765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0162b30 msgr2=0x7f13a00b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:02.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 --2- 172.21.15.53:0/3424374765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0162b30 0x7f13a00b53b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1398009d90 tx=0x7f139802a5d0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 -- 172.21.15.53:0/3424374765 shutdown_connections 2024-05-10T10:34:02.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 --2- 172.21.15.53:0/3424374765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0162b30 0x7f13a00b53b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 --2- 172.21.15.53:0/3424374765 >> v2:172.21.15.135:3300/0 conn(0x7f13a0162750 0x7f13a00d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 -- 172.21.15.53:0/3424374765 >> 172.21.15.53:0/3424374765 conn(0x7f13a009e3b0 msgr2=0x7f13a00d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:02.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 -- 172.21.15.53:0/3424374765 shutdown_connections 2024-05-10T10:34:02.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.368+0000 7f13a6124640 1 -- 172.21.15.53:0/3424374765 wait complete. 2024-05-10T10:34:02.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.369+0000 7f13a6124640 1 Processor -- start 2024-05-10T10:34:02.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.370+0000 7f13a6124640 1 -- start start 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.370+0000 7f13a6124640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0162750 0x7f13a0157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.370+0000 7f13a6124640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0162b30 0x7f13a0159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.370+0000 7f13a6124640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0159750 0x7f13a015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.370+0000 7f13a6124640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00d0d30 con 0x7f13a0162b30 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.370+0000 7f13a6124640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00d0bb0 con 0x7f13a0162750 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.370+0000 7f13a6124640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f13a00d3d40 con 0x7f13a0159750 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0162b30 0x7f13a0159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0162b30 0x7f13a0159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58904/0 (socket says 172.21.15.53:58904) 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 -- 172.21.15.53:0/2867167563 learned_addr learned my addr 172.21.15.53:0/2867167563 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139ffff640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0159750 0x7f13a015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:02.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139f7fe640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0162750 0x7f13a0157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 -- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0159750 msgr2=0x7f13a015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0159750 0x7f13a015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 -- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0162750 msgr2=0x7f13a0157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0162750 0x7f13a0157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 -- 172.21.15.53:0/2867167563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a015a650 con 0x7f13a0162b30 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f139effd640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0162b30 0x7f13a0159210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f139000c460 tx=0x7f139000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f1397fff640 1 -- 172.21.15.53:0/2867167563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390017070 con 0x7f13a0162b30 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f1397fff640 1 -- 172.21.15.53:0/2867167563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1390005ce0 con 0x7f13a0162b30 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a0159db0 con 0x7f13a0162b30 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f1397fff640 1 -- 172.21.15.53:0/2867167563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390007310 con 0x7f13a0162b30 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.371+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a015a370 con 0x7f13a0162b30 2024-05-10T10:34:02.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.372+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a009e880 con 0x7f13a0162b30 2024-05-10T10:34:02.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.373+0000 7f1397fff640 1 -- 172.21.15.53:0/2867167563 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f139001a070 con 0x7f13a0162b30 2024-05-10T10:34:02.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.373+0000 7f1397fff640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:02.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.374+0000 7f1397fff640 1 -- 172.21.15.53:0/2867167563 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1390096d40 con 0x7f13a0162b30 2024-05-10T10:34:02.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.374+0000 7f139f7fe640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:02.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.374+0000 7f139f7fe640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1388009560 tx=0x7f1388009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:02.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.378+0000 7f1397fff640 1 -- 172.21.15.53:0/2867167563 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f13a009e880 con 0x7f13a0162b30 2024-05-10T10:34:02.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.611+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f13a006a220 con 0x7f13a0162b30 2024-05-10T10:34:02.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.611+0000 7f1397fff640 1 -- 172.21.15.53:0/2867167563 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f139005fc90 con 0x7f13a0162b30 2024-05-10T10:34:02.612 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T10:34:02.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 msgr2=0x7f1380078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:02.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1388009560 tx=0x7f1388009190 comp rx=0 tx=0).stop 2024-05-10T10:34:02.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0162b30 msgr2=0x7f13a0159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0162b30 0x7f13a0159210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f139000c460 tx=0x7f139000c920 comp rx=0 tx=0).stop 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 shutdown_connections 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a0159750 0x7f13a015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0162b30 0x7f13a0159210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 --2- 172.21.15.53:0/2867167563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0162750 0x7f13a0157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 >> 172.21.15.53:0/2867167563 conn(0x7f13a009e3b0 msgr2=0x7f13a00d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 shutdown_connections 2024-05-10T10:34:02.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:02.613+0000 7f13a6124640 1 -- 172.21.15.53:0/2867167563 wait complete. 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T10:34:02.625 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:34:02.626 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-10T10:34:02.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:02.626 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:02.643 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f0 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f1 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::ec4:7aff:fe88:8083/64 scope link 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:34:02.644 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether c2:a9:8a:1d:d2:37 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::bc8e:1cff:fe55:1639/64 scope link tentative 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether c2:a9:8a:1d:d2:37 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::c0a9:8aff:fe1d:d237/64 scope link tentative 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:02.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:02.645 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:34:02.708 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-05-10T10:34:02.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:02.710 DEBUG:teuthology.orchestra.run.smithi135:> route 2024-05-10T10:34:02.766 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2024-05-10T10:34:02.766 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:34:02.766 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T10:34:02.766 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T10:34:02.767 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:34:02.767 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:02.767 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:02.767 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T10:34:02.767 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T10:34:02.767 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T10:34:02.767 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:02.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:02.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:02.893 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-05-10T10:34:02.909 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T10:34:02.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:02.909 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2024-05-10T10:34:02.965 INFO:teuthology.orchestra.run.smithi135.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-10T10:34:02.965 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:02.965 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:02.965 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T10:34:02.965 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-10T10:34:02.966 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:03.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:03.077 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:03.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:03.083 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-10T10:34:03.108 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:34:03.108 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:34:03.108 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:34:03.108 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.108 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-05-10T10:34:03.109 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.109 INFO:teuthology.orchestra.run.smithi135.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:34:03.109 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether be:85:ab:ed:cf:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T10:34:03.109 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T10:34:03.109 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.109 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::bc85:abff:feed:cfe5/64 scope link 2024-05-10T10:34:03.109 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:03.110 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-10T10:34:03.176 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:34:03.176 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:34:03.176 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:34:03.176 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.176 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-05-10T10:34:03.177 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.177 INFO:teuthology.orchestra.run.smithi135.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:34:03.177 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether be:85:ab:ed:cf:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T10:34:03.177 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T10:34:03.177 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.177 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::bc85:abff:feed:cfe5/64 scope link 2024-05-10T10:34:03.177 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:03.178 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-10T10:34:03.178 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:03.179 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:03.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-10T10:34:03.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-10T10:34:03.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-10T10:34:03.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-10T10:34:03.179 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-10T10:34:03.179 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:03.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:03.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:03.331 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:03.331 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-05-10T10:34:03.331 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.1 up 2024-05-10T10:34:03.331 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.1 master ceph-brx 2024-05-10T10:34:03.331 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-05-10T10:34:03.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:03.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:34:03.408 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-10T10:34:03.423 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-10T10:34:03.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:34:03.424 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-10T10:34:03.479 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:34:03.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:03.480 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T10:34:03.563 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T10:34:03.563 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-10T10:34:03.563 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-05-10T10:34:03.572 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T10:34:03.572 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-10T10:34:03.572 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-05-10T10:34:03.572 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:34:03.572 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:34:03.577 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:34:03.578 INFO:teuthology.orchestra.run.smithi135.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-10T10:34:03.578 INFO:teuthology.orchestra.run.smithi135.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:34:03.578 INFO:teuthology.orchestra.run.smithi135.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:34:03.578 INFO:teuthology.orchestra.run.smithi135.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:34:03.578 INFO:teuthology.orchestra.run.smithi135.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:34:03.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:34:03.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:03.579 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-10T10:34:03.645 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:03.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:03.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:03.742 INFO:teuthology.orchestra.run.smithi135.stdout:client4690 2024-05-10T10:34:03.742 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:03.742 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:03.759 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:03.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:03.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:03.854 INFO:teuthology.orchestra.run.smithi135.stdout:client4690 2024-05-10T10:34:03.854 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:03.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:03.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:03.909 INFO:teuthology.orchestra.run.smithi135.stdout:client4690 2024-05-10T10:34:03.910 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/543b513e-90c7-47ff-9cdd-f7fa4d978734.client4690/status') 2024-05-10T10:34:03.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:03.968 INFO:teuthology.orchestra.run.smithi135.stdout:instance: client.4690 (3)192.168.0.2:0/250851353 2024-05-10T10:34:03.968 INFO:teuthology.orchestra.run.smithi135.stdout:blocklisted: false 2024-05-10T10:34:03.969 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records in 2024-05-10T10:34:03.969 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records out 2024-05-10T10:34:03.969 INFO:teuthology.orchestra.run.smithi135.stderr:68 bytes copied, 2.3013e-05 s, 3.0 MB/s 2024-05-10T10:34:03.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-05-10T10:34:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:03.976 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-10T10:34:04.016 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-10T10:34:04.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.133+0000 7f28c07d0640 1 Processor -- start 2024-05-10T10:34:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.133+0000 7f28c07d0640 1 -- start start 2024-05-10T10:34:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28c07d0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28b815f4e0 0x7f28b806d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28c07d0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b815f8c0 0x7f28b806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28c07d0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f28b806e450 0x7f28b80afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28c07d0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f28b80b23f0 con 0x7f28b815f4e0 2024-05-10T10:34:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28c07d0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f28b80b4150 con 0x7f28b806e450 2024-05-10T10:34:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28c07d0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f28b806e8c0 con 0x7f28b815f8c0 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28b815f4e0 0x7f28b806d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28b815f4e0 0x7f28b806d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58370/0 (socket says 172.21.15.53:58370) 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 -- 172.21.15.53:0/1040229297 learned_addr learned my addr 172.21.15.53:0/1040229297 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28bed6d640 1 --2- 172.21.15.53:0/1040229297 >> v2:172.21.15.135:3300/0 conn(0x7f28b806e450 0x7f28b80afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28bdd6b640 1 --2- 172.21.15.53:0/1040229297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b815f8c0 0x7f28b806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 -- 172.21.15.53:0/1040229297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b815f8c0 msgr2=0x7f28b806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 --2- 172.21.15.53:0/1040229297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b815f8c0 0x7f28b806df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 -- 172.21.15.53:0/1040229297 >> v2:172.21.15.135:3300/0 conn(0x7f28b806e450 msgr2=0x7f28b80afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 --2- 172.21.15.53:0/1040229297 >> v2:172.21.15.135:3300/0 conn(0x7f28b806e450 0x7f28b80afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.134+0000 7f28be56c640 1 -- 172.21.15.53:0/1040229297 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b80b3540 con 0x7f28b815f4e0 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.135+0000 7f28be56c640 1 --2- 172.21.15.53:0/1040229297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f4e0 0x7f28b806d9d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f28b8073990 tx=0x7f28b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce616ce902e81a89 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.135+0000 7f28bd56a640 1 -- 172.21.15.53:0/1040229297 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4037070 con 0x7f28b815f4e0 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.135+0000 7f28bd56a640 1 -- 172.21.15.53:0/1040229297 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b4032440 con 0x7f28b815f4e0 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.135+0000 7f28bd56a640 1 -- 172.21.15.53:0/1040229297 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4032810 con 0x7f28b815f4e0 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 -- 172.21.15.53:0/1040229297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f4e0 msgr2=0x7f28b806d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 --2- 172.21.15.53:0/1040229297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f4e0 0x7f28b806d9d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f28b8073990 tx=0x7f28b402a140 comp rx=0 tx=0).stop 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 -- 172.21.15.53:0/1040229297 shutdown_connections 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 --2- 172.21.15.53:0/1040229297 >> v2:172.21.15.135:3300/0 conn(0x7f28b806e450 0x7f28b80afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 --2- 172.21.15.53:0/1040229297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b815f8c0 0x7f28b806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 --2- 172.21.15.53:0/1040229297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f4e0 0x7f28b806d9d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 -- 172.21.15.53:0/1040229297 >> 172.21.15.53:0/1040229297 conn(0x7f28b80d2df0 msgr2=0x7f28b80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 -- 172.21.15.53:0/1040229297 shutdown_connections 2024-05-10T10:34:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.136+0000 7f28c07d0640 1 -- 172.21.15.53:0/1040229297 wait complete. 2024-05-10T10:34:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.137+0000 7f28c07d0640 1 Processor -- start 2024-05-10T10:34:04.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.137+0000 7f28c07d0640 1 -- start start 2024-05-10T10:34:04.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28c07d0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b806e450 0x7f28b81f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28c07d0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b815f4e0 0x7f28b8159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28c07d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f8c0 0x7f28b81f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28c07d0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80b3de0 con 0x7f28b815f8c0 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28c07d0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80b3b00 con 0x7f28b815f4e0 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28c07d0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f28b80b45e0 con 0x7f28b806e450 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28bed6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f8c0 0x7f28b81f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28bed6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f8c0 0x7f28b81f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58384/0 (socket says 172.21.15.53:58384) 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28bed6d640 1 -- 172.21.15.53:0/678497492 learned_addr learned my addr 172.21.15.53:0/678497492 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28bdd6b640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b815f4e0 0x7f28b8159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.138+0000 7f28be56c640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b806e450 0x7f28b81f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28bed6d640 1 -- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b806e450 msgr2=0x7f28b81f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28bed6d640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b806e450 0x7f28b81f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28bed6d640 1 -- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b815f4e0 msgr2=0x7f28b8159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28bed6d640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b815f4e0 0x7f28b8159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28bed6d640 1 -- 172.21.15.53:0/678497492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b81f8b60 con 0x7f28b815f8c0 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28bed6d640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f8c0 0x7f28b81f48b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007d20 tx=0x7f28b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28aeffd640 1 -- 172.21.15.53:0/678497492 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0014420 con 0x7f28b815f8c0 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28aeffd640 1 -- 172.21.15.53:0/678497492 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b0019070 con 0x7f28b815f8c0 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b81f8e40 con 0x7f28b815f8c0 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28aeffd640 1 -- 172.21.15.53:0/678497492 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0014820 con 0x7f28b815f8c0 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.139+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b81f50f0 con 0x7f28b815f8c0 2024-05-10T10:34:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.140+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b81f9360 con 0x7f28b815f8c0 2024-05-10T10:34:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.141+0000 7f28aeffd640 1 -- 172.21.15.53:0/678497492 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f28b00149c0 con 0x7f28b815f8c0 2024-05-10T10:34:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.141+0000 7f28aeffd640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f289c076300 0x7f289c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.141+0000 7f28aeffd640 1 -- 172.21.15.53:0/678497492 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f28b0097ef0 con 0x7f28b815f8c0 2024-05-10T10:34:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.141+0000 7f28be56c640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f289c076300 0x7f289c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.142+0000 7f28be56c640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f289c076300 0x7f289c0787c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28b4034000 tx=0x7f28b402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.146+0000 7f28aeffd640 1 -- 172.21.15.53:0/678497492 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f28b81f9360 con 0x7f28b815f8c0 2024-05-10T10:34:04.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.339+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28b806a220 con 0x7f28b815f8c0 2024-05-10T10:34:04.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.340+0000 7f28aeffd640 1 -- 172.21.15.53:0/678497492 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f28b0060f50 con 0x7f28b815f8c0 2024-05-10T10:34:04.341 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:04.341 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f289c076300 msgr2=0x7f289c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f289c076300 0x7f289c0787c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28b4034000 tx=0x7f28b402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f8c0 msgr2=0x7f28b81f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f8c0 0x7f28b81f48b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007d20 tx=0x7f28b000e4b0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 shutdown_connections 2024-05-10T10:34:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f289c076300 0x7f289c0787c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b815f8c0 0x7f28b81f48b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28b815f4e0 0x7f28b8159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 --2- 172.21.15.53:0/678497492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28b806e450 0x7f28b81f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 >> 172.21.15.53:0/678497492 conn(0x7f28b80d2df0 msgr2=0x7f28b806d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 shutdown_connections 2024-05-10T10:34:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.342+0000 7f28c07d0640 1 -- 172.21.15.53:0/678497492 wait complete. 2024-05-10T10:34:04.354 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:33:28.171977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T10:33:31.754471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T10:33:32.820243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T10:33:36.516220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T10:34:04.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:34:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.460+0000 7f730f220640 1 Processor -- start 2024-05-10T10:34:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.460+0000 7f730f220640 1 -- start start 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730f220640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080cf3e0 0x7f73080cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730f220640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73080cd500 0x7f73080cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730f220640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f730806e720 con 0x7f73080b2dd0 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730f220640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f730806de80 con 0x7f73080cd500 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730f220640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f730806e160 con 0x7f73080cf3e0 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730d7bd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f73080b2dd0 0x7f73080b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51696/0 (socket says 172.21.15.53:51696) 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730d7bd640 1 -- 172.21.15.53:0/1614007780 learned_addr learned my addr 172.21.15.53:0/1614007780 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f7307fff640 1 --2- 172.21.15.53:0/1614007780 >> v2:172.21.15.135:3300/0 conn(0x7f73080cd500 0x7f73080cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730cfbc640 1 --2- 172.21.15.53:0/1614007780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080cf3e0 0x7f73080cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f73077fe640 1 -- 172.21.15.53:0/1614007780 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358114295 0 0) 0x7f730806e720 con 0x7f73080b2dd0 2024-05-10T10:34:04.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f73077fe640 1 -- 172.21.15.53:0/1614007780 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72f8003520 con 0x7f73080b2dd0 2024-05-10T10:34:04.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730cfbc640 1 -- 172.21.15.53:0/1614007780 >> v2:172.21.15.135:3300/0 conn(0x7f73080cd500 msgr2=0x7f73080cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:04.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730cfbc640 1 --2- 172.21.15.53:0/1614007780 >> v2:172.21.15.135:3300/0 conn(0x7f73080cd500 0x7f73080cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730cfbc640 1 -- 172.21.15.53:0/1614007780 >> v1:172.21.15.53:6789/0 conn(0x7f73080b2dd0 legacy=0x7f73080b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f730cfbc640 1 -- 172.21.15.53:0/1614007780 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730806e440 con 0x7f73080cf3e0 2024-05-10T10:34:04.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.461+0000 7f7307fff640 1 --2- 172.21.15.53:0/1614007780 >> v2:172.21.15.135:3300/0 conn(0x7f73080cd500 0x7f73080cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:04.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.462+0000 7f730cfbc640 1 --2- 172.21.15.53:0/1614007780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080cf3e0 0x7f73080cf8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e11d20b67a2c4fa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.462+0000 7f73077fe640 1 -- 172.21.15.53:0/1614007780 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc037070 con 0x7f73080cf3e0 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.462+0000 7f73077fe640 1 -- 172.21.15.53:0/1614007780 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc032440 con 0x7f73080cf3e0 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.462+0000 7f73077fe640 1 -- 172.21.15.53:0/1614007780 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc032810 con 0x7f73080cf3e0 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 -- 172.21.15.53:0/1614007780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080cf3e0 msgr2=0x7f73080cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 --2- 172.21.15.53:0/1614007780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080cf3e0 0x7f73080cf8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02a180 comp rx=0 tx=0).stop 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 -- 172.21.15.53:0/1614007780 shutdown_connections 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 --2- 172.21.15.53:0/1614007780 >> v2:172.21.15.135:3300/0 conn(0x7f73080cd500 0x7f73080cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 --2- 172.21.15.53:0/1614007780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080cf3e0 0x7f73080cf8c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 -- 172.21.15.53:0/1614007780 >> 172.21.15.53:0/1614007780 conn(0x7f73080d2eb0 msgr2=0x7f73080d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 -- 172.21.15.53:0/1614007780 shutdown_connections 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.463+0000 7f730f220640 1 -- 172.21.15.53:0/1614007780 wait complete. 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.464+0000 7f730f220640 1 Processor -- start 2024-05-10T10:34:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.464+0000 7f730f220640 1 -- start start 2024-05-10T10:34:04.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730f220640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080b2dd0 0x7f73081558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730f220640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73080cd500 0x7f73080c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730f220640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73080cf3e0 0x7f73080c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730f220640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73080d2190 con 0x7f73080cf3e0 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730f220640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73080b4510 con 0x7f73080cd500 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730f220640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f73080d2310 con 0x7f73080b2dd0 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73080cf3e0 0x7f73080c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73080cf3e0 0x7f73080c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58390/0 (socket says 172.21.15.53:58390) 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 -- 172.21.15.53:0/3885487526 learned_addr learned my addr 172.21.15.53:0/3885487526 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f7307fff640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73080cd500 0x7f73080c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730cfbc640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080b2dd0 0x7f73081558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 -- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080b2dd0 msgr2=0x7f73081558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080b2dd0 0x7f73081558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 -- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73080cd500 msgr2=0x7f73080c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73080cd500 0x7f73080c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 -- 172.21.15.53:0/3885487526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73080c7670 con 0x7f73080cf3e0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730cfbc640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080b2dd0 0x7f73081558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.465+0000 7f730d7bd640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73080cf3e0 0x7f73080c9190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f730000cb40 tx=0x7f730000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.466+0000 7f73057fa640 1 -- 172.21.15.53:0/3885487526 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300017070 con 0x7f73080cf3e0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.466+0000 7f73057fa640 1 -- 172.21.15.53:0/3885487526 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f730000aca0 con 0x7f73080cf3e0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.466+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73080c7950 con 0x7f73080cf3e0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.466+0000 7f73057fa640 1 -- 172.21.15.53:0/3885487526 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300012620 con 0x7f73080cf3e0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.466+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73080c7f10 con 0x7f73080cf3e0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.467+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730806a220 con 0x7f73080cf3e0 2024-05-10T10:34:04.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.467+0000 7f73057fa640 1 -- 172.21.15.53:0/3885487526 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f730001a070 con 0x7f73080cf3e0 2024-05-10T10:34:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.468+0000 7f73057fa640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72ec0762d0 0x7f72ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.468+0000 7f73057fa640 1 -- 172.21.15.53:0/3885487526 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f73000975f0 con 0x7f73080cf3e0 2024-05-10T10:34:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.468+0000 7f730cfbc640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72ec0762d0 0x7f72ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.469+0000 7f730cfbc640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72ec0762d0 0x7f72ec078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f72fc034000 tx=0x7f72fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.472+0000 7f73057fa640 1 -- 172.21.15.53:0/3885487526 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7300060650 con 0x7f73080cf3e0 2024-05-10T10:34:04.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.663+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f730806ef30 con 0x7f73080cf3e0 2024-05-10T10:34:04.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.663+0000 7f73057fa640 1 -- 172.21.15.53:0/3885487526 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f7300064950 con 0x7f73080cf3e0 2024-05-10T10:34:04.664 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72ec0762d0 msgr2=0x7f72ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72ec0762d0 0x7f72ec078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f72fc034000 tx=0x7f72fc02e000 comp rx=0 tx=0).stop 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73080cf3e0 msgr2=0x7f73080c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73080cf3e0 0x7f73080c9190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f730000cb40 tx=0x7f730000a060 comp rx=0 tx=0).stop 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 shutdown_connections 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72ec0762d0 0x7f72ec078790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73080cf3e0 0x7f73080c9190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73080cd500 0x7f73080c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 --2- 172.21.15.53:0/3885487526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73080b2dd0 0x7f73081558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 >> 172.21.15.53:0/3885487526 conn(0x7f73080d2eb0 msgr2=0x7f7308075990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 shutdown_connections 2024-05-10T10:34:04.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.665+0000 7f730f220640 1 -- 172.21.15.53:0/3885487526 wait complete. 2024-05-10T10:34:04.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T10:34:04.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.783+0000 7f4bbff04640 1 Processor -- start 2024-05-10T10:34:04.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.784+0000 7f4bbff04640 1 -- start start 2024-05-10T10:34:04.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.784+0000 7f4bbff04640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4bb80b52a0 0x7f4bb806f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.784+0000 7f4bbff04640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb80b68c0 con 0x7f4bb815d600 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.784+0000 7f4bbff04640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb80af980 con 0x7f4bb80730b0 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.784+0000 7f4bbff04640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb806f650 con 0x7f4bb80b52a0 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbdca0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4bb815d600 0x7f4bb80b61b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51702/0 (socket says 172.21.15.53:51702) 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbdca0640 1 -- 172.21.15.53:0/2889605417 learned_addr learned my addr 172.21.15.53:0/2889605417 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888874441 0 0) 0x7f4bb80b68c0 con 0x7f4bb815d600 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ba0003520 con 0x7f4bb815d600 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbd49f640 1 --2- 172.21.15.53:0/2889605417 >> v2:172.21.15.135:3300/0 conn(0x7f4bb80b52a0 0x7f4bb806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2180984691 0 0) 0x7f4ba0003520 con 0x7f4bb815d600 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 >> v2:172.21.15.135:3300/0 conn(0x7f4bb80b52a0 msgr2=0x7f4bb806f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 --2- 172.21.15.53:0/2889605417 >> v2:172.21.15.135:3300/0 conn(0x7f4bb80b52a0 0x7f4bb806f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 >> v1:172.21.15.135:6789/0 conn(0x7f4bb80730b0 legacy=0x7f4bb80b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb80afc20 con 0x7f4bb815d600 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4bb0001670 con 0x7f4bb815d600 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 >> v1:172.21.15.53:6789/0 conn(0x7f4bb815d600 legacy=0x7f4bb80b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ba0003a00 0x7f4ba0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ba0004800 0x7f4ba0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4ba000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba000c1b0 con 0x7f4ba0005290 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba000c360 con 0x7f4ba0003a00 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba000c560 con 0x7f4ba0004800 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbe4a1640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4ba000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbdca0640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ba0003a00 0x7f4ba0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.785+0000 7f4bbd49f640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ba0004800 0x7f4ba0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbe4a1640 1 -- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ba0004800 msgr2=0x7f4ba0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbe4a1640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ba0004800 0x7f4ba0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbe4a1640 1 -- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ba0003a00 msgr2=0x7f4ba0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbe4a1640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ba0003a00 0x7f4ba0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbe4a1640 1 -- 172.21.15.53:0/2889605417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba0003520 con 0x7f4ba0005290 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbdca0640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ba0003a00 0x7f4ba0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbe4a1640 1 -- 172.21.15.53:0/2889605417 reap_dead start 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbe4a1640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4ba000b9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80044b0 tx=0x7f4ba0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a1451868fbe488 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0007c40 con 0x7f4ba0005290 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba802ada0 con 0x7f4ba0005290 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.786+0000 7f4bbcc9e640 1 -- 172.21.15.53:0/2889605417 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8004080 con 0x7f4ba0005290 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.787+0000 7f4bbff04640 1 -- 172.21.15.53:0/2889605417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 msgr2=0x7f4ba000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.787+0000 7f4bbff04640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4ba000b9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80044b0 tx=0x7f4ba0005830 comp rx=0 tx=0).stop 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.787+0000 7f4bbff04640 1 -- 172.21.15.53:0/2889605417 shutdown_connections 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.787+0000 7f4bbff04640 1 --2- 172.21.15.53:0/2889605417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4ba000b9c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.787+0000 7f4bbff04640 1 -- 172.21.15.53:0/2889605417 >> 172.21.15.53:0/2889605417 conn(0x7f4bb80d1ed0 msgr2=0x7f4bb80d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:04.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.787+0000 7f4bbff04640 1 -- 172.21.15.53:0/2889605417 shutdown_connections 2024-05-10T10:34:04.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.787+0000 7f4bbff04640 1 -- 172.21.15.53:0/2889605417 wait complete. 2024-05-10T10:34:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.788+0000 7f4bbff04640 1 Processor -- start 2024-05-10T10:34:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.788+0000 7f4bbff04640 1 -- start start 2024-05-10T10:34:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbff04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4bb81f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbff04640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb81f92c0 0x7f4bb815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbff04640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815f160 0x7f4bb806c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbff04640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb80b04c0 con 0x7f4ba0005290 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbff04640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb80b01e0 con 0x7f4bb815f160 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbff04640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb806f110 con 0x7f4bb81f92c0 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4bb81f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4bb81f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58408/0 (socket says 172.21.15.53:58408) 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 -- 172.21.15.53:0/132144299 learned_addr learned my addr 172.21.15.53:0/132144299 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbd49f640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb81f92c0 0x7f4bb815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbe4a1640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815f160 0x7f4bb806c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 -- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb81f92c0 msgr2=0x7f4bb815ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb81f92c0 0x7f4bb815ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 -- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815f160 msgr2=0x7f4bb806c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815f160 0x7f4bb806c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.789+0000 7f4bbdca0640 1 -- 172.21.15.53:0/132144299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb8160500 con 0x7f4ba0005290 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.790+0000 7f4bbdca0640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4bb81f52c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007310 tx=0x7f4bb0007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.790+0000 7f4bb67fc640 1 -- 172.21.15.53:0/132144299 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000f960 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.790+0000 7f4bb67fc640 1 -- 172.21.15.53:0/132144299 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb0018070 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.790+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81f9740 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.790+0000 7f4bb67fc640 1 -- 172.21.15.53:0/132144299 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000fd00 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.790+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81f5bf0 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.791+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb8158e30 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.791+0000 7f4bb67fc640 1 -- 172.21.15.53:0/132144299 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4bb001d460 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.792+0000 7f4bb67fc640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ba40762d0 0x7f4ba4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.792+0000 7f4bb67fc640 1 -- 172.21.15.53:0/132144299 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4bb00977c0 con 0x7f4ba0005290 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.793+0000 7f4bbd49f640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ba40762d0 0x7f4ba4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.793+0000 7f4bbd49f640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ba40762d0 0x7f4ba4078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80d0c10 tx=0x7f4bac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:04.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.797+0000 7f4bb67fc640 1 -- 172.21.15.53:0/132144299 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4bb8158e30 con 0x7f4ba0005290 2024-05-10T10:34:04.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.986+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4bb80693f0 con 0x7f4ba0005290 2024-05-10T10:34:04.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.987+0000 7f4bb67fc640 1 -- 172.21.15.53:0/132144299 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f4bb8160500 con 0x7f4ba0005290 2024-05-10T10:34:04.987 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ba40762d0 msgr2=0x7f4ba4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ba40762d0 0x7f4ba4078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80d0c10 tx=0x7f4bac008040 comp rx=0 tx=0).stop 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 msgr2=0x7f4bb81f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4bb81f52c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007310 tx=0x7f4bb0007710 comp rx=0 tx=0).stop 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 shutdown_connections 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ba40762d0 0x7f4ba4078790 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815f160 0x7f4bb806c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb81f92c0 0x7f4bb815ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 --2- 172.21.15.53:0/132144299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ba0005290 0x7f4bb81f52c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 >> 172.21.15.53:0/132144299 conn(0x7f4bb80d1ed0 msgr2=0x7f4bb80b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 shutdown_connections 2024-05-10T10:34:04.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:04.989+0000 7f4bbff04640 1 -- 172.21.15.53:0/132144299 wait complete. 2024-05-10T10:34:05.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T10:34:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.106+0000 7f8576f24640 1 Processor -- start 2024-05-10T10:34:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.106+0000 7f8576f24640 1 -- start start 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f8576f24640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8570162bc0 0x7f85700d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f8576f24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f85700b3500 0x7f85700d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f8576f24640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8570162fa0 con 0x7f85700d2230 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f8576f24640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f85700d1170 con 0x7f8570162bc0 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f8576f24640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f85700d2f70 con 0x7f85700b3500 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f8574cc0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8570162bc0 0x7f85700d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f856ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f85700d2230 0x7f85700cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51706/0 (socket says 172.21.15.53:51706) 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f856ffff640 1 -- 172.21.15.53:0/2057538246 learned_addr learned my addr 172.21.15.53:0/2057538246 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.107+0000 7f85754c1640 1 --2- 172.21.15.53:0/2057538246 >> v2:172.21.15.135:3300/0 conn(0x7f85700b3500 0x7f85700d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f8574cc0640 1 -- 172.21.15.53:0/2057538246 >> v2:172.21.15.135:3300/0 conn(0x7f85700b3500 msgr2=0x7f85700d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f8574cc0640 1 --2- 172.21.15.53:0/2057538246 >> v2:172.21.15.135:3300/0 conn(0x7f85700b3500 0x7f85700d3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f8574cc0640 1 -- 172.21.15.53:0/2057538246 >> v1:172.21.15.53:6789/0 conn(0x7f85700d2230 legacy=0x7f85700cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f8574cc0640 1 -- 172.21.15.53:0/2057538246 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85700d3250 con 0x7f8570162bc0 2024-05-10T10:34:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f856f7fe640 1 -- 172.21.15.53:0/2057538246 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3952717601 0 0) 0x7f8570162fa0 con 0x7f85700d2230 2024-05-10T10:34:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f85754c1640 1 --2- 172.21.15.53:0/2057538246 >> v2:172.21.15.135:3300/0 conn(0x7f85700b3500 0x7f85700d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f8574cc0640 1 --2- 172.21.15.53:0/2057538246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8570162bc0 0x7f85700d3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f857008e9e0 tx=0x7f856002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ad54d02c845ec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f856f7fe640 1 -- 172.21.15.53:0/2057538246 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560037070 con 0x7f8570162bc0 2024-05-10T10:34:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f856f7fe640 1 -- 172.21.15.53:0/2057538246 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8560032440 con 0x7f8570162bc0 2024-05-10T10:34:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.108+0000 7f856f7fe640 1 -- 172.21.15.53:0/2057538246 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856002ae60 con 0x7f8570162bc0 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 -- 172.21.15.53:0/2057538246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8570162bc0 msgr2=0x7f85700d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 --2- 172.21.15.53:0/2057538246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8570162bc0 0x7f85700d3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f857008e9e0 tx=0x7f856002a110 comp rx=0 tx=0).stop 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 -- 172.21.15.53:0/2057538246 shutdown_connections 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 --2- 172.21.15.53:0/2057538246 >> v2:172.21.15.135:3300/0 conn(0x7f85700b3500 0x7f85700d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 --2- 172.21.15.53:0/2057538246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8570162bc0 0x7f85700d3770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 -- 172.21.15.53:0/2057538246 >> 172.21.15.53:0/2057538246 conn(0x7f85700d5190 msgr2=0x7f85700d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 -- 172.21.15.53:0/2057538246 shutdown_connections 2024-05-10T10:34:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.109+0000 7f8576f24640 1 -- 172.21.15.53:0/2057538246 wait complete. 2024-05-10T10:34:05.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.110+0000 7f8576f24640 1 Processor -- start 2024-05-10T10:34:05.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.110+0000 7f8576f24640 1 -- start start 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8576f24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b3500 0x7f85701589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8576f24640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f85700d2230 0x7f85700b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8576f24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570162bc0 0x7f85700b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8576f24640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f85700b1cf0 con 0x7f85700b3500 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8576f24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85700d5f90 con 0x7f8570162bc0 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8576f24640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f85700b4030 con 0x7f85700d2230 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8574cc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b3500 0x7f85701589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8574cc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b3500 0x7f85701589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58418/0 (socket says 172.21.15.53:58418) 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8574cc0640 1 -- 172.21.15.53:0/1543952756 learned_addr learned my addr 172.21.15.53:0/1543952756 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f85754c1640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570162bc0 0x7f85700b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.111+0000 7f8574cc0640 1 -- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f85700d2230 msgr2=0x7f85700b99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f856ffff640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f85700d2230 0x7f85700b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f8574cc0640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f85700d2230 0x7f85700b99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f8574cc0640 1 -- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570162bc0 msgr2=0x7f85700b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f8574cc0640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570162bc0 0x7f85700b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f8574cc0640 1 -- 172.21.15.53:0/1543952756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8570158f30 con 0x7f85700b3500 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f85754c1640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570162bc0 0x7f85700b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f856ffff640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f85700d2230 0x7f85700b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f8574cc0640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b3500 0x7f85701589f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8560009890 tx=0x7f85600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f856d7fa640 1 -- 172.21.15.53:0/1543952756 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560037040 con 0x7f85700b3500 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f856d7fa640 1 -- 172.21.15.53:0/1543952756 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8560042070 con 0x7f85700b3500 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701f8be0 con 0x7f85700b3500 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f856d7fa640 1 -- 172.21.15.53:0/1543952756 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560030bd0 con 0x7f85700b3500 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.112+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701f91a0 con 0x7f85700b3500 2024-05-10T10:34:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.113+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857006a220 con 0x7f85700b3500 2024-05-10T10:34:05.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.113+0000 7f856d7fa640 1 -- 172.21.15.53:0/1543952756 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8560047460 con 0x7f85700b3500 2024-05-10T10:34:05.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.114+0000 7f856d7fa640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f855c0762d0 0x7f855c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.114+0000 7f856d7fa640 1 -- 172.21.15.53:0/1543952756 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f85600b8410 con 0x7f85700b3500 2024-05-10T10:34:05.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.114+0000 7f856ffff640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f855c0762d0 0x7f855c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.115+0000 7f856ffff640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f855c0762d0 0x7f855c078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8564007fd0 tx=0x7f8564009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.118+0000 7f856d7fa640 1 -- 172.21.15.53:0/1543952756 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8560032e20 con 0x7f85700b3500 2024-05-10T10:34:05.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.307+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f85700d1da0 con 0x7f85700b3500 2024-05-10T10:34:05.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.308+0000 7f856d7fa640 1 -- 172.21.15.53:0/1543952756 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f8560081ac0 con 0x7f85700b3500 2024-05-10T10:34:05.308 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2024-05-10T10:34:05.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f855c0762d0 msgr2=0x7f855c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f855c0762d0 0x7f855c078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8564007fd0 tx=0x7f8564009040 comp rx=0 tx=0).stop 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b3500 msgr2=0x7f85701589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b3500 0x7f85701589f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8560009890 tx=0x7f85600046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 shutdown_connections 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f855c0762d0 0x7f855c078790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570162bc0 0x7f85700b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f85700d2230 0x7f85700b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 --2- 172.21.15.53:0/1543952756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b3500 0x7f85701589f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.309+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 >> 172.21.15.53:0/1543952756 conn(0x7f85700d5190 msgr2=0x7f85700cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:05.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.310+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 shutdown_connections 2024-05-10T10:34:05.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.310+0000 7f8576f24640 1 -- 172.21.15.53:0/1543952756 wait complete. 2024-05-10T10:34:05.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T10:34:05.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.428+0000 7f8c61b0a640 1 Processor -- start 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.428+0000 7f8c61b0a640 1 -- start start 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.428+0000 7f8c61b0a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c5c0cfc00 0x7f8c5c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.428+0000 7f8c61b0a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c5c0c8df0 0x7f8c5c0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.428+0000 7f8c61b0a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c5c0b64b0 con 0x7f8c5c162a80 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.428+0000 7f8c61b0a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5c0d08d0 con 0x7f8c5c0c8df0 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.428+0000 7f8c61b0a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5c0d0bb0 con 0x7f8c5c0cfc00 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c5c0c8df0 0x7f8c5c0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8c5c162a80 0x7f8c5c0d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51712/0 (socket says 172.21.15.53:51712) 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5b7fe640 1 -- 172.21.15.53:0/559119350 learned_addr learned my addr 172.21.15.53:0/559119350 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5affd640 1 --2- 172.21.15.53:0/559119350 >> v2:172.21.15.135:3300/0 conn(0x7f8c5c0cfc00 0x7f8c5c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5bfff640 1 -- 172.21.15.53:0/559119350 >> v2:172.21.15.135:3300/0 conn(0x7f8c5c0cfc00 msgr2=0x7f8c5c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:05.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/559119350 >> v2:172.21.15.135:3300/0 conn(0x7f8c5c0cfc00 0x7f8c5c0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5bfff640 1 -- 172.21.15.53:0/559119350 >> v1:172.21.15.53:6789/0 conn(0x7f8c5c162a80 legacy=0x7f8c5c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5bfff640 1 -- 172.21.15.53:0/559119350 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c0d05d0 con 0x7f8c5c0c8df0 2024-05-10T10:34:05.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5a7fc640 1 -- 172.21.15.53:0/559119350 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 302898010 0 0) 0x7f8c5c0b64b0 con 0x7f8c5c162a80 2024-05-10T10:34:05.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.429+0000 7f8c5affd640 1 --2- 172.21.15.53:0/559119350 >> v2:172.21.15.135:3300/0 conn(0x7f8c5c0cfc00 0x7f8c5c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/559119350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c0c8df0 0x7f8c5c0c9220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c08f4b0 tx=0x7f8c5402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fca473e32f00304 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c5a7fc640 1 -- 172.21.15.53:0/559119350 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54007740 con 0x7f8c5c0c8df0 2024-05-10T10:34:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c5a7fc640 1 -- 172.21.15.53:0/559119350 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c540078e0 con 0x7f8c5c0c8df0 2024-05-10T10:34:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c5a7fc640 1 -- 172.21.15.53:0/559119350 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54007cb0 con 0x7f8c5c0c8df0 2024-05-10T10:34:05.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 -- 172.21.15.53:0/559119350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c0c8df0 msgr2=0x7f8c5c0c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/559119350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c0c8df0 0x7f8c5c0c9220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c08f4b0 tx=0x7f8c5402a710 comp rx=0 tx=0).stop 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 -- 172.21.15.53:0/559119350 shutdown_connections 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/559119350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c0c8df0 0x7f8c5c0c9220 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/559119350 >> v2:172.21.15.135:3300/0 conn(0x7f8c5c0cfc00 0x7f8c5c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 -- 172.21.15.53:0/559119350 >> 172.21.15.53:0/559119350 conn(0x7f8c5c0d50f0 msgr2=0x7f8c5c0d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 -- 172.21.15.53:0/559119350 shutdown_connections 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.430+0000 7f8c61b0a640 1 -- 172.21.15.53:0/559119350 wait complete. 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.432+0000 7f8c61b0a640 1 Processor -- start 2024-05-10T10:34:05.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.432+0000 7f8c61b0a640 1 -- start start 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c5c0c8df0 0x7f8c5c15bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c0cfc00 0x7f8c5c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c162a80 0x7f8c5c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c0d5e40 con 0x7f8c5c162a80 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c0b6140 con 0x7f8c5c0cfc00 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5c162e60 con 0x7f8c5c0c8df0 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c162a80 0x7f8c5c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c162a80 0x7f8c5c0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58430/0 (socket says 172.21.15.53:58430) 2024-05-10T10:34:05.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 -- 172.21.15.53:0/2723326493 learned_addr learned my addr 172.21.15.53:0/2723326493 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5affd640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c0cfc00 0x7f8c5c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5b7fe640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c5c0c8df0 0x7f8c5c15bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 -- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c5c0c8df0 msgr2=0x7f8c5c15bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c5c0c8df0 0x7f8c5c15bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 -- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c0cfc00 msgr2=0x7f8c5c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c0cfc00 0x7f8c5c0b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 -- 172.21.15.53:0/2723326493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c0b28a0 con 0x7f8c5c162a80 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5affd640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c0cfc00 0x7f8c5c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c5bfff640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c162a80 0x7f8c5c0b47e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c5402ad00 tx=0x7f8c54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c43fff640 1 -- 172.21.15.53:0/2723326493 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54040030 con 0x7f8c5c162a80 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c43fff640 1 -- 172.21.15.53:0/2723326493 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c54031a90 con 0x7f8c5c162a80 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c0b2b80 con 0x7f8c5c162a80 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c43fff640 1 -- 172.21.15.53:0/2723326493 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54031e60 con 0x7f8c5c162a80 2024-05-10T10:34:05.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.433+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c0b3140 con 0x7f8c5c162a80 2024-05-10T10:34:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.435+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5c08e820 con 0x7f8c5c162a80 2024-05-10T10:34:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.435+0000 7f8c43fff640 1 -- 172.21.15.53:0/2723326493 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8c54043030 con 0x7f8c5c162a80 2024-05-10T10:34:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.436+0000 7f8c43fff640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c480762d0 0x7f8c48078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.436+0000 7f8c43fff640 1 -- 172.21.15.53:0/2723326493 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8c540b8930 con 0x7f8c5c162a80 2024-05-10T10:34:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.436+0000 7f8c5b7fe640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c480762d0 0x7f8c48078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.436+0000 7f8c5b7fe640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c480762d0 0x7f8c48078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000f60 tx=0x7f8c4c007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.440+0000 7f8c43fff640 1 -- 172.21.15.53:0/2723326493 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8c5c08e820 con 0x7f8c5c162a80 2024-05-10T10:34:05.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.634+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8c5c0693f0 con 0x7f8c5c162a80 2024-05-10T10:34:05.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.634+0000 7f8c43fff640 1 -- 172.21.15.53:0/2723326493 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f8c54081f80 con 0x7f8c5c162a80 2024-05-10T10:34:05.635 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c480762d0 msgr2=0x7f8c48078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c480762d0 0x7f8c48078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000f60 tx=0x7f8c4c007b50 comp rx=0 tx=0).stop 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c162a80 msgr2=0x7f8c5c0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c162a80 0x7f8c5c0b47e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c5402ad00 tx=0x7f8c54004a40 comp rx=0 tx=0).stop 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 shutdown_connections 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c480762d0 0x7f8c48078790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c5c162a80 0x7f8c5c0b47e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c0cfc00 0x7f8c5c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 --2- 172.21.15.53:0/2723326493 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c5c0c8df0 0x7f8c5c15bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 >> 172.21.15.53:0/2723326493 conn(0x7f8c5c0d50f0 msgr2=0x7f8c5c0cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:05.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 shutdown_connections 2024-05-10T10:34:05.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.636+0000 7f8c61b0a640 1 -- 172.21.15.53:0/2723326493 wait complete. 2024-05-10T10:34:05.648 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-10T10:34:05.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.750+0000 7f5a0ee2a640 1 Processor -- start 2024-05-10T10:34:05.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.750+0000 7f5a0ee2a640 1 -- start start 2024-05-10T10:34:05.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a0ee2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162750 0x7f5a080d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a0ee2a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5a080d5cb0 0x7f5a080d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a0ee2a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a080b45f0 con 0x7f5a08162b30 2024-05-10T10:34:05.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a0ee2a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5a080b4f20 con 0x7f5a080d5cb0 2024-05-10T10:34:05.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a0ee2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5a080d0fe0 con 0x7f5a08162750 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5a080d5cb0 0x7f5a080d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a0d3c7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5a08162b30 0x7f5a080d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51724/0 (socket says 172.21.15.53:51724) 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.751+0000 7f5a0d3c7640 1 -- 172.21.15.53:0/501808684 learned_addr learned my addr 172.21.15.53:0/501808684 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a0cbc6640 1 --2- 172.21.15.53:0/501808684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162750 0x7f5a080d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a07fff640 1 -- 172.21.15.53:0/501808684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162750 msgr2=0x7f5a080d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a07fff640 1 --2- 172.21.15.53:0/501808684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162750 0x7f5a080d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a07fff640 1 -- 172.21.15.53:0/501808684 >> v1:172.21.15.53:6789/0 conn(0x7f5a08162b30 legacy=0x7f5a080d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a07fff640 1 -- 172.21.15.53:0/501808684 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a080b48d0 con 0x7f5a080d5cb0 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a077fe640 1 -- 172.21.15.53:0/501808684 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076693947 0 0) 0x7f5a080b45f0 con 0x7f5a08162b30 2024-05-10T10:34:05.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a0cbc6640 1 --2- 172.21.15.53:0/501808684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162750 0x7f5a080d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:05.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.752+0000 7f5a07fff640 1 --2- 172.21.15.53:0/501808684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080d5cb0 0x7f5a080d3630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0097a0 tx=0x7f59fc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d1468a8fc1dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a077fe640 1 -- 172.21.15.53:0/501808684 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc037070 con 0x7f5a080d5cb0 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a077fe640 1 -- 172.21.15.53:0/501808684 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59fc02ae50 con 0x7f5a080d5cb0 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a077fe640 1 -- 172.21.15.53:0/501808684 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc032660 con 0x7f5a080d5cb0 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/501808684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080d5cb0 msgr2=0x7f5a080d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/501808684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080d5cb0 0x7f5a080d3630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0097a0 tx=0x7f59fc02a100 comp rx=0 tx=0).stop 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/501808684 shutdown_connections 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/501808684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080d5cb0 0x7f5a080d3630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/501808684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162750 0x7f5a080d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/501808684 >> 172.21.15.53:0/501808684 conn(0x7f5a0809e3b0 msgr2=0x7f5a080d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/501808684 shutdown_connections 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.753+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/501808684 wait complete. 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.754+0000 7f5a0ee2a640 1 Processor -- start 2024-05-10T10:34:05.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.754+0000 7f5a0ee2a640 1 -- start start 2024-05-10T10:34:05.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0ee2a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a08162750 0x7f5a081f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0ee2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162b30 0x7f5a080b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0ee2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080b3670 0x7f5a080cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0ee2a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a080d0d30 con 0x7f5a080b3670 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0ee2a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a080d0bb0 con 0x7f5a08162750 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0ee2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5a080b63c0 con 0x7f5a08162b30 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0d3c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080b3670 0x7f5a080cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0d3c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080b3670 0x7f5a080cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58450/0 (socket says 172.21.15.53:58450) 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.755+0000 7f5a0d3c7640 1 -- 172.21.15.53:0/1237077306 learned_addr learned my addr 172.21.15.53:0/1237077306 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0cbc6640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a08162750 0x7f5a081f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a07fff640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162b30 0x7f5a080b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0d3c7640 1 -- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162b30 msgr2=0x7f5a080b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0d3c7640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162b30 0x7f5a080b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0d3c7640 1 -- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a08162750 msgr2=0x7f5a081f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:05.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0d3c7640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a08162750 0x7f5a081f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0d3c7640 1 -- 172.21.15.53:0/1237077306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a080cd1a0 con 0x7f5a080b3670 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0cbc6640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a08162750 0x7f5a081f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0d3c7640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080b3670 0x7f5a080cc400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c700 tx=0x7f5a0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a057fa640 1 -- 172.21.15.53:0/1237077306 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00018070 con 0x7f5a080b3670 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a057fa640 1 -- 172.21.15.53:0/1237077306 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0000ce10 con 0x7f5a080b3670 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a057fa640 1 -- 172.21.15.53:0/1237077306 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000a880 con 0x7f5a080b3670 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a080cd480 con 0x7f5a080b3670 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.756+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a080cda40 con 0x7f5a080b3670 2024-05-10T10:34:05.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.757+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0806a220 con 0x7f5a080b3670 2024-05-10T10:34:05.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.758+0000 7f5a057fa640 1 -- 172.21.15.53:0/1237077306 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5a0001b070 con 0x7f5a080b3670 2024-05-10T10:34:05.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.758+0000 7f5a057fa640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59f40762d0 0x7f59f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:05.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.758+0000 7f5a057fa640 1 -- 172.21.15.53:0/1237077306 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5a00096e70 con 0x7f5a080b3670 2024-05-10T10:34:05.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.758+0000 7f5a0cbc6640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59f40762d0 0x7f59f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:05.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.759+0000 7f5a0cbc6640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59f40762d0 0x7f59f4078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f59f800aa30 tx=0x7f59f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:05.762+0000 7f5a057fa640 1 -- 172.21.15.53:0/1237077306 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5a00060520 con 0x7f5a080b3670 2024-05-10T10:34:06.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.032+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f5a080b4210 con 0x7f5a080b3670 2024-05-10T10:34:06.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.651+0000 7f5a057fa640 1 -- 172.21.15.53:0/1237077306 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v47) ==== 145+0+0 (secure 0 0 0) 0x7f5a0005fe20 con 0x7f5a080b3670 2024-05-10T10:34:06.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59f40762d0 msgr2=0x7f59f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59f40762d0 0x7f59f4078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f59f800aa30 tx=0x7f59f8008040 comp rx=0 tx=0).stop 2024-05-10T10:34:06.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080b3670 msgr2=0x7f5a080cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080b3670 0x7f5a080cc400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c700 tx=0x7f5a0000cbc0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 shutdown_connections 2024-05-10T10:34:06.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59f40762d0 0x7f59f4078790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a080b3670 0x7f5a080cc400 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5a08162b30 0x7f5a080b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 --2- 172.21.15.53:0/1237077306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a08162750 0x7f5a081f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.653+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 >> 172.21.15.53:0/1237077306 conn(0x7f5a0809e3b0 msgr2=0x7f5a080d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:06.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.654+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 shutdown_connections 2024-05-10T10:34:06.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.654+0000 7f5a0ee2a640 1 -- 172.21.15.53:0/1237077306 wait complete. 2024-05-10T10:34:06.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T10:34:06.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.780+0000 7ff59a7c8640 1 Processor -- start 2024-05-10T10:34:06.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.780+0000 7ff59a7c8640 1 -- start start 2024-05-10T10:34:06.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff59a7c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff594162a80 0x7ff5940d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff59a7c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5940cfc00 0x7ff5940d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff59a7c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5940d3410 con 0x7ff5940d1720 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff59a7c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff5940b64b0 con 0x7ff5940cfc00 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff59a7c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff5940d08d0 con 0x7ff594162a80 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff5937fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5940cfc00 0x7ff5940d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff593fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff594162a80 0x7ff5940d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff5937fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5940cfc00 0x7ff5940d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:52768/0 (socket says 172.21.15.53:52768) 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff593fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff594162a80 0x7ff5940d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:58190/0 (socket says 172.21.15.53:58190) 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff5937fe640 1 -- 172.21.15.53:0/4211958947 learned_addr learned my addr 172.21.15.53:0/4211958947 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff593fff640 1 -- 172.21.15.53:0/4211958947 >> v2:172.21.15.135:3300/0 conn(0x7ff5940cfc00 msgr2=0x7ff5940d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff593fff640 1 --2- 172.21.15.53:0/4211958947 >> v2:172.21.15.135:3300/0 conn(0x7ff5940cfc00 0x7ff5940d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff593fff640 1 -- 172.21.15.53:0/4211958947 >> v1:172.21.15.135:6789/0 conn(0x7ff5940d1720 legacy=0x7ff5940d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff593fff640 1 -- 172.21.15.53:0/4211958947 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940d0bb0 con 0x7ff594162a80 2024-05-10T10:34:06.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.781+0000 7ff5937fe640 1 --2- 172.21.15.53:0/4211958947 >> v2:172.21.15.135:3300/0 conn(0x7ff5940cfc00 0x7ff5940d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.782+0000 7ff593fff640 1 --2- 172.21.15.53:0/4211958947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff594162a80 0x7ff5940d2ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e4c25c5798a01f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.782+0000 7ff592ffd640 1 -- 172.21.15.53:0/4211958947 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584037070 con 0x7ff594162a80 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.782+0000 7ff592ffd640 1 -- 172.21.15.53:0/4211958947 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff584032440 con 0x7ff594162a80 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.782+0000 7ff592ffd640 1 -- 172.21.15.53:0/4211958947 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584032810 con 0x7ff594162a80 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4211958947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff594162a80 msgr2=0x7ff5940d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4211958947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff594162a80 0x7ff5940d2ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402a210 comp rx=0 tx=0).stop 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4211958947 shutdown_connections 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4211958947 >> v2:172.21.15.135:3300/0 conn(0x7ff5940cfc00 0x7ff5940d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4211958947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff594162a80 0x7ff5940d2ed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4211958947 >> 172.21.15.53:0/4211958947 conn(0x7ff5940d50f0 msgr2=0x7ff5940d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4211958947 shutdown_connections 2024-05-10T10:34:06.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.783+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4211958947 wait complete. 2024-05-10T10:34:06.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.784+0000 7ff59a7c8640 1 Processor -- start 2024-05-10T10:34:06.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.784+0000 7ff59a7c8640 1 -- start start 2024-05-10T10:34:06.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff59a7c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5940cfc00 0x7ff5941543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:06.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff59a7c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5940d1720 0x7ff59409f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:06.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff59a7c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff594162a80 0x7ff594160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff59a7c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940b6140 con 0x7ff5940cfc00 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff59a7c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940b6790 con 0x7ff594162a80 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff59a7c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff5940d5e40 con 0x7ff5940d1720 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5940cfc00 0x7ff5941543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5940cfc00 0x7ff5941543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58458/0 (socket says 172.21.15.53:58458) 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 -- 172.21.15.53:0/4050386200 learned_addr learned my addr 172.21.15.53:0/4050386200 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff598d65640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff594162a80 0x7ff594160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff5937fe640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5940d1720 0x7ff59409f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 -- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5940d1720 msgr2=0x7ff59409f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5940d1720 0x7ff59409f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 -- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff594162a80 msgr2=0x7ff594160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff594162a80 0x7ff594160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff593fff640 1 -- 172.21.15.53:0/4050386200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940a0210 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.785+0000 7ff598d65640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff594162a80 0x7ff594160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.786+0000 7ff593fff640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5940cfc00 0x7ff5941543c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff584000c00 tx=0x7ff5840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.786+0000 7ff590ff9640 1 -- 172.21.15.53:0/4050386200 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584037040 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.786+0000 7ff590ff9640 1 -- 172.21.15.53:0/4050386200 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff584042070 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.786+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5940a04f0 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.786+0000 7ff590ff9640 1 -- 172.21.15.53:0/4050386200 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584030920 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.786+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5940a0ab0 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.787+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5940693f0 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.787+0000 7ff590ff9640 1 -- 172.21.15.53:0/4050386200 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff584030ac0 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.788+0000 7ff590ff9640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5800762d0 0x7ff580078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.788+0000 7ff590ff9640 1 -- 172.21.15.53:0/4050386200 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff5840b8370 con 0x7ff5940cfc00 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.788+0000 7ff5937fe640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5800762d0 0x7ff580078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:06.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.789+0000 7ff5937fe640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5800762d0 0x7ff580078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff59415d3a0 tx=0x7ff588008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:06.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.792+0000 7ff590ff9640 1 -- 172.21.15.53:0/4050386200 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff5840812b0 con 0x7ff5940cfc00 2024-05-10T10:34:06.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.984+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff5940b5e80 con 0x7ff5940cfc00 2024-05-10T10:34:06.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.987+0000 7ff590ff9640 1 -- 172.21.15.53:0/4050386200 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7ff5840855b0 con 0x7ff5940cfc00 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5800762d0 msgr2=0x7ff580078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5800762d0 0x7ff580078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff59415d3a0 tx=0x7ff588008040 comp rx=0 tx=0).stop 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5940cfc00 msgr2=0x7ff5941543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5940cfc00 0x7ff5941543c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff584000c00 tx=0x7ff5840046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 shutdown_connections 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5800762d0 0x7ff580078790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff594162a80 0x7ff594160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5940d1720 0x7ff59409f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 --2- 172.21.15.53:0/4050386200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5940cfc00 0x7ff5941543c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 >> 172.21.15.53:0/4050386200 conn(0x7ff5940d50f0 msgr2=0x7ff5940d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:06.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 shutdown_connections 2024-05-10T10:34:06.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:06.989+0000 7ff59a7c8640 1 -- 172.21.15.53:0/4050386200 wait complete. 2024-05-10T10:34:07.001 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:34:07.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:34:07.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:34:07.093 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:34:07.093 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:07.093 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:07.110 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:07.111 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:07.167 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:07.167 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:07.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:34:07.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.367+0000 7f145db24640 1 Processor -- start 2024-05-10T10:34:07.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.368+0000 7f145db24640 1 -- start start 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.368+0000 7f145db24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f14580d3450 0x7f14580d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.368+0000 7f145db24640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145806f3c0 con 0x7f14580702e0 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.368+0000 7f145db24640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14580d16f0 con 0x7f145815f290 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.368+0000 7f145db24640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f14580d14c0 con 0x7f14580d3450 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.368+0000 7f1457fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f14580702e0 0x7f145806d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51734/0 (socket says 172.21.15.53:51734) 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.368+0000 7f1457fff640 1 -- 172.21.15.53:0/4212678646 learned_addr learned my addr 172.21.15.53:0/4212678646 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249135542 0 0) 0x7f145806f3c0 con 0x7f14580702e0 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1440003520 con 0x7f14580702e0 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f1456ffd640 1 --2- 172.21.15.53:0/4212678646 >> v2:172.21.15.135:3300/0 conn(0x7f14580d3450 0x7f14580d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1888597437 0 0) 0x7f1440003520 con 0x7f14580702e0 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 >> v2:172.21.15.135:3300/0 conn(0x7f14580d3450 msgr2=0x7f14580d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 --2- 172.21.15.53:0/4212678646 >> v2:172.21.15.135:3300/0 conn(0x7f14580d3450 0x7f14580d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 >> v1:172.21.15.135:6789/0 conn(0x7f145815f290 legacy=0x7f14580d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14580d19d0 con 0x7f14580702e0 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f14500019f0 con 0x7f14580702e0 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 >> v1:172.21.15.53:6789/0 conn(0x7f14580702e0 legacy=0x7f145806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14400039b0 0x7f1440003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1440004800 0x7f1440004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14400051c0 0x7f144000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f144000c040 con 0x7f1440004800 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f144000c1f0 con 0x7f14400039b0 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f144000c3f0 con 0x7f14400051c0 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f1456ffd640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1440004800 0x7f1440004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f14577fe640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14400039b0 0x7f1440003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f1457fff640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14400051c0 0x7f144000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f1456ffd640 1 -- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14400051c0 msgr2=0x7f144000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f1456ffd640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14400051c0 0x7f144000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f1456ffd640 1 -- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14400039b0 msgr2=0x7f1440003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.369+0000 7f1456ffd640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14400039b0 0x7f1440003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.370+0000 7f1457fff640 1 -- 172.21.15.53:0/4212678646 reap_dead start 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.370+0000 7f1456ffd640 1 -- 172.21.15.53:0/4212678646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1440003520 con 0x7f1440004800 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.370+0000 7f14577fe640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14400039b0 0x7f1440003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.370+0000 7f1456ffd640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1440004800 0x7f1440004c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f144c009ac0 tx=0x7f1440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12aba4ee3b040d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.370+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c036070 con 0x7f1440004800 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.370+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c031400 con 0x7f1440004800 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.370+0000 7f14567fc640 1 -- 172.21.15.53:0/4212678646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c009690 con 0x7f1440004800 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.371+0000 7f145db24640 1 -- 172.21.15.53:0/4212678646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1440004800 msgr2=0x7f1440004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.371+0000 7f145db24640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1440004800 0x7f1440004c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f144c009ac0 tx=0x7f1440005830 comp rx=0 tx=0).stop 2024-05-10T10:34:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.371+0000 7f145db24640 1 -- 172.21.15.53:0/4212678646 shutdown_connections 2024-05-10T10:34:07.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.371+0000 7f145db24640 1 --2- 172.21.15.53:0/4212678646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1440004800 0x7f1440004c80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.371+0000 7f145db24640 1 -- 172.21.15.53:0/4212678646 >> 172.21.15.53:0/4212678646 conn(0x7f145809ba20 msgr2=0x7f145809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:07.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.371+0000 7f145db24640 1 -- 172.21.15.53:0/4212678646 shutdown_connections 2024-05-10T10:34:07.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.371+0000 7f145db24640 1 -- 172.21.15.53:0/4212678646 wait complete. 2024-05-10T10:34:07.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.372+0000 7f145db24640 1 Processor -- start 2024-05-10T10:34:07.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.372+0000 7f145db24640 1 -- start start 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f145db24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440004800 0x7f145815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f145db24640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14580a0680 0x7f14580b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f145db24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14580a0ae0 0x7f14580b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f145db24640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14580d1f90 con 0x7f14580a0ae0 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f145db24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14580d2300 con 0x7f1440004800 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f145db24640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f145815f670 con 0x7f14580a0680 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f1457fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14580a0ae0 0x7f14580b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f1457fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14580a0ae0 0x7f14580b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58476/0 (socket says 172.21.15.53:58476) 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f1457fff640 1 -- 172.21.15.53:0/2846981497 learned_addr learned my addr 172.21.15.53:0/2846981497 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f14577fe640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440004800 0x7f145815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.373+0000 7f1456ffd640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14580a0680 0x7f14580b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f1457fff640 1 -- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14580a0680 msgr2=0x7f14580b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f1457fff640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14580a0680 0x7f14580b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f1457fff640 1 -- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440004800 msgr2=0x7f145815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f1457fff640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440004800 0x7f145815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f1457fff640 1 -- 172.21.15.53:0/2846981497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14581f8b20 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f14577fe640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440004800 0x7f145815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f1457fff640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14580a0ae0 0x7f14580b7110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14580701d0 tx=0x7f1450007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f143ffff640 1 -- 172.21.15.53:0/2846981497 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145000eb00 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f143ffff640 1 -- 172.21.15.53:0/2846981497 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1450018070 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f143ffff640 1 -- 172.21.15.53:0/2846981497 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450013420 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14581f8e00 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.374+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14581f93c0 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.375+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14580693f0 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.376+0000 7f143ffff640 1 -- 172.21.15.53:0/2846981497 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f14500135c0 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.376+0000 7f143ffff640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1444076300 0x7f14440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.377+0000 7f143ffff640 1 -- 172.21.15.53:0/2846981497 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1450097b50 con 0x7f14580a0ae0 2024-05-10T10:34:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.377+0000 7f14577fe640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1444076300 0x7f14440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.377+0000 7f14577fe640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1444076300 0x7f14440787c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1448007fd0 tx=0x7f1448009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:07.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.381+0000 7f143ffff640 1 -- 172.21.15.53:0/2846981497 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1450061200 con 0x7f14580a0ae0 2024-05-10T10:34:07.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.588+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f145806f3c0 con 0x7f14580a0ae0 2024-05-10T10:34:07.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.588+0000 7f143ffff640 1 -- 172.21.15.53:0/2846981497 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f14500691f0 con 0x7f14580a0ae0 2024-05-10T10:34:07.589 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1444076300 msgr2=0x7f14440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1444076300 0x7f14440787c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1448007fd0 tx=0x7f1448009040 comp rx=0 tx=0).stop 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14580a0ae0 msgr2=0x7f14580b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14580a0ae0 0x7f14580b7110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14580701d0 tx=0x7f1450007bc0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 shutdown_connections 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1444076300 0x7f14440787c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14580a0ae0 0x7f14580b7110 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14580a0680 0x7f14580b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 --2- 172.21.15.53:0/2846981497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440004800 0x7f145815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 >> 172.21.15.53:0/2846981497 conn(0x7f145809ba20 msgr2=0x7f145806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:07.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 shutdown_connections 2024-05-10T10:34:07.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.590+0000 7f145db24640 1 -- 172.21.15.53:0/2846981497 wait complete. 2024-05-10T10:34:07.602 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:34:07.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:07.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-10T10:34:07.629 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:34:07.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:07.630 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:34:07.686 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:07.686 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:07.706 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:07.706 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:07.766 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:07.766 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:07.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:34:07.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.930+0000 7f59e651b640 1 Processor -- start 2024-05-10T10:34:07.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.930+0000 7f59e651b640 1 -- start start 2024-05-10T10:34:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59e651b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59e015f390 0x7f59e00d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59e651b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f59e00b4c00 0x7f59e00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59e651b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59e00cfd10 con 0x7f59e00d5190 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59e651b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f59e00cff90 con 0x7f59e015f390 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59e651b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f59e0163d00 con 0x7f59e00b4c00 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59dffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59e015f390 0x7f59e00d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59dffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59e015f390 0x7f59e00d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58480/0 (socket says 172.21.15.53:58480) 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.931+0000 7f59dffff640 1 -- 172.21.15.53:0/3361977146 learned_addr learned my addr 172.21.15.53:0/3361977146 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.932+0000 7f59e4ab8640 1 --2- 172.21.15.53:0/3361977146 >> v2:172.21.15.135:3300/0 conn(0x7f59e00b4c00 0x7f59e00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.932+0000 7f59dffff640 1 -- 172.21.15.53:0/3361977146 >> v2:172.21.15.135:3300/0 conn(0x7f59e00b4c00 msgr2=0x7f59e00b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.932+0000 7f59dffff640 1 --2- 172.21.15.53:0/3361977146 >> v2:172.21.15.135:3300/0 conn(0x7f59e00b4c00 0x7f59e00b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.932+0000 7f59dffff640 1 -- 172.21.15.53:0/3361977146 >> v1:172.21.15.135:6789/0 conn(0x7f59e00d5190 legacy=0x7f59e00d55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.932+0000 7f59dffff640 1 -- 172.21.15.53:0/3361977146 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e00d0250 con 0x7f59e015f390 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.932+0000 7f59e4ab8640 1 --2- 172.21.15.53:0/3361977146 >> v2:172.21.15.135:3300/0 conn(0x7f59e00b4c00 0x7f59e00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.932+0000 7f59dffff640 1 --2- 172.21.15.53:0/3361977146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e00d0a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59e006cda0 tx=0x7f59d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e532cecb41003b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59deffd640 1 -- 172.21.15.53:0/3361977146 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0037070 con 0x7f59e015f390 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59deffd640 1 -- 172.21.15.53:0/3361977146 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d0032440 con 0x7f59e015f390 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59deffd640 1 -- 172.21.15.53:0/3361977146 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0032810 con 0x7f59e015f390 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 -- 172.21.15.53:0/3361977146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 msgr2=0x7f59e00d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 --2- 172.21.15.53:0/3361977146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e00d0a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59e006cda0 tx=0x7f59d002a230 comp rx=0 tx=0).stop 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 -- 172.21.15.53:0/3361977146 shutdown_connections 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 --2- 172.21.15.53:0/3361977146 >> v2:172.21.15.135:3300/0 conn(0x7f59e00b4c00 0x7f59e00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 --2- 172.21.15.53:0/3361977146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e00d0a50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 -- 172.21.15.53:0/3361977146 >> 172.21.15.53:0/3361977146 conn(0x7f59e008ecb0 msgr2=0x7f59e009e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 -- 172.21.15.53:0/3361977146 shutdown_connections 2024-05-10T10:34:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.933+0000 7f59e651b640 1 -- 172.21.15.53:0/3361977146 wait complete. 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.934+0000 7f59e651b640 1 Processor -- start 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e651b640 1 -- start start 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e651b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59e00b4c00 0x7f59e01f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e651b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59e00d5190 0x7f59e015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e651b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e0160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e651b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f59e00d0f90 con 0x7f59e015f390 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e651b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59e00b5570 con 0x7f59e00d5190 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e651b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f59e00d1110 con 0x7f59e00b4c00 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e4ab8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e0160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e4ab8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e0160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58490/0 (socket says 172.21.15.53:58490) 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.935+0000 7f59e4ab8640 1 -- 172.21.15.53:0/2752190210 learned_addr learned my addr 172.21.15.53:0/2752190210 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59df7fe640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59e00d5190 0x7f59e015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59dffff640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59e00b4c00 0x7f59e01f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e4ab8640 1 -- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59e00b4c00 msgr2=0x7f59e01f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.940 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e4ab8640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59e00b4c00 0x7f59e01f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e4ab8640 1 -- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59e00d5190 msgr2=0x7f59e015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e4ab8640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59e00d5190 0x7f59e015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e4ab8640 1 -- 172.21.15.53:0/2752190210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e015f770 con 0x7f59e015f390 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59dffff640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59e00b4c00 0x7f59e01f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e4ab8640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e0160af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f59d800cef0 tx=0x7f59d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59dcff9640 1 -- 172.21.15.53:0/2752190210 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d801a070 con 0x7f59e015f390 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59dcff9640 1 -- 172.21.15.53:0/2752190210 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d800d860 con 0x7f59e015f390 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59dcff9640 1 -- 172.21.15.53:0/2752190210 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d800dc50 con 0x7f59e015f390 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e015fa50 con 0x7f59e015f390 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.936+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e0160010 con 0x7f59e015f390 2024-05-10T10:34:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.937+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e006a220 con 0x7f59e015f390 2024-05-10T10:34:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.938+0000 7f59dcff9640 1 -- 172.21.15.53:0/2752190210 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f59d801d070 con 0x7f59e015f390 2024-05-10T10:34:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.938+0000 7f59dcff9640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59cc0762d0 0x7f59cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.939+0000 7f59dcff9640 1 -- 172.21.15.53:0/2752190210 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f59d8098060 con 0x7f59e015f390 2024-05-10T10:34:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.939+0000 7f59dffff640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59cc0762d0 0x7f59cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:07.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.939+0000 7f59dffff640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59cc0762d0 0x7f59cc078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59d0034000 tx=0x7f59d002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:07.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:07.943+0000 7f59dcff9640 1 -- 172.21.15.53:0/2752190210 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f59d8061180 con 0x7f59e015f390 2024-05-10T10:34:08.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.146+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f59e00b5890 con 0x7f59e015f390 2024-05-10T10:34:08.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.146+0000 7f59dcff9640 1 -- 172.21.15.53:0/2752190210 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f59d8065480 con 0x7f59e015f390 2024-05-10T10:34:08.147 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:34:08.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59cc0762d0 msgr2=0x7f59cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:08.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59cc0762d0 0x7f59cc078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59d0034000 tx=0x7f59d002e000 comp rx=0 tx=0).stop 2024-05-10T10:34:08.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 msgr2=0x7f59e0160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:08.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e0160af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f59d800cef0 tx=0x7f59d800a710 comp rx=0 tx=0).stop 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 shutdown_connections 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59cc0762d0 0x7f59cc078790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59e015f390 0x7f59e0160af0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59e00d5190 0x7f59e015c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 --2- 172.21.15.53:0/2752190210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59e00b4c00 0x7f59e01f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.148+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 >> 172.21.15.53:0/2752190210 conn(0x7f59e008ecb0 msgr2=0x7f59e00b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.149+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 shutdown_connections 2024-05-10T10:34:08.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.149+0000 7f59e651b640 1 -- 172.21.15.53:0/2752190210 wait complete. 2024-05-10T10:34:08.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T10:34:08.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:08.160 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-10T10:34:08.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:34:08.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:08.193 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T10:34:08.267 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:08.267 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:08.283 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:08.283 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:08.339 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:08.339 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:08.396 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:34:08.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.541+0000 7f82e8bf0640 1 Processor -- start 2024-05-10T10:34:08.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.541+0000 7f82e8bf0640 1 -- start start 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e8bf0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82e4162750 0x7f82e40d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e8bf0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e40d5cb0 0x7f82e40d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e8bf0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82e4162b30 0x7f82e40d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e8bf0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f82e40b4f20 con 0x7f82e4162750 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e8bf0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f82e40d0fe0 con 0x7f82e4162b30 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e8bf0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f82e40b45f0 con 0x7f82e40d5cb0 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82e4162750 0x7f82e40d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82e4162750 0x7f82e40d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58504/0 (socket says 172.21.15.53:58504) 2024-05-10T10:34:08.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 -- 172.21.15.53:0/3414709884 learned_addr learned my addr 172.21.15.53:0/3414709884 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:08.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 -- 172.21.15.53:0/3414709884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e40d5cb0 msgr2=0x7f82e40d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:08.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e2d9d640 1 --2- 172.21.15.53:0/3414709884 >> v2:172.21.15.135:3300/0 conn(0x7f82e4162b30 0x7f82e40d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:08.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 --2- 172.21.15.53:0/3414709884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e40d5cb0 0x7f82e40d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 -- 172.21.15.53:0/3414709884 >> v2:172.21.15.135:3300/0 conn(0x7f82e4162b30 msgr2=0x7f82e40d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:08.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 --2- 172.21.15.53:0/3414709884 >> v2:172.21.15.135:3300/0 conn(0x7f82e4162b30 0x7f82e40d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.542+0000 7f82e259c640 1 -- 172.21.15.53:0/3414709884 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e40b48d0 con 0x7f82e4162750 2024-05-10T10:34:08.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.543+0000 7f82e259c640 1 --2- 172.21.15.53:0/3414709884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e4162750 0x7f82e40d0670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82e40cc0d0 tx=0x7f82d402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b013723ed93c6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:08.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.543+0000 7f82e159a640 1 -- 172.21.15.53:0/3414709884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4037070 con 0x7f82e4162750 2024-05-10T10:34:08.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.543+0000 7f82e159a640 1 -- 172.21.15.53:0/3414709884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d4032440 con 0x7f82e4162750 2024-05-10T10:34:08.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.543+0000 7f82e159a640 1 -- 172.21.15.53:0/3414709884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4032810 con 0x7f82e4162750 2024-05-10T10:34:08.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 -- 172.21.15.53:0/3414709884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e4162750 msgr2=0x7f82e40d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:08.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/3414709884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e4162750 0x7f82e40d0670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82e40cc0d0 tx=0x7f82d402a140 comp rx=0 tx=0).stop 2024-05-10T10:34:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 -- 172.21.15.53:0/3414709884 shutdown_connections 2024-05-10T10:34:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/3414709884 >> v2:172.21.15.135:3300/0 conn(0x7f82e4162b30 0x7f82e40d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/3414709884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e40d5cb0 0x7f82e40d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/3414709884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e4162750 0x7f82e40d0670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 -- 172.21.15.53:0/3414709884 >> 172.21.15.53:0/3414709884 conn(0x7f82e409e3b0 msgr2=0x7f82e40d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 -- 172.21.15.53:0/3414709884 shutdown_connections 2024-05-10T10:34:08.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.544+0000 7f82e8bf0640 1 -- 172.21.15.53:0/3414709884 wait complete. 2024-05-10T10:34:08.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.545+0000 7f82e8bf0640 1 Processor -- start 2024-05-10T10:34:08.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.545+0000 7f82e8bf0640 1 -- start start 2024-05-10T10:34:08.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e8bf0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e4162750 0x7f82e4156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:08.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e8bf0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82e4162b30 0x7f82e40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:08.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e8bf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40ccee0 0x7f82e406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:08.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e8bf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82e40d0d30 con 0x7f82e40ccee0 2024-05-10T10:34:08.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e8bf0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82e40d0bb0 con 0x7f82e4162b30 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e8bf0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f82e40d1660 con 0x7f82e4162750 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40ccee0 0x7f82e406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40ccee0 0x7f82e406dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58514/0 (socket says 172.21.15.53:58514) 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e2d9d640 1 -- 172.21.15.53:0/1746220499 learned_addr learned my addr 172.21.15.53:0/1746220499 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e259c640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e4162750 0x7f82e4156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e2d9d640 1 -- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e4162750 msgr2=0x7f82e4156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e2d9d640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e4162750 0x7f82e4156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.546+0000 7f82e1d9b640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82e4162b30 0x7f82e40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e2d9d640 1 -- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82e4162b30 msgr2=0x7f82e40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e2d9d640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82e4162b30 0x7f82e40cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e2d9d640 1 -- 172.21.15.53:0/1746220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e41f8060 con 0x7f82e40ccee0 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e1d9b640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82e4162b30 0x7f82e40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e2d9d640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40ccee0 0x7f82e406dbe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00cc30 tx=0x7f82dc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82d2ffd640 1 -- 172.21.15.53:0/1746220499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00ce40 con 0x7f82e40ccee0 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82d2ffd640 1 -- 172.21.15.53:0/1746220499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82dc003990 con 0x7f82e40ccee0 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e41f8de0 con 0x7f82e40ccee0 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82d2ffd640 1 -- 172.21.15.53:0/1746220499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc003d40 con 0x7f82e40ccee0 2024-05-10T10:34:08.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.547+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e41f85e0 con 0x7f82e40ccee0 2024-05-10T10:34:08.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.548+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e406a220 con 0x7f82e40ccee0 2024-05-10T10:34:08.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.549+0000 7f82d2ffd640 1 -- 172.21.15.53:0/1746220499 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f82dc004000 con 0x7f82e40ccee0 2024-05-10T10:34:08.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.549+0000 7f82d2ffd640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f82c40762d0 0x7f82c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:08.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.549+0000 7f82d2ffd640 1 -- 172.21.15.53:0/1746220499 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f82dc097d80 con 0x7f82e40ccee0 2024-05-10T10:34:08.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.549+0000 7f82e259c640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f82c40762d0 0x7f82c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:08.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.550+0000 7f82e259c640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f82c40762d0 0x7f82c4078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f82d4034000 tx=0x7f82d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:08.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.553+0000 7f82d2ffd640 1 -- 172.21.15.53:0/1746220499 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f82dc014030 con 0x7f82e40ccee0 2024-05-10T10:34:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.760+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f82e4075100 con 0x7f82e40ccee0 2024-05-10T10:34:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.760+0000 7f82d2ffd640 1 -- 172.21.15.53:0/1746220499 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f82dc0613a0 con 0x7f82e40ccee0 2024-05-10T10:34:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:34:08.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f82c40762d0 msgr2=0x7f82c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f82c40762d0 0x7f82c4078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f82d4034000 tx=0x7f82d402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40ccee0 msgr2=0x7f82e406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40ccee0 0x7f82e406dbe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00cc30 tx=0x7f82dc00a5c0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 shutdown_connections 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f82c40762d0 0x7f82c4078790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40ccee0 0x7f82e406dbe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82e4162b30 0x7f82e40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 --2- 172.21.15.53:0/1746220499 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82e4162750 0x7f82e4156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 >> 172.21.15.53:0/1746220499 conn(0x7f82e409e3b0 msgr2=0x7f82e40d54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 shutdown_connections 2024-05-10T10:34:08.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:08.763+0000 7f82e8bf0640 1 -- 172.21.15.53:0/1746220499 wait complete. 2024-05-10T10:34:08.774 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-10T10:34:08.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:08.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-10T10:34:08.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:34:08.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:08.802 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-10T10:34:08.860 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:08.860 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:08.881 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:08.881 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:08.940 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:08.940 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:09.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:34:09.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.105+0000 7f6a26e15640 1 Processor -- start 2024-05-10T10:34:09.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.105+0000 7f6a26e15640 1 -- start start 2024-05-10T10:34:09.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a26e15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a2015f390 0x7f6a200d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a26e15640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200d5190 0x7f6a200d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a26e15640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a200cfd10 con 0x7f6a200b4c00 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a26e15640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6a200cff90 con 0x7f6a2015f390 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a26e15640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6a20163d00 con 0x7f6a200d5190 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a24bb1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a2015f390 0x7f6a200d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a253b2640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6a200b4c00 0x7f6a200b5090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51744/0 (socket says 172.21.15.53:51744) 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a253b2640 1 -- 172.21.15.53:0/4012036010 learned_addr learned my addr 172.21.15.53:0/4012036010 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a1ffff640 1 --2- 172.21.15.53:0/4012036010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200d5190 0x7f6a200d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a24bb1640 1 -- 172.21.15.53:0/4012036010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200d5190 msgr2=0x7f6a200d55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a24bb1640 1 --2- 172.21.15.53:0/4012036010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200d5190 0x7f6a200d55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a24bb1640 1 -- 172.21.15.53:0/4012036010 >> v1:172.21.15.53:6789/0 conn(0x7f6a200b4c00 legacy=0x7f6a200b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a24bb1640 1 -- 172.21.15.53:0/4012036010 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a200d0250 con 0x7f6a2015f390 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.106+0000 7f6a1f7fe640 1 -- 172.21.15.53:0/4012036010 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164560834 0 0) 0x7f6a18001490 con 0x7f6a200b4c00 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.107+0000 7f6a24bb1640 1 --2- 172.21.15.53:0/4012036010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200d0a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6a140097a0 tx=0x7f6a1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7cb521d6bd21ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.107+0000 7f6a1f7fe640 1 -- 172.21.15.53:0/4012036010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14037070 con 0x7f6a2015f390 2024-05-10T10:34:09.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.107+0000 7f6a1f7fe640 1 -- 172.21.15.53:0/4012036010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a14032440 con 0x7f6a2015f390 2024-05-10T10:34:09.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.107+0000 7f6a1f7fe640 1 -- 172.21.15.53:0/4012036010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14032810 con 0x7f6a2015f390 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 -- 172.21.15.53:0/4012036010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 msgr2=0x7f6a200d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 --2- 172.21.15.53:0/4012036010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200d0a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6a140097a0 tx=0x7f6a1402a180 comp rx=0 tx=0).stop 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 -- 172.21.15.53:0/4012036010 shutdown_connections 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 --2- 172.21.15.53:0/4012036010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200d5190 0x7f6a200d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 --2- 172.21.15.53:0/4012036010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200d0a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 -- 172.21.15.53:0/4012036010 >> 172.21.15.53:0/4012036010 conn(0x7f6a2008ecb0 msgr2=0x7f6a2009e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 -- 172.21.15.53:0/4012036010 shutdown_connections 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.108+0000 7f6a26e15640 1 -- 172.21.15.53:0/4012036010 wait complete. 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.109+0000 7f6a26e15640 1 Processor -- start 2024-05-10T10:34:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.109+0000 7f6a26e15640 1 -- start start 2024-05-10T10:34:09.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a26e15640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200b4c00 0x7f6a200cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a26e15640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a200d5190 0x7f6a2014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a26e15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a26e15640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6a200d0f90 con 0x7f6a2015f390 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a26e15640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a200d5af0 con 0x7f6a200d5190 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a26e15640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6a200d1110 con 0x7f6a200b4c00 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58530/0 (socket says 172.21.15.53:58530) 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 -- 172.21.15.53:0/975673914 learned_addr learned my addr 172.21.15.53:0/975673914 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a24bb1640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200b4c00 0x7f6a200cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a1ffff640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a200d5190 0x7f6a2014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 -- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200b4c00 msgr2=0x7f6a200cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200b4c00 0x7f6a200cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 -- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a200d5190 msgr2=0x7f6a2014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a200d5190 0x7f6a2014c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 -- 172.21.15.53:0/975673914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a200cc8c0 con 0x7f6a2015f390 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a1ffff640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a200d5190 0x7f6a2014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.110+0000 7f6a253b2640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200ce630 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c700 tx=0x7f6a1800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.111+0000 7f6a1d7fa640 1 -- 172.21.15.53:0/975673914 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18018070 con 0x7f6a2015f390 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.111+0000 7f6a1d7fa640 1 -- 172.21.15.53:0/975673914 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1800ce10 con 0x7f6a2015f390 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.111+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a200ccba0 con 0x7f6a2015f390 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.111+0000 7f6a1d7fa640 1 -- 172.21.15.53:0/975673914 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800a860 con 0x7f6a2015f390 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.111+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a200cd160 con 0x7f6a2015f390 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.112+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2006a220 con 0x7f6a2015f390 2024-05-10T10:34:09.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.112+0000 7f6a1d7fa640 1 -- 172.21.15.53:0/975673914 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6a1801b070 con 0x7f6a2015f390 2024-05-10T10:34:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.113+0000 7f6a1d7fa640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6a040762d0 0x7f6a04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.113+0000 7f6a1d7fa640 1 -- 172.21.15.53:0/975673914 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6a18096e90 con 0x7f6a2015f390 2024-05-10T10:34:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.114+0000 7f6a24bb1640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6a040762d0 0x7f6a04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.114+0000 7f6a24bb1640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6a040762d0 0x7f6a04078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a14034000 tx=0x7f6a1402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.118+0000 7f6a1d7fa640 1 -- 172.21.15.53:0/975673914 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6a200cfd10 con 0x7f6a2015f390 2024-05-10T10:34:09.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.323+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6a2015e880 con 0x7f6a2015f390 2024-05-10T10:34:09.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.324+0000 7f6a1d7fa640 1 -- 172.21.15.53:0/975673914 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f6a1805fdd0 con 0x7f6a2015f390 2024-05-10T10:34:09.324 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6a040762d0 msgr2=0x7f6a04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6a040762d0 0x7f6a04078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a14034000 tx=0x7f6a1402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 msgr2=0x7f6a200ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200ce630 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c700 tx=0x7f6a1800cbc0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 shutdown_connections 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6a040762d0 0x7f6a04078790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2015f390 0x7f6a200ce630 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a200d5190 0x7f6a2014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 --2- 172.21.15.53:0/975673914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6a200b4c00 0x7f6a200cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 >> 172.21.15.53:0/975673914 conn(0x7f6a2008ecb0 msgr2=0x7f6a2006e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 shutdown_connections 2024-05-10T10:34:09.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.326+0000 7f6a26e15640 1 -- 172.21.15.53:0/975673914 wait complete. 2024-05-10T10:34:09.337 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-10T10:34:09.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:09.338 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-10T10:34:09.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:34:09.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:09.370 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-10T10:34:09.427 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T10:34:09.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.531+0000 7f2c0ce08640 1 Processor -- start 2024-05-10T10:34:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.531+0000 7f2c0ce08640 1 -- start start 2024-05-10T10:34:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c0ce08640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2c081608b0 0x7f2c080d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c0ce08640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 0x7f2c080d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c0ce08640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c08160c90 con 0x7f2c080b2900 2024-05-10T10:34:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c0ce08640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2c080ce5f0 con 0x7f2c081608b0 2024-05-10T10:34:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c0ce08640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2c080ce8d0 con 0x7f2c080d1230 2024-05-10T10:34:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c06d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2c080b2900 0x7f2c080b2d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51752/0 (socket says 172.21.15.53:51752) 2024-05-10T10:34:09.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c06d9d640 1 -- 172.21.15.53:0/2636673021 learned_addr learned my addr 172.21.15.53:0/2636673021 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c0659c640 1 --2- 172.21.15.53:0/2636673021 >> v2:172.21.15.135:3300/0 conn(0x7f2c081608b0 0x7f2c080d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2bfffff640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 0x7f2c080d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457976392 0 0) 0x7f2c00001490 con 0x7f2c080b2900 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bf4003520 con 0x7f2c080b2900 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1298200549 0 0) 0x7f2bf4003520 con 0x7f2c080b2900 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 msgr2=0x7f2c080d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 0x7f2c080d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 >> v2:172.21.15.135:3300/0 conn(0x7f2c081608b0 msgr2=0x7f2c080d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 --2- 172.21.15.53:0/2636673021 >> v2:172.21.15.135:3300/0 conn(0x7f2c081608b0 0x7f2c080d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c080cebb0 con 0x7f2c080b2900 2024-05-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f2c00001bc0 con 0x7f2c080b2900 2024-05-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c0659c640 1 --2- 172.21.15.53:0/2636673021 >> v2:172.21.15.135:3300/0 conn(0x7f2c081608b0 0x7f2c080d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 >> v1:172.21.15.53:6789/0 conn(0x7f2c080b2900 legacy=0x7f2c080b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 0x7f2bf40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bf4003f30 0x7f2bf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2bf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080ce5f0 con 0x7f2bf4004d40 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.532+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080ce8d0 con 0x7f2bf4003f30 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf400b9c0 con 0x7f2c080d1230 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c06d9d640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2bf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c0659c640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 0x7f2bf40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2bfffff640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bf4003f30 0x7f2bf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c06d9d640 1 -- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 msgr2=0x7f2bf40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c06d9d640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080d1230 0x7f2bf40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c06d9d640 1 -- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bf4003f30 msgr2=0x7f2bf4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:09.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c06d9d640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bf4003f30 0x7f2bf4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.533+0000 7f2c06d9d640 1 -- 172.21.15.53:0/2636673021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf4003520 con 0x7f2bf4004d40 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c06d9d640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2bf40051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000aa60 tx=0x7f2bf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d04ddf1e0fb5d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000a6b0 con 0x7f2bf4004d40 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0002f070 con 0x7f2bf4004d40 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c05d9b640 1 -- 172.21.15.53:0/2636673021 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00034470 con 0x7f2bf4004d40 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2636673021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 msgr2=0x7f2bf40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c0ce08640 1 --2- 172.21.15.53:0/2636673021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2bf40051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000aa60 tx=0x7f2bf4005830 comp rx=0 tx=0).stop 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c06d9d640 1 -- 172.21.15.53:0/2636673021 reap_dead start 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2636673021 shutdown_connections 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2636673021 >> 172.21.15.53:0/2636673021 conn(0x7f2c080d2df0 msgr2=0x7f2c080d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:09.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2636673021 shutdown_connections 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.534+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2636673021 wait complete. 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 Processor -- start 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 -- start start 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2c0814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c080cb650 0x7f2c080cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080ca8f0 0x7f2c080cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080d1f10 con 0x7f2bf4004d40 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080d1d90 con 0x7f2c080cb650 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0ce08640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2c080d03c0 con 0x7f2c080ca8f0 2024-05-10T10:34:09.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2c0814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2c0814f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58536/0 (socket says 172.21.15.53:58536) 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.536+0000 7f2c0659c640 1 -- 172.21.15.53:0/2242930638 learned_addr learned my addr 172.21.15.53:0/2242930638 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c06d9d640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080ca8f0 0x7f2c080cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2bfffff640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c080cb650 0x7f2c080cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0659c640 1 -- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080ca8f0 msgr2=0x7f2c080cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0659c640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080ca8f0 0x7f2c080cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0659c640 1 -- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c080cb650 msgr2=0x7f2c080cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0659c640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c080cb650 0x7f2c080cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0659c640 1 -- 172.21.15.53:0/2242930638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c080cb2b0 con 0x7f2bf4004d40 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2bfffff640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c080cb650 0x7f2c080cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0659c640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2c0814f2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8011d00 tx=0x7f2bf800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2bfeffd640 1 -- 172.21.15.53:0/2242930638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf800fcc0 con 0x7f2bf4004d40 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c080c71c0 con 0x7f2bf4004d40 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2bfeffd640 1 -- 172.21.15.53:0/2242930638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf800fe80 con 0x7f2bf4004d40 2024-05-10T10:34:09.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2bfeffd640 1 -- 172.21.15.53:0/2242930638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8004260 con 0x7f2bf4004d40 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.537+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c080c7740 con 0x7f2bf4004d40 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.538+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0806a220 con 0x7f2bf4004d40 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.539+0000 7f2bfeffd640 1 -- 172.21.15.53:0/2242930638 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2bf8012040 con 0x7f2bf4004d40 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.539+0000 7f2bfeffd640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2be80762d0 0x7f2be8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.539+0000 7f2bfeffd640 1 -- 172.21.15.53:0/2242930638 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2bf8097f10 con 0x7f2bf4004d40 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.539+0000 7f2bfffff640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2be80762d0 0x7f2be8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.540+0000 7f2bfffff640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2be80762d0 0x7f2be8078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2c081f8920 tx=0x7f2bf0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.544+0000 7f2bfeffd640 1 -- 172.21.15.53:0/2242930638 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2c080cb2b0 con 0x7f2bf4004d40 2024-05-10T10:34:09.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.774+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c080b0c30 con 0x7f2bf4004d40 2024-05-10T10:34:09.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.775+0000 7f2bfeffd640 1 -- 172.21.15.53:0/2242930638 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f2bf8019030 con 0x7f2bf4004d40 2024-05-10T10:34:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:09.776 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:34:09.776 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:34:09.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2be80762d0 msgr2=0x7f2be8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2be80762d0 0x7f2be8078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2c081f8920 tx=0x7f2bf0008040 comp rx=0 tx=0).stop 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 msgr2=0x7f2c0814f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2c0814f2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8011d00 tx=0x7f2bf800f610 comp rx=0 tx=0).stop 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 shutdown_connections 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2be80762d0 0x7f2be8078790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2c080ca8f0 0x7f2c080cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c080cb650 0x7f2c080cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 --2- 172.21.15.53:0/2242930638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bf4004d40 0x7f2c0814f2c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 >> 172.21.15.53:0/2242930638 conn(0x7f2c080d2df0 msgr2=0x7f2c080cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 shutdown_connections 2024-05-10T10:34:09.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.777+0000 7f2c0ce08640 1 -- 172.21.15.53:0/2242930638 wait complete. 2024-05-10T10:34:09.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:09.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.892+0000 7f5b84532640 1 Processor -- start 2024-05-10T10:34:09.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.892+0000 7f5b84532640 1 -- start start 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b84532640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c162750 0x7f5b7c0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b84532640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5b7c162b30 0x7f5b7c0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b84532640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7c0b45f0 con 0x7f5b7c0d5cb0 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b84532640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7c0b4f20 con 0x7f5b7c162b30 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b84532640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7c0d0fe0 con 0x7f5b7c162750 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b82acf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5b7c162b30 0x7f5b7c0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b81acd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5b7c0d5cb0 0x7f5b7c0d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51766/0 (socket says 172.21.15.53:51766) 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b81acd640 1 -- 172.21.15.53:0/994848313 learned_addr learned my addr 172.21.15.53:0/994848313 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b822ce640 1 --2- 172.21.15.53:0/994848313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c162750 0x7f5b7c0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b82acf640 1 -- 172.21.15.53:0/994848313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c162750 msgr2=0x7f5b7c0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:09.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b82acf640 1 --2- 172.21.15.53:0/994848313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c162750 0x7f5b7c0d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b82acf640 1 -- 172.21.15.53:0/994848313 >> v1:172.21.15.53:6789/0 conn(0x7f5b7c0d5cb0 legacy=0x7f5b7c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b82acf640 1 -- 172.21.15.53:0/994848313 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c0b48d0 con 0x7f5b7c162b30 2024-05-10T10:34:09.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.893+0000 7f5b822ce640 1 --2- 172.21.15.53:0/994848313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c162750 0x7f5b7c0d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:09.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.894+0000 7f5b82acf640 1 --2- 172.21.15.53:0/994848313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162b30 0x7f5b7c0b53b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009d90 tx=0x7f5b7402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95aba9b99b7b6130 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.894+0000 7f5b812cc640 1 -- 172.21.15.53:0/994848313 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74031600 con 0x7f5b7c162b30 2024-05-10T10:34:09.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.894+0000 7f5b812cc640 1 -- 172.21.15.53:0/994848313 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b740317a0 con 0x7f5b7c162b30 2024-05-10T10:34:09.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.894+0000 7f5b812cc640 1 -- 172.21.15.53:0/994848313 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74031b70 con 0x7f5b7c162b30 2024-05-10T10:34:09.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 -- 172.21.15.53:0/994848313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162b30 msgr2=0x7f5b7c0b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 --2- 172.21.15.53:0/994848313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162b30 0x7f5b7c0b53b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009d90 tx=0x7f5b7402a5d0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 -- 172.21.15.53:0/994848313 shutdown_connections 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 --2- 172.21.15.53:0/994848313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162b30 0x7f5b7c0b53b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 --2- 172.21.15.53:0/994848313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c162750 0x7f5b7c0d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 -- 172.21.15.53:0/994848313 >> 172.21.15.53:0/994848313 conn(0x7f5b7c09e3b0 msgr2=0x7f5b7c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 -- 172.21.15.53:0/994848313 shutdown_connections 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.895+0000 7f5b84532640 1 -- 172.21.15.53:0/994848313 wait complete. 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.896+0000 7f5b84532640 1 Processor -- start 2024-05-10T10:34:09.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.896+0000 7f5b84532640 1 -- start start 2024-05-10T10:34:09.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162750 0x7f5b7c1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b7c162b30 0x7f5b7c0b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c15b920 0x7f5b7c1f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0d0d30 con 0x7f5b7c162750 2024-05-10T10:34:09.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0d0bb0 con 0x7f5b7c162b30 2024-05-10T10:34:09.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7c0d3d40 con 0x7f5b7c15b920 2024-05-10T10:34:09.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162750 0x7f5b7c1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162750 0x7f5b7c1f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58558/0 (socket says 172.21.15.53:58558) 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 -- 172.21.15.53:0/2500881312 learned_addr learned my addr 172.21.15.53:0/2500881312 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b82acf640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c15b920 0x7f5b7c1f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b81acd640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b7c162b30 0x7f5b7c0b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 -- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c15b920 msgr2=0x7f5b7c1f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c15b920 0x7f5b7c1f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 -- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b7c162b30 msgr2=0x7f5b7c0b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b7c162b30 0x7f5b7c0b90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 -- 172.21.15.53:0/2500881312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c1f5bd0 con 0x7f5b7c162750 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b822ce640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162750 0x7f5b7c1f5690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c7f0 tx=0x7f5b7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b72ffd640 1 -- 172.21.15.53:0/2500881312 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78015070 con 0x7f5b7c162750 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b72ffd640 1 -- 172.21.15.53:0/2500881312 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78007a30 con 0x7f5b7c162750 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b72ffd640 1 -- 172.21.15.53:0/2500881312 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78007e00 con 0x7f5b7c162750 2024-05-10T10:34:09.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c1f5e70 con 0x7f5b7c162750 2024-05-10T10:34:09.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.897+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c1f6430 con 0x7f5b7c162750 2024-05-10T10:34:09.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.898+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c06a220 con 0x7f5b7c162750 2024-05-10T10:34:09.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.899+0000 7f5b72ffd640 1 -- 172.21.15.53:0/2500881312 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b78018070 con 0x7f5b7c162750 2024-05-10T10:34:09.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.899+0000 7f5b72ffd640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b600762d0 0x7f5b60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:09.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.900+0000 7f5b72ffd640 1 -- 172.21.15.53:0/2500881312 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5b78097800 con 0x7f5b7c162750 2024-05-10T10:34:09.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.900+0000 7f5b81acd640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b600762d0 0x7f5b60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:09.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.900+0000 7f5b81acd640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b600762d0 0x7f5b60078790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0b6410 tx=0x7f5b6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:09.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:09.904+0000 7f5b72ffd640 1 -- 172.21.15.53:0/2500881312 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5b78060eb0 con 0x7f5b7c162750 2024-05-10T10:34:10.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.134+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b7c161060 con 0x7f5b7c162750 2024-05-10T10:34:10.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.135+0000 7f5b72ffd640 1 -- 172.21.15.53:0/2500881312 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f5b780607b0 con 0x7f5b7c162750 2024-05-10T10:34:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:10.136 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:34:10.136 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b600762d0 msgr2=0x7f5b60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b600762d0 0x7f5b60078790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0b6410 tx=0x7f5b6c009040 comp rx=0 tx=0).stop 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162750 msgr2=0x7f5b7c1f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162750 0x7f5b7c1f5690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c7f0 tx=0x7f5b7800cbf0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 shutdown_connections 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b600762d0 0x7f5b60078790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b7c15b920 0x7f5b7c1f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b7c162b30 0x7f5b7c0b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 --2- 172.21.15.53:0/2500881312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b7c162750 0x7f5b7c1f5690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 >> 172.21.15.53:0/2500881312 conn(0x7f5b7c09e3b0 msgr2=0x7f5b7c0d4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 shutdown_connections 2024-05-10T10:34:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.137+0000 7f5b84532640 1 -- 172.21.15.53:0/2500881312 wait complete. 2024-05-10T10:34:10.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:10.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.252+0000 7f0867a2a640 1 Processor -- start 2024-05-10T10:34:10.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.252+0000 7f0867a2a640 1 -- start start 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0867a2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 0x7f08600b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0867a2a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08600d59d0 0x7f08600d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0867a2a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600d2d20 con 0x7f08600b47a0 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0867a2a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0860162d20 con 0x7f08600d59d0 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0867a2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f08600d3000 con 0x7f0860162940 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0864fc5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08600d59d0 0x7f08600d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0864fc5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08600d59d0 0x7f08600d3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58568/0 (socket says 172.21.15.53:58568) 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0864fc5640 1 -- 172.21.15.53:0/965220658 learned_addr learned my addr 172.21.15.53:0/965220658 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0864fc5640 1 -- 172.21.15.53:0/965220658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 msgr2=0x7f08600b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0864fc5640 1 --2- 172.21.15.53:0/965220658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 0x7f08600b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0864fc5640 1 -- 172.21.15.53:0/965220658 >> v1:172.21.15.135:6789/0 conn(0x7f08600b47a0 legacy=0x7f0860162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:10.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.253+0000 7f0864fc5640 1 -- 172.21.15.53:0/965220658 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600d32e0 con 0x7f08600d59d0 2024-05-10T10:34:10.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.254+0000 7f0864fc5640 1 --2- 172.21.15.53:0/965220658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600d59d0 0x7f08600d3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0850009870 tx=0x7f085002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ec2314e436eba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.254+0000 7f0857fff640 1 -- 172.21.15.53:0/965220658 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850037070 con 0x7f08600d59d0 2024-05-10T10:34:10.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.254+0000 7f0857fff640 1 -- 172.21.15.53:0/965220658 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085002ae10 con 0x7f08600d59d0 2024-05-10T10:34:10.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.254+0000 7f0857fff640 1 -- 172.21.15.53:0/965220658 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850032660 con 0x7f08600d59d0 2024-05-10T10:34:10.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 -- 172.21.15.53:0/965220658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600d59d0 msgr2=0x7f08600d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 --2- 172.21.15.53:0/965220658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600d59d0 0x7f08600d3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0850009870 tx=0x7f085002a100 comp rx=0 tx=0).stop 2024-05-10T10:34:10.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 -- 172.21.15.53:0/965220658 shutdown_connections 2024-05-10T10:34:10.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 --2- 172.21.15.53:0/965220658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600d59d0 0x7f08600d3670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 --2- 172.21.15.53:0/965220658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 0x7f08600b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 -- 172.21.15.53:0/965220658 >> 172.21.15.53:0/965220658 conn(0x7f086009e380 msgr2=0x7f08600d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:10.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 -- 172.21.15.53:0/965220658 shutdown_connections 2024-05-10T10:34:10.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.255+0000 7f0867a2a640 1 -- 172.21.15.53:0/965220658 wait complete. 2024-05-10T10:34:10.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.256+0000 7f0867a2a640 1 Processor -- start 2024-05-10T10:34:10.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.256+0000 7f0867a2a640 1 -- start start 2024-05-10T10:34:10.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0867a2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600b47a0 0x7f08601522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0867a2a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08600d59d0 0x7f08600b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0867a2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 0x7f08600ba740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0867a2a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08600d2390 con 0x7f08600b47a0 2024-05-10T10:34:10.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0867a2a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08600cfc50 con 0x7f08600d59d0 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0867a2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f08600d2510 con 0x7f0860162940 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600b47a0 0x7f08601522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600b47a0 0x7f08601522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58576/0 (socket says 172.21.15.53:58576) 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 -- 172.21.15.53:0/3219862606 learned_addr learned my addr 172.21.15.53:0/3219862606 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0865fc7640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 0x7f08600ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0864fc5640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08600d59d0 0x7f08600b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 -- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 msgr2=0x7f08600ba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 0x7f08600ba740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 -- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08600d59d0 msgr2=0x7f08600b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08600d59d0 0x7f08600b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f08657c6640 1 -- 172.21.15.53:0/3219862606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600b1c70 con 0x7f08600b47a0 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.257+0000 7f0864fc5640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08600d59d0 0x7f08600b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.258+0000 7f08657c6640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600b47a0 0x7f08601522e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f085c00b4a0 tx=0x7f085c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.258+0000 7f0855ffb640 1 -- 172.21.15.53:0/3219862606 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c014070 con 0x7f08600b47a0 2024-05-10T10:34:10.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.258+0000 7f0855ffb640 1 -- 172.21.15.53:0/3219862606 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c010610 con 0x7f08600b47a0 2024-05-10T10:34:10.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.258+0000 7f0855ffb640 1 -- 172.21.15.53:0/3219862606 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c010a00 con 0x7f08600b47a0 2024-05-10T10:34:10.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.258+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08600b1f10 con 0x7f08600b47a0 2024-05-10T10:34:10.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.258+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08600b24d0 con 0x7f08600b47a0 2024-05-10T10:34:10.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.259+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086006a220 con 0x7f08600b47a0 2024-05-10T10:34:10.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.259+0000 7f0855ffb640 1 -- 172.21.15.53:0/3219862606 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f085c017070 con 0x7f08600b47a0 2024-05-10T10:34:10.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.260+0000 7f0855ffb640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08440762d0 0x7f0844078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.260+0000 7f0855ffb640 1 -- 172.21.15.53:0/3219862606 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f085c0972a0 con 0x7f08600b47a0 2024-05-10T10:34:10.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.261+0000 7f0864fc5640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08440762d0 0x7f0844078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.261+0000 7f0864fc5640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08440762d0 0x7f0844078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0850034000 tx=0x7f085002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.265+0000 7f0855ffb640 1 -- 172.21.15.53:0/3219862606 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f085c060270 con 0x7f08600b47a0 2024-05-10T10:34:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.459+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0860161f50 con 0x7f08600b47a0 2024-05-10T10:34:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.460+0000 7f0855ffb640 1 -- 172.21.15.53:0/3219862606 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f085c064570 con 0x7f08600b47a0 2024-05-10T10:34:10.461 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:10.461 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:10.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08440762d0 msgr2=0x7f0844078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08440762d0 0x7f0844078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0850034000 tx=0x7f085002e000 comp rx=0 tx=0).stop 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600b47a0 msgr2=0x7f08601522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600b47a0 0x7f08601522e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f085c00b4a0 tx=0x7f085c00b960 comp rx=0 tx=0).stop 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 shutdown_connections 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08440762d0 0x7f0844078790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0860162940 0x7f08600ba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08600d59d0 0x7f08600b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 --2- 172.21.15.53:0/3219862606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08600b47a0 0x7f08601522e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 >> 172.21.15.53:0/3219862606 conn(0x7f086009e380 msgr2=0x7f08600b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 shutdown_connections 2024-05-10T10:34:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.463+0000 7f0867a2a640 1 -- 172.21.15.53:0/3219862606 wait complete. 2024-05-10T10:34:10.475 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:34:10.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:10.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.580+0000 7f3e5c1c8640 1 Processor -- start 2024-05-10T10:34:10.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.580+0000 7f3e5c1c8640 1 -- start start 2024-05-10T10:34:10.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5c1c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3e5415f3b0 0x7f3e540d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5c1c8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3e540d5d30 0x7f3e540d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5c1c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e540d30b0 con 0x7f3e5415f790 2024-05-10T10:34:10.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5c1c8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3e540d3390 con 0x7f3e540d5d30 2024-05-10T10:34:10.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5c1c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3e540c8c80 con 0x7f3e5415f3b0 2024-05-10T10:34:10.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5a765640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3e540d5d30 0x7f3e540d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5a765640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3e540d5d30 0x7f3e540d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58586/0 (socket says 172.21.15.53:58586) 2024-05-10T10:34:10.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.581+0000 7f3e5a765640 1 -- 172.21.15.53:0/607498725 learned_addr learned my addr 172.21.15.53:0/607498725 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:10.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.582+0000 7f3e5a765640 1 -- 172.21.15.53:0/607498725 >> v2:172.21.15.135:3300/0 conn(0x7f3e5415f3b0 msgr2=0x7f3e540d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:10.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.582+0000 7f3e5a765640 1 --2- 172.21.15.53:0/607498725 >> v2:172.21.15.135:3300/0 conn(0x7f3e5415f3b0 0x7f3e540d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.582+0000 7f3e5a765640 1 -- 172.21.15.53:0/607498725 >> v1:172.21.15.135:6789/0 conn(0x7f3e5415f790 legacy=0x7f3e540c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:10.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.582+0000 7f3e5a765640 1 -- 172.21.15.53:0/607498725 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e540c8f60 con 0x7f3e540d5d30 2024-05-10T10:34:10.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.582+0000 7f3e5a765640 1 --2- 172.21.15.53:0/607498725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e540d5d30 0x7f3e540d3c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e5408e820 tx=0x7f3e4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc43442297530d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e58f62640 1 -- 172.21.15.53:0/607498725 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c036070 con 0x7f3e540d5d30 2024-05-10T10:34:10.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e58f62640 1 -- 172.21.15.53:0/607498725 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4c031600 con 0x7f3e540d5d30 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e58f62640 1 -- 172.21.15.53:0/607498725 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0319d0 con 0x7f3e540d5d30 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/607498725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e540d5d30 msgr2=0x7f3e540d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/607498725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e540d5d30 0x7f3e540d3c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e5408e820 tx=0x7f3e4c02a6f0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/607498725 shutdown_connections 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/607498725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e540d5d30 0x7f3e540d3c30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/607498725 >> v2:172.21.15.135:3300/0 conn(0x7f3e5415f3b0 0x7f3e540d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/607498725 >> 172.21.15.53:0/607498725 conn(0x7f3e540d5130 msgr2=0x7f3e540d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/607498725 shutdown_connections 2024-05-10T10:34:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.583+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/607498725 wait complete. 2024-05-10T10:34:10.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.584+0000 7f3e5c1c8640 1 Processor -- start 2024-05-10T10:34:10.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e5c1c8640 1 -- start start 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e5c1c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e540d5d30 0x7f3e541504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e5c1c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e5415f3b0 0x7f3e540ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e5c1c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3e540cd120 0x7f3e540b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e5c1c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3e540d0e60 con 0x7f3e5415f3b0 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e5c1c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e540d0ce0 con 0x7f3e540d5d30 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e5c1c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3e540d0fe0 con 0x7f3e540cd120 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e59763640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e5415f3b0 0x7f3e540ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e59763640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e5415f3b0 0x7f3e540ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58594/0 (socket says 172.21.15.53:58594) 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.585+0000 7f3e59763640 1 -- 172.21.15.53:0/1127511265 learned_addr learned my addr 172.21.15.53:0/1127511265 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e59f64640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e540d5d30 0x7f3e541504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e5a765640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3e540cd120 0x7f3e540b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e59763640 1 -- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3e540cd120 msgr2=0x7f3e540b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e59763640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3e540cd120 0x7f3e540b8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e59763640 1 -- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e540d5d30 msgr2=0x7f3e541504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e59763640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e540d5d30 0x7f3e541504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e59763640 1 -- 172.21.15.53:0/1127511265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e540cd6d0 con 0x7f3e5415f3b0 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e59763640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e5415f3b0 0x7f3e540ccbe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b3f0 tx=0x7f3e4400b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e4a7fc640 1 -- 172.21.15.53:0/1127511265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44015020 con 0x7f3e5415f3b0 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e4a7fc640 1 -- 172.21.15.53:0/1127511265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e44011830 con 0x7f3e5415f3b0 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e540cd9b0 con 0x7f3e5415f3b0 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e4a7fc640 1 -- 172.21.15.53:0/1127511265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44011c00 con 0x7f3e5415f3b0 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.586+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e540cdf70 con 0x7f3e5415f3b0 2024-05-10T10:34:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.587+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5406a220 con 0x7f3e5415f3b0 2024-05-10T10:34:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.588+0000 7f3e4a7fc640 1 -- 172.21.15.53:0/1127511265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3e44018020 con 0x7f3e5415f3b0 2024-05-10T10:34:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.589+0000 7f3e4a7fc640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3e38076300 0x7f3e380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.589+0000 7f3e4a7fc640 1 -- 172.21.15.53:0/1127511265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3e440976d0 con 0x7f3e5415f3b0 2024-05-10T10:34:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.589+0000 7f3e59f64640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3e38076300 0x7f3e380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.590+0000 7f3e59f64640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3e38076300 0x7f3e380787c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e541621c0 tx=0x7f3e50009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.593+0000 7f3e4a7fc640 1 -- 172.21.15.53:0/1127511265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3e4401f020 con 0x7f3e5415f3b0 2024-05-10T10:34:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.826+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e540c86c0 con 0x7f3e5415f3b0 2024-05-10T10:34:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.827+0000 7f3e4a7fc640 1 -- 172.21.15.53:0/1127511265 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f3e44060cf0 con 0x7f3e5415f3b0 2024-05-10T10:34:10.828 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:10.828 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:34:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:34:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.828+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3e38076300 msgr2=0x7f3e380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3e38076300 0x7f3e380787c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e541621c0 tx=0x7f3e50009110 comp rx=0 tx=0).stop 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e5415f3b0 msgr2=0x7f3e540ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e5415f3b0 0x7f3e540ccbe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b3f0 tx=0x7f3e4400b7f0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 shutdown_connections 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3e38076300 0x7f3e380787c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3e540cd120 0x7f3e540b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e5415f3b0 0x7f3e540ccbe0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 --2- 172.21.15.53:0/1127511265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e540d5d30 0x7f3e541504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 >> 172.21.15.53:0/1127511265 conn(0x7f3e540d5130 msgr2=0x7f3e540c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 shutdown_connections 2024-05-10T10:34:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.829+0000 7f3e5c1c8640 1 -- 172.21.15.53:0/1127511265 wait complete. 2024-05-10T10:34:10.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.943+0000 7fdd9cb78640 1 Processor -- start 2024-05-10T10:34:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.943+0000 7fdd9cb78640 1 -- start start 2024-05-10T10:34:10.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd9cb78640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd98162dd0 0x7fdd980cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd9cb78640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd980b5e80 con 0x7fdd980d2270 2024-05-10T10:34:10.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd9cb78640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd980d5c50 con 0x7fdd980d3980 2024-05-10T10:34:10.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd9cb78640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fdd980b5c30 con 0x7fdd98162dd0 2024-05-10T10:34:10.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd8ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd98162dd0 0x7fdd980cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd96d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdd980d2270 0x7fdd980d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51774/0 (socket says 172.21.15.53:51774) 2024-05-10T10:34:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd8ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd98162dd0 0x7fdd980cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58600/0 (socket says 172.21.15.53:58600) 2024-05-10T10:34:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd96d9d640 1 -- 172.21.15.53:0/1605995572 learned_addr learned my addr 172.21.15.53:0/1605995572 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd8ffff640 1 -- 172.21.15.53:0/1605995572 >> v1:172.21.15.135:6789/0 conn(0x7fdd980d3980 legacy=0x7fdd981626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd8ffff640 1 -- 172.21.15.53:0/1605995572 >> v1:172.21.15.53:6789/0 conn(0x7fdd980d2270 legacy=0x7fdd980d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd8ffff640 1 -- 172.21.15.53:0/1605995572 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd980d5f30 con 0x7fdd98162dd0 2024-05-10T10:34:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.944+0000 7fdd95d9b640 1 -- 172.21.15.53:0/1605995572 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137078471 0 0) 0x7fdd980b5e80 con 0x7fdd980d2270 2024-05-10T10:34:10.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.945+0000 7fdd8ffff640 1 --2- 172.21.15.53:0/1605995572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd98162dd0 0x7fdd980cfeb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdd800097a0 tx=0x7fdd8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39450fece51d9f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.945+0000 7fdd95d9b640 1 -- 172.21.15.53:0/1605995572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80037070 con 0x7fdd98162dd0 2024-05-10T10:34:10.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.945+0000 7fdd95d9b640 1 -- 172.21.15.53:0/1605995572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd80032440 con 0x7fdd98162dd0 2024-05-10T10:34:10.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.945+0000 7fdd95d9b640 1 -- 172.21.15.53:0/1605995572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80032810 con 0x7fdd98162dd0 2024-05-10T10:34:10.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.946+0000 7fdd9cb78640 1 -- 172.21.15.53:0/1605995572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd98162dd0 msgr2=0x7fdd980cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.946+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/1605995572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd98162dd0 0x7fdd980cfeb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdd800097a0 tx=0x7fdd8002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.946+0000 7fdd9cb78640 1 -- 172.21.15.53:0/1605995572 shutdown_connections 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.946+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/1605995572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd98162dd0 0x7fdd980cfeb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.946+0000 7fdd9cb78640 1 -- 172.21.15.53:0/1605995572 >> 172.21.15.53:0/1605995572 conn(0x7fdd980d5130 msgr2=0x7fdd980d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.946+0000 7fdd9cb78640 1 -- 172.21.15.53:0/1605995572 shutdown_connections 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.946+0000 7fdd9cb78640 1 -- 172.21.15.53:0/1605995572 wait complete. 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.947+0000 7fdd9cb78640 1 Processor -- start 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.947+0000 7fdd9cb78640 1 -- start start 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd9cb78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd980d2270 0x7fdd980ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd9cb78640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd980d3980 0x7fdd981535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd9cb78640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd98162dd0 0x7fdd980ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd9cb78640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd980b6890 con 0x7fdd980d3980 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd9cb78640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd980d31f0 con 0x7fdd980d2270 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd9cb78640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdd980b6160 con 0x7fdd98162dd0 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd980d3980 0x7fdd981535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd980d3980 0x7fdd981535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58602/0 (socket says 172.21.15.53:58602) 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 -- 172.21.15.53:0/2820071540 learned_addr learned my addr 172.21.15.53:0/2820071540 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd9659c640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd980d2270 0x7fdd980ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 -- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd98162dd0 msgr2=0x7fdd980ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd98162dd0 0x7fdd980ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 -- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd980d2270 msgr2=0x7fdd980ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd980d2270 0x7fdd980ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.948+0000 7fdd8ffff640 1 -- 172.21.15.53:0/2820071540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd98153af0 con 0x7fdd980d3980 2024-05-10T10:34:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.949+0000 7fdd8ffff640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd980d3980 0x7fdd981535b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.949+0000 7fdd8effd640 1 -- 172.21.15.53:0/2820071540 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80041070 con 0x7fdd980d3980 2024-05-10T10:34:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.949+0000 7fdd8effd640 1 -- 172.21.15.53:0/2820071540 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8003c930 con 0x7fdd980d3980 2024-05-10T10:34:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.949+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd980cea10 con 0x7fdd980d3980 2024-05-10T10:34:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.949+0000 7fdd8effd640 1 -- 172.21.15.53:0/2820071540 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80037040 con 0x7fdd980d3980 2024-05-10T10:34:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.949+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd980cd510 con 0x7fdd980d3980 2024-05-10T10:34:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.950+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9806a220 con 0x7fdd980d3980 2024-05-10T10:34:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.950+0000 7fdd8effd640 1 -- 172.21.15.53:0/2820071540 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdd80044070 con 0x7fdd980d3980 2024-05-10T10:34:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.951+0000 7fdd8effd640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd78076300 0x7fdd780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.951+0000 7fdd8effd640 1 -- 172.21.15.53:0/2820071540 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdd800b83b0 con 0x7fdd980d3980 2024-05-10T10:34:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.951+0000 7fdd9659c640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd78076300 0x7fdd780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.952+0000 7fdd9659c640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd78076300 0x7fdd780787c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd88007fd0 tx=0x7fdd88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:10.955+0000 7fdd8effd640 1 -- 172.21.15.53:0/2820071540 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdd98153af0 con 0x7fdd980d3980 2024-05-10T10:34:11.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.186+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd980693f0 con 0x7fdd980d3980 2024-05-10T10:34:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.186+0000 7fdd8effd640 1 -- 172.21.15.53:0/2820071540 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7fdd80081a60 con 0x7fdd980d3980 2024-05-10T10:34:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:11.187 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/395283671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":395283671},{"type":"v1","addr":"172.21.15.53:6833","nonce":395283671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/421607356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":421607356},{"type":"v1","addr":"172.21.15.53:6835","nonce":421607356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2186497982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2186497982},{"type":"v1","addr":"172.21.15.135:6837","nonce":2186497982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:33:33.864840+0000","modified":"2024-05-10T10:33:46.950725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/4124737039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4124737039},{"type":"v1","addr":"172.21.15.135:6836","nonce":4124737039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-05-10T10:34:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2024-05-10T10:34:11.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd78076300 msgr2=0x7fdd780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd78076300 0x7fdd780787c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd88007fd0 tx=0x7fdd88009040 comp rx=0 tx=0).stop 2024-05-10T10:34:11.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd980d3980 msgr2=0x7fdd981535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd980d3980 0x7fdd981535b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd800046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 shutdown_connections 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd78076300 0x7fdd780787c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd98162dd0 0x7fdd980ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd980d3980 0x7fdd981535b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.188+0000 7fdd9cb78640 1 --2- 172.21.15.53:0/2820071540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd980d2270 0x7fdd980ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.189+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 >> 172.21.15.53:0/2820071540 conn(0x7fdd980d5130 msgr2=0x7fdd980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.189+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 shutdown_connections 2024-05-10T10:34:11.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.189+0000 7fdd9cb78640 1 -- 172.21.15.53:0/2820071540 wait complete. 2024-05-10T10:34:11.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:11.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.303+0000 7faf8b872640 1 Processor -- start 2024-05-10T10:34:11.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.303+0000 7faf8b872640 1 -- start start 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf8b872640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf8b872640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faf840d59d0 0x7faf840d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf8b872640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf840b47a0 0x7faf840d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf8b872640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7faf84162d20 con 0x7faf840b47a0 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf8b872640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7faf840d2d20 con 0x7faf840d59d0 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf8b872640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7faf840d3000 con 0x7faf84162940 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf89e0f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf840b47a0 0x7faf840d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf89e0f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf840b47a0 0x7faf840d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58616/0 (socket says 172.21.15.53:58616) 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.304+0000 7faf89e0f640 1 -- 172.21.15.53:0/1400114837 learned_addr learned my addr 172.21.15.53:0/1400114837 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf88e0d640 1 --2- 172.21.15.53:0/1400114837 >> v2:172.21.15.135:3300/0 conn(0x7faf840d59d0 0x7faf840d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf8960e640 1 --2- 172.21.15.53:0/1400114837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf89e0f640 1 -- 172.21.15.53:0/1400114837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 msgr2=0x7faf840b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:11.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf89e0f640 1 --2- 172.21.15.53:0/1400114837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf89e0f640 1 -- 172.21.15.53:0/1400114837 >> v2:172.21.15.135:3300/0 conn(0x7faf840d59d0 msgr2=0x7faf840d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:11.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf89e0f640 1 --2- 172.21.15.53:0/1400114837 >> v2:172.21.15.135:3300/0 conn(0x7faf840d59d0 0x7faf840d3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf89e0f640 1 -- 172.21.15.53:0/1400114837 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf840d32e0 con 0x7faf840b47a0 2024-05-10T10:34:11.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf8960e640 1 --2- 172.21.15.53:0/1400114837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:11.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.305+0000 7faf89e0f640 1 --2- 172.21.15.53:0/1400114837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840b47a0 0x7faf840d3bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf8408f8a0 tx=0x7faf7c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e9e5777c0b41ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:11.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf7bfff640 1 -- 172.21.15.53:0/1400114837 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c036070 con 0x7faf840b47a0 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf7bfff640 1 -- 172.21.15.53:0/1400114837 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7c031600 con 0x7faf840b47a0 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf7bfff640 1 -- 172.21.15.53:0/1400114837 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0319d0 con 0x7faf840b47a0 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 -- 172.21.15.53:0/1400114837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840b47a0 msgr2=0x7faf840d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 --2- 172.21.15.53:0/1400114837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840b47a0 0x7faf840d3bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf8408f8a0 tx=0x7faf7c02a570 comp rx=0 tx=0).stop 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 -- 172.21.15.53:0/1400114837 shutdown_connections 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 --2- 172.21.15.53:0/1400114837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840b47a0 0x7faf840d3bb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 --2- 172.21.15.53:0/1400114837 >> v2:172.21.15.135:3300/0 conn(0x7faf840d59d0 0x7faf840d3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 --2- 172.21.15.53:0/1400114837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 -- 172.21.15.53:0/1400114837 >> 172.21.15.53:0/1400114837 conn(0x7faf8409e380 msgr2=0x7faf840d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 -- 172.21.15.53:0/1400114837 shutdown_connections 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.306+0000 7faf8b872640 1 -- 172.21.15.53:0/1400114837 wait complete. 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.307+0000 7faf8b872640 1 Processor -- start 2024-05-10T10:34:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf8b872640 1 -- start start 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf8b872640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf840b47a0 0x7faf840b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf8b872640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840d59d0 0x7faf840b9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf8b872640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf8b872640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf840d2390 con 0x7faf840d59d0 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf8b872640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf840cfc50 con 0x7faf840b47a0 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf8b872640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7faf840d2510 con 0x7faf84162940 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf88e0d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840d59d0 0x7faf840b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf88e0d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840d59d0 0x7faf840b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58620/0 (socket says 172.21.15.53:58620) 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.308+0000 7faf88e0d640 1 -- 172.21.15.53:0/2121283278 learned_addr learned my addr 172.21.15.53:0/2121283278 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf88e0d640 1 -- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 msgr2=0x7faf840b9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf88e0d640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf88e0d640 1 -- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf840b47a0 msgr2=0x7faf840b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf8960e640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf840b47a0 0x7faf840b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf88e0d640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf840b47a0 0x7faf840b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf88e0d640 1 -- 172.21.15.53:0/2121283278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf840b8130 con 0x7faf840d59d0 2024-05-10T10:34:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf88e0d640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840d59d0 0x7faf840b9780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf7400d830 tx=0x7faf7400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf79ffb640 1 -- 172.21.15.53:0/2121283278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74018030 con 0x7faf840d59d0 2024-05-10T10:34:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf79ffb640 1 -- 172.21.15.53:0/2121283278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7400be10 con 0x7faf840d59d0 2024-05-10T10:34:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf79ffb640 1 -- 172.21.15.53:0/2121283278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740098b0 con 0x7faf840d59d0 2024-05-10T10:34:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf840b2960 con 0x7faf840d59d0 2024-05-10T10:34:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.309+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf840b2f20 con 0x7faf840d59d0 2024-05-10T10:34:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.310+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf8406a220 con 0x7faf840d59d0 2024-05-10T10:34:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.311+0000 7faf79ffb640 1 -- 172.21.15.53:0/2121283278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faf7401b070 con 0x7faf840d59d0 2024-05-10T10:34:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.311+0000 7faf79ffb640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faf680762d0 0x7faf68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.312+0000 7faf79ffb640 1 -- 172.21.15.53:0/2121283278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf74097af0 con 0x7faf840d59d0 2024-05-10T10:34:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.312+0000 7faf8960e640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faf680762d0 0x7faf68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.312+0000 7faf8960e640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faf680762d0 0x7faf68078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf8000b840 tx=0x7faf80008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:11.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.316+0000 7faf79ffb640 1 -- 172.21.15.53:0/2121283278 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7faf74061110 con 0x7faf840d59d0 2024-05-10T10:34:11.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.509+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf8415ddc0 con 0x7faf840d59d0 2024-05-10T10:34:11.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.510+0000 7faf79ffb640 1 -- 172.21.15.53:0/2121283278 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7faf74060a10 con 0x7faf840d59d0 2024-05-10T10:34:11.511 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:11.511 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":24,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:33:38.904220+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:33:31.754471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T10:33:32.820243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T10:33:36.516220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:11.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faf680762d0 msgr2=0x7faf68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faf680762d0 0x7faf68078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf8000b840 tx=0x7faf80008040 comp rx=0 tx=0).stop 2024-05-10T10:34:11.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840d59d0 msgr2=0x7faf840b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840d59d0 0x7faf840b9780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf7400d830 tx=0x7faf7400dcf0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 shutdown_connections 2024-05-10T10:34:11.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faf680762d0 0x7faf68078790 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf84162940 0x7faf840b9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf840d59d0 0x7faf840b9780 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 --2- 172.21.15.53:0/2121283278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf840b47a0 0x7faf840b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 >> 172.21.15.53:0/2121283278 conn(0x7faf8409e380 msgr2=0x7faf840d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:11.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 shutdown_connections 2024-05-10T10:34:11.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.513+0000 7faf8b872640 1 -- 172.21.15.53:0/2121283278 wait complete. 2024-05-10T10:34:11.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:34:11.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.629+0000 7ff10e02c640 1 Processor -- start 2024-05-10T10:34:11.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.629+0000 7ff10e02c640 1 -- start start 2024-05-10T10:34:11.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff10e02c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1080d3980 0x7ff1081626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff10e02c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff108162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff10e02c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1080b5e80 con 0x7ff1080d2270 2024-05-10T10:34:11.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff10e02c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff1080b5c30 con 0x7ff1080d3980 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff10e02c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff1080d5c50 con 0x7ff1080d0ec0 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff1077fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1080d3980 0x7ff1081626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff1077fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1080d3980 0x7ff1081626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58622/0 (socket says 172.21.15.53:58622) 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff1077fe640 1 -- 172.21.15.53:0/1011955175 learned_addr learned my addr 172.21.15.53:0/1011955175 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff107fff640 1 --2- 172.21.15.53:0/1011955175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff108162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff1077fe640 1 -- 172.21.15.53:0/1011955175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 msgr2=0x7ff108162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff1077fe640 1 --2- 172.21.15.53:0/1011955175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff108162c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff1077fe640 1 -- 172.21.15.53:0/1011955175 >> v1:172.21.15.135:6789/0 conn(0x7ff1080d2270 legacy=0x7ff1080b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.630+0000 7ff1077fe640 1 -- 172.21.15.53:0/1011955175 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1080d5f30 con 0x7ff1080d3980 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.631+0000 7ff1077fe640 1 --2- 172.21.15.53:0/1011955175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d3980 0x7ff1081626c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009870 tx=0x7ff0f802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27da1b9c212d7b45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.631+0000 7ff1067fc640 1 -- 172.21.15.53:0/1011955175 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8037070 con 0x7ff1080d3980 2024-05-10T10:34:11.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.631+0000 7ff1067fc640 1 -- 172.21.15.53:0/1011955175 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f8032440 con 0x7ff1080d3980 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.631+0000 7ff1067fc640 1 -- 172.21.15.53:0/1011955175 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8032810 con 0x7ff1080d3980 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 -- 172.21.15.53:0/1011955175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d3980 msgr2=0x7ff1081626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1011955175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d3980 0x7ff1081626c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009870 tx=0x7ff0f802a210 comp rx=0 tx=0).stop 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 -- 172.21.15.53:0/1011955175 shutdown_connections 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1011955175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff108162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1011955175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d3980 0x7ff1081626c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 -- 172.21.15.53:0/1011955175 >> 172.21.15.53:0/1011955175 conn(0x7ff1080d5130 msgr2=0x7ff1080d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 -- 172.21.15.53:0/1011955175 shutdown_connections 2024-05-10T10:34:11.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.632+0000 7ff10e02c640 1 -- 172.21.15.53:0/1011955175 wait complete. 2024-05-10T10:34:11.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.633+0000 7ff10e02c640 1 Processor -- start 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.633+0000 7ff10e02c640 1 -- start start 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff10e02c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff10815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff10e02c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d2270 0x7ff1080ba750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff10e02c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1080d3980 0x7ff1080b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff10e02c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080b6890 con 0x7ff1080d2270 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff10e02c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080d31f0 con 0x7ff1080d3980 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff10e02c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff1080b6160 con 0x7ff1080d0ec0 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d2270 0x7ff1080ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d2270 0x7ff1080ba750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58634/0 (socket says 172.21.15.53:58634) 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 -- 172.21.15.53:0/1687901256 learned_addr learned my addr 172.21.15.53:0/1687901256 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff1077fe640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff10815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff107fff640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1080d3980 0x7ff1080b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 -- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 msgr2=0x7ff10815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff10815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 -- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1080d3980 msgr2=0x7ff1080b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1080d3980 0x7ff1080b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 -- 172.21.15.53:0/1687901256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1080b7df0 con 0x7ff1080d2270 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff106ffd640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d2270 0x7ff1080ba750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00c400 tx=0x7ff0fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.634+0000 7ff0effff640 1 -- 172.21.15.53:0/1687901256 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc011790 con 0x7ff1080d2270 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.635+0000 7ff0effff640 1 -- 172.21.15.53:0/1687901256 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0fc017070 con 0x7ff1080d2270 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.635+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1080b80d0 con 0x7ff1080d2270 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.635+0000 7ff0effff640 1 -- 172.21.15.53:0/1687901256 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc011bb0 con 0x7ff1080d2270 2024-05-10T10:34:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.635+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1080b8690 con 0x7ff1080d2270 2024-05-10T10:34:11.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.636+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10806a220 con 0x7ff1080d2270 2024-05-10T10:34:11.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.636+0000 7ff0effff640 1 -- 172.21.15.53:0/1687901256 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff0fc01c460 con 0x7ff1080d2270 2024-05-10T10:34:11.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.637+0000 7ff0effff640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0f40762d0 0x7ff0f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.637+0000 7ff0effff640 1 -- 172.21.15.53:0/1687901256 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff0fc097670 con 0x7ff1080d2270 2024-05-10T10:34:11.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.637+0000 7ff1077fe640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0f40762d0 0x7ff0f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.637+0000 7ff1077fe640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0f40762d0 0x7ff0f4078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8034000 tx=0x7ff0f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:11.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.641+0000 7ff0effff640 1 -- 172.21.15.53:0/1687901256 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff0fc060630 con 0x7ff1080d2270 2024-05-10T10:34:11.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.873+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff1080b7670 con 0x7ff1080d2270 2024-05-10T10:34:11.874 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:34:11.873+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:34:11.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.875+0000 7ff0effff640 1 -- 172.21.15.53:0/1687901256 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7ff0fc064930 con 0x7ff1080d2270 2024-05-10T10:34:11.875 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:34:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0f40762d0 msgr2=0x7ff0f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0f40762d0 0x7ff0f4078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8034000 tx=0x7ff0f802e000 comp rx=0 tx=0).stop 2024-05-10T10:34:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d2270 msgr2=0x7ff1080ba750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d2270 0x7ff1080ba750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00c400 tx=0x7ff0fc00c8c0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 shutdown_connections 2024-05-10T10:34:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0f40762d0 0x7ff0f4078790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1080d3980 0x7ff1080b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080d2270 0x7ff1080ba750 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 --2- 172.21.15.53:0/1687901256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1080d0ec0 0x7ff10815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 >> 172.21.15.53:0/1687901256 conn(0x7ff1080d5130 msgr2=0x7ff10815f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 shutdown_connections 2024-05-10T10:34:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.877+0000 7ff10e02c640 1 -- 172.21.15.53:0/1687901256 wait complete. 2024-05-10T10:34:11.882 INFO:tasks.ceph.mds.b.smithi135.stderr: -4045> 2024-05-10T10:33:39.453+0000 7fbc79f60640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:34:11.889 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:34:11.914 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:34:11.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.994+0000 7fdd3a3b6640 1 Processor -- start 2024-05-10T10:34:11.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.994+0000 7fdd3a3b6640 1 -- start start 2024-05-10T10:34:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd3a3b6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd340d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd3a3b6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdd340d2120 0x7fdd340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd3a3b6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd340d41d0 con 0x7fdd340b4aa0 2024-05-10T10:34:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd3a3b6640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdd340cfd50 con 0x7fdd340d2120 2024-05-10T10:34:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd3a3b6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdd340b5d90 con 0x7fdd340d3df0 2024-05-10T10:34:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd33fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd340d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd38953640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdd340d2120 0x7fdd340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd33fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd340d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:58376/0 (socket says 172.21.15.53:58376) 2024-05-10T10:34:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.995+0000 7fdd33fff640 1 -- 172.21.15.53:0/3775760982 learned_addr learned my addr 172.21.15.53:0/3775760982 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.996+0000 7fdd33fff640 1 -- 172.21.15.53:0/3775760982 >> v2:172.21.15.135:3300/0 conn(0x7fdd340d2120 msgr2=0x7fdd340b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.996+0000 7fdd33fff640 1 --2- 172.21.15.53:0/3775760982 >> v2:172.21.15.135:3300/0 conn(0x7fdd340d2120 0x7fdd340b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.996+0000 7fdd33fff640 1 -- 172.21.15.53:0/3775760982 >> v1:172.21.15.135:6789/0 conn(0x7fdd340b4aa0 legacy=0x7fdd340d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.996+0000 7fdd33fff640 1 -- 172.21.15.53:0/3775760982 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd340b54f0 con 0x7fdd340d3df0 2024-05-10T10:34:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.996+0000 7fdd33fff640 1 --2- 172.21.15.53:0/3775760982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd340d2d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009870 tx=0x7fdd2402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=501a28ea0a80ea67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.996+0000 7fdd32ffd640 1 -- 172.21.15.53:0/3775760982 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24037070 con 0x7fdd340d3df0 2024-05-10T10:34:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd32ffd640 1 -- 172.21.15.53:0/3775760982 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd24032440 con 0x7fdd340d3df0 2024-05-10T10:34:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd32ffd640 1 -- 172.21.15.53:0/3775760982 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24032810 con 0x7fdd340d3df0 2024-05-10T10:34:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/3775760982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 msgr2=0x7fdd340d2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/3775760982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd340d2d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009870 tx=0x7fdd2402a270 comp rx=0 tx=0).stop 2024-05-10T10:34:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/3775760982 shutdown_connections 2024-05-10T10:34:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/3775760982 >> v2:172.21.15.135:3300/0 conn(0x7fdd340d2120 0x7fdd340b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/3775760982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd340d2d10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/3775760982 >> 172.21.15.53:0/3775760982 conn(0x7fdd3409e450 msgr2=0x7fdd340d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/3775760982 shutdown_connections 2024-05-10T10:34:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.997+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/3775760982 wait complete. 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.998+0000 7fdd3a3b6640 1 Processor -- start 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.998+0000 7fdd3a3b6640 1 -- start start 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd3a3b6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd340b4aa0 0x7fdd34158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd3a3b6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd340d2120 0x7fdd340b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd3a3b6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd3406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd3a3b6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340cff80 con 0x7fdd340d2120 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd3a3b6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340b5ab0 con 0x7fdd340b4aa0 2024-05-10T10:34:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd3a3b6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdd340b6120 con 0x7fdd340d3df0 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd340d2120 0x7fdd340b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd340d2120 0x7fdd340b9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58640/0 (socket says 172.21.15.53:58640) 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd337fe640 1 -- 172.21.15.53:0/147594595 learned_addr learned my addr 172.21.15.53:0/147594595 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd33fff640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd340b4aa0 0x7fdd34158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:11.999+0000 7fdd38953640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd3406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd337fe640 1 -- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 msgr2=0x7fdd3406dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd337fe640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd3406dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd337fe640 1 -- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd340b4aa0 msgr2=0x7fdd34158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd337fe640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd340b4aa0 0x7fdd34158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd337fe640 1 -- 172.21.15.53:0/147594595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd340bacb0 con 0x7fdd340d2120 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd33fff640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd340b4aa0 0x7fdd34158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd337fe640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd340d2120 0x7fdd340b9e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00c400 tx=0x7fdd1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd30ff9640 1 -- 172.21.15.53:0/147594595 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0052d0 con 0x7fdd340d2120 2024-05-10T10:34:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd30ff9640 1 -- 172.21.15.53:0/147594595 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c005490 con 0x7fdd340d2120 2024-05-10T10:34:12.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd30ff9640 1 -- 172.21.15.53:0/147594595 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c007260 con 0x7fdd340d2120 2024-05-10T10:34:12.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd340ba6f0 con 0x7fdd340d2120 2024-05-10T10:34:12.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.000+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd340c8a00 con 0x7fdd340d2120 2024-05-10T10:34:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.001+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3406a220 con 0x7fdd340d2120 2024-05-10T10:34:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.002+0000 7fdd30ff9640 1 -- 172.21.15.53:0/147594595 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdd1c007400 con 0x7fdd340d2120 2024-05-10T10:34:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.002+0000 7fdd30ff9640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd140762d0 0x7fdd14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.003+0000 7fdd30ff9640 1 -- 172.21.15.53:0/147594595 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6618+0+0 (secure 0 0 0) 0x7fdd1c096ea0 con 0x7fdd340d2120 2024-05-10T10:34:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.003+0000 7fdd33fff640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd140762d0 0x7fdd14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.003+0000 7fdd33fff640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd140762d0 0x7fdd14078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd24034000 tx=0x7fdd2402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:12.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.007+0000 7fdd30ff9640 1 -- 172.21.15.53:0/147594595 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdd1c05fd90 con 0x7fdd340d2120 2024-05-10T10:34:12.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.236+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdd3408f200 con 0x7fdd340d2120 2024-05-10T10:34:12.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.879+0000 7fdd30ff9640 1 -- 172.21.15.53:0/147594595 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fdd1c064090 con 0x7fdd340d2120 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd140762d0 msgr2=0x7fdd14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd140762d0 0x7fdd14078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd24034000 tx=0x7fdd2402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd340d2120 msgr2=0x7fdd340b9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd340d2120 0x7fdd340b9e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00c400 tx=0x7fdd1c00c8c0 comp rx=0 tx=0).stop 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 shutdown_connections 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd140762d0 0x7fdd14078790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd340d3df0 0x7fdd3406dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd340d2120 0x7fdd340b9e50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 --2- 172.21.15.53:0/147594595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd340b4aa0 0x7fdd34158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:12.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.881+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 >> 172.21.15.53:0/147594595 conn(0x7fdd3409e450 msgr2=0x7fdd340b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:12.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.882+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 shutdown_connections 2024-05-10T10:34:12.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:12.882+0000 7fdd3a3b6640 1 -- 172.21.15.53:0/147594595 wait complete. 2024-05-10T10:34:12.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:34:13.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.004+0000 7f1ed0874640 1 Processor -- start 2024-05-10T10:34:13.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.004+0000 7f1ed0874640 1 -- start start 2024-05-10T10:34:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1ed0874640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1ecc15f370 0x7f1ecc0d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1ed0874640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc162e00 0x7f1ecc163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1ed0874640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ecc0716f0 con 0x7f1ecc1639a0 2024-05-10T10:34:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1ed0874640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1ecc0d2510 con 0x7f1ecc15f370 2024-05-10T10:34:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1ed0874640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1ecc070220 con 0x7f1ecc162e00 2024-05-10T10:34:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1eca59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1ecc15f370 0x7f1ecc0d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1ecad9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc162e00 0x7f1ecc163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1eca59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1ecc15f370 0x7f1ecc0d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:39270/0 (socket says 172.21.15.53:39270) 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1eca59c640 1 -- 172.21.15.53:0/4206616386 learned_addr learned my addr 172.21.15.53:0/4206616386 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.005+0000 7f1ec9d9b640 1 --1- 172.21.15.53:0/4206616386 >> v1:172.21.15.135:6789/0 conn(0x7f1ecc1639a0 0x7f1ecc070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:58474/0 (socket says 172.21.15.53:58474) 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.006+0000 7f1eca59c640 1 -- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc162e00 msgr2=0x7f1ecc163290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.006+0000 7f1eca59c640 1 --2- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc162e00 0x7f1ecc163290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.006+0000 7f1eca59c640 1 -- 172.21.15.53:0/4206616386 >> v1:172.21.15.135:6789/0 conn(0x7f1ecc1639a0 legacy=0x7f1ecc070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.006+0000 7f1eca59c640 1 -- 172.21.15.53:0/4206616386 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc070500 con 0x7f1ecc15f370 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.006+0000 7f1ec959a640 1 -- 172.21.15.53:0/4206616386 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936727498 0 0) 0x7f1ecc0716f0 con 0x7f1ecc1639a0 2024-05-10T10:34:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.006+0000 7f1ecad9d640 1 --2- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc162e00 0x7f1ecc163290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.006+0000 7f1eca59c640 1 --2- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0d1cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009870 tx=0x7f1ebc02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b3bbdf3b1798497 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ec959a640 1 -- 172.21.15.53:0/4206616386 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc037070 con 0x7f1ecc15f370 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ec959a640 1 -- 172.21.15.53:0/4206616386 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ebc02ae50 con 0x7f1ecc15f370 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ec959a640 1 -- 172.21.15.53:0/4206616386 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc032860 con 0x7f1ecc15f370 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 -- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc15f370 msgr2=0x7f1ecc0d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 --2- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0d1cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009870 tx=0x7f1ebc02a300 comp rx=0 tx=0).stop 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 -- 172.21.15.53:0/4206616386 shutdown_connections 2024-05-10T10:34:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 --2- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc162e00 0x7f1ecc163290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 --2- 172.21.15.53:0/4206616386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0d1cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 -- 172.21.15.53:0/4206616386 >> 172.21.15.53:0/4206616386 conn(0x7f1ecc0d2e10 msgr2=0x7f1ecc0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 -- 172.21.15.53:0/4206616386 shutdown_connections 2024-05-10T10:34:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.007+0000 7f1ed0874640 1 -- 172.21.15.53:0/4206616386 wait complete. 2024-05-10T10:34:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 Processor -- start 2024-05-10T10:34:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 -- start start 2024-05-10T10:34:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc162e00 0x7f1ecc151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc1639a0 0x7f1ecc0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc0d2200 con 0x7f1ecc15f370 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc070ac0 con 0x7f1ecc162e00 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1ed0874640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1ecc0cd460 con 0x7f1ecc1639a0 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1eca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1eca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59552/0 (socket says 172.21.15.53:59552) 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.009+0000 7f1eca59c640 1 -- 172.21.15.53:0/920790373 learned_addr learned my addr 172.21.15.53:0/920790373 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ecad9d640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc1639a0 0x7f1ecc0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ec9d9b640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc162e00 0x7f1ecc151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1eca59c640 1 -- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc1639a0 msgr2=0x7f1ecc0cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1eca59c640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc1639a0 0x7f1ecc0cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1eca59c640 1 -- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc162e00 msgr2=0x7f1ecc151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1eca59c640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc162e00 0x7f1ecc151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1eca59c640 1 -- 172.21.15.53:0/920790373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc1586d0 con 0x7f1ecc15f370 2024-05-10T10:34:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ecad9d640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc1639a0 0x7f1ecc0cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ec9d9b640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc162e00 0x7f1ecc151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1eca59c640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0c6940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097c0 tx=0x7f1ebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ebaffd640 1 -- 172.21.15.53:0/920790373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc037040 con 0x7f1ecc15f370 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc0c6e80 con 0x7f1ecc15f370 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ebaffd640 1 -- 172.21.15.53:0/920790373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ebc004230 con 0x7f1ecc15f370 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ebaffd640 1 -- 172.21.15.53:0/920790373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003610 con 0x7f1ecc15f370 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.010+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc0cb350 con 0x7f1ecc15f370 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.011+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc06a220 con 0x7f1ecc15f370 2024-05-10T10:34:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.012+0000 7f1ebaffd640 1 -- 172.21.15.53:0/920790373 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ebc03a070 con 0x7f1ecc15f370 2024-05-10T10:34:13.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.012+0000 7f1ebaffd640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eac0762d0 0x7f1eac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:13.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.012+0000 7f1ebaffd640 1 -- 172.21.15.53:0/920790373 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6618+0+0 (secure 0 0 0) 0x7f1ebc0b7fa0 con 0x7f1ecc15f370 2024-05-10T10:34:13.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.013+0000 7f1ec9d9b640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eac0762d0 0x7f1eac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:13.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.013+0000 7f1ec9d9b640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eac0762d0 0x7f1eac078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc15aaf0 tx=0x7f1ec0007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:13.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.017+0000 7f1ebaffd640 1 -- 172.21.15.53:0/920790373 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1ebc041020 con 0x7f1ecc15f370 2024-05-10T10:34:13.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.212+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ecc0b8d90 con 0x7f1ecc15f370 2024-05-10T10:34:13.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:34:13.694 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:34:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.881+0000 7f1ebaffd640 1 -- 172.21.15.53:0/920790373 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7f1ebc0814f0 con 0x7f1ecc15f370 2024-05-10T10:34:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:34:13.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eac0762d0 msgr2=0x7f1eac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:13.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eac0762d0 0x7f1eac078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc15aaf0 tx=0x7f1ec0007ca0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ecc15f370 msgr2=0x7f1ecc0c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:13.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0c6940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097c0 tx=0x7f1ebc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 shutdown_connections 2024-05-10T10:34:13.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eac0762d0 0x7f1eac078790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ecc1639a0 0x7f1ecc0cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc162e00 0x7f1ecc151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 --2- 172.21.15.53:0/920790373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ecc15f370 0x7f1ecc0c6940 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097c0 tx=0x7f1ebc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:13.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 >> 172.21.15.53:0/920790373 conn(0x7f1ecc0d2e10 msgr2=0x7f1ecc0759c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:13.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 shutdown_connections 2024-05-10T10:34:13.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:13.884+0000 7f1ed0874640 1 -- 172.21.15.53:0/920790373 wait complete. 2024-05-10T10:34:13.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:34:14.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.011+0000 7f6214f14640 1 Processor -- start 2024-05-10T10:34:14.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.012+0000 7f6214f14640 1 -- start start 2024-05-10T10:34:14.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.012+0000 7f6214f14640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f62100b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:14.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.012+0000 7f6214f14640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f62100737a0 0x7f62100aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f6214f14640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62100b4d20 con 0x7f62100b0ee0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f6214f14640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f62100b4f90 con 0x7f62100737a0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f6214f14640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f62100b5270 con 0x7f621015bda0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f62100737a0 0x7f62100aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620dd9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f62100b0ee0 0x7f6210073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33948/0 (socket says 172.21.15.53:33948) 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620dd9b640 1 -- 172.21.15.53:0/3954032177 learned_addr learned my addr 172.21.15.53:0/3954032177 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620e59c640 1 --2- 172.21.15.53:0/3954032177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f62100b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620ed9d640 1 -- 172.21.15.53:0/3954032177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 msgr2=0x7f62100b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620ed9d640 1 --2- 172.21.15.53:0/3954032177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f62100b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620ed9d640 1 -- 172.21.15.53:0/3954032177 >> v1:172.21.15.53:6789/0 conn(0x7f62100b0ee0 legacy=0x7f6210073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620ed9d640 1 -- 172.21.15.53:0/3954032177 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62100b5550 con 0x7f62100737a0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.013+0000 7f620d59a640 1 -- 172.21.15.53:0/3954032177 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945514215 0 0) 0x7f62100b4d20 con 0x7f62100b0ee0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.014+0000 7f620ed9d640 1 --2- 172.21.15.53:0/3954032177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f62100aed60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6208009d90 tx=0x7f620802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d25083ee023df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:14.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.014+0000 7f620d59a640 1 -- 172.21.15.53:0/3954032177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208007740 con 0x7f62100737a0 2024-05-10T10:34:14.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.014+0000 7f620d59a640 1 -- 172.21.15.53:0/3954032177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62080078e0 con 0x7f62100737a0 2024-05-10T10:34:14.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.014+0000 7f620d59a640 1 -- 172.21.15.53:0/3954032177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208007cb0 con 0x7f62100737a0 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.014+0000 7f6214f14640 1 -- 172.21.15.53:0/3954032177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 msgr2=0x7f62100aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.014+0000 7f6214f14640 1 --2- 172.21.15.53:0/3954032177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f62100aed60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6208009d90 tx=0x7f620802a660 comp rx=0 tx=0).stop 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.015+0000 7f6214f14640 1 -- 172.21.15.53:0/3954032177 shutdown_connections 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.015+0000 7f6214f14640 1 --2- 172.21.15.53:0/3954032177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f62100aed60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.015+0000 7f6214f14640 1 --2- 172.21.15.53:0/3954032177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f62100b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.015+0000 7f6214f14640 1 -- 172.21.15.53:0/3954032177 >> 172.21.15.53:0/3954032177 conn(0x7f62100b4530 msgr2=0x7f62100d1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.015+0000 7f6214f14640 1 -- 172.21.15.53:0/3954032177 shutdown_connections 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.015+0000 7f6214f14640 1 -- 172.21.15.53:0/3954032177 wait complete. 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.016+0000 7f6214f14640 1 Processor -- start 2024-05-10T10:34:14.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.016+0000 7f6214f14640 1 -- start start 2024-05-10T10:34:14.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f6214f14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f6210156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f6214f14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100b0ee0 0x7f6210155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f6214f14640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f621009ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f6214f14640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f621015c300 con 0x7f62100737a0 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f6214f14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f621015c180 con 0x7f62100b0ee0 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f6214f14640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6210071360 con 0x7f621015bda0 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f6210156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f6210156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59570/0 (socket says 172.21.15.53:59570) 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 -- 172.21.15.53:0/9341095 learned_addr learned my addr 172.21.15.53:0/9341095 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:14.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620ed9d640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f621009ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620dd9b640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100b0ee0 0x7f6210155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 -- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 msgr2=0x7f621009ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f621009ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 -- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100b0ee0 msgr2=0x7f6210155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100b0ee0 0x7f6210155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 -- 172.21.15.53:0/9341095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62101573d0 con 0x7f62100737a0 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.017+0000 7f620e59c640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f6210156e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f620000d620 tx=0x7f620000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.018+0000 7f6206ffd640 1 -- 172.21.15.53:0/9341095 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000b620 con 0x7f62100737a0 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.018+0000 7f6206ffd640 1 -- 172.21.15.53:0/9341095 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6200019070 con 0x7f62100737a0 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.018+0000 7f6206ffd640 1 -- 172.21.15.53:0/9341095 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000ba00 con 0x7f62100737a0 2024-05-10T10:34:14.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.018+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62101f62f0 con 0x7f62100737a0 2024-05-10T10:34:14.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.018+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62101f68b0 con 0x7f62100737a0 2024-05-10T10:34:14.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.019+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62100693f0 con 0x7f62100737a0 2024-05-10T10:34:14.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.019+0000 7f6206ffd640 1 -- 172.21.15.53:0/9341095 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f620000bba0 con 0x7f62100737a0 2024-05-10T10:34:14.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.020+0000 7f6206ffd640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61f0076300 0x7f61f00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:14.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.020+0000 7f6206ffd640 1 -- 172.21.15.53:0/9341095 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6180+0+0 (secure 0 0 0) 0x7f6200097a00 con 0x7f62100737a0 2024-05-10T10:34:14.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.020+0000 7f620dd9b640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61f0076300 0x7f61f00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:14.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.021+0000 7f620dd9b640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61f0076300 0x7f61f00787c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6210073f30 tx=0x7f61f8007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:14.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.024+0000 7f6206ffd640 1 -- 172.21.15.53:0/9341095 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6200060bc0 con 0x7f62100737a0 2024-05-10T10:34:14.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.216+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f62100d1070 con 0x7f62100737a0 2024-05-10T10:34:14.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.888+0000 7f6206ffd640 1 -- 172.21.15.53:0/9341095 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f6200069030 con 0x7f62100737a0 2024-05-10T10:34:14.889 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61f0076300 msgr2=0x7f61f00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61f0076300 0x7f61f00787c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6210073f30 tx=0x7f61f8007d50 comp rx=0 tx=0).stop 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 msgr2=0x7f6210156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f6210156e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f620000d620 tx=0x7f620000dae0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 shutdown_connections 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61f0076300 0x7f61f00787c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f621015bda0 0x7f621009ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100b0ee0 0x7f6210155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 --2- 172.21.15.53:0/9341095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100737a0 0x7f6210156e90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 >> 172.21.15.53:0/9341095 conn(0x7f62100b4530 msgr2=0x7f621006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:14.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 shutdown_connections 2024-05-10T10:34:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:14.891+0000 7f6214f14640 1 -- 172.21.15.53:0/9341095 wait complete. 2024-05-10T10:34:14.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-10T10:34:15.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.012+0000 7f26ccdaf640 1 Processor -- start 2024-05-10T10:34:15.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.012+0000 7f26ccdaf640 1 -- start start 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26ccdaf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26c815d010 0x7f26c80cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26ccdaf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26ccdaf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c80add50 con 0x7f26c80d1370 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26ccdaf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f26c80d17e0 con 0x7f26c815d010 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26ccdaf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80ae030 con 0x7f26c80d04e0 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26c815d010 0x7f26c80cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c6d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f26c80d1370 0x7f26c806dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33952/0 (socket says 172.21.15.53:33952) 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c6d9d640 1 -- 172.21.15.53:0/2602956526 learned_addr learned my addr 172.21.15.53:0/2602956526 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26c815d010 0x7f26c80cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59580/0 (socket says 172.21.15.53:59580) 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c5d9b640 1 --2- 172.21.15.53:0/2602956526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c659c640 1 -- 172.21.15.53:0/2602956526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 msgr2=0x7f26c80d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:15.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c659c640 1 --2- 172.21.15.53:0/2602956526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c80d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c659c640 1 -- 172.21.15.53:0/2602956526 >> v1:172.21.15.53:6789/0 conn(0x7f26c80d1370 legacy=0x7f26c806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:15.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.013+0000 7f26c659c640 1 -- 172.21.15.53:0/2602956526 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80b5af0 con 0x7f26c815d010 2024-05-10T10:34:15.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.014+0000 7f26c659c640 1 --2- 172.21.15.53:0/2602956526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c815d010 0x7f26c80cfe90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26b00097a0 tx=0x7f26b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e3f3d847ae0220 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:15.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.014+0000 7f26c559a640 1 -- 172.21.15.53:0/2602956526 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0037070 con 0x7f26c815d010 2024-05-10T10:34:15.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.014+0000 7f26c559a640 1 -- 172.21.15.53:0/2602956526 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b0032440 con 0x7f26c815d010 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.014+0000 7f26c559a640 1 -- 172.21.15.53:0/2602956526 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0032810 con 0x7f26c815d010 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2602956526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c815d010 msgr2=0x7f26c80cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2602956526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c815d010 0x7f26c80cfe90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26b00097a0 tx=0x7f26b002a180 comp rx=0 tx=0).stop 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2602956526 shutdown_connections 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2602956526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c80d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2602956526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c815d010 0x7f26c80cfe90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2602956526 >> 172.21.15.53:0/2602956526 conn(0x7f26c806c4a0 msgr2=0x7f26c806c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2602956526 shutdown_connections 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.015+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2602956526 wait complete. 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.016+0000 7f26ccdaf640 1 Processor -- start 2024-05-10T10:34:15.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.016+0000 7f26ccdaf640 1 -- start start 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26ccdaf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c809d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26ccdaf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c80d1370 0x7f26c814aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26ccdaf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c815d010 0x7f26c814b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26ccdaf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b0580 con 0x7f26c80d1370 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26ccdaf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b0400 con 0x7f26c815d010 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26ccdaf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80b4c10 con 0x7f26c80d04e0 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c80d1370 0x7f26c814aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c80d1370 0x7f26c814aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59592/0 (socket says 172.21.15.53:59592) 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 -- 172.21.15.53:0/2622245947 learned_addr learned my addr 172.21.15.53:0/2622245947 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c659c640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c809d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:15.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c6d9d640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c815d010 0x7f26c814b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 -- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 msgr2=0x7f26c809d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c809d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 -- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c815d010 msgr2=0x7f26c814b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c815d010 0x7f26c814b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 -- 172.21.15.53:0/2622245947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c809ddd0 con 0x7f26c80d1370 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c659c640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c809d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c6d9d640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c815d010 0x7f26c814b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26c5d9b640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c80d1370 0x7f26c814aee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9b0 tx=0x7f26b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.017+0000 7f26beffd640 1 -- 172.21.15.53:0/2622245947 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8017070 con 0x7f26c80d1370 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.018+0000 7f26beffd640 1 -- 172.21.15.53:0/2622245947 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b800c9a0 con 0x7f26c80d1370 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.018+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c809c7f0 con 0x7f26c80d1370 2024-05-10T10:34:15.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.018+0000 7f26beffd640 1 -- 172.21.15.53:0/2622245947 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b800cd90 con 0x7f26c80d1370 2024-05-10T10:34:15.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.018+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c809cdb0 con 0x7f26c80d1370 2024-05-10T10:34:15.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.019+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c80b5910 con 0x7f26c80d1370 2024-05-10T10:34:15.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.019+0000 7f26beffd640 1 -- 172.21.15.53:0/2622245947 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f26b801a070 con 0x7f26c80d1370 2024-05-10T10:34:15.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.020+0000 7f26beffd640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26a80762d0 0x7f26a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:15.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.020+0000 7f26beffd640 1 -- 172.21.15.53:0/2622245947 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5766+0+0 (secure 0 0 0) 0x7f26b8097720 con 0x7f26c80d1370 2024-05-10T10:34:15.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.020+0000 7f26c659c640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26a80762d0 0x7f26a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:15.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.021+0000 7f26c659c640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26a80762d0 0x7f26a8078790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26b0034000 tx=0x7f26b002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:15.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.024+0000 7f26beffd640 1 -- 172.21.15.53:0/2622245947 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f26c80b5910 con 0x7f26c80d1370 2024-05-10T10:34:15.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.213+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f26c81547b0 con 0x7f26c80d1370 2024-05-10T10:34:15.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.890+0000 7f26beffd640 1 -- 172.21.15.53:0/2622245947 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v28) ==== 178+0+0 (secure 0 0 0) 0x7f26b8060fa0 con 0x7f26c80d1370 2024-05-10T10:34:15.891 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data_ec' removed 2024-05-10T10:34:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.892+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26a80762d0 msgr2=0x7f26a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.892+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26a80762d0 0x7f26a8078790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26b0034000 tx=0x7f26b002e000 comp rx=0 tx=0).stop 2024-05-10T10:34:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c80d1370 msgr2=0x7f26c814aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c80d1370 0x7f26c814aee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9b0 tx=0x7f26b800ee70 comp rx=0 tx=0).stop 2024-05-10T10:34:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 shutdown_connections 2024-05-10T10:34:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f26a80762d0 0x7f26a8078790 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26b0034000 tx=0x7f26b002e000 comp rx=0 tx=0).stop 2024-05-10T10:34:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c815d010 0x7f26c814b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c80d1370 0x7f26c814aee0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 --2- 172.21.15.53:0/2622245947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c80d04e0 0x7f26c809d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.893+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 >> 172.21.15.53:0/2622245947 conn(0x7f26c806c4a0 msgr2=0x7f26c80b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:15.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.894+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 shutdown_connections 2024-05-10T10:34:15.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:15.894+0000 7f26ccdaf640 1 -- 172.21.15.53:0/2622245947 wait complete. 2024-05-10T10:34:15.913 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:34:15.914 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T10:34:15.914 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:34:15.914 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:15.914 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:34:15.915 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T10:34:15.915 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:34:15.915 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:15.915 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:34:15.916 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T10:34:15.916 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:34:15.916 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:15.917 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:34:15.917 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T10:34:15.917 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:34:15.917 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:15.925 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:34:15.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:34:15.927 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:34:15.927 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:34:15.928 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:34:15.928 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:34:15.929 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:34:15.929 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:34:15.969 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:34:15.969 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:15.970 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:34:15.971 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:15.972 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:34:15.972 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:15.972 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:34:15.973 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:15.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T10:34:16.065 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:34:16.065 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:34:16.066 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:34:16.073 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:34:16.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.109+0000 7fb3f7189640 1 Processor -- start 2024-05-10T10:34:16.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.109+0000 7fb3f7189640 1 -- start start 2024-05-10T10:34:16.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.109+0000 7fb3f7189640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb3f00b5520 0x7fb3f00d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3f7189640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3f00b5900 0x7fb3f00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3f7189640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f00d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3f7189640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f00d5c60 con 0x7fb3f00b5900 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3f7189640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f00d41f0 con 0x7fb3f00b5520 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3f7189640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f00d5eb0 con 0x7fb3f00b62e0 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3f00b5900 0x7fb3f00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3f00b5900 0x7fb3f00d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59644/0 (socket says 172.21.15.53:59644) 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 -- 172.21.15.53:0/2996921426 learned_addr learned my addr 172.21.15.53:0/2996921426 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3f5726640 1 --2- 172.21.15.53:0/2996921426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f00d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3f4f25640 1 --2- 172.21.15.53:0/2996921426 >> v2:172.21.15.135:3300/0 conn(0x7fb3f00b5520 0x7fb3f00d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 -- 172.21.15.53:0/2996921426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 msgr2=0x7fb3f00d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 --2- 172.21.15.53:0/2996921426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f00d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 -- 172.21.15.53:0/2996921426 >> v2:172.21.15.135:3300/0 conn(0x7fb3f00b5520 msgr2=0x7fb3f00d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 --2- 172.21.15.53:0/2996921426 >> v2:172.21.15.135:3300/0 conn(0x7fb3f00b5520 0x7fb3f00d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.110+0000 7fb3effff640 1 -- 172.21.15.53:0/2996921426 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f00d0a30 con 0x7fb3f00b5900 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.111+0000 7fb3effff640 1 --2- 172.21.15.53:0/2996921426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f00d1de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0074950 tx=0x7fb3dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84c4a95903e504 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.111+0000 7fb3ef7fe640 1 -- 172.21.15.53:0/2996921426 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc037070 con 0x7fb3f00b5900 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.111+0000 7fb3ef7fe640 1 -- 172.21.15.53:0/2996921426 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc032440 con 0x7fb3f00b5900 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.111+0000 7fb3ef7fe640 1 -- 172.21.15.53:0/2996921426 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc032810 con 0x7fb3f00b5900 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.111+0000 7fb3f7189640 1 -- 172.21.15.53:0/2996921426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 msgr2=0x7fb3f00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.111+0000 7fb3f7189640 1 --2- 172.21.15.53:0/2996921426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f00d1de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0074950 tx=0x7fb3dc02a1e0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.112+0000 7fb3f7189640 1 -- 172.21.15.53:0/2996921426 shutdown_connections 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.112+0000 7fb3f7189640 1 --2- 172.21.15.53:0/2996921426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f00d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.112+0000 7fb3f7189640 1 --2- 172.21.15.53:0/2996921426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f00d1de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.112+0000 7fb3f7189640 1 --2- 172.21.15.53:0/2996921426 >> v2:172.21.15.135:3300/0 conn(0x7fb3f00b5520 0x7fb3f00d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.112+0000 7fb3f7189640 1 -- 172.21.15.53:0/2996921426 >> 172.21.15.53:0/2996921426 conn(0x7fb3f00d5170 msgr2=0x7fb3f00d5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.112+0000 7fb3f7189640 1 -- 172.21.15.53:0/2996921426 shutdown_connections 2024-05-10T10:34:16.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.112+0000 7fb3f7189640 1 -- 172.21.15.53:0/2996921426 wait complete. 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.113+0000 7fb3f7189640 1 Processor -- start 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.113+0000 7fb3f7189640 1 -- start start 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f7189640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3f00b5520 0x7fb3f00b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f7189640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f0158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:16.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f7189640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f0156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f7189640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f00b5f10 con 0x7fb3f00b5900 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f7189640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f00b5d90 con 0x7fb3f00b5520 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f7189640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f00b6090 con 0x7fb3f00b62e0 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f0158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f0158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59650/0 (socket says 172.21.15.53:59650) 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 -- 172.21.15.53:0/851887812 learned_addr learned my addr 172.21.15.53:0/851887812 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f4f25640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3f00b5520 0x7fb3f00b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f5726640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f0156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 -- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 msgr2=0x7fb3f0156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f0156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 -- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3f00b5520 msgr2=0x7fb3f00b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3f00b5520 0x7fb3f00b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 -- 172.21.15.53:0/851887812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f01f65c0 con 0x7fb3f00b5900 2024-05-10T10:34:16.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f4f25640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3f00b5520 0x7fb3f00b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3effff640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f0158fd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc02a220 tx=0x7fb3dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3ed7fa640 1 -- 172.21.15.53:0/851887812 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc041020 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3ed7fa640 1 -- 172.21.15.53:0/851887812 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc03d620 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.114+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f01f68a0 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.115+0000 7fb3ed7fa640 1 -- 172.21.15.53:0/851887812 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc037040 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.115+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f01f6e60 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.116+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f00693f0 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.116+0000 7fb3ed7fa640 1 -- 172.21.15.53:0/851887812 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb3dc044020 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.117+0000 7fb3ed7fa640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3d40762d0 0x7fb3d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.117+0000 7fb3ed7fa640 1 -- 172.21.15.53:0/851887812 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5335+0+0 (secure 0 0 0) 0x7fb3dc0b7bf0 con 0x7fb3f00b5900 2024-05-10T10:34:16.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.117+0000 7fb3f4f25640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3d40762d0 0x7fb3d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:16.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.118+0000 7fb3f4f25640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3d40762d0 0x7fb3d4078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009b10 tx=0x7fb3e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:16.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.121+0000 7fb3ed7fa640 1 -- 172.21.15.53:0/851887812 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb3dc04b030 con 0x7fb3f00b5900 2024-05-10T10:34:16.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.324+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb3f00d0000 con 0x7fb3f00b5900 2024-05-10T10:34:16.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.909+0000 7fb3ed7fa640 1 -- 172.21.15.53:0/851887812 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) ==== 97+0+0 (secure 0 0 0) 0x7fb3dc081620 con 0x7fb3f00b5900 2024-05-10T10:34:16.909 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2024-05-10T10:34:16.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.911+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3d40762d0 msgr2=0x7fb3d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:16.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.911+0000 7fb3f7189640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3d40762d0 0x7fb3d4078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009b10 tx=0x7fb3e4008040 comp rx=0 tx=0).stop 2024-05-10T10:34:16.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.911+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 msgr2=0x7fb3f0158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.911+0000 7fb3f7189640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f0158fd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc02a220 tx=0x7fb3dc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 shutdown_connections 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3d40762d0 0x7fb3d4078790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3f00b62e0 0x7fb3f0156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3f00b5900 0x7fb3f0158fd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 --2- 172.21.15.53:0/851887812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3f00b5520 0x7fb3f00b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 >> 172.21.15.53:0/851887812 conn(0x7fb3f00d5170 msgr2=0x7fb3f00b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 shutdown_connections 2024-05-10T10:34:16.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:16.912+0000 7fb3f7189640 1 -- 172.21.15.53:0/851887812 wait complete. 2024-05-10T10:34:16.927 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T10:34:17.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.036+0000 7fdbe8b38640 1 Processor -- start 2024-05-10T10:34:17.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.036+0000 7fdbe8b38640 1 -- start start 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.037+0000 7fdbe8b38640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdbe4162750 0x7fdbe40d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.037+0000 7fdbe8b38640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40d5cb0 0x7fdbe40d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.037+0000 7fdbe8b38640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe40b45f0 con 0x7fdbe4162b30 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.037+0000 7fdbe8b38640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdbe40b4f20 con 0x7fdbe4162750 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.037+0000 7fdbe8b38640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe40d0fe0 con 0x7fdbe40d5cb0 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.037+0000 7fdbe2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdbe4162b30 0x7fdbe40d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33984/0 (socket says 172.21.15.53:33984) 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe2d9d640 1 -- 172.21.15.53:0/2957347793 learned_addr learned my addr 172.21.15.53:0/2957347793 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:17.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe259c640 1 --2- 172.21.15.53:0/2957347793 >> v2:172.21.15.135:3300/0 conn(0x7fdbe4162750 0x7fdbe40d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe1d9b640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40d5cb0 0x7fdbe40d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557721686 0 0) 0x7fdbe40b45f0 con 0x7fdbe4162b30 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbd0003520 con 0x7fdbe4162b30 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 384789675 0 0) 0x7fdbd0003520 con 0x7fdbe4162b30 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40d5cb0 msgr2=0x7fdbe40d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40d5cb0 0x7fdbe40d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 >> v2:172.21.15.135:3300/0 conn(0x7fdbe4162750 msgr2=0x7fdbe40d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 --2- 172.21.15.53:0/2957347793 >> v2:172.21.15.135:3300/0 conn(0x7fdbe4162750 0x7fdbe40d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe40b48d0 con 0x7fdbe4162b30 2024-05-10T10:34:17.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fdbdc001670 con 0x7fdbe4162b30 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 >> v1:172.21.15.53:6789/0 conn(0x7fdbe4162b30 legacy=0x7fdbe40d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbd0004800 0x7fdbd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbd0005180 0x7fdbd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe40b4f20 con 0x7fdbd00039b0 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe40d0fe0 con 0x7fdbd0005180 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd0004630 con 0x7fdbd0004800 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe259c640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe2d9d640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbd0005180 0x7fdbd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.038+0000 7fdbe1d9b640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbd0004800 0x7fdbd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.039+0000 7fdbe259c640 1 -- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbd0004800 msgr2=0x7fdbd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.039+0000 7fdbe259c640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbd0004800 0x7fdbd0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.039+0000 7fdbe259c640 1 -- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbd0005180 msgr2=0x7fdbd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.039+0000 7fdbe259c640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbd0005180 0x7fdbd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.039+0000 7fdbe259c640 1 -- 172.21.15.53:0/2957347793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd0003520 con 0x7fdbd00039b0 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.039+0000 7fdbe259c640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbd0003dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009fc0 tx=0x7fdbd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15976b4cb9adc30f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4009c10 con 0x7fdbd00039b0 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd4036070 con 0x7fdbd00039b0 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe159a640 1 -- 172.21.15.53:0/2957347793 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd402d070 con 0x7fdbd00039b0 2024-05-10T10:34:17.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe8b38640 1 -- 172.21.15.53:0/2957347793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 msgr2=0x7fdbd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe8b38640 1 --2- 172.21.15.53:0/2957347793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbd0003dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009fc0 tx=0x7fdbd0005830 comp rx=0 tx=0).stop 2024-05-10T10:34:17.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe2d9d640 1 -- 172.21.15.53:0/2957347793 reap_dead start 2024-05-10T10:34:17.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe8b38640 1 -- 172.21.15.53:0/2957347793 shutdown_connections 2024-05-10T10:34:17.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe8b38640 1 -- 172.21.15.53:0/2957347793 >> 172.21.15.53:0/2957347793 conn(0x7fdbe409e3b0 msgr2=0x7fdbe40d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:17.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe8b38640 1 -- 172.21.15.53:0/2957347793 shutdown_connections 2024-05-10T10:34:17.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.040+0000 7fdbe8b38640 1 -- 172.21.15.53:0/2957347793 wait complete. 2024-05-10T10:34:17.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.042+0000 7fdbe8b38640 1 Processor -- start 2024-05-10T10:34:17.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.042+0000 7fdbe8b38640 1 -- start start 2024-05-10T10:34:17.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.042+0000 7fdbe8b38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbe4158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.042+0000 7fdbe8b38640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe4160170 0x7fdbe41605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.042+0000 7fdbe8b38640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40cc3d0 0x7fdbe40cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe8b38640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe40d0d30 con 0x7fdbd00039b0 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe8b38640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe40d0bb0 con 0x7fdbe4160170 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe8b38640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe40b63c0 con 0x7fdbe40cc3d0 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbe4158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbe4158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59668/0 (socket says 172.21.15.53:59668) 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 -- 172.21.15.53:0/3040650672 learned_addr learned my addr 172.21.15.53:0/3040650672 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe1d9b640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe4160170 0x7fdbe41605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe2d9d640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40cc3d0 0x7fdbe40cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 -- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40cc3d0 msgr2=0x7fdbe40cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40cc3d0 0x7fdbe40cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 -- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe4160170 msgr2=0x7fdbe41605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe4160170 0x7fdbe41605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 -- 172.21.15.53:0/3040650672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe415a380 con 0x7fdbd00039b0 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe259c640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbe4158a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400e230 tx=0x7fdbd4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbdaffd640 1 -- 172.21.15.53:0/3040650672 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4009c10 con 0x7fdbd00039b0 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbdaffd640 1 -- 172.21.15.53:0/3040650672 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd4003bb0 con 0x7fdbd00039b0 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe415a660 con 0x7fdbd00039b0 2024-05-10T10:34:17.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbdaffd640 1 -- 172.21.15.53:0/3040650672 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4004020 con 0x7fdbd00039b0 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.043+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe415ac20 con 0x7fdbd00039b0 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.044+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe406a220 con 0x7fdbd00039b0 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.045+0000 7fdbdaffd640 1 -- 172.21.15.53:0/3040650672 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdbd4005d10 con 0x7fdbd00039b0 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.046+0000 7fdbdaffd640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbc4076300 0x7fdbc40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.046+0000 7fdbdaffd640 1 -- 172.21.15.53:0/3040650672 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fdbd40b86e0 con 0x7fdbd00039b0 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.046+0000 7fdbe1d9b640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbc4076300 0x7fdbc40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.046+0000 7fdbe1d9b640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbc4076300 0x7fdbc40787c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40b4170 tx=0x7fdbcc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.050+0000 7fdbdaffd640 1 -- 172.21.15.53:0/3040650672 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdbd4082290 con 0x7fdbd00039b0 2024-05-10T10:34:17.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.313+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdbe40d13b0 con 0x7fdbd00039b0 2024-05-10T10:34:17.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.313+0000 7fdbdaffd640 1 -- 172.21.15.53:0/3040650672 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7fdbd408a280 con 0x7fdbd00039b0 2024-05-10T10:34:17.314 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:17.314 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:34:17.314 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2024-05-10T10:34:17.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg==", 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ==", 2024-05-10T10:34:17.315 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA==", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:17.316 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew==", 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.317 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA==", 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA==", 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.318 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw==", 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2024-05-10T10:34:17.319 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA==", 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g==", 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A==", 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA==", 2024-05-10T10:34:17.321 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg==", 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg==", 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g==", 2024-05-10T10:34:17.323 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw==", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:34:17.324 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw==", 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.325 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1mKY2tIhAAzPSpOQLOFy1GEKdl5bPBEw==", 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-mds", 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDt9z1mS95VLRAA83tQMsrRDFpvzdLDzLgL+A==", 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.326 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-mds" 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-mgr", 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDt9z1mKelVLRAAN037dNHguBLv4WvsDvkq5w==", 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-osd", 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDt9z1mi/FVLRAALQwHajMoImYiJrN/UQJgng==", 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.327 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-osd" 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rbd", 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDt9z1mKflVLRAAmJr6SbMYg6ADYUBunzHuww==", 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDt9z1mDwFWLRAAdwfwAuNuKU3pjtbiEXu5cg==", 2024-05-10T10:34:17.328 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rgw", 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDt9z1myQhWLRAALPScxCEIi2VCYj3SsSZYbA==", 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2024-05-10T10:34:17.329 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw==", 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ==", 2024-05-10T10:34:17.330 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:17.331 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:34:17.331 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:34:17.331 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:17.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:17.331 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:34:17.331 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:34:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.315+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbc4076300 msgr2=0x7fdbc40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.315+0000 7fdbe8b38640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbc4076300 0x7fdbc40787c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40b4170 tx=0x7fdbcc009190 comp rx=0 tx=0).stop 2024-05-10T10:34:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.315+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 msgr2=0x7fdbe4158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.315+0000 7fdbe8b38640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbe4158a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400e230 tx=0x7fdbd4009750 comp rx=0 tx=0).stop 2024-05-10T10:34:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 shutdown_connections 2024-05-10T10:34:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbc4076300 0x7fdbc40787c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe40cc3d0 0x7fdbe40cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe4160170 0x7fdbe41605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 --2- 172.21.15.53:0/3040650672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd00039b0 0x7fdbe4158a70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 >> 172.21.15.53:0/3040650672 conn(0x7fdbe409e3b0 msgr2=0x7fdbe40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 shutdown_connections 2024-05-10T10:34:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.316+0000 7fdbe8b38640 1 -- 172.21.15.53:0/3040650672 wait complete. 2024-05-10T10:34:17.334 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-10T10:34:17.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.438+0000 7fab97546640 1 Processor -- start 2024-05-10T10:34:17.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.438+0000 7fab97546640 1 -- start start 2024-05-10T10:34:17.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.439+0000 7fab97546640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab90162c90 0x7fab900d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.439+0000 7fab97546640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab900d3070 con 0x7fab900d2110 2024-05-10T10:34:17.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.439+0000 7fab97546640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab900d5c00 con 0x7fab900d0bf0 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.439+0000 7fab97546640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fab900d2e20 con 0x7fab90162c90 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.439+0000 7fab952e2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab90162c90 0x7fab900d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.439+0000 7fab94ae1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fab900d2110 0x7fab900c9430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33994/0 (socket says 172.21.15.53:33994) 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.439+0000 7fab94ae1640 1 -- 172.21.15.53:0/1155127277 learned_addr learned my addr 172.21.15.53:0/1155127277 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.440+0000 7fab952e2640 1 -- 172.21.15.53:0/1155127277 >> v1:172.21.15.135:6789/0 conn(0x7fab900d0bf0 legacy=0x7fab900c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.440+0000 7fab952e2640 1 -- 172.21.15.53:0/1155127277 >> v1:172.21.15.53:6789/0 conn(0x7fab900d2110 legacy=0x7fab900c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.440+0000 7fab8ffff640 1 -- 172.21.15.53:0/1155127277 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240513298 0 0) 0x7fab900d3070 con 0x7fab900d2110 2024-05-10T10:34:17.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.440+0000 7fab952e2640 1 -- 172.21.15.53:0/1155127277 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab900c8c30 con 0x7fab90162c90 2024-05-10T10:34:17.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.440+0000 7fab952e2640 1 --2- 172.21.15.53:0/1155127277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab90162c90 0x7fab900d3620 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fab80009d90 tx=0x7fab8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc465c11ad7745a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.440+0000 7fab8ffff640 1 -- 172.21.15.53:0/1155127277 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80007740 con 0x7fab90162c90 2024-05-10T10:34:17.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab8ffff640 1 -- 172.21.15.53:0/1155127277 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab800078e0 con 0x7fab90162c90 2024-05-10T10:34:17.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab8ffff640 1 -- 172.21.15.53:0/1155127277 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80007cb0 con 0x7fab90162c90 2024-05-10T10:34:17.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab97546640 1 -- 172.21.15.53:0/1155127277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab90162c90 msgr2=0x7fab900d3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab97546640 1 --2- 172.21.15.53:0/1155127277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab90162c90 0x7fab900d3620 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fab80009d90 tx=0x7fab8002a6a0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab97546640 1 -- 172.21.15.53:0/1155127277 shutdown_connections 2024-05-10T10:34:17.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab97546640 1 --2- 172.21.15.53:0/1155127277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab90162c90 0x7fab900d3620 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab97546640 1 -- 172.21.15.53:0/1155127277 >> 172.21.15.53:0/1155127277 conn(0x7fab900d5040 msgr2=0x7fab900d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:17.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab97546640 1 -- 172.21.15.53:0/1155127277 shutdown_connections 2024-05-10T10:34:17.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.441+0000 7fab97546640 1 -- 172.21.15.53:0/1155127277 wait complete. 2024-05-10T10:34:17.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.442+0000 7fab97546640 1 Processor -- start 2024-05-10T10:34:17.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.442+0000 7fab97546640 1 -- start start 2024-05-10T10:34:17.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab97546640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab900d0bf0 0x7fab9009f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab97546640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab900d2110 0x7fab900ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab97546640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab90162c90 0x7fab9015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab97546640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab900d5ee0 con 0x7fab900d0bf0 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab97546640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab900b6740 con 0x7fab900d2110 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab97546640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fab900d3b60 con 0x7fab90162c90 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab952e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab900d0bf0 0x7fab9009f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab952e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab900d0bf0 0x7fab9009f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59680/0 (socket says 172.21.15.53:59680) 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab952e2640 1 -- 172.21.15.53:0/3796730133 learned_addr learned my addr 172.21.15.53:0/3796730133 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab94ae1640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab900d2110 0x7fab900ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.443+0000 7fab95ae3640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab90162c90 0x7fab9015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab952e2640 1 -- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab90162c90 msgr2=0x7fab9015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab952e2640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab90162c90 0x7fab9015c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab952e2640 1 -- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab900d2110 msgr2=0x7fab900ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab952e2640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab900d2110 0x7fab900ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab952e2640 1 -- 172.21.15.53:0/3796730133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab900ba780 con 0x7fab900d0bf0 2024-05-10T10:34:17.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab952e2640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab900d0bf0 0x7fab9009f900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab80009d90 tx=0x7fab80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab8dffb640 1 -- 172.21.15.53:0/3796730133 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80040070 con 0x7fab900d0bf0 2024-05-10T10:34:17.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab8dffb640 1 -- 172.21.15.53:0/3796730133 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab80031c40 con 0x7fab900d0bf0 2024-05-10T10:34:17.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab900baa60 con 0x7fab900d0bf0 2024-05-10T10:34:17.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab8dffb640 1 -- 172.21.15.53:0/3796730133 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80003990 con 0x7fab900d0bf0 2024-05-10T10:34:17.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.444+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab900b7f40 con 0x7fab900d0bf0 2024-05-10T10:34:17.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.445+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9006a220 con 0x7fab900d0bf0 2024-05-10T10:34:17.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.446+0000 7fab8dffb640 1 -- 172.21.15.53:0/3796730133 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fab80039040 con 0x7fab900d0bf0 2024-05-10T10:34:17.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.446+0000 7fab8dffb640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fab7c0762d0 0x7fab7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.446+0000 7fab8dffb640 1 -- 172.21.15.53:0/3796730133 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fab800b83f0 con 0x7fab900d0bf0 2024-05-10T10:34:17.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.447+0000 7fab94ae1640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fab7c0762d0 0x7fab7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.447+0000 7fab94ae1640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fab7c0762d0 0x7fab7c078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab900c9a60 tx=0x7fab84007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.451+0000 7fab8dffb640 1 -- 172.21.15.53:0/3796730133 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fab80081e90 con 0x7fab900d0bf0 2024-05-10T10:34:17.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.714+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fab900cb9a0 con 0x7fab900d0bf0 2024-05-10T10:34:17.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.716+0000 7fab8dffb640 1 -- 172.21.15.53:0/3796730133 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fab80081790 con 0x7fab900d0bf0 2024-05-10T10:34:17.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fab7c0762d0 msgr2=0x7fab7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fab7c0762d0 0x7fab7c078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab900c9a60 tx=0x7fab84007b50 comp rx=0 tx=0).stop 2024-05-10T10:34:17.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab900d0bf0 msgr2=0x7fab9009f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab900d0bf0 0x7fab9009f900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab80009d90 tx=0x7fab80004a40 comp rx=0 tx=0).stop 2024-05-10T10:34:17.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 shutdown_connections 2024-05-10T10:34:17.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fab7c0762d0 0x7fab7c078790 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab90162c90 0x7fab9015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab900d2110 0x7fab900ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 --2- 172.21.15.53:0/3796730133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab900d0bf0 0x7fab9009f900 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 >> 172.21.15.53:0/3796730133 conn(0x7fab900d5040 msgr2=0x7fab9015f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:17.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 shutdown_connections 2024-05-10T10:34:17.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.718+0000 7fab97546640 1 -- 172.21.15.53:0/3796730133 wait complete. 2024-05-10T10:34:17.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-10T10:34:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.834+0000 7fe81be85640 1 Processor -- start 2024-05-10T10:34:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.834+0000 7fe81be85640 1 -- start start 2024-05-10T10:34:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe81be85640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 0x7fe8140cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe81be85640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81415fad0 con 0x7fe8140cea20 2024-05-10T10:34:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe81be85640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81406e1a0 con 0x7fe8140b3ba0 2024-05-10T10:34:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe81be85640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe8140d27b0 con 0x7fe81415f6f0 2024-05-10T10:34:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe81a422640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe8140cea20 0x7fe8140d1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34004/0 (socket says 172.21.15.53:34004) 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe81a422640 1 -- 172.21.15.53:0/2853550016 learned_addr learned my addr 172.21.15.53:0/2853550016 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921326115 0 0) 0x7fe81415fad0 con 0x7fe8140cea20 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7fc003520 con 0x7fe8140cea20 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.835+0000 7fe819c21640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 0x7fe8140cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1276077435 0 0) 0x7fe7fc003520 con 0x7fe8140cea20 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 msgr2=0x7fe8140cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 0x7fe8140cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 >> v1:172.21.15.135:6789/0 conn(0x7fe8140b3ba0 legacy=0x7fe8140d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81406e480 con 0x7fe8140cea20 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fe80c001670 con 0x7fe8140cea20 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 >> v1:172.21.15.53:6789/0 conn(0x7fe8140cea20 legacy=0x7fe8140d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 0x7fe7fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc003f60 0x7fe7fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc004d40 0x7fe7fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140d27b0 con 0x7fe7fc003f60 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc00baa0 con 0x7fe7fc004d40 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe7fc00bca0 con 0x7fe81415f6f0 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe819420640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc003f60 0x7fe7fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe819c21640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 0x7fe7fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe81a422640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc004d40 0x7fe7fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe819420640 1 -- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 msgr2=0x7fe7fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe819420640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81415f6f0 0x7fe7fc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe819420640 1 -- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc004d40 msgr2=0x7fe7fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe819420640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc004d40 0x7fe7fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.836+0000 7fe819420640 1 -- 172.21.15.53:0/2853550016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc003520 con 0x7fe7fc003f60 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.837+0000 7fe819420640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc003f60 0x7fe7fc004800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe808001750 tx=0x7fe7fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ac67efa90965d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.837+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808001370 con 0x7fe7fc003f60 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.837+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8080052d0 con 0x7fe7fc003f60 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.837+0000 7fe818c1f640 1 -- 172.21.15.53:0/2853550016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80802abe0 con 0x7fe7fc003f60 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.838+0000 7fe81be85640 1 -- 172.21.15.53:0/2853550016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc003f60 msgr2=0x7fe7fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.838+0000 7fe81be85640 1 --2- 172.21.15.53:0/2853550016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc003f60 0x7fe7fc004800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe808001750 tx=0x7fe7fc005830 comp rx=0 tx=0).stop 2024-05-10T10:34:17.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.838+0000 7fe81a422640 1 -- 172.21.15.53:0/2853550016 reap_dead start 2024-05-10T10:34:17.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.838+0000 7fe81be85640 1 -- 172.21.15.53:0/2853550016 shutdown_connections 2024-05-10T10:34:17.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.838+0000 7fe81be85640 1 -- 172.21.15.53:0/2853550016 >> 172.21.15.53:0/2853550016 conn(0x7fe8140d2d70 msgr2=0x7fe8140d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:17.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.838+0000 7fe81be85640 1 -- 172.21.15.53:0/2853550016 shutdown_connections 2024-05-10T10:34:17.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.838+0000 7fe81be85640 1 -- 172.21.15.53:0/2853550016 wait complete. 2024-05-10T10:34:17.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.839+0000 7fe81be85640 1 Processor -- start 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.839+0000 7fe81be85640 1 -- start start 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe81be85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc003f60 0x7fe8140c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe81be85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe814159300 0x7fe8140cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe81be85640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81414d120 0x7fe814162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe81be85640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140cd4f0 con 0x7fe814159300 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe81be85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140d24d0 con 0x7fe7fc003f60 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe81be85640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe8140d1e60 con 0x7fe81414d120 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe814159300 0x7fe8140cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe814159300 0x7fe8140cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59698/0 (socket says 172.21.15.53:59698) 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 -- 172.21.15.53:0/4139626225 learned_addr learned my addr 172.21.15.53:0/4139626225 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe81a422640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81414d120 0x7fe814162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819c21640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc003f60 0x7fe8140c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 -- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81414d120 msgr2=0x7fe814162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81414d120 0x7fe814162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 -- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc003f60 msgr2=0x7fe8140c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc003f60 0x7fe8140c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 -- 172.21.15.53:0/4139626225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe814159780 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.840+0000 7fe819420640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe814159300 0x7fe8140cb5e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe808008fd0 tx=0x7fe808001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.841+0000 7fe8127fc640 1 -- 172.21.15.53:0/4139626225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808040020 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.841+0000 7fe8127fc640 1 -- 172.21.15.53:0/4139626225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe808007680 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.841+0000 7fe8127fc640 1 -- 172.21.15.53:0/4139626225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808007a30 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.841+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8140ca540 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.841+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8140cab00 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.842+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81406a220 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.842+0000 7fe8127fc640 1 -- 172.21.15.53:0/4139626225 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe808043020 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.843+0000 7fe8127fc640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe800076300 0x7fe8000787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.843+0000 7fe8127fc640 1 -- 172.21.15.53:0/4139626225 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fe8080b73a0 con 0x7fe814159300 2024-05-10T10:34:17.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.843+0000 7fe819c21640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe800076300 0x7fe8000787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:17.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.844+0000 7fe819c21640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe800076300 0x7fe8000787c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe804007520 tx=0x7fe80400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:17.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:17.847+0000 7fe8127fc640 1 -- 172.21.15.53:0/4139626225 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe80804a030 con 0x7fe814159300 2024-05-10T10:34:18.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.115+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fe81415d8e0 con 0x7fe814159300 2024-05-10T10:34:18.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.118+0000 7fe8127fc640 1 -- 172.21.15.53:0/4139626225 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fe808080ec0 con 0x7fe814159300 2024-05-10T10:34:18.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.119+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe800076300 msgr2=0x7fe8000787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.119+0000 7fe81be85640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe800076300 0x7fe8000787c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe804007520 tx=0x7fe80400b040 comp rx=0 tx=0).stop 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe814159300 msgr2=0x7fe8140cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe814159300 0x7fe8140cb5e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe808008fd0 tx=0x7fe808001120 comp rx=0 tx=0).stop 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 shutdown_connections 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe800076300 0x7fe8000787c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe81414d120 0x7fe814162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe814159300 0x7fe8140cb5e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 --2- 172.21.15.53:0/4139626225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7fc003f60 0x7fe8140c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 >> 172.21.15.53:0/4139626225 conn(0x7fe8140d2d70 msgr2=0x7fe8140714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 shutdown_connections 2024-05-10T10:34:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.120+0000 7fe81be85640 1 -- 172.21.15.53:0/4139626225 wait complete. 2024-05-10T10:34:18.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-10T10:34:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.236+0000 7f8e7f023640 1 Processor -- start 2024-05-10T10:34:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.236+0000 7f8e7f023640 1 -- start start 2024-05-10T10:34:18.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.236+0000 7f8e7f023640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780b6490 0x7f8e780d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.236+0000 7f8e7f023640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e780cfd00 0x7f8e780d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.236+0000 7f8e7f023640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e780d0570 con 0x7f8e780d1f90 2024-05-10T10:34:18.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.236+0000 7f8e7f023640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8e780d1180 con 0x7f8e780cfd00 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.236+0000 7f8e7f023640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8e780d0850 con 0x7f8e780b6490 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e77fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e780cfd00 0x7f8e780d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e77fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e780cfd00 0x7f8e780d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59712/0 (socket says 172.21.15.53:59712) 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e77fff640 1 -- 172.21.15.53:0/3151333354 learned_addr learned my addr 172.21.15.53:0/3151333354 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e7cdbf640 1 --2- 172.21.15.53:0/3151333354 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780b6490 0x7f8e780d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e77fff640 1 -- 172.21.15.53:0/3151333354 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780b6490 msgr2=0x7f8e780d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e77fff640 1 --2- 172.21.15.53:0/3151333354 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780b6490 0x7f8e780d5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e77fff640 1 -- 172.21.15.53:0/3151333354 >> v1:172.21.15.135:6789/0 conn(0x7f8e780d1f90 legacy=0x7f8e7815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.237+0000 7f8e77fff640 1 -- 172.21.15.53:0/3151333354 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e780d0b30 con 0x7f8e780cfd00 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e77fff640 1 --2- 172.21.15.53:0/3151333354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780cfd00 0x7f8e780d1a50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009870 tx=0x7f8e6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607bbbbcb4d76533 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e777fe640 1 -- 172.21.15.53:0/3151333354 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64037070 con 0x7f8e780cfd00 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e777fe640 1 -- 172.21.15.53:0/3151333354 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e64032440 con 0x7f8e780cfd00 2024-05-10T10:34:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e777fe640 1 -- 172.21.15.53:0/3151333354 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64032810 con 0x7f8e780cfd00 2024-05-10T10:34:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 -- 172.21.15.53:0/3151333354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780cfd00 msgr2=0x7f8e780d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 --2- 172.21.15.53:0/3151333354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780cfd00 0x7f8e780d1a50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009870 tx=0x7f8e6402a270 comp rx=0 tx=0).stop 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 -- 172.21.15.53:0/3151333354 shutdown_connections 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 --2- 172.21.15.53:0/3151333354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780cfd00 0x7f8e780d1a50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 --2- 172.21.15.53:0/3151333354 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780b6490 0x7f8e780d5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 -- 172.21.15.53:0/3151333354 >> 172.21.15.53:0/3151333354 conn(0x7f8e780d5040 msgr2=0x7f8e780d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 -- 172.21.15.53:0/3151333354 shutdown_connections 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.238+0000 7f8e7f023640 1 -- 172.21.15.53:0/3151333354 wait complete. 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.240+0000 7f8e7f023640 1 Processor -- start 2024-05-10T10:34:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.240+0000 7f8e7f023640 1 -- start start 2024-05-10T10:34:18.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.240+0000 7f8e7f023640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780b6490 0x7f8e7815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.240+0000 7f8e7f023640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780cfd00 0x7f8e7815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7f023640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e780d1f90 0x7f8e7815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7f023640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e780d0160 con 0x7f8e780b6490 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7f023640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8e780b6870 con 0x7f8e780d1f90 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7f023640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8e780d02e0 con 0x7f8e780cfd00 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780b6490 0x7f8e7815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780b6490 0x7f8e7815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59726/0 (socket says 172.21.15.53:59726) 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 -- 172.21.15.53:0/2059533175 learned_addr learned my addr 172.21.15.53:0/2059533175 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e77fff640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780cfd00 0x7f8e7815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7d5c0640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e780d1f90 0x7f8e7815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 -- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780cfd00 msgr2=0x7f8e7815f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780cfd00 0x7f8e7815f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 -- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e780d1f90 msgr2=0x7f8e7815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e780d1f90 0x7f8e7815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 -- 172.21.15.53:0/2059533175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7815fe20 con 0x7f8e780b6490 2024-05-10T10:34:18.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7cdbf640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780b6490 0x7f8e7815c4c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00d800 tx=0x7f8e6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e757fa640 1 -- 172.21.15.53:0/2059533175 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00b6c0 con 0x7f8e780b6490 2024-05-10T10:34:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e757fa640 1 -- 172.21.15.53:0/2059533175 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6c019070 con 0x7f8e780b6490 2024-05-10T10:34:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e780cc860 con 0x7f8e780b6490 2024-05-10T10:34:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e757fa640 1 -- 172.21.15.53:0/2059533175 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00bac0 con 0x7f8e780b6490 2024-05-10T10:34:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.241+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e780cce20 con 0x7f8e780b6490 2024-05-10T10:34:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.242+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e78162ca0 con 0x7f8e780b6490 2024-05-10T10:34:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.243+0000 7f8e757fa640 1 -- 172.21.15.53:0/2059533175 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8e6c00bc60 con 0x7f8e780b6490 2024-05-10T10:34:18.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.244+0000 7f8e757fa640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e5c0762d0 0x7f8e5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.244+0000 7f8e757fa640 1 -- 172.21.15.53:0/2059533175 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7f8e6c0978d0 con 0x7f8e780b6490 2024-05-10T10:34:18.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.244+0000 7f8e77fff640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e5c0762d0 0x7f8e5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.244+0000 7f8e77fff640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e5c0762d0 0x7f8e5c078790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8e64034000 tx=0x7f8e6402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:18.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.248+0000 7f8e757fa640 1 -- 172.21.15.53:0/2059533175 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8e78162ca0 con 0x7f8e780b6490 2024-05-10T10:34:18.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.516+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f8e780693f0 con 0x7f8e780b6490 2024-05-10T10:34:18.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.518+0000 7f8e757fa640 1 -- 172.21.15.53:0/2059533175 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f8e6c061480 con 0x7f8e780b6490 2024-05-10T10:34:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e5c0762d0 msgr2=0x7f8e5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e5c0762d0 0x7f8e5c078790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8e64034000 tx=0x7f8e6402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780b6490 msgr2=0x7f8e7815c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780b6490 0x7f8e7815c4c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00d800 tx=0x7f8e6c00dcc0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 shutdown_connections 2024-05-10T10:34:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e5c0762d0 0x7f8e5c078790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e780d1f90 0x7f8e7815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e780cfd00 0x7f8e7815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 --2- 172.21.15.53:0/2059533175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e780b6490 0x7f8e7815c4c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 >> 172.21.15.53:0/2059533175 conn(0x7f8e780d5040 msgr2=0x7f8e780b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 shutdown_connections 2024-05-10T10:34:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.521+0000 7f8e7f023640 1 -- 172.21.15.53:0/2059533175 wait complete. 2024-05-10T10:34:18.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-10T10:34:18.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.636+0000 7fa8495f2640 1 Processor -- start 2024-05-10T10:34:18.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.636+0000 7fa8495f2640 1 -- start start 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.637+0000 7fa8495f2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 0x7fa8440d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.637+0000 7fa8495f2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8440d0570 con 0x7fa8440b6490 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.637+0000 7fa8495f2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8440d0850 con 0x7fa8440d1f90 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.637+0000 7fa8495f2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa8440d1180 con 0x7fa8440d5b60 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.637+0000 7fa842ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa8440b6490 0x7fa8440cfd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34008/0 (socket says 172.21.15.53:34008) 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.637+0000 7fa842ffd640 1 -- 172.21.15.53:0/2065651796 learned_addr learned my addr 172.21.15.53:0/2065651796 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa8427fc640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 0x7fa8440d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503234076 0 0) 0x7fa8440d0570 con 0x7fa8440b6490 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa82c003520 con 0x7fa8440b6490 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1684844518 0 0) 0x7fa82c003520 con 0x7fa8440b6490 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 msgr2=0x7fa8440d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 0x7fa8440d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 >> v1:172.21.15.135:6789/0 conn(0x7fa8440d1f90 legacy=0x7fa84415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8440d0b30 con 0x7fa8440b6490 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fa8340019f0 con 0x7fa8440b6490 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 >> v1:172.21.15.53:6789/0 conn(0x7fa8440b6490 legacy=0x7fa8440cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c0039e0 0x7fa82c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82c004390 0x7fa82c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 0x7fa82c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8440d1180 con 0x7fa82c004390 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c00b9c0 con 0x7fa82c0039e0 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa82c00bbc0 con 0x7fa8440d5b60 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa8427fc640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82c004390 0x7fa82c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa842ffd640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c0039e0 0x7fa82c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.638+0000 7fa8437fe640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 0x7fa82c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa8427fc640 1 -- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 msgr2=0x7fa82c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa8427fc640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8440d5b60 0x7fa82c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa8427fc640 1 -- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c0039e0 msgr2=0x7fa82c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa8427fc640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c0039e0 0x7fa82c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa8427fc640 1 -- 172.21.15.53:0/2065651796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c003520 con 0x7fa82c004390 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa8427fc640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82c004390 0x7fa82c004860 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa838009ac0 tx=0x7fa82c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c683fdd5ed0f3dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838036070 con 0x7fa82c004390 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa838031400 con 0x7fa82c004390 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.639+0000 7fa841ffb640 1 -- 172.21.15.53:0/2065651796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838009690 con 0x7fa82c004390 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.640+0000 7fa8495f2640 1 -- 172.21.15.53:0/2065651796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82c004390 msgr2=0x7fa82c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.640+0000 7fa8495f2640 1 --2- 172.21.15.53:0/2065651796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82c004390 0x7fa82c004860 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa838009ac0 tx=0x7fa82c005830 comp rx=0 tx=0).stop 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.640+0000 7fa8437fe640 1 -- 172.21.15.53:0/2065651796 reap_dead start 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.640+0000 7fa8495f2640 1 -- 172.21.15.53:0/2065651796 shutdown_connections 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.640+0000 7fa8495f2640 1 -- 172.21.15.53:0/2065651796 >> 172.21.15.53:0/2065651796 conn(0x7fa8440d5040 msgr2=0x7fa8440d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.640+0000 7fa8495f2640 1 -- 172.21.15.53:0/2065651796 shutdown_connections 2024-05-10T10:34:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.640+0000 7fa8495f2640 1 -- 172.21.15.53:0/2065651796 wait complete. 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.641+0000 7fa8495f2640 1 Processor -- start 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.641+0000 7fa8495f2640 1 -- start start 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8495f2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c004390 0x7fa8440cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8495f2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8441f4310 0x7fa8441f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8495f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8440cdd70 0x7fa8440ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8495f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8440b5670 con 0x7fa8440cdd70 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8495f2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8440b6870 con 0x7fa82c004390 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8495f2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa8440b57f0 con 0x7fa8441f4310 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8440cdd70 0x7fa8440ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8440cdd70 0x7fa8440ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59752/0 (socket says 172.21.15.53:59752) 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8437fe640 1 -- 172.21.15.53:0/2798763534 learned_addr learned my addr 172.21.15.53:0/2798763534 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa8427fc640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8441f4310 0x7fa8441f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.642+0000 7fa842ffd640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c004390 0x7fa8440cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8437fe640 1 -- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8441f4310 msgr2=0x7fa8441f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8437fe640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8441f4310 0x7fa8441f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8437fe640 1 -- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c004390 msgr2=0x7fa8440cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8437fe640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c004390 0x7fa8440cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8437fe640 1 -- 172.21.15.53:0/2798763534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8441f4cd0 con 0x7fa8440cdd70 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa842ffd640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c004390 0x7fa8440cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8437fe640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8440cdd70 0x7fa8440ce220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa84408f510 tx=0x7fa83c00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa82b7fe640 1 -- 172.21.15.53:0/2798763534 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c019070 con 0x7fa8440cdd70 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa82b7fe640 1 -- 172.21.15.53:0/2798763534 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c0050d0 con 0x7fa8440cdd70 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8440cead0 con 0x7fa8440cdd70 2024-05-10T10:34:18.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa82b7fe640 1 -- 172.21.15.53:0/2798763534 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007030 con 0x7fa8440cdd70 2024-05-10T10:34:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.643+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa844150040 con 0x7fa8440cdd70 2024-05-10T10:34:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.644+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8440693f0 con 0x7fa8440cdd70 2024-05-10T10:34:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.645+0000 7fa82b7fe640 1 -- 172.21.15.53:0/2798763534 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa83c01c070 con 0x7fa8440cdd70 2024-05-10T10:34:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.645+0000 7fa82b7fe640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8300762d0 0x7fa830078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.646+0000 7fa82b7fe640 1 -- 172.21.15.53:0/2798763534 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fa83c0963a0 con 0x7fa8440cdd70 2024-05-10T10:34:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.646+0000 7fa842ffd640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8300762d0 0x7fa830078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.646+0000 7fa842ffd640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8300762d0 0x7fa830078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa834000f80 tx=0x7fa834006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:18.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.650+0000 7fa82b7fe640 1 -- 172.21.15.53:0/2798763534 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa83c05f900 con 0x7fa8440cdd70 2024-05-10T10:34:18.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.915+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fa8440d3400 con 0x7fa8440cdd70 2024-05-10T10:34:18.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.916+0000 7fa82b7fe640 1 -- 172.21.15.53:0/2798763534 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fa83c063c00 con 0x7fa8440cdd70 2024-05-10T10:34:18.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8300762d0 msgr2=0x7fa830078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8300762d0 0x7fa830078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa834000f80 tx=0x7fa834006040 comp rx=0 tx=0).stop 2024-05-10T10:34:18.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8440cdd70 msgr2=0x7fa8440ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:18.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8440cdd70 0x7fa8440ce220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa84408f510 tx=0x7fa83c00fc00 comp rx=0 tx=0).stop 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 shutdown_connections 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8300762d0 0x7fa830078790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8440cdd70 0x7fa8440ce220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8441f4310 0x7fa8441f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 --2- 172.21.15.53:0/2798763534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82c004390 0x7fa8440cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 >> 172.21.15.53:0/2798763534 conn(0x7fa8440d5040 msgr2=0x7fa8440b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 shutdown_connections 2024-05-10T10:34:18.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:18.919+0000 7fa8495f2640 1 -- 172.21.15.53:0/2798763534 wait complete. 2024-05-10T10:34:18.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-10T10:34:19.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.035+0000 7f31e0408640 1 Processor -- start 2024-05-10T10:34:19.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.035+0000 7f31e0408640 1 -- start start 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31e0408640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ac0 0x7f31d80b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31e0408640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31d80d1ea0 0x7f31d80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31e0408640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d80b5d30 con 0x7f31d80b54c0 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31e0408640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f31d80d0a70 con 0x7f31d80d1ea0 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31e0408640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31d80d5de0 con 0x7f31d80d1ac0 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31dd9a3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31d80d1ea0 0x7f31d80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de1a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ac0 0x7f31d80b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de1a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ac0 0x7f31d80b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:37112/0 (socket says 172.21.15.53:37112) 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31dd9a3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31d80d1ea0 0x7f31d80b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:39420/0 (socket says 172.21.15.53:39420) 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de1a4640 1 -- 172.21.15.53:0/2573117484 learned_addr learned my addr 172.21.15.53:0/2573117484 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de9a5640 1 --1- 172.21.15.53:0/2573117484 >> v1:172.21.15.135:6789/0 conn(0x7f31d80b54c0 0x7f31d80c8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:58546/0 (socket says 172.21.15.53:58546) 2024-05-10T10:34:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de1a4640 1 -- 172.21.15.53:0/2573117484 >> v2:172.21.15.135:3300/0 conn(0x7f31d80d1ea0 msgr2=0x7f31d80b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:19.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de1a4640 1 --2- 172.21.15.53:0/2573117484 >> v2:172.21.15.135:3300/0 conn(0x7f31d80d1ea0 0x7f31d80b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de1a4640 1 -- 172.21.15.53:0/2573117484 >> v1:172.21.15.135:6789/0 conn(0x7f31d80b54c0 legacy=0x7f31d80c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.036+0000 7f31de1a4640 1 -- 172.21.15.53:0/2573117484 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d80b5950 con 0x7f31d80d1ac0 2024-05-10T10:34:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.037+0000 7f31de1a4640 1 --2- 172.21.15.53:0/2573117484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ac0 0x7f31d80b6010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009870 tx=0x7f31d402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50cb6634f01a03cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.037+0000 7f31dd1a2640 1 -- 172.21.15.53:0/2573117484 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4037070 con 0x7f31d80d1ac0 2024-05-10T10:34:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.037+0000 7f31dd1a2640 1 -- 172.21.15.53:0/2573117484 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d4032440 con 0x7f31d80d1ac0 2024-05-10T10:34:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.037+0000 7f31dd1a2640 1 -- 172.21.15.53:0/2573117484 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4032810 con 0x7f31d80d1ac0 2024-05-10T10:34:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 -- 172.21.15.53:0/2573117484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ac0 msgr2=0x7f31d80b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 --2- 172.21.15.53:0/2573117484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ac0 0x7f31d80b6010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009870 tx=0x7f31d402a270 comp rx=0 tx=0).stop 2024-05-10T10:34:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 -- 172.21.15.53:0/2573117484 shutdown_connections 2024-05-10T10:34:19.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 --2- 172.21.15.53:0/2573117484 >> v2:172.21.15.135:3300/0 conn(0x7f31d80d1ea0 0x7f31d80b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 --2- 172.21.15.53:0/2573117484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ac0 0x7f31d80b6010 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 -- 172.21.15.53:0/2573117484 >> 172.21.15.53:0/2573117484 conn(0x7f31d80d5120 msgr2=0x7f31d80d5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:19.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 -- 172.21.15.53:0/2573117484 shutdown_connections 2024-05-10T10:34:19.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.038+0000 7f31e0408640 1 -- 172.21.15.53:0/2573117484 wait complete. 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.039+0000 7f31e0408640 1 Processor -- start 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.039+0000 7f31e0408640 1 -- start start 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31e0408640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d80b54c0 0x7f31d80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31e0408640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d80d1ac0 0x7f31d814a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31e0408640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ea0 0x7f31d8160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31e0408640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31d80d1080 con 0x7f31d80d1ac0 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31e0408640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31d80d0f00 con 0x7f31d80b54c0 2024-05-10T10:34:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31e0408640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31d80d3f10 con 0x7f31d80d1ea0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d80d1ac0 0x7f31d814a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d80d1ac0 0x7f31d814a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59756/0 (socket says 172.21.15.53:59756) 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 -- 172.21.15.53:0/594159966 learned_addr learned my addr 172.21.15.53:0/594159966 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31de1a4640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d80b54c0 0x7f31d80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31de9a5640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ea0 0x7f31d8160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 -- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ea0 msgr2=0x7f31d8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ea0 0x7f31d8160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 -- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d80b54c0 msgr2=0x7f31d80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d80b54c0 0x7f31d80ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.040+0000 7f31dd9a3640 1 -- 172.21.15.53:0/594159966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d81f6420 con 0x7f31d80d1ac0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.041+0000 7f31dd9a3640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d80d1ac0 0x7f31d814a0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00e870 tx=0x7f31cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.041+0000 7f31caffd640 1 -- 172.21.15.53:0/594159966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc017070 con 0x7f31d80d1ac0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.041+0000 7f31caffd640 1 -- 172.21.15.53:0/594159966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31cc00c940 con 0x7f31d80d1ac0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.041+0000 7f31caffd640 1 -- 172.21.15.53:0/594159966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00cd30 con 0x7f31d80d1ac0 2024-05-10T10:34:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.041+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d80ce230 con 0x7f31d80d1ac0 2024-05-10T10:34:19.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.041+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d80ce7f0 con 0x7f31d80d1ac0 2024-05-10T10:34:19.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.042+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d806a220 con 0x7f31d80d1ac0 2024-05-10T10:34:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.043+0000 7f31caffd640 1 -- 172.21.15.53:0/594159966 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f31cc01a070 con 0x7f31d80d1ac0 2024-05-10T10:34:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.043+0000 7f31caffd640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31bc0762d0 0x7f31bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.043+0000 7f31caffd640 1 -- 172.21.15.53:0/594159966 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7f31cc097600 con 0x7f31d80d1ac0 2024-05-10T10:34:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.043+0000 7f31de1a4640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31bc0762d0 0x7f31bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.044+0000 7f31de1a4640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31bc0762d0 0x7f31bc078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31d4034000 tx=0x7f31d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.047+0000 7f31caffd640 1 -- 172.21.15.53:0/594159966 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f31cc0611b0 con 0x7f31d80d1ac0 2024-05-10T10:34:19.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.322+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f31d80d15e0 con 0x7f31d80d1ac0 2024-05-10T10:34:19.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.324+0000 7f31caffd640 1 -- 172.21.15.53:0/594159966 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f31cc060ab0 con 0x7f31d80d1ac0 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31bc0762d0 msgr2=0x7f31bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31bc0762d0 0x7f31bc078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31d4034000 tx=0x7f31d402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d80d1ac0 msgr2=0x7f31d814a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d80d1ac0 0x7f31d814a0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00e870 tx=0x7f31cc00ed30 comp rx=0 tx=0).stop 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 shutdown_connections 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31bc0762d0 0x7f31bc078790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d80d1ea0 0x7f31d8160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d80d1ac0 0x7f31d814a0f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 --2- 172.21.15.53:0/594159966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d80b54c0 0x7f31d80ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 >> 172.21.15.53:0/594159966 conn(0x7f31d80d5120 msgr2=0x7f31d80b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:19.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 shutdown_connections 2024-05-10T10:34:19.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.326+0000 7f31e0408640 1 -- 172.21.15.53:0/594159966 wait complete. 2024-05-10T10:34:19.338 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-10T10:34:19.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.441+0000 7f1a182d3640 1 Processor -- start 2024-05-10T10:34:19.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.442+0000 7f1a182d3640 1 -- start start 2024-05-10T10:34:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.442+0000 7f1a182d3640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1006fb10 0x7f1a100cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.442+0000 7f1a182d3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1a100724d0 0x7f1a10072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.442+0000 7f1a182d3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a10146d10 con 0x7f1a100d0100 2024-05-10T10:34:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.442+0000 7f1a182d3640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1a10072040 con 0x7f1a100724d0 2024-05-10T10:34:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.442+0000 7f1a182d3640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1006ff70 con 0x7f1a1006fb10 2024-05-10T10:34:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a1586e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1006fb10 0x7f1a100cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a16870640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1a100724d0 0x7f1a10072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a1586e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1006fb10 0x7f1a100cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:37130/0 (socket says 172.21.15.53:37130) 2024-05-10T10:34:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a16870640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1a100724d0 0x7f1a10072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:39446/0 (socket says 172.21.15.53:39446) 2024-05-10T10:34:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a1586e640 1 -- 172.21.15.53:0/2808249123 learned_addr learned my addr 172.21.15.53:0/2808249123 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a1606f640 1 --1- 172.21.15.53:0/2808249123 >> v1:172.21.15.135:6789/0 conn(0x7f1a100d0100 0x7f1a100ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:58552/0 (socket says 172.21.15.53:58552) 2024-05-10T10:34:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a16870640 1 -- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1006fb10 msgr2=0x7f1a100cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:19.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a16870640 1 --2- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1006fb10 0x7f1a100cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a16870640 1 -- 172.21.15.53:0/2808249123 >> v1:172.21.15.135:6789/0 conn(0x7f1a100d0100 legacy=0x7f1a100ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a16870640 1 -- 172.21.15.53:0/2808249123 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1006dfb0 con 0x7f1a100724d0 2024-05-10T10:34:19.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.443+0000 7f1a1586e640 1 --2- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1006fb10 0x7f1a100cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.444+0000 7f1a16870640 1 --2- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a100724d0 0x7f1a10072960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009dc0 tx=0x7f1a0802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92841d98f5583b0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.444+0000 7f1a1506d640 1 -- 172.21.15.53:0/2808249123 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0802ae60 con 0x7f1a100724d0 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.444+0000 7f1a1506d640 1 -- 172.21.15.53:0/2808249123 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a08007740 con 0x7f1a100724d0 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.444+0000 7f1a1506d640 1 -- 172.21.15.53:0/2808249123 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08007b10 con 0x7f1a100724d0 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 -- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a100724d0 msgr2=0x7f1a10072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 --2- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a100724d0 0x7f1a10072960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009dc0 tx=0x7f1a0802a680 comp rx=0 tx=0).stop 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 -- 172.21.15.53:0/2808249123 shutdown_connections 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 --2- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1006fb10 0x7f1a100cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 --2- 172.21.15.53:0/2808249123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a100724d0 0x7f1a10072960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 -- 172.21.15.53:0/2808249123 >> 172.21.15.53:0/2808249123 conn(0x7f1a100d0c90 msgr2=0x7f1a100d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 -- 172.21.15.53:0/2808249123 shutdown_connections 2024-05-10T10:34:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.445+0000 7f1a182d3640 1 -- 172.21.15.53:0/2808249123 wait complete. 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.446+0000 7f1a182d3640 1 Processor -- start 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.446+0000 7f1a182d3640 1 -- start start 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a182d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1006fb10 0x7f1a100b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a182d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a100724d0 0x7f1a100cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a182d3640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a100d0100 0x7f1a101f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a182d3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1a100701f0 con 0x7f1a100724d0 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a182d3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1006e530 con 0x7f1a1006fb10 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a182d3640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1a100d04e0 con 0x7f1a100d0100 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a1586e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a100724d0 0x7f1a100cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a1586e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a100724d0 0x7f1a100cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59762/0 (socket says 172.21.15.53:59762) 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a1586e640 1 -- 172.21.15.53:0/1835459780 learned_addr learned my addr 172.21.15.53:0/1835459780 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a16870640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a100d0100 0x7f1a101f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.447+0000 7f1a1606f640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1006fb10 0x7f1a100b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a1586e640 1 -- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a100d0100 msgr2=0x7f1a101f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a1586e640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a100d0100 0x7f1a101f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a1586e640 1 -- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1006fb10 msgr2=0x7f1a100b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a1586e640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1006fb10 0x7f1a100b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a1586e640 1 -- 172.21.15.53:0/1835459780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a100c4660 con 0x7f1a100724d0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a1586e640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a100724d0 0x7f1a100cbd20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000e9b0 tx=0x7f1a0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a06ffd640 1 -- 172.21.15.53:0/1835459780 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00017070 con 0x7f1a100724d0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a06ffd640 1 -- 172.21.15.53:0/1835459780 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a0000c9a0 con 0x7f1a100724d0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a100caf60 con 0x7f1a100724d0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a06ffd640 1 -- 172.21.15.53:0/1835459780 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000cd50 con 0x7f1a100724d0 2024-05-10T10:34:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.448+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a100cb520 con 0x7f1a100724d0 2024-05-10T10:34:19.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.449+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a1006a220 con 0x7f1a100724d0 2024-05-10T10:34:19.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.450+0000 7f1a06ffd640 1 -- 172.21.15.53:0/1835459780 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1a0001a070 con 0x7f1a100724d0 2024-05-10T10:34:19.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.450+0000 7f1a06ffd640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19f40762d0 0x7f19f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.450+0000 7f1a06ffd640 1 -- 172.21.15.53:0/1835459780 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7f1a00097280 con 0x7f1a100724d0 2024-05-10T10:34:19.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.450+0000 7f1a1606f640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19f40762d0 0x7f19f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.451+0000 7f1a1606f640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19f40762d0 0x7f19f4078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a0c007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.454+0000 7f1a06ffd640 1 -- 172.21.15.53:0/1835459780 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1a00060750 con 0x7f1a100724d0 2024-05-10T10:34:19.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.720+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f1a101f56d0 con 0x7f1a100724d0 2024-05-10T10:34:19.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.722+0000 7f1a06ffd640 1 -- 172.21.15.53:0/1835459780 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f1a00064a50 con 0x7f1a100724d0 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19f40762d0 msgr2=0x7f19f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19f40762d0 0x7f19f4078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a0c007e80 comp rx=0 tx=0).stop 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a100724d0 msgr2=0x7f1a100cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a100724d0 0x7f1a100cbd20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000e9b0 tx=0x7f1a0000ee70 comp rx=0 tx=0).stop 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 shutdown_connections 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19f40762d0 0x7f19f4078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a100d0100 0x7f1a101f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a100724d0 0x7f1a100cbd20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 --2- 172.21.15.53:0/1835459780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1006fb10 0x7f1a100b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 >> 172.21.15.53:0/1835459780 conn(0x7f1a100d0c90 msgr2=0x7f1a100ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:19.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 shutdown_connections 2024-05-10T10:34:19.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.724+0000 7f1a182d3640 1 -- 172.21.15.53:0/1835459780 wait complete. 2024-05-10T10:34:19.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:34:19.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.839+0000 7f341e86d640 1 Processor -- start 2024-05-10T10:34:19.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.839+0000 7f341e86d640 1 -- start start 2024-05-10T10:34:19.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.839+0000 7f341e86d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3418162750 0x7f34180d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.839+0000 7f341e86d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34180d5cb0 0x7f34180d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f341e86d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34180b45f0 con 0x7f3418162b30 2024-05-10T10:34:19.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f341e86d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f34180b4f20 con 0x7f34180d5cb0 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f341e86d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f34180d0fe0 con 0x7f3418162750 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f340ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34180d5cb0 0x7f34180d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f340ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34180d5cb0 0x7f34180d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59774/0 (socket says 172.21.15.53:59774) 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f340ffff640 1 -- 172.21.15.53:0/3135851429 learned_addr learned my addr 172.21.15.53:0/3135851429 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f3417fff640 1 --2- 172.21.15.53:0/3135851429 >> v2:172.21.15.135:3300/0 conn(0x7f3418162750 0x7f34180d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f340ffff640 1 -- 172.21.15.53:0/3135851429 >> v2:172.21.15.135:3300/0 conn(0x7f3418162750 msgr2=0x7f34180d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f340ffff640 1 --2- 172.21.15.53:0/3135851429 >> v2:172.21.15.135:3300/0 conn(0x7f3418162750 0x7f34180d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f340ffff640 1 -- 172.21.15.53:0/3135851429 >> v1:172.21.15.135:6789/0 conn(0x7f3418162b30 legacy=0x7f34180d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.840+0000 7f340ffff640 1 -- 172.21.15.53:0/3135851429 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34180b48d0 con 0x7f34180d5cb0 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f340ffff640 1 --2- 172.21.15.53:0/3135851429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34180d5cb0 0x7f34180d3630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f34180b7c40 tx=0x7f340002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257ad814b2159d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f34177fe640 1 -- 172.21.15.53:0/3135851429 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400037070 con 0x7f34180d5cb0 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f34177fe640 1 -- 172.21.15.53:0/3135851429 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3400032440 con 0x7f34180d5cb0 2024-05-10T10:34:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f34177fe640 1 -- 172.21.15.53:0/3135851429 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400032810 con 0x7f34180d5cb0 2024-05-10T10:34:19.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f341e86d640 1 -- 172.21.15.53:0/3135851429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34180d5cb0 msgr2=0x7f34180d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f341e86d640 1 --2- 172.21.15.53:0/3135851429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34180d5cb0 0x7f34180d3630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f34180b7c40 tx=0x7f340002a1a0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f341e86d640 1 -- 172.21.15.53:0/3135851429 shutdown_connections 2024-05-10T10:34:19.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f341e86d640 1 --2- 172.21.15.53:0/3135851429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34180d5cb0 0x7f34180d3630 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f341e86d640 1 --2- 172.21.15.53:0/3135851429 >> v2:172.21.15.135:3300/0 conn(0x7f3418162750 0x7f34180d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.841+0000 7f341e86d640 1 -- 172.21.15.53:0/3135851429 >> 172.21.15.53:0/3135851429 conn(0x7f341809e3b0 msgr2=0x7f34180d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.842+0000 7f341e86d640 1 -- 172.21.15.53:0/3135851429 shutdown_connections 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.842+0000 7f341e86d640 1 -- 172.21.15.53:0/3135851429 wait complete. 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 Processor -- start 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 -- start start 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3418162750 0x7f34180b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3418162b30 0x7f34181f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34180ba700 0x7f341815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34180d0d30 con 0x7f3418162750 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f34180d0bb0 con 0x7f34180ba700 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.843+0000 7f341e86d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f34180b63c0 con 0x7f3418162b30 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3418162750 0x7f34180b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3418162750 0x7f34180b8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59782/0 (socket says 172.21.15.53:59782) 2024-05-10T10:34:19.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 -- 172.21.15.53:0/2835661510 learned_addr learned my addr 172.21.15.53:0/2835661510 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f340ffff640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3418162b30 0x7f34181f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f341ce0a640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34180ba700 0x7f341815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 -- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3418162b30 msgr2=0x7f34181f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3418162b30 0x7f34181f8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 -- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34180ba700 msgr2=0x7f341815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34180ba700 0x7f341815acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 -- 172.21.15.53:0/2835661510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34181f9a90 con 0x7f3418162750 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f340ffff640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3418162b30 0x7f34181f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f341ce0a640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34180ba700 0x7f341815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f3417fff640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3418162750 0x7f34180b8f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f340800d800 tx=0x7f340800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f34157fa640 1 -- 172.21.15.53:0/2835661510 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340800b6c0 con 0x7f3418162750 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f34157fa640 1 -- 172.21.15.53:0/2835661510 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408019070 con 0x7f3418162750 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f34157fa640 1 -- 172.21.15.53:0/2835661510 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340800ba80 con 0x7f3418162750 2024-05-10T10:34:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34180b7e90 con 0x7f3418162750 2024-05-10T10:34:19.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.844+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34180b8450 con 0x7f3418162750 2024-05-10T10:34:19.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.845+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341806a220 con 0x7f3418162750 2024-05-10T10:34:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.846+0000 7f34157fa640 1 -- 172.21.15.53:0/2835661510 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f340800bc20 con 0x7f3418162750 2024-05-10T10:34:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.846+0000 7f34157fa640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33f80762d0 0x7f33f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.846+0000 7f34157fa640 1 -- 172.21.15.53:0/2835661510 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7f34080979e0 con 0x7f3418162750 2024-05-10T10:34:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.847+0000 7f340ffff640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33f80762d0 0x7f33f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.847+0000 7f340ffff640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33f80762d0 0x7f33f8078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3400034000 tx=0x7f340002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:19.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:19.850+0000 7f34157fa640 1 -- 172.21.15.53:0/2835661510 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3408061460 con 0x7f3418162750 2024-05-10T10:34:20.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.044+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f341806dca0 con 0x7f3418162750 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.044+0000 7f34157fa640 1 -- 172.21.15.53:0/2835661510 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7f3408060d60 con 0x7f3418162750 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:20.045 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:34:20.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33f80762d0 msgr2=0x7f33f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:20.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33f80762d0 0x7f33f8078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3400034000 tx=0x7f340002e000 comp rx=0 tx=0).stop 2024-05-10T10:34:20.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3418162750 msgr2=0x7f34180b8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:20.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3418162750 0x7f34180b8f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f340800d800 tx=0x7f340800dcc0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 shutdown_connections 2024-05-10T10:34:20.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33f80762d0 0x7f33f8078790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34180ba700 0x7f341815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3418162b30 0x7f34181f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 --2- 172.21.15.53:0/2835661510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3418162750 0x7f34180b8f30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 >> 172.21.15.53:0/2835661510 conn(0x7f341809e3b0 msgr2=0x7f341806dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.046+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 shutdown_connections 2024-05-10T10:34:20.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.047+0000 7f341e86d640 1 -- 172.21.15.53:0/2835661510 wait complete. 2024-05-10T10:34:20.058 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:34:20.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:34:20.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.165+0000 7efe4dd40640 1 Processor -- start 2024-05-10T10:34:20.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.165+0000 7efe4dd40640 1 -- start start 2024-05-10T10:34:20.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.166+0000 7efe4dd40640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe480cf780 0x7efe480cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.166+0000 7efe4dd40640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe480d0120 con 0x7efe480cd450 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.166+0000 7efe4dd40640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe480d1580 con 0x7efe480b4210 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.166+0000 7efe4dd40640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efe480d3c70 con 0x7efe480cf780 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.166+0000 7efe47fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efe480cd450 0x7efe480cd8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34018/0 (socket says 172.21.15.53:34018) 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.166+0000 7efe47fff640 1 -- 172.21.15.53:0/3382391614 learned_addr learned my addr 172.21.15.53:0/3382391614 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe46ffd640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe480cf780 0x7efe480cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 102110538 0 0) 0x7efe480d0120 con 0x7efe480cd450 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe30003520 con 0x7efe480cd450 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1619226521 0 0) 0x7efe480d1580 con 0x7efe480b4210 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe480d0120 con 0x7efe480b4210 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2271854507 0 0) 0x7efe30003520 con 0x7efe480cd450 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe480cf780 msgr2=0x7efe480cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe480cf780 0x7efe480cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 >> v1:172.21.15.135:6789/0 conn(0x7efe480b4210 legacy=0x7efe4806e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe480d1860 con 0x7efe480cd450 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7efe400019f0 con 0x7efe480cd450 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 >> v1:172.21.15.53:6789/0 conn(0x7efe480cd450 legacy=0x7efe480cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe30003b70 0x7efe30003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe30004800 0x7efe30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe300051c0 0x7efe3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe3000bfb0 con 0x7efe300051c0 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe3000c130 con 0x7efe30004800 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efe3000c330 con 0x7efe30003b70 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe46ffd640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe480cf780 0x7efe480cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe47fff640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe300051c0 0x7efe3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe477fe640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe30003b70 0x7efe30003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.167+0000 7efe46ffd640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe30004800 0x7efe30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe47fff640 1 -- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe30003b70 msgr2=0x7efe30003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe47fff640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe30003b70 0x7efe30003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe47fff640 1 -- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe30004800 msgr2=0x7efe30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe47fff640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe30004800 0x7efe30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe47fff640 1 -- 172.21.15.53:0/3382391614 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30003520 con 0x7efe300051c0 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe47fff640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe300051c0 0x7efe3000b9c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efe4000a800 tx=0x7efe30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1beb91429d6da4a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4000a430 con 0x7efe300051c0 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4002f070 con 0x7efe300051c0 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.168+0000 7efe467fc640 1 -- 172.21.15.53:0/3382391614 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4002bc90 con 0x7efe300051c0 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.169+0000 7efe4dd40640 1 -- 172.21.15.53:0/3382391614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe300051c0 msgr2=0x7efe3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.169+0000 7efe4dd40640 1 --2- 172.21.15.53:0/3382391614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe300051c0 0x7efe3000b9c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efe4000a800 tx=0x7efe30005830 comp rx=0 tx=0).stop 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.169+0000 7efe47fff640 1 -- 172.21.15.53:0/3382391614 reap_dead start 2024-05-10T10:34:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.169+0000 7efe4dd40640 1 -- 172.21.15.53:0/3382391614 shutdown_connections 2024-05-10T10:34:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.169+0000 7efe4dd40640 1 -- 172.21.15.53:0/3382391614 >> 172.21.15.53:0/3382391614 conn(0x7efe480d2e00 msgr2=0x7efe480d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.169+0000 7efe4dd40640 1 -- 172.21.15.53:0/3382391614 shutdown_connections 2024-05-10T10:34:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.169+0000 7efe4dd40640 1 -- 172.21.15.53:0/3382391614 wait complete. 2024-05-10T10:34:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.170+0000 7efe4dd40640 1 Processor -- start 2024-05-10T10:34:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.170+0000 7efe4dd40640 1 -- start start 2024-05-10T10:34:20.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe4dd40640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe300051c0 0x7efe48162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe4dd40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe4814d380 0x7efe4814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe4dd40640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe4814cb10 0x7efe48071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe4dd40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe480d1e20 con 0x7efe4814d380 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe4dd40640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe4806e840 con 0x7efe300051c0 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe4dd40640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efe480d2100 con 0x7efe4814cb10 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe46ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe4814d380 0x7efe4814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe46ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe4814d380 0x7efe4814fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59808/0 (socket says 172.21.15.53:59808) 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe46ffd640 1 -- 172.21.15.53:0/518415190 learned_addr learned my addr 172.21.15.53:0/518415190 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe477fe640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe300051c0 0x7efe48162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.171+0000 7efe47fff640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe4814cb10 0x7efe48071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe46ffd640 1 -- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe4814cb10 msgr2=0x7efe48071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe46ffd640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe4814cb10 0x7efe48071390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe46ffd640 1 -- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe300051c0 msgr2=0x7efe48162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe46ffd640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe300051c0 0x7efe48162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe46ffd640 1 -- 172.21.15.53:0/518415190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe48150330 con 0x7efe4814d380 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe477fe640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe300051c0 0x7efe48162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe47fff640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe4814cb10 0x7efe48071390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe46ffd640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe4814d380 0x7efe4814fdf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efe3c011c30 tx=0x7efe3c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe2ffff640 1 -- 172.21.15.53:0/518415190 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00fd20 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe2ffff640 1 -- 172.21.15.53:0/518415190 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe3c014440 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe48162760 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe2ffff640 1 -- 172.21.15.53:0/518415190 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0147f0 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.172+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe480cbbc0 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.173+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe480693f0 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.174+0000 7efe2ffff640 1 -- 172.21.15.53:0/518415190 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efe3c012040 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.174+0000 7efe2ffff640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efe340762d0 0x7efe34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.174+0000 7efe2ffff640 1 -- 172.21.15.53:0/518415190 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7efe3c097c90 con 0x7efe4814d380 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.175+0000 7efe477fe640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efe340762d0 0x7efe34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.175+0000 7efe477fe640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efe340762d0 0x7efe34078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe380036b0 tx=0x7efe38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.178+0000 7efe2ffff640 1 -- 172.21.15.53:0/518415190 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7efe3c019030 con 0x7efe4814d380 2024-05-10T10:34:20.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.370+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efe480b8c80 con 0x7efe4814d380 2024-05-10T10:34:20.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.917+0000 7efe2ffff640 1 -- 172.21.15.53:0/518415190 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7efe3c061730 con 0x7efe4814d380 2024-05-10T10:34:21.014 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efe340762d0 msgr2=0x7efe34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efe340762d0 0x7efe34078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe380036b0 tx=0x7efe38009040 comp rx=0 tx=0).stop 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe4814d380 msgr2=0x7efe4814fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe4814d380 0x7efe4814fdf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efe3c011c30 tx=0x7efe3c00f610 comp rx=0 tx=0).stop 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 shutdown_connections 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efe340762d0 0x7efe34078790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe4814cb10 0x7efe48071390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe4814d380 0x7efe4814fdf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 --2- 172.21.15.53:0/518415190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe300051c0 0x7efe48162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 >> 172.21.15.53:0/518415190 conn(0x7efe480d2e00 msgr2=0x7efe480b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:21.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 shutdown_connections 2024-05-10T10:34:21.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:20.920+0000 7efe4dd40640 1 -- 172.21.15.53:0/518415190 wait complete. 2024-05-10T10:34:21.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:34:21.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.141+0000 7f7d22191640 1 Processor -- start 2024-05-10T10:34:21.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.141+0000 7f7d22191640 1 -- start start 2024-05-10T10:34:21.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d22191640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d1c162900 0x7f7d1c0d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d22191640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c0d4190 con 0x7f7d1c162ce0 2024-05-10T10:34:21.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d22191640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c0cfc40 con 0x7f7d1c0d3950 2024-05-10T10:34:21.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d22191640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1c0d3eb0 con 0x7f7d1c162900 2024-05-10T10:34:21.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d1b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d1c162900 0x7f7d1c0d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d1affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d1c162ce0 0x7f7d1c0b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34032/0 (socket says 172.21.15.53:34032) 2024-05-10T10:34:21.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d1affd640 1 -- 172.21.15.53:0/2691538072 learned_addr learned my addr 172.21.15.53:0/2691538072 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:21.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d1b7fe640 1 -- 172.21.15.53:0/2691538072 >> v1:172.21.15.135:6789/0 conn(0x7f7d1c0d3950 legacy=0x7f7d1c0d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d1b7fe640 1 -- 172.21.15.53:0/2691538072 >> v1:172.21.15.53:6789/0 conn(0x7f7d1c162ce0 legacy=0x7f7d1c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.142+0000 7f7d1b7fe640 1 -- 172.21.15.53:0/2691538072 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c0cfee0 con 0x7f7d1c162900 2024-05-10T10:34:21.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.143+0000 7f7d1b7fe640 1 --2- 172.21.15.53:0/2691538072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c0d3410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0097a0 tx=0x7f7d0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2b47ca9442d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:21.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.143+0000 7f7d1a7fc640 1 -- 172.21.15.53:0/2691538072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c037070 con 0x7f7d1c162900 2024-05-10T10:34:21.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.143+0000 7f7d1a7fc640 1 -- 172.21.15.53:0/2691538072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c032440 con 0x7f7d1c162900 2024-05-10T10:34:21.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.143+0000 7f7d1a7fc640 1 -- 172.21.15.53:0/2691538072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c032810 con 0x7f7d1c162900 2024-05-10T10:34:21.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.144+0000 7f7d22191640 1 -- 172.21.15.53:0/2691538072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 msgr2=0x7f7d1c0d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.144+0000 7f7d22191640 1 --2- 172.21.15.53:0/2691538072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c0d3410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0097a0 tx=0x7f7d0c02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.144+0000 7f7d22191640 1 -- 172.21.15.53:0/2691538072 shutdown_connections 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.144+0000 7f7d22191640 1 --2- 172.21.15.53:0/2691538072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c0d3410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.144+0000 7f7d22191640 1 -- 172.21.15.53:0/2691538072 >> 172.21.15.53:0/2691538072 conn(0x7f7d1c09e430 msgr2=0x7f7d1c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.144+0000 7f7d22191640 1 -- 172.21.15.53:0/2691538072 shutdown_connections 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.144+0000 7f7d22191640 1 -- 172.21.15.53:0/2691538072 wait complete. 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.145+0000 7f7d22191640 1 Processor -- start 2024-05-10T10:34:21.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.145+0000 7f7d22191640 1 -- start start 2024-05-10T10:34:21.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d22191640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1c0d3950 0x7f7d1c0b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d22191640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c06d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d22191640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1c162ce0 0x7f7d1c06d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d22191640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c0b4300 con 0x7f7d1c162900 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d22191640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c0b4180 con 0x7f7d1c0d3950 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d22191640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c0b4ff0 con 0x7f7d1c162ce0 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d1affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c06d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d1affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c06d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59830/0 (socket says 172.21.15.53:59830) 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d1affd640 1 -- 172.21.15.53:0/1631397604 learned_addr learned my addr 172.21.15.53:0/1631397604 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d1b7fe640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1c0d3950 0x7f7d1c0b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d1bfff640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1c162ce0 0x7f7d1c06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d1affd640 1 -- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1c162ce0 msgr2=0x7f7d1c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.146+0000 7f7d1affd640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1c162ce0 0x7f7d1c06d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d1affd640 1 -- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1c0d3950 msgr2=0x7f7d1c0b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d1affd640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1c0d3950 0x7f7d1c0b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d1affd640 1 -- 172.21.15.53:0/1631397604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c0ba4d0 con 0x7f7d1c162900 2024-05-10T10:34:21.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d1bfff640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1c162ce0 0x7f7d1c06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:21.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d1affd640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c06d0e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c4d0 tx=0x7f7d1000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:21.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d03fff640 1 -- 172.21.15.53:0/1631397604 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10017070 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d03fff640 1 -- 172.21.15.53:0/1631397604 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d10005ce0 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d03fff640 1 -- 172.21.15.53:0/1631397604 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10007270 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c15d360 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.147+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c15d920 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.148+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c06a220 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.149+0000 7f7d03fff640 1 -- 172.21.15.53:0/1631397604 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d1001a070 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.149+0000 7f7d03fff640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d080762d0 0x7f7d08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.149+0000 7f7d03fff640 1 -- 172.21.15.53:0/1631397604 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5619+0+0 (secure 0 0 0) 0x7f7d10096850 con 0x7f7d1c162900 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.149+0000 7f7d1b7fe640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d080762d0 0x7f7d08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:21.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.150+0000 7f7d1b7fe640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d080762d0 0x7f7d08078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c034000 tx=0x7f7d0c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:21.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.154+0000 7f7d03fff640 1 -- 172.21.15.53:0/1631397604 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7d1c0d4190 con 0x7f7d1c162900 2024-05-10T10:34:21.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.346+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7d1c0b99b0 con 0x7f7d1c162900 2024-05-10T10:34:21.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.919+0000 7f7d03fff640 1 -- 172.21.15.53:0/1631397604 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7f7d1005fbb0 con 0x7f7d1c162900 2024-05-10T10:34:21.921 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:34:21.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d080762d0 msgr2=0x7f7d08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d080762d0 0x7f7d08078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c034000 tx=0x7f7d0c02e000 comp rx=0 tx=0).stop 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 msgr2=0x7f7d1c06d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c06d0e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c4d0 tx=0x7f7d1000c990 comp rx=0 tx=0).stop 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 shutdown_connections 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d080762d0 0x7f7d08078790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1c162ce0 0x7f7d1c06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1c162900 0x7f7d1c06d0e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 --2- 172.21.15.53:0/1631397604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1c0d3950 0x7f7d1c0b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.922+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 >> 172.21.15.53:0/1631397604 conn(0x7f7d1c09e430 msgr2=0x7f7d1c15e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.923+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 shutdown_connections 2024-05-10T10:34:21.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:21.923+0000 7f7d22191640 1 -- 172.21.15.53:0/1631397604 wait complete. 2024-05-10T10:34:21.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:34:22.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.064+0000 7f042e1af640 1 Processor -- start 2024-05-10T10:34:22.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.064+0000 7f042e1af640 1 -- start start 2024-05-10T10:34:22.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f042e1af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280d3df0 0x7f04280d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:22.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f042e1af640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04280b4aa0 0x7f04280b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:22.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f042e1af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f04280d1a10 0x7f04280d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:22.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f042e1af640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f04280cfd50 con 0x7f04280b4aa0 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f042e1af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f04280b5d90 con 0x7f04280d1a10 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f042e1af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f04280d41d0 con 0x7f04280d3df0 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04280b4aa0 0x7f04280b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f04280b4aa0 0x7f04280b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59836/0 (socket says 172.21.15.53:59836) 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 -- 172.21.15.53:0/3166413640 learned_addr learned my addr 172.21.15.53:0/3166413640 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 -- 172.21.15.53:0/3166413640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280d3df0 msgr2=0x7f04280d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 --2- 172.21.15.53:0/3166413640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280d3df0 0x7f04280d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 -- 172.21.15.53:0/3166413640 >> v2:172.21.15.135:3300/0 conn(0x7f04280d1a10 msgr2=0x7f04280d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 --2- 172.21.15.53:0/3166413640 >> v2:172.21.15.135:3300/0 conn(0x7f04280d1a10 0x7f04280d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.065+0000 7f0426ffd640 1 -- 172.21.15.53:0/3166413640 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04280b54f0 con 0x7f04280b4aa0 2024-05-10T10:34:22.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.066+0000 7f0426ffd640 1 --2- 172.21.15.53:0/3166413640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280b4aa0 0x7f04280b4f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f042808e580 tx=0x7f041c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d3c27fe9c1177d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.066+0000 7f04267fc640 1 -- 172.21.15.53:0/3166413640 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c037070 con 0x7f04280b4aa0 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.066+0000 7f04267fc640 1 -- 172.21.15.53:0/3166413640 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c032440 con 0x7f04280b4aa0 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.066+0000 7f04267fc640 1 -- 172.21.15.53:0/3166413640 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c032810 con 0x7f04280b4aa0 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 -- 172.21.15.53:0/3166413640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280b4aa0 msgr2=0x7f04280b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 --2- 172.21.15.53:0/3166413640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280b4aa0 0x7f04280b4f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f042808e580 tx=0x7f041c02a1e0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 -- 172.21.15.53:0/3166413640 shutdown_connections 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 --2- 172.21.15.53:0/3166413640 >> v2:172.21.15.135:3300/0 conn(0x7f04280d1a10 0x7f04280d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 --2- 172.21.15.53:0/3166413640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280b4aa0 0x7f04280b4f00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 --2- 172.21.15.53:0/3166413640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280d3df0 0x7f04280d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 -- 172.21.15.53:0/3166413640 >> 172.21.15.53:0/3166413640 conn(0x7f042809e450 msgr2=0x7f04280d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:22.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 -- 172.21.15.53:0/3166413640 shutdown_connections 2024-05-10T10:34:22.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.067+0000 7f042e1af640 1 -- 172.21.15.53:0/3166413640 wait complete. 2024-05-10T10:34:22.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.068+0000 7f042e1af640 1 Processor -- start 2024-05-10T10:34:22.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.068+0000 7f042e1af640 1 -- start start 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f042e1af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280b4aa0 0x7f04280c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f042e1af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280d1a10 0x7f042814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f042e1af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04280d3df0 0x7f04281ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f042e1af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04280cff80 con 0x7f04280d1a10 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f042e1af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04280b5ab0 con 0x7f04280d3df0 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f042e1af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f04280b6120 con 0x7f04280b4aa0 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280d1a10 0x7f042814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280d1a10 0x7f042814f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59842/0 (socket says 172.21.15.53:59842) 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 -- 172.21.15.53:0/2980990743 learned_addr learned my addr 172.21.15.53:0/2980990743 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0427fff640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04280d3df0 0x7f04281ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f04277fe640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280b4aa0 0x7f04280c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 -- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280b4aa0 msgr2=0x7f04280c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280b4aa0 0x7f04280c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 -- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04280d3df0 msgr2=0x7f04281ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04280d3df0 0x7f04281ea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 -- 172.21.15.53:0/2980990743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04280c9b20 con 0x7f04280d1a10 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.069+0000 7f0426ffd640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280d1a10 0x7f042814f510 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f041c02a220 tx=0x7f041c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.070+0000 7f040ffff640 1 -- 172.21.15.53:0/2980990743 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c041070 con 0x7f04280d1a10 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.070+0000 7f040ffff640 1 -- 172.21.15.53:0/2980990743 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c03c870 con 0x7f04280d1a10 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.070+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0428160f30 con 0x7f04280d1a10 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.070+0000 7f040ffff640 1 -- 172.21.15.53:0/2980990743 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c037040 con 0x7f04280d1a10 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.070+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04281614f0 con 0x7f04280d1a10 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.071+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04280d43e0 con 0x7f04280d1a10 2024-05-10T10:34:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.072+0000 7f040ffff640 1 -- 172.21.15.53:0/2980990743 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f041c044070 con 0x7f04280d1a10 2024-05-10T10:34:22.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.072+0000 7f040ffff640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04140762d0 0x7f0414078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:22.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.072+0000 7f040ffff640 1 -- 172.21.15.53:0/2980990743 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 6001+0+0 (secure 0 0 0) 0x7f041c0b8650 con 0x7f04280d1a10 2024-05-10T10:34:22.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.072+0000 7f04277fe640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04140762d0 0x7f0414078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:22.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.073+0000 7f04277fe640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04140762d0 0x7f0414078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0418009890 tx=0x7f0418008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:22.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.076+0000 7f040ffff640 1 -- 172.21.15.53:0/2980990743 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f04280d43e0 con 0x7f04280d1a10 2024-05-10T10:34:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.304+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f04280d2cf0 con 0x7f04280d1a10 2024-05-10T10:34:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.942+0000 7f040ffff640 1 -- 172.21.15.53:0/2980990743 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f04280d2cf0 con 0x7f04280d1a10 2024-05-10T10:34:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 5 and data pool 6 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04140762d0 msgr2=0x7f0414078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04140762d0 0x7f0414078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0418009890 tx=0x7f0418008040 comp rx=0 tx=0).stop 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280d1a10 msgr2=0x7f042814f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280d1a10 0x7f042814f510 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f041c02a220 tx=0x7f041c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 shutdown_connections 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f04140762d0 0x7f0414078790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04280d3df0 0x7f04281ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04280d1a10 0x7f042814f510 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 --2- 172.21.15.53:0/2980990743 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04280b4aa0 0x7f04280c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.953+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 >> 172.21.15.53:0/2980990743 conn(0x7f042809e450 msgr2=0x7f04280b9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:22.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.954+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 shutdown_connections 2024-05-10T10:34:22.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:22.954+0000 7f042e1af640 1 -- 172.21.15.53:0/2980990743 wait complete. 2024-05-10T10:34:22.976 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:23.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.083+0000 7f4e2b73f640 1 Processor -- start 2024-05-10T10:34:23.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.083+0000 7f4e2b73f640 1 -- start start 2024-05-10T10:34:23.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e2b73f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4e240b0940 0x7f4e24167a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e2b73f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e24146be0 con 0x7f4e2406c2c0 2024-05-10T10:34:23.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e2b73f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e240719d0 con 0x7f4e24146750 2024-05-10T10:34:23.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e2b73f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4e240af4a0 con 0x7f4e240b0940 2024-05-10T10:34:23.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e294db640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4e2406c2c0 0x7f4e2406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60016/0 (socket says 172.21.15.53:60016) 2024-05-10T10:34:23.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e294db640 1 -- 172.21.15.53:0/2609681720 learned_addr learned my addr 172.21.15.53:0/2609681720 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:23.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219655519 0 0) 0x7f4e14001490 con 0x7f4e2406c2c0 2024-05-10T10:34:23.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e29cdc640 1 --2- 172.21.15.53:0/2609681720 >> v2:172.21.15.135:3300/0 conn(0x7f4e240b0940 0x7f4e24167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e0c003520 con 0x7f4e2406c2c0 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1438641413 0 0) 0x7f4e0c003520 con 0x7f4e2406c2c0 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 >> v2:172.21.15.135:3300/0 conn(0x7f4e240b0940 msgr2=0x7f4e24167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 --2- 172.21.15.53:0/2609681720 >> v2:172.21.15.135:3300/0 conn(0x7f4e240b0940 0x7f4e24167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 >> v1:172.21.15.135:6789/0 conn(0x7f4e24146750 legacy=0x7f4e24070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e24071410 con 0x7f4e2406c2c0 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4e14001bc0 con 0x7f4e2406c2c0 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 >> v1:172.21.15.53:6789/0 conn(0x7f4e2406c2c0 legacy=0x7f4e2406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e0c0039b0 0x7f4e0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.084+0000 7f4e23fff640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e0c0051c0 0x7f4e0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4e240af4a0 con 0x7f4e0c004800 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c00c060 con 0x7f4e0c0051c0 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0c00c260 con 0x7f4e0c0039b0 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e28cda640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e294db640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e0c0039b0 0x7f4e0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e29cdc640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e0c0051c0 0x7f4e0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e28cda640 1 -- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e0c0039b0 msgr2=0x7f4e0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:23.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e28cda640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e0c0039b0 0x7f4e0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e28cda640 1 -- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e0c0051c0 msgr2=0x7f4e0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:23.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e28cda640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e0c0051c0 0x7f4e0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e28cda640 1 -- 172.21.15.53:0/2609681720 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c003520 con 0x7f4e0c004800 2024-05-10T10:34:23.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e29cdc640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e0c0051c0 0x7f4e0c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:23.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.085+0000 7f4e29cdc640 1 -- 172.21.15.53:0/2609681720 reap_dead start 2024-05-10T10:34:23.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e28cda640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e0c004c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e18001750 tx=0x7f4e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a803df8e8ca0a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18001370 con 0x7f4e0c004800 2024-05-10T10:34:23.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e180052d0 con 0x7f4e0c004800 2024-05-10T10:34:23.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e23fff640 1 -- 172.21.15.53:0/2609681720 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1802abc0 con 0x7f4e0c004800 2024-05-10T10:34:23.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2609681720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 msgr2=0x7f4e0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e0c004c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e18001750 tx=0x7f4e0c005830 comp rx=0 tx=0).stop 2024-05-10T10:34:23.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2609681720 shutdown_connections 2024-05-10T10:34:23.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2609681720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e0c004c80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2609681720 >> 172.21.15.53:0/2609681720 conn(0x7f4e240d1d70 msgr2=0x7f4e240d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:23.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2609681720 shutdown_connections 2024-05-10T10:34:23.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.086+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2609681720 wait complete. 2024-05-10T10:34:23.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 Processor -- start 2024-05-10T10:34:23.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 -- start start 2024-05-10T10:34:23.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e241e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e2415c4b0 0x7f4e240cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e240cfde0 0x7f4e2409ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4e240b1530 con 0x7f4e0c004800 2024-05-10T10:34:23.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e240af130 con 0x7f4e240cfde0 2024-05-10T10:34:23.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e2b73f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4e240b16b0 con 0x7f4e2415c4b0 2024-05-10T10:34:23.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e294db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e241e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e294db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e241e58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44294/0 (socket says 172.21.15.53:44294) 2024-05-10T10:34:23.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.088+0000 7f4e294db640 1 -- 172.21.15.53:0/2911725503 learned_addr learned my addr 172.21.15.53:0/2911725503 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:23.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e28cda640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e2415c4b0 0x7f4e240cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e29cdc640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e240cfde0 0x7f4e2409ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e294db640 1 -- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e2415c4b0 msgr2=0x7f4e240cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e294db640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e2415c4b0 0x7f4e240cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e294db640 1 -- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e240cfde0 msgr2=0x7f4e2409ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e294db640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e240cfde0 0x7f4e2409ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e294db640 1 -- 172.21.15.53:0/2911725503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e241e5df0 con 0x7f4e0c004800 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e29cdc640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e240cfde0 0x7f4e2409ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e294db640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e241e58b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400ab40 tx=0x7f4e14007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e21ffb640 1 -- 172.21.15.53:0/2911725503 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14007aa0 con 0x7f4e0c004800 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e21ffb640 1 -- 172.21.15.53:0/2911725503 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e14018070 con 0x7f4e0c004800 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2409cee0 con 0x7f4e0c004800 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e21ffb640 1 -- 172.21.15.53:0/2911725503 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14013630 con 0x7f4e0c004800 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.089+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2409d4a0 con 0x7f4e0c004800 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.090+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e240693f0 con 0x7f4e0c004800 2024-05-10T10:34:23.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.091+0000 7f4e21ffb640 1 -- 172.21.15.53:0/2911725503 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4e140137d0 con 0x7f4e0c004800 2024-05-10T10:34:23.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.092+0000 7f4e21ffb640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4e100762d0 0x7f4e10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.092+0000 7f4e21ffb640 1 -- 172.21.15.53:0/2911725503 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6101+0+0 (secure 0 0 0) 0x7f4e14098090 con 0x7f4e0c004800 2024-05-10T10:34:23.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.092+0000 7f4e28cda640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4e100762d0 0x7f4e10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.092+0000 7f4e28cda640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4e100762d0 0x7f4e10078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e18032000 tx=0x7f4e1802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.096+0000 7f4e21ffb640 1 -- 172.21.15.53:0/2911725503 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4e24146be0 con 0x7f4e0c004800 2024-05-10T10:34:23.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.288+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e240d01c0 con 0x7f4e0c004800 2024-05-10T10:34:23.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.289+0000 7f4e21ffb640 1 -- 172.21.15.53:0/2911725503 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16955 (secure 0 0 0) 0x7f4e14061860 con 0x7f4e0c004800 2024-05-10T10:34:23.291 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:23.291 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":35,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:22.306371+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:20.374693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:21.350324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4e100762d0 msgr2=0x7f4e10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4e100762d0 0x7f4e10078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e18032000 tx=0x7f4e1802e000 comp rx=0 tx=0).stop 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 msgr2=0x7f4e241e58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e241e58b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400ab40 tx=0x7f4e14007060 comp rx=0 tx=0).stop 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 shutdown_connections 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4e100762d0 0x7f4e10078790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e240cfde0 0x7f4e2409ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4e2415c4b0 0x7f4e240cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 --2- 172.21.15.53:0/2911725503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e0c004800 0x7f4e241e58b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 >> 172.21.15.53:0/2911725503 conn(0x7f4e240d1d70 msgr2=0x7f4e240759c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:23.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 shutdown_connections 2024-05-10T10:34:23.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.291+0000 7f4e2b73f640 1 -- 172.21.15.53:0/2911725503 wait complete. 2024-05-10T10:34:23.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:23.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.406+0000 7f4a176e1640 1 Processor -- start 2024-05-10T10:34:23.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.406+0000 7f4a176e1640 1 -- start start 2024-05-10T10:34:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a176e1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4a100d59d0 0x7f4a100b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a176e1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a100d2d20 con 0x7f4a10162940 2024-05-10T10:34:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a176e1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a100d3000 con 0x7f4a100d3d80 2024-05-10T10:34:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a176e1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4a10162d20 con 0x7f4a100d59d0 2024-05-10T10:34:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a1547d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4a10162940 0x7f4a100d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60028/0 (socket says 172.21.15.53:60028) 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a1547d640 1 -- 172.21.15.53:0/2456219794 learned_addr learned my addr 172.21.15.53:0/2456219794 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a15c7e640 1 --2- 172.21.15.53:0/2456219794 >> v2:172.21.15.135:3300/0 conn(0x7f4a100d59d0 0x7f4a100b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645538408 0 0) 0x7f4a100d2d20 con 0x7f4a10162940 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.407+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49fc003520 con 0x7f4a10162940 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3698932603 0 0) 0x7f49fc003520 con 0x7f4a10162940 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 >> v2:172.21.15.135:3300/0 conn(0x7f4a100d59d0 msgr2=0x7f4a100b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 --2- 172.21.15.53:0/2456219794 >> v2:172.21.15.135:3300/0 conn(0x7f4a100d59d0 0x7f4a100b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 >> v1:172.21.15.135:6789/0 conn(0x7f4a100d3d80 legacy=0x7f4a100d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a100d32e0 con 0x7f4a10162940 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4a0c001670 con 0x7f4a10162940 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 >> v1:172.21.15.53:6789/0 conn(0x7f4a10162940 legacy=0x7f4a100d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc00c130 con 0x7f49fc004800 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc00c2e0 con 0x7f49fc0051c0 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc00c4e0 con 0x7f49fc0039b0 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a14c7c640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a1547d640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a15c7e640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a14c7c640 1 -- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0039b0 msgr2=0x7f49fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a14c7c640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc0039b0 0x7f49fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a14c7c640 1 -- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0051c0 msgr2=0x7f49fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a14c7c640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49fc0051c0 0x7f49fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a14c7c640 1 -- 172.21.15.53:0/2456219794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc003520 con 0x7f49fc004800 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.408+0000 7f4a15c7e640 1 -- 172.21.15.53:0/2456219794 reap_dead start 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.409+0000 7f4a14c7c640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001720 tx=0x7f49fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db13a44ee0c55d0b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.409+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00001370 con 0x7f49fc004800 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.409+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a000052d0 con 0x7f49fc004800 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.409+0000 7f4a07fff640 1 -- 172.21.15.53:0/2456219794 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0002ab90 con 0x7f49fc004800 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.410+0000 7f4a176e1640 1 -- 172.21.15.53:0/2456219794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 msgr2=0x7f49fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.410+0000 7f4a176e1640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001720 tx=0x7f49fc005830 comp rx=0 tx=0).stop 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.410+0000 7f4a176e1640 1 -- 172.21.15.53:0/2456219794 shutdown_connections 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.410+0000 7f4a176e1640 1 --2- 172.21.15.53:0/2456219794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49fc004800 0x7f49fc004c80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.410+0000 7f4a176e1640 1 -- 172.21.15.53:0/2456219794 >> 172.21.15.53:0/2456219794 conn(0x7f4a1009e380 msgr2=0x7f4a100d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.410+0000 7f4a176e1640 1 -- 172.21.15.53:0/2456219794 shutdown_connections 2024-05-10T10:34:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.410+0000 7f4a176e1640 1 -- 172.21.15.53:0/2456219794 wait complete. 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.411+0000 7f4a176e1640 1 Processor -- start 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.411+0000 7f4a176e1640 1 -- start start 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a176e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc004800 0x7f4a101520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a176e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a1014e0b0 0x7f4a100ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a176e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a101f8530 0x7f4a101f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a176e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100d2390 con 0x7f4a1014e0b0 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a176e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100cfc50 con 0x7f4a101f8530 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a176e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4a100d2510 con 0x7f49fc004800 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a1014e0b0 0x7f4a100ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a1014e0b0 0x7f4a100ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44318/0 (socket says 172.21.15.53:44318) 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 -- 172.21.15.53:0/3302724384 learned_addr learned my addr 172.21.15.53:0/3302724384 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a15c7e640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a101f8530 0x7f4a101f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a1547d640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc004800 0x7f4a101520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 -- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc004800 msgr2=0x7f4a101520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc004800 0x7f4a101520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 -- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a101f8530 msgr2=0x7f4a101f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a101f8530 0x7f4a101f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 -- 172.21.15.53:0/3302724384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a100cf0e0 con 0x7f4a1014e0b0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.412+0000 7f4a14c7c640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a1014e0b0 0x7f4a100ceba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.413+0000 7f4a05ffb640 1 -- 172.21.15.53:0/3302724384 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00040030 con 0x7f4a1014e0b0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.413+0000 7f4a05ffb640 1 -- 172.21.15.53:0/3302724384 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a00005480 con 0x7f4a1014e0b0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.413+0000 7f4a05ffb640 1 -- 172.21.15.53:0/3302724384 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00001370 con 0x7f4a1014e0b0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.413+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a100b8ce0 con 0x7f4a1014e0b0 2024-05-10T10:34:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.413+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1014e8e0 con 0x7f4a1014e0b0 2024-05-10T10:34:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.414+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1014bd40 con 0x7f4a1014e0b0 2024-05-10T10:34:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.415+0000 7f4a05ffb640 1 -- 172.21.15.53:0/3302724384 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a00043070 con 0x7f4a1014e0b0 2024-05-10T10:34:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.415+0000 7f4a05ffb640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40762d0 0x7f49f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.415+0000 7f4a05ffb640 1 -- 172.21.15.53:0/3302724384 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6101+0+0 (secure 0 0 0) 0x7f4a000b6e70 con 0x7f4a1014e0b0 2024-05-10T10:34:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.415+0000 7f4a1547d640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40762d0 0x7f49f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.416+0000 7f4a1547d640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40762d0 0x7f49f4078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c001cf0 tx=0x7f4a0c001b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.419+0000 7f4a05ffb640 1 -- 172.21.15.53:0/3302724384 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4a1014bd40 con 0x7f4a1014e0b0 2024-05-10T10:34:23.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.610+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a1006a220 con 0x7f4a1014e0b0 2024-05-10T10:34:23.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.611+0000 7f4a05ffb640 1 -- 172.21.15.53:0/3302724384 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16955 (secure 0 0 0) 0x7f4a0004b020 con 0x7f4a1014e0b0 2024-05-10T10:34:23.613 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:23.613 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":35,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:22.306371+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:20.374693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:21.350324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:23.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40762d0 msgr2=0x7f49f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40762d0 0x7f49f4078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c001cf0 tx=0x7f4a0c001b80 comp rx=0 tx=0).stop 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a1014e0b0 msgr2=0x7f4a100ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a1014e0b0 0x7f4a100ceba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00001120 comp rx=0 tx=0).stop 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 shutdown_connections 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f40762d0 0x7f49f4078790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a101f8530 0x7f4a101f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a1014e0b0 0x7f4a100ceba0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 --2- 172.21.15.53:0/3302724384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49fc004800 0x7f4a101520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 >> 172.21.15.53:0/3302724384 conn(0x7f4a1009e380 msgr2=0x7f4a1006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 shutdown_connections 2024-05-10T10:34:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.614+0000 7f4a176e1640 1 -- 172.21.15.53:0/3302724384 wait complete. 2024-05-10T10:34:23.627 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:34:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.732+0000 7f5206664640 1 Processor -- start 2024-05-10T10:34:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.733+0000 7f5206664640 1 -- start start 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.733+0000 7f5206664640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f520015f400 0x7f520006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.733+0000 7f5206664640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f52000d17f0 0x7f52000b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.733+0000 7f5206664640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52000d1c40 con 0x7f520006e290 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.733+0000 7f5206664640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f520006e6f0 con 0x7f520015f400 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.733+0000 7f5206664640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f52000d1ed0 con 0x7f52000d17f0 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51fffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f520015f400 0x7f520006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51f7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f520006e290 0x7f52000d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60042/0 (socket says 172.21.15.53:60042) 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51fffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f520015f400 0x7f520006dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44324/0 (socket says 172.21.15.53:44324) 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51f7fff640 1 -- 172.21.15.53:0/1697600345 learned_addr learned my addr 172.21.15.53:0/1697600345 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f5204c01640 1 --2- 172.21.15.53:0/1697600345 >> v2:172.21.15.135:3300/0 conn(0x7f52000d17f0 0x7f52000b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51fffff640 1 -- 172.21.15.53:0/1697600345 >> v2:172.21.15.135:3300/0 conn(0x7f52000d17f0 msgr2=0x7f52000b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:23.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51fffff640 1 --2- 172.21.15.53:0/1697600345 >> v2:172.21.15.135:3300/0 conn(0x7f52000d17f0 0x7f52000b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51fffff640 1 -- 172.21.15.53:0/1697600345 >> v1:172.21.15.53:6789/0 conn(0x7f520006e290 legacy=0x7f52000d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51ff7fe640 1 -- 172.21.15.53:0/1697600345 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3561198442 0 0) 0x7f52000d1c40 con 0x7f520006e290 2024-05-10T10:34:23.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.734+0000 7f51fffff640 1 -- 172.21.15.53:0/1697600345 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52000d21b0 con 0x7f520015f400 2024-05-10T10:34:23.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.735+0000 7f51fffff640 1 --2- 172.21.15.53:0/1697600345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f520006dd50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52000d0b70 tx=0x7f51f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c144fbc2aac8a475 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.735+0000 7f51ff7fe640 1 -- 172.21.15.53:0/1697600345 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0037070 con 0x7f520015f400 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.735+0000 7f51ff7fe640 1 -- 172.21.15.53:0/1697600345 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0032440 con 0x7f520015f400 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.735+0000 7f51ff7fe640 1 -- 172.21.15.53:0/1697600345 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0032810 con 0x7f520015f400 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.735+0000 7f5206664640 1 -- 172.21.15.53:0/1697600345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 msgr2=0x7f520006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.736+0000 7f5206664640 1 --2- 172.21.15.53:0/1697600345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f520006dd50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52000d0b70 tx=0x7f51f002a1a0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.736+0000 7f5206664640 1 -- 172.21.15.53:0/1697600345 shutdown_connections 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.736+0000 7f5206664640 1 --2- 172.21.15.53:0/1697600345 >> v2:172.21.15.135:3300/0 conn(0x7f52000d17f0 0x7f52000b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.736+0000 7f5206664640 1 --2- 172.21.15.53:0/1697600345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f520006dd50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.736+0000 7f5206664640 1 -- 172.21.15.53:0/1697600345 >> 172.21.15.53:0/1697600345 conn(0x7f520009ba20 msgr2=0x7f520009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.736+0000 7f5206664640 1 -- 172.21.15.53:0/1697600345 shutdown_connections 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.736+0000 7f5206664640 1 -- 172.21.15.53:0/1697600345 wait complete. 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.737+0000 7f5206664640 1 Processor -- start 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.737+0000 7f5206664640 1 -- start start 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5206664640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520006e290 0x7f520015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5206664640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52000d17f0 0x7f5200158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5206664640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f5200158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5206664640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f520015f7e0 con 0x7f520015f400 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5206664640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52000d2770 con 0x7f520006e290 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5206664640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f520015b0c0 con 0x7f52000d17f0 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f5200158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f5200158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44336/0 (socket says 172.21.15.53:44336) 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 -- 172.21.15.53:0/2675030367 learned_addr learned my addr 172.21.15.53:0/2675030367 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f51fffff640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520006e290 0x7f520015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f51f7fff640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52000d17f0 0x7f5200158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 -- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52000d17f0 msgr2=0x7f5200158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52000d17f0 0x7f5200158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 -- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520006e290 msgr2=0x7f520015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520006e290 0x7f520015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 -- 172.21.15.53:0/2675030367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520015b3a0 con 0x7f520015f400 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f5204c01640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f5200158c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51f800efc0 tx=0x7f51f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f51fd7fa640 1 -- 172.21.15.53:0/2675030367 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8018020 con 0x7f520015f400 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.738+0000 7f51fd7fa640 1 -- 172.21.15.53:0/2675030367 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8009b10 con 0x7f520015f400 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.739+0000 7f51fd7fa640 1 -- 172.21.15.53:0/2675030367 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80146c0 con 0x7f520015f400 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.739+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520015b680 con 0x7f520015f400 2024-05-10T10:34:23.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.739+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52000b3580 con 0x7f520015f400 2024-05-10T10:34:23.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.740+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52000693f0 con 0x7f520015f400 2024-05-10T10:34:23.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.740+0000 7f51fd7fa640 1 -- 172.21.15.53:0/2675030367 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f51f8005d10 con 0x7f520015f400 2024-05-10T10:34:23.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.741+0000 7f51fd7fa640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51e0076300 0x7f51e00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:23.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.741+0000 7f51fd7fa640 1 -- 172.21.15.53:0/2675030367 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6101+0+0 (secure 0 0 0) 0x7f51f8098370 con 0x7f520015f400 2024-05-10T10:34:23.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.741+0000 7f51fffff640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51e0076300 0x7f51e00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:23.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.741+0000 7f51fffff640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51e0076300 0x7f51e00787c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f0034000 tx=0x7f51f002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:23.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.745+0000 7f51fd7fa640 1 -- 172.21.15.53:0/2675030367 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f51f8022030 con 0x7f520015f400 2024-05-10T10:34:23.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:23.975+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5200162890 con 0x7f520015f400 2024-05-10T10:34:24.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.953+0000 7f51fd7fa640 1 -- 172.21.15.53:0/2675030367 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f51f8010040 con 0x7f520015f400 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51e0076300 msgr2=0x7f51e00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51e0076300 0x7f51e00787c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f0034000 tx=0x7f51f002e000 comp rx=0 tx=0).stop 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 msgr2=0x7f5200158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f5200158c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51f800efc0 tx=0x7f51f800c490 comp rx=0 tx=0).stop 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 shutdown_connections 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51e0076300 0x7f51e00787c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f520015f400 0x7f5200158c80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52000d17f0 0x7f5200158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:24.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 --2- 172.21.15.53:0/2675030367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f520006e290 0x7f520015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 >> 172.21.15.53:0/2675030367 conn(0x7f520009ba20 msgr2=0x7f52000710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 shutdown_connections 2024-05-10T10:34:24.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:24.956+0000 7f5206664640 1 -- 172.21.15.53:0/2675030367 wait complete. 2024-05-10T10:34:24.971 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T10:34:24.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:34:25.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.083+0000 7f130df9d640 1 Processor -- start 2024-05-10T10:34:25.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.083+0000 7f130df9d640 1 -- start start 2024-05-10T10:34:25.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f130df9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13080d3df0 0x7f13080d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:25.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f130df9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f13080b4aa0 0x7f13080d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:25.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f130df9d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13080d41d0 con 0x7f13080b4f30 2024-05-10T10:34:25.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f130df9d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f13080cfd50 con 0x7f13080b4aa0 2024-05-10T10:34:25.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f130df9d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f13080b5d90 con 0x7f13080d3df0 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f1306ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f13080b4aa0 0x7f13080d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f1306ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f13080b4aa0 0x7f13080d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44348/0 (socket says 172.21.15.53:44348) 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f1306ffd640 1 -- 172.21.15.53:0/2188553990 learned_addr learned my addr 172.21.15.53:0/2188553990 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f13077fe640 1 --2- 172.21.15.53:0/2188553990 >> v2:172.21.15.135:3300/0 conn(0x7f13080d3df0 0x7f13080d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f1306ffd640 1 -- 172.21.15.53:0/2188553990 >> v2:172.21.15.135:3300/0 conn(0x7f13080d3df0 msgr2=0x7f13080d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f1306ffd640 1 --2- 172.21.15.53:0/2188553990 >> v2:172.21.15.135:3300/0 conn(0x7f13080d3df0 0x7f13080d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f1306ffd640 1 -- 172.21.15.53:0/2188553990 >> v1:172.21.15.135:6789/0 conn(0x7f13080b4f30 legacy=0x7f13080d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.084+0000 7f1306ffd640 1 -- 172.21.15.53:0/2188553990 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13080b54f0 con 0x7f13080b4aa0 2024-05-10T10:34:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.085+0000 7f1306ffd640 1 --2- 172.21.15.53:0/2188553990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080d1a10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f130806cbe0 tx=0x7f12fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcbb31e6d8bcb2d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:25.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.085+0000 7f13067fc640 1 -- 172.21.15.53:0/2188553990 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc037070 con 0x7f13080b4aa0 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.085+0000 7f13067fc640 1 -- 172.21.15.53:0/2188553990 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12fc032440 con 0x7f13080b4aa0 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.085+0000 7f13067fc640 1 -- 172.21.15.53:0/2188553990 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc032810 con 0x7f13080b4aa0 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 -- 172.21.15.53:0/2188553990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 msgr2=0x7f13080d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 --2- 172.21.15.53:0/2188553990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080d1a10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f130806cbe0 tx=0x7f12fc02a230 comp rx=0 tx=0).stop 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 -- 172.21.15.53:0/2188553990 shutdown_connections 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 --2- 172.21.15.53:0/2188553990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080d1a10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 --2- 172.21.15.53:0/2188553990 >> v2:172.21.15.135:3300/0 conn(0x7f13080d3df0 0x7f13080d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 -- 172.21.15.53:0/2188553990 >> 172.21.15.53:0/2188553990 conn(0x7f130809e450 msgr2=0x7f13080d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 -- 172.21.15.53:0/2188553990 shutdown_connections 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.086+0000 7f130df9d640 1 -- 172.21.15.53:0/2188553990 wait complete. 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.087+0000 7f130df9d640 1 Processor -- start 2024-05-10T10:34:25.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.087+0000 7f130df9d640 1 -- start start 2024-05-10T10:34:25.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f130df9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f130df9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13080b4f30 0x7f130815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f130df9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13080d3df0 0x7f130815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f130df9d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13080cff80 con 0x7f13080b4aa0 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f130df9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13080b5ab0 con 0x7f13080b4f30 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f130df9d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f13080b6120 con 0x7f13080d3df0 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44350/0 (socket says 172.21.15.53:44350) 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 -- 172.21.15.53:0/3148078470 learned_addr learned my addr 172.21.15.53:0/3148078470 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f1306ffd640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13080b4f30 0x7f130815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f1307fff640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13080d3df0 0x7f130815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 -- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13080d3df0 msgr2=0x7f130815f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13080d3df0 0x7f130815f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 -- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13080b4f30 msgr2=0x7f130815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13080b4f30 0x7f130815bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f13077fe640 1 -- 172.21.15.53:0/3148078470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13080cd600 con 0x7f13080b4aa0 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.088+0000 7f1307fff640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13080d3df0 0x7f130815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.089+0000 7f13077fe640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080ce6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d5f0 tx=0x7f12f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.089+0000 7f12effff640 1 -- 172.21.15.53:0/3148078470 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80125f0 con 0x7f13080b4aa0 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.089+0000 7f12effff640 1 -- 172.21.15.53:0/3148078470 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f8019070 con 0x7f13080b4aa0 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.089+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13080cd8e0 con 0x7f13080b4aa0 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.089+0000 7f12effff640 1 -- 172.21.15.53:0/3148078470 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80129d0 con 0x7f13080b4aa0 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.089+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13080cdea0 con 0x7f13080b4aa0 2024-05-10T10:34:25.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.090+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130806a220 con 0x7f13080b4aa0 2024-05-10T10:34:25.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.091+0000 7f12effff640 1 -- 172.21.15.53:0/3148078470 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f12f8012b70 con 0x7f13080b4aa0 2024-05-10T10:34:25.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.091+0000 7f12effff640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f12f4076300 0x7f12f40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:25.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.091+0000 7f12effff640 1 -- 172.21.15.53:0/3148078470 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f12f8097cc0 con 0x7f13080b4aa0 2024-05-10T10:34:25.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.091+0000 7f1306ffd640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f12f4076300 0x7f12f40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:25.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.092+0000 7f1306ffd640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f12f4076300 0x7f12f40787c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12fc034000 tx=0x7f12fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:25.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.095+0000 7f12effff640 1 -- 172.21.15.53:0/3148078470 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f12f8061400 con 0x7f13080b4aa0 2024-05-10T10:34:25.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.328+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f13080d27c0 con 0x7f13080b4aa0 2024-05-10T10:34:25.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.954+0000 7f12effff640 1 -- 172.21.15.53:0/3148078470 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f12f8060d00 con 0x7f13080b4aa0 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f12f4076300 msgr2=0x7f12f40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f12f4076300 0x7f12f40787c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12fc034000 tx=0x7f12fc02e000 comp rx=0 tx=0).stop 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 msgr2=0x7f13080ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080ce6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d5f0 tx=0x7f12f800dab0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 shutdown_connections 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f12f4076300 0x7f12f40787c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13080d3df0 0x7f130815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13080b4f30 0x7f130815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 --2- 172.21.15.53:0/3148078470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13080b4aa0 0x7f13080ce6a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:25.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 >> 172.21.15.53:0/3148078470 conn(0x7f130809e450 msgr2=0x7f13080d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:25.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 shutdown_connections 2024-05-10T10:34:25.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:25.957+0000 7f130df9d640 1 -- 172.21.15.53:0/3148078470 wait complete. 2024-05-10T10:34:25.972 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:34:25.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:26.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.081+0000 7f3184e85640 1 Processor -- start 2024-05-10T10:34:26.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.081+0000 7f3184e85640 1 -- start start 2024-05-10T10:34:26.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f3184e85640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31801598c0 0x7f318006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f3184e85640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f318006ec40 0x7f31800c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f3184e85640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318006f1c0 con 0x7f31800c71c0 2024-05-10T10:34:26.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f3184e85640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f31800c7610 con 0x7f31801598c0 2024-05-10T10:34:26.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f3184e85640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f318006f4a0 con 0x7f318006ec40 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f317e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31801598c0 0x7f318006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f317e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31801598c0 0x7f318006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44356/0 (socket says 172.21.15.53:44356) 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f317e59c640 1 -- 172.21.15.53:0/572232566 learned_addr learned my addr 172.21.15.53:0/572232566 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f3177fff640 1 --2- 172.21.15.53:0/572232566 >> v2:172.21.15.135:3300/0 conn(0x7f318006ec40 0x7f31800c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f317e59c640 1 -- 172.21.15.53:0/572232566 >> v2:172.21.15.135:3300/0 conn(0x7f318006ec40 msgr2=0x7f31800c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f317e59c640 1 --2- 172.21.15.53:0/572232566 >> v2:172.21.15.135:3300/0 conn(0x7f318006ec40 0x7f31800c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f317e59c640 1 -- 172.21.15.53:0/572232566 >> v1:172.21.15.135:6789/0 conn(0x7f31800c71c0 legacy=0x7f318015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.082+0000 7f317e59c640 1 -- 172.21.15.53:0/572232566 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318006f780 con 0x7f31801598c0 2024-05-10T10:34:26.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.083+0000 7f317e59c640 1 --2- 172.21.15.53:0/572232566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318006e700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3180070400 tx=0x7f317002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9921bcaf07508034 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.083+0000 7f317dd9b640 1 -- 172.21.15.53:0/572232566 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170036020 con 0x7f31801598c0 2024-05-10T10:34:26.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.083+0000 7f317dd9b640 1 -- 172.21.15.53:0/572232566 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3170032640 con 0x7f31801598c0 2024-05-10T10:34:26.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.083+0000 7f317dd9b640 1 -- 172.21.15.53:0/572232566 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170032a10 con 0x7f31801598c0 2024-05-10T10:34:26.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 -- 172.21.15.53:0/572232566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 msgr2=0x7f318006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 --2- 172.21.15.53:0/572232566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318006e700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3180070400 tx=0x7f317002c940 comp rx=0 tx=0).stop 2024-05-10T10:34:26.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 -- 172.21.15.53:0/572232566 shutdown_connections 2024-05-10T10:34:26.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 --2- 172.21.15.53:0/572232566 >> v2:172.21.15.135:3300/0 conn(0x7f318006ec40 0x7f31800c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 --2- 172.21.15.53:0/572232566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318006e700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 -- 172.21.15.53:0/572232566 >> 172.21.15.53:0/572232566 conn(0x7f31800b2900 msgr2=0x7f31800b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:26.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 -- 172.21.15.53:0/572232566 shutdown_connections 2024-05-10T10:34:26.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.084+0000 7f3184e85640 1 -- 172.21.15.53:0/572232566 wait complete. 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.085+0000 7f3184e85640 1 Processor -- start 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.085+0000 7f3184e85640 1 -- start start 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f3184e85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f318006ec40 0x7f318015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f3184e85640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31800c71c0 0x7f31800cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f3184e85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f3184e85640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f318015c6e0 con 0x7f31801598c0 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f3184e85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31800ce920 con 0x7f318006ec40 2024-05-10T10:34:26.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f3184e85640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3180159ca0 con 0x7f31800c71c0 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f317ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f317ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318014f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44362/0 (socket says 172.21.15.53:44362) 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.086+0000 7f317ed9d640 1 -- 172.21.15.53:0/1818633891 learned_addr learned my addr 172.21.15.53:0/1818633891 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f3177fff640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31800c71c0 0x7f31800cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f317e59c640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f318006ec40 0x7f318015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f317ed9d640 1 -- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31800c71c0 msgr2=0x7f31800cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f317ed9d640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31800c71c0 0x7f31800cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f317ed9d640 1 -- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f318006ec40 msgr2=0x7f318015aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f317ed9d640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f318006ec40 0x7f318015aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f317ed9d640 1 -- 172.21.15.53:0/1818633891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318015b140 con 0x7f31801598c0 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f317ed9d640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318014f8b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f317800c7a0 tx=0x7f317800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f3176ffd640 1 -- 172.21.15.53:0/1818633891 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178003b70 con 0x7f31801598c0 2024-05-10T10:34:26.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f3176ffd640 1 -- 172.21.15.53:0/1818633891 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3178003d30 con 0x7f31801598c0 2024-05-10T10:34:26.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f3176ffd640 1 -- 172.21.15.53:0/1818633891 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317800aa90 con 0x7f31801598c0 2024-05-10T10:34:26.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318015b420 con 0x7f31801598c0 2024-05-10T10:34:26.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.087+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3180072430 con 0x7f31801598c0 2024-05-10T10:34:26.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.088+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318015b940 con 0x7f31801598c0 2024-05-10T10:34:26.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.089+0000 7f3176ffd640 1 -- 172.21.15.53:0/1818633891 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f317800ac30 con 0x7f31801598c0 2024-05-10T10:34:26.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.089+0000 7f3176ffd640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31600762d0 0x7f3160078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.089+0000 7f3176ffd640 1 -- 172.21.15.53:0/1818633891 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f31780978e0 con 0x7f31801598c0 2024-05-10T10:34:26.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.089+0000 7f317e59c640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31600762d0 0x7f3160078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.090+0000 7f317e59c640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31600762d0 0x7f3160078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3170037000 tx=0x7f317003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.094+0000 7f3176ffd640 1 -- 172.21.15.53:0/1818633891 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f318015b940 con 0x7f31801598c0 2024-05-10T10:34:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.326+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f318006a220 con 0x7f31801598c0 2024-05-10T10:34:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.326+0000 7f3176ffd640 1 -- 172.21.15.53:0/1818633891 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7f3178060f00 con 0x7f31801598c0 2024-05-10T10:34:26.328 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:26.328 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:26.328 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.328+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31600762d0 msgr2=0x7f3160078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.328+0000 7f3184e85640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31600762d0 0x7f3160078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3170037000 tx=0x7f317003d040 comp rx=0 tx=0).stop 2024-05-10T10:34:26.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.328+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 msgr2=0x7f318014f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.328+0000 7f3184e85640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318014f8b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f317800c7a0 tx=0x7f317800cc60 comp rx=0 tx=0).stop 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 shutdown_connections 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31600762d0 0x7f3160078790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31801598c0 0x7f318014f8b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31800c71c0 0x7f31800cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 --2- 172.21.15.53:0/1818633891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f318006ec40 0x7f318015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 >> 172.21.15.53:0/1818633891 conn(0x7f31800b2900 msgr2=0x7f318015f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 shutdown_connections 2024-05-10T10:34:26.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.329+0000 7f3184e85640 1 -- 172.21.15.53:0/1818633891 wait complete. 2024-05-10T10:34:26.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.444+0000 7fb66edaf640 1 Processor -- start 2024-05-10T10:34:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.444+0000 7fb66edaf640 1 -- start start 2024-05-10T10:34:26.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.444+0000 7fb66edaf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb66815f3b0 0x7fb6680d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.444+0000 7fb66edaf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6680d30b0 con 0x7fb6680d5d30 2024-05-10T10:34:26.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.444+0000 7fb66edaf640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6680d3390 con 0x7fb66815f790 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.444+0000 7fb66edaf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb6680c8c80 con 0x7fb66815f3b0 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb66d34c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb6680d5d30 0x7fb6680d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60050/0 (socket says 172.21.15.53:60050) 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb66d34c640 1 -- 172.21.15.53:0/2768031953 learned_addr learned my addr 172.21.15.53:0/2768031953 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/2768031953 >> v2:172.21.15.135:3300/0 conn(0x7fb66815f3b0 0x7fb6680d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980584741 0 0) 0x7fb6680d30b0 con 0x7fb6680d5d30 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb650003520 con 0x7fb6680d5d30 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 893810411 0 0) 0x7fb650003520 con 0x7fb6680d5d30 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 >> v2:172.21.15.135:3300/0 conn(0x7fb66815f3b0 msgr2=0x7fb6680d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 --2- 172.21.15.53:0/2768031953 >> v2:172.21.15.135:3300/0 conn(0x7fb66815f3b0 0x7fb6680d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 >> v1:172.21.15.135:6789/0 conn(0x7fb66815f790 legacy=0x7fb6680c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6680c8f60 con 0x7fb6680d5d30 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/2768031953 >> v2:172.21.15.135:3300/0 conn(0x7fb66815f3b0 0x7fb6680d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fb6600019f0 con 0x7fb6680d5d30 2024-05-10T10:34:26.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 >> v1:172.21.15.53:6789/0 conn(0x7fb6680d5d30 legacy=0x7fb6680d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb650003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb650004800 0x7fb650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6500051c0 0x7fb65000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680c8c80 con 0x7fb6500039b0 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb65000c110 con 0x7fb6500051c0 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb65000c310 con 0x7fb650004800 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb650003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb667fff640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb650004800 0x7fb650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb66d34c640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6500051c0 0x7fb65000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.445+0000 7fb66cb4b640 1 -- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb650004800 msgr2=0x7fb650004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb650004800 0x7fb650004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb66cb4b640 1 -- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6500051c0 msgr2=0x7fb65000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:26.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6500051c0 0x7fb65000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb66cb4b640 1 -- 172.21.15.53:0/2768031953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb650003520 con 0x7fb6500039b0 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb66d34c640 1 -- 172.21.15.53:0/2768031953 reap_dead start 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb667fff640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb650004800 0x7fb650004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb650003dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb6580098a0 tx=0x7fb650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e311a7adac6ec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65800f040 con 0x7fb6500039b0 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65802ace0 con 0x7fb6500039b0 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.446+0000 7fb6677fe640 1 -- 172.21.15.53:0/2768031953 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658004250 con 0x7fb6500039b0 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.447+0000 7fb66edaf640 1 -- 172.21.15.53:0/2768031953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 msgr2=0x7fb650003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.447+0000 7fb66edaf640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb650003dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb6580098a0 tx=0x7fb650005830 comp rx=0 tx=0).stop 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.447+0000 7fb66edaf640 1 -- 172.21.15.53:0/2768031953 shutdown_connections 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.447+0000 7fb66edaf640 1 --2- 172.21.15.53:0/2768031953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb650003dd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.447+0000 7fb66edaf640 1 -- 172.21.15.53:0/2768031953 >> 172.21.15.53:0/2768031953 conn(0x7fb6680d5130 msgr2=0x7fb6680d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.447+0000 7fb66edaf640 1 -- 172.21.15.53:0/2768031953 shutdown_connections 2024-05-10T10:34:26.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.447+0000 7fb66edaf640 1 -- 172.21.15.53:0/2768031953 wait complete. 2024-05-10T10:34:26.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.448+0000 7fb66edaf640 1 Processor -- start 2024-05-10T10:34:26.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.448+0000 7fb66edaf640 1 -- start start 2024-05-10T10:34:26.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66edaf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb6681475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66edaf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6680cc200 0x7fb6680cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66edaf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb668153cb0 0x7fb668154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66edaf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680d3870 con 0x7fb6500039b0 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66edaf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680d36f0 con 0x7fb668153cb0 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66edaf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680d39f0 con 0x7fb6680cc200 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb6681475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb6681475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44376/0 (socket says 172.21.15.53:44376) 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 -- 172.21.15.53:0/914604925 learned_addr learned my addr 172.21.15.53:0/914604925 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb667fff640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6680cc200 0x7fb6680cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66d34c640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb668153cb0 0x7fb668154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 -- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6680cc200 msgr2=0x7fb6680cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6680cc200 0x7fb6680cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 -- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb668153cb0 msgr2=0x7fb668154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb668153cb0 0x7fb668154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb66cb4b640 1 -- 172.21.15.53:0/914604925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6680b9af0 con 0x7fb6500039b0 2024-05-10T10:34:26.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.449+0000 7fb667fff640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6680cc200 0x7fb6680cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.450+0000 7fb66d34c640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb668153cb0 0x7fb668154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.450+0000 7fb66cb4b640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb6681475e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb6580099c0 tx=0x7fb658004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.450+0000 7fb6657fa640 1 -- 172.21.15.53:0/914604925 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658041070 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.450+0000 7fb6657fa640 1 -- 172.21.15.53:0/914604925 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65802aae0 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.450+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6680b9dd0 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.450+0000 7fb6657fa640 1 -- 172.21.15.53:0/914604925 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65803d430 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.450+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6680ba390 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.451+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66806a220 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.452+0000 7fb6657fa640 1 -- 172.21.15.53:0/914604925 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb658044070 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.452+0000 7fb6657fa640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6540762d0 0x7fb654078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.452+0000 7fb6657fa640 1 -- 172.21.15.53:0/914604925 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fb6580b7e70 con 0x7fb6500039b0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.452+0000 7fb667fff640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6540762d0 0x7fb654078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.453+0000 7fb667fff640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6540762d0 0x7fb654078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb65c007fd0 tx=0x7fb65c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.457+0000 7fb6657fa640 1 -- 172.21.15.53:0/914604925 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb65802acb0 con 0x7fb6500039b0 2024-05-10T10:34:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.647+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb668154670 con 0x7fb6500039b0 2024-05-10T10:34:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.648+0000 7fb6657fa640 1 -- 172.21.15.53:0/914604925 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16942 (secure 0 0 0) 0x7fb658081640 con 0x7fb6500039b0 2024-05-10T10:34:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:26.650 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:23.943083+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:20.374693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:21.350324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6540762d0 msgr2=0x7fb654078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6540762d0 0x7fb654078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb65c007fd0 tx=0x7fb65c009040 comp rx=0 tx=0).stop 2024-05-10T10:34:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 msgr2=0x7fb6681475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb6681475e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb6580099c0 tx=0x7fb658004730 comp rx=0 tx=0).stop 2024-05-10T10:34:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 shutdown_connections 2024-05-10T10:34:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6540762d0 0x7fb654078790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb668153cb0 0x7fb668154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6680cc200 0x7fb6680cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 --2- 172.21.15.53:0/914604925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6500039b0 0x7fb6681475e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 >> 172.21.15.53:0/914604925 conn(0x7fb6680d5130 msgr2=0x7fb6680d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 shutdown_connections 2024-05-10T10:34:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.650+0000 7fb66edaf640 1 -- 172.21.15.53:0/914604925 wait complete. 2024-05-10T10:34:26.663 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:34:26.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:34:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.766+0000 7f3ae5151640 1 Processor -- start 2024-05-10T10:34:26.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.766+0000 7f3ae5151640 1 -- start start 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3ae5151640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3ae00d5d30 0x7f3ae016a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3ae5151640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae00d30b0 con 0x7f3ae015f3b0 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3ae5151640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae00d3390 con 0x7f3ae015f790 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3ae5151640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae00c8c80 con 0x7f3ae00d5d30 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3aded9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3ae015f3b0 0x7f3ae00d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60056/0 (socket says 172.21.15.53:60056) 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3aded9d640 1 -- 172.21.15.53:0/4277690002 learned_addr learned my addr 172.21.15.53:0/4277690002 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3adf59e640 1 --2- 172.21.15.53:0/4277690002 >> v2:172.21.15.135:3300/0 conn(0x7f3ae00d5d30 0x7f3ae016a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3addd9b640 1 -- 172.21.15.53:0/4277690002 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527639023 0 0) 0x7f3ae00d30b0 con 0x7f3ae015f3b0 2024-05-10T10:34:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3addd9b640 1 -- 172.21.15.53:0/4277690002 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac8003520 con 0x7f3ae015f3b0 2024-05-10T10:34:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3addd9b640 1 -- 172.21.15.53:0/4277690002 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2171769221 0 0) 0x7f3ac8003520 con 0x7f3ae015f3b0 2024-05-10T10:34:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3adf59e640 1 -- 172.21.15.53:0/4277690002 >> v1:172.21.15.135:6789/0 conn(0x7f3ae015f790 legacy=0x7f3ae00cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3adf59e640 1 -- 172.21.15.53:0/4277690002 >> v1:172.21.15.53:6789/0 conn(0x7f3ae015f3b0 legacy=0x7f3ae00d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.767+0000 7f3adf59e640 1 -- 172.21.15.53:0/4277690002 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae00c8f60 con 0x7f3ae00d5d30 2024-05-10T10:34:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.768+0000 7f3adf59e640 1 --2- 172.21.15.53:0/4277690002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae00d5d30 0x7f3ae016a0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009d90 tx=0x7f3ad802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7820c3cbfdabc06f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.768+0000 7f3addd9b640 1 -- 172.21.15.53:0/4277690002 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8035020 con 0x7f3ae00d5d30 2024-05-10T10:34:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.768+0000 7f3addd9b640 1 -- 172.21.15.53:0/4277690002 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad8007540 con 0x7f3ae00d5d30 2024-05-10T10:34:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.768+0000 7f3addd9b640 1 -- 172.21.15.53:0/4277690002 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0001670 con 0x7f3ae00d5d30 2024-05-10T10:34:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.769+0000 7f3ae5151640 1 -- 172.21.15.53:0/4277690002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae00d5d30 msgr2=0x7f3ae016a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.769+0000 7f3ae5151640 1 --2- 172.21.15.53:0/4277690002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae00d5d30 0x7f3ae016a0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009d90 tx=0x7f3ad802a5b0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.769+0000 7f3ae5151640 1 -- 172.21.15.53:0/4277690002 shutdown_connections 2024-05-10T10:34:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.769+0000 7f3ae5151640 1 --2- 172.21.15.53:0/4277690002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae00d5d30 0x7f3ae016a0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.769+0000 7f3ae5151640 1 -- 172.21.15.53:0/4277690002 >> 172.21.15.53:0/4277690002 conn(0x7f3ae00d5130 msgr2=0x7f3ae00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.769+0000 7f3ae5151640 1 -- 172.21.15.53:0/4277690002 shutdown_connections 2024-05-10T10:34:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.769+0000 7f3ae5151640 1 -- 172.21.15.53:0/4277690002 wait complete. 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.770+0000 7f3ae5151640 1 Processor -- start 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3ae5151640 1 -- start start 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3ae5151640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00d5d30 0x7f3ae00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3ae5151640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae015f3b0 0x7f3ae00d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3ae5151640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae00b7d20 0x7f3ae014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3ae5151640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00d3f80 con 0x7f3ae00d5d30 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3ae5151640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00d3e00 con 0x7f3ae015f3b0 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3ae5151640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae00ca770 con 0x7f3ae00b7d20 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3aded9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00d5d30 0x7f3ae00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3aded9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00d5d30 0x7f3ae00cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44384/0 (socket says 172.21.15.53:44384) 2024-05-10T10:34:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.771+0000 7f3aded9d640 1 -- 172.21.15.53:0/2360776190 learned_addr learned my addr 172.21.15.53:0/2360776190 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3ade59c640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae015f3b0 0x7f3ae00d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3adf59e640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae00b7d20 0x7f3ae014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3aded9d640 1 -- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae00b7d20 msgr2=0x7f3ae014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3aded9d640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae00b7d20 0x7f3ae014f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3aded9d640 1 -- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae015f3b0 msgr2=0x7f3ae00d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3aded9d640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae015f3b0 0x7f3ae00d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3aded9d640 1 -- 172.21.15.53:0/2360776190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae00b8220 con 0x7f3ae00d5d30 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3aded9d640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00d5d30 0x7f3ae00cc200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c7b0 tx=0x7f3ad000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3ac77fe640 1 -- 172.21.15.53:0/2360776190 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0018070 con 0x7f3ae00d5d30 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3ac77fe640 1 -- 172.21.15.53:0/2360776190 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad00072d0 con 0x7f3ae00d5d30 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae00b8500 con 0x7f3ae00d5d30 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3ac77fe640 1 -- 172.21.15.53:0/2360776190 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00047e0 con 0x7f3ae00d5d30 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.772+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae00b8ac0 con 0x7f3ae00d5d30 2024-05-10T10:34:26.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.773+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae006a220 con 0x7f3ae00d5d30 2024-05-10T10:34:26.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.774+0000 7f3ac77fe640 1 -- 172.21.15.53:0/2360776190 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3ad001b070 con 0x7f3ae00d5d30 2024-05-10T10:34:26.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.774+0000 7f3ac77fe640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3acc0762d0 0x7f3acc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:26.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.775+0000 7f3ac77fe640 1 -- 172.21.15.53:0/2360776190 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f3ad0096ca0 con 0x7f3ae00d5d30 2024-05-10T10:34:26.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.775+0000 7f3ade59c640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3acc0762d0 0x7f3acc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:26.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.775+0000 7f3ade59c640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3acc0762d0 0x7f3acc078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00d1a00 tx=0x7f3ad4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:26.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.779+0000 7f3ac77fe640 1 -- 172.21.15.53:0/2360776190 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3ad00603d0 con 0x7f3ae00d5d30 2024-05-10T10:34:26.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.968+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ae00c85b0 con 0x7f3acc0762d0 2024-05-10T10:34:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.975+0000 7f3ac77fe640 1 -- 172.21.15.53:0/2360776190 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251874 (secure 0 0 0) 0x7f3ae00c85b0 con 0x7f3acc0762d0 2024-05-10T10:34:26.979 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:34:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3acc0762d0 msgr2=0x7f3acc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3acc0762d0 0x7f3acc078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00d1a00 tx=0x7f3ad4009040 comp rx=0 tx=0).stop 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00d5d30 msgr2=0x7f3ae00cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00d5d30 0x7f3ae00cc200 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c7b0 tx=0x7f3ad000cc70 comp rx=0 tx=0).stop 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 shutdown_connections 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3acc0762d0 0x7f3acc078790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ae00b7d20 0x7f3ae014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ae015f3b0 0x7f3ae00d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 --2- 172.21.15.53:0/2360776190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00d5d30 0x7f3ae00cc200 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 >> 172.21.15.53:0/2360776190 conn(0x7f3ae00d5130 msgr2=0x7f3ae015e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 shutdown_connections 2024-05-10T10:34:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:26.978+0000 7f3ae5151640 1 -- 172.21.15.53:0/2360776190 wait complete. 2024-05-10T10:34:26.991 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-05-10T10:34:26.168337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237108,"kb_used_data":4272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737612,"statfs":{"total":773094113280,"available":772851314688,"internally_reserved":0,"allocated":4374528,"data_stored":2501860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.248436"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959783+0000","last_change":"2024-05-10T10:34:22.945801+0000","last_active":"2024-05-10T10:34:22.959783+0000","last_peered":"2024-05-10T10:34:22.959783+0000","last_clean":"2024-05-10T10:34:22.959783+0000","last_became_active":"2024-05-10T10:34:22.945537+0000","last_became_peered":"2024-05-10T10:34:22.945537+0000","last_unstale":"2024-05-10T10:34:22.959783+0000","last_undegraded":"2024-05-10T10:34:22.959783+0000","last_fullsized":"2024-05-10T10:34:22.959783+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:34:35.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.967102+0000","last_change":"2024-05-10T10:34:22.962579+0000","last_active":"2024-05-10T10:34:22.967102+0000","last_peered":"2024-05-10T10:34:22.967102+0000","last_clean":"2024-05-10T10:34:22.967102+0000","last_became_active":"2024-05-10T10:34:22.959848+0000","last_became_peered":"2024-05-10T10:34:22.959848+0000","last_unstale":"2024-05-10T10:34:22.967102+0000","last_undegraded":"2024-05-10T10:34:22.967102+0000","last_fullsized":"2024-05-10T10:34:22.967102+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:00:18.302819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951038+0000","last_change":"2024-05-10T10:34:22.949313+0000","last_active":"2024-05-10T10:34:22.951038+0000","last_peered":"2024-05-10T10:34:22.951038+0000","last_clean":"2024-05-10T10:34:22.951038+0000","last_became_active":"2024-05-10T10:34:22.948996+0000","last_became_peered":"2024-05-10T10:34:22.948996+0000","last_unstale":"2024-05-10T10:34:22.951038+0000","last_undegraded":"2024-05-10T10:34:22.951038+0000","last_fullsized":"2024-05-10T10:34:22.951038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:40.757906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959454+0000","last_change":"2024-05-10T10:34:22.953297+0000","last_active":"2024-05-10T10:34:22.959454+0000","last_peered":"2024-05-10T10:34:22.959454+0000","last_clean":"2024-05-10T10:34:22.959454+0000","last_became_active":"2024-05-10T10:34:22.952991+0000","last_became_peered":"2024-05-10T10:34:22.952991+0000","last_unstale":"2024-05-10T10:34:22.959454+0000","last_undegraded":"2024-05-10T10:34:22.959454+0000","last_fullsized":"2024-05-10T10:34:22.959454+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:28:55.588056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963981+0000","last_change":"2024-05-10T10:34:22.959595+0000","last_active":"2024-05-10T10:34:22.963981+0000","last_peered":"2024-05-10T10:34:22.963981+0000","last_clean":"2024-05-10T10:34:22.963981+0000","last_became_active":"2024-05-10T10:34:22.958958+0000","last_became_peered":"2024-05-10T10:34:22.958958+0000","last_unstale":"2024-05-10T10:34:22.963981+0000","last_undegraded":"2024-05-10T10:34:22.963981+0000","last_fullsized":"2024-05-10T10:34:22.963981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:36.084771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963007+0000","last_change":"2024-05-10T10:34:22.961576+0000","last_active":"2024-05-10T10:34:22.963007+0000","last_peered":"2024-05-10T10:34:22.963007+0000","last_clean":"2024-05-10T10:34:22.963007+0000","last_became_active":"2024-05-10T10:34:22.961244+0000","last_became_peered":"2024-05-10T10:34:22.961244+0000","last_unstale":"2024-05-10T10:34:22.963007+0000","last_undegraded":"2024-05-10T10:34:22.963007+0000","last_fullsized":"2024-05-10T10:34:22.963007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966159+0000","last_change":"2024-05-10T10:34:22.961167+0000","last_active":"2024-05-10T10:34:22.966159+0000","last_peered":"2024-05-10T10:34:22.966159+0000","last_clean":"2024-05-10T10:34:22.966159+0000","last_became_active":"2024-05-10T10:34:22.960890+0000","last_became_peered":"2024-05-10T10:34:22.960890+0000","last_unstale":"2024-05-10T10:34:22.966159+0000","last_undegraded":"2024-05-10T10:34:22.966159+0000","last_fullsized":"2024-05-10T10:34:22.966159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:31.277579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959640+0000","last_change":"2024-05-10T10:34:22.955448+0000","last_active":"2024-05-10T10:34:22.959640+0000","last_peered":"2024-05-10T10:34:22.959640+0000","last_clean":"2024-05-10T10:34:22.959640+0000","last_became_active":"2024-05-10T10:34:22.955112+0000","last_became_peered":"2024-05-10T10:34:22.955112+0000","last_unstale":"2024-05-10T10:34:22.959640+0000","last_undegraded":"2024-05-10T10:34:22.959640+0000","last_fullsized":"2024-05-10T10:34:22.959640+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:31.277579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.953273+0000","last_change":"2024-05-10T10:34:22.951156+0000","last_active":"2024-05-10T10:34:22.953273+0000","last_peered":"2024-05-10T10:34:22.953273+0000","last_clean":"2024-05-10T10:34:22.953273+0000","last_became_active":"2024-05-10T10:34:22.950802+0000","last_became_peered":"2024-05-10T10:34:22.950802+0000","last_unstale":"2024-05-10T10:34:22.953273+0000","last_undegraded":"2024-05-10T10:34:22.953273+0000","last_fullsized":"2024-05-10T10:34:22.953273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.949518+0000","last_change":"2024-05-10T10:34:22.947395+0000","last_active":"2024-05-10T10:34:22.949518+0000","last_peered":"2024-05-10T10:34:22.949518+0000","last_clean":"2024-05-10T10:34:22.949518+0000","last_became_active":"2024-05-10T10:34:22.946849+0000","last_became_peered":"2024-05-10T10:34:22.946849+0000","last_unstale":"2024-05-10T10:34:22.949518+0000","last_undegraded":"2024-05-10T10:34:22.949518+0000","last_fullsized":"2024-05-10T10:34:22.949518+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:37.110604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950113+0000","last_change":"2024-05-10T10:34:22.948251+0000","last_active":"2024-05-10T10:34:22.950113+0000","last_peered":"2024-05-10T10:34:22.950113+0000","last_clean":"2024-05-10T10:34:22.950113+0000","last_became_active":"2024-05-10T10:34:22.947987+0000","last_became_peered":"2024-05-10T10:34:22.947987+0000","last_unstale":"2024-05-10T10:34:22.950113+0000","last_undegraded":"2024-05-10T10:34:22.950113+0000","last_fullsized":"2024-05-10T10:34:22.950113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:01.044100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.965702+0000","last_change":"2024-05-10T10:34:22.962882+0000","last_active":"2024-05-10T10:34:22.965702+0000","last_peered":"2024-05-10T10:34:22.965702+0000","last_clean":"2024-05-10T10:34:22.965702+0000","last_became_active":"2024-05-10T10:34:22.962553+0000","last_became_peered":"2024-05-10T10:34:22.962553+0000","last_unstale":"2024-05-10T10:34:22.965702+0000","last_undegraded":"2024-05-10T10:34:22.965702+0000","last_fullsized":"2024-05-10T10:34:22.965702+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.949689+0000","last_change":"2024-05-10T10:34:22.947582+0000","last_active":"2024-05-10T10:34:22.949689+0000","last_peered":"2024-05-10T10:34:22.949689+0000","last_clean":"2024-05-10T10:34:22.949689+0000","last_became_active":"2024-05-10T10:34:22.947290+0000","last_became_peered":"2024-05-10T10:34:22.947290+0000","last_unstale":"2024-05-10T10:34:22.949689+0000","last_undegraded":"2024-05-10T10:34:22.949689+0000","last_fullsized":"2024-05-10T10:34:22.949689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:34:35.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.958292+0000","last_change":"2024-05-10T10:34:22.953517+0000","last_active":"2024-05-10T10:34:22.958292+0000","last_peered":"2024-05-10T10:34:22.958292+0000","last_clean":"2024-05-10T10:34:22.958292+0000","last_became_active":"2024-05-10T10:34:22.953023+0000","last_became_peered":"2024-05-10T10:34:22.953023+0000","last_unstale":"2024-05-10T10:34:22.958292+0000","last_undegraded":"2024-05-10T10:34:22.958292+0000","last_fullsized":"2024-05-10T10:34:22.958292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966136+0000","last_change":"2024-05-10T10:34:22.959846+0000","last_active":"2024-05-10T10:34:22.966136+0000","last_peered":"2024-05-10T10:34:22.966136+0000","last_clean":"2024-05-10T10:34:22.966136+0000","last_became_active":"2024-05-10T10:34:22.959574+0000","last_became_peered":"2024-05-10T10:34:22.959574+0000","last_unstale":"2024-05-10T10:34:22.966136+0000","last_undegraded":"2024-05-10T10:34:22.966136+0000","last_fullsized":"2024-05-10T10:34:22.966136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:53:54.272898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951170+0000","last_change":"2024-05-10T10:34:22.947531+0000","last_active":"2024-05-10T10:34:22.951170+0000","last_peered":"2024-05-10T10:34:22.951170+0000","last_clean":"2024-05-10T10:34:22.951170+0000","last_became_active":"2024-05-10T10:34:22.947286+0000","last_became_peered":"2024-05-10T10:34:22.947286+0000","last_unstale":"2024-05-10T10:34:22.951170+0000","last_undegraded":"2024-05-10T10:34:22.951170+0000","last_fullsized":"2024-05-10T10:34:22.951170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955306+0000","last_change":"2024-05-10T10:34:22.952207+0000","last_active":"2024-05-10T10:34:22.955306+0000","last_peered":"2024-05-10T10:34:22.955306+0000","last_clean":"2024-05-10T10:34:22.955306+0000","last_became_active":"2024-05-10T10:34:22.951646+0000","last_became_peered":"2024-05-10T10:34:22.951646+0000","last_unstale":"2024-05-10T10:34:22.955306+0000","last_undegraded":"2024-05-10T10:34:22.955306+0000","last_fullsized":"2024-05-10T10:34:22.955306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:37.110604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955205+0000","last_change":"2024-05-10T10:34:22.951407+0000","last_active":"2024-05-10T10:34:22.955205+0000","last_peered":"2024-05-10T10:34:22.955205+0000","last_clean":"2024-05-10T10:34:22.955205+0000","last_became_active":"2024-05-10T10:34:22.950728+0000","last_became_peered":"2024-05-10T10:34:22.950728+0000","last_unstale":"2024-05-10T10:34:22.955205+0000","last_undegraded":"2024-05-10T10:34:22.955205+0000","last_fullsized":"2024-05-10T10:34:22.955205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:26:23.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964476+0000","last_change":"2024-05-10T10:34:22.960531+0000","last_active":"2024-05-10T10:34:22.964476+0000","last_peered":"2024-05-10T10:34:22.964476+0000","last_clean":"2024-05-10T10:34:22.964476+0000","last_became_active":"2024-05-10T10:34:22.959148+0000","last_became_peered":"2024-05-10T10:34:22.959148+0000","last_unstale":"2024-05-10T10:34:22.964476+0000","last_undegraded":"2024-05-10T10:34:22.964476+0000","last_fullsized":"2024-05-10T10:34:22.964476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:46.071001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954480+0000","last_change":"2024-05-10T10:34:22.951236+0000","last_active":"2024-05-10T10:34:22.954480+0000","last_peered":"2024-05-10T10:34:22.954480+0000","last_clean":"2024-05-10T10:34:22.954480+0000","last_became_active":"2024-05-10T10:34:22.950788+0000","last_became_peered":"2024-05-10T10:34:22.950788+0000","last_unstale":"2024-05-10T10:34:22.954480+0000","last_undegraded":"2024-05-10T10:34:22.954480+0000","last_fullsized":"2024-05-10T10:34:22.954480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:05:51.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.949636+0000","last_change":"2024-05-10T10:34:22.947425+0000","last_active":"2024-05-10T10:34:22.949636+0000","last_peered":"2024-05-10T10:34:22.949636+0000","last_clean":"2024-05-10T10:34:22.949636+0000","last_became_active":"2024-05-10T10:34:22.946646+0000","last_became_peered":"2024-05-10T10:34:22.946646+0000","last_unstale":"2024-05-10T10:34:22.949636+0000","last_undegraded":"2024-05-10T10:34:22.949636+0000","last_fullsized":"2024-05-10T10:34:22.949636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:14:25.324350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966334+0000","last_change":"2024-05-10T10:34:22.963155+0000","last_active":"2024-05-10T10:34:22.966334+0000","last_peered":"2024-05-10T10:34:22.966334+0000","last_clean":"2024-05-10T10:34:22.966334+0000","last_became_active":"2024-05-10T10:34:22.962830+0000","last_became_peered":"2024-05-10T10:34:22.962830+0000","last_unstale":"2024-05-10T10:34:22.966334+0000","last_undegraded":"2024-05-10T10:34:22.966334+0000","last_fullsized":"2024-05-10T10:34:22.966334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:35.506101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951151+0000","last_change":"2024-05-10T10:34:22.945113+0000","last_active":"2024-05-10T10:34:22.951151+0000","last_peered":"2024-05-10T10:34:22.951151+0000","last_clean":"2024-05-10T10:34:22.951151+0000","last_became_active":"2024-05-10T10:34:22.944812+0000","last_became_peered":"2024-05-10T10:34:22.944812+0000","last_unstale":"2024-05-10T10:34:22.951151+0000","last_undegraded":"2024-05-10T10:34:22.951151+0000","last_fullsized":"2024-05-10T10:34:22.951151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964902+0000","last_change":"2024-05-10T10:34:22.958535+0000","last_active":"2024-05-10T10:34:22.964902+0000","last_peered":"2024-05-10T10:34:22.964902+0000","last_clean":"2024-05-10T10:34:22.964902+0000","last_became_active":"2024-05-10T10:34:22.958217+0000","last_became_peered":"2024-05-10T10:34:22.958217+0000","last_unstale":"2024-05-10T10:34:22.964902+0000","last_undegraded":"2024-05-10T10:34:22.964902+0000","last_fullsized":"2024-05-10T10:34:22.964902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:37.648074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955388+0000","last_change":"2024-05-10T10:34:22.952256+0000","last_active":"2024-05-10T10:34:22.955388+0000","last_peered":"2024-05-10T10:34:22.955388+0000","last_clean":"2024-05-10T10:34:22.955388+0000","last_became_active":"2024-05-10T10:34:22.951880+0000","last_became_peered":"2024-05-10T10:34:22.951880+0000","last_unstale":"2024-05-10T10:34:22.955388+0000","last_undegraded":"2024-05-10T10:34:22.955388+0000","last_fullsized":"2024-05-10T10:34:22.955388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:44.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954080+0000","last_change":"2024-05-10T10:34:22.950997+0000","last_active":"2024-05-10T10:34:22.954080+0000","last_peered":"2024-05-10T10:34:22.954080+0000","last_clean":"2024-05-10T10:34:22.954080+0000","last_became_active":"2024-05-10T10:34:22.950702+0000","last_became_peered":"2024-05-10T10:34:22.950702+0000","last_unstale":"2024-05-10T10:34:22.954080+0000","last_undegraded":"2024-05-10T10:34:22.954080+0000","last_fullsized":"2024-05-10T10:34:22.954080+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:40.757906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964847+0000","last_change":"2024-05-10T10:34:22.961969+0000","last_active":"2024-05-10T10:34:22.964847+0000","last_peered":"2024-05-10T10:34:22.964847+0000","last_clean":"2024-05-10T10:34:22.964847+0000","last_became_active":"2024-05-10T10:34:22.961424+0000","last_became_peered":"2024-05-10T10:34:22.961424+0000","last_unstale":"2024-05-10T10:34:22.964847+0000","last_undegraded":"2024-05-10T10:34:22.964847+0000","last_fullsized":"2024-05-10T10:34:22.964847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:25.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.969503+0000","last_change":"2024-05-10T10:34:22.967078+0000","last_active":"2024-05-10T10:34:22.969503+0000","last_peered":"2024-05-10T10:34:22.969503+0000","last_clean":"2024-05-10T10:34:22.969503+0000","last_became_active":"2024-05-10T10:34:22.966799+0000","last_became_peered":"2024-05-10T10:34:22.966799+0000","last_unstale":"2024-05-10T10:34:22.969503+0000","last_undegraded":"2024-05-10T10:34:22.969503+0000","last_fullsized":"2024-05-10T10:34:22.969503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:47:58.867430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955090+0000","last_change":"2024-05-10T10:34:22.954373+0000","last_active":"2024-05-10T10:34:22.955090+0000","last_peered":"2024-05-10T10:34:22.955090+0000","last_clean":"2024-05-10T10:34:22.955090+0000","last_became_active":"2024-05-10T10:34:22.953436+0000","last_became_peered":"2024-05-10T10:34:22.953436+0000","last_unstale":"2024-05-10T10:34:22.955090+0000","last_undegraded":"2024-05-10T10:34:22.955090+0000","last_fullsized":"2024-05-10T10:34:22.955090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:59.543871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.962692+0000","last_change":"2024-05-10T10:34:22.961643+0000","last_active":"2024-05-10T10:34:22.962692+0000","last_peered":"2024-05-10T10:34:22.962692+0000","last_clean":"2024-05-10T10:34:22.962692+0000","last_became_active":"2024-05-10T10:34:22.961356+0000","last_became_peered":"2024-05-10T10:34:22.961356+0000","last_unstale":"2024-05-10T10:34:22.962692+0000","last_undegraded":"2024-05-10T10:34:22.962692+0000","last_fullsized":"2024-05-10T10:34:22.962692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:45.805155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963979+0000","last_change":"2024-05-10T10:34:22.959527+0000","last_active":"2024-05-10T10:34:22.963979+0000","last_peered":"2024-05-10T10:34:22.963979+0000","last_clean":"2024-05-10T10:34:22.963979+0000","last_became_active":"2024-05-10T10:34:22.959229+0000","last_became_peered":"2024-05-10T10:34:22.959229+0000","last_unstale":"2024-05-10T10:34:22.963979+0000","last_undegraded":"2024-05-10T10:34:22.963979+0000","last_fullsized":"2024-05-10T10:34:22.963979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:01.044100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.961733+0000","last_change":"2024-05-10T10:34:22.960222+0000","last_active":"2024-05-10T10:34:22.961733+0000","last_peered":"2024-05-10T10:34:22.961733+0000","last_clean":"2024-05-10T10:34:22.961733+0000","last_became_active":"2024-05-10T10:34:22.959929+0000","last_became_peered":"2024-05-10T10:34:22.959929+0000","last_unstale":"2024-05-10T10:34:22.961733+0000","last_undegraded":"2024-05-10T10:34:22.961733+0000","last_fullsized":"2024-05-10T10:34:22.961733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954584+0000","last_change":"2024-05-10T10:34:22.950597+0000","last_active":"2024-05-10T10:34:22.954584+0000","last_peered":"2024-05-10T10:34:22.954584+0000","last_clean":"2024-05-10T10:34:22.954584+0000","last_became_active":"2024-05-10T10:34:22.950209+0000","last_became_peered":"2024-05-10T10:34:22.950209+0000","last_unstale":"2024-05-10T10:34:22.954584+0000","last_undegraded":"2024-05-10T10:34:22.954584+0000","last_fullsized":"2024-05-10T10:34:22.954584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966479+0000","last_change":"2024-05-10T10:34:21.939198+0000","last_active":"2024-05-10T10:34:22.966479+0000","last_peered":"2024-05-10T10:34:22.966479+0000","last_clean":"2024-05-10T10:34:22.966479+0000","last_became_active":"2024-05-10T10:34:21.938678+0000","last_became_peered":"2024-05-10T10:34:21.938678+0000","last_unstale":"2024-05-10T10:34:22.966479+0000","last_undegraded":"2024-05-10T10:34:22.966479+0000","last_fullsized":"2024-05-10T10:34:22.966479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:43.544910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954939+0000","last_change":"2024-05-10T10:34:22.952036+0000","last_active":"2024-05-10T10:34:22.954939+0000","last_peered":"2024-05-10T10:34:22.954939+0000","last_clean":"2024-05-10T10:34:22.954939+0000","last_became_active":"2024-05-10T10:34:22.951769+0000","last_became_peered":"2024-05-10T10:34:22.951769+0000","last_unstale":"2024-05-10T10:34:22.954939+0000","last_undegraded":"2024-05-10T10:34:22.954939+0000","last_fullsized":"2024-05-10T10:34:22.954939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:18.194365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963571+0000","last_change":"2024-05-10T10:34:21.943676+0000","last_active":"2024-05-10T10:34:22.963571+0000","last_peered":"2024-05-10T10:34:22.963571+0000","last_clean":"2024-05-10T10:34:22.963571+0000","last_became_active":"2024-05-10T10:34:21.943365+0000","last_became_peered":"2024-05-10T10:34:21.943365+0000","last_unstale":"2024-05-10T10:34:22.963571+0000","last_undegraded":"2024-05-10T10:34:22.963571+0000","last_fullsized":"2024-05-10T10:34:22.963571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:43.624004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.957588+0000","last_change":"2024-05-10T10:34:22.953375+0000","last_active":"2024-05-10T10:34:22.957588+0000","last_peered":"2024-05-10T10:34:22.957588+0000","last_clean":"2024-05-10T10:34:22.957588+0000","last_became_active":"2024-05-10T10:34:22.953047+0000","last_became_peered":"2024-05-10T10:34:22.953047+0000","last_unstale":"2024-05-10T10:34:22.957588+0000","last_undegraded":"2024-05-10T10:34:22.957588+0000","last_fullsized":"2024-05-10T10:34:22.957588+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:18.194365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.967147+0000","last_change":"2024-05-10T10:34:21.941982+0000","last_active":"2024-05-10T10:34:22.967147+0000","last_peered":"2024-05-10T10:34:22.967147+0000","last_clean":"2024-05-10T10:34:22.967147+0000","last_became_active":"2024-05-10T10:34:21.941554+0000","last_became_peered":"2024-05-10T10:34:21.941554+0000","last_unstale":"2024-05-10T10:34:22.967147+0000","last_undegraded":"2024-05-10T10:34:22.967147+0000","last_fullsized":"2024-05-10T10:34:22.967147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:24.394698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959824+0000","last_change":"2024-05-10T10:34:22.956620+0000","last_active":"2024-05-10T10:34:22.959824+0000","last_peered":"2024-05-10T10:34:22.959824+0000","last_clean":"2024-05-10T10:34:22.959824+0000","last_became_active":"2024-05-10T10:34:22.956297+0000","last_became_peered":"2024-05-10T10:34:22.956297+0000","last_unstale":"2024-05-10T10:34:22.959824+0000","last_undegraded":"2024-05-10T10:34:22.959824+0000","last_fullsized":"2024-05-10T10:34:22.959824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:47:58.867430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.968438+0000","last_change":"2024-05-10T10:34:21.938057+0000","last_active":"2024-05-10T10:34:22.968438+0000","last_peered":"2024-05-10T10:34:22.968438+0000","last_clean":"2024-05-10T10:34:22.968438+0000","last_became_active":"2024-05-10T10:34:21.937804+0000","last_became_peered":"2024-05-10T10:34:21.937804+0000","last_unstale":"2024-05-10T10:34:22.968438+0000","last_undegraded":"2024-05-10T10:34:22.968438+0000","last_fullsized":"2024-05-10T10:34:22.968438+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:58.539394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951179+0000","last_change":"2024-05-10T10:34:22.947366+0000","last_active":"2024-05-10T10:34:22.951179+0000","last_peered":"2024-05-10T10:34:22.951179+0000","last_clean":"2024-05-10T10:34:22.951179+0000","last_became_active":"2024-05-10T10:34:22.944886+0000","last_became_peered":"2024-05-10T10:34:22.944886+0000","last_unstale":"2024-05-10T10:34:22.951179+0000","last_undegraded":"2024-05-10T10:34:22.951179+0000","last_fullsized":"2024-05-10T10:34:22.951179+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:38.753738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963534+0000","last_change":"2024-05-10T10:34:21.939856+0000","last_active":"2024-05-10T10:34:22.963534+0000","last_peered":"2024-05-10T10:34:22.963534+0000","last_clean":"2024-05-10T10:34:22.963534+0000","last_became_active":"2024-05-10T10:34:21.939559+0000","last_became_peered":"2024-05-10T10:34:21.939559+0000","last_unstale":"2024-05-10T10:34:22.963534+0000","last_undegraded":"2024-05-10T10:34:22.963534+0000","last_fullsized":"2024-05-10T10:34:22.963534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:17.198949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955802+0000","last_change":"2024-05-10T10:34:22.951040+0000","last_active":"2024-05-10T10:34:22.955802+0000","last_peered":"2024-05-10T10:34:22.955802+0000","last_clean":"2024-05-10T10:34:22.955802+0000","last_became_active":"2024-05-10T10:34:22.950137+0000","last_became_peered":"2024-05-10T10:34:22.950137+0000","last_unstale":"2024-05-10T10:34:22.955802+0000","last_undegraded":"2024-05-10T10:34:22.955802+0000","last_fullsized":"2024-05-10T10:34:22.955802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:38.542678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963335+0000","last_change":"2024-05-10T10:34:21.949021+0000","last_active":"2024-05-10T10:34:22.963335+0000","last_peered":"2024-05-10T10:34:22.963335+0000","last_clean":"2024-05-10T10:34:22.963335+0000","last_became_active":"2024-05-10T10:34:21.948782+0000","last_became_peered":"2024-05-10T10:34:21.948782+0000","last_unstale":"2024-05-10T10:34:22.963335+0000","last_undegraded":"2024-05-10T10:34:22.963335+0000","last_fullsized":"2024-05-10T10:34:22.963335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:37.749261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950333+0000","last_change":"2024-05-10T10:34:22.949084+0000","last_active":"2024-05-10T10:34:22.950333+0000","last_peered":"2024-05-10T10:34:22.950333+0000","last_clean":"2024-05-10T10:34:22.950333+0000","last_became_active":"2024-05-10T10:34:22.948827+0000","last_became_peered":"2024-05-10T10:34:22.948827+0000","last_unstale":"2024-05-10T10:34:22.950333+0000","last_undegraded":"2024-05-10T10:34:22.950333+0000","last_fullsized":"2024-05-10T10:34:22.950333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:40.757906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.938267+0000","last_change":"2024-05-10T10:34:21.939497+0000","last_active":"2024-05-10T10:34:22.938267+0000","last_peered":"2024-05-10T10:34:22.938267+0000","last_clean":"2024-05-10T10:34:22.938267+0000","last_became_active":"2024-05-10T10:34:21.939228+0000","last_became_peered":"2024-05-10T10:34:21.939228+0000","last_unstale":"2024-05-10T10:34:22.938267+0000","last_undegraded":"2024-05-10T10:34:22.938267+0000","last_fullsized":"2024-05-10T10:34:22.938267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:44.800678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.961304+0000","last_change":"2024-05-10T10:34:22.959971+0000","last_active":"2024-05-10T10:34:22.961304+0000","last_peered":"2024-05-10T10:34:22.961304+0000","last_clean":"2024-05-10T10:34:22.961304+0000","last_became_active":"2024-05-10T10:34:22.959314+0000","last_became_peered":"2024-05-10T10:34:22.959314+0000","last_unstale":"2024-05-10T10:34:22.961304+0000","last_undegraded":"2024-05-10T10:34:22.961304+0000","last_fullsized":"2024-05-10T10:34:22.961304+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:30.812601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936785+0000","last_change":"2024-05-10T10:34:21.939449+0000","last_active":"2024-05-10T10:34:22.936785+0000","last_peered":"2024-05-10T10:34:22.936785+0000","last_clean":"2024-05-10T10:34:22.936785+0000","last_became_active":"2024-05-10T10:34:21.939184+0000","last_became_peered":"2024-05-10T10:34:21.939184+0000","last_unstale":"2024-05-10T10:34:22.936785+0000","last_undegraded":"2024-05-10T10:34:22.936785+0000","last_fullsized":"2024-05-10T10:34:22.936785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:43.624004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959427+0000","last_change":"2024-05-10T10:34:22.952546+0000","last_active":"2024-05-10T10:34:22.959427+0000","last_peered":"2024-05-10T10:34:22.959427+0000","last_clean":"2024-05-10T10:34:22.959427+0000","last_became_active":"2024-05-10T10:34:22.952246+0000","last_became_peered":"2024-05-10T10:34:22.952246+0000","last_unstale":"2024-05-10T10:34:22.959427+0000","last_undegraded":"2024-05-10T10:34:22.959427+0000","last_fullsized":"2024-05-10T10:34:22.959427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:03.869048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.939522+0000","last_change":"2024-05-10T10:34:21.941345+0000","last_active":"2024-05-10T10:34:22.939522+0000","last_peered":"2024-05-10T10:34:22.939522+0000","last_clean":"2024-05-10T10:34:22.939522+0000","last_became_active":"2024-05-10T10:34:21.938931+0000","last_became_peered":"2024-05-10T10:34:21.938931+0000","last_unstale":"2024-05-10T10:34:22.939522+0000","last_undegraded":"2024-05-10T10:34:22.939522+0000","last_fullsized":"2024-05-10T10:34:22.939522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:34.501624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955932+0000","last_change":"2024-05-10T10:34:22.952832+0000","last_active":"2024-05-10T10:34:22.955932+0000","last_peered":"2024-05-10T10:34:22.955932+0000","last_clean":"2024-05-10T10:34:22.955932+0000","last_became_active":"2024-05-10T10:34:22.952551+0000","last_became_peered":"2024-05-10T10:34:22.952551+0000","last_unstale":"2024-05-10T10:34:22.955932+0000","last_undegraded":"2024-05-10T10:34:22.955932+0000","last_fullsized":"2024-05-10T10:34:22.955932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:34:35.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.940703+0000","last_change":"2024-05-10T10:34:21.936303+0000","last_active":"2024-05-10T10:34:22.940703+0000","last_peered":"2024-05-10T10:34:22.940703+0000","last_clean":"2024-05-10T10:34:22.940703+0000","last_became_active":"2024-05-10T10:34:21.933569+0000","last_became_peered":"2024-05-10T10:34:21.933569+0000","last_unstale":"2024-05-10T10:34:22.940703+0000","last_undegraded":"2024-05-10T10:34:22.940703+0000","last_fullsized":"2024-05-10T10:34:22.940703+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:39.753429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950763+0000","last_change":"2024-05-10T10:34:22.947178+0000","last_active":"2024-05-10T10:34:22.950763+0000","last_peered":"2024-05-10T10:34:22.950763+0000","last_clean":"2024-05-10T10:34:22.950763+0000","last_became_active":"2024-05-10T10:34:22.946906+0000","last_became_peered":"2024-05-10T10:34:22.946906+0000","last_unstale":"2024-05-10T10:34:22.950763+0000","last_undegraded":"2024-05-10T10:34:22.950763+0000","last_fullsized":"2024-05-10T10:34:22.950763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:25.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963773+0000","last_change":"2024-05-10T10:34:21.937948+0000","last_active":"2024-05-10T10:34:22.963773+0000","last_peered":"2024-05-10T10:34:22.963773+0000","last_clean":"2024-05-10T10:34:22.963773+0000","last_became_active":"2024-05-10T10:34:21.937589+0000","last_became_peered":"2024-05-10T10:34:21.937589+0000","last_unstale":"2024-05-10T10:34:22.963773+0000","last_undegraded":"2024-05-10T10:34:22.963773+0000","last_fullsized":"2024-05-10T10:34:22.963773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:17.189889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966261+0000","last_change":"2024-05-10T10:34:22.960422+0000","last_active":"2024-05-10T10:34:22.966261+0000","last_peered":"2024-05-10T10:34:22.966261+0000","last_clean":"2024-05-10T10:34:22.966261+0000","last_became_active":"2024-05-10T10:34:22.960132+0000","last_became_peered":"2024-05-10T10:34:22.960132+0000","last_unstale":"2024-05-10T10:34:22.966261+0000","last_undegraded":"2024-05-10T10:34:22.966261+0000","last_fullsized":"2024-05-10T10:34:22.966261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.976847+0000","last_change":"2024-05-10T10:34:21.948576+0000","last_active":"2024-05-10T10:34:22.976847+0000","last_peered":"2024-05-10T10:34:22.976847+0000","last_clean":"2024-05-10T10:34:22.976847+0000","last_became_active":"2024-05-10T10:34:21.948217+0000","last_became_peered":"2024-05-10T10:34:21.948217+0000","last_unstale":"2024-05-10T10:34:22.976847+0000","last_undegraded":"2024-05-10T10:34:22.976847+0000","last_fullsized":"2024-05-10T10:34:22.976847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:00.039624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.953489+0000","last_change":"2024-05-10T10:34:22.951035+0000","last_active":"2024-05-10T10:34:22.953489+0000","last_peered":"2024-05-10T10:34:22.953489+0000","last_clean":"2024-05-10T10:34:22.953489+0000","last_became_active":"2024-05-10T10:34:22.950382+0000","last_became_peered":"2024-05-10T10:34:22.950382+0000","last_unstale":"2024-05-10T10:34:22.953489+0000","last_undegraded":"2024-05-10T10:34:22.953489+0000","last_fullsized":"2024-05-10T10:34:22.953489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:18.194365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.967853+0000","last_change":"2024-05-10T10:34:21.937199+0000","last_active":"2024-05-10T10:34:22.967853+0000","last_peered":"2024-05-10T10:34:22.967853+0000","last_clean":"2024-05-10T10:34:22.967853+0000","last_became_active":"2024-05-10T10:34:21.936927+0000","last_became_peered":"2024-05-10T10:34:21.936927+0000","last_unstale":"2024-05-10T10:34:22.967853+0000","last_undegraded":"2024-05-10T10:34:22.967853+0000","last_fullsized":"2024-05-10T10:34:22.967853+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:14:24.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959688+0000","last_change":"2024-05-10T10:34:22.952267+0000","last_active":"2024-05-10T10:34:22.959688+0000","last_peered":"2024-05-10T10:34:22.959688+0000","last_clean":"2024-05-10T10:34:22.959688+0000","last_became_active":"2024-05-10T10:34:22.952003+0000","last_became_peered":"2024-05-10T10:34:22.952003+0000","last_unstale":"2024-05-10T10:34:22.959688+0000","last_undegraded":"2024-05-10T10:34:22.959688+0000","last_fullsized":"2024-05-10T10:34:22.959688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:36:34.714282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.938210+0000","last_change":"2024-05-10T10:34:21.940676+0000","last_active":"2024-05-10T10:34:22.938210+0000","last_peered":"2024-05-10T10:34:22.938210+0000","last_clean":"2024-05-10T10:34:22.938210+0000","last_became_active":"2024-05-10T10:34:21.938910+0000","last_became_peered":"2024-05-10T10:34:21.938910+0000","last_unstale":"2024-05-10T10:34:22.938210+0000","last_undegraded":"2024-05-10T10:34:22.938210+0000","last_fullsized":"2024-05-10T10:34:22.938210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:58.539394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963912+0000","last_change":"2024-05-10T10:34:22.959723+0000","last_active":"2024-05-10T10:34:22.963912+0000","last_peered":"2024-05-10T10:34:22.963912+0000","last_clean":"2024-05-10T10:34:22.963912+0000","last_became_active":"2024-05-10T10:34:22.959441+0000","last_became_peered":"2024-05-10T10:34:22.959441+0000","last_unstale":"2024-05-10T10:34:22.963912+0000","last_undegraded":"2024-05-10T10:34:22.963912+0000","last_fullsized":"2024-05-10T10:34:22.963912+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:38.542678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945548+0000","last_change":"2024-05-10T10:34:21.938877+0000","last_active":"2024-05-10T10:34:22.945548+0000","last_peered":"2024-05-10T10:34:22.945548+0000","last_clean":"2024-05-10T10:34:22.945548+0000","last_became_active":"2024-05-10T10:34:21.938552+0000","last_became_peered":"2024-05-10T10:34:21.938552+0000","last_unstale":"2024-05-10T10:34:22.945548+0000","last_undegraded":"2024-05-10T10:34:22.945548+0000","last_fullsized":"2024-05-10T10:34:22.945548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:38:13.381884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960247+0000","last_change":"2024-05-10T10:34:22.951954+0000","last_active":"2024-05-10T10:34:22.960247+0000","last_peered":"2024-05-10T10:34:22.960247+0000","last_clean":"2024-05-10T10:34:22.960247+0000","last_became_active":"2024-05-10T10:34:22.951656+0000","last_became_peered":"2024-05-10T10:34:22.951656+0000","last_unstale":"2024-05-10T10:34:22.960247+0000","last_undegraded":"2024-05-10T10:34:22.960247+0000","last_fullsized":"2024-05-10T10:34:22.960247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.110682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.969371+0000","last_change":"2024-05-10T10:34:21.949056+0000","last_active":"2024-05-10T10:34:22.969371+0000","last_peered":"2024-05-10T10:34:22.969371+0000","last_clean":"2024-05-10T10:34:22.969371+0000","last_became_active":"2024-05-10T10:34:21.948803+0000","last_became_peered":"2024-05-10T10:34:21.948803+0000","last_unstale":"2024-05-10T10:34:22.969371+0000","last_undegraded":"2024-05-10T10:34:22.969371+0000","last_fullsized":"2024-05-10T10:34:22.969371+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:24.394698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.968307+0000","last_change":"2024-05-10T10:34:22.965209+0000","last_active":"2024-05-10T10:34:22.968307+0000","last_peered":"2024-05-10T10:34:22.968307+0000","last_clean":"2024-05-10T10:34:22.968307+0000","last_became_active":"2024-05-10T10:34:22.963079+0000","last_became_peered":"2024-05-10T10:34:22.963079+0000","last_unstale":"2024-05-10T10:34:22.968307+0000","last_undegraded":"2024-05-10T10:34:22.968307+0000","last_fullsized":"2024-05-10T10:34:22.968307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:18:02.863461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963887+0000","last_change":"2024-05-10T10:34:21.939434+0000","last_active":"2024-05-10T10:34:22.963887+0000","last_peered":"2024-05-10T10:34:22.963887+0000","last_clean":"2024-05-10T10:34:22.963887+0000","last_became_active":"2024-05-10T10:34:21.939172+0000","last_became_peered":"2024-05-10T10:34:21.939172+0000","last_unstale":"2024-05-10T10:34:22.963887+0000","last_undegraded":"2024-05-10T10:34:22.963887+0000","last_fullsized":"2024-05-10T10:34:22.963887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:46.129934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.948387+0000","last_change":"2024-05-10T10:34:22.944487+0000","last_active":"2024-05-10T10:34:22.948387+0000","last_peered":"2024-05-10T10:34:22.948387+0000","last_clean":"2024-05-10T10:34:22.948387+0000","last_became_active":"2024-05-10T10:34:22.944184+0000","last_became_peered":"2024-05-10T10:34:22.944184+0000","last_unstale":"2024-05-10T10:34:22.948387+0000","last_undegraded":"2024-05-10T10:34:22.948387+0000","last_fullsized":"2024-05-10T10:34:22.948387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:35.506101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963135+0000","last_change":"2024-05-10T10:34:21.945201+0000","last_active":"2024-05-10T10:34:22.963135+0000","last_peered":"2024-05-10T10:34:22.963135+0000","last_clean":"2024-05-10T10:34:22.963135+0000","last_became_active":"2024-05-10T10:34:21.944968+0000","last_became_peered":"2024-05-10T10:34:21.944968+0000","last_unstale":"2024-05-10T10:34:22.963135+0000","last_undegraded":"2024-05-10T10:34:22.963135+0000","last_fullsized":"2024-05-10T10:34:22.963135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:24.394698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.946875+0000","last_change":"2024-05-10T10:34:22.941905+0000","last_active":"2024-05-10T10:34:22.946875+0000","last_peered":"2024-05-10T10:34:22.946875+0000","last_clean":"2024-05-10T10:34:22.946875+0000","last_became_active":"2024-05-10T10:34:22.941646+0000","last_became_peered":"2024-05-10T10:34:22.941646+0000","last_unstale":"2024-05-10T10:34:22.946875+0000","last_undegraded":"2024-05-10T10:34:22.946875+0000","last_fullsized":"2024-05-10T10:34:22.946875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:38.753738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964190+0000","last_change":"2024-05-10T10:34:21.948228+0000","last_active":"2024-05-10T10:34:22.964190+0000","last_peered":"2024-05-10T10:34:22.964190+0000","last_clean":"2024-05-10T10:34:22.964190+0000","last_became_active":"2024-05-10T10:34:21.947889+0000","last_became_peered":"2024-05-10T10:34:21.947889+0000","last_unstale":"2024-05-10T10:34:22.964190+0000","last_undegraded":"2024-05-10T10:34:22.964190+0000","last_fullsized":"2024-05-10T10:34:22.964190+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:20.566743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966537+0000","last_change":"2024-05-10T10:34:22.964710+0000","last_active":"2024-05-10T10:34:22.966537+0000","last_peered":"2024-05-10T10:34:22.966537+0000","last_clean":"2024-05-10T10:34:22.966537+0000","last_became_active":"2024-05-10T10:34:22.964225+0000","last_became_peered":"2024-05-10T10:34:22.964225+0000","last_unstale":"2024-05-10T10:34:22.966537+0000","last_undegraded":"2024-05-10T10:34:22.966537+0000","last_fullsized":"2024-05-10T10:34:22.966537+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:58.457557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.937616+0000","last_change":"2024-05-10T10:34:21.943638+0000","last_active":"2024-05-10T10:34:22.937616+0000","last_peered":"2024-05-10T10:34:22.937616+0000","last_clean":"2024-05-10T10:34:22.937616+0000","last_became_active":"2024-05-10T10:34:21.943335+0000","last_became_peered":"2024-05-10T10:34:21.943335+0000","last_unstale":"2024-05-10T10:34:22.937616+0000","last_undegraded":"2024-05-10T10:34:22.937616+0000","last_fullsized":"2024-05-10T10:34:22.937616+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:43.624004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.969312+0000","last_change":"2024-05-10T10:34:22.962472+0000","last_active":"2024-05-10T10:34:22.969312+0000","last_peered":"2024-05-10T10:34:22.969312+0000","last_clean":"2024-05-10T10:34:22.969312+0000","last_became_active":"2024-05-10T10:34:22.960525+0000","last_became_peered":"2024-05-10T10:34:22.960525+0000","last_unstale":"2024-05-10T10:34:22.969312+0000","last_undegraded":"2024-05-10T10:34:22.969312+0000","last_fullsized":"2024-05-10T10:34:22.969312+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.110682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964521+0000","last_change":"2024-05-10T10:34:21.937201+0000","last_active":"2024-05-10T10:34:22.964521+0000","last_peered":"2024-05-10T10:34:22.964521+0000","last_clean":"2024-05-10T10:34:22.964521+0000","last_became_active":"2024-05-10T10:34:21.936888+0000","last_became_peered":"2024-05-10T10:34:21.936888+0000","last_unstale":"2024-05-10T10:34:22.964521+0000","last_undegraded":"2024-05-10T10:34:22.964521+0000","last_fullsized":"2024-05-10T10:34:22.964521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:11.892379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950593+0000","last_change":"2024-05-10T10:34:22.946531+0000","last_active":"2024-05-10T10:34:22.950593+0000","last_peered":"2024-05-10T10:34:22.950593+0000","last_clean":"2024-05-10T10:34:22.950593+0000","last_became_active":"2024-05-10T10:34:22.946269+0000","last_became_peered":"2024-05-10T10:34:22.946269+0000","last_unstale":"2024-05-10T10:34:22.950593+0000","last_undegraded":"2024-05-10T10:34:22.950593+0000","last_fullsized":"2024-05-10T10:34:22.950593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:45.805155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945681+0000","last_change":"2024-05-10T10:34:21.935279+0000","last_active":"2024-05-10T10:34:22.945681+0000","last_peered":"2024-05-10T10:34:22.945681+0000","last_clean":"2024-05-10T10:34:22.945681+0000","last_became_active":"2024-05-10T10:34:21.934970+0000","last_became_peered":"2024-05-10T10:34:21.934970+0000","last_unstale":"2024-05-10T10:34:22.945681+0000","last_undegraded":"2024-05-10T10:34:22.945681+0000","last_fullsized":"2024-05-10T10:34:22.945681+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:47:34.131705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955891+0000","last_change":"2024-05-10T10:34:22.951580+0000","last_active":"2024-05-10T10:34:22.955891+0000","last_peered":"2024-05-10T10:34:22.955891+0000","last_clean":"2024-05-10T10:34:22.955891+0000","last_became_active":"2024-05-10T10:34:22.951307+0000","last_became_peered":"2024-05-10T10:34:22.951307+0000","last_unstale":"2024-05-10T10:34:22.955891+0000","last_undegraded":"2024-05-10T10:34:22.955891+0000","last_fullsized":"2024-05-10T10:34:22.955891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.110682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936745+0000","last_change":"2024-05-10T10:34:21.948225+0000","last_active":"2024-05-10T10:34:22.936745+0000","last_peered":"2024-05-10T10:34:22.936745+0000","last_clean":"2024-05-10T10:34:22.936745+0000","last_became_active":"2024-05-10T10:34:21.947894+0000","last_became_peered":"2024-05-10T10:34:21.947894+0000","last_unstale":"2024-05-10T10:34:22.936745+0000","last_undegraded":"2024-05-10T10:34:22.936745+0000","last_fullsized":"2024-05-10T10:34:22.936745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:17.189889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960225+0000","last_change":"2024-05-10T10:34:22.948513+0000","last_active":"2024-05-10T10:34:22.960225+0000","last_peered":"2024-05-10T10:34:22.960225+0000","last_clean":"2024-05-10T10:34:22.960225+0000","last_became_active":"2024-05-10T10:34:22.948186+0000","last_became_peered":"2024-05-10T10:34:22.948186+0000","last_unstale":"2024-05-10T10:34:22.960225+0000","last_undegraded":"2024-05-10T10:34:22.960225+0000","last_fullsized":"2024-05-10T10:34:22.960225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:38.542678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945559+0000","last_change":"2024-05-10T10:34:21.935491+0000","last_active":"2024-05-10T10:34:22.945559+0000","last_peered":"2024-05-10T10:34:22.945559+0000","last_clean":"2024-05-10T10:34:22.945559+0000","last_became_active":"2024-05-10T10:34:21.935236+0000","last_became_peered":"2024-05-10T10:34:21.935236+0000","last_unstale":"2024-05-10T10:34:22.945559+0000","last_undegraded":"2024-05-10T10:34:22.945559+0000","last_fullsized":"2024-05-10T10:34:22.945559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:44.846873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954453+0000","last_change":"2024-05-10T10:34:22.950668+0000","last_active":"2024-05-10T10:34:22.954453+0000","last_peered":"2024-05-10T10:34:22.954453+0000","last_clean":"2024-05-10T10:34:22.954453+0000","last_became_active":"2024-05-10T10:34:22.950361+0000","last_became_peered":"2024-05-10T10:34:22.950361+0000","last_unstale":"2024-05-10T10:34:22.954453+0000","last_undegraded":"2024-05-10T10:34:22.954453+0000","last_fullsized":"2024-05-10T10:34:22.954453+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:38.753738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960495+0000","last_change":"2024-05-10T10:34:21.938021+0000","last_active":"2024-05-10T10:34:22.960495+0000","last_peered":"2024-05-10T10:34:22.960495+0000","last_clean":"2024-05-10T10:34:22.960495+0000","last_became_active":"2024-05-10T10:34:21.937596+0000","last_became_peered":"2024-05-10T10:34:21.937596+0000","last_unstale":"2024-05-10T10:34:22.960495+0000","last_undegraded":"2024-05-10T10:34:22.960495+0000","last_fullsized":"2024-05-10T10:34:22.960495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:33.340892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955909+0000","last_change":"2024-05-10T10:34:22.952729+0000","last_active":"2024-05-10T10:34:22.955909+0000","last_peered":"2024-05-10T10:34:22.955909+0000","last_clean":"2024-05-10T10:34:22.955909+0000","last_became_active":"2024-05-10T10:34:22.952425+0000","last_became_peered":"2024-05-10T10:34:22.952425+0000","last_unstale":"2024-05-10T10:34:22.955909+0000","last_undegraded":"2024-05-10T10:34:22.955909+0000","last_fullsized":"2024-05-10T10:34:22.955909+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:14:25.324350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.940902+0000","last_change":"2024-05-10T10:34:21.930825+0000","last_active":"2024-05-10T10:34:22.940902+0000","last_peered":"2024-05-10T10:34:22.940902+0000","last_clean":"2024-05-10T10:34:22.940902+0000","last_became_active":"2024-05-10T10:34:21.930537+0000","last_became_peered":"2024-05-10T10:34:21.930537+0000","last_unstale":"2024-05-10T10:34:22.940902+0000","last_undegraded":"2024-05-10T10:34:22.940902+0000","last_fullsized":"2024-05-10T10:34:22.940902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:10.889571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960040+0000","last_change":"2024-05-10T10:34:22.953808+0000","last_active":"2024-05-10T10:34:22.960040+0000","last_peered":"2024-05-10T10:34:22.960040+0000","last_clean":"2024-05-10T10:34:22.960040+0000","last_became_active":"2024-05-10T10:34:22.950344+0000","last_became_peered":"2024-05-10T10:34:22.950344+0000","last_unstale":"2024-05-10T10:34:22.960040+0000","last_undegraded":"2024-05-10T10:34:22.960040+0000","last_fullsized":"2024-05-10T10:34:22.960040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:53:54.272898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.938164+0000","last_change":"2024-05-10T10:34:21.940012+0000","last_active":"2024-05-10T10:34:22.938164+0000","last_peered":"2024-05-10T10:34:22.938164+0000","last_clean":"2024-05-10T10:34:22.938164+0000","last_became_active":"2024-05-10T10:34:21.939678+0000","last_became_peered":"2024-05-10T10:34:21.939678+0000","last_unstale":"2024-05-10T10:34:22.938164+0000","last_undegraded":"2024-05-10T10:34:22.938164+0000","last_fullsized":"2024-05-10T10:34:22.938164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:26:22.232697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960616+0000","last_change":"2024-05-10T10:34:22.956959+0000","last_active":"2024-05-10T10:34:22.960616+0000","last_peered":"2024-05-10T10:34:22.960616+0000","last_clean":"2024-05-10T10:34:22.960616+0000","last_became_active":"2024-05-10T10:34:22.956610+0000","last_became_peered":"2024-05-10T10:34:22.956610+0000","last_unstale":"2024-05-10T10:34:22.960616+0000","last_undegraded":"2024-05-10T10:34:22.960616+0000","last_fullsized":"2024-05-10T10:34:22.960616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.940257+0000","last_change":"2024-05-10T10:34:21.941866+0000","last_active":"2024-05-10T10:34:22.940257+0000","last_peered":"2024-05-10T10:34:22.940257+0000","last_clean":"2024-05-10T10:34:22.940257+0000","last_became_active":"2024-05-10T10:34:21.941582+0000","last_became_peered":"2024-05-10T10:34:21.941582+0000","last_unstale":"2024-05-10T10:34:22.940257+0000","last_undegraded":"2024-05-10T10:34:22.940257+0000","last_fullsized":"2024-05-10T10:34:22.940257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:58.539394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.957104+0000","last_change":"2024-05-10T10:34:22.955983+0000","last_active":"2024-05-10T10:34:22.957104+0000","last_peered":"2024-05-10T10:34:22.957104+0000","last_clean":"2024-05-10T10:34:22.957104+0000","last_became_active":"2024-05-10T10:34:22.955621+0000","last_became_peered":"2024-05-10T10:34:22.955621+0000","last_unstale":"2024-05-10T10:34:22.957104+0000","last_undegraded":"2024-05-10T10:34:22.957104+0000","last_fullsized":"2024-05-10T10:34:22.957104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945574+0000","last_change":"2024-05-10T10:34:21.939706+0000","last_active":"2024-05-10T10:34:22.945574+0000","last_peered":"2024-05-10T10:34:22.945574+0000","last_clean":"2024-05-10T10:34:22.945574+0000","last_became_active":"2024-05-10T10:34:21.938741+0000","last_became_peered":"2024-05-10T10:34:21.938741+0000","last_unstale":"2024-05-10T10:34:22.945574+0000","last_undegraded":"2024-05-10T10:34:22.945574+0000","last_fullsized":"2024-05-10T10:34:22.945574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:35.640358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959033+0000","last_change":"2024-05-10T10:34:22.957086+0000","last_active":"2024-05-10T10:34:22.959033+0000","last_peered":"2024-05-10T10:34:22.959033+0000","last_clean":"2024-05-10T10:34:22.959033+0000","last_became_active":"2024-05-10T10:34:22.956471+0000","last_became_peered":"2024-05-10T10:34:22.956471+0000","last_unstale":"2024-05-10T10:34:22.959033+0000","last_undegraded":"2024-05-10T10:34:22.959033+0000","last_fullsized":"2024-05-10T10:34:22.959033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936967+0000","last_change":"2024-05-10T10:34:21.938215+0000","last_active":"2024-05-10T10:34:22.936967+0000","last_peered":"2024-05-10T10:34:22.936967+0000","last_clean":"2024-05-10T10:34:22.936967+0000","last_became_active":"2024-05-10T10:34:21.937820+0000","last_became_peered":"2024-05-10T10:34:21.937820+0000","last_unstale":"2024-05-10T10:34:22.936967+0000","last_undegraded":"2024-05-10T10:34:22.936967+0000","last_fullsized":"2024-05-10T10:34:22.936967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:44.800678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955271+0000","last_change":"2024-05-10T10:34:22.952057+0000","last_active":"2024-05-10T10:34:22.955271+0000","last_peered":"2024-05-10T10:34:22.955271+0000","last_clean":"2024-05-10T10:34:22.955271+0000","last_became_active":"2024-05-10T10:34:22.951800+0000","last_became_peered":"2024-05-10T10:34:22.951800+0000","last_unstale":"2024-05-10T10:34:22.955271+0000","last_undegraded":"2024-05-10T10:34:22.955271+0000","last_fullsized":"2024-05-10T10:34:22.955271+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:35.506101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.943244+0000","last_change":"2024-05-10T10:34:21.938066+0000","last_active":"2024-05-10T10:34:22.943244+0000","last_peered":"2024-05-10T10:34:22.943244+0000","last_clean":"2024-05-10T10:34:22.943244+0000","last_became_active":"2024-05-10T10:34:21.937775+0000","last_became_peered":"2024-05-10T10:34:21.937775+0000","last_unstale":"2024-05-10T10:34:22.943244+0000","last_undegraded":"2024-05-10T10:34:22.943244+0000","last_fullsized":"2024-05-10T10:34:22.943244+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:36.106128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960060+0000","last_change":"2024-05-10T10:34:22.957395+0000","last_active":"2024-05-10T10:34:22.960060+0000","last_peered":"2024-05-10T10:34:22.960060+0000","last_clean":"2024-05-10T10:34:22.960060+0000","last_became_active":"2024-05-10T10:34:22.957140+0000","last_became_peered":"2024-05-10T10:34:22.957140+0000","last_unstale":"2024-05-10T10:34:22.960060+0000","last_undegraded":"2024-05-10T10:34:22.960060+0000","last_fullsized":"2024-05-10T10:34:22.960060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:36:34.714282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936481+0000","last_change":"2024-05-10T10:34:21.948267+0000","last_active":"2024-05-10T10:34:22.936481+0000","last_peered":"2024-05-10T10:34:22.936481+0000","last_clean":"2024-05-10T10:34:22.936481+0000","last_became_active":"2024-05-10T10:34:21.947897+0000","last_became_peered":"2024-05-10T10:34:21.947897+0000","last_unstale":"2024-05-10T10:34:22.936481+0000","last_undegraded":"2024-05-10T10:34:22.936481+0000","last_fullsized":"2024-05-10T10:34:22.936481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:39.753429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":140,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.935917+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:34:22.935917+0000","last_peered":"2024-05-10T10:34:22.935917+0000","last_clean":"2024-05-10T10:34:22.935917+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:34:22.935917+0000","last_undegraded":"2024-05-10T10:34:22.935917+0000","last_fullsized":"2024-05-10T10:34:22.935917+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":751820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":161452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":168731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":169223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":161452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":161452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":168709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":759021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:34:26.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:34:27.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.134+0000 7f530a420640 1 Processor -- start 2024-05-10T10:34:27.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.134+0000 7f530a420640 1 -- start start 2024-05-10T10:34:27.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f530a420640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5304162940 0x7f53040b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f530a420640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53040d59d0 0x7f53040d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f530a420640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f53040b47a0 0x7f53040d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f530a420640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5304162d20 con 0x7f53040d59d0 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f530a420640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f53040d2d20 con 0x7f53040b47a0 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f530a420640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f53040d3000 con 0x7f5304162940 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f53037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53040d59d0 0x7f53040d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f53037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53040d59d0 0x7f53040d3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44386/0 (socket says 172.21.15.53:44386) 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.135+0000 7f53037fe640 1 -- 172.21.15.53:0/2451516330 learned_addr learned my addr 172.21.15.53:0/2451516330 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f53089bd640 1 --2- 172.21.15.53:0/2451516330 >> v2:172.21.15.135:3300/0 conn(0x7f53040b47a0 0x7f53040d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f5303fff640 1 --2- 172.21.15.53:0/2451516330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5304162940 0x7f53040b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f53037fe640 1 -- 172.21.15.53:0/2451516330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5304162940 msgr2=0x7f53040b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:27.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f53037fe640 1 --2- 172.21.15.53:0/2451516330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5304162940 0x7f53040b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f53037fe640 1 -- 172.21.15.53:0/2451516330 >> v2:172.21.15.135:3300/0 conn(0x7f53040b47a0 msgr2=0x7f53040d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:27.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f53037fe640 1 --2- 172.21.15.53:0/2451516330 >> v2:172.21.15.135:3300/0 conn(0x7f53040b47a0 0x7f53040d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f53037fe640 1 -- 172.21.15.53:0/2451516330 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53040d32e0 con 0x7f53040d59d0 2024-05-10T10:34:27.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.136+0000 7f53037fe640 1 --2- 172.21.15.53:0/2451516330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53040d59d0 0x7f53040d3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53040cf9a0 tx=0x7f52f802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb3be20a640ec2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f5302ffd640 1 -- 172.21.15.53:0/2451516330 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8037070 con 0x7f53040d59d0 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f5302ffd640 1 -- 172.21.15.53:0/2451516330 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f802ae00 con 0x7f53040d59d0 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f5302ffd640 1 -- 172.21.15.53:0/2451516330 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8032660 con 0x7f53040d59d0 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 -- 172.21.15.53:0/2451516330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53040d59d0 msgr2=0x7f53040d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 --2- 172.21.15.53:0/2451516330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53040d59d0 0x7f53040d3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53040cf9a0 tx=0x7f52f802a0b0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 -- 172.21.15.53:0/2451516330 shutdown_connections 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 --2- 172.21.15.53:0/2451516330 >> v2:172.21.15.135:3300/0 conn(0x7f53040b47a0 0x7f53040d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 --2- 172.21.15.53:0/2451516330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53040d59d0 0x7f53040d3670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 --2- 172.21.15.53:0/2451516330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5304162940 0x7f53040b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 -- 172.21.15.53:0/2451516330 >> 172.21.15.53:0/2451516330 conn(0x7f530409e380 msgr2=0x7f53040d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 -- 172.21.15.53:0/2451516330 shutdown_connections 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.137+0000 7f530a420640 1 -- 172.21.15.53:0/2451516330 wait complete. 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.138+0000 7f530a420640 1 Processor -- start 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f530a420640 1 -- start start 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f530a420640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53040b47a0 0x7f53040cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f530a420640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53040d59d0 0x7f53041540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f530a420640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304162940 0x7f53040b9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f530a420640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f53040d2390 con 0x7f5304162940 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f530a420640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53040cfc50 con 0x7f53040b47a0 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f530a420640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f53040d2510 con 0x7f53040d59d0 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f53089bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304162940 0x7f53040b9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f53089bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304162940 0x7f53040b9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44388/0 (socket says 172.21.15.53:44388) 2024-05-10T10:34:27.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.139+0000 7f53089bd640 1 -- 172.21.15.53:0/595384294 learned_addr learned my addr 172.21.15.53:0/595384294 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f53037fe640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53040d59d0 0x7f53041540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f5303fff640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53040b47a0 0x7f53040cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f53089bd640 1 -- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53040d59d0 msgr2=0x7f53041540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f53089bd640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53040d59d0 0x7f53041540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f53089bd640 1 -- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53040b47a0 msgr2=0x7f53040cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f53089bd640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53040b47a0 0x7f53040cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f53089bd640 1 -- 172.21.15.53:0/595384294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53040c9a90 con 0x7f5304162940 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f53089bd640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304162940 0x7f53040b9e90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52fc007d20 tx=0x7f52fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f5300ff9640 1 -- 172.21.15.53:0/595384294 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc014420 con 0x7f5304162940 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f5300ff9640 1 -- 172.21.15.53:0/595384294 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52fc019070 con 0x7f5304162940 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53040c9d70 con 0x7f5304162940 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f5300ff9640 1 -- 172.21.15.53:0/595384294 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc014820 con 0x7f5304162940 2024-05-10T10:34:27.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.140+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53040ca330 con 0x7f5304162940 2024-05-10T10:34:27.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.141+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530406a220 con 0x7f5304162940 2024-05-10T10:34:27.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.142+0000 7f5300ff9640 1 -- 172.21.15.53:0/595384294 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f52fc0149c0 con 0x7f5304162940 2024-05-10T10:34:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.142+0000 7f5300ff9640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52f00762d0 0x7f52f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.143+0000 7f5300ff9640 1 -- 172.21.15.53:0/595384294 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f52fc097ee0 con 0x7f5304162940 2024-05-10T10:34:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.143+0000 7f5303fff640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52f00762d0 0x7f52f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.143+0000 7f5303fff640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52f00762d0 0x7f52f0078790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52f4006fd0 tx=0x7f52f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.147+0000 7f5300ff9640 1 -- 172.21.15.53:0/595384294 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f52fc061740 con 0x7f5304162940 2024-05-10T10:34:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.332+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5304154c30 con 0x7f52f00762d0 2024-05-10T10:34:27.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.339+0000 7f5300ff9640 1 -- 172.21.15.53:0/595384294 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251874 (secure 0 0 0) 0x7f52f400f860 con 0x7f52f00762d0 2024-05-10T10:34:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:27.344 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:34:27.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52f00762d0 msgr2=0x7f52f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52f00762d0 0x7f52f0078790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52f4006fd0 tx=0x7f52f4008040 comp rx=0 tx=0).stop 2024-05-10T10:34:27.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304162940 msgr2=0x7f53040b9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304162940 0x7f53040b9e90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52fc007d20 tx=0x7f52fc00e4b0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 shutdown_connections 2024-05-10T10:34:27.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52f00762d0 0x7f52f0078790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304162940 0x7f53040b9e90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53040d59d0 0x7f53041540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 --2- 172.21.15.53:0/595384294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53040b47a0 0x7f53040cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 >> 172.21.15.53:0/595384294 conn(0x7f530409e380 msgr2=0x7f53040b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:27.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 shutdown_connections 2024-05-10T10:34:27.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.342+0000 7f530a420640 1 -- 172.21.15.53:0/595384294 wait complete. 2024-05-10T10:34:27.354 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-05-10T10:34:26.168337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237108,"kb_used_data":4272,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737612,"statfs":{"total":773094113280,"available":772851314688,"internally_reserved":0,"allocated":4374528,"data_stored":2501860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.248436"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959783+0000","last_change":"2024-05-10T10:34:22.945801+0000","last_active":"2024-05-10T10:34:22.959783+0000","last_peered":"2024-05-10T10:34:22.959783+0000","last_clean":"2024-05-10T10:34:22.959783+0000","last_became_active":"2024-05-10T10:34:22.945537+0000","last_became_peered":"2024-05-10T10:34:22.945537+0000","last_unstale":"2024-05-10T10:34:22.959783+0000","last_undegraded":"2024-05-10T10:34:22.959783+0000","last_fullsized":"2024-05-10T10:34:22.959783+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:34:35.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.967102+0000","last_change":"2024-05-10T10:34:22.962579+0000","last_active":"2024-05-10T10:34:22.967102+0000","last_peered":"2024-05-10T10:34:22.967102+0000","last_clean":"2024-05-10T10:34:22.967102+0000","last_became_active":"2024-05-10T10:34:22.959848+0000","last_became_peered":"2024-05-10T10:34:22.959848+0000","last_unstale":"2024-05-10T10:34:22.967102+0000","last_undegraded":"2024-05-10T10:34:22.967102+0000","last_fullsized":"2024-05-10T10:34:22.967102+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:00:18.302819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951038+0000","last_change":"2024-05-10T10:34:22.949313+0000","last_active":"2024-05-10T10:34:22.951038+0000","last_peered":"2024-05-10T10:34:22.951038+0000","last_clean":"2024-05-10T10:34:22.951038+0000","last_became_active":"2024-05-10T10:34:22.948996+0000","last_became_peered":"2024-05-10T10:34:22.948996+0000","last_unstale":"2024-05-10T10:34:22.951038+0000","last_undegraded":"2024-05-10T10:34:22.951038+0000","last_fullsized":"2024-05-10T10:34:22.951038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:40.757906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959454+0000","last_change":"2024-05-10T10:34:22.953297+0000","last_active":"2024-05-10T10:34:22.959454+0000","last_peered":"2024-05-10T10:34:22.959454+0000","last_clean":"2024-05-10T10:34:22.959454+0000","last_became_active":"2024-05-10T10:34:22.952991+0000","last_became_peered":"2024-05-10T10:34:22.952991+0000","last_unstale":"2024-05-10T10:34:22.959454+0000","last_undegraded":"2024-05-10T10:34:22.959454+0000","last_fullsized":"2024-05-10T10:34:22.959454+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:28:55.588056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963981+0000","last_change":"2024-05-10T10:34:22.959595+0000","last_active":"2024-05-10T10:34:22.963981+0000","last_peered":"2024-05-10T10:34:22.963981+0000","last_clean":"2024-05-10T10:34:22.963981+0000","last_became_active":"2024-05-10T10:34:22.958958+0000","last_became_peered":"2024-05-10T10:34:22.958958+0000","last_unstale":"2024-05-10T10:34:22.963981+0000","last_undegraded":"2024-05-10T10:34:22.963981+0000","last_fullsized":"2024-05-10T10:34:22.963981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:03:36.084771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963007+0000","last_change":"2024-05-10T10:34:22.961576+0000","last_active":"2024-05-10T10:34:22.963007+0000","last_peered":"2024-05-10T10:34:22.963007+0000","last_clean":"2024-05-10T10:34:22.963007+0000","last_became_active":"2024-05-10T10:34:22.961244+0000","last_became_peered":"2024-05-10T10:34:22.961244+0000","last_unstale":"2024-05-10T10:34:22.963007+0000","last_undegraded":"2024-05-10T10:34:22.963007+0000","last_fullsized":"2024-05-10T10:34:22.963007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966159+0000","last_change":"2024-05-10T10:34:22.961167+0000","last_active":"2024-05-10T10:34:22.966159+0000","last_peered":"2024-05-10T10:34:22.966159+0000","last_clean":"2024-05-10T10:34:22.966159+0000","last_became_active":"2024-05-10T10:34:22.960890+0000","last_became_peered":"2024-05-10T10:34:22.960890+0000","last_unstale":"2024-05-10T10:34:22.966159+0000","last_undegraded":"2024-05-10T10:34:22.966159+0000","last_fullsized":"2024-05-10T10:34:22.966159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:31.277579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959640+0000","last_change":"2024-05-10T10:34:22.955448+0000","last_active":"2024-05-10T10:34:22.959640+0000","last_peered":"2024-05-10T10:34:22.959640+0000","last_clean":"2024-05-10T10:34:22.959640+0000","last_became_active":"2024-05-10T10:34:22.955112+0000","last_became_peered":"2024-05-10T10:34:22.955112+0000","last_unstale":"2024-05-10T10:34:22.959640+0000","last_undegraded":"2024-05-10T10:34:22.959640+0000","last_fullsized":"2024-05-10T10:34:22.959640+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:31.277579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.953273+0000","last_change":"2024-05-10T10:34:22.951156+0000","last_active":"2024-05-10T10:34:22.953273+0000","last_peered":"2024-05-10T10:34:22.953273+0000","last_clean":"2024-05-10T10:34:22.953273+0000","last_became_active":"2024-05-10T10:34:22.950802+0000","last_became_peered":"2024-05-10T10:34:22.950802+0000","last_unstale":"2024-05-10T10:34:22.953273+0000","last_undegraded":"2024-05-10T10:34:22.953273+0000","last_fullsized":"2024-05-10T10:34:22.953273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.949518+0000","last_change":"2024-05-10T10:34:22.947395+0000","last_active":"2024-05-10T10:34:22.949518+0000","last_peered":"2024-05-10T10:34:22.949518+0000","last_clean":"2024-05-10T10:34:22.949518+0000","last_became_active":"2024-05-10T10:34:22.946849+0000","last_became_peered":"2024-05-10T10:34:22.946849+0000","last_unstale":"2024-05-10T10:34:22.949518+0000","last_undegraded":"2024-05-10T10:34:22.949518+0000","last_fullsized":"2024-05-10T10:34:22.949518+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:37.110604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950113+0000","last_change":"2024-05-10T10:34:22.948251+0000","last_active":"2024-05-10T10:34:22.950113+0000","last_peered":"2024-05-10T10:34:22.950113+0000","last_clean":"2024-05-10T10:34:22.950113+0000","last_became_active":"2024-05-10T10:34:22.947987+0000","last_became_peered":"2024-05-10T10:34:22.947987+0000","last_unstale":"2024-05-10T10:34:22.950113+0000","last_undegraded":"2024-05-10T10:34:22.950113+0000","last_fullsized":"2024-05-10T10:34:22.950113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:01.044100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.965702+0000","last_change":"2024-05-10T10:34:22.962882+0000","last_active":"2024-05-10T10:34:22.965702+0000","last_peered":"2024-05-10T10:34:22.965702+0000","last_clean":"2024-05-10T10:34:22.965702+0000","last_became_active":"2024-05-10T10:34:22.962553+0000","last_became_peered":"2024-05-10T10:34:22.962553+0000","last_unstale":"2024-05-10T10:34:22.965702+0000","last_undegraded":"2024-05-10T10:34:22.965702+0000","last_fullsized":"2024-05-10T10:34:22.965702+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.949689+0000","last_change":"2024-05-10T10:34:22.947582+0000","last_active":"2024-05-10T10:34:22.949689+0000","last_peered":"2024-05-10T10:34:22.949689+0000","last_clean":"2024-05-10T10:34:22.949689+0000","last_became_active":"2024-05-10T10:34:22.947290+0000","last_became_peered":"2024-05-10T10:34:22.947290+0000","last_unstale":"2024-05-10T10:34:22.949689+0000","last_undegraded":"2024-05-10T10:34:22.949689+0000","last_fullsized":"2024-05-10T10:34:22.949689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:34:35.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.958292+0000","last_change":"2024-05-10T10:34:22.953517+0000","last_active":"2024-05-10T10:34:22.958292+0000","last_peered":"2024-05-10T10:34:22.958292+0000","last_clean":"2024-05-10T10:34:22.958292+0000","last_became_active":"2024-05-10T10:34:22.953023+0000","last_became_peered":"2024-05-10T10:34:22.953023+0000","last_unstale":"2024-05-10T10:34:22.958292+0000","last_undegraded":"2024-05-10T10:34:22.958292+0000","last_fullsized":"2024-05-10T10:34:22.958292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966136+0000","last_change":"2024-05-10T10:34:22.959846+0000","last_active":"2024-05-10T10:34:22.966136+0000","last_peered":"2024-05-10T10:34:22.966136+0000","last_clean":"2024-05-10T10:34:22.966136+0000","last_became_active":"2024-05-10T10:34:22.959574+0000","last_became_peered":"2024-05-10T10:34:22.959574+0000","last_unstale":"2024-05-10T10:34:22.966136+0000","last_undegraded":"2024-05-10T10:34:22.966136+0000","last_fullsized":"2024-05-10T10:34:22.966136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:53:54.272898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951170+0000","last_change":"2024-05-10T10:34:22.947531+0000","last_active":"2024-05-10T10:34:22.951170+0000","last_peered":"2024-05-10T10:34:22.951170+0000","last_clean":"2024-05-10T10:34:22.951170+0000","last_became_active":"2024-05-10T10:34:22.947286+0000","last_became_peered":"2024-05-10T10:34:22.947286+0000","last_unstale":"2024-05-10T10:34:22.951170+0000","last_undegraded":"2024-05-10T10:34:22.951170+0000","last_fullsized":"2024-05-10T10:34:22.951170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955306+0000","last_change":"2024-05-10T10:34:22.952207+0000","last_active":"2024-05-10T10:34:22.955306+0000","last_peered":"2024-05-10T10:34:22.955306+0000","last_clean":"2024-05-10T10:34:22.955306+0000","last_became_active":"2024-05-10T10:34:22.951646+0000","last_became_peered":"2024-05-10T10:34:22.951646+0000","last_unstale":"2024-05-10T10:34:22.955306+0000","last_undegraded":"2024-05-10T10:34:22.955306+0000","last_fullsized":"2024-05-10T10:34:22.955306+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:37.110604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955205+0000","last_change":"2024-05-10T10:34:22.951407+0000","last_active":"2024-05-10T10:34:22.955205+0000","last_peered":"2024-05-10T10:34:22.955205+0000","last_clean":"2024-05-10T10:34:22.955205+0000","last_became_active":"2024-05-10T10:34:22.950728+0000","last_became_peered":"2024-05-10T10:34:22.950728+0000","last_unstale":"2024-05-10T10:34:22.955205+0000","last_undegraded":"2024-05-10T10:34:22.955205+0000","last_fullsized":"2024-05-10T10:34:22.955205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:26:23.237174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964476+0000","last_change":"2024-05-10T10:34:22.960531+0000","last_active":"2024-05-10T10:34:22.964476+0000","last_peered":"2024-05-10T10:34:22.964476+0000","last_clean":"2024-05-10T10:34:22.964476+0000","last_became_active":"2024-05-10T10:34:22.959148+0000","last_became_peered":"2024-05-10T10:34:22.959148+0000","last_unstale":"2024-05-10T10:34:22.964476+0000","last_undegraded":"2024-05-10T10:34:22.964476+0000","last_fullsized":"2024-05-10T10:34:22.964476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:46.071001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954480+0000","last_change":"2024-05-10T10:34:22.951236+0000","last_active":"2024-05-10T10:34:22.954480+0000","last_peered":"2024-05-10T10:34:22.954480+0000","last_clean":"2024-05-10T10:34:22.954480+0000","last_became_active":"2024-05-10T10:34:22.950788+0000","last_became_peered":"2024-05-10T10:34:22.950788+0000","last_unstale":"2024-05-10T10:34:22.954480+0000","last_undegraded":"2024-05-10T10:34:22.954480+0000","last_fullsized":"2024-05-10T10:34:22.954480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:05:51.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.949636+0000","last_change":"2024-05-10T10:34:22.947425+0000","last_active":"2024-05-10T10:34:22.949636+0000","last_peered":"2024-05-10T10:34:22.949636+0000","last_clean":"2024-05-10T10:34:22.949636+0000","last_became_active":"2024-05-10T10:34:22.946646+0000","last_became_peered":"2024-05-10T10:34:22.946646+0000","last_unstale":"2024-05-10T10:34:22.949636+0000","last_undegraded":"2024-05-10T10:34:22.949636+0000","last_fullsized":"2024-05-10T10:34:22.949636+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:14:25.324350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966334+0000","last_change":"2024-05-10T10:34:22.963155+0000","last_active":"2024-05-10T10:34:22.966334+0000","last_peered":"2024-05-10T10:34:22.966334+0000","last_clean":"2024-05-10T10:34:22.966334+0000","last_became_active":"2024-05-10T10:34:22.962830+0000","last_became_peered":"2024-05-10T10:34:22.962830+0000","last_unstale":"2024-05-10T10:34:22.966334+0000","last_undegraded":"2024-05-10T10:34:22.966334+0000","last_fullsized":"2024-05-10T10:34:22.966334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:35.506101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951151+0000","last_change":"2024-05-10T10:34:22.945113+0000","last_active":"2024-05-10T10:34:22.951151+0000","last_peered":"2024-05-10T10:34:22.951151+0000","last_clean":"2024-05-10T10:34:22.951151+0000","last_became_active":"2024-05-10T10:34:22.944812+0000","last_became_peered":"2024-05-10T10:34:22.944812+0000","last_unstale":"2024-05-10T10:34:22.951151+0000","last_undegraded":"2024-05-10T10:34:22.951151+0000","last_fullsized":"2024-05-10T10:34:22.951151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964902+0000","last_change":"2024-05-10T10:34:22.958535+0000","last_active":"2024-05-10T10:34:22.964902+0000","last_peered":"2024-05-10T10:34:22.964902+0000","last_clean":"2024-05-10T10:34:22.964902+0000","last_became_active":"2024-05-10T10:34:22.958217+0000","last_became_peered":"2024-05-10T10:34:22.958217+0000","last_unstale":"2024-05-10T10:34:22.964902+0000","last_undegraded":"2024-05-10T10:34:22.964902+0000","last_fullsized":"2024-05-10T10:34:22.964902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:00:37.648074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955388+0000","last_change":"2024-05-10T10:34:22.952256+0000","last_active":"2024-05-10T10:34:22.955388+0000","last_peered":"2024-05-10T10:34:22.955388+0000","last_clean":"2024-05-10T10:34:22.955388+0000","last_became_active":"2024-05-10T10:34:22.951880+0000","last_became_peered":"2024-05-10T10:34:22.951880+0000","last_unstale":"2024-05-10T10:34:22.955388+0000","last_undegraded":"2024-05-10T10:34:22.955388+0000","last_fullsized":"2024-05-10T10:34:22.955388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:44.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954080+0000","last_change":"2024-05-10T10:34:22.950997+0000","last_active":"2024-05-10T10:34:22.954080+0000","last_peered":"2024-05-10T10:34:22.954080+0000","last_clean":"2024-05-10T10:34:22.954080+0000","last_became_active":"2024-05-10T10:34:22.950702+0000","last_became_peered":"2024-05-10T10:34:22.950702+0000","last_unstale":"2024-05-10T10:34:22.954080+0000","last_undegraded":"2024-05-10T10:34:22.954080+0000","last_fullsized":"2024-05-10T10:34:22.954080+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:40.757906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964847+0000","last_change":"2024-05-10T10:34:22.961969+0000","last_active":"2024-05-10T10:34:22.964847+0000","last_peered":"2024-05-10T10:34:22.964847+0000","last_clean":"2024-05-10T10:34:22.964847+0000","last_became_active":"2024-05-10T10:34:22.961424+0000","last_became_peered":"2024-05-10T10:34:22.961424+0000","last_unstale":"2024-05-10T10:34:22.964847+0000","last_undegraded":"2024-05-10T10:34:22.964847+0000","last_fullsized":"2024-05-10T10:34:22.964847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:25.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.969503+0000","last_change":"2024-05-10T10:34:22.967078+0000","last_active":"2024-05-10T10:34:22.969503+0000","last_peered":"2024-05-10T10:34:22.969503+0000","last_clean":"2024-05-10T10:34:22.969503+0000","last_became_active":"2024-05-10T10:34:22.966799+0000","last_became_peered":"2024-05-10T10:34:22.966799+0000","last_unstale":"2024-05-10T10:34:22.969503+0000","last_undegraded":"2024-05-10T10:34:22.969503+0000","last_fullsized":"2024-05-10T10:34:22.969503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:47:58.867430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955090+0000","last_change":"2024-05-10T10:34:22.954373+0000","last_active":"2024-05-10T10:34:22.955090+0000","last_peered":"2024-05-10T10:34:22.955090+0000","last_clean":"2024-05-10T10:34:22.955090+0000","last_became_active":"2024-05-10T10:34:22.953436+0000","last_became_peered":"2024-05-10T10:34:22.953436+0000","last_unstale":"2024-05-10T10:34:22.955090+0000","last_undegraded":"2024-05-10T10:34:22.955090+0000","last_fullsized":"2024-05-10T10:34:22.955090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:59.543871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.962692+0000","last_change":"2024-05-10T10:34:22.961643+0000","last_active":"2024-05-10T10:34:22.962692+0000","last_peered":"2024-05-10T10:34:22.962692+0000","last_clean":"2024-05-10T10:34:22.962692+0000","last_became_active":"2024-05-10T10:34:22.961356+0000","last_became_peered":"2024-05-10T10:34:22.961356+0000","last_unstale":"2024-05-10T10:34:22.962692+0000","last_undegraded":"2024-05-10T10:34:22.962692+0000","last_fullsized":"2024-05-10T10:34:22.962692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:45.805155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963979+0000","last_change":"2024-05-10T10:34:22.959527+0000","last_active":"2024-05-10T10:34:22.963979+0000","last_peered":"2024-05-10T10:34:22.963979+0000","last_clean":"2024-05-10T10:34:22.963979+0000","last_became_active":"2024-05-10T10:34:22.959229+0000","last_became_peered":"2024-05-10T10:34:22.959229+0000","last_unstale":"2024-05-10T10:34:22.963979+0000","last_undegraded":"2024-05-10T10:34:22.963979+0000","last_fullsized":"2024-05-10T10:34:22.963979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:01.044100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.961733+0000","last_change":"2024-05-10T10:34:22.960222+0000","last_active":"2024-05-10T10:34:22.961733+0000","last_peered":"2024-05-10T10:34:22.961733+0000","last_clean":"2024-05-10T10:34:22.961733+0000","last_became_active":"2024-05-10T10:34:22.959929+0000","last_became_peered":"2024-05-10T10:34:22.959929+0000","last_unstale":"2024-05-10T10:34:22.961733+0000","last_undegraded":"2024-05-10T10:34:22.961733+0000","last_fullsized":"2024-05-10T10:34:22.961733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954584+0000","last_change":"2024-05-10T10:34:22.950597+0000","last_active":"2024-05-10T10:34:22.954584+0000","last_peered":"2024-05-10T10:34:22.954584+0000","last_clean":"2024-05-10T10:34:22.954584+0000","last_became_active":"2024-05-10T10:34:22.950209+0000","last_became_peered":"2024-05-10T10:34:22.950209+0000","last_unstale":"2024-05-10T10:34:22.954584+0000","last_undegraded":"2024-05-10T10:34:22.954584+0000","last_fullsized":"2024-05-10T10:34:22.954584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966479+0000","last_change":"2024-05-10T10:34:21.939198+0000","last_active":"2024-05-10T10:34:22.966479+0000","last_peered":"2024-05-10T10:34:22.966479+0000","last_clean":"2024-05-10T10:34:22.966479+0000","last_became_active":"2024-05-10T10:34:21.938678+0000","last_became_peered":"2024-05-10T10:34:21.938678+0000","last_unstale":"2024-05-10T10:34:22.966479+0000","last_undegraded":"2024-05-10T10:34:22.966479+0000","last_fullsized":"2024-05-10T10:34:22.966479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:04:43.544910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954939+0000","last_change":"2024-05-10T10:34:22.952036+0000","last_active":"2024-05-10T10:34:22.954939+0000","last_peered":"2024-05-10T10:34:22.954939+0000","last_clean":"2024-05-10T10:34:22.954939+0000","last_became_active":"2024-05-10T10:34:22.951769+0000","last_became_peered":"2024-05-10T10:34:22.951769+0000","last_unstale":"2024-05-10T10:34:22.954939+0000","last_undegraded":"2024-05-10T10:34:22.954939+0000","last_fullsized":"2024-05-10T10:34:22.954939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:18.194365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963571+0000","last_change":"2024-05-10T10:34:21.943676+0000","last_active":"2024-05-10T10:34:22.963571+0000","last_peered":"2024-05-10T10:34:22.963571+0000","last_clean":"2024-05-10T10:34:22.963571+0000","last_became_active":"2024-05-10T10:34:21.943365+0000","last_became_peered":"2024-05-10T10:34:21.943365+0000","last_unstale":"2024-05-10T10:34:22.963571+0000","last_undegraded":"2024-05-10T10:34:22.963571+0000","last_fullsized":"2024-05-10T10:34:22.963571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:43.624004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.957588+0000","last_change":"2024-05-10T10:34:22.953375+0000","last_active":"2024-05-10T10:34:22.957588+0000","last_peered":"2024-05-10T10:34:22.957588+0000","last_clean":"2024-05-10T10:34:22.957588+0000","last_became_active":"2024-05-10T10:34:22.953047+0000","last_became_peered":"2024-05-10T10:34:22.953047+0000","last_unstale":"2024-05-10T10:34:22.957588+0000","last_undegraded":"2024-05-10T10:34:22.957588+0000","last_fullsized":"2024-05-10T10:34:22.957588+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:18.194365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.967147+0000","last_change":"2024-05-10T10:34:21.941982+0000","last_active":"2024-05-10T10:34:22.967147+0000","last_peered":"2024-05-10T10:34:22.967147+0000","last_clean":"2024-05-10T10:34:22.967147+0000","last_became_active":"2024-05-10T10:34:21.941554+0000","last_became_peered":"2024-05-10T10:34:21.941554+0000","last_unstale":"2024-05-10T10:34:22.967147+0000","last_undegraded":"2024-05-10T10:34:22.967147+0000","last_fullsized":"2024-05-10T10:34:22.967147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:24.394698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959824+0000","last_change":"2024-05-10T10:34:22.956620+0000","last_active":"2024-05-10T10:34:22.959824+0000","last_peered":"2024-05-10T10:34:22.959824+0000","last_clean":"2024-05-10T10:34:22.959824+0000","last_became_active":"2024-05-10T10:34:22.956297+0000","last_became_peered":"2024-05-10T10:34:22.956297+0000","last_unstale":"2024-05-10T10:34:22.959824+0000","last_undegraded":"2024-05-10T10:34:22.959824+0000","last_fullsized":"2024-05-10T10:34:22.959824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:47:58.867430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.968438+0000","last_change":"2024-05-10T10:34:21.938057+0000","last_active":"2024-05-10T10:34:22.968438+0000","last_peered":"2024-05-10T10:34:22.968438+0000","last_clean":"2024-05-10T10:34:22.968438+0000","last_became_active":"2024-05-10T10:34:21.937804+0000","last_became_peered":"2024-05-10T10:34:21.937804+0000","last_unstale":"2024-05-10T10:34:22.968438+0000","last_undegraded":"2024-05-10T10:34:22.968438+0000","last_fullsized":"2024-05-10T10:34:22.968438+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:58.539394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.951179+0000","last_change":"2024-05-10T10:34:22.947366+0000","last_active":"2024-05-10T10:34:22.951179+0000","last_peered":"2024-05-10T10:34:22.951179+0000","last_clean":"2024-05-10T10:34:22.951179+0000","last_became_active":"2024-05-10T10:34:22.944886+0000","last_became_peered":"2024-05-10T10:34:22.944886+0000","last_unstale":"2024-05-10T10:34:22.951179+0000","last_undegraded":"2024-05-10T10:34:22.951179+0000","last_fullsized":"2024-05-10T10:34:22.951179+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:38.753738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963534+0000","last_change":"2024-05-10T10:34:21.939856+0000","last_active":"2024-05-10T10:34:22.963534+0000","last_peered":"2024-05-10T10:34:22.963534+0000","last_clean":"2024-05-10T10:34:22.963534+0000","last_became_active":"2024-05-10T10:34:21.939559+0000","last_became_peered":"2024-05-10T10:34:21.939559+0000","last_unstale":"2024-05-10T10:34:22.963534+0000","last_undegraded":"2024-05-10T10:34:22.963534+0000","last_fullsized":"2024-05-10T10:34:22.963534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:17.198949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955802+0000","last_change":"2024-05-10T10:34:22.951040+0000","last_active":"2024-05-10T10:34:22.955802+0000","last_peered":"2024-05-10T10:34:22.955802+0000","last_clean":"2024-05-10T10:34:22.955802+0000","last_became_active":"2024-05-10T10:34:22.950137+0000","last_became_peered":"2024-05-10T10:34:22.950137+0000","last_unstale":"2024-05-10T10:34:22.955802+0000","last_undegraded":"2024-05-10T10:34:22.955802+0000","last_fullsized":"2024-05-10T10:34:22.955802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:38.542678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963335+0000","last_change":"2024-05-10T10:34:21.949021+0000","last_active":"2024-05-10T10:34:22.963335+0000","last_peered":"2024-05-10T10:34:22.963335+0000","last_clean":"2024-05-10T10:34:22.963335+0000","last_became_active":"2024-05-10T10:34:21.948782+0000","last_became_peered":"2024-05-10T10:34:21.948782+0000","last_unstale":"2024-05-10T10:34:22.963335+0000","last_undegraded":"2024-05-10T10:34:22.963335+0000","last_fullsized":"2024-05-10T10:34:22.963335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:37.749261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950333+0000","last_change":"2024-05-10T10:34:22.949084+0000","last_active":"2024-05-10T10:34:22.950333+0000","last_peered":"2024-05-10T10:34:22.950333+0000","last_clean":"2024-05-10T10:34:22.950333+0000","last_became_active":"2024-05-10T10:34:22.948827+0000","last_became_peered":"2024-05-10T10:34:22.948827+0000","last_unstale":"2024-05-10T10:34:22.950333+0000","last_undegraded":"2024-05-10T10:34:22.950333+0000","last_fullsized":"2024-05-10T10:34:22.950333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:40.757906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.938267+0000","last_change":"2024-05-10T10:34:21.939497+0000","last_active":"2024-05-10T10:34:22.938267+0000","last_peered":"2024-05-10T10:34:22.938267+0000","last_clean":"2024-05-10T10:34:22.938267+0000","last_became_active":"2024-05-10T10:34:21.939228+0000","last_became_peered":"2024-05-10T10:34:21.939228+0000","last_unstale":"2024-05-10T10:34:22.938267+0000","last_undegraded":"2024-05-10T10:34:22.938267+0000","last_fullsized":"2024-05-10T10:34:22.938267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:44.800678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.961304+0000","last_change":"2024-05-10T10:34:22.959971+0000","last_active":"2024-05-10T10:34:22.961304+0000","last_peered":"2024-05-10T10:34:22.961304+0000","last_clean":"2024-05-10T10:34:22.961304+0000","last_became_active":"2024-05-10T10:34:22.959314+0000","last_became_peered":"2024-05-10T10:34:22.959314+0000","last_unstale":"2024-05-10T10:34:22.961304+0000","last_undegraded":"2024-05-10T10:34:22.961304+0000","last_fullsized":"2024-05-10T10:34:22.961304+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:13:30.812601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936785+0000","last_change":"2024-05-10T10:34:21.939449+0000","last_active":"2024-05-10T10:34:22.936785+0000","last_peered":"2024-05-10T10:34:22.936785+0000","last_clean":"2024-05-10T10:34:22.936785+0000","last_became_active":"2024-05-10T10:34:21.939184+0000","last_became_peered":"2024-05-10T10:34:21.939184+0000","last_unstale":"2024-05-10T10:34:22.936785+0000","last_undegraded":"2024-05-10T10:34:22.936785+0000","last_fullsized":"2024-05-10T10:34:22.936785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:43.624004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959427+0000","last_change":"2024-05-10T10:34:22.952546+0000","last_active":"2024-05-10T10:34:22.959427+0000","last_peered":"2024-05-10T10:34:22.959427+0000","last_clean":"2024-05-10T10:34:22.959427+0000","last_became_active":"2024-05-10T10:34:22.952246+0000","last_became_peered":"2024-05-10T10:34:22.952246+0000","last_unstale":"2024-05-10T10:34:22.959427+0000","last_undegraded":"2024-05-10T10:34:22.959427+0000","last_fullsized":"2024-05-10T10:34:22.959427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:03.869048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.939522+0000","last_change":"2024-05-10T10:34:21.941345+0000","last_active":"2024-05-10T10:34:22.939522+0000","last_peered":"2024-05-10T10:34:22.939522+0000","last_clean":"2024-05-10T10:34:22.939522+0000","last_became_active":"2024-05-10T10:34:21.938931+0000","last_became_peered":"2024-05-10T10:34:21.938931+0000","last_unstale":"2024-05-10T10:34:22.939522+0000","last_undegraded":"2024-05-10T10:34:22.939522+0000","last_fullsized":"2024-05-10T10:34:22.939522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:34.501624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955932+0000","last_change":"2024-05-10T10:34:22.952832+0000","last_active":"2024-05-10T10:34:22.955932+0000","last_peered":"2024-05-10T10:34:22.955932+0000","last_clean":"2024-05-10T10:34:22.955932+0000","last_became_active":"2024-05-10T10:34:22.952551+0000","last_became_peered":"2024-05-10T10:34:22.952551+0000","last_unstale":"2024-05-10T10:34:22.955932+0000","last_undegraded":"2024-05-10T10:34:22.955932+0000","last_fullsized":"2024-05-10T10:34:22.955932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:34:35.159467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.940703+0000","last_change":"2024-05-10T10:34:21.936303+0000","last_active":"2024-05-10T10:34:22.940703+0000","last_peered":"2024-05-10T10:34:22.940703+0000","last_clean":"2024-05-10T10:34:22.940703+0000","last_became_active":"2024-05-10T10:34:21.933569+0000","last_became_peered":"2024-05-10T10:34:21.933569+0000","last_unstale":"2024-05-10T10:34:22.940703+0000","last_undegraded":"2024-05-10T10:34:22.940703+0000","last_fullsized":"2024-05-10T10:34:22.940703+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:39.753429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950763+0000","last_change":"2024-05-10T10:34:22.947178+0000","last_active":"2024-05-10T10:34:22.950763+0000","last_peered":"2024-05-10T10:34:22.950763+0000","last_clean":"2024-05-10T10:34:22.950763+0000","last_became_active":"2024-05-10T10:34:22.946906+0000","last_became_peered":"2024-05-10T10:34:22.946906+0000","last_unstale":"2024-05-10T10:34:22.950763+0000","last_undegraded":"2024-05-10T10:34:22.950763+0000","last_fullsized":"2024-05-10T10:34:22.950763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:25.399175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963773+0000","last_change":"2024-05-10T10:34:21.937948+0000","last_active":"2024-05-10T10:34:22.963773+0000","last_peered":"2024-05-10T10:34:22.963773+0000","last_clean":"2024-05-10T10:34:22.963773+0000","last_became_active":"2024-05-10T10:34:21.937589+0000","last_became_peered":"2024-05-10T10:34:21.937589+0000","last_unstale":"2024-05-10T10:34:22.963773+0000","last_undegraded":"2024-05-10T10:34:22.963773+0000","last_fullsized":"2024-05-10T10:34:22.963773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:17.189889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966261+0000","last_change":"2024-05-10T10:34:22.960422+0000","last_active":"2024-05-10T10:34:22.966261+0000","last_peered":"2024-05-10T10:34:22.966261+0000","last_clean":"2024-05-10T10:34:22.966261+0000","last_became_active":"2024-05-10T10:34:22.960132+0000","last_became_peered":"2024-05-10T10:34:22.960132+0000","last_unstale":"2024-05-10T10:34:22.966261+0000","last_undegraded":"2024-05-10T10:34:22.966261+0000","last_fullsized":"2024-05-10T10:34:22.966261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:34.345368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.976847+0000","last_change":"2024-05-10T10:34:21.948576+0000","last_active":"2024-05-10T10:34:22.976847+0000","last_peered":"2024-05-10T10:34:22.976847+0000","last_clean":"2024-05-10T10:34:22.976847+0000","last_became_active":"2024-05-10T10:34:21.948217+0000","last_became_peered":"2024-05-10T10:34:21.948217+0000","last_unstale":"2024-05-10T10:34:22.976847+0000","last_undegraded":"2024-05-10T10:34:22.976847+0000","last_fullsized":"2024-05-10T10:34:22.976847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:42:00.039624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.953489+0000","last_change":"2024-05-10T10:34:22.951035+0000","last_active":"2024-05-10T10:34:22.953489+0000","last_peered":"2024-05-10T10:34:22.953489+0000","last_clean":"2024-05-10T10:34:22.953489+0000","last_became_active":"2024-05-10T10:34:22.950382+0000","last_became_peered":"2024-05-10T10:34:22.950382+0000","last_unstale":"2024-05-10T10:34:22.953489+0000","last_undegraded":"2024-05-10T10:34:22.953489+0000","last_fullsized":"2024-05-10T10:34:22.953489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:18.194365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.967853+0000","last_change":"2024-05-10T10:34:21.937199+0000","last_active":"2024-05-10T10:34:22.967853+0000","last_peered":"2024-05-10T10:34:22.967853+0000","last_clean":"2024-05-10T10:34:22.967853+0000","last_became_active":"2024-05-10T10:34:21.936927+0000","last_became_peered":"2024-05-10T10:34:21.936927+0000","last_unstale":"2024-05-10T10:34:22.967853+0000","last_undegraded":"2024-05-10T10:34:22.967853+0000","last_fullsized":"2024-05-10T10:34:22.967853+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:14:24.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959688+0000","last_change":"2024-05-10T10:34:22.952267+0000","last_active":"2024-05-10T10:34:22.959688+0000","last_peered":"2024-05-10T10:34:22.959688+0000","last_clean":"2024-05-10T10:34:22.959688+0000","last_became_active":"2024-05-10T10:34:22.952003+0000","last_became_peered":"2024-05-10T10:34:22.952003+0000","last_unstale":"2024-05-10T10:34:22.959688+0000","last_undegraded":"2024-05-10T10:34:22.959688+0000","last_fullsized":"2024-05-10T10:34:22.959688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:36:34.714282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.938210+0000","last_change":"2024-05-10T10:34:21.940676+0000","last_active":"2024-05-10T10:34:22.938210+0000","last_peered":"2024-05-10T10:34:22.938210+0000","last_clean":"2024-05-10T10:34:22.938210+0000","last_became_active":"2024-05-10T10:34:21.938910+0000","last_became_peered":"2024-05-10T10:34:21.938910+0000","last_unstale":"2024-05-10T10:34:22.938210+0000","last_undegraded":"2024-05-10T10:34:22.938210+0000","last_fullsized":"2024-05-10T10:34:22.938210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:58.539394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963912+0000","last_change":"2024-05-10T10:34:22.959723+0000","last_active":"2024-05-10T10:34:22.963912+0000","last_peered":"2024-05-10T10:34:22.963912+0000","last_clean":"2024-05-10T10:34:22.963912+0000","last_became_active":"2024-05-10T10:34:22.959441+0000","last_became_peered":"2024-05-10T10:34:22.959441+0000","last_unstale":"2024-05-10T10:34:22.963912+0000","last_undegraded":"2024-05-10T10:34:22.963912+0000","last_fullsized":"2024-05-10T10:34:22.963912+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:38.542678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945548+0000","last_change":"2024-05-10T10:34:21.938877+0000","last_active":"2024-05-10T10:34:22.945548+0000","last_peered":"2024-05-10T10:34:22.945548+0000","last_clean":"2024-05-10T10:34:22.945548+0000","last_became_active":"2024-05-10T10:34:21.938552+0000","last_became_peered":"2024-05-10T10:34:21.938552+0000","last_unstale":"2024-05-10T10:34:22.945548+0000","last_undegraded":"2024-05-10T10:34:22.945548+0000","last_fullsized":"2024-05-10T10:34:22.945548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:38:13.381884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960247+0000","last_change":"2024-05-10T10:34:22.951954+0000","last_active":"2024-05-10T10:34:22.960247+0000","last_peered":"2024-05-10T10:34:22.960247+0000","last_clean":"2024-05-10T10:34:22.960247+0000","last_became_active":"2024-05-10T10:34:22.951656+0000","last_became_peered":"2024-05-10T10:34:22.951656+0000","last_unstale":"2024-05-10T10:34:22.960247+0000","last_undegraded":"2024-05-10T10:34:22.960247+0000","last_fullsized":"2024-05-10T10:34:22.960247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.110682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.969371+0000","last_change":"2024-05-10T10:34:21.949056+0000","last_active":"2024-05-10T10:34:22.969371+0000","last_peered":"2024-05-10T10:34:22.969371+0000","last_clean":"2024-05-10T10:34:22.969371+0000","last_became_active":"2024-05-10T10:34:21.948803+0000","last_became_peered":"2024-05-10T10:34:21.948803+0000","last_unstale":"2024-05-10T10:34:22.969371+0000","last_undegraded":"2024-05-10T10:34:22.969371+0000","last_fullsized":"2024-05-10T10:34:22.969371+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:24.394698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.968307+0000","last_change":"2024-05-10T10:34:22.965209+0000","last_active":"2024-05-10T10:34:22.968307+0000","last_peered":"2024-05-10T10:34:22.968307+0000","last_clean":"2024-05-10T10:34:22.968307+0000","last_became_active":"2024-05-10T10:34:22.963079+0000","last_became_peered":"2024-05-10T10:34:22.963079+0000","last_unstale":"2024-05-10T10:34:22.968307+0000","last_undegraded":"2024-05-10T10:34:22.968307+0000","last_fullsized":"2024-05-10T10:34:22.968307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:18:02.863461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963887+0000","last_change":"2024-05-10T10:34:21.939434+0000","last_active":"2024-05-10T10:34:22.963887+0000","last_peered":"2024-05-10T10:34:22.963887+0000","last_clean":"2024-05-10T10:34:22.963887+0000","last_became_active":"2024-05-10T10:34:21.939172+0000","last_became_peered":"2024-05-10T10:34:21.939172+0000","last_unstale":"2024-05-10T10:34:22.963887+0000","last_undegraded":"2024-05-10T10:34:22.963887+0000","last_fullsized":"2024-05-10T10:34:22.963887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:46.129934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.948387+0000","last_change":"2024-05-10T10:34:22.944487+0000","last_active":"2024-05-10T10:34:22.948387+0000","last_peered":"2024-05-10T10:34:22.948387+0000","last_clean":"2024-05-10T10:34:22.948387+0000","last_became_active":"2024-05-10T10:34:22.944184+0000","last_became_peered":"2024-05-10T10:34:22.944184+0000","last_unstale":"2024-05-10T10:34:22.948387+0000","last_undegraded":"2024-05-10T10:34:22.948387+0000","last_fullsized":"2024-05-10T10:34:22.948387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:35.506101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.963135+0000","last_change":"2024-05-10T10:34:21.945201+0000","last_active":"2024-05-10T10:34:22.963135+0000","last_peered":"2024-05-10T10:34:22.963135+0000","last_clean":"2024-05-10T10:34:22.963135+0000","last_became_active":"2024-05-10T10:34:21.944968+0000","last_became_peered":"2024-05-10T10:34:21.944968+0000","last_unstale":"2024-05-10T10:34:22.963135+0000","last_undegraded":"2024-05-10T10:34:22.963135+0000","last_fullsized":"2024-05-10T10:34:22.963135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:24.394698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.946875+0000","last_change":"2024-05-10T10:34:22.941905+0000","last_active":"2024-05-10T10:34:22.946875+0000","last_peered":"2024-05-10T10:34:22.946875+0000","last_clean":"2024-05-10T10:34:22.946875+0000","last_became_active":"2024-05-10T10:34:22.941646+0000","last_became_peered":"2024-05-10T10:34:22.941646+0000","last_unstale":"2024-05-10T10:34:22.946875+0000","last_undegraded":"2024-05-10T10:34:22.946875+0000","last_fullsized":"2024-05-10T10:34:22.946875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:38.753738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964190+0000","last_change":"2024-05-10T10:34:21.948228+0000","last_active":"2024-05-10T10:34:22.964190+0000","last_peered":"2024-05-10T10:34:22.964190+0000","last_clean":"2024-05-10T10:34:22.964190+0000","last_became_active":"2024-05-10T10:34:21.947889+0000","last_became_peered":"2024-05-10T10:34:21.947889+0000","last_unstale":"2024-05-10T10:34:22.964190+0000","last_undegraded":"2024-05-10T10:34:22.964190+0000","last_fullsized":"2024-05-10T10:34:22.964190+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:20.566743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.966537+0000","last_change":"2024-05-10T10:34:22.964710+0000","last_active":"2024-05-10T10:34:22.966537+0000","last_peered":"2024-05-10T10:34:22.966537+0000","last_clean":"2024-05-10T10:34:22.966537+0000","last_became_active":"2024-05-10T10:34:22.964225+0000","last_became_peered":"2024-05-10T10:34:22.964225+0000","last_unstale":"2024-05-10T10:34:22.966537+0000","last_undegraded":"2024-05-10T10:34:22.966537+0000","last_fullsized":"2024-05-10T10:34:22.966537+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:58.457557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.937616+0000","last_change":"2024-05-10T10:34:21.943638+0000","last_active":"2024-05-10T10:34:22.937616+0000","last_peered":"2024-05-10T10:34:22.937616+0000","last_clean":"2024-05-10T10:34:22.937616+0000","last_became_active":"2024-05-10T10:34:21.943335+0000","last_became_peered":"2024-05-10T10:34:21.943335+0000","last_unstale":"2024-05-10T10:34:22.937616+0000","last_undegraded":"2024-05-10T10:34:22.937616+0000","last_fullsized":"2024-05-10T10:34:22.937616+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:43.624004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.969312+0000","last_change":"2024-05-10T10:34:22.962472+0000","last_active":"2024-05-10T10:34:22.969312+0000","last_peered":"2024-05-10T10:34:22.969312+0000","last_clean":"2024-05-10T10:34:22.969312+0000","last_became_active":"2024-05-10T10:34:22.960525+0000","last_became_peered":"2024-05-10T10:34:22.960525+0000","last_unstale":"2024-05-10T10:34:22.969312+0000","last_undegraded":"2024-05-10T10:34:22.969312+0000","last_fullsized":"2024-05-10T10:34:22.969312+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.110682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.964521+0000","last_change":"2024-05-10T10:34:21.937201+0000","last_active":"2024-05-10T10:34:22.964521+0000","last_peered":"2024-05-10T10:34:22.964521+0000","last_clean":"2024-05-10T10:34:22.964521+0000","last_became_active":"2024-05-10T10:34:21.936888+0000","last_became_peered":"2024-05-10T10:34:21.936888+0000","last_unstale":"2024-05-10T10:34:22.964521+0000","last_undegraded":"2024-05-10T10:34:22.964521+0000","last_fullsized":"2024-05-10T10:34:22.964521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:08:11.892379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.950593+0000","last_change":"2024-05-10T10:34:22.946531+0000","last_active":"2024-05-10T10:34:22.950593+0000","last_peered":"2024-05-10T10:34:22.950593+0000","last_clean":"2024-05-10T10:34:22.950593+0000","last_became_active":"2024-05-10T10:34:22.946269+0000","last_became_peered":"2024-05-10T10:34:22.946269+0000","last_unstale":"2024-05-10T10:34:22.950593+0000","last_undegraded":"2024-05-10T10:34:22.950593+0000","last_fullsized":"2024-05-10T10:34:22.950593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:45.805155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945681+0000","last_change":"2024-05-10T10:34:21.935279+0000","last_active":"2024-05-10T10:34:22.945681+0000","last_peered":"2024-05-10T10:34:22.945681+0000","last_clean":"2024-05-10T10:34:22.945681+0000","last_became_active":"2024-05-10T10:34:21.934970+0000","last_became_peered":"2024-05-10T10:34:21.934970+0000","last_unstale":"2024-05-10T10:34:22.945681+0000","last_undegraded":"2024-05-10T10:34:22.945681+0000","last_fullsized":"2024-05-10T10:34:22.945681+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:47:34.131705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955891+0000","last_change":"2024-05-10T10:34:22.951580+0000","last_active":"2024-05-10T10:34:22.955891+0000","last_peered":"2024-05-10T10:34:22.955891+0000","last_clean":"2024-05-10T10:34:22.955891+0000","last_became_active":"2024-05-10T10:34:22.951307+0000","last_became_peered":"2024-05-10T10:34:22.951307+0000","last_unstale":"2024-05-10T10:34:22.955891+0000","last_undegraded":"2024-05-10T10:34:22.955891+0000","last_fullsized":"2024-05-10T10:34:22.955891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.110682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936745+0000","last_change":"2024-05-10T10:34:21.948225+0000","last_active":"2024-05-10T10:34:22.936745+0000","last_peered":"2024-05-10T10:34:22.936745+0000","last_clean":"2024-05-10T10:34:22.936745+0000","last_became_active":"2024-05-10T10:34:21.947894+0000","last_became_peered":"2024-05-10T10:34:21.947894+0000","last_unstale":"2024-05-10T10:34:22.936745+0000","last_undegraded":"2024-05-10T10:34:22.936745+0000","last_fullsized":"2024-05-10T10:34:22.936745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:46:17.189889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960225+0000","last_change":"2024-05-10T10:34:22.948513+0000","last_active":"2024-05-10T10:34:22.960225+0000","last_peered":"2024-05-10T10:34:22.960225+0000","last_clean":"2024-05-10T10:34:22.960225+0000","last_became_active":"2024-05-10T10:34:22.948186+0000","last_became_peered":"2024-05-10T10:34:22.948186+0000","last_unstale":"2024-05-10T10:34:22.960225+0000","last_undegraded":"2024-05-10T10:34:22.960225+0000","last_fullsized":"2024-05-10T10:34:22.960225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:27:38.542678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945559+0000","last_change":"2024-05-10T10:34:21.935491+0000","last_active":"2024-05-10T10:34:22.945559+0000","last_peered":"2024-05-10T10:34:22.945559+0000","last_clean":"2024-05-10T10:34:22.945559+0000","last_became_active":"2024-05-10T10:34:21.935236+0000","last_became_peered":"2024-05-10T10:34:21.935236+0000","last_unstale":"2024-05-10T10:34:22.945559+0000","last_undegraded":"2024-05-10T10:34:22.945559+0000","last_fullsized":"2024-05-10T10:34:22.945559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:44.846873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.954453+0000","last_change":"2024-05-10T10:34:22.950668+0000","last_active":"2024-05-10T10:34:22.954453+0000","last_peered":"2024-05-10T10:34:22.954453+0000","last_clean":"2024-05-10T10:34:22.954453+0000","last_became_active":"2024-05-10T10:34:22.950361+0000","last_became_peered":"2024-05-10T10:34:22.950361+0000","last_unstale":"2024-05-10T10:34:22.954453+0000","last_undegraded":"2024-05-10T10:34:22.954453+0000","last_fullsized":"2024-05-10T10:34:22.954453+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:08:38.753738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960495+0000","last_change":"2024-05-10T10:34:21.938021+0000","last_active":"2024-05-10T10:34:22.960495+0000","last_peered":"2024-05-10T10:34:22.960495+0000","last_clean":"2024-05-10T10:34:22.960495+0000","last_became_active":"2024-05-10T10:34:21.937596+0000","last_became_peered":"2024-05-10T10:34:21.937596+0000","last_unstale":"2024-05-10T10:34:22.960495+0000","last_undegraded":"2024-05-10T10:34:22.960495+0000","last_fullsized":"2024-05-10T10:34:22.960495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:40:33.340892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955909+0000","last_change":"2024-05-10T10:34:22.952729+0000","last_active":"2024-05-10T10:34:22.955909+0000","last_peered":"2024-05-10T10:34:22.955909+0000","last_clean":"2024-05-10T10:34:22.955909+0000","last_became_active":"2024-05-10T10:34:22.952425+0000","last_became_peered":"2024-05-10T10:34:22.952425+0000","last_unstale":"2024-05-10T10:34:22.955909+0000","last_undegraded":"2024-05-10T10:34:22.955909+0000","last_fullsized":"2024-05-10T10:34:22.955909+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:14:25.324350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.940902+0000","last_change":"2024-05-10T10:34:21.930825+0000","last_active":"2024-05-10T10:34:22.940902+0000","last_peered":"2024-05-10T10:34:22.940902+0000","last_clean":"2024-05-10T10:34:22.940902+0000","last_became_active":"2024-05-10T10:34:21.930537+0000","last_became_peered":"2024-05-10T10:34:21.930537+0000","last_unstale":"2024-05-10T10:34:22.940902+0000","last_undegraded":"2024-05-10T10:34:22.940902+0000","last_fullsized":"2024-05-10T10:34:22.940902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:10.889571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960040+0000","last_change":"2024-05-10T10:34:22.953808+0000","last_active":"2024-05-10T10:34:22.960040+0000","last_peered":"2024-05-10T10:34:22.960040+0000","last_clean":"2024-05-10T10:34:22.960040+0000","last_became_active":"2024-05-10T10:34:22.950344+0000","last_became_peered":"2024-05-10T10:34:22.950344+0000","last_unstale":"2024-05-10T10:34:22.960040+0000","last_undegraded":"2024-05-10T10:34:22.960040+0000","last_fullsized":"2024-05-10T10:34:22.960040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:53:54.272898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.938164+0000","last_change":"2024-05-10T10:34:21.940012+0000","last_active":"2024-05-10T10:34:22.938164+0000","last_peered":"2024-05-10T10:34:22.938164+0000","last_clean":"2024-05-10T10:34:22.938164+0000","last_became_active":"2024-05-10T10:34:21.939678+0000","last_became_peered":"2024-05-10T10:34:21.939678+0000","last_unstale":"2024-05-10T10:34:22.938164+0000","last_undegraded":"2024-05-10T10:34:22.938164+0000","last_fullsized":"2024-05-10T10:34:22.938164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:26:22.232697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960616+0000","last_change":"2024-05-10T10:34:22.956959+0000","last_active":"2024-05-10T10:34:22.960616+0000","last_peered":"2024-05-10T10:34:22.960616+0000","last_clean":"2024-05-10T10:34:22.960616+0000","last_became_active":"2024-05-10T10:34:22.956610+0000","last_became_peered":"2024-05-10T10:34:22.956610+0000","last_unstale":"2024-05-10T10:34:22.960616+0000","last_undegraded":"2024-05-10T10:34:22.960616+0000","last_fullsized":"2024-05-10T10:34:22.960616+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.940257+0000","last_change":"2024-05-10T10:34:21.941866+0000","last_active":"2024-05-10T10:34:22.940257+0000","last_peered":"2024-05-10T10:34:22.940257+0000","last_clean":"2024-05-10T10:34:22.940257+0000","last_became_active":"2024-05-10T10:34:21.941582+0000","last_became_peered":"2024-05-10T10:34:21.941582+0000","last_unstale":"2024-05-10T10:34:22.940257+0000","last_undegraded":"2024-05-10T10:34:22.940257+0000","last_fullsized":"2024-05-10T10:34:22.940257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:46:58.539394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.957104+0000","last_change":"2024-05-10T10:34:22.955983+0000","last_active":"2024-05-10T10:34:22.957104+0000","last_peered":"2024-05-10T10:34:22.957104+0000","last_clean":"2024-05-10T10:34:22.957104+0000","last_became_active":"2024-05-10T10:34:22.955621+0000","last_became_peered":"2024-05-10T10:34:22.955621+0000","last_unstale":"2024-05-10T10:34:22.957104+0000","last_undegraded":"2024-05-10T10:34:22.957104+0000","last_fullsized":"2024-05-10T10:34:22.957104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:18.203426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.945574+0000","last_change":"2024-05-10T10:34:21.939706+0000","last_active":"2024-05-10T10:34:22.945574+0000","last_peered":"2024-05-10T10:34:22.945574+0000","last_clean":"2024-05-10T10:34:22.945574+0000","last_became_active":"2024-05-10T10:34:21.938741+0000","last_became_peered":"2024-05-10T10:34:21.938741+0000","last_unstale":"2024-05-10T10:34:22.945574+0000","last_undegraded":"2024-05-10T10:34:22.945574+0000","last_fullsized":"2024-05-10T10:34:22.945574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:13:35.640358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.959033+0000","last_change":"2024-05-10T10:34:22.957086+0000","last_active":"2024-05-10T10:34:22.959033+0000","last_peered":"2024-05-10T10:34:22.959033+0000","last_clean":"2024-05-10T10:34:22.959033+0000","last_became_active":"2024-05-10T10:34:22.956471+0000","last_became_peered":"2024-05-10T10:34:22.956471+0000","last_unstale":"2024-05-10T10:34:22.959033+0000","last_undegraded":"2024-05-10T10:34:22.959033+0000","last_fullsized":"2024-05-10T10:34:22.959033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:42:47.134411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936967+0000","last_change":"2024-05-10T10:34:21.938215+0000","last_active":"2024-05-10T10:34:22.936967+0000","last_peered":"2024-05-10T10:34:22.936967+0000","last_clean":"2024-05-10T10:34:22.936967+0000","last_became_active":"2024-05-10T10:34:21.937820+0000","last_became_peered":"2024-05-10T10:34:21.937820+0000","last_unstale":"2024-05-10T10:34:22.936967+0000","last_undegraded":"2024-05-10T10:34:22.936967+0000","last_fullsized":"2024-05-10T10:34:22.936967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:44.800678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.955271+0000","last_change":"2024-05-10T10:34:22.952057+0000","last_active":"2024-05-10T10:34:22.955271+0000","last_peered":"2024-05-10T10:34:22.955271+0000","last_clean":"2024-05-10T10:34:22.955271+0000","last_became_active":"2024-05-10T10:34:22.951800+0000","last_became_peered":"2024-05-10T10:34:22.951800+0000","last_unstale":"2024-05-10T10:34:22.955271+0000","last_undegraded":"2024-05-10T10:34:22.955271+0000","last_fullsized":"2024-05-10T10:34:22.955271+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:35.506101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.943244+0000","last_change":"2024-05-10T10:34:21.938066+0000","last_active":"2024-05-10T10:34:22.943244+0000","last_peered":"2024-05-10T10:34:22.943244+0000","last_clean":"2024-05-10T10:34:22.943244+0000","last_became_active":"2024-05-10T10:34:21.937775+0000","last_became_peered":"2024-05-10T10:34:21.937775+0000","last_unstale":"2024-05-10T10:34:22.943244+0000","last_undegraded":"2024-05-10T10:34:22.943244+0000","last_fullsized":"2024-05-10T10:34:22.943244+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:28:36.106128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.960060+0000","last_change":"2024-05-10T10:34:22.957395+0000","last_active":"2024-05-10T10:34:22.960060+0000","last_peered":"2024-05-10T10:34:22.960060+0000","last_clean":"2024-05-10T10:34:22.960060+0000","last_became_active":"2024-05-10T10:34:22.957140+0000","last_became_peered":"2024-05-10T10:34:22.957140+0000","last_unstale":"2024-05-10T10:34:22.960060+0000","last_undegraded":"2024-05-10T10:34:22.960060+0000","last_fullsized":"2024-05-10T10:34:22.960060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:21.917854+0000","last_clean_scrub_stamp":"2024-05-10T10:34:21.917854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:36:34.714282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.936481+0000","last_change":"2024-05-10T10:34:21.948267+0000","last_active":"2024-05-10T10:34:22.936481+0000","last_peered":"2024-05-10T10:34:22.936481+0000","last_clean":"2024-05-10T10:34:22.936481+0000","last_became_active":"2024-05-10T10:34:21.947897+0000","last_became_peered":"2024-05-10T10:34:21.947897+0000","last_unstale":"2024-05-10T10:34:22.936481+0000","last_undegraded":"2024-05-10T10:34:22.936481+0000","last_fullsized":"2024-05-10T10:34:22.936481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:20.913377+0000","last_clean_scrub_stamp":"2024-05-10T10:34:20.913377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:39.753429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":140,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-10T10:34:22.935917+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:34:22.935917+0000","last_peered":"2024-05-10T10:34:22.935917+0000","last_clean":"2024-05-10T10:34:22.935917+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:34:22.935917+0000","last_undegraded":"2024-05-10T10:34:22.935917+0000","last_fullsized":"2024-05-10T10:34:22.935917+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":751820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":161452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":168731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":169223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":161452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":161452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":168709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":759021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:34:27.358 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:34:27.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T10:34:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.503+0000 7fbd05178640 1 Processor -- start 2024-05-10T10:34:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.503+0000 7fbd05178640 1 -- start start 2024-05-10T10:34:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbd05178640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd000cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbd05178640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd0015cfb0 con 0x7fbd00160530 2024-05-10T10:34:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbd05178640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd0015d290 con 0x7fbd0006f030 2024-05-10T10:34:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbd05178640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbd000ccb60 con 0x7fbd000cf240 2024-05-10T10:34:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbcfe59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbd00160530 0x7fbd0015c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60060/0 (socket says 172.21.15.53:60060) 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbcfe59c640 1 -- 172.21.15.53:0/1602984484 learned_addr learned my addr 172.21.15.53:0/1602984484 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbcff59e640 1 --2- 172.21.15.53:0/1602984484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd000cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbcfdd9b640 1 -- 172.21.15.53:0/1602984484 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3793878732 0 0) 0x7fbd0015cfb0 con 0x7fbd00160530 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.504+0000 7fbcfdd9b640 1 -- 172.21.15.53:0/1602984484 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce8003520 con 0x7fbd00160530 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.505+0000 7fbcfdd9b640 1 -- 172.21.15.53:0/1602984484 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 62908899 0 0) 0x7fbd0015d290 con 0x7fbd0006f030 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.505+0000 7fbcfdd9b640 1 -- 172.21.15.53:0/1602984484 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd0015cfb0 con 0x7fbd0006f030 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.505+0000 7fbcff59e640 1 -- 172.21.15.53:0/1602984484 >> v1:172.21.15.135:6789/0 conn(0x7fbd0006f030 legacy=0x7fbd0006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.505+0000 7fbcff59e640 1 -- 172.21.15.53:0/1602984484 >> v1:172.21.15.53:6789/0 conn(0x7fbd00160530 legacy=0x7fbd0015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.505+0000 7fbcff59e640 1 -- 172.21.15.53:0/1602984484 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0015fd30 con 0x7fbd000cf240 2024-05-10T10:34:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.505+0000 7fbcff59e640 1 --2- 172.21.15.53:0/1602984484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd000cf6d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009dc0 tx=0x7fbcf802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316f35c6b2aad4e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbcfdd9b640 1 -- 172.21.15.53:0/1602984484 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8007740 con 0x7fbd000cf240 2024-05-10T10:34:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbcfdd9b640 1 -- 172.21.15.53:0/1602984484 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf80078e0 con 0x7fbd000cf240 2024-05-10T10:34:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbcfdd9b640 1 -- 172.21.15.53:0/1602984484 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8007c90 con 0x7fbd000cf240 2024-05-10T10:34:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbd05178640 1 -- 172.21.15.53:0/1602984484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 msgr2=0x7fbd000cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbd05178640 1 --2- 172.21.15.53:0/1602984484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd000cf6d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009dc0 tx=0x7fbcf802a6d0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbd05178640 1 -- 172.21.15.53:0/1602984484 shutdown_connections 2024-05-10T10:34:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbd05178640 1 --2- 172.21.15.53:0/1602984484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd000cf6d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbd05178640 1 -- 172.21.15.53:0/1602984484 >> 172.21.15.53:0/1602984484 conn(0x7fbd000d0c90 msgr2=0x7fbd000d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbd05178640 1 -- 172.21.15.53:0/1602984484 shutdown_connections 2024-05-10T10:34:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.506+0000 7fbd05178640 1 -- 172.21.15.53:0/1602984484 wait complete. 2024-05-10T10:34:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.508+0000 7fbd05178640 1 Processor -- start 2024-05-10T10:34:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.508+0000 7fbd05178640 1 -- start start 2024-05-10T10:34:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.508+0000 7fbd05178640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd0006f030 0x7fbd00153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbd05178640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd001f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbd05178640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd00160530 0x7fbd000b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbd05178640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0015fa40 con 0x7fbd0006f030 2024-05-10T10:34:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbd05178640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd00160990 con 0x7fbd00160530 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbd05178640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbd000cb910 con 0x7fbd000cf240 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd0006f030 0x7fbd00153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd0006f030 0x7fbd00153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44404/0 (socket says 172.21.15.53:44404) 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 -- 172.21.15.53:0/370181627 learned_addr learned my addr 172.21.15.53:0/370181627 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfe59c640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd001f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcff59e640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd00160530 0x7fbd000b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 -- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 msgr2=0x7fbd001f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd001f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 -- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd00160530 msgr2=0x7fbd000b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd00160530 0x7fbd000b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 -- 172.21.15.53:0/370181627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd000b0420 con 0x7fbd0006f030 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfe59c640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd001f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcff59e640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd00160530 0x7fbd000b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbcfed9d640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd0006f030 0x7fbd00153fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c430 tx=0x7fbcf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbce77fe640 1 -- 172.21.15.53:0/370181627 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0017070 con 0x7fbd0006f030 2024-05-10T10:34:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbce77fe640 1 -- 172.21.15.53:0/370181627 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf00052f0 con 0x7fbd0006f030 2024-05-10T10:34:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd001f8ab0 con 0x7fbd0006f030 2024-05-10T10:34:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbce77fe640 1 -- 172.21.15.53:0/370181627 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0003830 con 0x7fbd0006f030 2024-05-10T10:34:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.509+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd001f9070 con 0x7fbd0006f030 2024-05-10T10:34:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.511+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0006a220 con 0x7fbd0006f030 2024-05-10T10:34:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.511+0000 7fbce77fe640 1 -- 172.21.15.53:0/370181627 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbcf001a070 con 0x7fbd0006f030 2024-05-10T10:34:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.511+0000 7fbce77fe640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbcec0762d0 0x7fbcec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.512+0000 7fbce77fe640 1 -- 172.21.15.53:0/370181627 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fbcf0096bc0 con 0x7fbd0006f030 2024-05-10T10:34:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.512+0000 7fbcfe59c640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbcec0762d0 0x7fbcec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.512+0000 7fbcfe59c640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbcec0762d0 0x7fbcec078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd000b68a0 tx=0x7fbcf4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.516+0000 7fbce77fe640 1 -- 172.21.15.53:0/370181627 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbd000b0420 con 0x7fbd0006f030 2024-05-10T10:34:27.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.780+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbd000cf020 con 0x7fbd0006f030 2024-05-10T10:34:27.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.783+0000 7fbce77fe640 1 -- 172.21.15.53:0/370181627 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7fbcf0060310 con 0x7fbd0006f030 2024-05-10T10:34:27.784 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:34:27.784 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:34:27.784 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2024-05-10T10:34:27.784 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2024-05-10T10:34:27.785 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-10T10:34:27.785 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbcec0762d0 msgr2=0x7fbcec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbcec0762d0 0x7fbcec078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd000b68a0 tx=0x7fbcf4009040 comp rx=0 tx=0).stop 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd0006f030 msgr2=0x7fbd00153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd0006f030 0x7fbd00153fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c430 tx=0x7fbcf000c8f0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 shutdown_connections 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbcec0762d0 0x7fbcec078790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd00160530 0x7fbd000b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd000cf240 0x7fbd001f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 --2- 172.21.15.53:0/370181627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd0006f030 0x7fbd00153fc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 >> 172.21.15.53:0/370181627 conn(0x7fbd000d0c90 msgr2=0x7fbd00159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 shutdown_connections 2024-05-10T10:34:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.785+0000 7fbd05178640 1 -- 172.21.15.53:0/370181627 wait complete. 2024-05-10T10:34:27.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.899+0000 7fd0205f2640 1 Processor -- start 2024-05-10T10:34:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.899+0000 7fd0205f2640 1 -- start start 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.900+0000 7fd0205f2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd01815f290 0x7fd0180d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.900+0000 7fd0205f2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0180702e0 0x7fd0180d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.900+0000 7fd0205f2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0180d3280 0x7fd0180cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.900+0000 7fd0205f2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd0180d14c0 con 0x7fd0180d3280 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.900+0000 7fd0205f2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd0180d2300 con 0x7fd01815f290 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.900+0000 7fd0205f2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd01806f3c0 con 0x7fd0180702e0 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0180d3280 0x7fd0180cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0180d3280 0x7fd0180cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44410/0 (socket says 172.21.15.53:44410) 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 -- 172.21.15.53:0/4270862238 learned_addr learned my addr 172.21.15.53:0/4270862238 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01e38e640 1 --2- 172.21.15.53:0/4270862238 >> v2:172.21.15.135:3300/0 conn(0x7fd01815f290 0x7fd0180d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01db8d640 1 --2- 172.21.15.53:0/4270862238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0180702e0 0x7fd0180d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 -- 172.21.15.53:0/4270862238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0180702e0 msgr2=0x7fd0180d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 --2- 172.21.15.53:0/4270862238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0180702e0 0x7fd0180d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 -- 172.21.15.53:0/4270862238 >> v2:172.21.15.135:3300/0 conn(0x7fd01815f290 msgr2=0x7fd0180d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 --2- 172.21.15.53:0/4270862238 >> v2:172.21.15.135:3300/0 conn(0x7fd01815f290 0x7fd0180d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01eb8f640 1 -- 172.21.15.53:0/4270862238 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0180d16f0 con 0x7fd0180d3280 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01e38e640 1 --2- 172.21.15.53:0/4270862238 >> v2:172.21.15.135:3300/0 conn(0x7fd01815f290 0x7fd0180d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.901+0000 7fd01db8d640 1 --2- 172.21.15.53:0/4270862238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0180702e0 0x7fd0180d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd01eb8f640 1 --2- 172.21.15.53:0/4270862238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd0180cf770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd018072530 tx=0x7fd00802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0128353bec807c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd01d38c640 1 -- 172.21.15.53:0/4270862238 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008036070 con 0x7fd0180d3280 2024-05-10T10:34:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd01d38c640 1 -- 172.21.15.53:0/4270862238 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008031600 con 0x7fd0180d3280 2024-05-10T10:34:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd01d38c640 1 -- 172.21.15.53:0/4270862238 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0080319d0 con 0x7fd0180d3280 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd0205f2640 1 -- 172.21.15.53:0/4270862238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 msgr2=0x7fd0180cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd0205f2640 1 --2- 172.21.15.53:0/4270862238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd0180cf770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd018072530 tx=0x7fd00802a730 comp rx=0 tx=0).stop 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd0205f2640 1 -- 172.21.15.53:0/4270862238 shutdown_connections 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd0205f2640 1 --2- 172.21.15.53:0/4270862238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd0180cf770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd0205f2640 1 --2- 172.21.15.53:0/4270862238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0180702e0 0x7fd0180d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd0205f2640 1 --2- 172.21.15.53:0/4270862238 >> v2:172.21.15.135:3300/0 conn(0x7fd01815f290 0x7fd0180d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.902+0000 7fd0205f2640 1 -- 172.21.15.53:0/4270862238 >> 172.21.15.53:0/4270862238 conn(0x7fd01809ba20 msgr2=0x7fd01809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.903+0000 7fd0205f2640 1 -- 172.21.15.53:0/4270862238 shutdown_connections 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.903+0000 7fd0205f2640 1 -- 172.21.15.53:0/4270862238 wait complete. 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 Processor -- start 2024-05-10T10:34:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 -- start start 2024-05-10T10:34:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0180702e0 0x7fd018159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd01815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd01815f290 0x7fd0180c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd0180d1f90 con 0x7fd0180d3280 2024-05-10T10:34:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0180d1cb0 con 0x7fd0180702e0 2024-05-10T10:34:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd0205f2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd01815f670 con 0x7fd01815f290 2024-05-10T10:34:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.904+0000 7fd01db8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd01815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd01815a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44418/0 (socket says 172.21.15.53:44418) 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 -- 172.21.15.53:0/226244761 learned_addr learned my addr 172.21.15.53:0/226244761 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01eb8f640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd01815f290 0x7fd0180c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01e38e640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0180702e0 0x7fd018159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 -- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd01815f290 msgr2=0x7fd0180c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd01815f290 0x7fd0180c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 -- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0180702e0 msgr2=0x7fd018159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0180702e0 0x7fd018159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 -- 172.21.15.53:0/226244761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd018159fa0 con 0x7fd0180d3280 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd01db8d640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd01815a860 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00d800 tx=0x7fd00c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd016ffd640 1 -- 172.21.15.53:0/226244761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c018070 con 0x7fd0180d3280 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd016ffd640 1 -- 172.21.15.53:0/226244761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00c004030 con 0x7fd0180d3280 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd016ffd640 1 -- 172.21.15.53:0/226244761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c004400 con 0x7fd0180d3280 2024-05-10T10:34:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01815a240 con 0x7fd0180d3280 2024-05-10T10:34:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.905+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0180c6e60 con 0x7fd0180d3280 2024-05-10T10:34:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.906+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0180693f0 con 0x7fd0180d3280 2024-05-10T10:34:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.907+0000 7fd016ffd640 1 -- 172.21.15.53:0/226244761 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd00c0096a0 con 0x7fd0180d3280 2024-05-10T10:34:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.907+0000 7fd016ffd640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd004076300 0x7fd0040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.907+0000 7fd016ffd640 1 -- 172.21.15.53:0/226244761 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fd00c0979e0 con 0x7fd0180d3280 2024-05-10T10:34:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.908+0000 7fd01e38e640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd004076300 0x7fd0040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.908+0000 7fd01e38e640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd004076300 0x7fd0040787c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd010006e80 tx=0x7fd010008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:27.913 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:34:27.912+0000 7fca746bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:34:27.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:27.912+0000 7fd016ffd640 1 -- 172.21.15.53:0/226244761 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd00c060ac0 con 0x7fd0180d3280 2024-05-10T10:34:28.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.145+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0180c9bc0 con 0x7fd0180d3280 2024-05-10T10:34:28.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.145+0000 7fd016ffd640 1 -- 172.21.15.53:0/226244761 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7fd00c064dc0 con 0x7fd0180d3280 2024-05-10T10:34:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:28.147 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:28.147 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:28.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd004076300 msgr2=0x7fd0040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd004076300 0x7fd0040787c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd010006e80 tx=0x7fd010008040 comp rx=0 tx=0).stop 2024-05-10T10:34:28.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 msgr2=0x7fd01815a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd01815a860 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00d800 tx=0x7fd00c00dcc0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 shutdown_connections 2024-05-10T10:34:28.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd004076300 0x7fd0040787c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd01815f290 0x7fd0180c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0180d3280 0x7fd01815a860 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 --2- 172.21.15.53:0/226244761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0180702e0 0x7fd018159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 >> 172.21.15.53:0/226244761 conn(0x7fd01809ba20 msgr2=0x7fd0180d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 shutdown_connections 2024-05-10T10:34:28.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.147+0000 7fd0205f2640 1 -- 172.21.15.53:0/226244761 wait complete. 2024-05-10T10:34:28.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:34:22.306333+0000', 'modified': '2024-05-10T10:34:25.330760+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4878}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4878': {'gid': 4878, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/561700189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 561700189}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 561700189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4878, 'qdb_cluster': [4878]} 2024-05-10T10:34:28.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:34:28.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T10:34:28.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.671+0000 7f55a7a47640 1 Processor -- start 2024-05-10T10:34:28.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.671+0000 7f55a7a47640 1 -- start start 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f55a7a47640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 0x7f55a00c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f55a7a47640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a00cd090 con 0x7f55a01611e0 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f55a7a47640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a00cd370 con 0x7f55a015ce60 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f55a7a47640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f55a00cb790 con 0x7f55a00cb300 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f55a4fe2640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f55a01611e0 0x7f55a015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60072/0 (socket says 172.21.15.53:60072) 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f55a4fe2640 1 -- 172.21.15.53:0/3996686751 learned_addr learned my addr 172.21.15.53:0/3996686751 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 0x7f55a00c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233210989 0 0) 0x7f55a00cd090 con 0x7f55a01611e0 2024-05-10T10:34:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f558c003520 con 0x7f55a01611e0 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 394536488 0 0) 0x7f558c003520 con 0x7f55a01611e0 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 msgr2=0x7f55a00c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 0x7f55a00c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 >> v1:172.21.15.135:6789/0 conn(0x7f55a015ce60 legacy=0x7f55a0160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a00cd650 con 0x7f55a01611e0 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f5590001670 con 0x7f55a01611e0 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 >> v1:172.21.15.53:6789/0 conn(0x7f55a01611e0 legacy=0x7f55a015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f558c0039e0 0x7f558c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c004390 0x7f558c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 0x7f558c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f558c00ba70 con 0x7f558c004390 2024-05-10T10:34:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f558c00bbf0 con 0x7f558c0039e0 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.672+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f558c00bdf0 con 0x7f55a00cb300 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c004390 0x7f558c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a57e3640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f558c0039e0 0x7f558c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 0x7f558c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a4fe2640 1 -- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 msgr2=0x7f558c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 0x7f558c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a4fe2640 1 -- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f558c0039e0 msgr2=0x7f558c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f558c0039e0 0x7f558c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a4fe2640 1 -- 172.21.15.53:0/3996686751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c003520 con 0x7f558c004390 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.673+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55a00cb300 0x7f558c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c004390 0x7f558c004860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5590007fd0 tx=0x7f558c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46faf2c5eec6cea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590007ca0 con 0x7f558c004390 2024-05-10T10:34:28.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5590005ad0 con 0x7f558c004390 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f5597fff640 1 -- 172.21.15.53:0/3996686751 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590007070 con 0x7f558c004390 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a7a47640 1 -- 172.21.15.53:0/3996686751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c004390 msgr2=0x7f558c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a7a47640 1 --2- 172.21.15.53:0/3996686751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c004390 0x7f558c004860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5590007fd0 tx=0x7f558c005830 comp rx=0 tx=0).stop 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a5fe4640 1 -- 172.21.15.53:0/3996686751 reap_dead start 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a7a47640 1 -- 172.21.15.53:0/3996686751 shutdown_connections 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a7a47640 1 -- 172.21.15.53:0/3996686751 >> 172.21.15.53:0/3996686751 conn(0x7f55a00b2900 msgr2=0x7f55a00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a7a47640 1 -- 172.21.15.53:0/3996686751 shutdown_connections 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.674+0000 7f55a7a47640 1 -- 172.21.15.53:0/3996686751 wait complete. 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 Processor -- start 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 -- start start 2024-05-10T10:34:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f558c004390 0x7f55a00b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a00d10b0 0x7f55a01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a009ba20 0x7f55a009beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55a00cde50 con 0x7f55a009ba20 2024-05-10T10:34:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55a00ccd60 con 0x7f55a00d10b0 2024-05-10T10:34:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a7a47640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f55a00cdfd0 con 0x7f558c004390 2024-05-10T10:34:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a5fe4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a009ba20 0x7f55a009beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.676+0000 7f55a5fe4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a009ba20 0x7f55a009beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44442/0 (socket says 172.21.15.53:44442) 2024-05-10T10:34:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a5fe4640 1 -- 172.21.15.53:0/1341419719 learned_addr learned my addr 172.21.15.53:0/1341419719 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a5fe4640 1 -- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f558c004390 msgr2=0x7f55a00b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f558c004390 0x7f55a00b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a5fe4640 1 -- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a00d10b0 msgr2=0x7f55a01585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a00d10b0 0x7f55a01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a5fe4640 1 -- 172.21.15.53:0/1341419719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a00b4b80 con 0x7f55a009ba20 2024-05-10T10:34:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a009ba20 0x7f55a009beb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55a0074230 tx=0x7f5598009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f5595ffb640 1 -- 172.21.15.53:0/1341419719 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598010e50 con 0x7f55a009ba20 2024-05-10T10:34:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f5595ffb640 1 -- 172.21.15.53:0/1341419719 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5598003990 con 0x7f55a009ba20 2024-05-10T10:34:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a00b42e0 con 0x7f55a009ba20 2024-05-10T10:34:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f5595ffb640 1 -- 172.21.15.53:0/1341419719 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598003dc0 con 0x7f55a009ba20 2024-05-10T10:34:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.677+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a00b48a0 con 0x7f55a009ba20 2024-05-10T10:34:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.679+0000 7f5595ffb640 1 -- 172.21.15.53:0/1341419719 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f559801e020 con 0x7f55a009ba20 2024-05-10T10:34:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.679+0000 7f5595ffb640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55840762d0 0x7f5584078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.680+0000 7f5595ffb640 1 -- 172.21.15.53:0/1341419719 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f5598097f90 con 0x7f55a009ba20 2024-05-10T10:34:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.680+0000 7f55a57e3640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55840762d0 0x7f5584078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.680+0000 7f55a57e3640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55840762d0 0x7f5584078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f559c0018b0 tx=0x7f559c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.680+0000 7f55aa765740 1 Processor -- start 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55aa765740 1 -- start start 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c86f50 0x5577d4c89760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c89ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55aa765740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c59bb0 0x5577d4c5d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55aa765740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5577d4bc38d0 con 0x5577d4c59bb0 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5577d4b7cc10 con 0x5577d4c87330 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5577d4bbf890 con 0x5577d4c86f50 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55a5fe4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c59bb0 0x5577d4c5d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55a5fe4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c59bb0 0x5577d4c5d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44448/0 (socket says 172.21.15.53:44448) 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55a5fe4640 1 -- 172.21.15.53:0/65150263 learned_addr learned my addr 172.21.15.53:0/65150263 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55a57e3640 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c86f50 0x5577d4c89760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.681+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c89ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55a5fe4640 1 -- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c86f50 msgr2=0x5577d4c89760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c86f50 0x5577d4c89760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55a5fe4640 1 -- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 msgr2=0x5577d4c89ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c89ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55a5fe4640 1 -- 172.21.15.53:0/65150263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d4b845d0 con 0x5577d4c59bb0 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c59bb0 0x5577d4c5d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5598011010 tx=0x7f5598060a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339cda75af21feb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f558affd640 1 -- 172.21.15.53:0/65150263 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598060fc0 con 0x5577d4c59bb0 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f558affd640 1 -- 172.21.15.53:0/65150263 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5598065370 con 0x5577d4c59bb0 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f558affd640 1 -- 172.21.15.53:0/65150263 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598069b20 con 0x5577d4c59bb0 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55aa765740 1 -- 172.21.15.53:0/65150263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c59bb0 msgr2=0x5577d4c5d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.682+0000 7f55aa765740 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c59bb0 0x5577d4c5d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5598011010 tx=0x7f5598060a40 comp rx=0 tx=0).stop 2024-05-10T10:34:28.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 -- 172.21.15.53:0/65150263 shutdown_connections 2024-05-10T10:34:28.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c59bb0 0x5577d4c5d3c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c89ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 --2- 172.21.15.53:0/65150263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c86f50 0x5577d4c89760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 -- 172.21.15.53:0/65150263 >> 172.21.15.53:0/65150263 conn(0x5577d4c5be20 msgr2=0x5577d4ca4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 -- 172.21.15.53:0/65150263 shutdown_connections 2024-05-10T10:34:28.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 -- 172.21.15.53:0/65150263 wait complete. 2024-05-10T10:34:28.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 -- start start 2024-05-10T10:34:28.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.683+0000 7f55aa765740 1 Processor -- start 2024-05-10T10:34:28.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c86f50 0x5577d4ca77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55aa765740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c87330 0x5577d4c7ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4c7f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55aa765740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5577d4c5efd0 con 0x5577d4c87330 2024-05-10T10:34:28.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5577d4b80760 con 0x5577d4c86f50 2024-05-10T10:34:28.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5577d4c5f2b0 con 0x5577d4c7f010 2024-05-10T10:34:28.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55aa765740 10 client.0 _ll_register_callbacks cb 0x5577d4bb9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:34:28.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55a4fe2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c87330 0x5577d4c7ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55a4fe2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c87330 0x5577d4c7ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44464/0 (socket says 172.21.15.53:44464) 2024-05-10T10:34:28.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55a4fe2640 1 -- 172.21.15.53:0/4233442057 learned_addr learned my addr 172.21.15.53:0/4233442057 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:28.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4c7f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55a57e3640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c86f50 0x5577d4ca77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55a4fe2640 1 -- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 msgr2=0x5577d4c7f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.684+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4c7f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55a4fe2640 1 -- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c86f50 msgr2=0x5577d4ca77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c86f50 0x5577d4ca77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55a4fe2640 1 -- 172.21.15.53:0/4233442057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d48d52d0 con 0x5577d4c87330 2024-05-10T10:34:28.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55a57e3640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c86f50 0x5577d4ca77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:28.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4c7f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:28.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c87330 0x5577d4c7ead0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f559002ad60 tx=0x7f55a00cd820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4fb813263def40 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f557ffff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:34:28.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f557ffff640 1 -- 172.21.15.53:0/4233442057 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590043020 con 0x5577d4c87330 2024-05-10T10:34:28.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f557ffff640 1 -- 172.21.15.53:0/4233442057 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f55900047e0 con 0x5577d4c87330 2024-05-10T10:34:28.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5598065370 con 0x5577d4c87330 2024-05-10T10:34:28.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f557ffff640 1 -- 172.21.15.53:0/4233442057 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f559004d050 con 0x5577d4c87330 2024-05-10T10:34:28.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 10 client.5025 fetch_fsmap learned FSMap version 18 2024-05-10T10:34:28.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5577d499f6b0 con 0x5577d4c87330 2024-05-10T10:34:28.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f557ffff640 1 -- 172.21.15.53:0/4233442057 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5598065370 con 0x5577d4c87330 2024-05-10T10:34:28.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f557ffff640 1 -- 172.21.15.53:0/4233442057 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f5590050020 con 0x5577d4c87330 2024-05-10T10:34:28.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 10 client.5025 fetch_fsmap finished waiting for FSMap version 18 2024-05-10T10:34:28.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 10 client.5025 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:34:28.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 20 client.5025 populate_metadata read hostname 'smithi053' 2024-05-10T10:34:28.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4c7dde0 0x5577d4cc32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 4 client.5025 mds_command: new command op to 4878 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:34:28.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 --> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577d4b47760 con 0x5577d4c7dde0 2024-05-10T10:34:28.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.685+0000 7f55a57e3640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4c7dde0 0x5577d4cc32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.686+0000 7f55a57e3640 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4c7dde0 0x5577d4cc32e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce079f097499fe1 server_cookie=3db97805400844b4 in_seq=0 out_seq=0 2024-05-10T10:34:28.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.686+0000 7f557ffff640 10 client.5025 ms_handle_connect on v2:172.21.15.53:6834/561700189 2024-05-10T10:34:28.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.686+0000 7f557ffff640 1 -- 172.21.15.53:0/4233442057 <== mds.0 v2:172.21.15.53:6834/561700189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577d4b47760 con 0x5577d4c7dde0 2024-05-10T10:34:28.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.686+0000 7f557ffff640 10 client.5025 handle_command_reply: tid=0 2024-05-10T10:34:28.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.686+0000 7f55aa765740 1 client.5025 shutdown 2024-05-10T10:34:28.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.686+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c87330 msgr2=0x5577d4c7ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.686+0000 7f55aa765740 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c87330 0x5577d4c7ead0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f559002ad60 tx=0x7f55a00cd820 comp rx=0 tx=0).stop 2024-05-10T10:34:28.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 shutdown_connections 2024-05-10T10:34:28.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4c7dde0 0x5577d4cc32e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4c7f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c87330 0x5577d4c7ead0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f557ffff640 0 client.5025 ms_handle_reset on v2:172.21.15.53:6834/561700189 2024-05-10T10:34:28.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 --2- 172.21.15.53:0/4233442057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c86f50 0x5577d4ca77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 >> 172.21.15.53:0/4233442057 conn(0x5577d4c59bb0 msgr2=0x5577d4c58a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 shutdown_connections 2024-05-10T10:34:28.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 1 -- 172.21.15.53:0/4233442057 wait complete. 2024-05-10T10:34:28.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.687+0000 7f55aa765740 20 client.5025 trim_cache size 0 max 16384 2024-05-10T10:34:28.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 Processor -- start 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 -- start start 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c7f010 0x5577d4c58f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c87330 0x5577d4c7fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01fff20 con 0x5577d4c86f50 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01fef70 con 0x5577d4c7f010 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.706+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f55a01fe2d0 con 0x5577d4c87330 2024-05-10T10:34:28.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44478/0 (socket says 172.21.15.53:44478) 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 -- 172.21.15.53:0/968661018 learned_addr learned my addr 172.21.15.53:0/968661018 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c87330 0x5577d4c7fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a57e3640 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c7f010 0x5577d4c58f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 -- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c87330 msgr2=0x5577d4c7fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c87330 0x5577d4c7fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 -- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c7f010 msgr2=0x5577d4c58f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c7f010 0x5577d4c58f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 -- 172.21.15.53:0/968661018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d4c89af0 con 0x5577d4c86f50 2024-05-10T10:34:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7f470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5590001d10 tx=0x7f559003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c48d4ab51e39784 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f558a7fc640 1 -- 172.21.15.53:0/968661018 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590043070 con 0x5577d4c86f50 2024-05-10T10:34:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f558a7fc640 1 -- 172.21.15.53:0/968661018 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5590038040 con 0x5577d4c86f50 2024-05-10T10:34:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.707+0000 7f558a7fc640 1 -- 172.21.15.53:0/968661018 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590004ab0 con 0x5577d4c86f50 2024-05-10T10:34:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 -- 172.21.15.53:0/968661018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 msgr2=0x5577d4c7f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7f470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5590001d10 tx=0x7f559003c000 comp rx=0 tx=0).stop 2024-05-10T10:34:28.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 -- 172.21.15.53:0/968661018 shutdown_connections 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c87330 0x5577d4c7fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7f470 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 --2- 172.21.15.53:0/968661018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c7f010 0x5577d4c58f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 -- 172.21.15.53:0/968661018 >> 172.21.15.53:0/968661018 conn(0x5577d4c59bb0 msgr2=0x5577d4c60a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 -- 172.21.15.53:0/968661018 shutdown_connections 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 -- 172.21.15.53:0/968661018 wait complete. 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 -- start start 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.708+0000 7f55aa765740 1 Processor -- start 2024-05-10T10:34:28.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4d0f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55aa765740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55aa765740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c5f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55aa765740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5577d4c81020 con 0x5577d4c86f50 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01fe920 con 0x5577d4c87330 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55aa765740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5577d4c81680 con 0x5577d4c7f010 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55aa765740 10 client.0 _ll_register_callbacks cb 0x7f55980095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.709+0000 7f55a4fe2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44490/0 (socket says 172.21.15.53:44490) 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 -- 172.21.15.53:0/1478069865 learned_addr learned my addr 172.21.15.53:0/1478069865 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a57e3640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4d0f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c5f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 -- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 msgr2=0x5577d4d0f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4d0f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 -- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 msgr2=0x5577d4c5f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c5f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 -- 172.21.15.53:0/1478069865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d4d0fb70 con 0x5577d4c86f50 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a57e3640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4d0f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a5fe4640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c5f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55a4fe2640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7bd20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5590001eb0 tx=0x7f5590009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd4a06ea38f8c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f557ffff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f557ffff640 1 -- 172.21.15.53:0/1478069865 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559005a060 con 0x5577d4c86f50 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f557ffff640 1 -- 172.21.15.53:0/1478069865 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f5590007270 con 0x5577d4c86f50 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5590038040 con 0x5577d4c86f50 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f557ffff640 1 -- 172.21.15.53:0/1478069865 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5590060050 con 0x5577d4c86f50 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55aa765740 10 client.5031 fetch_fsmap learned FSMap version 18 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5577d4c7c490 con 0x5577d4c86f50 2024-05-10T10:34:28.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.710+0000 7f557ffff640 1 -- 172.21.15.53:0/1478069865 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5590038040 con 0x5577d4c86f50 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f557ffff640 1 -- 172.21.15.53:0/1478069865 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f5590063020 con 0x5577d4c86f50 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 10 client.5031 fetch_fsmap finished waiting for FSMap version 18 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 10 client.5031 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 20 client.5031 populate_metadata read hostname 'smithi053' 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4cb35a0 0x5577d4c9a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 4 client.5031 mds_command: new command op to 4878 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 --> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] -- command(tid 0: {"prefix": "status"}) -- 0x5577d4bd14e0 con 0x5577d4cb35a0 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55a57e3640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4cb35a0 0x5577d4c9a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f557ffff640 10 client.5031 ms_handle_connect on v2:172.21.15.53:6834/561700189 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55a57e3640 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4cb35a0 0x5577d4c9a0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e95aa62bd7400c92 server_cookie=9a2448061f5a8f09 in_seq=0 out_seq=0 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f557ffff640 1 -- 172.21.15.53:0/1478069865 <== mds.0 v2:172.21.15.53:6834/561700189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5577d4bd14e0 con 0x5577d4cb35a0 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f557ffff640 10 client.5031 handle_command_reply: tid=0 2024-05-10T10:34:28.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 client.5031 shutdown 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 msgr2=0x5577d4c7bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7bd20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5590001eb0 tx=0x7f5590009030 comp rx=0 tx=0).stop 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 shutdown_connections 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:6834/561700189,v1:172.21.15.53:6835/561700189] conn(0x5577d4cb35a0 0x5577d4c9a0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577d4c87330 0x5577d4c5f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577d4c86f50 0x5577d4c7bd20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f557ffff640 0 client.5031 ms_handle_reset on v2:172.21.15.53:6834/561700189 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 --2- 172.21.15.53:0/1478069865 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5577d4c7f010 0x5577d4d0f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.711+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 >> 172.21.15.53:0/1478069865 conn(0x5577d4c59bb0 msgr2=0x5577d4c58db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 shutdown_connections 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55aa765740 1 -- 172.21.15.53:0/1478069865 wait complete. 2024-05-10T10:34:28.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55aa765740 20 client.5031 trim_cache size 0 max 16384 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 4878, 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 5.7692229389999996, 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 18, 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 36, 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 36, 2024-05-10T10:34:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.637704668 2024-05-10T10:34:28.733 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55840762d0 msgr2=0x7f5584078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55840762d0 0x7f5584078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f559c0018b0 tx=0x7f559c009040 comp rx=0 tx=0).stop 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a009ba20 msgr2=0x7f55a009beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a009ba20 0x7f55a009beb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55a0074230 tx=0x7f5598009f90 comp rx=0 tx=0).stop 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 shutdown_connections 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f55840762d0 0x7f5584078790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a009ba20 0x7f55a009beb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a00d10b0 0x7f55a01585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 --2- 172.21.15.53:0/1341419719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f558c004390 0x7f55a00b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 >> 172.21.15.53:0/1341419719 conn(0x7f55a00b2900 msgr2=0x7f55a00af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 shutdown_connections 2024-05-10T10:34:28.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.712+0000 7f55a7a47640 1 -- 172.21.15.53:0/1341419719 wait complete. 2024-05-10T10:34:28.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:34:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.838+0000 7fd181245640 1 Processor -- start 2024-05-10T10:34:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.838+0000 7fd181245640 1 -- start start 2024-05-10T10:34:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.838+0000 7fd181245640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd17c0d5cb0 0x7fd17c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.838+0000 7fd181245640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c162b30 0x7fd17c0d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.838+0000 7fd181245640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17c0b45f0 con 0x7fd17c162750 2024-05-10T10:34:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.838+0000 7fd181245640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd17c0b4f20 con 0x7fd17c0d5cb0 2024-05-10T10:34:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.838+0000 7fd181245640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd17c0d0fe0 con 0x7fd17c162b30 2024-05-10T10:34:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd17c0d5cb0 0x7fd17c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd17c0d5cb0 0x7fd17c0d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44496/0 (socket says 172.21.15.53:44496) 2024-05-10T10:34:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17259c640 1 -- 172.21.15.53:0/3607353299 learned_addr learned my addr 172.21.15.53:0/3607353299 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17b59e640 1 --2- 172.21.15.53:0/3607353299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c162b30 0x7fd17c0d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17259c640 1 -- 172.21.15.53:0/3607353299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c162b30 msgr2=0x7fd17c0d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17259c640 1 --2- 172.21.15.53:0/3607353299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c162b30 0x7fd17c0d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17259c640 1 -- 172.21.15.53:0/3607353299 >> v1:172.21.15.135:6789/0 conn(0x7fd17c162750 legacy=0x7fd17c0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.839+0000 7fd17259c640 1 -- 172.21.15.53:0/3607353299 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c0b48d0 con 0x7fd17c0d5cb0 2024-05-10T10:34:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd17259c640 1 --2- 172.21.15.53:0/3607353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c0d5cb0 0x7fd17c0d3630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd164009870 tx=0x7fd16402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe4de4435e87164 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd17a59c640 1 -- 172.21.15.53:0/3607353299 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164037070 con 0x7fd17c0d5cb0 2024-05-10T10:34:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd17a59c640 1 -- 172.21.15.53:0/3607353299 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164032440 con 0x7fd17c0d5cb0 2024-05-10T10:34:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd17a59c640 1 -- 172.21.15.53:0/3607353299 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164032810 con 0x7fd17c0d5cb0 2024-05-10T10:34:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 -- 172.21.15.53:0/3607353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c0d5cb0 msgr2=0x7fd17c0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 --2- 172.21.15.53:0/3607353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c0d5cb0 0x7fd17c0d3630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd164009870 tx=0x7fd16402a1c0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 -- 172.21.15.53:0/3607353299 shutdown_connections 2024-05-10T10:34:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 --2- 172.21.15.53:0/3607353299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c162b30 0x7fd17c0d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 --2- 172.21.15.53:0/3607353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c0d5cb0 0x7fd17c0d3630 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 -- 172.21.15.53:0/3607353299 >> 172.21.15.53:0/3607353299 conn(0x7fd17c09e3b0 msgr2=0x7fd17c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 -- 172.21.15.53:0/3607353299 shutdown_connections 2024-05-10T10:34:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.840+0000 7fd181245640 1 -- 172.21.15.53:0/3607353299 wait complete. 2024-05-10T10:34:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.842+0000 7fd181245640 1 Processor -- start 2024-05-10T10:34:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.842+0000 7fd181245640 1 -- start start 2024-05-10T10:34:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd181245640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd17c162750 0x7fd17c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd181245640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c162b30 0x7fd17c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd181245640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c0b3ab0 0x7fd17c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd181245640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c0b53b0 con 0x7fd17c162b30 2024-05-10T10:34:28.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd181245640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c0d0d80 con 0x7fd17c162750 2024-05-10T10:34:28.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd181245640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd17c0b5530 con 0x7fd17c0b3ab0 2024-05-10T10:34:28.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c162b30 0x7fd17c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c162b30 0x7fd17c0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44498/0 (socket says 172.21.15.53:44498) 2024-05-10T10:34:28.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 -- 172.21.15.53:0/2590204182 learned_addr learned my addr 172.21.15.53:0/2590204182 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:28.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17ad9d640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd17c162750 0x7fd17c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17b59e640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c0b3ab0 0x7fd17c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 -- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c0b3ab0 msgr2=0x7fd17c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c0b3ab0 0x7fd17c15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 -- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd17c162750 msgr2=0x7fd17c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd17c162750 0x7fd17c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 -- 172.21.15.53:0/2590204182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c1f86c0 con 0x7fd17c162b30 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17ad9d640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd17c162750 0x7fd17c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.843+0000 7fd17259c640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c162b30 0x7fd17c0b3570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd164000c00 tx=0x7fd1640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.844+0000 7fd173fff640 1 -- 172.21.15.53:0/2590204182 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164041020 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.844+0000 7fd173fff640 1 -- 172.21.15.53:0/2590204182 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1640306c0 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.844+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c1f89a0 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.844+0000 7fd173fff640 1 -- 172.21.15.53:0/2590204182 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164037040 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.844+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c1f8f60 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.845+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17c06a220 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.845+0000 7fd173fff640 1 -- 172.21.15.53:0/2590204182 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd164044020 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.846+0000 7fd173fff640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd15c0762d0 0x7fd15c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.846+0000 7fd173fff640 1 -- 172.21.15.53:0/2590204182 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fd1640b7e00 con 0x7fd17c162b30 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.846+0000 7fd17ad9d640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd15c0762d0 0x7fd15c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.846+0000 7fd17ad9d640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd15c0762d0 0x7fd15c078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd16c001b70 tx=0x7fd16c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:28.850+0000 7fd173fff640 1 -- 172.21.15.53:0/2590204182 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd16404b030 con 0x7fd17c162b30 2024-05-10T10:34:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.083+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd17c0b3ec0 con 0x7fd17c162b30 2024-05-10T10:34:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.083+0000 7fd173fff640 1 -- 172.21.15.53:0/2590204182 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7fd164080f80 con 0x7fd17c162b30 2024-05-10T10:34:29.086 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:34:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd15c0762d0 msgr2=0x7fd15c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd15c0762d0 0x7fd15c078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd16c001b70 tx=0x7fd16c009040 comp rx=0 tx=0).stop 2024-05-10T10:34:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c162b30 msgr2=0x7fd17c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c162b30 0x7fd17c0b3570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd164000c00 tx=0x7fd1640046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 shutdown_connections 2024-05-10T10:34:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd15c0762d0 0x7fd15c078790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd17c0b3ab0 0x7fd17c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd17c162b30 0x7fd17c0b3570 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 --2- 172.21.15.53:0/2590204182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd17c162750 0x7fd17c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 >> 172.21.15.53:0/2590204182 conn(0x7fd17c09e3b0 msgr2=0x7fd17c06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 shutdown_connections 2024-05-10T10:34:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29.085+0000 7fd181245640 1 -- 172.21.15.53:0/2590204182 wait complete. 2024-05-10T10:34:29.098 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:34:29.099 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:34:29.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:34:29.099 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:34:29.099 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:34:29.099 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:34:29.099 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:34:29.099 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:34:29.100 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:34:29.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.100 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:34:29.118 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:34:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:34:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:34:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:34:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:29.119 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:34:29.120 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:34:29.120 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:34:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:34:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:34:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42700sec preferred_lft 42700sec 2024-05-10T10:34:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:34:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:29.122 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:34:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:34:29.122 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:34:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:34:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:34:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:34:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:29.124 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:34:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:34:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:34:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:29.124 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:34:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:34:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:34:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:34:29.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.126 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:34:29.183 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:34:29.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.185 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:34:29.243 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:34:29.243 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:34:29.243 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:34:29.243 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:34:29.243 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:34:29.244 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:34:29.244 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:34:29.244 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:34:29.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:34:29.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:34:29.245 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:34:29.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:34:29.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:29.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.375 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:34:29.392 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:34:29.393 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:34:29.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:34:29.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:34:29.459 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:34:29.460 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:34:29.515 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:34:29.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:34:29.516 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:34:29.571 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:34:29.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:34:29.656 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:34:29.656 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:34:29.656 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:34:29.665 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:34:29.665 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:34:29.665 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:34:29.665 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:34:29.665 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:34:29.671 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:34:29.671 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:34:29.671 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:34:29.671 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:34:29.671 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:34:29.671 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:34:29.672 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:34:29.672 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:34:29.673 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:34:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:34:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:34:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:34:29.673 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:34:29.674 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:34:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:34:29.700 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:34:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:34:29.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:29.803 INFO:teuthology.orchestra.run.smithi053.stdout:client4987 2024-05-10T10:34:29.804 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:29.804 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:29.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.821 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-05-10T10:34:29.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.882 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-05-10T10:34:29.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.940 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-05-10T10:34:29.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:29.998 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-05-10T10:34:30.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:30.057 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-05-10T10:34:30.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:30.114 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-10T10:34:30.174 INFO:teuthology.orchestra.run.smithi053.stdout:./ 2024-05-10T10:34:30.174 INFO:teuthology.orchestra.run.smithi053.stdout:./rfile 2024-05-10T10:34:30.174 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir 2024-05-10T10:34:30.174 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir/pfile 2024-05-10T10:34:30.175 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir/childdir 2024-05-10T10:34:30.175 INFO:teuthology.orchestra.run.smithi053.stdout:./parentdir/childdir/cfile 2024-05-10T10:34:30.175 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:34:30.175 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:34:30.175 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:34:30.175 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:34:30.175 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-10T10:34:30.175 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:34:30.282 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:34:30.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:30.294 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:34:30.294 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:34:30.294 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:34:30.294 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:34:30.294 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-05-10T10:34:30.294 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:34:30.399 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627778 2024-05-10T10:34:30.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:30.409 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:34:30.409 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:34:30.409 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:34:30.409 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:34:30.409 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-05-10T10:34:30.410 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:34:30.519 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2024-05-10T10:34:30.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:30.529 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:34:30.529 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:34:30.529 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:34:30.529 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:34:30.529 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-05-10T10:34:30.529 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:34:30.637 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628279 2024-05-10T10:34:30.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:30.647 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:34:30.647 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:34:30.647 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:34:30.647 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:34:30.647 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-05-10T10:34:30.647 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:34:30.756 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2024-05-10T10:34:30.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:30.765 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:34:30.765 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:34:30.765 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:34:30.765 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:34:30.765 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-05-10T10:34:30.765 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:34:30.885 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628280 2024-05-10T10:34:30.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:34:30.895 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:30.895 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:30.953 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:30.953 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:31.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:34:31.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.153+0000 7f6015990640 1 Processor -- start 2024-05-10T10:34:31.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.153+0000 7f6015990640 1 -- start start 2024-05-10T10:34:31.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.154+0000 7f6015990640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6010162dd0 0x7f60100cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.154+0000 7f6015990640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d2270 0x7f60100d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.154+0000 7f6015990640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60100b5e80 con 0x7f60100d3980 2024-05-10T10:34:31.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.154+0000 7f6015990640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f60100b5c30 con 0x7f6010162dd0 2024-05-10T10:34:31.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.154+0000 7f6015990640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f60100d5c50 con 0x7f60100d2270 2024-05-10T10:34:31.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.154+0000 7f6007fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6010162dd0 0x7f60100cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f600effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f60100d3980 0x7f60101626c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60078/0 (socket says 172.21.15.53:60078) 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f600effd640 1 -- 172.21.15.53:0/4151388914 learned_addr learned my addr 172.21.15.53:0/4151388914 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f600f7fe640 1 --2- 172.21.15.53:0/4151388914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d2270 0x7f60100d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f6007fff640 1 -- 172.21.15.53:0/4151388914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d2270 msgr2=0x7f60100d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f6007fff640 1 --2- 172.21.15.53:0/4151388914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d2270 0x7f60100d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f6007fff640 1 -- 172.21.15.53:0/4151388914 >> v1:172.21.15.53:6789/0 conn(0x7f60100d3980 legacy=0x7f60101626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f600e7fc640 1 -- 172.21.15.53:0/4151388914 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 286534885 0 0) 0x7f60100b5e80 con 0x7f60100d3980 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.155+0000 7f6007fff640 1 -- 172.21.15.53:0/4151388914 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60100d5f30 con 0x7f6010162dd0 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f6007fff640 1 --2- 172.21.15.53:0/4151388914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60100cfeb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80097a0 tx=0x7f5ff802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c82ea89bbf0462 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:31.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f600e7fc640 1 -- 172.21.15.53:0/4151388914 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8037070 con 0x7f6010162dd0 2024-05-10T10:34:31.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f600e7fc640 1 -- 172.21.15.53:0/4151388914 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff8032440 con 0x7f6010162dd0 2024-05-10T10:34:31.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f600e7fc640 1 -- 172.21.15.53:0/4151388914 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8032810 con 0x7f6010162dd0 2024-05-10T10:34:31.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f6015990640 1 -- 172.21.15.53:0/4151388914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 msgr2=0x7f60100cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f6015990640 1 --2- 172.21.15.53:0/4151388914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60100cfeb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80097a0 tx=0x7f5ff802a180 comp rx=0 tx=0).stop 2024-05-10T10:34:31.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f6015990640 1 -- 172.21.15.53:0/4151388914 shutdown_connections 2024-05-10T10:34:31.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f6015990640 1 --2- 172.21.15.53:0/4151388914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d2270 0x7f60100d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.156+0000 7f6015990640 1 --2- 172.21.15.53:0/4151388914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60100cfeb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.157+0000 7f6015990640 1 -- 172.21.15.53:0/4151388914 >> 172.21.15.53:0/4151388914 conn(0x7f60100d5130 msgr2=0x7f60100d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:31.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.157+0000 7f6015990640 1 -- 172.21.15.53:0/4151388914 shutdown_connections 2024-05-10T10:34:31.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.157+0000 7f6015990640 1 -- 172.21.15.53:0/4151388914 wait complete. 2024-05-10T10:34:31.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 Processor -- start 2024-05-10T10:34:31.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 -- start start 2024-05-10T10:34:31.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60100d2270 0x7f601014c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d3980 0x7f60100cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60101e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60100b6890 con 0x7f6010162dd0 2024-05-10T10:34:31.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60100d31f0 con 0x7f60100d2270 2024-05-10T10:34:31.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.158+0000 7f6015990640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f60100b6160 con 0x7f60100d3980 2024-05-10T10:34:31.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60101e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60101e8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44524/0 (socket says 172.21.15.53:44524) 2024-05-10T10:34:31.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 -- 172.21.15.53:0/3405435026 learned_addr learned my addr 172.21.15.53:0/3405435026 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:31.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600effd640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60100d2270 0x7f601014c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f6007fff640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d3980 0x7f60100cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 -- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d3980 msgr2=0x7f60100cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d3980 0x7f60100cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 -- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60100d2270 msgr2=0x7f601014c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60100d2270 0x7f601014c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 -- 172.21.15.53:0/3405435026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60100ced20 con 0x7f6010162dd0 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600f7fe640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60101e8f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6008007e30 tx=0x7f600800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600cff9640 1 -- 172.21.15.53:0/3405435026 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008014620 con 0x7f6010162dd0 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600cff9640 1 -- 172.21.15.53:0/3405435026 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6008019070 con 0x7f6010162dd0 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60100cf000 con 0x7f6010162dd0 2024-05-10T10:34:31.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f600cff9640 1 -- 172.21.15.53:0/3405435026 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008014a00 con 0x7f6010162dd0 2024-05-10T10:34:31.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.159+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60100ce9a0 con 0x7f6010162dd0 2024-05-10T10:34:31.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.161+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60100cf520 con 0x7f6010162dd0 2024-05-10T10:34:31.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.161+0000 7f600cff9640 1 -- 172.21.15.53:0/3405435026 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6008014ba0 con 0x7f6010162dd0 2024-05-10T10:34:31.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.162+0000 7f600cff9640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5ff00762d0 0x7f5ff0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.162+0000 7f600cff9640 1 -- 172.21.15.53:0/3405435026 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f6008098060 con 0x7f6010162dd0 2024-05-10T10:34:31.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.162+0000 7f600effd640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5ff00762d0 0x7f5ff0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.163+0000 7f600effd640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5ff00762d0 0x7f5ff0078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6000000f60 tx=0x7f6000007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:31.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.166+0000 7f600cff9640 1 -- 172.21.15.53:0/3405435026 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f60100cf520 con 0x7f6010162dd0 2024-05-10T10:34:31.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.372+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f601006a220 con 0x7f6010162dd0 2024-05-10T10:34:31.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.372+0000 7f600cff9640 1 -- 172.21.15.53:0/3405435026 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f6008061830 con 0x7f6010162dd0 2024-05-10T10:34:31.374 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:34:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5ff00762d0 msgr2=0x7f5ff0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5ff00762d0 0x7f5ff0078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6000000f60 tx=0x7f6000007b50 comp rx=0 tx=0).stop 2024-05-10T10:34:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 msgr2=0x7f60101e8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60101e8f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6008007e30 tx=0x7f600800e4b0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 shutdown_connections 2024-05-10T10:34:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5ff00762d0 0x7f5ff0078790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6010162dd0 0x7f60101e8f30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60100d3980 0x7f60100cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 --2- 172.21.15.53:0/3405435026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60100d2270 0x7f601014c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 >> 172.21.15.53:0/3405435026 conn(0x7f60100d5130 msgr2=0x7f60100d5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 shutdown_connections 2024-05-10T10:34:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.374+0000 7f6015990640 1 -- 172.21.15.53:0/3405435026 wait complete. 2024-05-10T10:34:31.388 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:34:31.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:31.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:34:31.445 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:34:31.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:34:31.445 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:34:31.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.614+0000 7f03e8c59640 1 Processor -- start 2024-05-10T10:34:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.614+0000 7f03e8c59640 1 -- start start 2024-05-10T10:34:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e8c59640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03e415f8c0 0x7f03e406e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e8c59640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e406e620 0x7f03e40d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e8c59640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e40b0a90 con 0x7f03e415f4e0 2024-05-10T10:34:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e8c59640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f03e40b23f0 con 0x7f03e415f8c0 2024-05-10T10:34:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e8c59640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f03e40b4150 con 0x7f03e406e620 2024-05-10T10:34:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03dbfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03e415f8c0 0x7f03e406e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e406e620 0x7f03e40d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e406e620 0x7f03e40d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:51362/0 (socket says 172.21.15.53:51362) 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03dbfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f03e415f8c0 0x7f03e406e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:43592/0 (socket says 172.21.15.53:43592) 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e259c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f03e415f4e0 0x7f03e406d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:49864/0 (socket says 172.21.15.53:49864) 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e2d9d640 1 -- 172.21.15.53:0/1593605268 learned_addr learned my addr 172.21.15.53:0/1593605268 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e2d9d640 1 -- 172.21.15.53:0/1593605268 >> v2:172.21.15.135:3300/0 conn(0x7f03e415f8c0 msgr2=0x7f03e406e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e2d9d640 1 --2- 172.21.15.53:0/1593605268 >> v2:172.21.15.135:3300/0 conn(0x7f03e415f8c0 0x7f03e406e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e2d9d640 1 -- 172.21.15.53:0/1593605268 >> v1:172.21.15.135:6789/0 conn(0x7f03e415f4e0 legacy=0x7f03e406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03e2d9d640 1 -- 172.21.15.53:0/1593605268 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e40b3540 con 0x7f03e406e620 2024-05-10T10:34:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.615+0000 7f03dbfff640 1 --2- 172.21.15.53:0/1593605268 >> v2:172.21.15.135:3300/0 conn(0x7f03e415f8c0 0x7f03e406e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.616+0000 7f03e2d9d640 1 --2- 172.21.15.53:0/1593605268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e406e620 0x7f03e40d0cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e50 tx=0x7f03dc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c6ac32578f6943 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.616+0000 7f03e1d9b640 1 -- 172.21.15.53:0/1593605268 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc036070 con 0x7f03e406e620 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.616+0000 7f03e1d9b640 1 -- 172.21.15.53:0/1593605268 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc031600 con 0x7f03e406e620 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.616+0000 7f03e1d9b640 1 -- 172.21.15.53:0/1593605268 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0319d0 con 0x7f03e406e620 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 -- 172.21.15.53:0/1593605268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e406e620 msgr2=0x7f03e40d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 --2- 172.21.15.53:0/1593605268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e406e620 0x7f03e40d0cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e50 tx=0x7f03dc02a680 comp rx=0 tx=0).stop 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 -- 172.21.15.53:0/1593605268 shutdown_connections 2024-05-10T10:34:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 --2- 172.21.15.53:0/1593605268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e406e620 0x7f03e40d0cf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 --2- 172.21.15.53:0/1593605268 >> v2:172.21.15.135:3300/0 conn(0x7f03e415f8c0 0x7f03e406e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 -- 172.21.15.53:0/1593605268 >> 172.21.15.53:0/1593605268 conn(0x7f03e40d2df0 msgr2=0x7f03e40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 -- 172.21.15.53:0/1593605268 shutdown_connections 2024-05-10T10:34:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.617+0000 7f03e8c59640 1 -- 172.21.15.53:0/1593605268 wait complete. 2024-05-10T10:34:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.618+0000 7f03e8c59640 1 Processor -- start 2024-05-10T10:34:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.618+0000 7f03e8c59640 1 -- start start 2024-05-10T10:34:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e8c59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03e406e620 0x7f03e41f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e8c59640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e415f4e0 0x7f03e40c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e8c59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03e415f8c0 0x7f03e4152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e8c59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40b3de0 con 0x7f03e415f8c0 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e8c59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03e40b3b00 con 0x7f03e406e620 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e8c59640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f03e40b45e0 con 0x7f03e415f4e0 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03e415f8c0 0x7f03e4152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03e415f8c0 0x7f03e4152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44538/0 (socket says 172.21.15.53:44538) 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 -- 172.21.15.53:0/3628399159 learned_addr learned my addr 172.21.15.53:0/3628399159 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e259c640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03e406e620 0x7f03e41f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03dbfff640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e415f4e0 0x7f03e40c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 -- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e415f4e0 msgr2=0x7f03e40c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e415f4e0 0x7f03e40c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 -- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03e406e620 msgr2=0x7f03e41f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03e406e620 0x7f03e41f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.619+0000 7f03e2d9d640 1 -- 172.21.15.53:0/3628399159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e415b630 con 0x7f03e415f8c0 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.620+0000 7f03dbfff640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e415f4e0 0x7f03e40c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.620+0000 7f03e2d9d640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03e415f8c0 0x7f03e4152100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e50 tx=0x7f03dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.620+0000 7f03daffd640 1 -- 172.21.15.53:0/3628399159 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc036050 con 0x7f03e415f8c0 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.620+0000 7f03daffd640 1 -- 172.21.15.53:0/3628399159 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc042070 con 0x7f03e415f8c0 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.620+0000 7f03daffd640 1 -- 172.21.15.53:0/3628399159 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc009510 con 0x7f03e415f8c0 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.620+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e415b910 con 0x7f03e415f8c0 2024-05-10T10:34:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.620+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e41f4b40 con 0x7f03e415f8c0 2024-05-10T10:34:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.621+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e4160720 con 0x7f03e415f8c0 2024-05-10T10:34:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.622+0000 7f03daffd640 1 -- 172.21.15.53:0/3628399159 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f03dc007bf0 con 0x7f03e415f8c0 2024-05-10T10:34:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.622+0000 7f03daffd640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03c40762d0 0x7f03c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.622+0000 7f03daffd640 1 -- 172.21.15.53:0/3628399159 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f03dc0b84e0 con 0x7f03e415f8c0 2024-05-10T10:34:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.622+0000 7f03e259c640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03c40762d0 0x7f03c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.623+0000 7f03e259c640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03c40762d0 0x7f03c4078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000f60 tx=0x7f03d4007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.626+0000 7f03daffd640 1 -- 172.21.15.53:0/3628399159 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f03dc081d40 con 0x7f03e415f8c0 2024-05-10T10:34:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.856+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e415e800 con 0x7f03e415f8c0 2024-05-10T10:34:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.857+0000 7f03daffd640 1 -- 172.21.15.53:0/3628399159 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7f03dc081640 con 0x7f03e415f8c0 2024-05-10T10:34:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:31.859 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03c40762d0 msgr2=0x7f03c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03c40762d0 0x7f03c4078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000f60 tx=0x7f03d4007cc0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03e415f8c0 msgr2=0x7f03e4152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03e415f8c0 0x7f03e4152100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e50 tx=0x7f03dc004a40 comp rx=0 tx=0).stop 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 shutdown_connections 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03c40762d0 0x7f03c4078790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03e415f8c0 0x7f03e4152100 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03e415f4e0 0x7f03e40c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 --2- 172.21.15.53:0/3628399159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03e406e620 0x7f03e41f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 >> 172.21.15.53:0/3628399159 conn(0x7f03e40d2df0 msgr2=0x7f03e4070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 shutdown_connections 2024-05-10T10:34:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:31.859+0000 7f03e8c59640 1 -- 172.21.15.53:0/3628399159 wait complete. 2024-05-10T10:34:31.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-10T10:34:32.004 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:34:32.004 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:34:32.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.107+0000 7fa97dbe4640 1 Processor -- start 2024-05-10T10:34:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.107+0000 7fa97dbe4640 1 -- start start 2024-05-10T10:34:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.107+0000 7fa97dbe4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 0x7fa97816a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.107+0000 7fa97dbe4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9780b0a90 con 0x7fa97815f4e0 2024-05-10T10:34:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.107+0000 7fa97dbe4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9780b3540 con 0x7fa97815f8c0 2024-05-10T10:34:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.107+0000 7fa97dbe4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa9780b23f0 con 0x7fa978168220 2024-05-10T10:34:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9777fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa97815f4e0 0x7fa97806d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60084/0 (socket says 172.21.15.53:60084) 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9777fe640 1 -- 172.21.15.53:0/4178337480 learned_addr learned my addr 172.21.15.53:0/4178337480 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa977fff640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 0x7fa97816a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104197254 0 0) 0x7fa9780b0a90 con 0x7fa97815f4e0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa960003520 con 0x7fa97815f4e0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948237047 0 0) 0x7fa9780b3540 con 0x7fa97815f8c0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9780b0a90 con 0x7fa97815f8c0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1312140402 0 0) 0x7fa960003520 con 0x7fa97815f4e0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 msgr2=0x7fa97816a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 0x7fa97816a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 >> v1:172.21.15.135:6789/0 conn(0x7fa97815f8c0 legacy=0x7fa97806e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9780b3820 con 0x7fa97815f4e0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fa968001670 con 0x7fa97815f4e0 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 >> v1:172.21.15.53:6789/0 conn(0x7fa97815f4e0 legacy=0x7fa97806d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 0x7fa960003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9600040d0 0x7fa960004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960004d40 0x7fa9600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780b23f0 con 0x7fa9600040d0 2024-05-10T10:34:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa96000bb50 con 0x7fa960004d40 2024-05-10T10:34:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa96000bd20 con 0x7fa978168220 2024-05-10T10:34:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.108+0000 7fa976ffd640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9600040d0 0x7fa960004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa977fff640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960004d40 0x7fa9600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa9777fe640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 0x7fa960003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa976ffd640 1 -- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 msgr2=0x7fa960003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa976ffd640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 0x7fa960003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa976ffd640 1 -- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960004d40 msgr2=0x7fa9600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa976ffd640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960004d40 0x7fa9600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa976ffd640 1 -- 172.21.15.53:0/4178337480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa960003520 con 0x7fa9600040d0 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa9777fe640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa978168220 0x7fa960003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.109+0000 7fa976ffd640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9600040d0 0x7fa960004800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa978073ad0 tx=0x7fa960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c634555012ce857 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0014f0 con 0x7fa9600040d0 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c0052d0 con 0x7fa9600040d0 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa9767fc640 1 -- 172.21.15.53:0/4178337480 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c02ac20 con 0x7fa9600040d0 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa97dbe4640 1 -- 172.21.15.53:0/4178337480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9600040d0 msgr2=0x7fa960004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa97dbe4640 1 --2- 172.21.15.53:0/4178337480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9600040d0 0x7fa960004800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa978073ad0 tx=0x7fa960005830 comp rx=0 tx=0).stop 2024-05-10T10:34:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa977fff640 1 -- 172.21.15.53:0/4178337480 reap_dead start 2024-05-10T10:34:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa97dbe4640 1 -- 172.21.15.53:0/4178337480 shutdown_connections 2024-05-10T10:34:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa97dbe4640 1 -- 172.21.15.53:0/4178337480 >> 172.21.15.53:0/4178337480 conn(0x7fa9780d2df0 msgr2=0x7fa9780d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa97dbe4640 1 -- 172.21.15.53:0/4178337480 shutdown_connections 2024-05-10T10:34:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.110+0000 7fa97dbe4640 1 -- 172.21.15.53:0/4178337480 wait complete. 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.111+0000 7fa97dbe4640 1 Processor -- start 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.112+0000 7fa97dbe4640 1 -- start start 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.112+0000 7fa97dbe4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9600040d0 0x7fa9781f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.112+0000 7fa97dbe4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9780c8660 0x7fa97815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa97dbe4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9780b7d20 0x7fa97815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa97dbe4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780b3de0 con 0x7fa9780c8660 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa97dbe4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780b4150 con 0x7fa9780b7d20 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa97dbe4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa97806e7f0 con 0x7fa9600040d0 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9780c8660 0x7fa97815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9780c8660 0x7fa97815bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44566/0 (socket says 172.21.15.53:44566) 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 -- 172.21.15.53:0/3801348062 learned_addr learned my addr 172.21.15.53:0/3801348062 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa977fff640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9780b7d20 0x7fa97815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa9777fe640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9600040d0 0x7fa9781f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 -- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9600040d0 msgr2=0x7fa9781f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9600040d0 0x7fa9781f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 -- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9780b7d20 msgr2=0x7fa97815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9780b7d20 0x7fa97815a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 -- 172.21.15.53:0/3801348062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9781f5a80 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa976ffd640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9780c8660 0x7fa97815bd30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa96c005040 tx=0x7fa96c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa95ffff640 1 -- 172.21.15.53:0/3801348062 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c040020 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa95ffff640 1 -- 172.21.15.53:0/3801348062 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c005480 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa95ffff640 1 -- 172.21.15.53:0/3801348062 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0014f0 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9780c8e40 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.113+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9780b53c0 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.115+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97806a220 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.115+0000 7fa95ffff640 1 -- 172.21.15.53:0/3801348062 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa96c043020 con 0x7fa9780c8660 2024-05-10T10:34:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.116+0000 7fa95ffff640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9640762d0 0x7fa964078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.116+0000 7fa95ffff640 1 -- 172.21.15.53:0/3801348062 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fa96c0b7580 con 0x7fa9780c8660 2024-05-10T10:34:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.116+0000 7fa9777fe640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9640762d0 0x7fa964078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.116+0000 7fa9777fe640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9640762d0 0x7fa964078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa9780c85f0 tx=0x7fa968007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.120+0000 7fa95ffff640 1 -- 172.21.15.53:0/3801348062 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa96c04a030 con 0x7fa9780c8660 2024-05-10T10:34:32.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.350+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9780714e0 con 0x7fa9780c8660 2024-05-10T10:34:32.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.350+0000 7fa95ffff640 1 -- 172.21.15.53:0/3801348062 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7fa96c080790 con 0x7fa9780c8660 2024-05-10T10:34:32.352 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:32.352 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:32.352 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:32.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.352+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9640762d0 msgr2=0x7fa964078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:32.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.352+0000 7fa97dbe4640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9640762d0 0x7fa964078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa9780c85f0 tx=0x7fa968007040 comp rx=0 tx=0).stop 2024-05-10T10:34:32.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.352+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9780c8660 msgr2=0x7fa97815bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:32.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.352+0000 7fa97dbe4640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9780c8660 0x7fa97815bd30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa96c005040 tx=0x7fa96c0012d0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 shutdown_connections 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9640762d0 0x7fa964078790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9780b7d20 0x7fa97815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9780c8660 0x7fa97815bd30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 --2- 172.21.15.53:0/3801348062 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9600040d0 0x7fa9781f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 >> 172.21.15.53:0/3801348062 conn(0x7fa9780d2df0 msgr2=0x7fa978070390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 shutdown_connections 2024-05-10T10:34:32.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:32.353+0000 7fa97dbe4640 1 -- 172.21.15.53:0/3801348062 wait complete. 2024-05-10T10:34:32.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-05-10T10:34:32.489 INFO:teuthology.orchestra.run.smithi053.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-05-10T10:34:32.490 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-05-10T10:34:42.491 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-05-10T10:34:42.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 Processor -- start 2024-05-10T10:34:42.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 -- start start 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012d4f9b0 0x564012d94540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 0x564012e272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012e1d540 0x564012e25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564012e29fc0 con 0x564012e1d540 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564012cf8f50 con 0x564012d4f9b0 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.543+0000 7f58aea1bf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x564012d9ea80 con 0x564012e28850 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012e1d540 0x564012e25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012e1d540 0x564012e25f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39068/0 (socket says 172.21.15.53:39068) 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 -- 172.21.15.53:0/1619997543 learned_addr learned my addr 172.21.15.53:0/1619997543 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58ac515640 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012d4f9b0 0x564012d94540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58abd14640 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 0x564012e272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 -- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 msgr2=0x564012e272f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 0x564012e272f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 -- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012d4f9b0 msgr2=0x564012d94540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012d4f9b0 0x564012d94540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 -- 172.21.15.53:0/1619997543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564012e23010 con 0x564012e1d540 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58acd16640 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012e1d540 0x564012e25f90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58a4025a20 tx=0x7f58a4050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b764a2c782dc26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58aad12640 1 -- 172.21.15.53:0/1619997543 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4055030 con 0x564012e1d540 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58aad12640 1 -- 172.21.15.53:0/1619997543 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a4051b20 con 0x564012e1d540 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.544+0000 7f58aad12640 1 -- 172.21.15.53:0/1619997543 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a405b430 con 0x564012e1d540 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 -- 172.21.15.53:0/1619997543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012e1d540 msgr2=0x564012e25f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012e1d540 0x564012e25f90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58a4025a20 tx=0x7f58a4050000 comp rx=0 tx=0).stop 2024-05-10T10:34:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 -- 172.21.15.53:0/1619997543 shutdown_connections 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012e1d540 0x564012e25f90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 0x564012e272f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/1619997543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012d4f9b0 0x564012d94540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 -- 172.21.15.53:0/1619997543 >> 172.21.15.53:0/1619997543 conn(0x564012d9edd0 msgr2=0x564012d50cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 -- 172.21.15.53:0/1619997543 shutdown_connections 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.545+0000 7f58aea1bf40 1 -- 172.21.15.53:0/1619997543 wait complete. 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.546+0000 7f58aea1bf40 1 Processor -- start 2024-05-10T10:34:42.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.546+0000 7f58aea1bf40 1 -- start start 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58aea1bf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012d4f9b0 0x564012e1f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58aea1bf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012e1d540 0x564012eba9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58aea1bf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 0x564012eb84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58aea1bf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564012e28070 con 0x564012d4f9b0 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58aea1bf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564012e26fe0 con 0x564012e1d540 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58aea1bf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x564012e1e060 con 0x564012e28850 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012d4f9b0 0x564012e1f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012d4f9b0 0x564012e1f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39084/0 (socket says 172.21.15.53:39084) 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 -- 172.21.15.53:0/2729321669 learned_addr learned my addr 172.21.15.53:0/2729321669 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58acd16640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 0x564012eb84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58abd14640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012e1d540 0x564012eba9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 -- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 msgr2=0x564012eb84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564012e28850 0x564012eb84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 -- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012e1d540 msgr2=0x564012eba9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564012e1d540 0x564012eba9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 -- 172.21.15.53:0/2729321669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564012e1d010 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58ac515640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012d4f9b0 0x564012e1f420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f589c008680 tx=0x7f589c008770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.547+0000 7f58a9d10640 1 -- 172.21.15.53:0/2729321669 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c019020 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.548+0000 7f58a9d10640 1 -- 172.21.15.53:0/2729321669 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589c015880 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.548+0000 7f58a9d10640 1 -- 172.21.15.53:0/2729321669 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c015c30 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.548+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564012ebb980 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.548+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564012ebb3c0 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.549+0000 7f58a9d10640 1 -- 172.21.15.53:0/2729321669 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f589c01c020 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.550+0000 7f58a9d10640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5898074cb0 0x7f5898077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.550+0000 7f58a9d10640 1 -- 172.21.15.53:0/2729321669 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f589c096d90 con 0x564012d4f9b0 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.550+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x564012ebaf20 0x564012e1b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.550+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x564012d51c10 con 0x564012ebaf20 2024-05-10T10:34:42.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.550+0000 7f58abd14640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5898074cb0 0x7f5898077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.550+0000 7f58acd16640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x564012ebaf20 0x564012e1b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58abd14640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5898074cb0 0x7f5898077170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x564012d94b60 tx=0x7f58a0001770 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58acd16640 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x564012ebaf20 0x564012e1b870 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58acd16640 1 -- 172.21.15.53:0/2729321669 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f58a4051cd0 con 0x564012ebaf20 2024-05-10T10:34:42.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x564012ebaf20 msgr2=0x564012e1b870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x564012ebaf20 0x564012e1b870 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5898074cb0 msgr2=0x7f5898077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5898074cb0 0x7f5898077170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x564012d94b60 tx=0x7f58a0001770 comp rx=0 tx=0).stop 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012d4f9b0 msgr2=0x564012e1f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 --2- 172.21.15.53:0/2729321669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564012d4f9b0 0x564012e1f420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f589c008680 tx=0x7f589c008770 comp rx=0 tx=0).stop 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58acd16640 1 -- 172.21.15.53:0/2729321669 reap_dead start 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 shutdown_connections 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.551+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 >> 172.21.15.53:0/2729321669 conn(0x564012d9edd0 msgr2=0x564012e25f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.552+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 shutdown_connections 2024-05-10T10:34:42.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.552+0000 7f58aea1bf40 1 -- 172.21.15.53:0/2729321669 wait complete. 2024-05-10T10:34:42.557 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2024-05-10T10:34:42.557 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-05-10T10:34:42.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.603+0000 7fc8cfe75f40 1 Processor -- start 2024-05-10T10:34:42.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.603+0000 7fc8cfe75f40 1 -- start start 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cfe75f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f917f781f0 0x55f917ffa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cfe75f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cfe75f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f918005790 0x55f917f5fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cfe75f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f917f61780 con 0x55f9180053b0 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cfe75f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f917ed8f50 con 0x55f918005790 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cfe75f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f917f61a60 con 0x55f917f781f0 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cd16e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cd16e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39094/0 (socket says 172.21.15.53:39094) 2024-05-10T10:34:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cd16e640 1 -- 172.21.15.53:0/3668904789 learned_addr learned my addr 172.21.15.53:0/3668904789 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8cd96f640 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f917f781f0 0x55f917ffa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.604+0000 7fc8ce170640 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f918005790 0x55f917f5fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8cd16e640 1 -- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f917f781f0 msgr2=0x55f917ffa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8cd16e640 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f917f781f0 0x55f917ffa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8cd16e640 1 -- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f918005790 msgr2=0x55f917f5fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8cd16e640 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f918005790 0x55f917f5fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8cd16e640 1 -- 172.21.15.53:0/3668904789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f918004620 con 0x55f9180053b0 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8ce170640 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f918005790 0x55f917f5fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8cd16e640 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8001dc0 tx=0x7fc8b800b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f504a4b433ad028 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8c7fff640 1 -- 172.21.15.53:0/3668904789 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8010640 con 0x55f9180053b0 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8c7fff640 1 -- 172.21.15.53:0/3668904789 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8009070 con 0x55f9180053b0 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.605+0000 7fc8c7fff640 1 -- 172.21.15.53:0/3668904789 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8010a40 con 0x55f9180053b0 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/3668904789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 msgr2=0x55f917f32b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8001dc0 tx=0x7fc8b800b590 comp rx=0 tx=0).stop 2024-05-10T10:34:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/3668904789 shutdown_connections 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f918005790 0x55f917f5fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/3668904789 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f917f781f0 0x55f917ffa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/3668904789 >> 172.21.15.53:0/3668904789 conn(0x55f917f5b800 msgr2=0x55f917f60ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/3668904789 shutdown_connections 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/3668904789 wait complete. 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 Processor -- start 2024-05-10T10:34:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.606+0000 7fc8cfe75f40 1 -- start start 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cfe75f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f917f781f0 0x55f917ff3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cfe75f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cfe75f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f918005790 0x55f917f330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cfe75f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f917f5aef0 con 0x55f9180053b0 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cfe75f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f918004970 con 0x55f917f781f0 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cfe75f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f917f32160 con 0x55f918005790 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39108/0 (socket says 172.21.15.53:39108) 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 -- 172.21.15.53:0/936593320 learned_addr learned my addr 172.21.15.53:0/936593320 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8ce170640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f918005790 0x55f917f330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd96f640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f917f781f0 0x55f917ff3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 -- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f918005790 msgr2=0x55f917f330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f918005790 0x55f917f330b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 -- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f917f781f0 msgr2=0x55f917ff3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f917f781f0 0x55f917ff3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.607+0000 7fc8cd16e640 1 -- 172.21.15.53:0/936593320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f917f530e0 con 0x55f9180053b0 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.608+0000 7fc8cd96f640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f917f781f0 0x55f917ff3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.608+0000 7fc8cd16e640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b5d0 tx=0x7fc8b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.608+0000 7fc8c6ffd640 1 -- 172.21.15.53:0/936593320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b801c630 con 0x55f9180053b0 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.608+0000 7fc8c6ffd640 1 -- 172.21.15.53:0/936593320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b801c7d0 con 0x55f9180053b0 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.608+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f917f75040 con 0x55f9180053b0 2024-05-10T10:34:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.608+0000 7fc8c6ffd640 1 -- 172.21.15.53:0/936593320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b801cba0 con 0x55f9180053b0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.608+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f917f75600 con 0x55f9180053b0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.609+0000 7fc8c6ffd640 1 -- 172.21.15.53:0/936593320 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc8b8024400 con 0x55f9180053b0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.610+0000 7fc8c6ffd640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc8b0074ce0 0x7fc8b00771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.610+0000 7fc8c6ffd640 1 -- 172.21.15.53:0/936593320 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fc8b80985a0 con 0x55f9180053b0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.610+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55f9180062c0 0x55f917f76030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.610+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x55f917f52c20 con 0x55f9180062c0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.610+0000 7fc8cd96f640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc8b0074ce0 0x7fc8b00771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.610+0000 7fc8ce170640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55f9180062c0 0x55f917f76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cd96f640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc8b0074ce0 0x7fc8b00771a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0057000 tx=0x7fc8c0007030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8ce170640 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55f9180062c0 0x55f917f76030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8ce170640 1 -- 172.21.15.53:0/936593320 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fc8c8009a30 con 0x55f9180062c0 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55f9180062c0 msgr2=0x55f917f76030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55f9180062c0 0x55f917f76030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc8b0074ce0 msgr2=0x7fc8b00771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc8b0074ce0 0x7fc8b00771a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0057000 tx=0x7fc8c0007030 comp rx=0 tx=0).stop 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 msgr2=0x55f917f32b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 --2- 172.21.15.53:0/936593320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f9180053b0 0x55f917f32b70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b5d0 tx=0x7fc8b8000bc0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8ce170640 1 -- 172.21.15.53:0/936593320 reap_dead start 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 shutdown_connections 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.611+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 >> 172.21.15.53:0/936593320 conn(0x55f917f5b800 msgr2=0x55f917ff7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.612+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 shutdown_connections 2024-05-10T10:34:42.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.612+0000 7fc8cfe75f40 1 -- 172.21.15.53:0/936593320 wait complete. 2024-05-10T10:34:42.618 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2024-05-10T10:34:42.619 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-10T10:34:42.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.664+0000 7fa52cae7f40 1 Processor -- start 2024-05-10T10:34:42.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.664+0000 7fa52cae7f40 1 -- start start 2024-05-10T10:34:42.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52cae7f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba8dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52cae7f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba847a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52cae7f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba847f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52cae7f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5584ba7a96d0 con 0x5584ba7a8d40 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52cae7f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5584ba720f50 con 0x5584ba77be30 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52cae7f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5584ba849300 con 0x5584ba848f20 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa529de0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba847a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa529de0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba847a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49922/0 (socket says 172.21.15.53:49922) 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa529de0640 1 -- 172.21.15.53:0/4121538049 learned_addr learned my addr 172.21.15.53:0/4121538049 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52ade2640 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba847f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.665+0000 7fa52a5e1640 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba8dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa529de0640 1 -- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 msgr2=0x5584ba847f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa529de0640 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba847f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa529de0640 1 -- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 msgr2=0x5584ba8dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa529de0640 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba8dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa529de0640 1 -- 172.21.15.53:0/4121538049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ba845db0 con 0x5584ba7a8d40 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52a5e1640 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba8dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52ade2640 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba847f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa529de0640 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba847a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b2e0 tx=0x7fa52000b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d5e606e6c72eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa528dde640 1 -- 172.21.15.53:0/4121538049 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520008070 con 0x5584ba7a8d40 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa528dde640 1 -- 172.21.15.53:0/4121538049 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52000d500 con 0x5584ba7a8d40 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa528dde640 1 -- 172.21.15.53:0/4121538049 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52000d8f0 con 0x5584ba7a8d40 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 -- 172.21.15.53:0/4121538049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 msgr2=0x5584ba847a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba847a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b2e0 tx=0x7fa52000b7a0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 -- 172.21.15.53:0/4121538049 shutdown_connections 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba847f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba847a20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/4121538049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba8dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 -- 172.21.15.53:0/4121538049 >> 172.21.15.53:0/4121538049 conn(0x5584ba7a6930 msgr2=0x5584ba7a6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 -- 172.21.15.53:0/4121538049 shutdown_connections 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.666+0000 7fa52cae7f40 1 -- 172.21.15.53:0/4121538049 wait complete. 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.667+0000 7fa52cae7f40 1 Processor -- start 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.667+0000 7fa52cae7f40 1 -- start start 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52cae7f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba7a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52cae7f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba77a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52cae7f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba8de550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52cae7f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5584ba846030 con 0x5584ba7a8d40 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52cae7f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5584ba84bd90 con 0x5584ba77be30 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52cae7f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5584ba84e480 con 0x5584ba848f20 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba77a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba77a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49924/0 (socket says 172.21.15.53:49924) 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 -- 172.21.15.53:0/3025570538 learned_addr learned my addr 172.21.15.53:0/3025570538 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52a5e1640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba7a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52ade2640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba8de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 -- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 msgr2=0x5584ba8de550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba8de550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 -- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 msgr2=0x5584ba7a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba7a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 -- 172.21.15.53:0/3025570538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ba7a5f10 con 0x5584ba7a8d40 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52ade2640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5584ba848f20 0x5584ba8de550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa52a5e1640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584ba77be30 0x5584ba7a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa529de0640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba77a060 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b7e0 tx=0x7fa520000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.668+0000 7fa51b7fe640 1 -- 172.21.15.53:0/3025570538 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52001c650 con 0x5584ba7a8d40 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.669+0000 7fa51b7fe640 1 -- 172.21.15.53:0/3025570538 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa520021020 con 0x5584ba7a8d40 2024-05-10T10:34:42.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.669+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5584ba7a5c30 con 0x5584ba7a8d40 2024-05-10T10:34:42.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.669+0000 7fa51b7fe640 1 -- 172.21.15.53:0/3025570538 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52001c850 con 0x5584ba7a8d40 2024-05-10T10:34:42.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.669+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ba845450 con 0x5584ba7a8d40 2024-05-10T10:34:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.670+0000 7fa51b7fe640 1 -- 172.21.15.53:0/3025570538 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa520017070 con 0x5584ba7a8d40 2024-05-10T10:34:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa51b7fe640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa50c074cb0 0x7fa50c077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa51b7fe640 1 -- 172.21.15.53:0/3025570538 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fa520098190 con 0x5584ba7a8d40 2024-05-10T10:34:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa52a5e1640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa50c074cb0 0x7fa50c077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x5584ba8df560 0x5584ba8dc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x5584ba8dce20 con 0x5584ba8df560 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa52ade2640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x5584ba8df560 0x5584ba8dc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa52a5e1640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa50c074cb0 0x7fa50c077170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa51c057000 tx=0x7fa51c052000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.671+0000 7fa52ade2640 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x5584ba8df560 0x5584ba8dc8e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52ade2640 1 -- 172.21.15.53:0/3025570538 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fa524011040 con 0x5584ba8df560 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x5584ba8df560 msgr2=0x5584ba8dc8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x5584ba8df560 0x5584ba8dc8e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa50c074cb0 msgr2=0x7fa50c077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa50c074cb0 0x7fa50c077170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa51c057000 tx=0x7fa51c052000 comp rx=0 tx=0).stop 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 msgr2=0x5584ba77a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 --2- 172.21.15.53:0/3025570538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584ba7a8d40 0x5584ba77a060 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b7e0 tx=0x7fa520000ed0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52ade2640 1 -- 172.21.15.53:0/3025570538 reap_dead start 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 shutdown_connections 2024-05-10T10:34:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 >> 172.21.15.53:0/3025570538 conn(0x5584ba7a6930 msgr2=0x5584ba7a6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 shutdown_connections 2024-05-10T10:34:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.672+0000 7fa52cae7f40 1 -- 172.21.15.53:0/3025570538 wait complete. 2024-05-10T10:34:42.679 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2024-05-10T10:34:42.680 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-10T10:34:42.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.722+0000 7f470071bf40 1 Processor -- start 2024-05-10T10:34:42.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.722+0000 7f470071bf40 1 -- start start 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f470071bf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb7b08d0 0x561aeb8860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f470071bf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb869270 0x561aeb867e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f470071bf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 0x561aeb884690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f470071bf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561aeb7b1260 con 0x561aeb869270 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f470071bf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561aeb7f9080 con 0x561aeb7b08d0 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f470071bf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561aeb869e70 con 0x561aeb88a380 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb869270 0x561aeb867e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb869270 0x561aeb867e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49942/0 (socket says 172.21.15.53:49942) 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 -- 172.21.15.53:0/3202292809 learned_addr learned my addr 172.21.15.53:0/3202292809 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd8ec640 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb7b08d0 0x561aeb8860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fe0ed640 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 0x561aeb884690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 -- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 msgr2=0x561aeb884690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 0x561aeb884690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 -- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb7b08d0 msgr2=0x561aeb8860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb7b08d0 0x561aeb8860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.723+0000 7f46fd0eb640 1 -- 172.21.15.53:0/3202292809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aeb718350 con 0x561aeb869270 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f46fd0eb640 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb869270 0x561aeb867e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f46e8000c00 tx=0x7f46e8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d28fc836718599 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f46f7fff640 1 -- 172.21.15.53:0/3202292809 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8062030 con 0x561aeb869270 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f46f7fff640 1 -- 172.21.15.53:0/3202292809 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e80544e0 con 0x561aeb869270 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f46f7fff640 1 -- 172.21.15.53:0/3202292809 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8054b70 con 0x561aeb869270 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 -- 172.21.15.53:0/3202292809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb869270 msgr2=0x561aeb867e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb869270 0x561aeb867e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f46e8000c00 tx=0x7f46e8051000 comp rx=0 tx=0).stop 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 -- 172.21.15.53:0/3202292809 shutdown_connections 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 0x561aeb884690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb869270 0x561aeb867e00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 --2- 172.21.15.53:0/3202292809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb7b08d0 0x561aeb8860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 -- 172.21.15.53:0/3202292809 >> 172.21.15.53:0/3202292809 conn(0x561aeb7ae7c0 msgr2=0x561aeb7ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 -- 172.21.15.53:0/3202292809 shutdown_connections 2024-05-10T10:34:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.724+0000 7f470071bf40 1 -- 172.21.15.53:0/3202292809 wait complete. 2024-05-10T10:34:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.725+0000 7f470071bf40 1 Processor -- start 2024-05-10T10:34:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.725+0000 7f470071bf40 1 -- start start 2024-05-10T10:34:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f470071bf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb7b08d0 0x561aeb864b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f470071bf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb869270 0x561aeb867e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f470071bf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 0x561aeb866920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f470071bf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561aeb868730 con 0x561aeb7b08d0 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f470071bf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561aeb868ff0 con 0x561aeb869270 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f470071bf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561aeb868d10 con 0x561aeb88a380 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb7b08d0 0x561aeb864b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb7b08d0 0x561aeb864b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49952/0 (socket says 172.21.15.53:49952) 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 -- 172.21.15.53:0/3521914275 learned_addr learned my addr 172.21.15.53:0/3521914275 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd0eb640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb869270 0x561aeb867e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fe0ed640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 0x561aeb866920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 -- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 msgr2=0x561aeb866920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561aeb88a380 0x561aeb866920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 -- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb869270 msgr2=0x561aeb867e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561aeb869270 0x561aeb867e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 -- 172.21.15.53:0/3521914275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aeb865040 con 0x561aeb7b08d0 2024-05-10T10:34:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46fd8ec640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb7b08d0 0x561aeb864b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e290 tx=0x7f46f000e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46f6ffd640 1 -- 172.21.15.53:0/3521914275 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0018020 con 0x561aeb7b08d0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.726+0000 7f46f6ffd640 1 -- 172.21.15.53:0/3521914275 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f0013870 con 0x561aeb7b08d0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.727+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561aeb870200 con 0x561aeb7b08d0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.727+0000 7f46f6ffd640 1 -- 172.21.15.53:0/3521914275 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0013c40 con 0x561aeb7b08d0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.727+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aeb86ff20 con 0x561aeb7b08d0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.728+0000 7f46f6ffd640 1 -- 172.21.15.53:0/3521914275 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f46f001b020 con 0x561aeb7b08d0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.728+0000 7f46f6ffd640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46e0074cb0 0x7f46e0077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.729+0000 7f46f6ffd640 1 -- 172.21.15.53:0/3521914275 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f46f0096700 con 0x561aeb7b08d0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.729+0000 7f46fd0eb640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46e0074cb0 0x7f46e0077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.729+0000 7f470071bf40 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x561aeb7dd390 0x561aeb7dd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.729+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x561aeb868340 con 0x561aeb7dd390 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.729+0000 7f46fe0ed640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x561aeb7dd390 0x561aeb7dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.729+0000 7f46fd0eb640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46e0074cb0 0x7f46e0077170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46e8065000 tx=0x7f46e8069040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.729+0000 7f46fe0ed640 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x561aeb7dd390 0x561aeb7dd770 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f46fe0ed640 1 -- 172.21.15.53:0/3521914275 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f46f80098e0 con 0x561aeb7dd390 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x561aeb7dd390 msgr2=0x561aeb7dd770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x561aeb7dd390 0x561aeb7dd770 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46e0074cb0 msgr2=0x7f46e0077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46e0074cb0 0x7f46e0077170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46e8065000 tx=0x7f46e8069040 comp rx=0 tx=0).stop 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb7b08d0 msgr2=0x561aeb864b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 --2- 172.21.15.53:0/3521914275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561aeb7b08d0 0x561aeb864b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e290 tx=0x7f46f000e750 comp rx=0 tx=0).stop 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f46fe0ed640 1 -- 172.21.15.53:0/3521914275 reap_dead start 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 shutdown_connections 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 >> 172.21.15.53:0/3521914275 conn(0x561aeb7ae7c0 msgr2=0x561aeb7f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 shutdown_connections 2024-05-10T10:34:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.730+0000 7f470071bf40 1 -- 172.21.15.53:0/3521914275 wait complete. 2024-05-10T10:34:42.736 INFO:teuthology.orchestra.run.smithi053.stdout:mytag 2024-05-10T10:34:42.737 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-10T10:34:42.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.781+0000 7fa6069dbf40 1 Processor -- start 2024-05-10T10:34:42.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.781+0000 7fa6069dbf40 1 -- start start 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.781+0000 7fa6069dbf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad57b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.781+0000 7fa6069dbf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad580570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.781+0000 7fa6069dbf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad585190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.781+0000 7fa6069dbf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e1ad586de0 con 0x55e1ad4f8b20 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.781+0000 7fa6069dbf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e1ad458f50 con 0x55e1ad5843f0 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa6069dbf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e1ad5870c0 con 0x55e1ad4e13e0 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad580570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad580570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49976/0 (socket says 172.21.15.53:49976) 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 -- 172.21.15.53:0/912691783 learned_addr learned my addr 172.21.15.53:0/912691783 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa604cd8640 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad585190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa6044d7640 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad57b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 -- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 msgr2=0x55e1ad57b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad57b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 -- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 msgr2=0x55e1ad585190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad585190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 -- 172.21.15.53:0/912691783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ad4b3270 con 0x55e1ad4f8b20 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa6044d7640 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad57b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa603cd6640 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad580570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b670 tx=0x7fa5f800bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c21fc9f4624ae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa602cd4640 1 -- 172.21.15.53:0/912691783 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8018070 con 0x55e1ad4f8b20 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa602cd4640 1 -- 172.21.15.53:0/912691783 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f8009990 con 0x55e1ad4f8b20 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.782+0000 7fa602cd4640 1 -- 172.21.15.53:0/912691783 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8009d60 con 0x55e1ad4f8b20 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 -- 172.21.15.53:0/912691783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 msgr2=0x55e1ad580570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad580570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b670 tx=0x7fa5f800bb30 comp rx=0 tx=0).stop 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 -- 172.21.15.53:0/912691783 shutdown_connections 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad585190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad580570 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/912691783 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad57b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 -- 172.21.15.53:0/912691783 >> 172.21.15.53:0/912691783 conn(0x55e1ad4db800 msgr2=0x55e1ad4b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 -- 172.21.15.53:0/912691783 shutdown_connections 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 -- 172.21.15.53:0/912691783 wait complete. 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 Processor -- start 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.783+0000 7fa6069dbf40 1 -- start start 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa6069dbf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad4af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa6069dbf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad581e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa6069dbf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad571df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa6069dbf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e1ad586080 con 0x55e1ad4f8b20 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa6069dbf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e1ad4b2f90 con 0x55e1ad5843f0 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa6069dbf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e1ad586360 con 0x55e1ad4e13e0 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa603cd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad581e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa603cd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad581e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49984/0 (socket says 172.21.15.53:49984) 2024-05-10T10:34:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa603cd6640 1 -- 172.21.15.53:0/911380290 learned_addr learned my addr 172.21.15.53:0/911380290 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa604cd8640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad571df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.784+0000 7fa6044d7640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad4af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa603cd6640 1 -- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 msgr2=0x55e1ad4af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa603cd6640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e1ad4e13e0 0x55e1ad4af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa603cd6640 1 -- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 msgr2=0x55e1ad571df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa603cd6640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad571df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa603cd6640 1 -- 172.21.15.53:0/911380290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ad583b90 con 0x55e1ad4f8b20 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa604cd8640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1ad5843f0 0x55e1ad571df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa603cd6640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad581e20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55e1ad4b39c0 tx=0x7fa5f8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa601cd2640 1 -- 172.21.15.53:0/911380290 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8018030 con 0x55e1ad4f8b20 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa601cd2640 1 -- 172.21.15.53:0/911380290 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f8022070 con 0x55e1ad4f8b20 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1ad570e60 con 0x55e1ad4f8b20 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa601cd2640 1 -- 172.21.15.53:0/911380290 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f801d680 con 0x55e1ad4f8b20 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.785+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ad4af5b0 con 0x55e1ad4f8b20 2024-05-10T10:34:42.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.787+0000 7fa601cd2640 1 -- 172.21.15.53:0/911380290 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa5f801d820 con 0x55e1ad4f8b20 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.787+0000 7fa601cd2640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa5f0074ce0 0x7fa5f00771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.787+0000 7fa601cd2640 1 -- 172.21.15.53:0/911380290 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fa5f8097f80 con 0x55e1ad4f8b20 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.787+0000 7fa6044d7640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa5f0074ce0 0x7fa5f00771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.787+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55e1ad581460 0x55e1ad574930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.787+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x55e1ad4e1a80 con 0x55e1ad581460 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa604cd8640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55e1ad581460 0x55e1ad574930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa6044d7640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa5f0074ce0 0x7fa5f00771a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4056000 tx=0x7fa5f4007030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa604cd8640 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55e1ad581460 0x55e1ad574930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa604cd8640 1 -- 172.21.15.53:0/911380290 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fa5fc00e040 con 0x55e1ad581460 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55e1ad581460 msgr2=0x55e1ad574930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55e1ad581460 0x55e1ad574930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa5f0074ce0 msgr2=0x7fa5f00771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa5f0074ce0 0x7fa5f00771a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4056000 tx=0x7fa5f4007030 comp rx=0 tx=0).stop 2024-05-10T10:34:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 msgr2=0x55e1ad581e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa6069dbf40 1 --2- 172.21.15.53:0/911380290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1ad4f8b20 0x55e1ad581e20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55e1ad4b39c0 tx=0x7fa5f8003020 comp rx=0 tx=0).stop 2024-05-10T10:34:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.788+0000 7fa604cd8640 1 -- 172.21.15.53:0/911380290 reap_dead start 2024-05-10T10:34:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.789+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 shutdown_connections 2024-05-10T10:34:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.789+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 >> 172.21.15.53:0/911380290 conn(0x55e1ad4db800 msgr2=0x55e1ad57b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.789+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 shutdown_connections 2024-05-10T10:34:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:42.789+0000 7fa6069dbf40 1 -- 172.21.15.53:0/911380290 wait complete. 2024-05-10T10:34:42.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:34:42.798 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:34:42.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:34:42.852 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:34:42.896 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:34:42.896 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:34:42.897 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:42.898 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:42.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:34:42.914 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:34:42.914 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:42.914 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:42.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:34:42.955 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:34:42.955 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:42.955 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:42.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:34:42.971 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:34:42.971 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:42.971 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:43.013 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:34:43.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:43.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.115+0000 7f39c47a4640 1 Processor -- start 2024-05-10T10:34:43.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.115+0000 7f39c47a4640 1 -- start start 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c47a4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39bc162750 0x7f39bc0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c47a4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f39bc162b30 0x7f39bc0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c47a4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39bc0b45f0 con 0x7f39bc0d5cb0 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c47a4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f39bc0b4f20 con 0x7f39bc162750 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c47a4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f39bc0d0fe0 con 0x7f39bc162b30 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2540640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39bc162750 0x7f39bc0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2540640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39bc162750 0x7f39bc0d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50000/0 (socket says 172.21.15.53:50000) 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2540640 1 -- 172.21.15.53:0/1251893668 learned_addr learned my addr 172.21.15.53:0/1251893668 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:43.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c1d3f640 1 --1- 172.21.15.53:0/1251893668 >> v1:172.21.15.53:6789/0 conn(0x7f39bc0d5cb0 0x7f39bc0d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50984/0 (socket says 172.21.15.53:50984) 2024-05-10T10:34:43.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2d41640 1 --2- 172.21.15.53:0/1251893668 >> v2:172.21.15.135:3300/0 conn(0x7f39bc162b30 0x7f39bc0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2540640 1 -- 172.21.15.53:0/1251893668 >> v2:172.21.15.135:3300/0 conn(0x7f39bc162b30 msgr2=0x7f39bc0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:43.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2540640 1 --2- 172.21.15.53:0/1251893668 >> v2:172.21.15.135:3300/0 conn(0x7f39bc162b30 0x7f39bc0b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2540640 1 -- 172.21.15.53:0/1251893668 >> v1:172.21.15.53:6789/0 conn(0x7f39bc0d5cb0 legacy=0x7f39bc0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c2540640 1 -- 172.21.15.53:0/1251893668 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc0b48d0 con 0x7f39bc162750 2024-05-10T10:34:43.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.116+0000 7f39c153e640 1 -- 172.21.15.53:0/1251893668 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676646672 0 0) 0x7f39bc0b45f0 con 0x7f39bc0d5cb0 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.117+0000 7f39c2540640 1 --2- 172.21.15.53:0/1251893668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc162750 0x7f39bc0d0670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0d32c0 tx=0x7f39b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b65f5c5986add2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.117+0000 7f39c153e640 1 -- 172.21.15.53:0/1251893668 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8037070 con 0x7f39bc162750 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.117+0000 7f39c153e640 1 -- 172.21.15.53:0/1251893668 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b8032440 con 0x7f39bc162750 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.117+0000 7f39c153e640 1 -- 172.21.15.53:0/1251893668 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8032810 con 0x7f39bc162750 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 -- 172.21.15.53:0/1251893668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc162750 msgr2=0x7f39bc0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1251893668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc162750 0x7f39bc0d0670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0d32c0 tx=0x7f39b802a1e0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 -- 172.21.15.53:0/1251893668 shutdown_connections 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1251893668 >> v2:172.21.15.135:3300/0 conn(0x7f39bc162b30 0x7f39bc0b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1251893668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc162750 0x7f39bc0d0670 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 -- 172.21.15.53:0/1251893668 >> 172.21.15.53:0/1251893668 conn(0x7f39bc09e3b0 msgr2=0x7f39bc0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 -- 172.21.15.53:0/1251893668 shutdown_connections 2024-05-10T10:34:43.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.118+0000 7f39c47a4640 1 -- 172.21.15.53:0/1251893668 wait complete. 2024-05-10T10:34:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.119+0000 7f39c47a4640 1 Processor -- start 2024-05-10T10:34:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.119+0000 7f39c47a4640 1 -- start start 2024-05-10T10:34:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c47a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39bc162750 0x7f39bc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c47a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39bc162b30 0x7f39bc154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c47a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc15fad0 0x7f39bc15ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c47a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0d0d30 con 0x7f39bc15fad0 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c47a4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0d0bb0 con 0x7f39bc162b30 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c47a4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f39bc0d3d40 con 0x7f39bc162750 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc15fad0 0x7f39bc15ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc15fad0 0x7f39bc15ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50016/0 (socket says 172.21.15.53:50016) 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 -- 172.21.15.53:0/1235675052 learned_addr learned my addr 172.21.15.53:0/1235675052 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c1d3f640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39bc162b30 0x7f39bc154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2540640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39bc162750 0x7f39bc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 -- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39bc162750 msgr2=0x7f39bc0b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39bc162750 0x7f39bc0b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 -- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39bc162b30 msgr2=0x7f39bc154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39bc162b30 0x7f39bc154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 -- 172.21.15.53:0/1235675052 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc0b1d40 con 0x7f39bc15fad0 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2540640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39bc162750 0x7f39bc0b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:43.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c1d3f640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39bc162b30 0x7f39bc154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:43.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.120+0000 7f39c2d41640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc15fad0 0x7f39bc15ff80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f39b4007e60 tx=0x7f39b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.121+0000 7f39b2ffd640 1 -- 172.21.15.53:0/1235675052 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4014420 con 0x7f39bc15fad0 2024-05-10T10:34:43.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.121+0000 7f39b2ffd640 1 -- 172.21.15.53:0/1235675052 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b4019070 con 0x7f39bc15fad0 2024-05-10T10:34:43.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.121+0000 7f39b2ffd640 1 -- 172.21.15.53:0/1235675052 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4014820 con 0x7f39bc15fad0 2024-05-10T10:34:43.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.121+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc0b1fe0 con 0x7f39bc15fad0 2024-05-10T10:34:43.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.121+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc0b25a0 con 0x7f39bc15fad0 2024-05-10T10:34:43.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.122+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc06a220 con 0x7f39bc15fad0 2024-05-10T10:34:43.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.122+0000 7f39b2ffd640 1 -- 172.21.15.53:0/1235675052 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f39b40149c0 con 0x7f39bc15fad0 2024-05-10T10:34:43.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.123+0000 7f39b2ffd640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39a00762d0 0x7f39a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.123+0000 7f39b2ffd640 1 -- 172.21.15.53:0/1235675052 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f39b4098000 con 0x7f39bc15fad0 2024-05-10T10:34:43.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.123+0000 7f39c2540640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39a00762d0 0x7f39a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.124+0000 7f39c2540640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39a00762d0 0x7f39a0078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39b8034000 tx=0x7f39b802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.127+0000 7f39b2ffd640 1 -- 172.21.15.53:0/1235675052 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f39b4061170 con 0x7f39bc15fad0 2024-05-10T10:34:43.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.360+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39bc1612c0 con 0x7f39bc15fad0 2024-05-10T10:34:43.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.360+0000 7f39b2ffd640 1 -- 172.21.15.53:0/1235675052 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7f39b4065470 con 0x7f39bc15fad0 2024-05-10T10:34:43.362 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:43.362 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:43.363 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:43.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39a00762d0 msgr2=0x7f39a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39a00762d0 0x7f39a0078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39b8034000 tx=0x7f39b802e000 comp rx=0 tx=0).stop 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc15fad0 msgr2=0x7f39bc15ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc15fad0 0x7f39bc15ff80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f39b4007e60 tx=0x7f39b400e4b0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 shutdown_connections 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39a00762d0 0x7f39a0078790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39bc15fad0 0x7f39bc15ff80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39bc162b30 0x7f39bc154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 --2- 172.21.15.53:0/1235675052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39bc162750 0x7f39bc0b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 >> 172.21.15.53:0/1235675052 conn(0x7f39bc09e3b0 msgr2=0x7f39bc0d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 shutdown_connections 2024-05-10T10:34:43.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.363+0000 7f39c47a4640 1 -- 172.21.15.53:0/1235675052 wait complete. 2024-05-10T10:34:43.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:43.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.477+0000 7fa18a446640 1 Processor -- start 2024-05-10T10:34:43.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.477+0000 7fa18a446640 1 -- start start 2024-05-10T10:34:43.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa18a446640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa184168220 0x7fa18416a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa18a446640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1840b0a90 con 0x7fa18415f4e0 2024-05-10T10:34:43.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa18a446640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1840b3540 con 0x7fa18415f8c0 2024-05-10T10:34:43.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa18a446640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa1840b23f0 con 0x7fa184168220 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa183fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa18415f4e0 0x7fa18406d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50996/0 (socket says 172.21.15.53:50996) 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa183fff640 1 -- 172.21.15.53:0/4054123585 learned_addr learned my addr 172.21.15.53:0/4054123585 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa1889e3640 1 --2- 172.21.15.53:0/4054123585 >> v2:172.21.15.135:3300/0 conn(0x7fa184168220 0x7fa18416a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3313480152 0 0) 0x7fa1840b0a90 con 0x7fa18415f4e0 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.478+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa170003520 con 0x7fa18415f4e0 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2343636774 0 0) 0x7fa170003520 con 0x7fa18415f4e0 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 >> v2:172.21.15.135:3300/0 conn(0x7fa184168220 msgr2=0x7fa18416a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:43.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 --2- 172.21.15.53:0/4054123585 >> v2:172.21.15.135:3300/0 conn(0x7fa184168220 0x7fa18416a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 >> v1:172.21.15.135:6789/0 conn(0x7fa18415f8c0 legacy=0x7fa18406e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1840b3820 con 0x7fa18415f4e0 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fa174001670 con 0x7fa18415f4e0 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 >> v1:172.21.15.53:6789/0 conn(0x7fa18415f4e0 legacy=0x7fa18406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1700039b0 0x7fa170003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa170004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1700051c0 0x7fa17000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840b23f0 con 0x7fa170004800 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa17000c160 con 0x7fa1700051c0 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa17000c360 con 0x7fa1700039b0 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa17bfff640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa170004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa183fff640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1700039b0 0x7fa170003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1889e3640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1700051c0 0x7fa17000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa17bfff640 1 -- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1700039b0 msgr2=0x7fa170003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa17bfff640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1700039b0 0x7fa170003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa17bfff640 1 -- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1700051c0 msgr2=0x7fa17000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa17bfff640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1700051c0 0x7fa17000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa17bfff640 1 -- 172.21.15.53:0/4054123585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa170003520 con 0x7fa170004800 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa1889e3640 1 -- 172.21.15.53:0/4054123585 reap_dead start 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.479+0000 7fa183fff640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1700039b0 0x7fa170003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.480+0000 7fa17bfff640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa170004c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa184073ad0 tx=0x7fa170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48310ce977ca88a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.480+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c001370 con 0x7fa170004800 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.480+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c0052d0 con 0x7fa170004800 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.480+0000 7fa1837fe640 1 -- 172.21.15.53:0/4054123585 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c02ab70 con 0x7fa170004800 2024-05-10T10:34:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.481+0000 7fa18a446640 1 -- 172.21.15.53:0/4054123585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 msgr2=0x7fa170004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.481+0000 7fa18a446640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa170004c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa184073ad0 tx=0x7fa170005830 comp rx=0 tx=0).stop 2024-05-10T10:34:43.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.481+0000 7fa18a446640 1 -- 172.21.15.53:0/4054123585 shutdown_connections 2024-05-10T10:34:43.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.481+0000 7fa18a446640 1 --2- 172.21.15.53:0/4054123585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa170004c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.481+0000 7fa18a446640 1 -- 172.21.15.53:0/4054123585 >> 172.21.15.53:0/4054123585 conn(0x7fa1840d2df0 msgr2=0x7fa1840d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:43.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.481+0000 7fa18a446640 1 -- 172.21.15.53:0/4054123585 shutdown_connections 2024-05-10T10:34:43.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.481+0000 7fa18a446640 1 -- 172.21.15.53:0/4054123585 wait complete. 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.482+0000 7fa18a446640 1 Processor -- start 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.482+0000 7fa18a446640 1 -- start start 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa184158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1841619e0 0x7fa1840c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa184159000 0x7fa1840c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840b3de0 con 0x7fa170004800 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840b4150 con 0x7fa184159000 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa1840b45e0 con 0x7fa1841619e0 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa184158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa184158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50040/0 (socket says 172.21.15.53:50040) 2024-05-10T10:34:43.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 -- 172.21.15.53:0/1109091644 learned_addr learned my addr 172.21.15.53:0/1109091644 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa17bfff640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1841619e0 0x7fa1840c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa1889e3640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa184159000 0x7fa1840c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 -- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1841619e0 msgr2=0x7fa1840c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1841619e0 0x7fa1840c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 -- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa184159000 msgr2=0x7fa1840c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa184159000 0x7fa1840c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 -- 172.21.15.53:0/1109091644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1840c89b0 con 0x7fa170004800 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa1889e3640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa184159000 0x7fa1840c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa183fff640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa184158ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa17400e730 tx=0x7fa17400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa1817fa640 1 -- 172.21.15.53:0/1109091644 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17400c660 con 0x7fa170004800 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa1817fa640 1 -- 172.21.15.53:0/1109091644 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa174018070 con 0x7fa170004800 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa1817fa640 1 -- 172.21.15.53:0/1109091644 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17400ca20 con 0x7fa170004800 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1841606e0 con 0x7fa170004800 2024-05-10T10:34:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.483+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa184160ca0 con 0x7fa170004800 2024-05-10T10:34:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.485+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18406a220 con 0x7fa170004800 2024-05-10T10:34:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.485+0000 7fa1817fa640 1 -- 172.21.15.53:0/1109091644 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa17400cbc0 con 0x7fa170004800 2024-05-10T10:34:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.486+0000 7fa1817fa640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa164076300 0x7fa1640787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.486+0000 7fa1817fa640 1 -- 172.21.15.53:0/1109091644 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fa1740972a0 con 0x7fa170004800 2024-05-10T10:34:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.486+0000 7fa17bfff640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa164076300 0x7fa1640787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.486+0000 7fa17bfff640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa164076300 0x7fa1640787c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa16c032000 tx=0x7fa16c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.490+0000 7fa1817fa640 1 -- 172.21.15.53:0/1109091644 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa1740609d0 con 0x7fa170004800 2024-05-10T10:34:43.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.718+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa184071db0 con 0x7fa170004800 2024-05-10T10:34:43.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.718+0000 7fa1817fa640 1 -- 172.21.15.53:0/1109091644 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7fa1740602d0 con 0x7fa170004800 2024-05-10T10:34:43.720 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:43.720 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:43.721 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:43.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa164076300 msgr2=0x7fa1640787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa164076300 0x7fa1640787c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa16c032000 tx=0x7fa16c02e000 comp rx=0 tx=0).stop 2024-05-10T10:34:43.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 msgr2=0x7fa184158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa184158ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa17400e730 tx=0x7fa17400ebf0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 shutdown_connections 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa164076300 0x7fa1640787c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa184159000 0x7fa1840c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1841619e0 0x7fa1840c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 --2- 172.21.15.53:0/1109091644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa170004800 0x7fa184158ac0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.720+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 >> 172.21.15.53:0/1109091644 conn(0x7fa1840d2df0 msgr2=0x7fa1840d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.721+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 shutdown_connections 2024-05-10T10:34:43.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.721+0000 7fa18a446640 1 -- 172.21.15.53:0/1109091644 wait complete. 2024-05-10T10:34:43.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:43.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:34:43.823 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:34:43.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.836+0000 7feb4e938640 1 Processor -- start 2024-05-10T10:34:43.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.836+0000 7feb4e938640 1 -- start start 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.837+0000 7feb4e938640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7feb48071360 0x7feb480b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.837+0000 7feb4e938640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb480b5cc0 con 0x7feb4815be70 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.837+0000 7feb4e938640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4815c250 con 0x7feb480aed60 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.837+0000 7feb4e938640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7feb4806fb70 con 0x7feb48071360 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.837+0000 7feb47fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feb4815be70 0x7feb48070880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51008/0 (socket says 172.21.15.53:51008) 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.837+0000 7feb47fff640 1 -- 172.21.15.53:0/738492066 learned_addr learned my addr 172.21.15.53:0/738492066 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb477fe640 1 --2- 172.21.15.53:0/738492066 >> v2:172.21.15.135:3300/0 conn(0x7feb48071360 0x7feb480b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 544421898 0 0) 0x7feb480b5cc0 con 0x7feb4815be70 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb30003520 con 0x7feb4815be70 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203273612 0 0) 0x7feb4815c250 con 0x7feb480aed60 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb480b5cc0 con 0x7feb480aed60 2024-05-10T10:34:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3265967949 0 0) 0x7feb30003520 con 0x7feb4815be70 2024-05-10T10:34:43.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb477fe640 1 -- 172.21.15.53:0/738492066 >> v1:172.21.15.135:6789/0 conn(0x7feb480aed60 legacy=0x7feb48073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb477fe640 1 -- 172.21.15.53:0/738492066 >> v1:172.21.15.53:6789/0 conn(0x7feb4815be70 legacy=0x7feb48070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.838+0000 7feb477fe640 1 -- 172.21.15.53:0/738492066 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb480b59e0 con 0x7feb48071360 2024-05-10T10:34:43.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.839+0000 7feb477fe640 1 --2- 172.21.15.53:0/738492066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 0x7feb480b69f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb380097a0 tx=0x7feb3802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af7d9315fc93878f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.839+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38037070 con 0x7feb48071360 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.839+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3802ac50 con 0x7feb48071360 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.839+0000 7feb46ffd640 1 -- 172.21.15.53:0/738492066 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c001670 con 0x7feb48071360 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.840+0000 7feb4e938640 1 -- 172.21.15.53:0/738492066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 msgr2=0x7feb480b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.840+0000 7feb4e938640 1 --2- 172.21.15.53:0/738492066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 0x7feb480b69f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feb380097a0 tx=0x7feb3802a1c0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.840+0000 7feb4e938640 1 -- 172.21.15.53:0/738492066 shutdown_connections 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.840+0000 7feb4e938640 1 --2- 172.21.15.53:0/738492066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 0x7feb480b69f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.840+0000 7feb4e938640 1 -- 172.21.15.53:0/738492066 >> 172.21.15.53:0/738492066 conn(0x7feb480d1d70 msgr2=0x7feb480d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.840+0000 7feb4e938640 1 -- 172.21.15.53:0/738492066 shutdown_connections 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.840+0000 7feb4e938640 1 -- 172.21.15.53:0/738492066 wait complete. 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.841+0000 7feb4e938640 1 Processor -- start 2024-05-10T10:34:43.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.841+0000 7feb4e938640 1 -- start start 2024-05-10T10:34:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4e938640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 0x7feb4815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4e938640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb480aed60 0x7feb4814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4e938640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb4815be70 0x7feb4814ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4e938640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb48070080 con 0x7feb4815be70 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4e938640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb480703f0 con 0x7feb48071360 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4e938640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7feb480b5690 con 0x7feb480aed60 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4ced5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb4815be70 0x7feb4814ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4ced5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb4815be70 0x7feb4814ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50042/0 (socket says 172.21.15.53:50042) 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb4ced5640 1 -- 172.21.15.53:0/4106745782 learned_addr learned my addr 172.21.15.53:0/4106745782 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb47fff640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 0x7feb4815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.842+0000 7feb477fe640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb480aed60 0x7feb4814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4ced5640 1 -- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb480aed60 msgr2=0x7feb4814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4ced5640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb480aed60 0x7feb4814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4ced5640 1 -- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 msgr2=0x7feb4815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4ced5640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 0x7feb4815e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4ced5640 1 -- 172.21.15.53:0/4106745782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4814b070 con 0x7feb4815be70 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4ced5640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb4815be70 0x7feb4814ab30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feb4000c7c0 tx=0x7feb4000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb44ff9640 1 -- 172.21.15.53:0/4106745782 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40003b70 con 0x7feb4815be70 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb44ff9640 1 -- 172.21.15.53:0/4106745782 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb40003d10 con 0x7feb4815be70 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb481f91d0 con 0x7feb4815be70 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb44ff9640 1 -- 172.21.15.53:0/4106745782 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4000aa50 con 0x7feb4815be70 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.843+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4815a360 con 0x7feb4815be70 2024-05-10T10:34:43.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.844+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb480693f0 con 0x7feb4815be70 2024-05-10T10:34:43.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.845+0000 7feb44ff9640 1 -- 172.21.15.53:0/4106745782 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feb4000abf0 con 0x7feb4815be70 2024-05-10T10:34:43.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.845+0000 7feb44ff9640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb340762d0 0x7feb34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:43.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.846+0000 7feb44ff9640 1 -- 172.21.15.53:0/4106745782 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7feb40097600 con 0x7feb4815be70 2024-05-10T10:34:43.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.846+0000 7feb47fff640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb340762d0 0x7feb34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:43.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.846+0000 7feb47fff640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb340762d0 0x7feb34078790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0017e0 tx=0x7feb3c001610 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:43.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:43.850+0000 7feb44ff9640 1 -- 172.21.15.53:0/4106745782 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7feb40060770 con 0x7feb4815be70 2024-05-10T10:34:44.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.041+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb480b62c0 con 0x7feb4815be70 2024-05-10T10:34:44.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.043+0000 7feb44ff9640 1 -- 172.21.15.53:0/4106745782 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16942 (secure 0 0 0) 0x7feb40064a70 con 0x7feb4815be70 2024-05-10T10:34:44.044 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:44.045 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:23.943083+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:20.374693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:21.350324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb340762d0 msgr2=0x7feb34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb340762d0 0x7feb34078790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0017e0 tx=0x7feb3c001610 comp rx=0 tx=0).stop 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb4815be70 msgr2=0x7feb4814ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb4815be70 0x7feb4814ab30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feb4000c7c0 tx=0x7feb4000cc80 comp rx=0 tx=0).stop 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 shutdown_connections 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb340762d0 0x7feb34078790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb4815be70 0x7feb4814ab30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb480aed60 0x7feb4814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 --2- 172.21.15.53:0/4106745782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb48071360 0x7feb4815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 >> 172.21.15.53:0/4106745782 conn(0x7feb480d1d70 msgr2=0x7feb48073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 shutdown_connections 2024-05-10T10:34:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.045+0000 7feb4e938640 1 -- 172.21.15.53:0/4106745782 wait complete. 2024-05-10T10:34:44.058 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:34:44.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:44.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.160+0000 7feb3b9f0640 1 Processor -- start 2024-05-10T10:34:44.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.161+0000 7feb3b9f0640 1 -- start start 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.161+0000 7feb3b9f0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb3415f3b0 0x7feb340d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.161+0000 7feb3b9f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7feb340d5d30 0x7feb340d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.161+0000 7feb3b9f0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb340d30b0 con 0x7feb3415f790 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.161+0000 7feb3b9f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7feb340d3390 con 0x7feb340d5d30 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.161+0000 7feb3b9f0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7feb340c8c80 con 0x7feb3415f3b0 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.161+0000 7feb38f8b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feb3415f790 0x7feb340c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51022/0 (socket says 172.21.15.53:51022) 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb38f8b640 1 -- 172.21.15.53:0/2551865561 learned_addr learned my addr 172.21.15.53:0/2551865561 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb39f8d640 1 --2- 172.21.15.53:0/2551865561 >> v2:172.21.15.135:3300/0 conn(0x7feb340d5d30 0x7feb340d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb3978c640 1 --2- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb3415f3b0 0x7feb340d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb33fff640 1 -- 172.21.15.53:0/2551865561 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3765363264 0 0) 0x7feb340d30b0 con 0x7feb3415f790 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb33fff640 1 -- 172.21.15.53:0/2551865561 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb1c003520 con 0x7feb3415f790 2024-05-10T10:34:44.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb39f8d640 1 -- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb3415f3b0 msgr2=0x7feb340d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:44.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb39f8d640 1 --2- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb3415f3b0 0x7feb340d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb39f8d640 1 -- 172.21.15.53:0/2551865561 >> v1:172.21.15.53:6789/0 conn(0x7feb3415f790 legacy=0x7feb340c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.162+0000 7feb39f8d640 1 -- 172.21.15.53:0/2551865561 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb340c8f60 con 0x7feb340d5d30 2024-05-10T10:34:44.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.163+0000 7feb39f8d640 1 --2- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb340d5d30 0x7feb340d3c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb24009d90 tx=0x7feb2402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf029e730dd9b902 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.163+0000 7feb33fff640 1 -- 172.21.15.53:0/2551865561 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24007740 con 0x7feb340d5d30 2024-05-10T10:34:44.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.163+0000 7feb33fff640 1 -- 172.21.15.53:0/2551865561 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb240078e0 con 0x7feb340d5d30 2024-05-10T10:34:44.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.163+0000 7feb33fff640 1 -- 172.21.15.53:0/2551865561 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24007cb0 con 0x7feb340d5d30 2024-05-10T10:34:44.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb340d5d30 msgr2=0x7feb340d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb340d5d30 0x7feb340d3c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb24009d90 tx=0x7feb2402a660 comp rx=0 tx=0).stop 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2551865561 shutdown_connections 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb340d5d30 0x7feb340d3c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2551865561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb3415f3b0 0x7feb340d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2551865561 >> 172.21.15.53:0/2551865561 conn(0x7feb340d5130 msgr2=0x7feb340d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2551865561 shutdown_connections 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.164+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2551865561 wait complete. 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.165+0000 7feb3b9f0640 1 Processor -- start 2024-05-10T10:34:44.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.165+0000 7feb3b9f0640 1 -- start start 2024-05-10T10:34:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3b9f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340d5d30 0x7feb341f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3b9f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb3415f3b0 0x7feb340a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3b9f0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb341f6010 0x7feb340a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3b9f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb340d0e60 con 0x7feb340d5d30 2024-05-10T10:34:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3b9f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb340d0ce0 con 0x7feb3415f3b0 2024-05-10T10:34:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3b9f0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7feb340d0fe0 con 0x7feb341f6010 2024-05-10T10:34:44.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340d5d30 0x7feb341f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340d5d30 0x7feb341f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50058/0 (socket says 172.21.15.53:50058) 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 -- 172.21.15.53:0/2114459596 learned_addr learned my addr 172.21.15.53:0/2114459596 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb39f8d640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb341f6010 0x7feb340a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb38f8b640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb3415f3b0 0x7feb340a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 -- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb341f6010 msgr2=0x7feb340a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb341f6010 0x7feb340a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 -- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb3415f3b0 msgr2=0x7feb340a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb3415f3b0 0x7feb340a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 -- 172.21.15.53:0/2114459596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3409f310 con 0x7feb340d5d30 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3978c640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340d5d30 0x7feb341f5ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00e9b0 tx=0x7feb2c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb31ffb640 1 -- 172.21.15.53:0/2114459596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c017070 con 0x7feb340d5d30 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb31ffb640 1 -- 172.21.15.53:0/2114459596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2c00c9a0 con 0x7feb340d5d30 2024-05-10T10:34:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb31ffb640 1 -- 172.21.15.53:0/2114459596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00cd90 con 0x7feb340d5d30 2024-05-10T10:34:44.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.166+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3409f580 con 0x7feb340d5d30 2024-05-10T10:34:44.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.167+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3409fb40 con 0x7feb340d5d30 2024-05-10T10:34:44.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.168+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3406a220 con 0x7feb340d5d30 2024-05-10T10:34:44.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.168+0000 7feb31ffb640 1 -- 172.21.15.53:0/2114459596 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feb2c01a070 con 0x7feb340d5d30 2024-05-10T10:34:44.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.169+0000 7feb31ffb640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb20076300 0x7feb200787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.169+0000 7feb31ffb640 1 -- 172.21.15.53:0/2114459596 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7feb2c097b30 con 0x7feb340d5d30 2024-05-10T10:34:44.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.169+0000 7feb38f8b640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb20076300 0x7feb200787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.169+0000 7feb38f8b640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb20076300 0x7feb200787c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feb3415e3e0 tx=0x7feb28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.173+0000 7feb31ffb640 1 -- 172.21.15.53:0/2114459596 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7feb2c061300 con 0x7feb340d5d30 2024-05-10T10:34:44.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.408+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb340c9a50 con 0x7feb340d5d30 2024-05-10T10:34:44.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.409+0000 7feb31ffb640 1 -- 172.21.15.53:0/2114459596 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7feb2c060c00 con 0x7feb340d5d30 2024-05-10T10:34:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:44.411 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:44.411 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb20076300 msgr2=0x7feb200787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb20076300 0x7feb200787c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feb3415e3e0 tx=0x7feb28009040 comp rx=0 tx=0).stop 2024-05-10T10:34:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340d5d30 msgr2=0x7feb341f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340d5d30 0x7feb341f5ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00e9b0 tx=0x7feb2c00ee70 comp rx=0 tx=0).stop 2024-05-10T10:34:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 shutdown_connections 2024-05-10T10:34:44.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feb20076300 0x7feb200787c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb341f6010 0x7feb340a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb3415f3b0 0x7feb340a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 --2- 172.21.15.53:0/2114459596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb340d5d30 0x7feb341f5ad0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 >> 172.21.15.53:0/2114459596 conn(0x7feb340d5130 msgr2=0x7feb340d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:44.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 shutdown_connections 2024-05-10T10:34:44.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.411+0000 7feb3b9f0640 1 -- 172.21.15.53:0/2114459596 wait complete. 2024-05-10T10:34:44.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:44.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.526+0000 7ffb1d70a640 1 Processor -- start 2024-05-10T10:34:44.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.526+0000 7ffb1d70a640 1 -- start start 2024-05-10T10:34:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb1d70a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb18146750 0x7ffb18070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb1d70a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffb180af930 0x7ffb180afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb1d70a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb18146bb0 con 0x7ffb1806c2c0 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb1d70a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ffb180af4a0 con 0x7ffb180af930 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb1d70a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ffb180719d0 con 0x7ffb18146750 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb177fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffb180af930 0x7ffb180afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb177fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffb180af930 0x7ffb180afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50060/0 (socket says 172.21.15.53:50060) 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.527+0000 7ffb177fe640 1 -- 172.21.15.53:0/622531643 learned_addr learned my addr 172.21.15.53:0/622531643 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.528+0000 7ffb167fc640 1 --2- 172.21.15.53:0/622531643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb18146750 0x7ffb18070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.528+0000 7ffb177fe640 1 -- 172.21.15.53:0/622531643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb18146750 msgr2=0x7ffb18070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.528+0000 7ffb177fe640 1 --2- 172.21.15.53:0/622531643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb18146750 0x7ffb18070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.528+0000 7ffb177fe640 1 -- 172.21.15.53:0/622531643 >> v1:172.21.15.135:6789/0 conn(0x7ffb1806c2c0 legacy=0x7ffb1806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.528+0000 7ffb177fe640 1 -- 172.21.15.53:0/622531643 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb18071150 con 0x7ffb180af930 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.528+0000 7ffb177fe640 1 --2- 172.21.15.53:0/622531643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb180af930 0x7ffb180afdc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009e50 tx=0x7ffb0802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6354eb832b11cb00 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb15ffb640 1 -- 172.21.15.53:0/622531643 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08036070 con 0x7ffb180af930 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb15ffb640 1 -- 172.21.15.53:0/622531643 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb08031600 con 0x7ffb180af930 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb15ffb640 1 -- 172.21.15.53:0/622531643 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080319d0 con 0x7ffb180af930 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 -- 172.21.15.53:0/622531643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb180af930 msgr2=0x7ffb180afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/622531643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb180af930 0x7ffb180afdc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009e50 tx=0x7ffb0802a710 comp rx=0 tx=0).stop 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 -- 172.21.15.53:0/622531643 shutdown_connections 2024-05-10T10:34:44.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/622531643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb180af930 0x7ffb180afdc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/622531643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb18146750 0x7ffb18070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 -- 172.21.15.53:0/622531643 >> 172.21.15.53:0/622531643 conn(0x7ffb180d1d70 msgr2=0x7ffb180d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:44.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 -- 172.21.15.53:0/622531643 shutdown_connections 2024-05-10T10:34:44.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.529+0000 7ffb1d70a640 1 -- 172.21.15.53:0/622531643 wait complete. 2024-05-10T10:34:44.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 Processor -- start 2024-05-10T10:34:44.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 -- start start 2024-05-10T10:34:44.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb1806c2c0 0x7ffb181e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb180af930 0x7ffb1815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb18146750 0x7ffb181e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb180af130 con 0x7ffb18146750 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb180716f0 con 0x7ffb1806c2c0 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.531+0000 7ffb1d70a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ffb180b1530 con 0x7ffb180af930 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb18146750 0x7ffb181e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb18146750 0x7ffb181e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50070/0 (socket says 172.21.15.53:50070) 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 -- 172.21.15.53:0/326388469 learned_addr learned my addr 172.21.15.53:0/326388469 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb16ffd640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb1806c2c0 0x7ffb181e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb167fc640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb180af930 0x7ffb1815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 -- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb180af930 msgr2=0x7ffb1815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb180af930 0x7ffb1815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 -- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb1806c2c0 msgr2=0x7ffb181e3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb1806c2c0 0x7ffb181e3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 -- 172.21.15.53:0/326388469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb181e4480 con 0x7ffb18146750 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb177fe640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb18146750 0x7ffb181e35f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009e50 tx=0x7ffb08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffaff7fe640 1 -- 172.21.15.53:0/326388469 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08041020 con 0x7ffb18146750 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffaff7fe640 1 -- 172.21.15.53:0/326388469 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb08007780 con 0x7ffb18146750 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb181e4760 con 0x7ffb18146750 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffaff7fe640 1 -- 172.21.15.53:0/326388469 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08036040 con 0x7ffb18146750 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.532+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb181e4d20 con 0x7ffb18146750 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.533+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb180693f0 con 0x7ffb18146750 2024-05-10T10:34:44.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.534+0000 7ffaff7fe640 1 -- 172.21.15.53:0/326388469 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb08044020 con 0x7ffb18146750 2024-05-10T10:34:44.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.535+0000 7ffaff7fe640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ffb040762d0 0x7ffb04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.535+0000 7ffaff7fe640 1 -- 172.21.15.53:0/326388469 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7ffb080b8560 con 0x7ffb18146750 2024-05-10T10:34:44.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.535+0000 7ffb16ffd640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ffb040762d0 0x7ffb04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.535+0000 7ffb16ffd640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ffb040762d0 0x7ffb04078790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb100015a0 tx=0x7ffb10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.539+0000 7ffaff7fe640 1 -- 172.21.15.53:0/326388469 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ffb0804b030 con 0x7ffb18146750 2024-05-10T10:34:44.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.772+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb18075290 con 0x7ffb18146750 2024-05-10T10:34:44.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.772+0000 7ffaff7fe640 1 -- 172.21.15.53:0/326388469 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5178 (secure 0 0 0) 0x7ffb08081cc0 con 0x7ffb18146750 2024-05-10T10:34:44.774 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:44.774 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:22.306333+0000","modified":"2024-05-10T10:34:25.330760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4878},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4878":{"gid":4878,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/561700189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":561700189},{"type":"v1","addr":"172.21.15.53:6835","nonce":561700189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4878,"qdb_cluster":[4878]},"id":2}]} 2024-05-10T10:34:44.774 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ffb040762d0 msgr2=0x7ffb04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ffb040762d0 0x7ffb04078790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb100015a0 tx=0x7ffb10009040 comp rx=0 tx=0).stop 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb18146750 msgr2=0x7ffb181e35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb18146750 0x7ffb181e35f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009e50 tx=0x7ffb08004a40 comp rx=0 tx=0).stop 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 shutdown_connections 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ffb040762d0 0x7ffb04078790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb18146750 0x7ffb181e35f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb180af930 0x7ffb1815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 --2- 172.21.15.53:0/326388469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb1806c2c0 0x7ffb181e3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 >> 172.21.15.53:0/326388469 conn(0x7ffb180d1d70 msgr2=0x7ffb180d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 shutdown_connections 2024-05-10T10:34:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.774+0000 7ffb1d70a640 1 -- 172.21.15.53:0/326388469 wait complete. 2024-05-10T10:34:44.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:44.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.889+0000 7f9d985dd640 1 Processor -- start 2024-05-10T10:34:44.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.889+0000 7f9d985dd640 1 -- start start 2024-05-10T10:34:44.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d985dd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d900d0230 0x7f9d900cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d985dd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d9015fc90 con 0x7f9d900cfbb0 2024-05-10T10:34:44.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d985dd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d900b8c40 con 0x7f9d900b8860 2024-05-10T10:34:44.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d985dd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9006ec30 con 0x7f9d900d0230 2024-05-10T10:34:44.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d95b78640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d900d0230 0x7f9d900cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d96b7a640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9d900cfbb0 0x7f9d900ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51034/0 (socket says 172.21.15.53:51034) 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d96b7a640 1 -- 172.21.15.53:0/1145653089 learned_addr learned my addr 172.21.15.53:0/1145653089 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d95b78640 1 -- 172.21.15.53:0/1145653089 >> v1:172.21.15.135:6789/0 conn(0x7f9d900b8860 legacy=0x7f9d9006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d95b78640 1 -- 172.21.15.53:0/1145653089 >> v1:172.21.15.53:6789/0 conn(0x7f9d900cfbb0 legacy=0x7f9d900ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d95377640 1 -- 172.21.15.53:0/1145653089 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4174739182 0 0) 0x7f9d9015fc90 con 0x7f9d900cfbb0 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.890+0000 7f9d95b78640 1 -- 172.21.15.53:0/1145653089 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d900cd570 con 0x7f9d900d0230 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.891+0000 7f9d95b78640 1 --2- 172.21.15.53:0/1145653089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d900cf670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d840097a0 tx=0x7f9d8402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d5bacc9a6107cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.891+0000 7f9d95377640 1 -- 172.21.15.53:0/1145653089 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84037070 con 0x7f9d900d0230 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.891+0000 7f9d95377640 1 -- 172.21.15.53:0/1145653089 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8402ae50 con 0x7f9d900d0230 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.891+0000 7f9d95377640 1 -- 172.21.15.53:0/1145653089 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84032660 con 0x7f9d900d0230 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.891+0000 7f9d985dd640 1 -- 172.21.15.53:0/1145653089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 msgr2=0x7f9d900cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.891+0000 7f9d985dd640 1 --2- 172.21.15.53:0/1145653089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d900cf670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9d840097a0 tx=0x7f9d8402a140 comp rx=0 tx=0).stop 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.892+0000 7f9d985dd640 1 -- 172.21.15.53:0/1145653089 shutdown_connections 2024-05-10T10:34:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.892+0000 7f9d985dd640 1 --2- 172.21.15.53:0/1145653089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d900cf670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.892+0000 7f9d985dd640 1 -- 172.21.15.53:0/1145653089 >> 172.21.15.53:0/1145653089 conn(0x7f9d900d2df0 msgr2=0x7f9d900d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.892+0000 7f9d985dd640 1 -- 172.21.15.53:0/1145653089 shutdown_connections 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.892+0000 7f9d985dd640 1 -- 172.21.15.53:0/1145653089 wait complete. 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.893+0000 7f9d985dd640 1 Processor -- start 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.893+0000 7f9d985dd640 1 -- start start 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.893+0000 7f9d985dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d900b8860 0x7f9d90162d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d985dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d900cfbb0 0x7f9d900c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d985dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d90154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d985dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d900cdb30 con 0x7f9d900d0230 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d985dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9006ee50 con 0x7f9d900b8860 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d985dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9d900cde10 con 0x7f9d900cfbb0 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d90154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d90154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50090/0 (socket says 172.21.15.53:50090) 2024-05-10T10:34:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 -- 172.21.15.53:0/3515714587 learned_addr learned my addr 172.21.15.53:0/3515714587 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96379640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d900b8860 0x7f9d90162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d95b78640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d900cfbb0 0x7f9d900c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 -- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d900cfbb0 msgr2=0x7f9d900c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d900cfbb0 0x7f9d900c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 -- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d900b8860 msgr2=0x7f9d90162d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d900b8860 0x7f9d90162d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 -- 172.21.15.53:0/3515714587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d900ca400 con 0x7f9d900d0230 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d95b78640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d900cfbb0 0x7f9d900c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d96b7a640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d90154170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c7a0 tx=0x7f9d8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d82ffd640 1 -- 172.21.15.53:0/3515714587 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88018070 con 0x7f9d900d0230 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d900ca6a0 con 0x7f9d900d0230 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d82ffd640 1 -- 172.21.15.53:0/3515714587 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8800a640 con 0x7f9d900d0230 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d82ffd640 1 -- 172.21.15.53:0/3515714587 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800aab0 con 0x7f9d900d0230 2024-05-10T10:34:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.894+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d900caf40 con 0x7f9d900d0230 2024-05-10T10:34:44.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.895+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9006a220 con 0x7f9d900d0230 2024-05-10T10:34:44.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.896+0000 7f9d82ffd640 1 -- 172.21.15.53:0/3515714587 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9d8801b070 con 0x7f9d900d0230 2024-05-10T10:34:44.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.897+0000 7f9d82ffd640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9d740762d0 0x7f9d74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:44.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.897+0000 7f9d82ffd640 1 -- 172.21.15.53:0/3515714587 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f9d88096d50 con 0x7f9d900d0230 2024-05-10T10:34:44.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.897+0000 7f9d96379640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9d740762d0 0x7f9d74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:44.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.898+0000 7f9d96379640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9d740762d0 0x7f9d74078790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c004ea0 tx=0x7f9d8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:44.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:44.901+0000 7f9d82ffd640 1 -- 172.21.15.53:0/3515714587 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9d8809a030 con 0x7f9d900d0230 2024-05-10T10:34:45.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.095+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d900d1e20 con 0x7f9d900d0230 2024-05-10T10:34:45.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.096+0000 7f9d82ffd640 1 -- 172.21.15.53:0/3515714587 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16942 (secure 0 0 0) 0x7f9d88060520 con 0x7f9d900d0230 2024-05-10T10:34:45.098 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:45.098 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:23.943083+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:20.374693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:21.350324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:45.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9d740762d0 msgr2=0x7f9d74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9d740762d0 0x7f9d74078790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c004ea0 tx=0x7f9d8c009040 comp rx=0 tx=0).stop 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 msgr2=0x7f9d90154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d90154170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c7a0 tx=0x7f9d8800cc60 comp rx=0 tx=0).stop 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 shutdown_connections 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9d740762d0 0x7f9d74078790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d900d0230 0x7f9d90154170 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d900cfbb0 0x7f9d900c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 --2- 172.21.15.53:0/3515714587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d900b8860 0x7f9d90162d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 >> 172.21.15.53:0/3515714587 conn(0x7f9d900d2df0 msgr2=0x7f9d900d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 shutdown_connections 2024-05-10T10:34:45.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.099+0000 7f9d985dd640 1 -- 172.21.15.53:0/3515714587 wait complete. 2024-05-10T10:34:45.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:34:45.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.215+0000 7f9552845640 1 Processor -- start 2024-05-10T10:34:45.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.215+0000 7f9552845640 1 -- start start 2024-05-10T10:34:45.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.215+0000 7f9552845640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f9552845640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f954c072330 con 0x7f954c071880 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f9552845640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f954c071d70 con 0x7f954c0d3510 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f9552845640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f954c072050 con 0x7f954c15d290 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f9543fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f954c071880 0x7f954c0d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51046/0 (socket says 172.21.15.53:51046) 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f9543fff640 1 -- 172.21.15.53:0/4059190126 learned_addr learned my addr 172.21.15.53:0/4059190126 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954bfff640 1 --2- 172.21.15.53:0/4059190126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954b7fe640 1 -- 172.21.15.53:0/4059190126 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851656081 0 0) 0x7f954c072330 con 0x7f954c071880 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954b7fe640 1 -- 172.21.15.53:0/4059190126 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9538003520 con 0x7f954c071880 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954b7fe640 1 -- 172.21.15.53:0/4059190126 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592129714 0 0) 0x7f954c071d70 con 0x7f954c0d3510 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954b7fe640 1 -- 172.21.15.53:0/4059190126 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f954c072330 con 0x7f954c0d3510 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954bfff640 1 -- 172.21.15.53:0/4059190126 >> v1:172.21.15.135:6789/0 conn(0x7f954c0d3510 legacy=0x7f954c0ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954bfff640 1 -- 172.21.15.53:0/4059190126 >> v1:172.21.15.53:6789/0 conn(0x7f954c071880 legacy=0x7f954c0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.216+0000 7f954bfff640 1 -- 172.21.15.53:0/4059190126 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c0cda70 con 0x7f954c15d290 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.217+0000 7f954bfff640 1 --2- 172.21.15.53:0/4059190126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c071340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f953c0097d0 tx=0x7f953c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac2aae1b9ba2a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.217+0000 7f954b7fe640 1 -- 172.21.15.53:0/4059190126 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c037070 con 0x7f954c15d290 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.217+0000 7f954b7fe640 1 -- 172.21.15.53:0/4059190126 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c032440 con 0x7f954c15d290 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.217+0000 7f954b7fe640 1 -- 172.21.15.53:0/4059190126 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c0327f0 con 0x7f954c15d290 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.218+0000 7f9552845640 1 -- 172.21.15.53:0/4059190126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 msgr2=0x7f954c071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.218+0000 7f9552845640 1 --2- 172.21.15.53:0/4059190126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c071340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f953c0097d0 tx=0x7f953c02a210 comp rx=0 tx=0).stop 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.218+0000 7f9552845640 1 -- 172.21.15.53:0/4059190126 shutdown_connections 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.218+0000 7f9552845640 1 --2- 172.21.15.53:0/4059190126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c071340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.218+0000 7f9552845640 1 -- 172.21.15.53:0/4059190126 >> 172.21.15.53:0/4059190126 conn(0x7f954c09ba20 msgr2=0x7f954c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.218+0000 7f9552845640 1 -- 172.21.15.53:0/4059190126 shutdown_connections 2024-05-10T10:34:45.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.218+0000 7f9552845640 1 -- 172.21.15.53:0/4059190126 wait complete. 2024-05-10T10:34:45.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.219+0000 7f9552845640 1 Processor -- start 2024-05-10T10:34:45.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.219+0000 7f9552845640 1 -- start start 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9552845640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954c071880 0x7f954c15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9552845640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f954c0d3510 0x7f954c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9552845640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c0b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9552845640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0d1c20 con 0x7f954c071880 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9552845640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0d1aa0 con 0x7f954c0d3510 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9552845640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f954c15e7b0 con 0x7f954c15d290 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954c071880 0x7f954c15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954c071880 0x7f954c15b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50098/0 (socket says 172.21.15.53:50098) 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 -- 172.21.15.53:0/622747219 learned_addr learned my addr 172.21.15.53:0/622747219 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9543fff640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f954c0d3510 0x7f954c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9550de2640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c0b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 -- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 msgr2=0x7f954c0b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c0b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 -- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f954c0d3510 msgr2=0x7f954c0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f954c0d3510 0x7f954c0b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f954bfff640 1 -- 172.21.15.53:0/622747219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c0b86d0 con 0x7f954c071880 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.220+0000 7f9543fff640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f954c0d3510 0x7f954c0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.221+0000 7f954bfff640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954c071880 0x7f954c15b1d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f953c000c00 tx=0x7f953c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.221+0000 7f95497fa640 1 -- 172.21.15.53:0/622747219 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c041020 con 0x7f954c071880 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.221+0000 7f95497fa640 1 -- 172.21.15.53:0/622747219 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c03d620 con 0x7f954c071880 2024-05-10T10:34:45.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.221+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c0b89b0 con 0x7f954c071880 2024-05-10T10:34:45.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.221+0000 7f95497fa640 1 -- 172.21.15.53:0/622747219 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c037040 con 0x7f954c071880 2024-05-10T10:34:45.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.221+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c15e440 con 0x7f954c071880 2024-05-10T10:34:45.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.222+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954c0d2ab0 con 0x7f954c071880 2024-05-10T10:34:45.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.222+0000 7f95497fa640 1 -- 172.21.15.53:0/622747219 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f953c044020 con 0x7f954c071880 2024-05-10T10:34:45.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.223+0000 7f95497fa640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f952c0762d0 0x7f952c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.223+0000 7f95497fa640 1 -- 172.21.15.53:0/622747219 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f953c0b7f20 con 0x7f954c071880 2024-05-10T10:34:45.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.223+0000 7f9543fff640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f952c0762d0 0x7f952c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.224+0000 7f9543fff640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f952c0762d0 0x7f952c078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9534007fd0 tx=0x7f9534009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.227+0000 7f95497fa640 1 -- 172.21.15.53:0/622747219 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f954c0d2ab0 con 0x7f954c071880 2024-05-10T10:34:45.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.459+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f954c0693f0 con 0x7f954c071880 2024-05-10T10:34:45.461 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:34:45.459+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:34:45.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.461+0000 7f95497fa640 1 -- 172.21.15.53:0/622747219 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f953c04b030 con 0x7f954c071880 2024-05-10T10:34:45.462 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f952c0762d0 msgr2=0x7f952c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f952c0762d0 0x7f952c078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9534007fd0 tx=0x7f9534009040 comp rx=0 tx=0).stop 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954c071880 msgr2=0x7f954c15b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954c071880 0x7f954c15b1d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f953c000c00 tx=0x7f953c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 shutdown_connections 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f952c0762d0 0x7f952c078790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f954c15d290 0x7f954c0b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f954c0d3510 0x7f954c0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 --2- 172.21.15.53:0/622747219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954c071880 0x7f954c15b1d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 >> 172.21.15.53:0/622747219 conn(0x7f954c09ba20 msgr2=0x7f954c0b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:45.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 shutdown_connections 2024-05-10T10:34:45.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.464+0000 7f9552845640 1 -- 172.21.15.53:0/622747219 wait complete. 2024-05-10T10:34:45.468 INFO:tasks.ceph.mds.c.smithi053.stderr: -6454> 2024-05-10T10:34:27.912+0000 7fca746bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:34:45.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:34:45.501 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:34:45.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.583+0000 7f274d55f640 1 Processor -- start 2024-05-10T10:34:45.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.583+0000 7f274d55f640 1 -- start start 2024-05-10T10:34:45.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f274d55f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2748160530 0x7f274815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f274d55f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f274815c9e0 0x7f27480cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f274d55f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274815cfb0 con 0x7f274806f030 2024-05-10T10:34:45.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f274d55f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f27480ccb60 con 0x7f2748160530 2024-05-10T10:34:45.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f274d55f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2748160990 con 0x7f274815c9e0 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f273e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2748160530 0x7f274815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f2746ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f274806f030 0x7f274806f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51064/0 (socket says 172.21.15.53:51064) 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f2746ffd640 1 -- 172.21.15.53:0/2320497139 learned_addr learned my addr 172.21.15.53:0/2320497139 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f273e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2748160530 0x7f274815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50120/0 (socket says 172.21.15.53:50120) 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f27477fe640 1 --2- 172.21.15.53:0/2320497139 >> v2:172.21.15.135:3300/0 conn(0x7f274815c9e0 0x7f27480cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f273e7fc640 1 -- 172.21.15.53:0/2320497139 >> v2:172.21.15.135:3300/0 conn(0x7f274815c9e0 msgr2=0x7f27480cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f273e7fc640 1 --2- 172.21.15.53:0/2320497139 >> v2:172.21.15.135:3300/0 conn(0x7f274815c9e0 0x7f27480cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f273e7fc640 1 -- 172.21.15.53:0/2320497139 >> v1:172.21.15.53:6789/0 conn(0x7f274806f030 legacy=0x7f274806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f273e7fc640 1 -- 172.21.15.53:0/2320497139 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274815d290 con 0x7f2748160530 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.584+0000 7f27467fc640 1 -- 172.21.15.53:0/2320497139 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2396461985 0 0) 0x7f274815cfb0 con 0x7f274806f030 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.585+0000 7f273e7fc640 1 --2- 172.21.15.53:0/2320497139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815c4a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f27480d07e0 tx=0x7f273002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd26b2477a6df4f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.585+0000 7f27467fc640 1 -- 172.21.15.53:0/2320497139 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730037070 con 0x7f2748160530 2024-05-10T10:34:45.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.585+0000 7f27467fc640 1 -- 172.21.15.53:0/2320497139 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2730032440 con 0x7f2748160530 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.585+0000 7f27467fc640 1 -- 172.21.15.53:0/2320497139 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730032810 con 0x7f2748160530 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.585+0000 7f274d55f640 1 -- 172.21.15.53:0/2320497139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 msgr2=0x7f274815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.585+0000 7f274d55f640 1 --2- 172.21.15.53:0/2320497139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815c4a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f27480d07e0 tx=0x7f273002a1a0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.586+0000 7f274d55f640 1 -- 172.21.15.53:0/2320497139 shutdown_connections 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.586+0000 7f274d55f640 1 --2- 172.21.15.53:0/2320497139 >> v2:172.21.15.135:3300/0 conn(0x7f274815c9e0 0x7f27480cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.586+0000 7f274d55f640 1 --2- 172.21.15.53:0/2320497139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815c4a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.586+0000 7f274d55f640 1 -- 172.21.15.53:0/2320497139 >> 172.21.15.53:0/2320497139 conn(0x7f27480d0c90 msgr2=0x7f27480d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.586+0000 7f274d55f640 1 -- 172.21.15.53:0/2320497139 shutdown_connections 2024-05-10T10:34:45.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.586+0000 7f274d55f640 1 -- 172.21.15.53:0/2320497139 wait complete. 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.587+0000 7f274d55f640 1 Processor -- start 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.587+0000 7f274d55f640 1 -- start start 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f274d55f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f274806f030 0x7f2748152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f274d55f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f274815c9e0 0x7f274815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f274d55f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f274d55f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f274815fa40 con 0x7f2748160530 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f274d55f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2748160010 con 0x7f274806f030 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f274d55f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f27480cb910 con 0x7f274815c9e0 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50126/0 (socket says 172.21.15.53:50126) 2024-05-10T10:34:45.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 -- 172.21.15.53:0/378738559 learned_addr learned my addr 172.21.15.53:0/378738559 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f2746ffd640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f274806f030 0x7f2748152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f273e7fc640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f274815c9e0 0x7f274815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 -- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f274815c9e0 msgr2=0x7f274815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f274815c9e0 0x7f274815d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 -- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f274806f030 msgr2=0x7f2748152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f274806f030 0x7f2748152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 -- 172.21.15.53:0/378738559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27480b0820 con 0x7f2748160530 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f2746ffd640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f274806f030 0x7f2748152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f27477fe640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815d9f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2740007d90 tx=0x7f274000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.588+0000 7f273ffff640 1 -- 172.21.15.53:0/378738559 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740014420 con 0x7f2748160530 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.589+0000 7f273ffff640 1 -- 172.21.15.53:0/378738559 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2740019070 con 0x7f2748160530 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.589+0000 7f273ffff640 1 -- 172.21.15.53:0/378738559 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740014820 con 0x7f2748160530 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.589+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27480b0b00 con 0x7f2748160530 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.589+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27480b10c0 con 0x7f2748160530 2024-05-10T10:34:45.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.590+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274806a220 con 0x7f2748160530 2024-05-10T10:34:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.590+0000 7f273ffff640 1 -- 172.21.15.53:0/378738559 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f27400149c0 con 0x7f2748160530 2024-05-10T10:34:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.591+0000 7f273ffff640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2728076300 0x7f27280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.591+0000 7f273ffff640 1 -- 172.21.15.53:0/378738559 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6187+0+0 (secure 0 0 0) 0x7f2740098010 con 0x7f2748160530 2024-05-10T10:34:45.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.591+0000 7f2746ffd640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2728076300 0x7f27280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:45.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.592+0000 7f2746ffd640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2728076300 0x7f27280787c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2738000f60 tx=0x7f2738009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:45.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.595+0000 7f273ffff640 1 -- 172.21.15.53:0/378738559 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2740061130 con 0x7f2748160530 2024-05-10T10:34:45.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:45.827+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f27480b6c10 con 0x7f2748160530 2024-05-10T10:34:46.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.465+0000 7f273ffff640 1 -- 172.21.15.53:0/378738559 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f2740065430 con 0x7f2748160530 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2728076300 msgr2=0x7f27280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2728076300 0x7f27280787c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2738000f60 tx=0x7f2738009040 comp rx=0 tx=0).stop 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 msgr2=0x7f274815d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815d9f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2740007d90 tx=0x7f274000e4b0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 shutdown_connections 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2728076300 0x7f27280787c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2748160530 0x7f274815d9f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f274815c9e0 0x7f274815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 --2- 172.21.15.53:0/378738559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f274806f030 0x7f2748152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 >> 172.21.15.53:0/378738559 conn(0x7f27480d0c90 msgr2=0x7f274806f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:46.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 shutdown_connections 2024-05-10T10:34:46.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.468+0000 7f274d55f640 1 -- 172.21.15.53:0/378738559 wait complete. 2024-05-10T10:34:46.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:34:46.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.638+0000 7fb318524640 1 Processor -- start 2024-05-10T10:34:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.638+0000 7fb318524640 1 -- start start 2024-05-10T10:34:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb318524640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb318524640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3100716f0 con 0x7fb3101639a0 2024-05-10T10:34:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb318524640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3100d2510 con 0x7fb31015f370 2024-05-10T10:34:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb318524640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb310070220 con 0x7fb310162e00 2024-05-10T10:34:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb315abf640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb3101639a0 0x7fb310070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51066/0 (socket says 172.21.15.53:51066) 2024-05-10T10:34:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb315abf640 1 -- 172.21.15.53:0/2323175293 learned_addr learned my addr 172.21.15.53:0/2323175293 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb316ac1640 1 --2- 172.21.15.53:0/2323175293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2187987498 0 0) 0x7fb3100716f0 con 0x7fb3101639a0 2024-05-10T10:34:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2fc003520 con 0x7fb3101639a0 2024-05-10T10:34:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 403946972 0 0) 0x7fb3100d2510 con 0x7fb31015f370 2024-05-10T10:34:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3100716f0 con 0x7fb31015f370 2024-05-10T10:34:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3420036090 0 0) 0x7fb2fc003520 con 0x7fb3101639a0 2024-05-10T10:34:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb316ac1640 1 -- 172.21.15.53:0/2323175293 >> v1:172.21.15.135:6789/0 conn(0x7fb31015f370 legacy=0x7fb3100d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb316ac1640 1 -- 172.21.15.53:0/2323175293 >> v1:172.21.15.53:6789/0 conn(0x7fb3101639a0 legacy=0x7fb310070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.639+0000 7fb316ac1640 1 -- 172.21.15.53:0/2323175293 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb310070500 con 0x7fb310162e00 2024-05-10T10:34:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.640+0000 7fb316ac1640 1 --2- 172.21.15.53:0/2323175293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310163290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb3080097d0 tx=0x7fb30802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a530be0bb404f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.640+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308037070 con 0x7fb310162e00 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.640+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30802acc0 con 0x7fb310162e00 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.640+0000 7fb3152be640 1 -- 172.21.15.53:0/2323175293 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300001670 con 0x7fb310162e00 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.641+0000 7fb318524640 1 -- 172.21.15.53:0/2323175293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 msgr2=0x7fb310163290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.641+0000 7fb318524640 1 --2- 172.21.15.53:0/2323175293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310163290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb3080097d0 tx=0x7fb30802a230 comp rx=0 tx=0).stop 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.641+0000 7fb318524640 1 -- 172.21.15.53:0/2323175293 shutdown_connections 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.641+0000 7fb318524640 1 --2- 172.21.15.53:0/2323175293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310163290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.641+0000 7fb318524640 1 -- 172.21.15.53:0/2323175293 >> 172.21.15.53:0/2323175293 conn(0x7fb3100d2e10 msgr2=0x7fb3100d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.641+0000 7fb318524640 1 -- 172.21.15.53:0/2323175293 shutdown_connections 2024-05-10T10:34:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.641+0000 7fb318524640 1 -- 172.21.15.53:0/2323175293 wait complete. 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.642+0000 7fb318524640 1 Processor -- start 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.642+0000 7fb318524640 1 -- start start 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb318524640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015f370 0x7fb310159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb318524640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb318524640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3101639a0 0x7fb310158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb318524640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3100cd460 con 0x7fb31015f370 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb318524640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb310070ac0 con 0x7fb3101639a0 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb318524640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb3100cd5e0 con 0x7fb310162e00 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015f370 0x7fb310159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015f370 0x7fb310159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50136/0 (socket says 172.21.15.53:50136) 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 -- 172.21.15.53:0/3111115169 learned_addr learned my addr 172.21.15.53:0/3111115169 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb315abf640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb316ac1640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3101639a0 0x7fb310158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 -- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 msgr2=0x7fb310154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 -- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3101639a0 msgr2=0x7fb310158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3101639a0 0x7fb310158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb3162c0640 1 -- 172.21.15.53:0/3111115169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb310158e90 con 0x7fb31015f370 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.643+0000 7fb316ac1640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3101639a0 0x7fb310158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.644+0000 7fb3162c0640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015f370 0x7fb310159f30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c830 tx=0x7fb30c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.644+0000 7fb306ffd640 1 -- 172.21.15.53:0/3111115169 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c003b70 con 0x7fb31015f370 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.644+0000 7fb306ffd640 1 -- 172.21.15.53:0/3111115169 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c003d10 con 0x7fb31015f370 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.644+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb310159170 con 0x7fb31015f370 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.644+0000 7fb306ffd640 1 -- 172.21.15.53:0/3111115169 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00aa90 con 0x7fb31015f370 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.644+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb310159a10 con 0x7fb31015f370 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.645+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb310159340 con 0x7fb31015f370 2024-05-10T10:34:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.646+0000 7fb306ffd640 1 -- 172.21.15.53:0/3111115169 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb30c00ac30 con 0x7fb31015f370 2024-05-10T10:34:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.646+0000 7fb306ffd640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2f40762d0 0x7fb2f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.646+0000 7fb306ffd640 1 -- 172.21.15.53:0/3111115169 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6187+0+0 (secure 0 0 0) 0x7fb30c0977f0 con 0x7fb31015f370 2024-05-10T10:34:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.647+0000 7fb315abf640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2f40762d0 0x7fb2f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.647+0000 7fb315abf640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2f40762d0 0x7fb2f4078790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb31015ac50 tx=0x7fb300009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.652+0000 7fb306ffd640 1 -- 172.21.15.53:0/3111115169 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb30c061030 con 0x7fb31015f370 2024-05-10T10:34:46.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:46.841+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb31006a220 con 0x7fb31015f370 2024-05-10T10:34:47.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.468+0000 7fb306ffd640 1 -- 172.21.15.53:0/3111115169 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7fb30c060930 con 0x7fb31015f370 2024-05-10T10:34:47.468 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:34:47.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.471+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2f40762d0 msgr2=0x7fb2f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:47.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.471+0000 7fb318524640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2f40762d0 0x7fb2f4078790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb31015ac50 tx=0x7fb300009040 comp rx=0 tx=0).stop 2024-05-10T10:34:47.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015f370 msgr2=0x7fb310159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:47.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015f370 0x7fb310159f30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c830 tx=0x7fb30c00ccf0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 shutdown_connections 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2f40762d0 0x7fb2f4078790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3101639a0 0x7fb310158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb310162e00 0x7fb310154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 --2- 172.21.15.53:0/3111115169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb31015f370 0x7fb310159f30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 >> 172.21.15.53:0/3111115169 conn(0x7fb3100d2e10 msgr2=0x7fb310070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 shutdown_connections 2024-05-10T10:34:47.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.472+0000 7fb318524640 1 -- 172.21.15.53:0/3111115169 wait complete. 2024-05-10T10:34:47.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:34:47.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.600+0000 7f1205c59640 1 Processor -- start 2024-05-10T10:34:47.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.600+0000 7f1205c59640 1 -- start start 2024-05-10T10:34:47.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f1205c59640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1200072ef0 0x7f12000d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:47.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f1205c59640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f12000d1370 0x7f120015c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:47.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f1205c59640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120015fa80 con 0x7f1200159920 2024-05-10T10:34:47.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f1205c59640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f120015fd60 con 0x7f1200072ef0 2024-05-10T10:34:47.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f1205c59640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1200160040 con 0x7f12000d1370 2024-05-10T10:34:47.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11feffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1200072ef0 0x7f12000d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:47.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11feffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1200072ef0 0x7f12000d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50144/0 (socket says 172.21.15.53:50144) 2024-05-10T10:34:47.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11feffd640 1 -- 172.21.15.53:0/1404714006 learned_addr learned my addr 172.21.15.53:0/1404714006 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:47.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11fffff640 1 --2- 172.21.15.53:0/1404714006 >> v2:172.21.15.135:3300/0 conn(0x7f12000d1370 0x7f120015c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:47.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11feffd640 1 -- 172.21.15.53:0/1404714006 >> v2:172.21.15.135:3300/0 conn(0x7f12000d1370 msgr2=0x7f120015c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:47.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11feffd640 1 --2- 172.21.15.53:0/1404714006 >> v2:172.21.15.135:3300/0 conn(0x7f12000d1370 0x7f120015c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11feffd640 1 -- 172.21.15.53:0/1404714006 >> v1:172.21.15.135:6789/0 conn(0x7f1200159920 legacy=0x7f12000b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:47.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.601+0000 7f11feffd640 1 -- 172.21.15.53:0/1404714006 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1200160320 con 0x7f1200072ef0 2024-05-10T10:34:47.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.602+0000 7f11feffd640 1 --2- 172.21.15.53:0/1404714006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f12000d0e30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1200072330 tx=0x7f11f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef97014f87889dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:47.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.602+0000 7f11fe7fc640 1 -- 172.21.15.53:0/1404714006 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4037070 con 0x7f1200072ef0 2024-05-10T10:34:47.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.602+0000 7f11fe7fc640 1 -- 172.21.15.53:0/1404714006 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f4032440 con 0x7f1200072ef0 2024-05-10T10:34:47.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f11fe7fc640 1 -- 172.21.15.53:0/1404714006 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4032810 con 0x7f1200072ef0 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 -- 172.21.15.53:0/1404714006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 msgr2=0x7f12000d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 --2- 172.21.15.53:0/1404714006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f12000d0e30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1200072330 tx=0x7f11f402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 -- 172.21.15.53:0/1404714006 shutdown_connections 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 --2- 172.21.15.53:0/1404714006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f12000d0e30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 --2- 172.21.15.53:0/1404714006 >> v2:172.21.15.135:3300/0 conn(0x7f12000d1370 0x7f120015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 -- 172.21.15.53:0/1404714006 >> 172.21.15.53:0/1404714006 conn(0x7f120006c4a0 msgr2=0x7f120006c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 -- 172.21.15.53:0/1404714006 shutdown_connections 2024-05-10T10:34:47.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.603+0000 7f1205c59640 1 -- 172.21.15.53:0/1404714006 wait complete. 2024-05-10T10:34:47.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.604+0000 7f1205c59640 1 Processor -- start 2024-05-10T10:34:47.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.604+0000 7f1205c59640 1 -- start start 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f1205c59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f120015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f1205c59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12000d1370 0x7f120014a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f1205c59640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1200159920 0x7f120014a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f1205c59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f12000b3db0 con 0x7f1200072ef0 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f1205c59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12001608e0 con 0x7f12000d1370 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f1205c59640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1200159d00 con 0x7f1200159920 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f120015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f120015de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50158/0 (socket says 172.21.15.53:50158) 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.605+0000 7f11ff7fe640 1 -- 172.21.15.53:0/3305949333 learned_addr learned my addr 172.21.15.53:0/3305949333 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11feffd640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12000d1370 0x7f120014a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11fffff640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1200159920 0x7f120014a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11ff7fe640 1 -- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1200159920 msgr2=0x7f120014a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:47.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11ff7fe640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1200159920 0x7f120014a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11ff7fe640 1 -- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12000d1370 msgr2=0x7f120014a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11ff7fe640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12000d1370 0x7f120014a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11ff7fe640 1 -- 172.21.15.53:0/3305949333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12000ce250 con 0x7f1200072ef0 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11feffd640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12000d1370 0x7f120014a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11ff7fe640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f120015de90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c3c0 tx=0x7f11f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11e7fff640 1 -- 172.21.15.53:0/3305949333 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00052d0 con 0x7f1200072ef0 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11e7fff640 1 -- 172.21.15.53:0/3305949333 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f0005490 con 0x7f1200072ef0 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12000ce530 con 0x7f1200072ef0 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f11e7fff640 1 -- 172.21.15.53:0/3305949333 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0007260 con 0x7f1200072ef0 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.606+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12000ceaf0 con 0x7f1200072ef0 2024-05-10T10:34:47.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.607+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120006a220 con 0x7f1200072ef0 2024-05-10T10:34:47.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.608+0000 7f11e7fff640 1 -- 172.21.15.53:0/3305949333 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f11f0007400 con 0x7f1200072ef0 2024-05-10T10:34:47.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.608+0000 7f11e7fff640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11ec0762d0 0x7f11ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:47.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.608+0000 7f11e7fff640 1 -- 172.21.15.53:0/3305949333 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5749+0+0 (secure 0 0 0) 0x7f11f0096aa0 con 0x7f1200072ef0 2024-05-10T10:34:47.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.608+0000 7f11feffd640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11ec0762d0 0x7f11ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:47.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.609+0000 7f11feffd640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11ec0762d0 0x7f11ec078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11f4034000 tx=0x7f11f402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:47.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.613+0000 7f11e7fff640 1 -- 172.21.15.53:0/3305949333 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f11f00603d0 con 0x7f1200072ef0 2024-05-10T10:34:47.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:47.802+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1200072aa0 con 0x7f1200072ef0 2024-05-10T10:34:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.472+0000 7f11e7fff640 1 -- 172.21.15.53:0/3305949333 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7f11f00683c0 con 0x7f1200072ef0 2024-05-10T10:34:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:34:48.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11ec0762d0 msgr2=0x7f11ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11ec0762d0 0x7f11ec078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f11f4034000 tx=0x7f11f402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:48.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 msgr2=0x7f120015de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f120015de90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c3c0 tx=0x7f11f000c880 comp rx=0 tx=0).stop 2024-05-10T10:34:48.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 shutdown_connections 2024-05-10T10:34:48.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11ec0762d0 0x7f11ec078790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1200159920 0x7f120014a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12000d1370 0x7f120014a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 --2- 172.21.15.53:0/3305949333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1200072ef0 0x7f120015de90 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c3c0 tx=0x7f11f000c880 comp rx=0 tx=0).stop 2024-05-10T10:34:48.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.476+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 >> 172.21.15.53:0/3305949333 conn(0x7f120006c4a0 msgr2=0x7f12000af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:48.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.477+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 shutdown_connections 2024-05-10T10:34:48.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.477+0000 7f1205c59640 1 -- 172.21.15.53:0/3305949333 wait complete. 2024-05-10T10:34:48.493 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T10:34:48.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.603+0000 7f09db87c640 1 Processor -- start 2024-05-10T10:34:48.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.604+0000 7f09db87c640 1 -- start start 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.604+0000 7f09db87c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f09d40b52a0 0x7f09d406f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.604+0000 7f09db87c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d40b68c0 con 0x7f09d415d600 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.604+0000 7f09db87c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d40af980 con 0x7f09d40730b0 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.604+0000 7f09db87c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f09d406f650 con 0x7f09d40b52a0 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d9618640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f09d415d600 0x7f09d40b61b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51072/0 (socket says 172.21.15.53:51072) 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d9618640 1 -- 172.21.15.53:0/1755122567 learned_addr learned my addr 172.21.15.53:0/1755122567 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415646335 0 0) 0x7f09d40b68c0 con 0x7f09d415d600 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d8e17640 1 --2- 172.21.15.53:0/1755122567 >> v2:172.21.15.135:3300/0 conn(0x7f09d40b52a0 0x7f09d406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09bc003520 con 0x7f09d415d600 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2371013769 0 0) 0x7f09bc003520 con 0x7f09d415d600 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 >> v2:172.21.15.135:3300/0 conn(0x7f09d40b52a0 msgr2=0x7f09d406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 --2- 172.21.15.53:0/1755122567 >> v2:172.21.15.135:3300/0 conn(0x7f09d40b52a0 0x7f09d406f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 >> v1:172.21.15.135:6789/0 conn(0x7f09d40730b0 legacy=0x7f09d40b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d40afc20 con 0x7f09d415d600 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f09c40019f0 con 0x7f09d415d600 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 >> v1:172.21.15.53:6789/0 conn(0x7f09d415d600 legacy=0x7f09d40b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09bc0039b0 0x7f09bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09bc004800 0x7f09bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0051c0 0x7f09bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc00c130 con 0x7f09bc0039b0 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc00c2e0 con 0x7f09bc004800 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f09bc00c4e0 con 0x7f09bc0051c0 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d9618640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09bc0039b0 0x7f09bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d8e17640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09bc004800 0x7f09bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.605+0000 7f09d9e19640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0051c0 0x7f09bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d9618640 1 -- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0051c0 msgr2=0x7f09bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d9618640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0051c0 0x7f09bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d9618640 1 -- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09bc004800 msgr2=0x7f09bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d9618640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09bc004800 0x7f09bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d9618640 1 -- 172.21.15.53:0/1755122567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc003520 con 0x7f09bc0039b0 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d9e19640 1 -- 172.21.15.53:0/1755122567 reap_dead start 2024-05-10T10:34:48.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d9618640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09bc0039b0 0x7f09bc003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a840 tx=0x7f09bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fe2619d0f7104d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.606+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c400a460 con 0x7f09bc0039b0 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c402f070 con 0x7f09bc0039b0 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09d3fff640 1 -- 172.21.15.53:0/1755122567 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c402bcb0 con 0x7f09bc0039b0 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09db87c640 1 -- 172.21.15.53:0/1755122567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09bc0039b0 msgr2=0x7f09bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09db87c640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09bc0039b0 0x7f09bc003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a840 tx=0x7f09bc005830 comp rx=0 tx=0).stop 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09db87c640 1 -- 172.21.15.53:0/1755122567 shutdown_connections 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09db87c640 1 --2- 172.21.15.53:0/1755122567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09bc0039b0 0x7f09bc003dd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09db87c640 1 -- 172.21.15.53:0/1755122567 >> 172.21.15.53:0/1755122567 conn(0x7f09d40d1ed0 msgr2=0x7f09d40d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09db87c640 1 -- 172.21.15.53:0/1755122567 shutdown_connections 2024-05-10T10:34:48.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.607+0000 7f09db87c640 1 -- 172.21.15.53:0/1755122567 wait complete. 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.608+0000 7f09db87c640 1 Processor -- start 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.608+0000 7f09db87c640 1 -- start start 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09db87c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0039b0 0x7f09d41600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09db87c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09d414e5f0 0x7f09d40d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09db87c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09d40d06b0 0x7f09d406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09db87c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09d40b04c0 con 0x7f09d414e5f0 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09db87c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09d40b01e0 con 0x7f09d40d06b0 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09db87c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f09d406f110 con 0x7f09bc0039b0 2024-05-10T10:34:48.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09d8e17640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09d414e5f0 0x7f09d40d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09d8e17640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09d414e5f0 0x7f09d40d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50178/0 (socket says 172.21.15.53:50178) 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09d8e17640 1 -- 172.21.15.53:0/3881743434 learned_addr learned my addr 172.21.15.53:0/3881743434 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09d9e19640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09d40d06b0 0x7f09d406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09d9618640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0039b0 0x7f09d41600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.609+0000 7f09d8e17640 1 -- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0039b0 msgr2=0x7f09d41600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d8e17640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0039b0 0x7f09d41600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d8e17640 1 -- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09d40d06b0 msgr2=0x7f09d406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d8e17640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09d40d06b0 0x7f09d406c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d8e17640 1 -- 172.21.15.53:0/3881743434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d415df20 con 0x7f09d414e5f0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d8e17640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09d414e5f0 0x7f09d40d0170 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f09c8011aa0 tx=0x7f09c8011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d1ffb640 1 -- 172.21.15.53:0/3881743434 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c800fb80 con 0x7f09d414e5f0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d1ffb640 1 -- 172.21.15.53:0/3881743434 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c800fd20 con 0x7f09d414e5f0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d415e200 con 0x7f09d414e5f0 2024-05-10T10:34:48.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09d1ffb640 1 -- 172.21.15.53:0/3881743434 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8014660 con 0x7f09d414e5f0 2024-05-10T10:34:48.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.610+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d415e7c0 con 0x7f09d414e5f0 2024-05-10T10:34:48.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.611+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d40693f0 con 0x7f09d414e5f0 2024-05-10T10:34:48.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.612+0000 7f09d1ffb640 1 -- 172.21.15.53:0/3881743434 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f09c8012040 con 0x7f09d414e5f0 2024-05-10T10:34:48.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.612+0000 7f09d1ffb640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09c0076300 0x7f09c00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.613+0000 7f09d1ffb640 1 -- 172.21.15.53:0/3881743434 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f09c8097970 con 0x7f09d414e5f0 2024-05-10T10:34:48.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.613+0000 7f09d9618640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09c0076300 0x7f09c00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.613+0000 7f09d9618640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09c0076300 0x7f09c00787c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f09c4033000 tx=0x7f09c4007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.617+0000 7f09d1ffb640 1 -- 172.21.15.53:0/3881743434 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f09c8019030 con 0x7f09d414e5f0 2024-05-10T10:34:48.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.813+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f09d40d0a90 con 0x7f09d414e5f0 2024-05-10T10:34:48.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.817+0000 7f09d1ffb640 1 -- 172.21.15.53:0/3881743434 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f09c80614d0 con 0x7f09d414e5f0 2024-05-10T10:34:48.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09c0076300 msgr2=0x7f09c00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09c0076300 0x7f09c00787c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f09c4033000 tx=0x7f09c4007000 comp rx=0 tx=0).stop 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09d414e5f0 msgr2=0x7f09d40d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09d414e5f0 0x7f09d40d0170 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f09c8011aa0 tx=0x7f09c8011f60 comp rx=0 tx=0).stop 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 shutdown_connections 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09c0076300 0x7f09c00787c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09d40d06b0 0x7f09d406c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09d414e5f0 0x7f09d40d0170 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 --2- 172.21.15.53:0/3881743434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09bc0039b0 0x7f09d41600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 >> 172.21.15.53:0/3881743434 conn(0x7f09d40d1ed0 msgr2=0x7f09d40b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 shutdown_connections 2024-05-10T10:34:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.820+0000 7f09db87c640 1 -- 172.21.15.53:0/3881743434 wait complete. 2024-05-10T10:34:48.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-10T10:34:48.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.934+0000 7fa682ada640 1 Processor -- start 2024-05-10T10:34:48.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.934+0000 7fa682ada640 1 -- start start 2024-05-10T10:34:48.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa682ada640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa67c0d3d80 0x7fa67c0b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa682ada640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa67c0d2d20 con 0x7fa67c0d59d0 2024-05-10T10:34:48.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa682ada640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa67c0d3000 con 0x7fa67c162940 2024-05-10T10:34:48.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa682ada640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa67c162d20 con 0x7fa67c0d3d80 2024-05-10T10:34:48.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa67bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa67c0d3d80 0x7fa67c0b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa67bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa67c0d3d80 0x7fa67c0b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50190/0 (socket says 172.21.15.53:50190) 2024-05-10T10:34:48.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa67bfff640 1 -- 172.21.15.53:0/2023053751 learned_addr learned my addr 172.21.15.53:0/2023053751 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:48.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.935+0000 7fa681077640 1 --1- 172.21.15.53:0/2023053751 >> v1:172.21.15.53:6789/0 conn(0x7fa67c0d59d0 0x7fa67c0d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51088/0 (socket says 172.21.15.53:51088) 2024-05-10T10:34:48.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67bfff640 1 -- 172.21.15.53:0/2023053751 >> v1:172.21.15.135:6789/0 conn(0x7fa67c162940 legacy=0x7fa67c0d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67bfff640 1 -- 172.21.15.53:0/2023053751 >> v1:172.21.15.53:6789/0 conn(0x7fa67c0d59d0 legacy=0x7fa67c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67b7fe640 1 -- 172.21.15.53:0/2023053751 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252540013 0 0) 0x7fa67c0d2d20 con 0x7fa67c0d59d0 2024-05-10T10:34:48.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67bfff640 1 -- 172.21.15.53:0/2023053751 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c0d32e0 con 0x7fa67c0d3d80 2024-05-10T10:34:48.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67bfff640 1 --2- 172.21.15.53:0/2023053751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d3d80 0x7fa67c0b5e20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa670009870 tx=0x7fa67002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a52f0ff0d62d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67b7fe640 1 -- 172.21.15.53:0/2023053751 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670037070 con 0x7fa67c0d3d80 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67b7fe640 1 -- 172.21.15.53:0/2023053751 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa670032440 con 0x7fa67c0d3d80 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.936+0000 7fa67b7fe640 1 -- 172.21.15.53:0/2023053751 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670032810 con 0x7fa67c0d3d80 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.937+0000 7fa682ada640 1 -- 172.21.15.53:0/2023053751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d3d80 msgr2=0x7fa67c0b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.937+0000 7fa682ada640 1 --2- 172.21.15.53:0/2023053751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d3d80 0x7fa67c0b5e20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa670009870 tx=0x7fa67002a270 comp rx=0 tx=0).stop 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.937+0000 7fa682ada640 1 -- 172.21.15.53:0/2023053751 shutdown_connections 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.937+0000 7fa682ada640 1 --2- 172.21.15.53:0/2023053751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d3d80 0x7fa67c0b5e20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.937+0000 7fa682ada640 1 -- 172.21.15.53:0/2023053751 >> 172.21.15.53:0/2023053751 conn(0x7fa67c09e380 msgr2=0x7fa67c0d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.937+0000 7fa682ada640 1 -- 172.21.15.53:0/2023053751 shutdown_connections 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.937+0000 7fa682ada640 1 -- 172.21.15.53:0/2023053751 wait complete. 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.938+0000 7fa682ada640 1 Processor -- start 2024-05-10T10:34:48.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.938+0000 7fa682ada640 1 -- start start 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa682ada640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c0d3d80 0x7fa67c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa682ada640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d59d0 0x7fa67c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa682ada640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67c162940 0x7fa67c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa682ada640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c0d2390 con 0x7fa67c0d59d0 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa682ada640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c0cfc50 con 0x7fa67c0d3d80 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa682ada640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa67c0d2510 con 0x7fa67c162940 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d59d0 0x7fa67c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d59d0 0x7fa67c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50192/0 (socket says 172.21.15.53:50192) 2024-05-10T10:34:48.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.939+0000 7fa67bfff640 1 -- 172.21.15.53:0/2164976391 learned_addr learned my addr 172.21.15.53:0/2164976391 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa681077640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67c162940 0x7fa67c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa680876640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c0d3d80 0x7fa67c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa67bfff640 1 -- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67c162940 msgr2=0x7fa67c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa67bfff640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67c162940 0x7fa67c0ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa67bfff640 1 -- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c0d3d80 msgr2=0x7fa67c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa67bfff640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c0d3d80 0x7fa67c155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa67bfff640 1 -- 172.21.15.53:0/2164976391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c0cb460 con 0x7fa67c0d59d0 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa67bfff640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d59d0 0x7fa67c0ca980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa670009870 tx=0x7fa6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa6797fa640 1 -- 172.21.15.53:0/2164976391 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670041020 con 0x7fa67c0d59d0 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa6797fa640 1 -- 172.21.15.53:0/2164976391 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa670030970 con 0x7fa67c0d59d0 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c0cb740 con 0x7fa67c0d59d0 2024-05-10T10:34:48.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa6797fa640 1 -- 172.21.15.53:0/2164976391 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670037040 con 0x7fa67c0d59d0 2024-05-10T10:34:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.940+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c0cbd00 con 0x7fa67c0d59d0 2024-05-10T10:34:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.941+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa67c06a220 con 0x7fa67c0d59d0 2024-05-10T10:34:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.942+0000 7fa6797fa640 1 -- 172.21.15.53:0/2164976391 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa670044020 con 0x7fa67c0d59d0 2024-05-10T10:34:48.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.943+0000 7fa6797fa640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:48.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.943+0000 7fa6797fa640 1 -- 172.21.15.53:0/2164976391 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7fa6700b7e70 con 0x7fa67c0d59d0 2024-05-10T10:34:48.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.943+0000 7fa680876640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:48.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.943+0000 7fa680876640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000f60 tx=0x7fa66c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:48.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:48.947+0000 7fa6797fa640 1 -- 172.21.15.53:0/2164976391 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa67004b030 con 0x7fa67c0d59d0 2024-05-10T10:34:49.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.218+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fa67c0d2730 con 0x7fa67c0d59d0 2024-05-10T10:34:49.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.474+0000 7fa6797fa640 1 -- 172.21.15.53:0/2164976391 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v85) ==== 142+0+0 (secure 0 0 0) 0x7fa670081940 con 0x7fa67c0d59d0 2024-05-10T10:34:49.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 msgr2=0x7fa668078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000f60 tx=0x7fa66c009040 comp rx=0 tx=0).stop 2024-05-10T10:34:49.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d59d0 msgr2=0x7fa67c0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d59d0 0x7fa67c0ca980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa670009870 tx=0x7fa6700046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 shutdown_connections 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67c162940 0x7fa67c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa67c0d59d0 0x7fa67c0ca980 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 --2- 172.21.15.53:0/2164976391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67c0d3d80 0x7fa67c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.476+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 >> 172.21.15.53:0/2164976391 conn(0x7fa67c09e380 msgr2=0x7fa67c0b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.477+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 shutdown_connections 2024-05-10T10:34:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.477+0000 7fa682ada640 1 -- 172.21.15.53:0/2164976391 wait complete. 2024-05-10T10:34:49.489 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-10T10:34:49.489 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-05-10T10:34:49.489 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-10T10:34:49.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:49.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.594+0000 7fd98f820640 1 Processor -- start 2024-05-10T10:34:49.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.594+0000 7fd98f820640 1 -- start start 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98f820640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd9880b8860 0x7fd98806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98f820640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9880cfd80 0x7fd9880ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98f820640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd98815fc90 con 0x7fd9880d0230 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98f820640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd98806ee50 con 0x7fd9880cfd80 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98f820640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd9880b8c40 con 0x7fd9880b8860 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98ddbd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9880cfd80 0x7fd9880ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98ddbd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9880cfd80 0x7fd9880ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50198/0 (socket says 172.21.15.53:50198) 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98ddbd640 1 -- 172.21.15.53:0/3605008623 learned_addr learned my addr 172.21.15.53:0/3605008623 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98d5bc640 1 --2- 172.21.15.53:0/3605008623 >> v2:172.21.15.135:3300/0 conn(0x7fd9880b8860 0x7fd98806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98ddbd640 1 -- 172.21.15.53:0/3605008623 >> v2:172.21.15.135:3300/0 conn(0x7fd9880b8860 msgr2=0x7fd98806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98ddbd640 1 --2- 172.21.15.53:0/3605008623 >> v2:172.21.15.135:3300/0 conn(0x7fd9880b8860 0x7fd98806e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98ddbd640 1 -- 172.21.15.53:0/3605008623 >> v1:172.21.15.135:6789/0 conn(0x7fd9880d0230 legacy=0x7fd9880cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:49.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.595+0000 7fd98ddbd640 1 -- 172.21.15.53:0/3605008623 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9880cd570 con 0x7fd9880cfd80 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.596+0000 7fd98ddbd640 1 --2- 172.21.15.53:0/3605008623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880cfd80 0x7fd9880ce330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd9880730a0 tx=0x7fd98002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9adae10ec8efd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.596+0000 7fd97ffff640 1 -- 172.21.15.53:0/3605008623 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980036070 con 0x7fd9880cfd80 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.596+0000 7fd97ffff640 1 -- 172.21.15.53:0/3605008623 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd980031600 con 0x7fd9880cfd80 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.596+0000 7fd97ffff640 1 -- 172.21.15.53:0/3605008623 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9800319d0 con 0x7fd9880cfd80 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 -- 172.21.15.53:0/3605008623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880cfd80 msgr2=0x7fd9880ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 --2- 172.21.15.53:0/3605008623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880cfd80 0x7fd9880ce330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd9880730a0 tx=0x7fd98002a660 comp rx=0 tx=0).stop 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 -- 172.21.15.53:0/3605008623 shutdown_connections 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 --2- 172.21.15.53:0/3605008623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880cfd80 0x7fd9880ce330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 --2- 172.21.15.53:0/3605008623 >> v2:172.21.15.135:3300/0 conn(0x7fd9880b8860 0x7fd98806e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 -- 172.21.15.53:0/3605008623 >> 172.21.15.53:0/3605008623 conn(0x7fd9880d2df0 msgr2=0x7fd9880d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 -- 172.21.15.53:0/3605008623 shutdown_connections 2024-05-10T10:34:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.597+0000 7fd98f820640 1 -- 172.21.15.53:0/3605008623 wait complete. 2024-05-10T10:34:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.598+0000 7fd98f820640 1 Processor -- start 2024-05-10T10:34:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.598+0000 7fd98f820640 1 -- start start 2024-05-10T10:34:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98f820640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880b8860 0x7fd9880c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98f820640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9880cfd80 0x7fd9880cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98f820640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9880d0230 0x7fd988162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98f820640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd9880cde10 con 0x7fd9880b8860 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98f820640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9880cdb30 con 0x7fd9880cfd80 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98f820640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd98806ea60 con 0x7fd9880d0230 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880b8860 0x7fd9880c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880b8860 0x7fd9880c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50212/0 (socket says 172.21.15.53:50212) 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 -- 172.21.15.53:0/3931982662 learned_addr learned my addr 172.21.15.53:0/3931982662 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98ddbd640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9880d0230 0x7fd988162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98cdbb640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9880cfd80 0x7fd9880cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 -- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9880d0230 msgr2=0x7fd988162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9880d0230 0x7fd988162480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 -- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9880cfd80 msgr2=0x7fd9880cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9880cfd80 0x7fd9880cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 -- 172.21.15.53:0/3931982662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9880ccd90 con 0x7fd9880b8860 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98ddbd640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9880d0230 0x7fd988162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98cdbb640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9880cfd80 0x7fd9880cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98d5bc640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880b8860 0x7fd9880c8380 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd98400d620 tx=0x7fd98400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd97dffb640 1 -- 172.21.15.53:0/3931982662 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98400b620 con 0x7fd9880b8860 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd97dffb640 1 -- 172.21.15.53:0/3931982662 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd984019070 con 0x7fd9880b8860 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9880c8a10 con 0x7fd9880b8860 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.599+0000 7fd97dffb640 1 -- 172.21.15.53:0/3931982662 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98400ba00 con 0x7fd9880b8860 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.600+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9880c8fd0 con 0x7fd9880b8860 2024-05-10T10:34:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.601+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9880711b0 con 0x7fd9880b8860 2024-05-10T10:34:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.601+0000 7fd97dffb640 1 -- 172.21.15.53:0/3931982662 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd98400bba0 con 0x7fd9880b8860 2024-05-10T10:34:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.602+0000 7fd97dffb640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd96c0762d0 0x7fd96c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.602+0000 7fd97dffb640 1 -- 172.21.15.53:0/3931982662 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7fd9840978e0 con 0x7fd9880b8860 2024-05-10T10:34:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.602+0000 7fd98cdbb640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd96c0762d0 0x7fd96c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.603+0000 7fd98cdbb640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd96c0762d0 0x7fd96c078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9881f8470 tx=0x7fd978009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.606+0000 7fd97dffb640 1 -- 172.21.15.53:0/3931982662 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd9880711b0 con 0x7fd9880b8860 2024-05-10T10:34:49.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.803+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9880d1e20 con 0x7fd9880b8860 2024-05-10T10:34:49.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.804+0000 7fd97dffb640 1 -- 172.21.15.53:0/3931982662 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13291 (secure 0 0 0) 0x7fd9840613b0 con 0x7fd9880b8860 2024-05-10T10:34:49.804 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:49.804 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":39,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:48.468862+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/561700189":"2024-05-11T10:34:45.461097+0000","172.21.15.53:6834/561700189":"2024-05-11T10:34:45.461097+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:49.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd96c0762d0 msgr2=0x7fd96c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd96c0762d0 0x7fd96c078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9881f8470 tx=0x7fd978009040 comp rx=0 tx=0).stop 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880b8860 msgr2=0x7fd9880c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880b8860 0x7fd9880c8380 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd98400d620 tx=0x7fd98400dae0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 shutdown_connections 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd96c0762d0 0x7fd96c078790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9880d0230 0x7fd988162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9880cfd80 0x7fd9880cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 --2- 172.21.15.53:0/3931982662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9880b8860 0x7fd9880c8380 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 >> 172.21.15.53:0/3931982662 conn(0x7fd9880d2df0 msgr2=0x7fd98815f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 shutdown_connections 2024-05-10T10:34:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.806+0000 7fd98f820640 1 -- 172.21.15.53:0/3931982662 wait complete. 2024-05-10T10:34:49.817 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:33:28.171977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T10:34:49.818 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:34:49.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.924+0000 7f9f9b009640 1 Processor -- start 2024-05-10T10:34:49.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.924+0000 7f9f9b009640 1 -- start start 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f9b009640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f94162900 0x7f9f940d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f9b009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162ce0 0x7f9f940b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f9b009640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f940d3950 0x7f9f940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f9b009640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9f940d4190 con 0x7f9f940d3950 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f9b009640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9f940cfc40 con 0x7f9f94162900 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f9b009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9f940d3eb0 con 0x7f9f94162ce0 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f940d3950 0x7f9f940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f940d3950 0x7f9f940b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50218/0 (socket says 172.21.15.53:50218) 2024-05-10T10:34:49.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 -- 172.21.15.53:0/3996898222 learned_addr learned my addr 172.21.15.53:0/3996898222 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:49.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 -- 172.21.15.53:0/3996898222 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162ce0 msgr2=0x7f9f940b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:49.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 --2- 172.21.15.53:0/3996898222 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162ce0 0x7f9f940b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 -- 172.21.15.53:0/3996898222 >> v2:172.21.15.135:3300/0 conn(0x7f9f94162900 msgr2=0x7f9f940d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:49.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 --2- 172.21.15.53:0/3996898222 >> v2:172.21.15.135:3300/0 conn(0x7f9f94162900 0x7f9f940d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.925+0000 7f9f995a6640 1 -- 172.21.15.53:0/3996898222 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f940cfee0 con 0x7f9f940d3950 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.926+0000 7f9f995a6640 1 --2- 172.21.15.53:0/3996898222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f940b4e20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9f9406c520 tx=0x7f9f8c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7239f1c52b796f5b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.926+0000 7f9f937fe640 1 -- 172.21.15.53:0/3996898222 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c036070 con 0x7f9f940d3950 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.926+0000 7f9f937fe640 1 -- 172.21.15.53:0/3996898222 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c031600 con 0x7f9f940d3950 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.926+0000 7f9f937fe640 1 -- 172.21.15.53:0/3996898222 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0319d0 con 0x7f9f940d3950 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 -- 172.21.15.53:0/3996898222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 msgr2=0x7f9f940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 --2- 172.21.15.53:0/3996898222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f940b4e20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9f9406c520 tx=0x7f9f8c02a610 comp rx=0 tx=0).stop 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 -- 172.21.15.53:0/3996898222 shutdown_connections 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 --2- 172.21.15.53:0/3996898222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f940b4e20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 --2- 172.21.15.53:0/3996898222 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162ce0 0x7f9f940b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 --2- 172.21.15.53:0/3996898222 >> v2:172.21.15.135:3300/0 conn(0x7f9f94162900 0x7f9f940d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 -- 172.21.15.53:0/3996898222 >> 172.21.15.53:0/3996898222 conn(0x7f9f9409e430 msgr2=0x7f9f940d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:49.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 -- 172.21.15.53:0/3996898222 shutdown_connections 2024-05-10T10:34:49.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.927+0000 7f9f9b009640 1 -- 172.21.15.53:0/3996898222 wait complete. 2024-05-10T10:34:49.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.928+0000 7f9f9b009640 1 Processor -- start 2024-05-10T10:34:49.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.928+0000 7f9f9b009640 1 -- start start 2024-05-10T10:34:49.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f9b009640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f9415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f9b009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162900 0x7f9f9415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f9b009640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f94162ce0 0x7f9f9415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f9b009640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9f940b4300 con 0x7f9f940d3950 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f9b009640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f940b4180 con 0x7f9f94162ce0 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f9b009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9f940d09c0 con 0x7f9f94162900 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f9415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f9415d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50230/0 (socket says 172.21.15.53:50230) 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 -- 172.21.15.53:0/1081058084 learned_addr learned my addr 172.21.15.53:0/1081058084 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f995a6640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f94162ce0 0x7f9f9415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f93fff640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162900 0x7f9f9415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 -- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162900 msgr2=0x7f9f9415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162900 0x7f9f9415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 -- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f94162ce0 msgr2=0x7f9f9415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f94162ce0 0x7f9f9415bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 -- 172.21.15.53:0/1081058084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9415df30 con 0x7f9f940d3950 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f93fff640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162900 0x7f9f9415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f98da5640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f9415d9f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400b4a0 tx=0x7f9f8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:49.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.929+0000 7f9f917fa640 1 -- 172.21.15.53:0/1081058084 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84014070 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.930+0000 7f9f917fa640 1 -- 172.21.15.53:0/1081058084 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f84010610 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.930+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f940cafd0 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.930+0000 7f9f917fa640 1 -- 172.21.15.53:0/1081058084 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84010a00 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.930+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f940cc5c0 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.930+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f9406a220 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.931+0000 7f9f917fa640 1 -- 172.21.15.53:0/1081058084 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9f84017070 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.932+0000 7f9f917fa640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f800762d0 0x7f9f80078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.932+0000 7f9f917fa640 1 -- 172.21.15.53:0/1081058084 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f9f84096d50 con 0x7f9f940d3950 2024-05-10T10:34:49.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.932+0000 7f9f93fff640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f800762d0 0x7f9f80078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:49.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.932+0000 7f9f93fff640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f800762d0 0x7f9f80078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f940c8b90 tx=0x7f9f88008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:49.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:49.936+0000 7f9f917fa640 1 -- 172.21.15.53:0/1081058084 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9f8409a030 con 0x7f9f940d3950 2024-05-10T10:34:50.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.124+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f9409e840 con 0x7f9f940d3950 2024-05-10T10:34:50.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.124+0000 7f9f917fa640 1 -- 172.21.15.53:0/1081058084 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f9f84060260 con 0x7f9f940d3950 2024-05-10T10:34:50.125 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-05-10T10:34:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f800762d0 msgr2=0x7f9f80078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f800762d0 0x7f9f80078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f940c8b90 tx=0x7f9f88008040 comp rx=0 tx=0).stop 2024-05-10T10:34:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 msgr2=0x7f9f9415d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f9415d9f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400b4a0 tx=0x7f9f8400b960 comp rx=0 tx=0).stop 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 shutdown_connections 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f800762d0 0x7f9f80078790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f94162ce0 0x7f9f9415bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f94162900 0x7f9f9415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 --2- 172.21.15.53:0/1081058084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f940d3950 0x7f9f9415d9f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.126+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 >> 172.21.15.53:0/1081058084 conn(0x7f9f9409e430 msgr2=0x7f9f9406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.127+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 shutdown_connections 2024-05-10T10:34:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.127+0000 7f9f9b009640 1 -- 172.21.15.53:0/1081058084 wait complete. 2024-05-10T10:34:50.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-10T10:34:50.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.245+0000 7f1c0c61d640 1 Processor -- start 2024-05-10T10:34:50.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.245+0000 7f1c0c61d640 1 -- start start 2024-05-10T10:34:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.246+0000 7f1c0c61d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1c0415ce60 0x7f1c040cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.246+0000 7f1c0c61d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0415d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.246+0000 7f1c0c61d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c040c6a60 con 0x7f1c040c65f0 2024-05-10T10:34:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.246+0000 7f1c0c61d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1c040cd090 con 0x7f1c0415ce60 2024-05-10T10:34:50.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.246+0000 7f1c0c61d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1c040cd370 con 0x7f1c041611e0 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0abba640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0a3b9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1c0415ce60 0x7f1c040cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c09bb8640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f1c040c65f0 0x7f1c04160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:33398/0 (socket says 172.21.15.53:33398) 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0a3b9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1c0415ce60 0x7f1c040cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:33750/0 (socket says 172.21.15.53:33750) 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c09bb8640 1 -- 172.21.15.53:0/16439458 learned_addr learned my addr 172.21.15.53:0/16439458 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0abba640 1 -- 172.21.15.53:0/16439458 >> v2:172.21.15.135:3300/0 conn(0x7f1c0415ce60 msgr2=0x7f1c040cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0abba640 1 --2- 172.21.15.53:0/16439458 >> v2:172.21.15.135:3300/0 conn(0x7f1c0415ce60 0x7f1c040cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0abba640 1 -- 172.21.15.53:0/16439458 >> v1:172.21.15.135:6789/0 conn(0x7f1c040c65f0 legacy=0x7f1c04160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0abba640 1 -- 172.21.15.53:0/16439458 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c040cd650 con 0x7f1c041611e0 2024-05-10T10:34:50.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.247+0000 7f1c0a3b9640 1 --2- 172.21.15.53:0/16439458 >> v2:172.21.15.135:3300/0 conn(0x7f1c0415ce60 0x7f1c040cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.248+0000 7f1c0abba640 1 --2- 172.21.15.53:0/16439458 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0415d240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009d90 tx=0x7f1bf402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ea2a8ed779eaf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.248+0000 7f1c093b7640 1 -- 172.21.15.53:0/16439458 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4031600 con 0x7f1c041611e0 2024-05-10T10:34:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.248+0000 7f1c093b7640 1 -- 172.21.15.53:0/16439458 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf40317a0 con 0x7f1c041611e0 2024-05-10T10:34:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.248+0000 7f1c093b7640 1 -- 172.21.15.53:0/16439458 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4031b70 con 0x7f1c041611e0 2024-05-10T10:34:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 -- 172.21.15.53:0/16439458 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 msgr2=0x7f1c0415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/16439458 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0415d240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009d90 tx=0x7f1bf402a5d0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 -- 172.21.15.53:0/16439458 shutdown_connections 2024-05-10T10:34:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/16439458 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0415d240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/16439458 >> v2:172.21.15.135:3300/0 conn(0x7f1c0415ce60 0x7f1c040cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 -- 172.21.15.53:0/16439458 >> 172.21.15.53:0/16439458 conn(0x7f1c040b2900 msgr2=0x7f1c040b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 -- 172.21.15.53:0/16439458 shutdown_connections 2024-05-10T10:34:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.249+0000 7f1c0c61d640 1 -- 172.21.15.53:0/16439458 wait complete. 2024-05-10T10:34:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.250+0000 7f1c0c61d640 1 Processor -- start 2024-05-10T10:34:50.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.250+0000 7f1c0c61d640 1 -- start start 2024-05-10T10:34:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0c61d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040c65f0 0x7f1c040b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0c61d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c0415ce60 0x7f1c0406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0c61d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0414bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0c61d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c040cde50 con 0x7f1c0415ce60 2024-05-10T10:34:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0c61d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c040ccd60 con 0x7f1c040c65f0 2024-05-10T10:34:50.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0c61d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1c040cdfd0 con 0x7f1c041611e0 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c0415ce60 0x7f1c0406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c0415ce60 0x7f1c0406dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50242/0 (socket says 172.21.15.53:50242) 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 -- 172.21.15.53:0/2060246846 learned_addr learned my addr 172.21.15.53:0/2060246846 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0abba640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0414bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0a3b9640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040c65f0 0x7f1c040b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 -- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 msgr2=0x7f1c0414bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0414bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 -- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040c65f0 msgr2=0x7f1c040b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040c65f0 0x7f1c040b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 -- 172.21.15.53:0/2060246846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0406e2d0 con 0x7f1c0415ce60 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c09bb8640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c0415ce60 0x7f1c0406dd90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c400 tx=0x7f1bf800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c02ffd640 1 -- 172.21.15.53:0/2060246846 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8017070 con 0x7f1c0415ce60 2024-05-10T10:34:50.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c02ffd640 1 -- 172.21.15.53:0/2060246846 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf8005ce0 con 0x7f1c0415ce60 2024-05-10T10:34:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.251+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0406e570 con 0x7f1c0415ce60 2024-05-10T10:34:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.252+0000 7f1c02ffd640 1 -- 172.21.15.53:0/2060246846 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8007310 con 0x7f1c0415ce60 2024-05-10T10:34:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.252+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0406eb30 con 0x7f1c0415ce60 2024-05-10T10:34:50.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.252+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0406a220 con 0x7f1c0415ce60 2024-05-10T10:34:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.253+0000 7f1c02ffd640 1 -- 172.21.15.53:0/2060246846 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1bf801a070 con 0x7f1c0415ce60 2024-05-10T10:34:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.254+0000 7f1c02ffd640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1bf00762d0 0x7f1bf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.254+0000 7f1c02ffd640 1 -- 172.21.15.53:0/2060246846 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f1bf8096860 con 0x7f1c0415ce60 2024-05-10T10:34:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.254+0000 7f1c0a3b9640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1bf00762d0 0x7f1bf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:50.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.255+0000 7f1c0a3b9640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1bf00762d0 0x7f1bf0078790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009560 tx=0x7f1bfc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:50.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.258+0000 7f1c02ffd640 1 -- 172.21.15.53:0/2060246846 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1bf805fc40 con 0x7f1c0415ce60 2024-05-10T10:34:50.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:50.524+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f1c040af170 con 0x7f1c0415ce60 2024-05-10T10:34:51.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.482+0000 7f1c02ffd640 1 -- 172.21.15.53:0/2060246846 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v87) ==== 152+0+0 (secure 0 0 0) 0x7f1bf8068280 con 0x7f1c0415ce60 2024-05-10T10:34:51.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1bf00762d0 msgr2=0x7f1bf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1bf00762d0 0x7f1bf0078790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009560 tx=0x7f1bfc009190 comp rx=0 tx=0).stop 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c0415ce60 msgr2=0x7f1c0406dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c0415ce60 0x7f1c0406dd90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c400 tx=0x7f1bf800c8c0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 shutdown_connections 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1bf00762d0 0x7f1bf0078790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c041611e0 0x7f1c0414bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c0415ce60 0x7f1c0406dd90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 --2- 172.21.15.53:0/2060246846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040c65f0 0x7f1c040b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 >> 172.21.15.53:0/2060246846 conn(0x7f1c040b2900 msgr2=0x7f1c0415d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 shutdown_connections 2024-05-10T10:34:51.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.485+0000 7f1c0c61d640 1 -- 172.21.15.53:0/2060246846 wait complete. 2024-05-10T10:34:51.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T10:34:51.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.612+0000 7f6e74872640 1 Processor -- start 2024-05-10T10:34:51.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.612+0000 7f6e74872640 1 -- start start 2024-05-10T10:34:51.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e74872640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e700d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e74872640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e70162ce0 0x7f6e700b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e74872640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6e700d3950 0x7f6e700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e74872640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6e700d4190 con 0x7f6e70162ce0 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e74872640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6e700cfc40 con 0x7f6e700d3950 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e74872640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6e700d3eb0 con 0x7f6e70162900 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e70162ce0 0x7f6e700b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e70162ce0 0x7f6e700b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50244/0 (socket says 172.21.15.53:50244) 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 -- 172.21.15.53:0/387819582 learned_addr learned my addr 172.21.15.53:0/387819582 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e6ed9d640 1 --2- 172.21.15.53:0/387819582 >> v2:172.21.15.135:3300/0 conn(0x7f6e700d3950 0x7f6e700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e6e59c640 1 --2- 172.21.15.53:0/387819582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e700d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 -- 172.21.15.53:0/387819582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 msgr2=0x7f6e700d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 --2- 172.21.15.53:0/387819582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e700d3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 -- 172.21.15.53:0/387819582 >> v2:172.21.15.135:3300/0 conn(0x7f6e700d3950 msgr2=0x7f6e700b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 --2- 172.21.15.53:0/387819582 >> v2:172.21.15.135:3300/0 conn(0x7f6e700d3950 0x7f6e700b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e67fff640 1 -- 172.21.15.53:0/387819582 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e700cfee0 con 0x7f6e70162ce0 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.613+0000 7f6e6ed9d640 1 --2- 172.21.15.53:0/387819582 >> v2:172.21.15.135:3300/0 conn(0x7f6e700d3950 0x7f6e700b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.614+0000 7f6e67fff640 1 --2- 172.21.15.53:0/387819582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e70162ce0 0x7f6e700b48e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e7008e560 tx=0x7f6e5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c5b22cbd53fb1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.614+0000 7f6e6dd9b640 1 -- 172.21.15.53:0/387819582 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58037070 con 0x7f6e70162ce0 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.614+0000 7f6e6dd9b640 1 -- 172.21.15.53:0/387819582 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e58032440 con 0x7f6e70162ce0 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.614+0000 7f6e6dd9b640 1 -- 172.21.15.53:0/387819582 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58032810 con 0x7f6e70162ce0 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 -- 172.21.15.53:0/387819582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e70162ce0 msgr2=0x7f6e700b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 --2- 172.21.15.53:0/387819582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e70162ce0 0x7f6e700b48e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e7008e560 tx=0x7f6e5802a140 comp rx=0 tx=0).stop 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 -- 172.21.15.53:0/387819582 shutdown_connections 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 --2- 172.21.15.53:0/387819582 >> v2:172.21.15.135:3300/0 conn(0x7f6e700d3950 0x7f6e700b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 --2- 172.21.15.53:0/387819582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e70162ce0 0x7f6e700b48e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 --2- 172.21.15.53:0/387819582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e700d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 -- 172.21.15.53:0/387819582 >> 172.21.15.53:0/387819582 conn(0x7f6e7009e430 msgr2=0x7f6e700d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:51.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 -- 172.21.15.53:0/387819582 shutdown_connections 2024-05-10T10:34:51.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.615+0000 7f6e74872640 1 -- 172.21.15.53:0/387819582 wait complete. 2024-05-10T10:34:51.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.616+0000 7f6e74872640 1 Processor -- start 2024-05-10T10:34:51.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.616+0000 7f6e74872640 1 -- start start 2024-05-10T10:34:51.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e74872640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e700d3950 0x7f6e700c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e74872640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e7014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e74872640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e70162ce0 0x7f6e7014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e74872640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700b4300 con 0x7f6e700d3950 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e74872640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700b4180 con 0x7f6e70162ce0 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e74872640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6e700d09c0 con 0x7f6e70162900 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e700d3950 0x7f6e700c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e700d3950 0x7f6e700c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50260/0 (socket says 172.21.15.53:50260) 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 -- 172.21.15.53:0/1502041260 learned_addr learned my addr 172.21.15.53:0/1502041260 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6ed9d640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e70162ce0 0x7f6e7014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e67fff640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e7014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 -- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 msgr2=0x7f6e7014ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e7014ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 -- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e70162ce0 msgr2=0x7f6e7014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e70162ce0 0x7f6e7014f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.617+0000 7f6e6e59c640 1 -- 172.21.15.53:0/1502041260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e700cdcb0 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.618+0000 7f6e6e59c640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e700d3950 0x7f6e700c8cf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000c7f0 tx=0x7f6e6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.618+0000 7f6e66ffd640 1 -- 172.21.15.53:0/1502041260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60015070 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.618+0000 7f6e66ffd640 1 -- 172.21.15.53:0/1502041260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e60004240 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.618+0000 7f6e66ffd640 1 -- 172.21.15.53:0/1502041260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60007620 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.618+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e700cdf90 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.618+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e700cbef0 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.619+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7006a220 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.619+0000 7f6e66ffd640 1 -- 172.21.15.53:0/1502041260 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6e600079d0 con 0x7f6e700d3950 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.620+0000 7f6e66ffd640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e50076300 0x7f6e500787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:51.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.620+0000 7f6e66ffd640 1 -- 172.21.15.53:0/1502041260 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f6e60097530 con 0x7f6e700d3950 2024-05-10T10:34:51.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.620+0000 7f6e67fff640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e50076300 0x7f6e500787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:51.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.621+0000 7f6e67fff640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e50076300 0x7f6e500787c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e58034000 tx=0x7f6e5802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:51.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.624+0000 7f6e66ffd640 1 -- 172.21.15.53:0/1502041260 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6e60061000 con 0x7f6e700d3950 2024-05-10T10:34:51.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.816+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6e7009e840 con 0x7f6e700d3950 2024-05-10T10:34:51.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.819+0000 7f6e66ffd640 1 -- 172.21.15.53:0/1502041260 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f6e60060900 con 0x7f6e700d3950 2024-05-10T10:34:51.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e50076300 msgr2=0x7f6e500787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e50076300 0x7f6e500787c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e58034000 tx=0x7f6e5802e000 comp rx=0 tx=0).stop 2024-05-10T10:34:51.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e700d3950 msgr2=0x7f6e700c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e700d3950 0x7f6e700c8cf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000c7f0 tx=0x7f6e6000ccb0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 shutdown_connections 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e50076300 0x7f6e500787c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e70162ce0 0x7f6e7014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6e70162900 0x7f6e7014ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 --2- 172.21.15.53:0/1502041260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e700d3950 0x7f6e700c8cf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 >> 172.21.15.53:0/1502041260 conn(0x7f6e7009e430 msgr2=0x7f6e700d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 shutdown_connections 2024-05-10T10:34:51.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:51.821+0000 7f6e74872640 1 -- 172.21.15.53:0/1502041260 wait complete. 2024-05-10T10:34:51.833 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:34:51.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:34:51.880 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:34:51.921 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:34:51.921 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:51.921 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:51.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:34:51.938 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:51.938 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:51.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:34:51.980 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:34:51.980 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:51.996 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:34:51.996 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:34:51.996 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:34:52.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:34:52.037 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T10:34:52.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:34:52.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.141+0000 7f58f2514640 1 Processor -- start 2024-05-10T10:34:52.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.141+0000 7f58f2514640 1 -- start start 2024-05-10T10:34:52.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58f2514640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58ec15f790 0x7f58ec0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58f2514640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58f2514640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ec0d30b0 con 0x7f58ec15f3b0 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58f2514640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f58ec0d3390 con 0x7f58ec15f790 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58f2514640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec0c8c80 con 0x7f58ec0d5d30 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58eb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58ec15f790 0x7f58ec0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58eb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58ec15f790 0x7f58ec0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50272/0 (socket says 172.21.15.53:50272) 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3236934790 learned_addr learned my addr 172.21.15.53:0/3236934790 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58f0ab1640 1 --2- 172.21.15.53:0/3236934790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3236934790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 msgr2=0x7f58ec0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58eb7fe640 1 --2- 172.21.15.53:0/3236934790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3236934790 >> v1:172.21.15.135:6789/0 conn(0x7f58ec15f3b0 legacy=0x7f58ec0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.142+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3236934790 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec0c8f60 con 0x7f58ec15f790 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.143+0000 7f58eb7fe640 1 --2- 172.21.15.53:0/3236934790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f790 0x7f58ec0cfcd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009870 tx=0x7f58e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dafb13e1ca634 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.143+0000 7f58eaffd640 1 -- 172.21.15.53:0/3236934790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0037070 con 0x7f58ec15f790 2024-05-10T10:34:52.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.143+0000 7f58eaffd640 1 -- 172.21.15.53:0/3236934790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e0032440 con 0x7f58ec15f790 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.143+0000 7f58eaffd640 1 -- 172.21.15.53:0/3236934790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0032810 con 0x7f58ec15f790 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 -- 172.21.15.53:0/3236934790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f790 msgr2=0x7f58ec0cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 --2- 172.21.15.53:0/3236934790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f790 0x7f58ec0cfcd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009870 tx=0x7f58e002a270 comp rx=0 tx=0).stop 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 -- 172.21.15.53:0/3236934790 shutdown_connections 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 --2- 172.21.15.53:0/3236934790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 --2- 172.21.15.53:0/3236934790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f790 0x7f58ec0cfcd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 -- 172.21.15.53:0/3236934790 >> 172.21.15.53:0/3236934790 conn(0x7f58ec0d5130 msgr2=0x7f58ec0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 -- 172.21.15.53:0/3236934790 shutdown_connections 2024-05-10T10:34:52.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.144+0000 7f58f2514640 1 -- 172.21.15.53:0/3236934790 wait complete. 2024-05-10T10:34:52.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.145+0000 7f58f2514640 1 Processor -- start 2024-05-10T10:34:52.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.145+0000 7f58f2514640 1 -- start start 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58f2514640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58f2514640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f3b0 0x7f58ec0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58f2514640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec15b820 0x7f58ec0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58f2514640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec0d3f80 con 0x7f58ec15f3b0 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58f2514640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec0d3e00 con 0x7f58ec15b820 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58f2514640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec0ca770 con 0x7f58ec0d5d30 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58eb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f3b0 0x7f58ec0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58eb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f3b0 0x7f58ec0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50280/0 (socket says 172.21.15.53:50280) 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3410595713 learned_addr learned my addr 172.21.15.53:0/3410595713 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58f0ab1640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec15b820 0x7f58ec0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.146+0000 7f58ebfff640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 msgr2=0x7f58ec0b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58eb7fe640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec15b820 msgr2=0x7f58ec0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58eb7fe640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec15b820 0x7f58ec0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58eb7fe640 1 -- 172.21.15.53:0/3410595713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec15a780 con 0x7f58ec15f3b0 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58f0ab1640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec15b820 0x7f58ec0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58ebfff640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58eb7fe640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f3b0 0x7f58ec0cab40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f58e0000c00 tx=0x7f58e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58e8ff9640 1 -- 172.21.15.53:0/3410595713 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0041020 con 0x7f58ec15f3b0 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58e8ff9640 1 -- 172.21.15.53:0/3410595713 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e00306c0 con 0x7f58ec15f3b0 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58e8ff9640 1 -- 172.21.15.53:0/3410595713 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0037040 con 0x7f58ec15f3b0 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec15aa60 con 0x7f58ec15f3b0 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.147+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec15b020 con 0x7f58ec15f3b0 2024-05-10T10:34:52.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.148+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ec06a220 con 0x7f58ec15f3b0 2024-05-10T10:34:52.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.149+0000 7f58e8ff9640 1 -- 172.21.15.53:0/3410595713 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58e0044020 con 0x7f58ec15f3b0 2024-05-10T10:34:52.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.149+0000 7f58e8ff9640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58d8076300 0x7f58d80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.149+0000 7f58e8ff9640 1 -- 172.21.15.53:0/3410595713 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f58e00b7c50 con 0x7f58ec15f3b0 2024-05-10T10:34:52.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.149+0000 7f58ebfff640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58d8076300 0x7f58d80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.150+0000 7f58ebfff640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58d8076300 0x7f58d80787c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58ec15e6d0 tx=0x7f58dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:52.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.153+0000 7f58e8ff9640 1 -- 172.21.15.53:0/3410595713 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f58e00bd020 con 0x7f58ec15f3b0 2024-05-10T10:34:52.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.390+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58ec0cbcc0 con 0x7f58ec15f3b0 2024-05-10T10:34:52.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.391+0000 7f58e8ff9640 1 -- 172.21.15.53:0/3410595713 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3866 (secure 0 0 0) 0x7f58e004b030 con 0x7f58ec15f3b0 2024-05-10T10:34:52.392 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:52.392 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/82825390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":82825390},{"type":"v1","addr":"172.21.15.135:6836","nonce":82825390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2766940165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2766940165},{"type":"v1","addr":"172.21.15.135:6837","nonce":2766940165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1199572987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1199572987},{"type":"v1","addr":"172.21.15.53:6833","nonce":1199572987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/788036663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":788036663},{"type":"v1","addr":"172.21.15.53:6835","nonce":788036663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-05-10T10:34:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 20 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58d8076300 msgr2=0x7f58d80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58d8076300 0x7f58d80787c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58ec15e6d0 tx=0x7f58dc009040 comp rx=0 tx=0).stop 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f3b0 msgr2=0x7f58ec0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f3b0 0x7f58ec0cab40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f58e0000c00 tx=0x7f58e00046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 shutdown_connections 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58d8076300 0x7f58d80787c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ec15b820 0x7f58ec0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ec15f3b0 0x7f58ec0cab40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 --2- 172.21.15.53:0/3410595713 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58ec0d5d30 0x7f58ec0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 >> 172.21.15.53:0/3410595713 conn(0x7f58ec0d5130 msgr2=0x7f58ec06e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 shutdown_connections 2024-05-10T10:34:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.393+0000 7f58f2514640 1 -- 172.21.15.53:0/3410595713 wait complete. 2024-05-10T10:34:52.405 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:34:52.405 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T10:34:52.406 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:34:52.406 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:52.406 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:34:52.406 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T10:34:52.406 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:34:52.406 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:52.406 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:34:52.407 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T10:34:52.407 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:34:52.407 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:52.407 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:34:52.407 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T10:34:52.407 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:34:52.407 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:34:52.415 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:34:52.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:34:52.417 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:34:52.417 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:34:52.418 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:34:52.419 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:34:52.419 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:34:52.420 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:34:52.459 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:34:52.460 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:52.461 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:34:52.461 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:52.462 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:34:52.462 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:52.463 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:34:52.463 DEBUG:teuthology.parallel:result is None 2024-05-10T10:34:52.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T10:34:52.538 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:34:52.539 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:34:52.545 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:34:52.545 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:34:52.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.572+0000 7f38a22a0640 1 Processor -- start 2024-05-10T10:34:52.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.572+0000 7f38a22a0640 1 -- start start 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f38a22a0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f389c15d290 0x7f389c071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f38a22a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c071880 0x7f389c0d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f38a22a0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f389c0d1aa0 0x7f389c0d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f38a22a0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f389c072330 con 0x7f389c15d290 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f38a22a0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f389c071d70 con 0x7f389c0d1aa0 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f38a22a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f389c072050 con 0x7f389c071880 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f389bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f389c15d290 0x7f389c071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f389bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f389c15d290 0x7f389c071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50328/0 (socket says 172.21.15.53:50328) 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f389bfff640 1 -- 172.21.15.53:0/2723426242 learned_addr learned my addr 172.21.15.53:0/2723426242 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f38a083d640 1 --2- 172.21.15.53:0/2723426242 >> v2:172.21.15.135:3300/0 conn(0x7f389c0d1aa0 0x7f389c0d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.573+0000 7f3893fff640 1 --2- 172.21.15.53:0/2723426242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c071880 0x7f389c0d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.574+0000 7f389bfff640 1 -- 172.21.15.53:0/2723426242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c071880 msgr2=0x7f389c0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.574+0000 7f389bfff640 1 --2- 172.21.15.53:0/2723426242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c071880 0x7f389c0d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.574+0000 7f389bfff640 1 -- 172.21.15.53:0/2723426242 >> v2:172.21.15.135:3300/0 conn(0x7f389c0d1aa0 msgr2=0x7f389c0d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.574+0000 7f389bfff640 1 --2- 172.21.15.53:0/2723426242 >> v2:172.21.15.135:3300/0 conn(0x7f389c0d1aa0 0x7f389c0d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.574+0000 7f389bfff640 1 -- 172.21.15.53:0/2723426242 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c0cda70 con 0x7f389c15d290 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.574+0000 7f38a083d640 1 --2- 172.21.15.53:0/2723426242 >> v2:172.21.15.135:3300/0 conn(0x7f389c0d1aa0 0x7f389c0d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.574+0000 7f389bfff640 1 --2- 172.21.15.53:0/2723426242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c15d290 0x7f389c071340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f389c09c660 tx=0x7f388c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a443541eef84e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f389b7fe640 1 -- 172.21.15.53:0/2723426242 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c037070 con 0x7f389c15d290 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f389b7fe640 1 -- 172.21.15.53:0/2723426242 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c032440 con 0x7f389c15d290 2024-05-10T10:34:52.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f389b7fe640 1 -- 172.21.15.53:0/2723426242 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c032810 con 0x7f389c15d290 2024-05-10T10:34:52.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 -- 172.21.15.53:0/2723426242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c15d290 msgr2=0x7f389c071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 --2- 172.21.15.53:0/2723426242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c15d290 0x7f389c071340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f389c09c660 tx=0x7f388c02a1e0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 -- 172.21.15.53:0/2723426242 shutdown_connections 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 --2- 172.21.15.53:0/2723426242 >> v2:172.21.15.135:3300/0 conn(0x7f389c0d1aa0 0x7f389c0d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 --2- 172.21.15.53:0/2723426242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c071880 0x7f389c0d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 --2- 172.21.15.53:0/2723426242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c15d290 0x7f389c071340 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 -- 172.21.15.53:0/2723426242 >> 172.21.15.53:0/2723426242 conn(0x7f389c09ba20 msgr2=0x7f389c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 -- 172.21.15.53:0/2723426242 shutdown_connections 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.575+0000 7f38a22a0640 1 -- 172.21.15.53:0/2723426242 wait complete. 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 Processor -- start 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 -- start start 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c071880 0x7f389c1e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c0d1aa0 0x7f389c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f389c15d290 0x7f389c1e7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0cf9e0 con 0x7f389c071880 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0cf860 con 0x7f389c15d290 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f38a22a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f389c0cfb60 con 0x7f389c0d1aa0 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.577+0000 7f389bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c071880 0x7f389c1e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c071880 0x7f389c1e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50342/0 (socket says 172.21.15.53:50342) 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 -- 172.21.15.53:0/3852591178 learned_addr learned my addr 172.21.15.53:0/3852591178 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f3893fff640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c0d1aa0 0x7f389c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f38a083d640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f389c15d290 0x7f389c1e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 -- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c0d1aa0 msgr2=0x7f389c151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c0d1aa0 0x7f389c151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 -- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f389c15d290 msgr2=0x7f389c1e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f389c15d290 0x7f389c1e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 -- 172.21.15.53:0/3852591178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c15c900 con 0x7f389c071880 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f389bfff640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c071880 0x7f389c1e6790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f388c000c00 tx=0x7f388c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:52.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f38997fa640 1 -- 172.21.15.53:0/3852591178 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c041070 con 0x7f389c071880 2024-05-10T10:34:52.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f38997fa640 1 -- 172.21.15.53:0/3852591178 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c032e50 con 0x7f389c071880 2024-05-10T10:34:52.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f38997fa640 1 -- 172.21.15.53:0/3852591178 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c037040 con 0x7f389c071880 2024-05-10T10:34:52.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c0b7530 con 0x7f389c071880 2024-05-10T10:34:52.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.578+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c0b77f0 con 0x7f389c071880 2024-05-10T10:34:52.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.579+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c071670 con 0x7f389c071880 2024-05-10T10:34:52.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.580+0000 7f38997fa640 1 -- 172.21.15.53:0/3852591178 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f388c044070 con 0x7f389c071880 2024-05-10T10:34:52.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.581+0000 7f38997fa640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f387c0762d0 0x7f387c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:52.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.581+0000 7f38997fa640 1 -- 172.21.15.53:0/3852591178 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f388c0b7c20 con 0x7f389c071880 2024-05-10T10:34:52.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.581+0000 7f3893fff640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f387c0762d0 0x7f387c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:52.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.581+0000 7f3893fff640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f387c0762d0 0x7f387c078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b8ad0 tx=0x7f3884009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:52.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.585+0000 7f38997fa640 1 -- 172.21.15.53:0/3852591178 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f389c071670 con 0x7f389c071880 2024-05-10T10:34:52.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:52.788+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f389c0693f0 con 0x7f389c071880 2024-05-10T10:34:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.836+0000 7f38997fa640 1 -- 172.21.15.53:0/3852591178 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) ==== 97+0+0 (secure 0 0 0) 0x7f388c081130 con 0x7f389c071880 2024-05-10T10:34:53.837 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f387c0762d0 msgr2=0x7f387c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f387c0762d0 0x7f387c078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b8ad0 tx=0x7f3884009190 comp rx=0 tx=0).stop 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c071880 msgr2=0x7f389c1e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c071880 0x7f389c1e6790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f388c000c00 tx=0x7f388c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 shutdown_connections 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f387c0762d0 0x7f387c078790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f389c15d290 0x7f389c1e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f389c0d1aa0 0x7f389c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 --2- 172.21.15.53:0/3852591178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f389c071880 0x7f389c1e6790 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 >> 172.21.15.53:0/3852591178 conn(0x7f389c09ba20 msgr2=0x7f389c0cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:53.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 shutdown_connections 2024-05-10T10:34:53.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.839+0000 7f38a22a0640 1 -- 172.21.15.53:0/3852591178 wait complete. 2024-05-10T10:34:53.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T10:34:53.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.964+0000 7fe1d5037640 1 Processor -- start 2024-05-10T10:34:53.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.964+0000 7fe1d5037640 1 -- start start 2024-05-10T10:34:53.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.964+0000 7fe1d5037640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f3b0 0x7fe1d00d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.964+0000 7fe1d5037640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe1d015f790 0x7fe1d00d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.964+0000 7fe1d5037640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe1d00d5d30 0x7fe1d00c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.964+0000 7fe1d5037640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d00d30b0 con 0x7fe1d015f790 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1d5037640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d00d3390 con 0x7fe1d00d5d30 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1d5037640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00c8c80 con 0x7fe1d015f3b0 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe1d015f790 0x7fe1d00d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe1d015f790 0x7fe1d00d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51452/0 (socket says 172.21.15.53:51452) 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 -- 172.21.15.53:0/1244430829 learned_addr learned my addr 172.21.15.53:0/1244430829 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ced9d640 1 --2- 172.21.15.53:0/1244430829 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f3b0 0x7fe1d00d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1cf59e640 1 --2- 172.21.15.53:0/1244430829 >> v2:172.21.15.135:3300/0 conn(0x7fe1d00d5d30 0x7fe1d00c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 -- 172.21.15.53:0/1244430829 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f3b0 msgr2=0x7fe1d00d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 --2- 172.21.15.53:0/1244430829 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f3b0 0x7fe1d00d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 -- 172.21.15.53:0/1244430829 >> v2:172.21.15.135:3300/0 conn(0x7fe1d00d5d30 msgr2=0x7fe1d00c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 --2- 172.21.15.53:0/1244430829 >> v2:172.21.15.135:3300/0 conn(0x7fe1d00d5d30 0x7fe1d00c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.965+0000 7fe1ce59c640 1 -- 172.21.15.53:0/1244430829 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00c8f60 con 0x7fe1d015f790 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.966+0000 7fe1ce59c640 1 --2- 172.21.15.53:0/1244430829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015f790 0x7fe1d00d3c30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00d36a0 tx=0x7fe1c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31b6f47cfe2c7b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:53.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.966+0000 7fe1cdd9b640 1 -- 172.21.15.53:0/1244430829 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4037070 con 0x7fe1d015f790 2024-05-10T10:34:53.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.966+0000 7fe1cdd9b640 1 -- 172.21.15.53:0/1244430829 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4032440 con 0x7fe1d015f790 2024-05-10T10:34:53.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.966+0000 7fe1cdd9b640 1 -- 172.21.15.53:0/1244430829 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4032810 con 0x7fe1d015f790 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 -- 172.21.15.53:0/1244430829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015f790 msgr2=0x7fe1d00d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 --2- 172.21.15.53:0/1244430829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015f790 0x7fe1d00d3c30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00d36a0 tx=0x7fe1c402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 -- 172.21.15.53:0/1244430829 shutdown_connections 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 --2- 172.21.15.53:0/1244430829 >> v2:172.21.15.135:3300/0 conn(0x7fe1d00d5d30 0x7fe1d00c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 --2- 172.21.15.53:0/1244430829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015f790 0x7fe1d00d3c30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 --2- 172.21.15.53:0/1244430829 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f3b0 0x7fe1d00d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 -- 172.21.15.53:0/1244430829 >> 172.21.15.53:0/1244430829 conn(0x7fe1d00d5130 msgr2=0x7fe1d00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 -- 172.21.15.53:0/1244430829 shutdown_connections 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.967+0000 7fe1d5037640 1 -- 172.21.15.53:0/1244430829 wait complete. 2024-05-10T10:34:53.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.968+0000 7fe1d5037640 1 Processor -- start 2024-05-10T10:34:53.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.968+0000 7fe1d5037640 1 -- start start 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1d5037640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d5d30 0x7fe1d015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1d5037640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f3b0 0x7fe1d01f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1d5037640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d01f80e0 0x7fe1d00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1d5037640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00c9e20 con 0x7fe1d00d5d30 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1d5037640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00c9ca0 con 0x7fe1d015f3b0 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1d5037640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00c9fa0 con 0x7fe1d01f80e0 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d5d30 0x7fe1d015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d5d30 0x7fe1d015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51460/0 (socket says 172.21.15.53:51460) 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 -- 172.21.15.53:0/3641121673 learned_addr learned my addr 172.21.15.53:0/3641121673 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ce59c640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f3b0 0x7fe1d01f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1cf59e640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d01f80e0 0x7fe1d00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 -- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d01f80e0 msgr2=0x7fe1d00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d01f80e0 0x7fe1d00cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 -- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f3b0 msgr2=0x7fe1d01f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f3b0 0x7fe1d01f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.969+0000 7fe1ced9d640 1 -- 172.21.15.53:0/3641121673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00ccef0 con 0x7fe1d00d5d30 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.970+0000 7fe1ced9d640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d5d30 0x7fe1d015b200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000e9b0 tx=0x7fe1c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.970+0000 7fe1b77fe640 1 -- 172.21.15.53:0/3641121673 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0017070 con 0x7fe1d00d5d30 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.970+0000 7fe1b77fe640 1 -- 172.21.15.53:0/3641121673 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c000c9a0 con 0x7fe1d00d5d30 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.970+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d00cd1d0 con 0x7fe1d00d5d30 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.970+0000 7fe1b77fe640 1 -- 172.21.15.53:0/3641121673 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000cd90 con 0x7fe1d00d5d30 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.970+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d00cd790 con 0x7fe1d00d5d30 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.971+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d006a220 con 0x7fe1d00d5d30 2024-05-10T10:34:53.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.972+0000 7fe1b77fe640 1 -- 172.21.15.53:0/3641121673 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe1c001a070 con 0x7fe1d00d5d30 2024-05-10T10:34:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.972+0000 7fe1b77fe640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1bc0762d0 0x7fe1bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.972+0000 7fe1b77fe640 1 -- 172.21.15.53:0/3641121673 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5249+0+0 (secure 0 0 0) 0x7fe1c0097770 con 0x7fe1d00d5d30 2024-05-10T10:34:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.972+0000 7fe1ce59c640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1bc0762d0 0x7fe1bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.973+0000 7fe1ce59c640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1bc0762d0 0x7fe1bc078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4034000 tx=0x7fe1c402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:53.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:53.976+0000 7fe1b77fe640 1 -- 172.21.15.53:0/3641121673 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe1c0061320 con 0x7fe1d00d5d30 2024-05-10T10:34:54.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.248+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe1d01f84c0 con 0x7fe1d00d5d30 2024-05-10T10:34:54.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.248+0000 7fe1b77fe640 1 -- 172.21.15.53:0/3641121673 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7fe1c0060c20 con 0x7fe1d00d5d30 2024-05-10T10:34:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:54.249 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:34:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg==", 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ==", 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2024-05-10T10:34:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA==", 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew==", 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA==", 2024-05-10T10:34:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA==", 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw==", 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA==", 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2024-05-10T10:34:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g==", 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A==", 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.257 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA==", 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg==", 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.259 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg==", 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g==", 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.260 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw==", 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:34:54.261 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw==", 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:34:54.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1mKY2tIhAAzPSpOQLOFy1GEKdl5bPBEw==", 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw==", 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2024-05-10T10:34:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ==", 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:34:54.265 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1bc0762d0 msgr2=0x7fe1bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1bc0762d0 0x7fe1bc078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4034000 tx=0x7fe1c402e000 comp rx=0 tx=0).stop 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d5d30 msgr2=0x7fe1d015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d5d30 0x7fe1d015b200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000e9b0 tx=0x7fe1c000ee70 comp rx=0 tx=0).stop 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 shutdown_connections 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1bc0762d0 0x7fe1bc078790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d01f80e0 0x7fe1d00cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f3b0 0x7fe1d01f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 --2- 172.21.15.53:0/3641121673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d5d30 0x7fe1d015b200 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 >> 172.21.15.53:0/3641121673 conn(0x7fe1d00d5130 msgr2=0x7fe1d00cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 shutdown_connections 2024-05-10T10:34:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.251+0000 7fe1d5037640 1 -- 172.21.15.53:0/3641121673 wait complete. 2024-05-10T10:34:54.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:34:54.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.372+0000 7fab134f8640 1 Processor -- start 2024-05-10T10:34:54.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.372+0000 7fab134f8640 1 -- start start 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab134f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c1608b0 0x7fab0c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab134f8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab0c0d03c0 0x7fab0c0d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab134f8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0c160c90 con 0x7fab0c0d1230 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab134f8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fab0c0ce5f0 con 0x7fab0c0d03c0 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab134f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c0ce8d0 con 0x7fab0c1608b0 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab11a95640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab0c0d03c0 0x7fab0c0d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab10a93640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fab0c0d1230 0x7fab0c0b2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32776/0 (socket says 172.21.15.53:32776) 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab10a93640 1 -- 172.21.15.53:0/2326820695 learned_addr learned my addr 172.21.15.53:0/2326820695 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab11294640 1 --2- 172.21.15.53:0/2326820695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c1608b0 0x7fab0c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab11a95640 1 -- 172.21.15.53:0/2326820695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c1608b0 msgr2=0x7fab0c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab11a95640 1 --2- 172.21.15.53:0/2326820695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c1608b0 0x7fab0c0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab11a95640 1 -- 172.21.15.53:0/2326820695 >> v1:172.21.15.53:6789/0 conn(0x7fab0c0d1230 legacy=0x7fab0c0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab11a95640 1 -- 172.21.15.53:0/2326820695 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c0cebb0 con 0x7fab0c0d03c0 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab03fff640 1 -- 172.21.15.53:0/2326820695 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622647955 0 0) 0x7faafc001110 con 0x7fab0c0d1230 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.373+0000 7fab11294640 1 --2- 172.21.15.53:0/2326820695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c1608b0 0x7fab0c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.374+0000 7fab11a95640 1 --2- 172.21.15.53:0/2326820695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c0d03c0 0x7fab0c0d16b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fab04009d90 tx=0x7fab0402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b743ed8cbc349aef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.374+0000 7fab03fff640 1 -- 172.21.15.53:0/2326820695 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04007740 con 0x7fab0c0d03c0 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.374+0000 7fab03fff640 1 -- 172.21.15.53:0/2326820695 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab040078e0 con 0x7fab0c0d03c0 2024-05-10T10:34:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.374+0000 7fab03fff640 1 -- 172.21.15.53:0/2326820695 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04007cb0 con 0x7fab0c0d03c0 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 -- 172.21.15.53:0/2326820695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c0d03c0 msgr2=0x7fab0c0d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 --2- 172.21.15.53:0/2326820695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c0d03c0 0x7fab0c0d16b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fab04009d90 tx=0x7fab0402a660 comp rx=0 tx=0).stop 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 -- 172.21.15.53:0/2326820695 shutdown_connections 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 --2- 172.21.15.53:0/2326820695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c0d03c0 0x7fab0c0d16b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 --2- 172.21.15.53:0/2326820695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c1608b0 0x7fab0c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 -- 172.21.15.53:0/2326820695 >> 172.21.15.53:0/2326820695 conn(0x7fab0c0d2df0 msgr2=0x7fab0c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 -- 172.21.15.53:0/2326820695 shutdown_connections 2024-05-10T10:34:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.375+0000 7fab134f8640 1 -- 172.21.15.53:0/2326820695 wait complete. 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.376+0000 7fab134f8640 1 Processor -- start 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.376+0000 7fab134f8640 1 -- start start 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab134f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c0d03c0 0x7fab0c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab134f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab0c0d1230 0x7fab0c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab134f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1608b0 0x7fab0c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab134f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0d1f10 con 0x7fab0c1608b0 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab134f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0d1d90 con 0x7fab0c0d1230 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab134f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c0b3010 con 0x7fab0c0d03c0 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1608b0 0x7fab0c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1608b0 0x7fab0c1556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51486/0 (socket says 172.21.15.53:51486) 2024-05-10T10:34:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 -- 172.21.15.53:0/3535421413 learned_addr learned my addr 172.21.15.53:0/3535421413 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab10a93640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab0c0d1230 0x7fab0c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11294640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c0d03c0 0x7fab0c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 -- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c0d03c0 msgr2=0x7fab0c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c0d03c0 0x7fab0c0cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 -- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab0c0d1230 msgr2=0x7fab0c153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab0c0d1230 0x7fab0c153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.377+0000 7fab11a95640 1 -- 172.21.15.53:0/3535421413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c154340 con 0x7fab0c1608b0 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.378+0000 7fab11a95640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1608b0 0x7fab0c1556b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fab04009eb0 tx=0x7fab04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.378+0000 7fab01ffb640 1 -- 172.21.15.53:0/3535421413 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04040070 con 0x7fab0c1608b0 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.378+0000 7fab01ffb640 1 -- 172.21.15.53:0/3535421413 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab04031c90 con 0x7fab0c1608b0 2024-05-10T10:34:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.378+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c154670 con 0x7fab0c1608b0 2024-05-10T10:34:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.378+0000 7fab01ffb640 1 -- 172.21.15.53:0/3535421413 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04045680 con 0x7fab0c1608b0 2024-05-10T10:34:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.378+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c154eb0 con 0x7fab0c1608b0 2024-05-10T10:34:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.379+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c06a220 con 0x7fab0c1608b0 2024-05-10T10:34:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.380+0000 7fab01ffb640 1 -- 172.21.15.53:0/3535421413 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fab04039040 con 0x7fab0c1608b0 2024-05-10T10:34:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.380+0000 7fab01ffb640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faaf00762d0 0x7faaf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.380+0000 7fab01ffb640 1 -- 172.21.15.53:0/3535421413 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5249+0+0 (secure 0 0 0) 0x7fab040b8cd0 con 0x7fab0c1608b0 2024-05-10T10:34:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.380+0000 7fab11294640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faaf00762d0 0x7faaf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.381+0000 7fab11294640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faaf00762d0 0x7faaf0078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0c97d0 tx=0x7fab08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.384+0000 7fab01ffb640 1 -- 172.21.15.53:0/3535421413 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fab0c154670 con 0x7fab0c1608b0 2024-05-10T10:34:54.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.578+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fab0c075140 con 0x7fab0c1608b0 2024-05-10T10:34:54.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.578+0000 7fab01ffb640 1 -- 172.21.15.53:0/3535421413 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) ==== 84+0+65 (secure 0 0 0) 0x7fab04082260 con 0x7fab0c1608b0 2024-05-10T10:34:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:54.580 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:34:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:34:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:34:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:34:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:34:54.580 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:34:54.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faaf00762d0 msgr2=0x7faaf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faaf00762d0 0x7faaf0078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0c97d0 tx=0x7fab08008040 comp rx=0 tx=0).stop 2024-05-10T10:34:54.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1608b0 msgr2=0x7fab0c1556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1608b0 0x7fab0c1556b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fab04009eb0 tx=0x7fab04004a40 comp rx=0 tx=0).stop 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 shutdown_connections 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faaf00762d0 0x7faaf0078790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1608b0 0x7fab0c1556b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab0c0d1230 0x7fab0c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 --2- 172.21.15.53:0/3535421413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab0c0d03c0 0x7fab0c0cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 >> 172.21.15.53:0/3535421413 conn(0x7fab0c0d2df0 msgr2=0x7fab0c0b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 shutdown_connections 2024-05-10T10:34:54.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.580+0000 7fab134f8640 1 -- 172.21.15.53:0/3535421413 wait complete. 2024-05-10T10:34:54.592 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:34:54.592 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:34:54.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.700+0000 7f0696c08640 1 Processor -- start 2024-05-10T10:34:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.700+0000 7f0696c08640 1 -- start start 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f0696c08640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f06900b48a0 0x7f06900b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f0696c08640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f06900d1aa0 0x7f06900d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f0696c08640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06900d0830 con 0x7f0690162990 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f0696c08640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f06900d10d0 con 0x7f06900b48a0 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f0696c08640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f06900d0b10 con 0x7f06900d1aa0 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f068ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f06900b48a0 0x7f06900b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f06949a4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0690162990 0x7f06900d59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32778/0 (socket says 172.21.15.53:32778) 2024-05-10T10:34:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f06949a4640 1 -- 172.21.15.53:0/1829792311 learned_addr learned my addr 172.21.15.53:0/1829792311 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f06951a5640 1 --2- 172.21.15.53:0/1829792311 >> v2:172.21.15.135:3300/0 conn(0x7f06900d1aa0 0x7f06900d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f068ffff640 1 -- 172.21.15.53:0/1829792311 >> v2:172.21.15.135:3300/0 conn(0x7f06900d1aa0 msgr2=0x7f06900d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f068ffff640 1 --2- 172.21.15.53:0/1829792311 >> v2:172.21.15.135:3300/0 conn(0x7f06900d1aa0 0x7f06900d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f068ffff640 1 -- 172.21.15.53:0/1829792311 >> v1:172.21.15.53:6789/0 conn(0x7f0690162990 legacy=0x7f06900d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f068ffff640 1 -- 172.21.15.53:0/1829792311 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06900d0df0 con 0x7f06900b48a0 2024-05-10T10:34:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.701+0000 7f068f7fe640 1 -- 172.21.15.53:0/1829792311 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252141628 0 0) 0x7f06900d0830 con 0x7f0690162990 2024-05-10T10:34:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.702+0000 7f068ffff640 1 --2- 172.21.15.53:0/1829792311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900b48a0 0x7f06900b5f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f069006cd30 tx=0x7f067c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dd997e599d82da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.702+0000 7f068f7fe640 1 -- 172.21.15.53:0/1829792311 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c037070 con 0x7f06900b48a0 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.702+0000 7f068f7fe640 1 -- 172.21.15.53:0/1829792311 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067c032440 con 0x7f06900b48a0 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.702+0000 7f068f7fe640 1 -- 172.21.15.53:0/1829792311 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c032810 con 0x7f06900b48a0 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 -- 172.21.15.53:0/1829792311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900b48a0 msgr2=0x7f06900b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 --2- 172.21.15.53:0/1829792311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900b48a0 0x7f06900b5f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f069006cd30 tx=0x7f067c02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 -- 172.21.15.53:0/1829792311 shutdown_connections 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 --2- 172.21.15.53:0/1829792311 >> v2:172.21.15.135:3300/0 conn(0x7f06900d1aa0 0x7f06900d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 --2- 172.21.15.53:0/1829792311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900b48a0 0x7f06900b5f30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 -- 172.21.15.53:0/1829792311 >> 172.21.15.53:0/1829792311 conn(0x7f069009e3f0 msgr2=0x7f06900d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 -- 172.21.15.53:0/1829792311 shutdown_connections 2024-05-10T10:34:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.703+0000 7f0696c08640 1 -- 172.21.15.53:0/1829792311 wait complete. 2024-05-10T10:34:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.704+0000 7f0696c08640 1 Processor -- start 2024-05-10T10:34:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.704+0000 7f0696c08640 1 -- start start 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f0696c08640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900b48a0 0x7f06901f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f0696c08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900d1aa0 0x7f069015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f0696c08640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0690162990 0x7f06901f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f0696c08640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f06900d3320 con 0x7f06900d1aa0 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f0696c08640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0690162d70 con 0x7f06900b48a0 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f0696c08640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f06900d34a0 con 0x7f0690162990 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900d1aa0 0x7f069015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900d1aa0 0x7f069015c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51516/0 (socket says 172.21.15.53:51516) 2024-05-10T10:34:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 -- 172.21.15.53:0/2948270322 learned_addr learned my addr 172.21.15.53:0/2948270322 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f06951a5640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0690162990 0x7f06901f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f06949a4640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900b48a0 0x7f06901f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 -- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0690162990 msgr2=0x7f06901f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0690162990 0x7f06901f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 -- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900b48a0 msgr2=0x7f06901f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900b48a0 0x7f06901f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 -- 172.21.15.53:0/2948270322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06901f5f70 con 0x7f06900d1aa0 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.705+0000 7f068ffff640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900d1aa0 0x7f069015c300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f067c035040 tx=0x7f067c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.706+0000 7f068d7fa640 1 -- 172.21.15.53:0/2948270322 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c037040 con 0x7f06900d1aa0 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.706+0000 7f068d7fa640 1 -- 172.21.15.53:0/2948270322 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067c004230 con 0x7f06900d1aa0 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.706+0000 7f068d7fa640 1 -- 172.21.15.53:0/2948270322 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c006e10 con 0x7f06900d1aa0 2024-05-10T10:34:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.706+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06901f59b0 con 0x7f06900d1aa0 2024-05-10T10:34:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.706+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069015bb00 con 0x7f06900d1aa0 2024-05-10T10:34:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.707+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069006a220 con 0x7f06900d1aa0 2024-05-10T10:34:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.707+0000 7f068d7fa640 1 -- 172.21.15.53:0/2948270322 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f067c030770 con 0x7f06900d1aa0 2024-05-10T10:34:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.708+0000 7f068d7fa640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06740762d0 0x7f0674078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.708+0000 7f068d7fa640 1 -- 172.21.15.53:0/2948270322 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5249+0+0 (secure 0 0 0) 0x7f067c0b8660 con 0x7f06900d1aa0 2024-05-10T10:34:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.708+0000 7f06949a4640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06740762d0 0x7f0674078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.709+0000 7f06949a4640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06740762d0 0x7f0674078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0684000f60 tx=0x7f0684007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:54.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.712+0000 7f068d7fa640 1 -- 172.21.15.53:0/2948270322 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f067c032e20 con 0x7f06900d1aa0 2024-05-10T10:34:54.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:54.904+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f06900d2710 con 0x7f06900d1aa0 2024-05-10T10:34:55.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.839+0000 7f068d7fa640 1 -- 172.21.15.53:0/2948270322 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) ==== 138+0+0 (secure 0 0 0) 0x7f067c082170 con 0x7f06900d1aa0 2024-05-10T10:34:55.840 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:34:55.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.843+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06740762d0 msgr2=0x7f0674078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:55.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.843+0000 7f0696c08640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06740762d0 0x7f0674078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0684000f60 tx=0x7f0684007c00 comp rx=0 tx=0).stop 2024-05-10T10:34:55.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.843+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900d1aa0 msgr2=0x7f069015c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:55.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.843+0000 7f0696c08640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900d1aa0 0x7f069015c300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f067c035040 tx=0x7f067c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.844+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 shutdown_connections 2024-05-10T10:34:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.844+0000 7f0696c08640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06740762d0 0x7f0674078790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.844+0000 7f0696c08640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0690162990 0x7f06901f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.844+0000 7f0696c08640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06900d1aa0 0x7f069015c300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.844+0000 7f0696c08640 1 --2- 172.21.15.53:0/2948270322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06900b48a0 0x7f06901f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.844+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 >> 172.21.15.53:0/2948270322 conn(0x7f069009e3f0 msgr2=0x7f06900757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.845+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 shutdown_connections 2024-05-10T10:34:55.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.845+0000 7f0696c08640 1 -- 172.21.15.53:0/2948270322 wait complete. 2024-05-10T10:34:55.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:34:55.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.977+0000 7f0a13b40640 1 Processor -- start 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.978+0000 7f0a13b40640 1 -- start start 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.978+0000 7f0a13b40640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c15f410 0x7f0a0c0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.978+0000 7f0a13b40640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0a0c0d0f10 0x7f0a0c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.978+0000 7f0a13b40640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0a0c0b54d0 0x7f0a0c0d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.978+0000 7f0a13b40640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0c0d2800 con 0x7f0a0c0d0f10 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.978+0000 7f0a13b40640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0c0d2ae0 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.978+0000 7f0a13b40640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0c0d01d0 con 0x7f0a0c15f410 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0a0c0d0f10 0x7f0a0c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0a0c0d0f10 0x7f0a0c0b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51526/0 (socket says 172.21.15.53:51526) 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 -- 172.21.15.53:0/2291084893 learned_addr learned my addr 172.21.15.53:0/2291084893 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a120dd640 1 --2- 172.21.15.53:0/2291084893 >> v2:172.21.15.135:3300/0 conn(0x7f0a0c0b54d0 0x7f0a0c0d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a118dc640 1 --2- 172.21.15.53:0/2291084893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c15f410 0x7f0a0c0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 -- 172.21.15.53:0/2291084893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c15f410 msgr2=0x7f0a0c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 --2- 172.21.15.53:0/2291084893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c15f410 0x7f0a0c0d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 -- 172.21.15.53:0/2291084893 >> v2:172.21.15.135:3300/0 conn(0x7f0a0c0b54d0 msgr2=0x7f0a0c0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 --2- 172.21.15.53:0/2291084893 >> v2:172.21.15.135:3300/0 conn(0x7f0a0c0b54d0 0x7f0a0c0d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.979+0000 7f0a110db640 1 -- 172.21.15.53:0/2291084893 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c0d04b0 con 0x7f0a0c0d0f10 2024-05-10T10:34:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a110db640 1 --2- 172.21.15.53:0/2291084893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0d0f10 0x7f0a0c0b4f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c06c840 tx=0x7f0a0002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed1068bbec7a4dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a108da640 1 -- 172.21.15.53:0/2291084893 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00037070 con 0x7f0a0c0d0f10 2024-05-10T10:34:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a108da640 1 -- 172.21.15.53:0/2291084893 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a00032440 con 0x7f0a0c0d0f10 2024-05-10T10:34:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a108da640 1 -- 172.21.15.53:0/2291084893 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00032810 con 0x7f0a0c0d0f10 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a13b40640 1 -- 172.21.15.53:0/2291084893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0d0f10 msgr2=0x7f0a0c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a13b40640 1 --2- 172.21.15.53:0/2291084893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0d0f10 0x7f0a0c0b4f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c06c840 tx=0x7f0a0002a140 comp rx=0 tx=0).stop 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a13b40640 1 -- 172.21.15.53:0/2291084893 shutdown_connections 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a13b40640 1 --2- 172.21.15.53:0/2291084893 >> v2:172.21.15.135:3300/0 conn(0x7f0a0c0b54d0 0x7f0a0c0d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a13b40640 1 --2- 172.21.15.53:0/2291084893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0d0f10 0x7f0a0c0b4f90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a13b40640 1 --2- 172.21.15.53:0/2291084893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c15f410 0x7f0a0c0d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.980+0000 7f0a13b40640 1 -- 172.21.15.53:0/2291084893 >> 172.21.15.53:0/2291084893 conn(0x7f0a0c08ed30 msgr2=0x7f0a0c09e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.981+0000 7f0a13b40640 1 -- 172.21.15.53:0/2291084893 shutdown_connections 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.981+0000 7f0a13b40640 1 -- 172.21.15.53:0/2291084893 wait complete. 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 Processor -- start 2024-05-10T10:34:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 -- start start 2024-05-10T10:34:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0b54d0 0x7f0a0c14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c0d0f10 0x7f0a0c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a0c15f410 0x7f0a0c0d4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c0cffb0 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c0cfe30 con 0x7f0a0c15f410 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.982+0000 7f0a13b40640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0c0d3c10 con 0x7f0a0c0d0f10 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0b54d0 0x7f0a0c14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0b54d0 0x7f0a0c14d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51530/0 (socket says 172.21.15.53:51530) 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 -- 172.21.15.53:0/515884810 learned_addr learned my addr 172.21.15.53:0/515884810 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a120dd640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a0c15f410 0x7f0a0c0d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a110db640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c0d0f10 0x7f0a0c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 -- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c0d0f10 msgr2=0x7f0a0c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c0d0f10 0x7f0a0c0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 -- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a0c15f410 msgr2=0x7f0a0c0d4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a0c15f410 0x7f0a0c0d4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 -- 172.21.15.53:0/515884810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c14dea0 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a110db640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c0d0f10 0x7f0a0c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a118dc640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0b54d0 0x7f0a0c14d960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f09fc007e40 tx=0x7f09fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a0a7fc640 1 -- 172.21.15.53:0/515884810 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc014620 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a0a7fc640 1 -- 172.21.15.53:0/515884810 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09fc019070 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c0ce060 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a0a7fc640 1 -- 172.21.15.53:0/515884810 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc014a00 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.983+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c0ce620 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.984+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0c06a220 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.985+0000 7f0a0a7fc640 1 -- 172.21.15.53:0/515884810 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f09fc014ba0 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.986+0000 7f0a0a7fc640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09f80762d0 0x7f09f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.986+0000 7f0a0a7fc640 1 -- 172.21.15.53:0/515884810 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5619+0+0 (secure 0 0 0) 0x7f09fc098c00 con 0x7f0a0c0b54d0 2024-05-10T10:34:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.986+0000 7f0a110db640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09f80762d0 0x7f09f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:55.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.987+0000 7f0a110db640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09f80762d0 0x7f09f8078790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0a00034000 tx=0x7f0a0002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:55.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:55.990+0000 7f0a0a7fc640 1 -- 172.21.15.53:0/515884810 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0a0c14dea0 con 0x7f0a0c0b54d0 2024-05-10T10:34:56.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.181+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0a0c0d6090 con 0x7f0a0c0b54d0 2024-05-10T10:34:56.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.844+0000 7f0a0a7fc640 1 -- 172.21.15.53:0/515884810 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) ==== 170+0+0 (secure 0 0 0) 0x7f09fc062620 con 0x7f0a0c0b54d0 2024-05-10T10:34:56.845 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:34:56.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.848+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09f80762d0 msgr2=0x7f09f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:56.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.848+0000 7f0a13b40640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09f80762d0 0x7f09f8078790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0a00034000 tx=0x7f0a0002e000 comp rx=0 tx=0).stop 2024-05-10T10:34:56.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.848+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0b54d0 msgr2=0x7f0a0c14d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:56.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.848+0000 7f0a13b40640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0b54d0 0x7f0a0c14d960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f09fc007e40 tx=0x7f09fc00e4b0 comp rx=0 tx=0).stop 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.849+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 shutdown_connections 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.849+0000 7f0a13b40640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f09f80762d0 0x7f09f8078790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.849+0000 7f0a13b40640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a0c15f410 0x7f0a0c0d4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.849+0000 7f0a13b40640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a0c0d0f10 0x7f0a0c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.849+0000 7f0a13b40640 1 --2- 172.21.15.53:0/515884810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a0c0b54d0 0x7f0a0c14d960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.849+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 >> 172.21.15.53:0/515884810 conn(0x7f0a0c08ed30 msgr2=0x7f0a0c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.849+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 shutdown_connections 2024-05-10T10:34:56.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.850+0000 7f0a13b40640 1 -- 172.21.15.53:0/515884810 wait complete. 2024-05-10T10:34:56.883 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.993+0000 7fd9f903f640 1 Processor -- start 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.993+0000 7fd9f903f640 1 -- start start 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.994+0000 7fd9f903f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd9f40cf3e0 0x7fd9f40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.994+0000 7fd9f903f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f406e720 con 0x7fd9f40d2190 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.994+0000 7fd9f903f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f406de80 con 0x7fd9f40cd500 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.994+0000 7fd9f903f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f406e160 con 0x7fd9f40cf3e0 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.994+0000 7fd9f359e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd9f40d2190 0x7fd9f40b34e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32794/0 (socket says 172.21.15.53:32794) 2024-05-10T10:34:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.994+0000 7fd9f359e640 1 -- 172.21.15.53:0/73583624 learned_addr learned my addr 172.21.15.53:0/73583624 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.994+0000 7fd9f2d9d640 1 --2- 172.21.15.53:0/73583624 >> v2:172.21.15.135:3300/0 conn(0x7fd9f40cf3e0 0x7fd9f40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518673764 0 0) 0x7fd9f406e720 con 0x7fd9f40d2190 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9e0003520 con 0x7fd9f40d2190 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2056447320 0 0) 0x7fd9e0003520 con 0x7fd9f40d2190 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 >> v2:172.21.15.135:3300/0 conn(0x7fd9f40cf3e0 msgr2=0x7fd9f40cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 --2- 172.21.15.53:0/73583624 >> v2:172.21.15.135:3300/0 conn(0x7fd9f40cf3e0 0x7fd9f40cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 >> v1:172.21.15.135:6789/0 conn(0x7fd9f40cd500 legacy=0x7fd9f40b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f406e440 con 0x7fd9f40d2190 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fd9ec001670 con 0x7fd9f40d2190 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 >> v1:172.21.15.53:6789/0 conn(0x7fd9f40d2190 legacy=0x7fd9f40b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e00039b0 0x7fd9e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9e0004800 0x7fd9e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9e00051c0 0x7fd9e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f406e160 con 0x7fd9e0004800 2024-05-10T10:34:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e000c060 con 0x7fd9e00039b0 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e000c260 con 0x7fd9e00051c0 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f259c640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9e0004800 0x7fd9e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f2d9d640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e00039b0 0x7fd9e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f359e640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9e00051c0 0x7fd9e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f259c640 1 -- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9e00051c0 msgr2=0x7fd9e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f259c640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9e00051c0 0x7fd9e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f259c640 1 -- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e00039b0 msgr2=0x7fd9e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f259c640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e00039b0 0x7fd9e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f259c640 1 -- 172.21.15.53:0/73583624 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e0003520 con 0x7fd9e0004800 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.995+0000 7fd9f359e640 1 -- 172.21.15.53:0/73583624 reap_dead start 2024-05-10T10:34:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.996+0000 7fd9f259c640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9e0004800 0x7fd9e0004c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc001720 tx=0x7fd9e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f632b45b1cf6406 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.996+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc001370 con 0x7fd9e0004800 2024-05-10T10:34:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.996+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc0052d0 con 0x7fd9e0004800 2024-05-10T10:34:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.996+0000 7fd9f1d9b640 1 -- 172.21.15.53:0/73583624 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc02ab90 con 0x7fd9e0004800 2024-05-10T10:34:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.997+0000 7fd9f903f640 1 -- 172.21.15.53:0/73583624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9e0004800 msgr2=0x7fd9e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.997+0000 7fd9f903f640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9e0004800 0x7fd9e0004c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc001720 tx=0x7fd9e0005830 comp rx=0 tx=0).stop 2024-05-10T10:34:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.997+0000 7fd9f903f640 1 -- 172.21.15.53:0/73583624 shutdown_connections 2024-05-10T10:34:57.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.997+0000 7fd9f903f640 1 --2- 172.21.15.53:0/73583624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9e0004800 0x7fd9e0004c80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.997+0000 7fd9f903f640 1 -- 172.21.15.53:0/73583624 >> 172.21.15.53:0/73583624 conn(0x7fd9f40d2eb0 msgr2=0x7fd9f40d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:57.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.997+0000 7fd9f903f640 1 -- 172.21.15.53:0/73583624 shutdown_connections 2024-05-10T10:34:57.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.997+0000 7fd9f903f640 1 -- 172.21.15.53:0/73583624 wait complete. 2024-05-10T10:34:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.998+0000 7fd9f903f640 1 Processor -- start 2024-05-10T10:34:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.998+0000 7fd9f903f640 1 -- start start 2024-05-10T10:34:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f903f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e0004800 0x7fd9f414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f903f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9f4071140 0x7fd9f40715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f903f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9f415d200 0x7fd9f415d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f903f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40cf8c0 con 0x7fd9f4071140 2024-05-10T10:34:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f903f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40b4510 con 0x7fd9e0004800 2024-05-10T10:34:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f903f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f40cfa40 con 0x7fd9f415d200 2024-05-10T10:34:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9f4071140 0x7fd9f40715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9f4071140 0x7fd9f40715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51538/0 (socket says 172.21.15.53:51538) 2024-05-10T10:34:57.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 -- 172.21.15.53:0/306599623 learned_addr learned my addr 172.21.15.53:0/306599623 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:57.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f2d9d640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e0004800 0x7fd9f414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f359e640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9f415d200 0x7fd9f415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 -- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9f415d200 msgr2=0x7fd9f415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9f415d200 0x7fd9f415d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 -- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e0004800 msgr2=0x7fd9f414c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e0004800 0x7fd9f414c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:56.999+0000 7fd9f259c640 1 -- 172.21.15.53:0/306599623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f414f210 con 0x7fd9f4071140 2024-05-10T10:34:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.000+0000 7fd9f259c640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9f4071140 0x7fd9f40715c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc001840 tx=0x7fd9dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.000+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/306599623 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc03f030 con 0x7fd9f4071140 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.000+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/306599623 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc005480 con 0x7fd9f4071140 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.000+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/306599623 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc037040 con 0x7fd9f4071140 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.000+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f414f4f0 con 0x7fd9f4071140 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.000+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f414fab0 con 0x7fd9f4071140 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.001+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f406a220 con 0x7fd9f4071140 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.002+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/306599623 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd9dc042070 con 0x7fd9f4071140 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.002+0000 7fd9eb7fe640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9d40762d0 0x7fd9d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.003+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/306599623 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6001+0+0 (secure 0 0 0) 0x7fd9dc0b6d20 con 0x7fd9f4071140 2024-05-10T10:34:57.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.003+0000 7fd9f2d9d640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9d40762d0 0x7fd9d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:57.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.003+0000 7fd9f2d9d640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9d40762d0 0x7fd9d4078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009fd0 tx=0x7fd9e400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:57.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.007+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/306599623 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd9dc0804c0 con 0x7fd9f4071140 2024-05-10T10:34:57.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.235+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd9f4070f00 con 0x7fd9f4071140 2024-05-10T10:34:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.846+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/306599623 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7fd9dc07fdc0 con 0x7fd9f4071140 2024-05-10T10:34:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 7 and data pool 8 2024-05-10T10:34:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.850+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9d40762d0 msgr2=0x7fd9d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.850+0000 7fd9f903f640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9d40762d0 0x7fd9d4078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009fd0 tx=0x7fd9e400b040 comp rx=0 tx=0).stop 2024-05-10T10:34:57.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.850+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9f4071140 msgr2=0x7fd9f40715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:57.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.850+0000 7fd9f903f640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9f4071140 0x7fd9f40715c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc001840 tx=0x7fd9dc001120 comp rx=0 tx=0).stop 2024-05-10T10:34:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.852+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 shutdown_connections 2024-05-10T10:34:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.852+0000 7fd9f903f640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9d40762d0 0x7fd9d4078790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.852+0000 7fd9f903f640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9f415d200 0x7fd9f415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.852+0000 7fd9f903f640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9f4071140 0x7fd9f40715c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.852+0000 7fd9f903f640 1 --2- 172.21.15.53:0/306599623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e0004800 0x7fd9f414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.852+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 >> 172.21.15.53:0/306599623 conn(0x7fd9f40d2eb0 msgr2=0x7fd9f40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:57.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.865+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 shutdown_connections 2024-05-10T10:34:57.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:57.866+0000 7fd9f903f640 1 -- 172.21.15.53:0/306599623 wait complete. 2024-05-10T10:34:57.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:58.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.001+0000 7f1fc786b640 1 Processor -- start 2024-05-10T10:34:58.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.001+0000 7f1fc786b640 1 -- start start 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc786b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fc00b8860 0x7f1fc006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc786b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00cfd80 0x7f1fc00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc786b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc015fc90 con 0x7f1fc00d0230 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc786b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc006ee50 con 0x7f1fc00b8860 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc786b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc00b8c40 con 0x7f1fc00cfd80 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc5607640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fc00b8860 0x7f1fc006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc5607640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fc00b8860 0x7f1fc006e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51548/0 (socket says 172.21.15.53:51548) 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc5607640 1 -- 172.21.15.53:0/3761062072 learned_addr learned my addr 172.21.15.53:0/3761062072 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:58.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc5607640 1 -- 172.21.15.53:0/3761062072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00cfd80 msgr2=0x7f1fc00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:58.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc5607640 1 --2- 172.21.15.53:0/3761062072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00cfd80 0x7f1fc00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc5607640 1 -- 172.21.15.53:0/3761062072 >> v1:172.21.15.135:6789/0 conn(0x7f1fc00d0230 legacy=0x7f1fc00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:58.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.002+0000 7f1fc5607640 1 -- 172.21.15.53:0/3761062072 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc00cd570 con 0x7f1fc00b8860 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.003+0000 7f1fc5607640 1 --2- 172.21.15.53:0/3761062072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00b8860 0x7f1fc006e520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009870 tx=0x7f1fbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d55d536a6ec8fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.003+0000 7f1fb3fff640 1 -- 172.21.15.53:0/3761062072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc037070 con 0x7f1fc00b8860 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.003+0000 7f1fb3fff640 1 -- 172.21.15.53:0/3761062072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fbc032440 con 0x7f1fc00b8860 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.003+0000 7f1fb3fff640 1 -- 172.21.15.53:0/3761062072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc032810 con 0x7f1fc00b8860 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 -- 172.21.15.53:0/3761062072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00b8860 msgr2=0x7f1fc006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 --2- 172.21.15.53:0/3761062072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00b8860 0x7f1fc006e520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009870 tx=0x7f1fbc02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 -- 172.21.15.53:0/3761062072 shutdown_connections 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 --2- 172.21.15.53:0/3761062072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00cfd80 0x7f1fc00ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 --2- 172.21.15.53:0/3761062072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00b8860 0x7f1fc006e520 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 -- 172.21.15.53:0/3761062072 >> 172.21.15.53:0/3761062072 conn(0x7f1fc00d2df0 msgr2=0x7f1fc00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 -- 172.21.15.53:0/3761062072 shutdown_connections 2024-05-10T10:34:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.004+0000 7f1fc786b640 1 -- 172.21.15.53:0/3761062072 wait complete. 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.005+0000 7f1fc786b640 1 Processor -- start 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.005+0000 7f1fc786b640 1 -- start start 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc786b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fc00b8860 0x7f1fc01556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc786b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00cfd80 0x7f1fc00c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc786b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00d0230 0x7f1fc00cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc786b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00cde10 con 0x7f1fc00cfd80 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc786b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00cdb30 con 0x7f1fc00b8860 2024-05-10T10:34:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc786b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc006ea60 con 0x7f1fc00d0230 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00cfd80 0x7f1fc00c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00cfd80 0x7f1fc00c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51560/0 (socket says 172.21.15.53:51560) 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 -- 172.21.15.53:0/1378288072 learned_addr learned my addr 172.21.15.53:0/1378288072 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc5607640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fc00b8860 0x7f1fc01556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc5e08640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00d0230 0x7f1fc00cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 -- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00d0230 msgr2=0x7f1fc00cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00d0230 0x7f1fc00cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 -- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fc00b8860 msgr2=0x7f1fc01556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fc00b8860 0x7f1fc01556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 -- 172.21.15.53:0/1378288072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc00c7bf0 con 0x7f1fc00cfd80 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.006+0000 7f1fc4e06640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00cfd80 0x7f1fc00c7330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c440 tx=0x7f1fb400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.007+0000 7f1fb1ffb640 1 -- 172.21.15.53:0/1378288072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4017070 con 0x7f1fc00cfd80 2024-05-10T10:34:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.007+0000 7f1fb1ffb640 1 -- 172.21.15.53:0/1378288072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb40035f0 con 0x7f1fc00cfd80 2024-05-10T10:34:58.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.007+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc00c7ed0 con 0x7f1fc00cfd80 2024-05-10T10:34:58.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.007+0000 7f1fb1ffb640 1 -- 172.21.15.53:0/1378288072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40039e0 con 0x7f1fc00cfd80 2024-05-10T10:34:58.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.007+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc00c7870 con 0x7f1fc00cfd80 2024-05-10T10:34:58.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.008+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc006a220 con 0x7f1fc00cfd80 2024-05-10T10:34:58.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.008+0000 7f1fb1ffb640 1 -- 172.21.15.53:0/1378288072 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1fb401a070 con 0x7f1fc00cfd80 2024-05-10T10:34:58.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.009+0000 7f1fb1ffb640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fa40762d0 0x7f1fa4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.009+0000 7f1fb1ffb640 1 -- 172.21.15.53:0/1378288072 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f1fb4096a90 con 0x7f1fc00cfd80 2024-05-10T10:34:58.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.009+0000 7f1fc5607640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fa40762d0 0x7f1fa4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.010+0000 7f1fc5607640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fa40762d0 0x7f1fa4078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc034000 tx=0x7f1fbc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.013+0000 7f1fb1ffb640 1 -- 172.21.15.53:0/1378288072 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1fb40010c0 con 0x7f1fc00cfd80 2024-05-10T10:34:58.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.209+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fc00cd100 con 0x7f1fc00cfd80 2024-05-10T10:34:58.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.210+0000 7f1fb1ffb640 1 -- 172.21.15.53:0/1378288072 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16955 (secure 0 0 0) 0x7f1fb4096020 con 0x7f1fc00cfd80 2024-05-10T10:34:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:58.211 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":47,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:57.236171+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:54.908375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:56.185211+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fa40762d0 msgr2=0x7f1fa4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fa40762d0 0x7f1fa4078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc034000 tx=0x7f1fbc02e000 comp rx=0 tx=0).stop 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00cfd80 msgr2=0x7f1fc00c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00cfd80 0x7f1fc00c7330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c440 tx=0x7f1fb400c900 comp rx=0 tx=0).stop 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 shutdown_connections 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fa40762d0 0x7f1fa4078790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fc00d0230 0x7f1fc00cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc00cfd80 0x7f1fc00c7330 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 --2- 172.21.15.53:0/1378288072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fc00b8860 0x7f1fc01556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 >> 172.21.15.53:0/1378288072 conn(0x7f1fc00d2df0 msgr2=0x7f1fc00d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 shutdown_connections 2024-05-10T10:34:58.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.212+0000 7f1fc786b640 1 -- 172.21.15.53:0/1378288072 wait complete. 2024-05-10T10:34:58.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:34:58.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.326+0000 7fd0f7fd4640 1 Processor -- start 2024-05-10T10:34:58.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.327+0000 7fd0f7fd4640 1 -- start start 2024-05-10T10:34:58.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f7fd4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0f00d5cb0 0x7fd0f00d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f7fd4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f7fd4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f00b45f0 con 0x7fd0f0162750 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f7fd4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f00b4f20 con 0x7fd0f00d5cb0 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f7fd4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f00d0fe0 con 0x7fd0f0162b30 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f556f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0f00d5cb0 0x7fd0f00d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f5d70640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd0f0162750 0x7fd0f00d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32810/0 (socket says 172.21.15.53:32810) 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f5d70640 1 -- 172.21.15.53:0/3630206016 learned_addr learned my addr 172.21.15.53:0/3630206016 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f6571640 1 --2- 172.21.15.53:0/3630206016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f556f640 1 -- 172.21.15.53:0/3630206016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 msgr2=0x7fd0f00d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f556f640 1 --2- 172.21.15.53:0/3630206016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f556f640 1 -- 172.21.15.53:0/3630206016 >> v1:172.21.15.53:6789/0 conn(0x7fd0f0162750 legacy=0x7fd0f00d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f556f640 1 -- 172.21.15.53:0/3630206016 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f00b48d0 con 0x7fd0f00d5cb0 2024-05-10T10:34:58.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.328+0000 7fd0f4d6e640 1 -- 172.21.15.53:0/3630206016 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3122349474 0 0) 0x7fd0f00b45f0 con 0x7fd0f0162750 2024-05-10T10:34:58.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.329+0000 7fd0f556f640 1 --2- 172.21.15.53:0/3630206016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f00d5cb0 0x7fd0f00d3630 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00097a0 tx=0x7fd0e002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdf1dd564c53c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.329+0000 7fd0f4d6e640 1 -- 172.21.15.53:0/3630206016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0037070 con 0x7fd0f00d5cb0 2024-05-10T10:34:58.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.329+0000 7fd0f4d6e640 1 -- 172.21.15.53:0/3630206016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e002ae50 con 0x7fd0f00d5cb0 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.329+0000 7fd0f4d6e640 1 -- 172.21.15.53:0/3630206016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0032660 con 0x7fd0f00d5cb0 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.329+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/3630206016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f00d5cb0 msgr2=0x7fd0f00d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.329+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/3630206016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f00d5cb0 0x7fd0f00d3630 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00097a0 tx=0x7fd0e002a100 comp rx=0 tx=0).stop 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.330+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/3630206016 shutdown_connections 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.330+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/3630206016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.330+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/3630206016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f00d5cb0 0x7fd0f00d3630 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.330+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/3630206016 >> 172.21.15.53:0/3630206016 conn(0x7fd0f009e3b0 msgr2=0x7fd0f00d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.330+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/3630206016 shutdown_connections 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.330+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/3630206016 wait complete. 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.331+0000 7fd0f7fd4640 1 Processor -- start 2024-05-10T10:34:58.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.331+0000 7fd0f7fd4640 1 -- start start 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.331+0000 7fd0f7fd4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f0162750 0x7fd0f01f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.331+0000 7fd0f7fd4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.331+0000 7fd0f7fd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f009f620 0x7fd0f009fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f7fd4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f00b53b0 con 0x7fd0f009f620 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f7fd4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f00d0d80 con 0x7fd0f0162750 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f7fd4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f00b5530 con 0x7fd0f0162b30 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f009f620 0x7fd0f009fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f009f620 0x7fd0f009fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51582/0 (socket says 172.21.15.53:51582) 2024-05-10T10:34:58.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 -- 172.21.15.53:0/2811454185 learned_addr learned my addr 172.21.15.53:0/2811454185 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:58.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f5d70640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f0162750 0x7fd0f01f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f556f640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 -- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 msgr2=0x7fd0f00cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 -- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f0162750 msgr2=0x7fd0f01f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f0162750 0x7fd0f01f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 -- 172.21.15.53:0/2811454185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f00cca90 con 0x7fd0f009f620 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f5d70640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f0162750 0x7fd0f01f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f6571640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f009f620 0x7fd0f009fad0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8007e60 tx=0x7fd0e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0e67fc640 1 -- 172.21.15.53:0/2811454185 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8014420 con 0x7fd0f009f620 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0e67fc640 1 -- 172.21.15.53:0/2811454185 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e8019070 con 0x7fd0f009f620 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f00ccd70 con 0x7fd0f009f620 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0e67fc640 1 -- 172.21.15.53:0/2811454185 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8014820 con 0x7fd0f009f620 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.332+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f009ee20 con 0x7fd0f009f620 2024-05-10T10:34:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.333+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f006a220 con 0x7fd0f009f620 2024-05-10T10:34:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.334+0000 7fd0e67fc640 1 -- 172.21.15.53:0/2811454185 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd0e80149c0 con 0x7fd0f009f620 2024-05-10T10:34:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.335+0000 7fd0e67fc640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0d4076300 0x7fd0d40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.335+0000 7fd0e67fc640 1 -- 172.21.15.53:0/2811454185 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7fd0e8097f10 con 0x7fd0f009f620 2024-05-10T10:34:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.335+0000 7fd0f5d70640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0d4076300 0x7fd0d40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.335+0000 7fd0f5d70640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0d4076300 0x7fd0d40787c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec000f60 tx=0x7fd0ec007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.339+0000 7fd0e67fc640 1 -- 172.21.15.53:0/2811454185 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd0e8061770 con 0x7fd0f009f620 2024-05-10T10:34:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.530+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0f00d1570 con 0x7fd0f009f620 2024-05-10T10:34:58.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.531+0000 7fd0e67fc640 1 -- 172.21.15.53:0/2811454185 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16955 (secure 0 0 0) 0x7fd0e8061070 con 0x7fd0f009f620 2024-05-10T10:34:58.532 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:34:58.532 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":47,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:57.236171+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:54.908375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:56.185211+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0d4076300 msgr2=0x7fd0d40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0d4076300 0x7fd0d40787c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec000f60 tx=0x7fd0ec007c10 comp rx=0 tx=0).stop 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f009f620 msgr2=0x7fd0f009fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f009f620 0x7fd0f009fad0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8007e60 tx=0x7fd0e800e4b0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 shutdown_connections 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0d4076300 0x7fd0d40787c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0f009f620 0x7fd0f009fad0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f0162b30 0x7fd0f00cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 --2- 172.21.15.53:0/2811454185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f0162750 0x7fd0f01f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 >> 172.21.15.53:0/2811454185 conn(0x7fd0f009e3b0 msgr2=0x7fd0f00d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:58.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 shutdown_connections 2024-05-10T10:34:58.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.533+0000 7fd0f7fd4640 1 -- 172.21.15.53:0/2811454185 wait complete. 2024-05-10T10:34:58.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:34:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.651+0000 7ff4423be640 1 Processor -- start 2024-05-10T10:34:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.651+0000 7ff4423be640 1 -- start start 2024-05-10T10:34:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff4423be640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff43c162900 0x7ff43c0d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff4423be640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162ce0 0x7ff43c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff4423be640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43c0d4190 con 0x7ff43c0d3950 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff4423be640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff43c0cfc40 con 0x7ff43c162900 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff4423be640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff43c0d3eb0 con 0x7ff43c162ce0 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff43b7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162ce0 0x7ff43c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff43bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff43c162900 0x7ff43c0d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff43b7fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162ce0 0x7ff43c0b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:47538/0 (socket says 172.21.15.53:47538) 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff43b7fe640 1 -- 172.21.15.53:0/3503313496 learned_addr learned my addr 172.21.15.53:0/3503313496 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.652+0000 7ff44095b640 1 --1- 172.21.15.53:0/3503313496 >> v1:172.21.15.135:6789/0 conn(0x7ff43c0d3950 0x7ff43c0d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:37976/0 (socket says 172.21.15.53:37976) 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.653+0000 7ff43b7fe640 1 -- 172.21.15.53:0/3503313496 >> v2:172.21.15.135:3300/0 conn(0x7ff43c162900 msgr2=0x7ff43c0d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.653+0000 7ff43b7fe640 1 --2- 172.21.15.53:0/3503313496 >> v2:172.21.15.135:3300/0 conn(0x7ff43c162900 0x7ff43c0d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.653+0000 7ff43b7fe640 1 -- 172.21.15.53:0/3503313496 >> v1:172.21.15.135:6789/0 conn(0x7ff43c0d3950 legacy=0x7ff43c0d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.653+0000 7ff43b7fe640 1 -- 172.21.15.53:0/3503313496 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c0cfee0 con 0x7ff43c162ce0 2024-05-10T10:34:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.653+0000 7ff43affd640 1 -- 172.21.15.53:0/3503313496 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 618544472 0 0) 0x7ff434001490 con 0x7ff43c0d3950 2024-05-10T10:34:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.653+0000 7ff43bfff640 1 --2- 172.21.15.53:0/3503313496 >> v2:172.21.15.135:3300/0 conn(0x7ff43c162900 0x7ff43c0d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:34:58.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff43b7fe640 1 --2- 172.21.15.53:0/3503313496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162ce0 0x7ff43c0b48e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff430009870 tx=0x7ff43002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d068b167d47542 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff43affd640 1 -- 172.21.15.53:0/3503313496 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430037070 con 0x7ff43c162ce0 2024-05-10T10:34:58.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff43affd640 1 -- 172.21.15.53:0/3503313496 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff430032440 con 0x7ff43c162ce0 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff43affd640 1 -- 172.21.15.53:0/3503313496 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430032810 con 0x7ff43c162ce0 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 -- 172.21.15.53:0/3503313496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162ce0 msgr2=0x7ff43c0b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 --2- 172.21.15.53:0/3503313496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162ce0 0x7ff43c0b48e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff430009870 tx=0x7ff43002a1c0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 -- 172.21.15.53:0/3503313496 shutdown_connections 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 --2- 172.21.15.53:0/3503313496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162ce0 0x7ff43c0b48e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 --2- 172.21.15.53:0/3503313496 >> v2:172.21.15.135:3300/0 conn(0x7ff43c162900 0x7ff43c0d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 -- 172.21.15.53:0/3503313496 >> 172.21.15.53:0/3503313496 conn(0x7ff43c09e430 msgr2=0x7ff43c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 -- 172.21.15.53:0/3503313496 shutdown_connections 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.654+0000 7ff4423be640 1 -- 172.21.15.53:0/3503313496 wait complete. 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.656+0000 7ff4423be640 1 Processor -- start 2024-05-10T10:34:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.656+0000 7ff4423be640 1 -- start start 2024-05-10T10:34:58.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.656+0000 7ff4423be640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff43c0d3950 0x7ff43c0cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.656+0000 7ff4423be640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162900 0x7ff43c0c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.656+0000 7ff4423be640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c162ce0 0x7ff43c0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff4423be640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c0b4300 con 0x7ff43c162ce0 2024-05-10T10:34:58.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff4423be640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c0b4180 con 0x7ff43c0d3950 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff4423be640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff43c0d10d0 con 0x7ff43c162900 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c162ce0 0x7ff43c0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c162ce0 0x7ff43c0b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51596/0 (socket says 172.21.15.53:51596) 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 -- 172.21.15.53:0/3352669546 learned_addr learned my addr 172.21.15.53:0/3352669546 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff43bfff640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff43c0d3950 0x7ff43c0cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff43b7fe640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162900 0x7ff43c0c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 -- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162900 msgr2=0x7ff43c0c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162900 0x7ff43c0c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 -- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff43c0d3950 msgr2=0x7ff43c0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff43c0d3950 0x7ff43c0cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 -- 172.21.15.53:0/3352669546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c0cdb40 con 0x7ff43c162ce0 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff44095b640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c162ce0 0x7ff43c0b3050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff43400c7c0 tx=0x7ff43400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff438ff9640 1 -- 172.21.15.53:0/3352669546 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434018070 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff438ff9640 1 -- 172.21.15.53:0/3352669546 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43400a5e0 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c0cde20 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.657+0000 7ff438ff9640 1 -- 172.21.15.53:0/3352669546 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43400a9d0 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.658+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c0ce3e0 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.658+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c06a220 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.659+0000 7ff438ff9640 1 -- 172.21.15.53:0/3352669546 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff43401b070 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.660+0000 7ff438ff9640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff428076300 0x7ff4280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.660+0000 7ff438ff9640 1 -- 172.21.15.53:0/3352669546 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7ff434096d50 con 0x7ff43c162ce0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.660+0000 7ff43bfff640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff428076300 0x7ff4280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:34:58.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.660+0000 7ff43bfff640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff428076300 0x7ff4280787c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0b3ac0 tx=0x7ff42c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:34:58.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.664+0000 7ff438ff9640 1 -- 172.21.15.53:0/3352669546 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff43c0d4190 con 0x7ff43c162ce0 2024-05-10T10:34:58.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:58.895+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff43c0d4e80 con 0x7ff43c162ce0 2024-05-10T10:34:59.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.876+0000 7ff438ff9640 1 -- 172.21.15.53:0/3352669546 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7ff43405fe30 con 0x7ff43c162ce0 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff428076300 msgr2=0x7ff4280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff428076300 0x7ff4280787c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0b3ac0 tx=0x7ff42c009190 comp rx=0 tx=0).stop 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c162ce0 msgr2=0x7ff43c0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c162ce0 0x7ff43c0b3050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff43400c7c0 tx=0x7ff43400cc80 comp rx=0 tx=0).stop 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 shutdown_connections 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff428076300 0x7ff4280787c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff43c162ce0 0x7ff43c0b3050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff43c162900 0x7ff43c0c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 --2- 172.21.15.53:0/3352669546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff43c0d3950 0x7ff43c0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 >> 172.21.15.53:0/3352669546 conn(0x7ff43c09e430 msgr2=0x7ff43c160730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:34:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 shutdown_connections 2024-05-10T10:34:59.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:34:59.879+0000 7ff4423be640 1 -- 172.21.15.53:0/3352669546 wait complete. 2024-05-10T10:34:59.893 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T10:34:59.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:35:00.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.006+0000 7f6d9ee6d640 1 Processor -- start 2024-05-10T10:35:00.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.006+0000 7f6d9ee6d640 1 -- start start 2024-05-10T10:35:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9ee6d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6d980b4f90 0x7f6d980b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9ee6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 0x7f6d980d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9ee6d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d980d2800 con 0x7f6d9815f410 2024-05-10T10:35:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9ee6d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6d980d2ae0 con 0x7f6d980b4f90 2024-05-10T10:35:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9ee6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6d980d01d0 con 0x7f6d980d36d0 2024-05-10T10:35:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9cc09640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6d9815f410 0x7f6d980d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32818/0 (socket says 172.21.15.53:32818) 2024-05-10T10:35:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9cc09640 1 -- 172.21.15.53:0/2416086634 learned_addr learned my addr 172.21.15.53:0/2416086634 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1411162815 0 0) 0x7f6d980d2800 con 0x7f6d9815f410 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d97fff640 1 --2- 172.21.15.53:0/2416086634 >> v2:172.21.15.135:3300/0 conn(0x7f6d980b4f90 0x7f6d980b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d88003520 con 0x7f6d9815f410 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d9d40a640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 0x7f6d980d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 11087765 0 0) 0x7f6d88003520 con 0x7f6d9815f410 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 msgr2=0x7f6d980d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 0x7f6d980d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 >> v2:172.21.15.135:3300/0 conn(0x7f6d980b4f90 msgr2=0x7f6d980b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 --2- 172.21.15.53:0/2416086634 >> v2:172.21.15.135:3300/0 conn(0x7f6d980b4f90 0x7f6d980b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d980d04b0 con 0x7f6d9815f410 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f6d8c001670 con 0x7f6d9815f410 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 >> v1:172.21.15.53:6789/0 conn(0x7f6d9815f410 legacy=0x7f6d980d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d880039b0 0x7f6d88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 0x7f6d88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d980d2ae0 con 0x7f6d88004d40 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d980d01d0 con 0x7f6d880039b0 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.007+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8800b9f0 con 0x7f6d980d36d0 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9d40a640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9cc09640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d880039b0 0x7f6d88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d97fff640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 0x7f6d88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9d40a640 1 -- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 msgr2=0x7f6d88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9d40a640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 0x7f6d88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9d40a640 1 -- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d880039b0 msgr2=0x7f6d88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9d40a640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d880039b0 0x7f6d88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9d40a640 1 -- 172.21.15.53:0/2416086634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d88003520 con 0x7f6d88004d40 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d97fff640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d980d36d0 0x7f6d88004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:00.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.008+0000 7f6d9d40a640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009fc0 tx=0x7f6d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a906b8044c1df7b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90009c10 con 0x7f6d88004d40 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d90036070 con 0x7f6d88004d40 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d977fe640 1 -- 172.21.15.53:0/2416086634 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90031880 con 0x7f6d88004d40 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2416086634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d88004d40 msgr2=0x7f6d880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d9ee6d640 1 --2- 172.21.15.53:0/2416086634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009fc0 tx=0x7f6d88005830 comp rx=0 tx=0).stop 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d9d40a640 1 -- 172.21.15.53:0/2416086634 reap_dead start 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2416086634 shutdown_connections 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2416086634 >> 172.21.15.53:0/2416086634 conn(0x7f6d9808ed30 msgr2=0x7f6d9809e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2416086634 shutdown_connections 2024-05-10T10:35:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.009+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2416086634 wait complete. 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.010+0000 7f6d9ee6d640 1 Processor -- start 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.011+0000 7f6d9ee6d640 1 -- start start 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.011+0000 7f6d9ee6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d88004d40 0x7f6d9814bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.011+0000 7f6d9ee6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d980b31c0 0x7f6d980b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.011+0000 7f6d9ee6d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d980b9a70 0x7f6d980b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.011+0000 7f6d9ee6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d980cffb0 con 0x7f6d980b31c0 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.011+0000 7f6d9ee6d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d980cfe30 con 0x7f6d980b9a70 2024-05-10T10:35:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.011+0000 7f6d9ee6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6d980d10e0 con 0x7f6d88004d40 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d980b31c0 0x7f6d980b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d980b31c0 0x7f6d980b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51616/0 (socket says 172.21.15.53:51616) 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 -- 172.21.15.53:0/2149741500 learned_addr learned my addr 172.21.15.53:0/2149741500 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d9cc09640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d88004d40 0x7f6d9814bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 -- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d88004d40 msgr2=0x7f6d9814bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d9d40a640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d980b9a70 0x7f6d980b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d88004d40 0x7f6d9814bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 -- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d980b9a70 msgr2=0x7f6d980b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d980b9a70 0x7f6d980b9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 -- 172.21.15.53:0/2149741500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d980ba7e0 con 0x7f6d980b31c0 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d97fff640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d980b31c0 0x7f6d980b3640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d840116d0 tx=0x7f6d84011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d957fa640 1 -- 172.21.15.53:0/2149741500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400f9c0 con 0x7f6d980b31c0 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d957fa640 1 -- 172.21.15.53:0/2149741500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8400fb60 con 0x7f6d980b31c0 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d980baac0 con 0x7f6d980b31c0 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d957fa640 1 -- 172.21.15.53:0/2149741500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84015460 con 0x7f6d980b31c0 2024-05-10T10:35:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.012+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d980c92d0 con 0x7f6d980b31c0 2024-05-10T10:35:00.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.013+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9806a220 con 0x7f6d980b31c0 2024-05-10T10:35:00.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.014+0000 7f6d957fa640 1 -- 172.21.15.53:0/2149741500 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6d84015600 con 0x7f6d980b31c0 2024-05-10T10:35:00.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.015+0000 7f6d957fa640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d7c076300 0x7f6d7c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:00.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.015+0000 7f6d957fa640 1 -- 172.21.15.53:0/2149741500 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f6d84098250 con 0x7f6d980b31c0 2024-05-10T10:35:00.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.015+0000 7f6d9cc09640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d7c076300 0x7f6d7c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:00.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.016+0000 7f6d9cc09640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d7c076300 0x7f6d7c0787c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c001050 tx=0x7f6d8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:00.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.019+0000 7f6d957fa640 1 -- 172.21.15.53:0/2149741500 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6d84019030 con 0x7f6d980b31c0 2024-05-10T10:35:00.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.255+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6d9815ebe0 con 0x7f6d980b31c0 2024-05-10T10:35:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.878+0000 7f6d957fa640 1 -- 172.21.15.53:0/2149741500 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f6d84061990 con 0x7f6d980b31c0 2024-05-10T10:35:00.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d7c076300 msgr2=0x7f6d7c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:00.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d7c076300 0x7f6d7c0787c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c001050 tx=0x7f6d8c009040 comp rx=0 tx=0).stop 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d980b31c0 msgr2=0x7f6d980b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d980b31c0 0x7f6d980b3640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d840116d0 tx=0x7f6d84011b90 comp rx=0 tx=0).stop 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 shutdown_connections 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6d7c076300 0x7f6d7c0787c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d980b9a70 0x7f6d980b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d980b31c0 0x7f6d980b3640 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 --2- 172.21.15.53:0/2149741500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6d88004d40 0x7f6d9814bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 >> 172.21.15.53:0/2149741500 conn(0x7f6d9808ed30 msgr2=0x7f6d980b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 shutdown_connections 2024-05-10T10:35:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:00.881+0000 7f6d9ee6d640 1 -- 172.21.15.53:0/2149741500 wait complete. 2024-05-10T10:35:00.896 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:35:00.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:01.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.006+0000 7f4a12245640 1 Processor -- start 2024-05-10T10:35:01.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.006+0000 7f4a12245640 1 -- start start 2024-05-10T10:35:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a12245640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c15f3b0 0x7f4a0c0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a12245640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4a0c15f790 0x7f4a0c0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a12245640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c0d30b0 con 0x7f4a0c0d5d30 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a12245640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0c0d3390 con 0x7f4a0c15f790 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a12245640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c0c8c80 con 0x7f4a0c15f3b0 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4a0c0d5d30 0x7f4a0c0c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32826/0 (socket says 172.21.15.53:32826) 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0bfff640 1 -- 172.21.15.53:0/2362761914 learned_addr learned my addr 172.21.15.53:0/2362761914 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0b7fe640 1 --2- 172.21.15.53:0/2362761914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c15f3b0 0x7f4a0c0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0affd640 1 --2- 172.21.15.53:0/2362761914 >> v2:172.21.15.135:3300/0 conn(0x7f4a0c15f790 0x7f4a0c0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0a7fc640 1 -- 172.21.15.53:0/2362761914 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837120844 0 0) 0x7f4a0c0d30b0 con 0x7f4a0c0d5d30 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0a7fc640 1 -- 172.21.15.53:0/2362761914 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f4003520 con 0x7f4a0c0d5d30 2024-05-10T10:35:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0b7fe640 1 -- 172.21.15.53:0/2362761914 >> v2:172.21.15.135:3300/0 conn(0x7f4a0c15f790 msgr2=0x7f4a0c0d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0b7fe640 1 --2- 172.21.15.53:0/2362761914 >> v2:172.21.15.135:3300/0 conn(0x7f4a0c15f790 0x7f4a0c0d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0b7fe640 1 -- 172.21.15.53:0/2362761914 >> v1:172.21.15.53:6789/0 conn(0x7f4a0c0d5d30 legacy=0x7f4a0c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.007+0000 7f4a0b7fe640 1 -- 172.21.15.53:0/2362761914 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c0c8f60 con 0x7f4a0c15f3b0 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.008+0000 7f4a0affd640 1 --2- 172.21.15.53:0/2362761914 >> v2:172.21.15.135:3300/0 conn(0x7f4a0c15f790 0x7f4a0c0d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.008+0000 7f4a0b7fe640 1 --2- 172.21.15.53:0/2362761914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c15f3b0 0x7f4a0c0d36f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097a0 tx=0x7f49fc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173bc5ad4d6f4623 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.008+0000 7f4a0a7fc640 1 -- 172.21.15.53:0/2362761914 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc037070 con 0x7f4a0c15f3b0 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a0a7fc640 1 -- 172.21.15.53:0/2362761914 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc032440 con 0x7f4a0c15f3b0 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a0a7fc640 1 -- 172.21.15.53:0/2362761914 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0327f0 con 0x7f4a0c15f3b0 2024-05-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 -- 172.21.15.53:0/2362761914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c15f3b0 msgr2=0x7f4a0c0d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 --2- 172.21.15.53:0/2362761914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c15f3b0 0x7f4a0c0d36f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097a0 tx=0x7f49fc02a180 comp rx=0 tx=0).stop 2024-05-10T10:35:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 -- 172.21.15.53:0/2362761914 shutdown_connections 2024-05-10T10:35:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 --2- 172.21.15.53:0/2362761914 >> v2:172.21.15.135:3300/0 conn(0x7f4a0c15f790 0x7f4a0c0d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 --2- 172.21.15.53:0/2362761914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c15f3b0 0x7f4a0c0d36f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 -- 172.21.15.53:0/2362761914 >> 172.21.15.53:0/2362761914 conn(0x7f4a0c0d5130 msgr2=0x7f4a0c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 -- 172.21.15.53:0/2362761914 shutdown_connections 2024-05-10T10:35:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.009+0000 7f4a12245640 1 -- 172.21.15.53:0/2362761914 wait complete. 2024-05-10T10:35:01.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.010+0000 7f4a12245640 1 Processor -- start 2024-05-10T10:35:01.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a12245640 1 -- start start 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a12245640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c0d5d30 0x7f4a0c1e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a12245640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0c15f3b0 0x7f4a0c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a12245640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c1571a0 0x7f4a0c158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a12245640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d0e60 con 0x7f4a0c1571a0 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a12245640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d0ce0 con 0x7f4a0c15f3b0 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a12245640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d0fe0 con 0x7f4a0c0d5d30 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c1571a0 0x7f4a0c158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c1571a0 0x7f4a0c158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51620/0 (socket says 172.21.15.53:51620) 2024-05-10T10:35:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.011+0000 7f4a0bfff640 1 -- 172.21.15.53:0/1641732914 learned_addr learned my addr 172.21.15.53:0/1641732914 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0b7fe640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c0d5d30 0x7f4a0c1e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0affd640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0c15f3b0 0x7f4a0c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0bfff640 1 -- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c0d5d30 msgr2=0x7f4a0c1e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0bfff640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c0d5d30 0x7f4a0c1e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0bfff640 1 -- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0c15f3b0 msgr2=0x7f4a0c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0bfff640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0c15f3b0 0x7f4a0c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0bfff640 1 -- 172.21.15.53:0/1641732914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c160b90 con 0x7f4a0c1571a0 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0affd640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0c15f3b0 0x7f4a0c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:01.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0b7fe640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c0d5d30 0x7f4a0c1e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a0bfff640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c1571a0 0x7f4a0c158cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400cbb0 tx=0x7f4a0400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f49f3fff640 1 -- 172.21.15.53:0/1641732914 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04017070 con 0x7f4a0c1571a0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f49f3fff640 1 -- 172.21.15.53:0/1641732914 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0400ad40 con 0x7f4a0c1571a0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c160e70 con 0x7f4a0c1571a0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f49f3fff640 1 -- 172.21.15.53:0/1641732914 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04012600 con 0x7f4a0c1571a0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.012+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c161430 con 0x7f4a0c1571a0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.013+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c06a220 con 0x7f4a0c1571a0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.014+0000 7f49f3fff640 1 -- 172.21.15.53:0/1641732914 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a0401a070 con 0x7f4a0c1571a0 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.014+0000 7f49f3fff640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f80762d0 0x7f49f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.014+0000 7f49f3fff640 1 -- 172.21.15.53:0/1641732914 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f4a04097450 con 0x7f4a0c1571a0 2024-05-10T10:35:01.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.015+0000 7f4a0b7fe640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f80762d0 0x7f49f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.015+0000 7f4a0b7fe640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f80762d0 0x7f49f8078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49fc034000 tx=0x7f49fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.019+0000 7f49f3fff640 1 -- 172.21.15.53:0/1641732914 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4a040605c0 con 0x7f4a0c1571a0 2024-05-10T10:35:01.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.249+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a0c0caef0 con 0x7f4a0c1571a0 2024-05-10T10:35:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.250+0000 7f49f3fff640 1 -- 172.21.15.53:0/1641732914 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7f4a040648c0 con 0x7f4a0c1571a0 2024-05-10T10:35:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:01.251 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f80762d0 msgr2=0x7f49f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f80762d0 0x7f49f8078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49fc034000 tx=0x7f49fc02e000 comp rx=0 tx=0).stop 2024-05-10T10:35:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c1571a0 msgr2=0x7f4a0c158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c1571a0 0x7f4a0c158cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400cbb0 tx=0x7f4a0400a430 comp rx=0 tx=0).stop 2024-05-10T10:35:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 shutdown_connections 2024-05-10T10:35:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49f80762d0 0x7f49f8078790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a0c1571a0 0x7f4a0c158cc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a0c15f3b0 0x7f4a0c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 --2- 172.21.15.53:0/1641732914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a0c0d5d30 0x7f4a0c1e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 >> 172.21.15.53:0/1641732914 conn(0x7f4a0c0d5130 msgr2=0x7f4a0c06e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 shutdown_connections 2024-05-10T10:35:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.252+0000 7f4a12245640 1 -- 172.21.15.53:0/1641732914 wait complete. 2024-05-10T10:35:01.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:35:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.369+0000 7fc315c15640 1 Processor -- start 2024-05-10T10:35:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.369+0000 7fc315c15640 1 -- start start 2024-05-10T10:35:01.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.369+0000 7fc315c15640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d5d30 0x7fc31016a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.369+0000 7fc315c15640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3100d30b0 con 0x7fc31015f3b0 2024-05-10T10:35:01.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.369+0000 7fc315c15640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3100d3390 con 0x7fc31015f790 2024-05-10T10:35:01.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.369+0000 7fc315c15640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc3100c8c80 con 0x7fc3100d5d30 2024-05-10T10:35:01.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc31015f3b0 0x7fc3100d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32830/0 (socket says 172.21.15.53:32830) 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30f7fe640 1 -- 172.21.15.53:0/3211060349 learned_addr learned my addr 172.21.15.53:0/3211060349 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30ffff640 1 --2- 172.21.15.53:0/3211060349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d5d30 0x7fc31016a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30e7fc640 1 -- 172.21.15.53:0/3211060349 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 925459350 0 0) 0x7fc3100d30b0 con 0x7fc31015f3b0 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30e7fc640 1 -- 172.21.15.53:0/3211060349 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2f8003520 con 0x7fc31015f3b0 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30e7fc640 1 -- 172.21.15.53:0/3211060349 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663150118 0 0) 0x7fc3100d3390 con 0x7fc31015f790 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30ffff640 1 -- 172.21.15.53:0/3211060349 >> v1:172.21.15.135:6789/0 conn(0x7fc31015f790 legacy=0x7fc3100cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30ffff640 1 -- 172.21.15.53:0/3211060349 >> v1:172.21.15.53:6789/0 conn(0x7fc31015f3b0 legacy=0x7fc3100d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.370+0000 7fc30ffff640 1 -- 172.21.15.53:0/3211060349 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3100c8f60 con 0x7fc3100d5d30 2024-05-10T10:35:01.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.371+0000 7fc30ffff640 1 --2- 172.21.15.53:0/3211060349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d5d30 0x7fc31016a0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc308009dc0 tx=0x7fc30802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd295aea17ba815 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.371+0000 7fc30e7fc640 1 -- 172.21.15.53:0/3211060349 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308035020 con 0x7fc3100d5d30 2024-05-10T10:35:01.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.371+0000 7fc30e7fc640 1 -- 172.21.15.53:0/3211060349 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc308007540 con 0x7fc3100d5d30 2024-05-10T10:35:01.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.371+0000 7fc30e7fc640 1 -- 172.21.15.53:0/3211060349 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300001670 con 0x7fc3100d5d30 2024-05-10T10:35:01.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.372+0000 7fc315c15640 1 -- 172.21.15.53:0/3211060349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d5d30 msgr2=0x7fc31016a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.372+0000 7fc315c15640 1 --2- 172.21.15.53:0/3211060349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d5d30 0x7fc31016a0c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc308009dc0 tx=0x7fc30802a640 comp rx=0 tx=0).stop 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.372+0000 7fc315c15640 1 -- 172.21.15.53:0/3211060349 shutdown_connections 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.372+0000 7fc315c15640 1 --2- 172.21.15.53:0/3211060349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d5d30 0x7fc31016a0c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.372+0000 7fc315c15640 1 -- 172.21.15.53:0/3211060349 >> 172.21.15.53:0/3211060349 conn(0x7fc3100d5130 msgr2=0x7fc3100d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.372+0000 7fc315c15640 1 -- 172.21.15.53:0/3211060349 shutdown_connections 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.372+0000 7fc315c15640 1 -- 172.21.15.53:0/3211060349 wait complete. 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.373+0000 7fc315c15640 1 Processor -- start 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.373+0000 7fc315c15640 1 -- start start 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc315c15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3100d5d30 0x7fc310150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc315c15640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc31015f3b0 0x7fc3100d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc315c15640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d1a50 0x7fc3100cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc315c15640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc3100d3f80 con 0x7fc3100d5d30 2024-05-10T10:35:01.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc315c15640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc3100d3e00 con 0x7fc31015f3b0 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc315c15640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc3100ca770 con 0x7fc3100d1a50 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3100d5d30 0x7fc310150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3100d5d30 0x7fc310150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51622/0 (socket says 172.21.15.53:51622) 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 -- 172.21.15.53:0/1082666095 learned_addr learned my addr 172.21.15.53:0/1082666095 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30ffff640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d1a50 0x7fc3100cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30effd640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc31015f3b0 0x7fc3100d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 -- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d1a50 msgr2=0x7fc3100cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d1a50 0x7fc3100cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 -- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc31015f3b0 msgr2=0x7fc3100d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc31015f3b0 0x7fc3100d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30f7fe640 1 -- 172.21.15.53:0/1082666095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3100cd2b0 con 0x7fc3100d5d30 2024-05-10T10:35:01.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30ffff640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d1a50 0x7fc3100cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.374+0000 7fc30effd640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc31015f3b0 0x7fc3100d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.375+0000 7fc30f7fe640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3100d5d30 0x7fc310150660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c790 tx=0x7fc30000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.375+0000 7fc2f7fff640 1 -- 172.21.15.53:0/1082666095 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300018070 con 0x7fc3100d5d30 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.375+0000 7fc2f7fff640 1 -- 172.21.15.53:0/1082666095 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3000074e0 con 0x7fc3100d5d30 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.375+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3100cd590 con 0x7fc3100d5d30 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.375+0000 7fc2f7fff640 1 -- 172.21.15.53:0/1082666095 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300004a00 con 0x7fc3100d5d30 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.375+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc310150ea0 con 0x7fc3100d5d30 2024-05-10T10:35:01.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.376+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31006a220 con 0x7fc3100d5d30 2024-05-10T10:35:01.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.376+0000 7fc2f7fff640 1 -- 172.21.15.53:0/1082666095 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc30001b070 con 0x7fc3100d5d30 2024-05-10T10:35:01.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.377+0000 7fc2f7fff640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2fc0762d0 0x7fc2fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.377+0000 7fc2f7fff640 1 -- 172.21.15.53:0/1082666095 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fc300096e90 con 0x7fc3100d5d30 2024-05-10T10:35:01.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.377+0000 7fc30effd640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2fc0762d0 0x7fc2fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.378+0000 7fc30effd640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2fc0762d0 0x7fc2fc078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc3101f7e40 tx=0x7fc304009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.381+0000 7fc2f7fff640 1 -- 172.21.15.53:0/1082666095 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc300060530 con 0x7fc3100d5d30 2024-05-10T10:35:01.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.576+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc310075ca0 con 0x7fc3100d5d30 2024-05-10T10:35:01.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.577+0000 7fc2f7fff640 1 -- 172.21.15.53:0/1082666095 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16942 (secure 0 0 0) 0x7fc300098090 con 0x7fc3100d5d30 2024-05-10T10:35:01.578 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:01.578 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":48,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:34:58.861358+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:54.908375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:56.185211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:35:01.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2fc0762d0 msgr2=0x7fc2fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2fc0762d0 0x7fc2fc078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc3101f7e40 tx=0x7fc304009040 comp rx=0 tx=0).stop 2024-05-10T10:35:01.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3100d5d30 msgr2=0x7fc310150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3100d5d30 0x7fc310150660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c790 tx=0x7fc30000cc50 comp rx=0 tx=0).stop 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 shutdown_connections 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2fc0762d0 0x7fc2fc078790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3100d1a50 0x7fc3100cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc31015f3b0 0x7fc3100d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 --2- 172.21.15.53:0/1082666095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3100d5d30 0x7fc310150660 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.579+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 >> 172.21.15.53:0/1082666095 conn(0x7fc3100d5130 msgr2=0x7fc31006d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.580+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 shutdown_connections 2024-05-10T10:35:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.580+0000 7fc315c15640 1 -- 172.21.15.53:0/1082666095 wait complete. 2024-05-10T10:35:01.592 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:35:01.592 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:35:01.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.695+0000 7ff54baa0640 1 Processor -- start 2024-05-10T10:35:01.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.695+0000 7ff54baa0640 1 -- start start 2024-05-10T10:35:01.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54baa0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5440d5cb0 0x7ff5440d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54baa0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 0x7ff5440d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54baa0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5440b45f0 con 0x7ff544162750 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54baa0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff5440b4f20 con 0x7ff5440d5cb0 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54baa0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff5440d0fe0 con 0x7ff544162b30 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54983c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff544162750 0x7ff5440d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32836/0 (socket says 172.21.15.53:32836) 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54983c640 1 -- 172.21.15.53:0/3187779121 learned_addr learned my addr 172.21.15.53:0/3187779121 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54a03d640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 0x7ff5440d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600448689 0 0) 0x7ff5440b45f0 con 0x7ff544162750 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54903b640 1 --2- 172.21.15.53:0/3187779121 >> v2:172.21.15.135:3300/0 conn(0x7ff5440d5cb0 0x7ff5440d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.696+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff530003520 con 0x7ff544162750 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 636071349 0 0) 0x7ff530003520 con 0x7ff544162750 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 msgr2=0x7ff5440d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:01.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 0x7ff5440d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 >> v2:172.21.15.135:3300/0 conn(0x7ff5440d5cb0 msgr2=0x7ff5440d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 --2- 172.21.15.53:0/3187779121 >> v2:172.21.15.135:3300/0 conn(0x7ff5440d5cb0 0x7ff5440d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5440b48d0 con 0x7ff544162750 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7ff540001670 con 0x7ff544162750 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54a03d640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 0x7ff5440d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 >> v1:172.21.15.53:6789/0 conn(0x7ff544162750 legacy=0x7ff5440d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 0x7ff5300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff530003fa0 0x7ff530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff530004d40 0x7ff5300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440b4f20 con 0x7ff530003fa0 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440d0fe0 con 0x7ff530004d40 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff53000bb20 con 0x7ff544162b30 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54903b640 1 --2- 172.21.15.53:0/3187779121 >> v2:172.21.15.135:3300/0 conn(0x7ff5440d5cb0 0x7ff5440d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54903b640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff530003fa0 0x7ff530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54a03d640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff530004d40 0x7ff5300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54983c640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 0x7ff5300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54903b640 1 -- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 msgr2=0x7ff5300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54903b640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff544162b30 0x7ff5300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54903b640 1 -- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff530004d40 msgr2=0x7ff5300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54903b640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff530004d40 0x7ff5300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.697+0000 7ff54903b640 1 -- 172.21.15.53:0/3187779121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff530003520 con 0x7ff530003fa0 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.698+0000 7ff54903b640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff530003fa0 0x7ff530004800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff5440d2860 tx=0x7ff530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94d79c135b7084b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.698+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534037070 con 0x7ff530003fa0 2024-05-10T10:35:01.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.698+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53402ad00 con 0x7ff530003fa0 2024-05-10T10:35:01.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.698+0000 7ff54883a640 1 -- 172.21.15.53:0/3187779121 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53400f040 con 0x7ff530003fa0 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.699+0000 7ff54baa0640 1 -- 172.21.15.53:0/3187779121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff530003fa0 msgr2=0x7ff530004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.699+0000 7ff54baa0640 1 --2- 172.21.15.53:0/3187779121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff530003fa0 0x7ff530004800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff5440d2860 tx=0x7ff530005830 comp rx=0 tx=0).stop 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.699+0000 7ff54a03d640 1 -- 172.21.15.53:0/3187779121 reap_dead start 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.699+0000 7ff54baa0640 1 -- 172.21.15.53:0/3187779121 shutdown_connections 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.699+0000 7ff54baa0640 1 -- 172.21.15.53:0/3187779121 >> 172.21.15.53:0/3187779121 conn(0x7ff54409e3b0 msgr2=0x7ff5440d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.699+0000 7ff54baa0640 1 -- 172.21.15.53:0/3187779121 shutdown_connections 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.699+0000 7ff54baa0640 1 -- 172.21.15.53:0/3187779121 wait complete. 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.700+0000 7ff54baa0640 1 Processor -- start 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.700+0000 7ff54baa0640 1 -- start start 2024-05-10T10:35:01.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54baa0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff530003fa0 0x7ff5441508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54baa0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5440ccac0 0x7ff54406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54baa0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5440ccf40 0x7ff544152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54baa0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440d3b70 con 0x7ff5440ccac0 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54baa0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440d0d80 con 0x7ff5440ccf40 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54baa0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff5440d3cf0 con 0x7ff530003fa0 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5440ccac0 0x7ff54406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5440ccac0 0x7ff54406e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51642/0 (socket says 172.21.15.53:51642) 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 -- 172.21.15.53:0/1166059667 learned_addr learned my addr 172.21.15.53:0/1166059667 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54983c640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff530003fa0 0x7ff5441508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54a03d640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5440ccf40 0x7ff544152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 -- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff530003fa0 msgr2=0x7ff5441508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff530003fa0 0x7ff5441508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 -- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5440ccf40 msgr2=0x7ff544152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5440ccf40 0x7ff544152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 -- 172.21.15.53:0/1166059667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5441531e0 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff54903b640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5440ccac0 0x7ff54406e0c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff534009960 tx=0x7ff534004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.701+0000 7ff53a7fc640 1 -- 172.21.15.53:0/1166059667 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534040020 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.702+0000 7ff53a7fc640 1 -- 172.21.15.53:0/1166059667 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534038040 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.702+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5441534c0 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.702+0000 7ff53a7fc640 1 -- 172.21.15.53:0/1166059667 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534037070 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.702+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff544153d60 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.703+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5441538b0 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.703+0000 7ff53a7fc640 1 -- 172.21.15.53:0/1166059667 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff534043020 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.704+0000 7ff53a7fc640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5280762d0 0x7ff528078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.704+0000 7ff53a7fc640 1 -- 172.21.15.53:0/1166059667 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7ff5340b75b0 con 0x7ff5440ccac0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.704+0000 7ff54983c640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5280762d0 0x7ff528078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:01.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.705+0000 7ff54983c640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5280762d0 0x7ff528078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff540001630 tx=0x7ff5400078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:01.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.709+0000 7ff53a7fc640 1 -- 172.21.15.53:0/1166059667 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff53404a030 con 0x7ff5440ccac0 2024-05-10T10:35:01.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.896+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff54406a220 con 0x7ff5280762d0 2024-05-10T10:35:01.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.903+0000 7ff53a7fc640 1 -- 172.21.15.53:0/1166059667 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251875 (secure 0 0 0) 0x7ff54406a220 con 0x7ff5280762d0 2024-05-10T10:35:01.906 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:01.908 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:35:01.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5280762d0 msgr2=0x7ff528078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5280762d0 0x7ff528078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff540001630 tx=0x7ff5400078f0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5440ccac0 msgr2=0x7ff54406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5440ccac0 0x7ff54406e0c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff534009960 tx=0x7ff534004730 comp rx=0 tx=0).stop 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 shutdown_connections 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5280762d0 0x7ff528078790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5440ccf40 0x7ff544152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5440ccac0 0x7ff54406e0c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 --2- 172.21.15.53:0/1166059667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff530003fa0 0x7ff5441508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.906+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 >> 172.21.15.53:0/1166059667 conn(0x7ff54409e3b0 msgr2=0x7ff5440b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.907+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 shutdown_connections 2024-05-10T10:35:01.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:01.907+0000 7ff54baa0640 1 -- 172.21.15.53:0/1166059667 wait complete. 2024-05-10T10:35:01.918 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-05-10T10:35:00.175444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238288,"kb_used_data":5468,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736432,"statfs":{"total":773094113280,"available":772850106368,"internally_reserved":0,"allocated":5599232,"data_stored":3079821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.334903"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873310+0000","last_change":"2024-05-10T10:34:57.871752+0000","last_active":"2024-05-10T10:34:57.873310+0000","last_peered":"2024-05-10T10:34:57.873310+0000","last_clean":"2024-05-10T10:34:57.873310+0000","last_became_active":"2024-05-10T10:34:57.871233+0000","last_became_peered":"2024-05-10T10:34:57.871233+0000","last_unstale":"2024-05-10T10:34:57.873310+0000","last_undegraded":"2024-05-10T10:34:57.873310+0000","last_fullsized":"2024-05-10T10:34:57.873310+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:51.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877036+0000","last_change":"2024-05-10T10:34:57.871584+0000","last_active":"2024-05-10T10:34:57.877036+0000","last_peered":"2024-05-10T10:34:57.877036+0000","last_clean":"2024-05-10T10:34:57.877036+0000","last_became_active":"2024-05-10T10:34:57.871310+0000","last_became_peered":"2024-05-10T10:34:57.871310+0000","last_unstale":"2024-05-10T10:34:57.877036+0000","last_undegraded":"2024-05-10T10:34:57.877036+0000","last_fullsized":"2024-05-10T10:34:57.877036+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:22.457663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882202+0000","last_change":"2024-05-10T10:34:57.877770+0000","last_active":"2024-05-10T10:34:57.882202+0000","last_peered":"2024-05-10T10:34:57.882202+0000","last_clean":"2024-05-10T10:34:57.882202+0000","last_became_active":"2024-05-10T10:34:57.877470+0000","last_became_peered":"2024-05-10T10:34:57.877470+0000","last_unstale":"2024-05-10T10:34:57.882202+0000","last_undegraded":"2024-05-10T10:34:57.882202+0000","last_fullsized":"2024-05-10T10:34:57.882202+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:51:39.244051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879883+0000","last_change":"2024-05-10T10:34:57.873074+0000","last_active":"2024-05-10T10:34:57.879883+0000","last_peered":"2024-05-10T10:34:57.879883+0000","last_clean":"2024-05-10T10:34:57.879883+0000","last_became_active":"2024-05-10T10:34:57.872825+0000","last_became_peered":"2024-05-10T10:34:57.872825+0000","last_unstale":"2024-05-10T10:34:57.879883+0000","last_undegraded":"2024-05-10T10:34:57.879883+0000","last_fullsized":"2024-05-10T10:34:57.879883+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877024+0000","last_change":"2024-05-10T10:34:57.868765+0000","last_active":"2024-05-10T10:34:57.877024+0000","last_peered":"2024-05-10T10:34:57.877024+0000","last_clean":"2024-05-10T10:34:57.877024+0000","last_became_active":"2024-05-10T10:34:57.868456+0000","last_became_peered":"2024-05-10T10:34:57.868456+0000","last_unstale":"2024-05-10T10:34:57.877024+0000","last_undegraded":"2024-05-10T10:34:57.877024+0000","last_fullsized":"2024-05-10T10:34:57.877024+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:37.953833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.863585+0000","last_change":"2024-05-10T10:34:57.862747+0000","last_active":"2024-05-10T10:34:57.863585+0000","last_peered":"2024-05-10T10:34:57.863585+0000","last_clean":"2024-05-10T10:34:57.863585+0000","last_became_active":"2024-05-10T10:34:57.862465+0000","last_became_peered":"2024-05-10T10:34:57.862465+0000","last_unstale":"2024-05-10T10:34:57.863585+0000","last_undegraded":"2024-05-10T10:34:57.863585+0000","last_fullsized":"2024-05-10T10:34:57.863585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:08.508199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882162+0000","last_change":"2024-05-10T10:34:57.877152+0000","last_active":"2024-05-10T10:34:57.882162+0000","last_peered":"2024-05-10T10:34:57.882162+0000","last_clean":"2024-05-10T10:34:57.882162+0000","last_became_active":"2024-05-10T10:34:57.876804+0000","last_became_peered":"2024-05-10T10:34:57.876804+0000","last_unstale":"2024-05-10T10:34:57.882162+0000","last_undegraded":"2024-05-10T10:34:57.882162+0000","last_fullsized":"2024-05-10T10:34:57.882162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:18.842628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873937+0000","last_change":"2024-05-10T10:34:57.864536+0000","last_active":"2024-05-10T10:34:57.873937+0000","last_peered":"2024-05-10T10:34:57.873937+0000","last_clean":"2024-05-10T10:34:57.873937+0000","last_became_active":"2024-05-10T10:34:57.864285+0000","last_became_peered":"2024-05-10T10:34:57.864285+0000","last_unstale":"2024-05-10T10:34:57.873937+0000","last_undegraded":"2024-05-10T10:34:57.873937+0000","last_fullsized":"2024-05-10T10:34:57.873937+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:34:11.896217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872326+0000","last_change":"2024-05-10T10:34:57.868059+0000","last_active":"2024-05-10T10:34:57.872326+0000","last_peered":"2024-05-10T10:34:57.872326+0000","last_clean":"2024-05-10T10:34:57.872326+0000","last_became_active":"2024-05-10T10:34:57.867717+0000","last_became_peered":"2024-05-10T10:34:57.867717+0000","last_unstale":"2024-05-10T10:34:57.872326+0000","last_undegraded":"2024-05-10T10:34:57.872326+0000","last_fullsized":"2024-05-10T10:34:57.872326+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:05:27.162854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884324+0000","last_change":"2024-05-10T10:34:57.882295+0000","last_active":"2024-05-10T10:34:57.884324+0000","last_peered":"2024-05-10T10:34:57.884324+0000","last_clean":"2024-05-10T10:34:57.884324+0000","last_became_active":"2024-05-10T10:34:57.881995+0000","last_became_peered":"2024-05-10T10:34:57.881995+0000","last_unstale":"2024-05-10T10:34:57.884324+0000","last_undegraded":"2024-05-10T10:34:57.884324+0000","last_fullsized":"2024-05-10T10:34:57.884324+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.212205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873683+0000","last_change":"2024-05-10T10:34:57.866921+0000","last_active":"2024-05-10T10:34:57.873683+0000","last_peered":"2024-05-10T10:34:57.873683+0000","last_clean":"2024-05-10T10:34:57.873683+0000","last_became_active":"2024-05-10T10:34:57.866648+0000","last_became_peered":"2024-05-10T10:34:57.866648+0000","last_unstale":"2024-05-10T10:34:57.873683+0000","last_undegraded":"2024-05-10T10:34:57.873683+0000","last_fullsized":"2024-05-10T10:34:57.873683+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:00.246868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.878058+0000","last_change":"2024-05-10T10:34:57.872641+0000","last_active":"2024-05-10T10:34:57.878058+0000","last_peered":"2024-05-10T10:34:57.878058+0000","last_clean":"2024-05-10T10:34:57.878058+0000","last_became_active":"2024-05-10T10:34:57.872279+0000","last_became_peered":"2024-05-10T10:34:57.872279+0000","last_unstale":"2024-05-10T10:34:57.878058+0000","last_undegraded":"2024-05-10T10:34:57.878058+0000","last_fullsized":"2024-05-10T10:34:57.878058+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:16.145569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.885082+0000","last_change":"2024-05-10T10:34:57.883170+0000","last_active":"2024-05-10T10:34:57.885082+0000","last_peered":"2024-05-10T10:34:57.885082+0000","last_clean":"2024-05-10T10:34:57.885082+0000","last_became_active":"2024-05-10T10:34:57.882880+0000","last_became_peered":"2024-05-10T10:34:57.882880+0000","last_unstale":"2024-05-10T10:34:57.885082+0000","last_undegraded":"2024-05-10T10:34:57.885082+0000","last_fullsized":"2024-05-10T10:34:57.885082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:44:29.426213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871918+0000","last_change":"2024-05-10T10:34:57.868800+0000","last_active":"2024-05-10T10:34:57.871918+0000","last_peered":"2024-05-10T10:34:57.871918+0000","last_clean":"2024-05-10T10:34:57.871918+0000","last_became_active":"2024-05-10T10:34:57.868500+0000","last_became_peered":"2024-05-10T10:34:57.868500+0000","last_unstale":"2024-05-10T10:34:57.871918+0000","last_undegraded":"2024-05-10T10:34:57.871918+0000","last_fullsized":"2024-05-10T10:34:57.871918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:52:08.552440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870781+0000","last_change":"2024-05-10T10:34:57.865318+0000","last_active":"2024-05-10T10:34:57.870781+0000","last_peered":"2024-05-10T10:34:57.870781+0000","last_clean":"2024-05-10T10:34:57.870781+0000","last_became_active":"2024-05-10T10:34:57.864429+0000","last_became_peered":"2024-05-10T10:34:57.864429+0000","last_unstale":"2024-05-10T10:34:57.870781+0000","last_undegraded":"2024-05-10T10:34:57.870781+0000","last_fullsized":"2024-05-10T10:34:57.870781+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:52:08.552440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.885053+0000","last_change":"2024-05-10T10:34:57.883110+0000","last_active":"2024-05-10T10:34:57.885053+0000","last_peered":"2024-05-10T10:34:57.885053+0000","last_clean":"2024-05-10T10:34:57.885053+0000","last_became_active":"2024-05-10T10:34:57.882795+0000","last_became_peered":"2024-05-10T10:34:57.882795+0000","last_unstale":"2024-05-10T10:34:57.885053+0000","last_undegraded":"2024-05-10T10:34:57.885053+0000","last_fullsized":"2024-05-10T10:34:57.885053+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:04.663516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873382+0000","last_change":"2024-05-10T10:34:57.870892+0000","last_active":"2024-05-10T10:34:57.873382+0000","last_peered":"2024-05-10T10:34:57.873382+0000","last_clean":"2024-05-10T10:34:57.873382+0000","last_became_active":"2024-05-10T10:34:57.870641+0000","last_became_peered":"2024-05-10T10:34:57.870641+0000","last_unstale":"2024-05-10T10:34:57.873382+0000","last_undegraded":"2024-05-10T10:34:57.873382+0000","last_fullsized":"2024-05-10T10:34:57.873382+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:49:37.243526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877017+0000","last_change":"2024-05-10T10:34:57.870949+0000","last_active":"2024-05-10T10:34:57.877017+0000","last_peered":"2024-05-10T10:34:57.877017+0000","last_clean":"2024-05-10T10:34:57.877017+0000","last_became_active":"2024-05-10T10:34:57.870644+0000","last_became_peered":"2024-05-10T10:34:57.870644+0000","last_unstale":"2024-05-10T10:34:57.877017+0000","last_undegraded":"2024-05-10T10:34:57.877017+0000","last_fullsized":"2024-05-10T10:34:57.877017+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:21.348868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.874588+0000","last_change":"2024-05-10T10:34:57.872700+0000","last_active":"2024-05-10T10:34:57.874588+0000","last_peered":"2024-05-10T10:34:57.874588+0000","last_clean":"2024-05-10T10:34:57.874588+0000","last_became_active":"2024-05-10T10:34:57.872404+0000","last_became_peered":"2024-05-10T10:34:57.872404+0000","last_unstale":"2024-05-10T10:34:57.874588+0000","last_undegraded":"2024-05-10T10:34:57.874588+0000","last_fullsized":"2024-05-10T10:34:57.874588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:21.348868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872214+0000","last_change":"2024-05-10T10:34:57.867885+0000","last_active":"2024-05-10T10:34:57.872214+0000","last_peered":"2024-05-10T10:34:57.872214+0000","last_clean":"2024-05-10T10:34:57.872214+0000","last_became_active":"2024-05-10T10:34:57.867584+0000","last_became_peered":"2024-05-10T10:34:57.867584+0000","last_unstale":"2024-05-10T10:34:57.872214+0000","last_undegraded":"2024-05-10T10:34:57.872214+0000","last_fullsized":"2024-05-10T10:34:57.872214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:49.016906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872168+0000","last_change":"2024-05-10T10:34:57.867423+0000","last_active":"2024-05-10T10:34:57.872168+0000","last_peered":"2024-05-10T10:34:57.872168+0000","last_clean":"2024-05-10T10:34:57.872168+0000","last_became_active":"2024-05-10T10:34:57.867126+0000","last_became_peered":"2024-05-10T10:34:57.867126+0000","last_unstale":"2024-05-10T10:34:57.872168+0000","last_undegraded":"2024-05-10T10:34:57.872168+0000","last_fullsized":"2024-05-10T10:34:57.872168+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:54.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872138+0000","last_change":"2024-05-10T10:34:57.867242+0000","last_active":"2024-05-10T10:34:57.872138+0000","last_peered":"2024-05-10T10:34:57.872138+0000","last_clean":"2024-05-10T10:34:57.872138+0000","last_became_active":"2024-05-10T10:34:57.866936+0000","last_became_peered":"2024-05-10T10:34:57.866936+0000","last_unstale":"2024-05-10T10:34:57.872138+0000","last_undegraded":"2024-05-10T10:34:57.872138+0000","last_fullsized":"2024-05-10T10:34:57.872138+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:20.026079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.874908+0000","last_change":"2024-05-10T10:34:57.869693+0000","last_active":"2024-05-10T10:34:57.874908+0000","last_peered":"2024-05-10T10:34:57.874908+0000","last_clean":"2024-05-10T10:34:57.874908+0000","last_became_active":"2024-05-10T10:34:57.869448+0000","last_became_peered":"2024-05-10T10:34:57.869448+0000","last_unstale":"2024-05-10T10:34:57.874908+0000","last_undegraded":"2024-05-10T10:34:57.874908+0000","last_fullsized":"2024-05-10T10:34:57.874908+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:35:10.081984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.875855+0000","last_change":"2024-05-10T10:34:57.871734+0000","last_active":"2024-05-10T10:34:57.875855+0000","last_peered":"2024-05-10T10:34:57.875855+0000","last_clean":"2024-05-10T10:34:57.875855+0000","last_became_active":"2024-05-10T10:34:57.871377+0000","last_became_peered":"2024-05-10T10:34:57.871377+0000","last_unstale":"2024-05-10T10:34:57.875855+0000","last_undegraded":"2024-05-10T10:34:57.875855+0000","last_fullsized":"2024-05-10T10:34:57.875855+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:01:13.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877007+0000","last_change":"2024-05-10T10:34:57.874707+0000","last_active":"2024-05-10T10:34:57.877007+0000","last_peered":"2024-05-10T10:34:57.877007+0000","last_clean":"2024-05-10T10:34:57.877007+0000","last_became_active":"2024-05-10T10:34:57.874327+0000","last_became_peered":"2024-05-10T10:34:57.874327+0000","last_unstale":"2024-05-10T10:34:57.877007+0000","last_undegraded":"2024-05-10T10:34:57.877007+0000","last_fullsized":"2024-05-10T10:34:57.877007+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:09.598550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872048+0000","last_change":"2024-05-10T10:34:57.863732+0000","last_active":"2024-05-10T10:34:57.872048+0000","last_peered":"2024-05-10T10:34:57.872048+0000","last_clean":"2024-05-10T10:34:57.872048+0000","last_became_active":"2024-05-10T10:34:57.863459+0000","last_became_peered":"2024-05-10T10:34:57.863459+0000","last_unstale":"2024-05-10T10:34:57.872048+0000","last_undegraded":"2024-05-10T10:34:57.872048+0000","last_fullsized":"2024-05-10T10:34:57.872048+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:18.842628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873721+0000","last_change":"2024-05-10T10:34:57.870768+0000","last_active":"2024-05-10T10:34:57.873721+0000","last_peered":"2024-05-10T10:34:57.873721+0000","last_clean":"2024-05-10T10:34:57.873721+0000","last_became_active":"2024-05-10T10:34:57.870408+0000","last_became_peered":"2024-05-10T10:34:57.870408+0000","last_unstale":"2024-05-10T10:34:57.873721+0000","last_undegraded":"2024-05-10T10:34:57.873721+0000","last_fullsized":"2024-05-10T10:34:57.873721+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:04.784690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880061+0000","last_change":"2024-05-10T10:34:57.874955+0000","last_active":"2024-05-10T10:34:57.880061+0000","last_peered":"2024-05-10T10:34:57.880061+0000","last_clean":"2024-05-10T10:34:57.880061+0000","last_became_active":"2024-05-10T10:34:57.874646+0000","last_became_peered":"2024-05-10T10:34:57.874646+0000","last_unstale":"2024-05-10T10:34:57.880061+0000","last_undegraded":"2024-05-10T10:34:57.880061+0000","last_fullsized":"2024-05-10T10:34:57.880061+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.212205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870193+0000","last_change":"2024-05-10T10:34:57.865667+0000","last_active":"2024-05-10T10:34:57.870193+0000","last_peered":"2024-05-10T10:34:57.870193+0000","last_clean":"2024-05-10T10:34:57.870193+0000","last_became_active":"2024-05-10T10:34:57.865368+0000","last_became_peered":"2024-05-10T10:34:57.865368+0000","last_unstale":"2024-05-10T10:34:57.870193+0000","last_undegraded":"2024-05-10T10:34:57.870193+0000","last_fullsized":"2024-05-10T10:34:57.870193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:04.663516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870872+0000","last_change":"2024-05-10T10:34:57.866422+0000","last_active":"2024-05-10T10:34:57.870872+0000","last_peered":"2024-05-10T10:34:57.870872+0000","last_clean":"2024-05-10T10:34:57.870872+0000","last_became_active":"2024-05-10T10:34:57.866189+0000","last_became_peered":"2024-05-10T10:34:57.866189+0000","last_unstale":"2024-05-10T10:34:57.870872+0000","last_undegraded":"2024-05-10T10:34:57.870872+0000","last_fullsized":"2024-05-10T10:34:57.870872+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:01:13.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881345+0000","last_change":"2024-05-10T10:34:57.876495+0000","last_active":"2024-05-10T10:34:57.881345+0000","last_peered":"2024-05-10T10:34:57.881345+0000","last_clean":"2024-05-10T10:34:57.881345+0000","last_became_active":"2024-05-10T10:34:57.876137+0000","last_became_peered":"2024-05-10T10:34:57.876137+0000","last_unstale":"2024-05-10T10:34:57.881345+0000","last_undegraded":"2024-05-10T10:34:57.881345+0000","last_fullsized":"2024-05-10T10:34:57.881345+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880333+0000","last_change":"2024-05-10T10:34:57.875770+0000","last_active":"2024-05-10T10:34:57.880333+0000","last_peered":"2024-05-10T10:34:57.880333+0000","last_clean":"2024-05-10T10:34:57.880333+0000","last_became_active":"2024-05-10T10:34:57.875555+0000","last_became_peered":"2024-05-10T10:34:57.875555+0000","last_unstale":"2024-05-10T10:34:57.880333+0000","last_undegraded":"2024-05-10T10:34:57.880333+0000","last_fullsized":"2024-05-10T10:34:57.880333+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:18.842628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877339+0000","last_change":"2024-05-10T10:34:57.872086+0000","last_active":"2024-05-10T10:34:57.877339+0000","last_peered":"2024-05-10T10:34:57.877339+0000","last_clean":"2024-05-10T10:34:57.877339+0000","last_became_active":"2024-05-10T10:34:57.871777+0000","last_became_peered":"2024-05-10T10:34:57.871777+0000","last_unstale":"2024-05-10T10:34:57.877339+0000","last_undegraded":"2024-05-10T10:34:57.877339+0000","last_fullsized":"2024-05-10T10:34:57.877339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:58.298444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877772+0000","last_change":"2024-05-10T10:34:56.859545+0000","last_active":"2024-05-10T10:34:57.877772+0000","last_peered":"2024-05-10T10:34:57.877772+0000","last_clean":"2024-05-10T10:34:57.877772+0000","last_became_active":"2024-05-10T10:34:56.859226+0000","last_became_peered":"2024-05-10T10:34:56.859226+0000","last_unstale":"2024-05-10T10:34:57.877772+0000","last_undegraded":"2024-05-10T10:34:57.877772+0000","last_fullsized":"2024-05-10T10:34:57.877772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:53:59.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873839+0000","last_change":"2024-05-10T10:34:57.864324+0000","last_active":"2024-05-10T10:34:57.873839+0000","last_peered":"2024-05-10T10:34:57.873839+0000","last_clean":"2024-05-10T10:34:57.873839+0000","last_became_active":"2024-05-10T10:34:57.864022+0000","last_became_peered":"2024-05-10T10:34:57.864022+0000","last_unstale":"2024-05-10T10:34:57.873839+0000","last_undegraded":"2024-05-10T10:34:57.873839+0000","last_fullsized":"2024-05-10T10:34:57.873839+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:22.457663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884311+0000","last_change":"2024-05-10T10:34:56.867469+0000","last_active":"2024-05-10T10:34:57.884311+0000","last_peered":"2024-05-10T10:34:57.884311+0000","last_clean":"2024-05-10T10:34:57.884311+0000","last_became_active":"2024-05-10T10:34:56.867176+0000","last_became_peered":"2024-05-10T10:34:56.867176+0000","last_unstale":"2024-05-10T10:34:57.884311+0000","last_undegraded":"2024-05-10T10:34:57.884311+0000","last_fullsized":"2024-05-10T10:34:57.884311+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:36.951524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872240+0000","last_change":"2024-05-10T10:34:57.869886+0000","last_active":"2024-05-10T10:34:57.872240+0000","last_peered":"2024-05-10T10:34:57.872240+0000","last_clean":"2024-05-10T10:34:57.872240+0000","last_became_active":"2024-05-10T10:34:57.869609+0000","last_became_peered":"2024-05-10T10:34:57.869609+0000","last_unstale":"2024-05-10T10:34:57.872240+0000","last_undegraded":"2024-05-10T10:34:57.872240+0000","last_fullsized":"2024-05-10T10:34:57.872240+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:04.784690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881954+0000","last_change":"2024-05-10T10:34:56.861449+0000","last_active":"2024-05-10T10:34:57.881954+0000","last_peered":"2024-05-10T10:34:57.881954+0000","last_clean":"2024-05-10T10:34:57.881954+0000","last_became_active":"2024-05-10T10:34:56.861155+0000","last_became_peered":"2024-05-10T10:34:56.861155+0000","last_unstale":"2024-05-10T10:34:57.881954+0000","last_undegraded":"2024-05-10T10:34:57.881954+0000","last_fullsized":"2024-05-10T10:34:57.881954+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:04.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881264+0000","last_change":"2024-05-10T10:34:57.875953+0000","last_active":"2024-05-10T10:34:57.881264+0000","last_peered":"2024-05-10T10:34:57.881264+0000","last_clean":"2024-05-10T10:34:57.881264+0000","last_became_active":"2024-05-10T10:34:57.875434+0000","last_became_peered":"2024-05-10T10:34:57.875434+0000","last_unstale":"2024-05-10T10:34:57.881264+0000","last_undegraded":"2024-05-10T10:34:57.881264+0000","last_fullsized":"2024-05-10T10:34:57.881264+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:29:57.259011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881773+0000","last_change":"2024-05-10T10:34:56.860462+0000","last_active":"2024-05-10T10:34:57.881773+0000","last_peered":"2024-05-10T10:34:57.881773+0000","last_clean":"2024-05-10T10:34:57.881773+0000","last_became_active":"2024-05-10T10:34:56.860067+0000","last_became_peered":"2024-05-10T10:34:56.860067+0000","last_unstale":"2024-05-10T10:34:57.881773+0000","last_undegraded":"2024-05-10T10:34:57.881773+0000","last_fullsized":"2024-05-10T10:34:57.881773+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:52:07.550131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877103+0000","last_change":"2024-05-10T10:34:57.870462+0000","last_active":"2024-05-10T10:34:57.877103+0000","last_peered":"2024-05-10T10:34:57.877103+0000","last_clean":"2024-05-10T10:34:57.877103+0000","last_became_active":"2024-05-10T10:34:57.868751+0000","last_became_peered":"2024-05-10T10:34:57.868751+0000","last_unstale":"2024-05-10T10:34:57.877103+0000","last_undegraded":"2024-05-10T10:34:57.877103+0000","last_fullsized":"2024-05-10T10:34:57.877103+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:08.508199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884009+0000","last_change":"2024-05-10T10:34:56.864551+0000","last_active":"2024-05-10T10:34:57.884009+0000","last_peered":"2024-05-10T10:34:57.884009+0000","last_clean":"2024-05-10T10:34:57.884009+0000","last_became_active":"2024-05-10T10:34:56.864051+0000","last_became_peered":"2024-05-10T10:34:56.864051+0000","last_unstale":"2024-05-10T10:34:57.884009+0000","last_undegraded":"2024-05-10T10:34:57.884009+0000","last_fullsized":"2024-05-10T10:34:57.884009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:50.072960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871679+0000","last_change":"2024-05-10T10:34:57.865586+0000","last_active":"2024-05-10T10:34:57.871679+0000","last_peered":"2024-05-10T10:34:57.871679+0000","last_clean":"2024-05-10T10:34:57.871679+0000","last_became_active":"2024-05-10T10:34:57.865246+0000","last_became_peered":"2024-05-10T10:34:57.865246+0000","last_unstale":"2024-05-10T10:34:57.871679+0000","last_undegraded":"2024-05-10T10:34:57.871679+0000","last_fullsized":"2024-05-10T10:34:57.871679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:49:37.243526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879905+0000","last_change":"2024-05-10T10:34:56.861850+0000","last_active":"2024-05-10T10:34:57.879905+0000","last_peered":"2024-05-10T10:34:57.879905+0000","last_clean":"2024-05-10T10:34:57.879905+0000","last_became_active":"2024-05-10T10:34:56.861284+0000","last_became_peered":"2024-05-10T10:34:56.861284+0000","last_unstale":"2024-05-10T10:34:57.879905+0000","last_undegraded":"2024-05-10T10:34:57.879905+0000","last_fullsized":"2024-05-10T10:34:57.879905+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:52:05.287263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.869252+0000","last_change":"2024-05-10T10:34:57.861124+0000","last_active":"2024-05-10T10:34:57.869252+0000","last_peered":"2024-05-10T10:34:57.869252+0000","last_clean":"2024-05-10T10:34:57.869252+0000","last_became_active":"2024-05-10T10:34:57.860834+0000","last_became_peered":"2024-05-10T10:34:57.860834+0000","last_unstale":"2024-05-10T10:34:57.869252+0000","last_undegraded":"2024-05-10T10:34:57.869252+0000","last_fullsized":"2024-05-10T10:34:57.869252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.861657+0000","last_change":"2024-05-10T10:34:56.860380+0000","last_active":"2024-05-10T10:34:57.861657+0000","last_peered":"2024-05-10T10:34:57.861657+0000","last_clean":"2024-05-10T10:34:57.861657+0000","last_became_active":"2024-05-10T10:34:56.860033+0000","last_became_peered":"2024-05-10T10:34:56.860033+0000","last_unstale":"2024-05-10T10:34:57.861657+0000","last_undegraded":"2024-05-10T10:34:57.861657+0000","last_fullsized":"2024-05-10T10:34:57.861657+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:29.174564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.876940+0000","last_change":"2024-05-10T10:34:57.871487+0000","last_active":"2024-05-10T10:34:57.876940+0000","last_peered":"2024-05-10T10:34:57.876940+0000","last_clean":"2024-05-10T10:34:57.876940+0000","last_became_active":"2024-05-10T10:34:57.871211+0000","last_became_peered":"2024-05-10T10:34:57.871211+0000","last_unstale":"2024-05-10T10:34:57.876940+0000","last_undegraded":"2024-05-10T10:34:57.876940+0000","last_fullsized":"2024-05-10T10:34:57.876940+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:09.598550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.855449+0000","last_change":"2024-05-10T10:34:56.872434+0000","last_active":"2024-05-10T10:34:57.855449+0000","last_peered":"2024-05-10T10:34:57.855449+0000","last_clean":"2024-05-10T10:34:57.855449+0000","last_became_active":"2024-05-10T10:34:56.872134+0000","last_became_peered":"2024-05-10T10:34:56.872134+0000","last_unstale":"2024-05-10T10:34:57.855449+0000","last_undegraded":"2024-05-10T10:34:57.855449+0000","last_fullsized":"2024-05-10T10:34:57.855449+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:53.787388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879960+0000","last_change":"2024-05-10T10:34:57.875401+0000","last_active":"2024-05-10T10:34:57.879960+0000","last_peered":"2024-05-10T10:34:57.879960+0000","last_clean":"2024-05-10T10:34:57.879960+0000","last_became_active":"2024-05-10T10:34:57.875099+0000","last_became_peered":"2024-05-10T10:34:57.875099+0000","last_unstale":"2024-05-10T10:34:57.879960+0000","last_undegraded":"2024-05-10T10:34:57.879960+0000","last_fullsized":"2024-05-10T10:34:57.879960+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:49.016906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.858509+0000","last_change":"2024-05-10T10:34:56.859542+0000","last_active":"2024-05-10T10:34:57.858509+0000","last_peered":"2024-05-10T10:34:57.858509+0000","last_clean":"2024-05-10T10:34:57.858509+0000","last_became_active":"2024-05-10T10:34:56.859260+0000","last_became_peered":"2024-05-10T10:34:56.859260+0000","last_unstale":"2024-05-10T10:34:57.858509+0000","last_undegraded":"2024-05-10T10:34:57.858509+0000","last_fullsized":"2024-05-10T10:34:57.858509+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:19:16.902820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.869354+0000","last_change":"2024-05-10T10:34:57.862046+0000","last_active":"2024-05-10T10:34:57.869354+0000","last_peered":"2024-05-10T10:34:57.869354+0000","last_clean":"2024-05-10T10:34:57.869354+0000","last_became_active":"2024-05-10T10:34:57.861769+0000","last_became_peered":"2024-05-10T10:34:57.861769+0000","last_unstale":"2024-05-10T10:34:57.869354+0000","last_undegraded":"2024-05-10T10:34:57.869354+0000","last_fullsized":"2024-05-10T10:34:57.869354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:01:13.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.861104+0000","last_change":"2024-05-10T10:34:56.863386+0000","last_active":"2024-05-10T10:34:57.861104+0000","last_peered":"2024-05-10T10:34:57.861104+0000","last_clean":"2024-05-10T10:34:57.861104+0000","last_became_active":"2024-05-10T10:34:56.863125+0000","last_became_peered":"2024-05-10T10:34:56.863125+0000","last_unstale":"2024-05-10T10:34:57.861104+0000","last_undegraded":"2024-05-10T10:34:57.861104+0000","last_fullsized":"2024-05-10T10:34:57.861104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:04.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871897+0000","last_change":"2024-05-10T10:34:57.868708+0000","last_active":"2024-05-10T10:34:57.871897+0000","last_peered":"2024-05-10T10:34:57.871897+0000","last_clean":"2024-05-10T10:34:57.871897+0000","last_became_active":"2024-05-10T10:34:57.868411+0000","last_became_peered":"2024-05-10T10:34:57.868411+0000","last_unstale":"2024-05-10T10:34:57.871897+0000","last_undegraded":"2024-05-10T10:34:57.871897+0000","last_fullsized":"2024-05-10T10:34:57.871897+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:19:17.905130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884721+0000","last_change":"2024-05-10T10:34:56.865480+0000","last_active":"2024-05-10T10:34:57.884721+0000","last_peered":"2024-05-10T10:34:57.884721+0000","last_clean":"2024-05-10T10:34:57.884721+0000","last_became_active":"2024-05-10T10:34:56.865196+0000","last_became_peered":"2024-05-10T10:34:56.865196+0000","last_unstale":"2024-05-10T10:34:57.884721+0000","last_undegraded":"2024-05-10T10:34:57.884721+0000","last_fullsized":"2024-05-10T10:34:57.884721+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:44:28.423904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877047+0000","last_change":"2024-05-10T10:34:57.870478+0000","last_active":"2024-05-10T10:34:57.877047+0000","last_peered":"2024-05-10T10:34:57.877047+0000","last_clean":"2024-05-10T10:34:57.877047+0000","last_became_active":"2024-05-10T10:34:57.869074+0000","last_became_peered":"2024-05-10T10:34:57.869074+0000","last_unstale":"2024-05-10T10:34:57.877047+0000","last_undegraded":"2024-05-10T10:34:57.877047+0000","last_fullsized":"2024-05-10T10:34:57.877047+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:54.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.885922+0000","last_change":"2024-05-10T10:34:56.867033+0000","last_active":"2024-05-10T10:34:57.885922+0000","last_peered":"2024-05-10T10:34:57.885922+0000","last_clean":"2024-05-10T10:34:57.885922+0000","last_became_active":"2024-05-10T10:34:56.866671+0000","last_became_peered":"2024-05-10T10:34:56.866671+0000","last_unstale":"2024-05-10T10:34:57.885922+0000","last_undegraded":"2024-05-10T10:34:57.885922+0000","last_fullsized":"2024-05-10T10:34:57.885922+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:51:38.241742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.869543+0000","last_change":"2024-05-10T10:34:57.862681+0000","last_active":"2024-05-10T10:34:57.869543+0000","last_peered":"2024-05-10T10:34:57.869543+0000","last_clean":"2024-05-10T10:34:57.869543+0000","last_became_active":"2024-05-10T10:34:57.862397+0000","last_became_peered":"2024-05-10T10:34:57.862397+0000","last_unstale":"2024-05-10T10:34:57.869543+0000","last_undegraded":"2024-05-10T10:34:57.869543+0000","last_fullsized":"2024-05-10T10:34:57.869543+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:21.348868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882662+0000","last_change":"2024-05-10T10:34:56.864313+0000","last_active":"2024-05-10T10:34:57.882662+0000","last_peered":"2024-05-10T10:34:57.882662+0000","last_clean":"2024-05-10T10:34:57.882662+0000","last_became_active":"2024-05-10T10:34:56.863747+0000","last_became_peered":"2024-05-10T10:34:56.863747+0000","last_unstale":"2024-05-10T10:34:57.882662+0000","last_undegraded":"2024-05-10T10:34:57.882662+0000","last_fullsized":"2024-05-10T10:34:57.882662+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:03.782381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884256+0000","last_change":"2024-05-10T10:34:57.883318+0000","last_active":"2024-05-10T10:34:57.884256+0000","last_peered":"2024-05-10T10:34:57.884256+0000","last_clean":"2024-05-10T10:34:57.884256+0000","last_became_active":"2024-05-10T10:34:57.883052+0000","last_became_peered":"2024-05-10T10:34:57.883052+0000","last_unstale":"2024-05-10T10:34:57.884256+0000","last_undegraded":"2024-05-10T10:34:57.884256+0000","last_fullsized":"2024-05-10T10:34:57.884256+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:34:11.896217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.862375+0000","last_change":"2024-05-10T10:34:56.866513+0000","last_active":"2024-05-10T10:34:57.862375+0000","last_peered":"2024-05-10T10:34:57.862375+0000","last_clean":"2024-05-10T10:34:57.862375+0000","last_became_active":"2024-05-10T10:34:56.866212+0000","last_became_peered":"2024-05-10T10:34:56.866212+0000","last_unstale":"2024-05-10T10:34:57.862375+0000","last_undegraded":"2024-05-10T10:34:57.862375+0000","last_fullsized":"2024-05-10T10:34:57.862375+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:07.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872073+0000","last_change":"2024-05-10T10:34:57.866243+0000","last_active":"2024-05-10T10:34:57.872073+0000","last_peered":"2024-05-10T10:34:57.872073+0000","last_clean":"2024-05-10T10:34:57.872073+0000","last_became_active":"2024-05-10T10:34:57.865939+0000","last_became_peered":"2024-05-10T10:34:57.865939+0000","last_unstale":"2024-05-10T10:34:57.872073+0000","last_undegraded":"2024-05-10T10:34:57.872073+0000","last_fullsized":"2024-05-10T10:34:57.872073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:36:41.686995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857409+0000","last_change":"2024-05-10T10:34:56.861868+0000","last_active":"2024-05-10T10:34:57.857409+0000","last_peered":"2024-05-10T10:34:57.857409+0000","last_clean":"2024-05-10T10:34:57.857409+0000","last_became_active":"2024-05-10T10:34:56.861533+0000","last_became_peered":"2024-05-10T10:34:56.861533+0000","last_unstale":"2024-05-10T10:34:57.857409+0000","last_undegraded":"2024-05-10T10:34:57.857409+0000","last_fullsized":"2024-05-10T10:34:57.857409+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:03.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879003+0000","last_change":"2024-05-10T10:34:57.875798+0000","last_active":"2024-05-10T10:34:57.879003+0000","last_peered":"2024-05-10T10:34:57.879003+0000","last_clean":"2024-05-10T10:34:57.879003+0000","last_became_active":"2024-05-10T10:34:57.875524+0000","last_became_peered":"2024-05-10T10:34:57.875524+0000","last_unstale":"2024-05-10T10:34:57.879003+0000","last_undegraded":"2024-05-10T10:34:57.879003+0000","last_fullsized":"2024-05-10T10:34:57.879003+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:58.298444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879554+0000","last_change":"2024-05-10T10:34:56.861950+0000","last_active":"2024-05-10T10:34:57.879554+0000","last_peered":"2024-05-10T10:34:57.879554+0000","last_clean":"2024-05-10T10:34:57.879554+0000","last_became_active":"2024-05-10T10:34:56.861652+0000","last_became_peered":"2024-05-10T10:34:56.861652+0000","last_unstale":"2024-05-10T10:34:57.879554+0000","last_undegraded":"2024-05-10T10:34:57.879554+0000","last_fullsized":"2024-05-10T10:34:57.879554+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:20.346559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877320+0000","last_change":"2024-05-10T10:34:57.874606+0000","last_active":"2024-05-10T10:34:57.877320+0000","last_peered":"2024-05-10T10:34:57.877320+0000","last_clean":"2024-05-10T10:34:57.877320+0000","last_became_active":"2024-05-10T10:34:57.874141+0000","last_became_peered":"2024-05-10T10:34:57.874141+0000","last_unstale":"2024-05-10T10:34:57.877320+0000","last_undegraded":"2024-05-10T10:34:57.877320+0000","last_fullsized":"2024-05-10T10:34:57.877320+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:29.475280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882409+0000","last_change":"2024-05-10T10:34:56.870750+0000","last_active":"2024-05-10T10:34:57.882409+0000","last_peered":"2024-05-10T10:34:57.882409+0000","last_clean":"2024-05-10T10:34:57.882409+0000","last_became_active":"2024-05-10T10:34:56.870433+0000","last_became_peered":"2024-05-10T10:34:56.870433+0000","last_unstale":"2024-05-10T10:34:57.882409+0000","last_undegraded":"2024-05-10T10:34:57.882409+0000","last_fullsized":"2024-05-10T10:34:57.882409+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:16.209896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871237+0000","last_change":"2024-05-10T10:34:57.869859+0000","last_active":"2024-05-10T10:34:57.871237+0000","last_peered":"2024-05-10T10:34:57.871237+0000","last_clean":"2024-05-10T10:34:57.871237+0000","last_became_active":"2024-05-10T10:34:57.869551+0000","last_became_peered":"2024-05-10T10:34:57.869551+0000","last_unstale":"2024-05-10T10:34:57.871237+0000","last_undegraded":"2024-05-10T10:34:57.871237+0000","last_fullsized":"2024-05-10T10:34:57.871237+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:54.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879016+0000","last_change":"2024-05-10T10:34:56.859624+0000","last_active":"2024-05-10T10:34:57.879016+0000","last_peered":"2024-05-10T10:34:57.879016+0000","last_clean":"2024-05-10T10:34:57.879016+0000","last_became_active":"2024-05-10T10:34:56.859338+0000","last_became_peered":"2024-05-10T10:34:56.859338+0000","last_unstale":"2024-05-10T10:34:57.879016+0000","last_undegraded":"2024-05-10T10:34:57.879016+0000","last_fullsized":"2024-05-10T10:34:57.879016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:11.030813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877216+0000","last_change":"2024-05-10T10:34:57.870212+0000","last_active":"2024-05-10T10:34:57.877216+0000","last_peered":"2024-05-10T10:34:57.877216+0000","last_clean":"2024-05-10T10:34:57.877216+0000","last_became_active":"2024-05-10T10:34:57.868839+0000","last_became_peered":"2024-05-10T10:34:57.868839+0000","last_unstale":"2024-05-10T10:34:57.877216+0000","last_undegraded":"2024-05-10T10:34:57.877216+0000","last_fullsized":"2024-05-10T10:34:57.877216+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.176874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877015+0000","last_change":"2024-05-10T10:34:56.865559+0000","last_active":"2024-05-10T10:34:57.877015+0000","last_peered":"2024-05-10T10:34:57.877015+0000","last_clean":"2024-05-10T10:34:57.877015+0000","last_became_active":"2024-05-10T10:34:56.865287+0000","last_became_peered":"2024-05-10T10:34:56.865287+0000","last_unstale":"2024-05-10T10:34:57.877015+0000","last_undegraded":"2024-05-10T10:34:57.877015+0000","last_fullsized":"2024-05-10T10:34:57.877015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:36.951524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873760+0000","last_change":"2024-05-10T10:34:57.871807+0000","last_active":"2024-05-10T10:34:57.873760+0000","last_peered":"2024-05-10T10:34:57.873760+0000","last_clean":"2024-05-10T10:34:57.873760+0000","last_became_active":"2024-05-10T10:34:57.871471+0000","last_became_peered":"2024-05-10T10:34:57.871471+0000","last_unstale":"2024-05-10T10:34:57.873760+0000","last_undegraded":"2024-05-10T10:34:57.873760+0000","last_fullsized":"2024-05-10T10:34:57.873760+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:14.233463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.860224+0000","last_change":"2024-05-10T10:34:56.871293+0000","last_active":"2024-05-10T10:34:57.860224+0000","last_peered":"2024-05-10T10:34:57.860224+0000","last_clean":"2024-05-10T10:34:57.860224+0000","last_became_active":"2024-05-10T10:34:56.871022+0000","last_became_peered":"2024-05-10T10:34:56.871022+0000","last_unstale":"2024-05-10T10:34:57.860224+0000","last_undegraded":"2024-05-10T10:34:57.860224+0000","last_fullsized":"2024-05-10T10:34:57.860224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:04.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.874661+0000","last_change":"2024-05-10T10:34:57.868372+0000","last_active":"2024-05-10T10:34:57.874661+0000","last_peered":"2024-05-10T10:34:57.874661+0000","last_clean":"2024-05-10T10:34:57.874661+0000","last_became_active":"2024-05-10T10:34:57.868059+0000","last_became_peered":"2024-05-10T10:34:57.868059+0000","last_unstale":"2024-05-10T10:34:57.874661+0000","last_undegraded":"2024-05-10T10:34:57.874661+0000","last_fullsized":"2024-05-10T10:34:57.874661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:58.298444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.883498+0000","last_change":"2024-05-10T10:34:56.868593+0000","last_active":"2024-05-10T10:34:57.883498+0000","last_peered":"2024-05-10T10:34:57.883498+0000","last_clean":"2024-05-10T10:34:57.883498+0000","last_became_active":"2024-05-10T10:34:56.868271+0000","last_became_peered":"2024-05-10T10:34:56.868271+0000","last_unstale":"2024-05-10T10:34:57.883498+0000","last_undegraded":"2024-05-10T10:34:57.883498+0000","last_fullsized":"2024-05-10T10:34:57.883498+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:52:05.287263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877092+0000","last_change":"2024-05-10T10:34:57.872953+0000","last_active":"2024-05-10T10:34:57.877092+0000","last_peered":"2024-05-10T10:34:57.877092+0000","last_clean":"2024-05-10T10:34:57.877092+0000","last_became_active":"2024-05-10T10:34:57.872677+0000","last_became_peered":"2024-05-10T10:34:57.872677+0000","last_unstale":"2024-05-10T10:34:57.877092+0000","last_undegraded":"2024-05-10T10:34:57.877092+0000","last_fullsized":"2024-05-10T10:34:57.877092+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:51.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857087+0000","last_change":"2024-05-10T10:34:56.864122+0000","last_active":"2024-05-10T10:34:57.857087+0000","last_peered":"2024-05-10T10:34:57.857087+0000","last_clean":"2024-05-10T10:34:57.857087+0000","last_became_active":"2024-05-10T10:34:56.863828+0000","last_became_peered":"2024-05-10T10:34:56.863828+0000","last_unstale":"2024-05-10T10:34:57.857087+0000","last_undegraded":"2024-05-10T10:34:57.857087+0000","last_fullsized":"2024-05-10T10:34:57.857087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:08.596241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870163+0000","last_change":"2024-05-10T10:34:57.865992+0000","last_active":"2024-05-10T10:34:57.870163+0000","last_peered":"2024-05-10T10:34:57.870163+0000","last_clean":"2024-05-10T10:34:57.870163+0000","last_became_active":"2024-05-10T10:34:57.865463+0000","last_became_peered":"2024-05-10T10:34:57.865463+0000","last_unstale":"2024-05-10T10:34:57.870163+0000","last_undegraded":"2024-05-10T10:34:57.870163+0000","last_fullsized":"2024-05-10T10:34:57.870163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:56:20.618702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.855880+0000","last_change":"2024-05-10T10:34:56.870819+0000","last_active":"2024-05-10T10:34:57.855880+0000","last_peered":"2024-05-10T10:34:57.855880+0000","last_clean":"2024-05-10T10:34:57.855880+0000","last_became_active":"2024-05-10T10:34:56.870473+0000","last_became_peered":"2024-05-10T10:34:56.870473+0000","last_unstale":"2024-05-10T10:34:57.855880+0000","last_undegraded":"2024-05-10T10:34:57.855880+0000","last_fullsized":"2024-05-10T10:34:57.855880+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:48.014597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872664+0000","last_change":"2024-05-10T10:34:57.870562+0000","last_active":"2024-05-10T10:34:57.872664+0000","last_peered":"2024-05-10T10:34:57.872664+0000","last_clean":"2024-05-10T10:34:57.872664+0000","last_became_active":"2024-05-10T10:34:57.870253+0000","last_became_peered":"2024-05-10T10:34:57.870253+0000","last_unstale":"2024-05-10T10:34:57.872664+0000","last_undegraded":"2024-05-10T10:34:57.872664+0000","last_fullsized":"2024-05-10T10:34:57.872664+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:49.016906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.849586+0000","last_change":"2024-05-10T10:34:56.857882+0000","last_active":"2024-05-10T10:34:57.849586+0000","last_peered":"2024-05-10T10:34:57.849586+0000","last_clean":"2024-05-10T10:34:57.849586+0000","last_became_active":"2024-05-10T10:34:56.857602+0000","last_became_peered":"2024-05-10T10:34:56.857602+0000","last_unstale":"2024-05-10T10:34:57.849586+0000","last_undegraded":"2024-05-10T10:34:57.849586+0000","last_fullsized":"2024-05-10T10:34:57.849586+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:26:09.426309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880356+0000","last_change":"2024-05-10T10:34:57.875747+0000","last_active":"2024-05-10T10:34:57.880356+0000","last_peered":"2024-05-10T10:34:57.880356+0000","last_clean":"2024-05-10T10:34:57.880356+0000","last_became_active":"2024-05-10T10:34:57.875375+0000","last_became_peered":"2024-05-10T10:34:57.875375+0000","last_unstale":"2024-05-10T10:34:57.880356+0000","last_undegraded":"2024-05-10T10:34:57.880356+0000","last_fullsized":"2024-05-10T10:34:57.880356+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880826+0000","last_change":"2024-05-10T10:34:56.869450+0000","last_active":"2024-05-10T10:34:57.880826+0000","last_peered":"2024-05-10T10:34:57.880826+0000","last_clean":"2024-05-10T10:34:57.880826+0000","last_became_active":"2024-05-10T10:34:56.869187+0000","last_became_peered":"2024-05-10T10:34:56.869187+0000","last_unstale":"2024-05-10T10:34:57.880826+0000","last_undegraded":"2024-05-10T10:34:57.880826+0000","last_fullsized":"2024-05-10T10:34:57.880826+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:52:05.287263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.863564+0000","last_change":"2024-05-10T10:34:57.861958+0000","last_active":"2024-05-10T10:34:57.863564+0000","last_peered":"2024-05-10T10:34:57.863564+0000","last_clean":"2024-05-10T10:34:57.863564+0000","last_became_active":"2024-05-10T10:34:57.861676+0000","last_became_peered":"2024-05-10T10:34:57.861676+0000","last_unstale":"2024-05-10T10:34:57.863564+0000","last_undegraded":"2024-05-10T10:34:57.863564+0000","last_fullsized":"2024-05-10T10:34:57.863564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.176874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.858071+0000","last_change":"2024-05-10T10:34:56.861703+0000","last_active":"2024-05-10T10:34:57.858071+0000","last_peered":"2024-05-10T10:34:57.858071+0000","last_clean":"2024-05-10T10:34:57.858071+0000","last_became_active":"2024-05-10T10:34:56.861365+0000","last_became_peered":"2024-05-10T10:34:56.861365+0000","last_unstale":"2024-05-10T10:34:57.858071+0000","last_undegraded":"2024-05-10T10:34:57.858071+0000","last_fullsized":"2024-05-10T10:34:57.858071+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:56:19.616392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879773+0000","last_change":"2024-05-10T10:34:57.875728+0000","last_active":"2024-05-10T10:34:57.879773+0000","last_peered":"2024-05-10T10:34:57.879773+0000","last_clean":"2024-05-10T10:34:57.879773+0000","last_became_active":"2024-05-10T10:34:57.875070+0000","last_became_peered":"2024-05-10T10:34:57.875070+0000","last_unstale":"2024-05-10T10:34:57.879773+0000","last_undegraded":"2024-05-10T10:34:57.879773+0000","last_fullsized":"2024-05-10T10:34:57.879773+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:35.349340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.856832+0000","last_change":"2024-05-10T10:34:56.862064+0000","last_active":"2024-05-10T10:34:57.856832+0000","last_peered":"2024-05-10T10:34:57.856832+0000","last_clean":"2024-05-10T10:34:57.856832+0000","last_became_active":"2024-05-10T10:34:56.861795+0000","last_became_peered":"2024-05-10T10:34:56.861795+0000","last_unstale":"2024-05-10T10:34:57.856832+0000","last_undegraded":"2024-05-10T10:34:57.856832+0000","last_fullsized":"2024-05-10T10:34:57.856832+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:13.231154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.878977+0000","last_change":"2024-05-10T10:34:57.874726+0000","last_active":"2024-05-10T10:34:57.878977+0000","last_peered":"2024-05-10T10:34:57.878977+0000","last_clean":"2024-05-10T10:34:57.878977+0000","last_became_active":"2024-05-10T10:34:57.874415+0000","last_became_peered":"2024-05-10T10:34:57.874415+0000","last_unstale":"2024-05-10T10:34:57.878977+0000","last_undegraded":"2024-05-10T10:34:57.878977+0000","last_fullsized":"2024-05-10T10:34:57.878977+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:22.457663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.859095+0000","last_change":"2024-05-10T10:34:56.860754+0000","last_active":"2024-05-10T10:34:57.859095+0000","last_peered":"2024-05-10T10:34:57.859095+0000","last_clean":"2024-05-10T10:34:57.859095+0000","last_became_active":"2024-05-10T10:34:56.860381+0000","last_became_peered":"2024-05-10T10:34:56.860381+0000","last_unstale":"2024-05-10T10:34:57.859095+0000","last_undegraded":"2024-05-10T10:34:57.859095+0000","last_fullsized":"2024-05-10T10:34:57.859095+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:57.296135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873763+0000","last_change":"2024-05-10T10:34:57.863630+0000","last_active":"2024-05-10T10:34:57.873763+0000","last_peered":"2024-05-10T10:34:57.873763+0000","last_clean":"2024-05-10T10:34:57.873763+0000","last_became_active":"2024-05-10T10:34:57.863377+0000","last_became_peered":"2024-05-10T10:34:57.863377+0000","last_unstale":"2024-05-10T10:34:57.873763+0000","last_undegraded":"2024-05-10T10:34:57.873763+0000","last_fullsized":"2024-05-10T10:34:57.873763+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:09.598550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857272+0000","last_change":"2024-05-10T10:34:56.868854+0000","last_active":"2024-05-10T10:34:57.857272+0000","last_peered":"2024-05-10T10:34:57.857272+0000","last_clean":"2024-05-10T10:34:57.857272+0000","last_became_active":"2024-05-10T10:34:56.868556+0000","last_became_peered":"2024-05-10T10:34:56.868556+0000","last_unstale":"2024-05-10T10:34:57.857272+0000","last_undegraded":"2024-05-10T10:34:57.857272+0000","last_fullsized":"2024-05-10T10:34:57.857272+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:21.455354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870141+0000","last_change":"2024-05-10T10:34:57.865756+0000","last_active":"2024-05-10T10:34:57.870141+0000","last_peered":"2024-05-10T10:34:57.870141+0000","last_clean":"2024-05-10T10:34:57.870141+0000","last_became_active":"2024-05-10T10:34:57.865083+0000","last_became_peered":"2024-05-10T10:34:57.865083+0000","last_unstale":"2024-05-10T10:34:57.870141+0000","last_undegraded":"2024-05-10T10:34:57.870141+0000","last_fullsized":"2024-05-10T10:34:57.870141+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:44:29.426213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.860050+0000","last_change":"2024-05-10T10:34:56.861607+0000","last_active":"2024-05-10T10:34:57.860050+0000","last_peered":"2024-05-10T10:34:57.860050+0000","last_clean":"2024-05-10T10:34:57.860050+0000","last_became_active":"2024-05-10T10:34:56.861299+0000","last_became_peered":"2024-05-10T10:34:56.861299+0000","last_unstale":"2024-05-10T10:34:57.860050+0000","last_undegraded":"2024-05-10T10:34:57.860050+0000","last_fullsized":"2024-05-10T10:34:57.860050+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:53:59.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872034+0000","last_change":"2024-05-10T10:34:57.866702+0000","last_active":"2024-05-10T10:34:57.872034+0000","last_peered":"2024-05-10T10:34:57.872034+0000","last_clean":"2024-05-10T10:34:57.872034+0000","last_became_active":"2024-05-10T10:34:57.866426+0000","last_became_peered":"2024-05-10T10:34:57.866426+0000","last_unstale":"2024-05-10T10:34:57.872034+0000","last_undegraded":"2024-05-10T10:34:57.872034+0000","last_fullsized":"2024-05-10T10:34:57.872034+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:39:12.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.854336+0000","last_change":"2024-05-10T10:34:56.868459+0000","last_active":"2024-05-10T10:34:57.854336+0000","last_peered":"2024-05-10T10:34:57.854336+0000","last_clean":"2024-05-10T10:34:57.854336+0000","last_became_active":"2024-05-10T10:34:56.868178+0000","last_became_peered":"2024-05-10T10:34:56.868178+0000","last_unstale":"2024-05-10T10:34:57.854336+0000","last_undegraded":"2024-05-10T10:34:57.854336+0000","last_fullsized":"2024-05-10T10:34:57.854336+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:49:36.241216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880033+0000","last_change":"2024-05-10T10:34:57.874732+0000","last_active":"2024-05-10T10:34:57.880033+0000","last_peered":"2024-05-10T10:34:57.880033+0000","last_clean":"2024-05-10T10:34:57.880033+0000","last_became_active":"2024-05-10T10:34:57.874247+0000","last_became_peered":"2024-05-10T10:34:57.874247+0000","last_unstale":"2024-05-10T10:34:57.880033+0000","last_undegraded":"2024-05-10T10:34:57.880033+0000","last_fullsized":"2024-05-10T10:34:57.880033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:14.233463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.854864+0000","last_change":"2024-05-10T10:34:56.871581+0000","last_active":"2024-05-10T10:34:57.854864+0000","last_peered":"2024-05-10T10:34:57.854864+0000","last_clean":"2024-05-10T10:34:57.854864+0000","last_became_active":"2024-05-10T10:34:56.871283+0000","last_became_peered":"2024-05-10T10:34:56.871283+0000","last_unstale":"2024-05-10T10:34:57.854864+0000","last_undegraded":"2024-05-10T10:34:57.854864+0000","last_fullsized":"2024-05-10T10:34:57.854864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:07.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":164,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857269+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:34:57.857269+0000","last_peered":"2024-05-10T10:34:57.857269+0000","last_clean":"2024-05-10T10:34:57.857269+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:34:57.857269+0000","last_undegraded":"2024-05-10T10:34:57.857269+0000","last_fullsized":"2024-05-10T10:34:57.857269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1138688,"data_stored":825039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":241838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":565248,"data_stored":241998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1159168,"data_stored":832262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:35:01.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:35:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.062+0000 7f58c5567640 1 Processor -- start 2024-05-10T10:35:02.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.062+0000 7f58c5567640 1 -- start start 2024-05-10T10:35:02.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58c5567640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f58c00d3d80 0x7f58c00b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58c5567640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 0x7f58c00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58c5567640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c00d2d20 con 0x7f58c0162940 2024-05-10T10:35:02.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58c5567640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f58c0162d20 con 0x7f58c00d3d80 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58c5567640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00d3000 con 0x7f58c00d59d0 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58beffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f58c0162940 0x7f58c00d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:32838/0 (socket says 172.21.15.53:32838) 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58beffd640 1 -- 172.21.15.53:0/1391463426 learned_addr learned my addr 172.21.15.53:0/1391463426 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073535109 0 0) 0x7f58c00d2d20 con 0x7f58c0162940 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58ac003520 con 0x7f58c0162940 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58bf7fe640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 0x7f58c00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58b7fff640 1 --2- 172.21.15.53:0/1391463426 >> v2:172.21.15.135:3300/0 conn(0x7f58c00d3d80 0x7f58c00b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3597085112 0 0) 0x7f58ac003520 con 0x7f58c0162940 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 msgr2=0x7f58c00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 0x7f58c00d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 >> v2:172.21.15.135:3300/0 conn(0x7f58c00d3d80 msgr2=0x7f58c00b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 --2- 172.21.15.53:0/1391463426 >> v2:172.21.15.135:3300/0 conn(0x7f58c00d3d80 0x7f58c00b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c00d32e0 con 0x7f58c0162940 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f58b0001670 con 0x7f58c0162940 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58b7fff640 1 --2- 172.21.15.53:0/1391463426 >> v2:172.21.15.135:3300/0 conn(0x7f58c00d3d80 0x7f58c00b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 >> v1:172.21.15.53:6789/0 conn(0x7f58c0162940 legacy=0x7f58c00d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 0x7f58ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ac004d40 0x7f58ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00d3000 con 0x7f58ac003f30 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac00ba70 con 0x7f58ac004d40 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.063+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac00bc40 con 0x7f58c00d59d0 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58b7fff640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58beffd640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 0x7f58ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58bf7fe640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ac004d40 0x7f58ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58b7fff640 1 -- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 msgr2=0x7f58ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58b7fff640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00d59d0 0x7f58ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58b7fff640 1 -- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ac004d40 msgr2=0x7f58ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58b7fff640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ac004d40 0x7f58ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58b7fff640 1 -- 172.21.15.53:0/1391463426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac003520 con 0x7f58ac003f30 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58bf7fe640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58ac004d40 0x7f58ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.064+0000 7f58b7fff640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58ac004800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f58a80098c0 tx=0x7f58ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c945f7c0fc162fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8037070 con 0x7f58ac003f30 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a802ad50 con 0x7f58ac003f30 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58be7fc640 1 -- 172.21.15.53:0/1391463426 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a800f040 con 0x7f58ac003f30 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58c5567640 1 -- 172.21.15.53:0/1391463426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 msgr2=0x7f58ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58c5567640 1 --2- 172.21.15.53:0/1391463426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58ac004800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f58a80098c0 tx=0x7f58ac005830 comp rx=0 tx=0).stop 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58bf7fe640 1 -- 172.21.15.53:0/1391463426 reap_dead start 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58c5567640 1 -- 172.21.15.53:0/1391463426 shutdown_connections 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58c5567640 1 -- 172.21.15.53:0/1391463426 >> 172.21.15.53:0/1391463426 conn(0x7f58c009e380 msgr2=0x7f58c00d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58c5567640 1 -- 172.21.15.53:0/1391463426 shutdown_connections 2024-05-10T10:35:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.065+0000 7f58c5567640 1 -- 172.21.15.53:0/1391463426 wait complete. 2024-05-10T10:35:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.066+0000 7f58c5567640 1 Processor -- start 2024-05-10T10:35:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58c5567640 1 -- start start 2024-05-10T10:35:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58c5567640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58c00b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58c5567640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00c8b90 0x7f58c00b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58c5567640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c00ba260 0x7f58c0160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58c5567640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00d2390 con 0x7f58ac003f30 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58c5567640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00cfc50 con 0x7f58c00ba260 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58c5567640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00d2510 con 0x7f58c00c8b90 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58beffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58c00b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58beffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58c00b7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51654/0 (socket says 172.21.15.53:51654) 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.067+0000 7f58beffd640 1 -- 172.21.15.53:0/2225002486 learned_addr learned my addr 172.21.15.53:0/2225002486 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58beffd640 1 -- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00c8b90 msgr2=0x7f58c00b9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58beffd640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00c8b90 0x7f58c00b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58beffd640 1 -- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c00ba260 msgr2=0x7f58c0160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58bf7fe640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c00ba260 0x7f58c0160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58beffd640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c00ba260 0x7f58c0160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58beffd640 1 -- 172.21.15.53:0/2225002486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c00ba710 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58bf7fe640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c00ba260 0x7f58c0160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58beffd640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58c00b7de0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e780 tx=0x7f58b000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58b77fe640 1 -- 172.21.15.53:0/2225002486 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000c8d0 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58b77fe640 1 -- 172.21.15.53:0/2225002486 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b0018070 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c00ba9e0 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58b77fe640 1 -- 172.21.15.53:0/2225002486 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000ccd0 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.068+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c0160210 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.069+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c006a220 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.070+0000 7f58b77fe640 1 -- 172.21.15.53:0/2225002486 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58b001d460 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.070+0000 7f58b77fe640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58a00762d0 0x7f58a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.070+0000 7f58b77fe640 1 -- 172.21.15.53:0/2225002486 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f58b00976b0 con 0x7f58ac003f30 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.070+0000 7f58b7fff640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58a00762d0 0x7f58a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.071+0000 7f58b7fff640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58a00762d0 0x7f58a0078790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f58a8034000 tx=0x7f58a802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.075+0000 7f58b77fe640 1 -- 172.21.15.53:0/2225002486 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f58b0060e80 con 0x7f58ac003f30 2024-05-10T10:35:02.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.261+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58c00d2730 con 0x7f58a00762d0 2024-05-10T10:35:02.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.268+0000 7f58b77fe640 1 -- 172.21.15.53:0/2225002486 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251875 (secure 0 0 0) 0x7f58c00d2730 con 0x7f58a00762d0 2024-05-10T10:35:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:02.272 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58a00762d0 msgr2=0x7f58a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58a00762d0 0x7f58a0078790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f58a8034000 tx=0x7f58a802c000 comp rx=0 tx=0).stop 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 msgr2=0x7f58c00b7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58c00b7de0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e780 tx=0x7f58b000ec40 comp rx=0 tx=0).stop 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 shutdown_connections 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58a00762d0 0x7f58a0078790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c00ba260 0x7f58c0160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58c00c8b90 0x7f58c00b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 --2- 172.21.15.53:0/2225002486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003f30 0x7f58c00b7de0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 >> 172.21.15.53:0/2225002486 conn(0x7f58c009e380 msgr2=0x7f58c00d3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 shutdown_connections 2024-05-10T10:35:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.271+0000 7f58c5567640 1 -- 172.21.15.53:0/2225002486 wait complete. 2024-05-10T10:35:02.283 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-05-10T10:35:02.175931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238288,"kb_used_data":5468,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736432,"statfs":{"total":773094113280,"available":772850106368,"internally_reserved":0,"allocated":5599232,"data_stored":3079821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001442"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873310+0000","last_change":"2024-05-10T10:34:57.871752+0000","last_active":"2024-05-10T10:34:57.873310+0000","last_peered":"2024-05-10T10:34:57.873310+0000","last_clean":"2024-05-10T10:34:57.873310+0000","last_became_active":"2024-05-10T10:34:57.871233+0000","last_became_peered":"2024-05-10T10:34:57.871233+0000","last_unstale":"2024-05-10T10:34:57.873310+0000","last_undegraded":"2024-05-10T10:34:57.873310+0000","last_fullsized":"2024-05-10T10:34:57.873310+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:51.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877036+0000","last_change":"2024-05-10T10:34:57.871584+0000","last_active":"2024-05-10T10:34:57.877036+0000","last_peered":"2024-05-10T10:34:57.877036+0000","last_clean":"2024-05-10T10:34:57.877036+0000","last_became_active":"2024-05-10T10:34:57.871310+0000","last_became_peered":"2024-05-10T10:34:57.871310+0000","last_unstale":"2024-05-10T10:34:57.877036+0000","last_undegraded":"2024-05-10T10:34:57.877036+0000","last_fullsized":"2024-05-10T10:34:57.877036+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:22.457663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882202+0000","last_change":"2024-05-10T10:34:57.877770+0000","last_active":"2024-05-10T10:34:57.882202+0000","last_peered":"2024-05-10T10:34:57.882202+0000","last_clean":"2024-05-10T10:34:57.882202+0000","last_became_active":"2024-05-10T10:34:57.877470+0000","last_became_peered":"2024-05-10T10:34:57.877470+0000","last_unstale":"2024-05-10T10:34:57.882202+0000","last_undegraded":"2024-05-10T10:34:57.882202+0000","last_fullsized":"2024-05-10T10:34:57.882202+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:51:39.244051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879883+0000","last_change":"2024-05-10T10:34:57.873074+0000","last_active":"2024-05-10T10:34:57.879883+0000","last_peered":"2024-05-10T10:34:57.879883+0000","last_clean":"2024-05-10T10:34:57.879883+0000","last_became_active":"2024-05-10T10:34:57.872825+0000","last_became_peered":"2024-05-10T10:34:57.872825+0000","last_unstale":"2024-05-10T10:34:57.879883+0000","last_undegraded":"2024-05-10T10:34:57.879883+0000","last_fullsized":"2024-05-10T10:34:57.879883+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877024+0000","last_change":"2024-05-10T10:34:57.868765+0000","last_active":"2024-05-10T10:34:57.877024+0000","last_peered":"2024-05-10T10:34:57.877024+0000","last_clean":"2024-05-10T10:34:57.877024+0000","last_became_active":"2024-05-10T10:34:57.868456+0000","last_became_peered":"2024-05-10T10:34:57.868456+0000","last_unstale":"2024-05-10T10:34:57.877024+0000","last_undegraded":"2024-05-10T10:34:57.877024+0000","last_fullsized":"2024-05-10T10:34:57.877024+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:37.953833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.863585+0000","last_change":"2024-05-10T10:34:57.862747+0000","last_active":"2024-05-10T10:34:57.863585+0000","last_peered":"2024-05-10T10:34:57.863585+0000","last_clean":"2024-05-10T10:34:57.863585+0000","last_became_active":"2024-05-10T10:34:57.862465+0000","last_became_peered":"2024-05-10T10:34:57.862465+0000","last_unstale":"2024-05-10T10:34:57.863585+0000","last_undegraded":"2024-05-10T10:34:57.863585+0000","last_fullsized":"2024-05-10T10:34:57.863585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:08.508199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882162+0000","last_change":"2024-05-10T10:34:57.877152+0000","last_active":"2024-05-10T10:34:57.882162+0000","last_peered":"2024-05-10T10:34:57.882162+0000","last_clean":"2024-05-10T10:34:57.882162+0000","last_became_active":"2024-05-10T10:34:57.876804+0000","last_became_peered":"2024-05-10T10:34:57.876804+0000","last_unstale":"2024-05-10T10:34:57.882162+0000","last_undegraded":"2024-05-10T10:34:57.882162+0000","last_fullsized":"2024-05-10T10:34:57.882162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:18.842628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873937+0000","last_change":"2024-05-10T10:34:57.864536+0000","last_active":"2024-05-10T10:34:57.873937+0000","last_peered":"2024-05-10T10:34:57.873937+0000","last_clean":"2024-05-10T10:34:57.873937+0000","last_became_active":"2024-05-10T10:34:57.864285+0000","last_became_peered":"2024-05-10T10:34:57.864285+0000","last_unstale":"2024-05-10T10:34:57.873937+0000","last_undegraded":"2024-05-10T10:34:57.873937+0000","last_fullsized":"2024-05-10T10:34:57.873937+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:34:11.896217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872326+0000","last_change":"2024-05-10T10:34:57.868059+0000","last_active":"2024-05-10T10:34:57.872326+0000","last_peered":"2024-05-10T10:34:57.872326+0000","last_clean":"2024-05-10T10:34:57.872326+0000","last_became_active":"2024-05-10T10:34:57.867717+0000","last_became_peered":"2024-05-10T10:34:57.867717+0000","last_unstale":"2024-05-10T10:34:57.872326+0000","last_undegraded":"2024-05-10T10:34:57.872326+0000","last_fullsized":"2024-05-10T10:34:57.872326+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:05:27.162854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884324+0000","last_change":"2024-05-10T10:34:57.882295+0000","last_active":"2024-05-10T10:34:57.884324+0000","last_peered":"2024-05-10T10:34:57.884324+0000","last_clean":"2024-05-10T10:34:57.884324+0000","last_became_active":"2024-05-10T10:34:57.881995+0000","last_became_peered":"2024-05-10T10:34:57.881995+0000","last_unstale":"2024-05-10T10:34:57.884324+0000","last_undegraded":"2024-05-10T10:34:57.884324+0000","last_fullsized":"2024-05-10T10:34:57.884324+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.212205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873683+0000","last_change":"2024-05-10T10:34:57.866921+0000","last_active":"2024-05-10T10:34:57.873683+0000","last_peered":"2024-05-10T10:34:57.873683+0000","last_clean":"2024-05-10T10:34:57.873683+0000","last_became_active":"2024-05-10T10:34:57.866648+0000","last_became_peered":"2024-05-10T10:34:57.866648+0000","last_unstale":"2024-05-10T10:34:57.873683+0000","last_undegraded":"2024-05-10T10:34:57.873683+0000","last_fullsized":"2024-05-10T10:34:57.873683+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:00.246868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.878058+0000","last_change":"2024-05-10T10:34:57.872641+0000","last_active":"2024-05-10T10:34:57.878058+0000","last_peered":"2024-05-10T10:34:57.878058+0000","last_clean":"2024-05-10T10:34:57.878058+0000","last_became_active":"2024-05-10T10:34:57.872279+0000","last_became_peered":"2024-05-10T10:34:57.872279+0000","last_unstale":"2024-05-10T10:34:57.878058+0000","last_undegraded":"2024-05-10T10:34:57.878058+0000","last_fullsized":"2024-05-10T10:34:57.878058+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:16.145569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.885082+0000","last_change":"2024-05-10T10:34:57.883170+0000","last_active":"2024-05-10T10:34:57.885082+0000","last_peered":"2024-05-10T10:34:57.885082+0000","last_clean":"2024-05-10T10:34:57.885082+0000","last_became_active":"2024-05-10T10:34:57.882880+0000","last_became_peered":"2024-05-10T10:34:57.882880+0000","last_unstale":"2024-05-10T10:34:57.885082+0000","last_undegraded":"2024-05-10T10:34:57.885082+0000","last_fullsized":"2024-05-10T10:34:57.885082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:44:29.426213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871918+0000","last_change":"2024-05-10T10:34:57.868800+0000","last_active":"2024-05-10T10:34:57.871918+0000","last_peered":"2024-05-10T10:34:57.871918+0000","last_clean":"2024-05-10T10:34:57.871918+0000","last_became_active":"2024-05-10T10:34:57.868500+0000","last_became_peered":"2024-05-10T10:34:57.868500+0000","last_unstale":"2024-05-10T10:34:57.871918+0000","last_undegraded":"2024-05-10T10:34:57.871918+0000","last_fullsized":"2024-05-10T10:34:57.871918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:52:08.552440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870781+0000","last_change":"2024-05-10T10:34:57.865318+0000","last_active":"2024-05-10T10:34:57.870781+0000","last_peered":"2024-05-10T10:34:57.870781+0000","last_clean":"2024-05-10T10:34:57.870781+0000","last_became_active":"2024-05-10T10:34:57.864429+0000","last_became_peered":"2024-05-10T10:34:57.864429+0000","last_unstale":"2024-05-10T10:34:57.870781+0000","last_undegraded":"2024-05-10T10:34:57.870781+0000","last_fullsized":"2024-05-10T10:34:57.870781+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:52:08.552440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.885053+0000","last_change":"2024-05-10T10:34:57.883110+0000","last_active":"2024-05-10T10:34:57.885053+0000","last_peered":"2024-05-10T10:34:57.885053+0000","last_clean":"2024-05-10T10:34:57.885053+0000","last_became_active":"2024-05-10T10:34:57.882795+0000","last_became_peered":"2024-05-10T10:34:57.882795+0000","last_unstale":"2024-05-10T10:34:57.885053+0000","last_undegraded":"2024-05-10T10:34:57.885053+0000","last_fullsized":"2024-05-10T10:34:57.885053+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:04.663516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873382+0000","last_change":"2024-05-10T10:34:57.870892+0000","last_active":"2024-05-10T10:34:57.873382+0000","last_peered":"2024-05-10T10:34:57.873382+0000","last_clean":"2024-05-10T10:34:57.873382+0000","last_became_active":"2024-05-10T10:34:57.870641+0000","last_became_peered":"2024-05-10T10:34:57.870641+0000","last_unstale":"2024-05-10T10:34:57.873382+0000","last_undegraded":"2024-05-10T10:34:57.873382+0000","last_fullsized":"2024-05-10T10:34:57.873382+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:49:37.243526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877017+0000","last_change":"2024-05-10T10:34:57.870949+0000","last_active":"2024-05-10T10:34:57.877017+0000","last_peered":"2024-05-10T10:34:57.877017+0000","last_clean":"2024-05-10T10:34:57.877017+0000","last_became_active":"2024-05-10T10:34:57.870644+0000","last_became_peered":"2024-05-10T10:34:57.870644+0000","last_unstale":"2024-05-10T10:34:57.877017+0000","last_undegraded":"2024-05-10T10:34:57.877017+0000","last_fullsized":"2024-05-10T10:34:57.877017+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:21.348868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.874588+0000","last_change":"2024-05-10T10:34:57.872700+0000","last_active":"2024-05-10T10:34:57.874588+0000","last_peered":"2024-05-10T10:34:57.874588+0000","last_clean":"2024-05-10T10:34:57.874588+0000","last_became_active":"2024-05-10T10:34:57.872404+0000","last_became_peered":"2024-05-10T10:34:57.872404+0000","last_unstale":"2024-05-10T10:34:57.874588+0000","last_undegraded":"2024-05-10T10:34:57.874588+0000","last_fullsized":"2024-05-10T10:34:57.874588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:21.348868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872214+0000","last_change":"2024-05-10T10:34:57.867885+0000","last_active":"2024-05-10T10:34:57.872214+0000","last_peered":"2024-05-10T10:34:57.872214+0000","last_clean":"2024-05-10T10:34:57.872214+0000","last_became_active":"2024-05-10T10:34:57.867584+0000","last_became_peered":"2024-05-10T10:34:57.867584+0000","last_unstale":"2024-05-10T10:34:57.872214+0000","last_undegraded":"2024-05-10T10:34:57.872214+0000","last_fullsized":"2024-05-10T10:34:57.872214+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:49.016906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872168+0000","last_change":"2024-05-10T10:34:57.867423+0000","last_active":"2024-05-10T10:34:57.872168+0000","last_peered":"2024-05-10T10:34:57.872168+0000","last_clean":"2024-05-10T10:34:57.872168+0000","last_became_active":"2024-05-10T10:34:57.867126+0000","last_became_peered":"2024-05-10T10:34:57.867126+0000","last_unstale":"2024-05-10T10:34:57.872168+0000","last_undegraded":"2024-05-10T10:34:57.872168+0000","last_fullsized":"2024-05-10T10:34:57.872168+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:54.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872138+0000","last_change":"2024-05-10T10:34:57.867242+0000","last_active":"2024-05-10T10:34:57.872138+0000","last_peered":"2024-05-10T10:34:57.872138+0000","last_clean":"2024-05-10T10:34:57.872138+0000","last_became_active":"2024-05-10T10:34:57.866936+0000","last_became_peered":"2024-05-10T10:34:57.866936+0000","last_unstale":"2024-05-10T10:34:57.872138+0000","last_undegraded":"2024-05-10T10:34:57.872138+0000","last_fullsized":"2024-05-10T10:34:57.872138+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:59:20.026079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.874908+0000","last_change":"2024-05-10T10:34:57.869693+0000","last_active":"2024-05-10T10:34:57.874908+0000","last_peered":"2024-05-10T10:34:57.874908+0000","last_clean":"2024-05-10T10:34:57.874908+0000","last_became_active":"2024-05-10T10:34:57.869448+0000","last_became_peered":"2024-05-10T10:34:57.869448+0000","last_unstale":"2024-05-10T10:34:57.874908+0000","last_undegraded":"2024-05-10T10:34:57.874908+0000","last_fullsized":"2024-05-10T10:34:57.874908+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:35:10.081984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.875855+0000","last_change":"2024-05-10T10:34:57.871734+0000","last_active":"2024-05-10T10:34:57.875855+0000","last_peered":"2024-05-10T10:34:57.875855+0000","last_clean":"2024-05-10T10:34:57.875855+0000","last_became_active":"2024-05-10T10:34:57.871377+0000","last_became_peered":"2024-05-10T10:34:57.871377+0000","last_unstale":"2024-05-10T10:34:57.875855+0000","last_undegraded":"2024-05-10T10:34:57.875855+0000","last_fullsized":"2024-05-10T10:34:57.875855+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:01:13.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877007+0000","last_change":"2024-05-10T10:34:57.874707+0000","last_active":"2024-05-10T10:34:57.877007+0000","last_peered":"2024-05-10T10:34:57.877007+0000","last_clean":"2024-05-10T10:34:57.877007+0000","last_became_active":"2024-05-10T10:34:57.874327+0000","last_became_peered":"2024-05-10T10:34:57.874327+0000","last_unstale":"2024-05-10T10:34:57.877007+0000","last_undegraded":"2024-05-10T10:34:57.877007+0000","last_fullsized":"2024-05-10T10:34:57.877007+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:09.598550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872048+0000","last_change":"2024-05-10T10:34:57.863732+0000","last_active":"2024-05-10T10:34:57.872048+0000","last_peered":"2024-05-10T10:34:57.872048+0000","last_clean":"2024-05-10T10:34:57.872048+0000","last_became_active":"2024-05-10T10:34:57.863459+0000","last_became_peered":"2024-05-10T10:34:57.863459+0000","last_unstale":"2024-05-10T10:34:57.872048+0000","last_undegraded":"2024-05-10T10:34:57.872048+0000","last_fullsized":"2024-05-10T10:34:57.872048+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:18.842628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873721+0000","last_change":"2024-05-10T10:34:57.870768+0000","last_active":"2024-05-10T10:34:57.873721+0000","last_peered":"2024-05-10T10:34:57.873721+0000","last_clean":"2024-05-10T10:34:57.873721+0000","last_became_active":"2024-05-10T10:34:57.870408+0000","last_became_peered":"2024-05-10T10:34:57.870408+0000","last_unstale":"2024-05-10T10:34:57.873721+0000","last_undegraded":"2024-05-10T10:34:57.873721+0000","last_fullsized":"2024-05-10T10:34:57.873721+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:04.784690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880061+0000","last_change":"2024-05-10T10:34:57.874955+0000","last_active":"2024-05-10T10:34:57.880061+0000","last_peered":"2024-05-10T10:34:57.880061+0000","last_clean":"2024-05-10T10:34:57.880061+0000","last_became_active":"2024-05-10T10:34:57.874646+0000","last_became_peered":"2024-05-10T10:34:57.874646+0000","last_unstale":"2024-05-10T10:34:57.880061+0000","last_undegraded":"2024-05-10T10:34:57.880061+0000","last_fullsized":"2024-05-10T10:34:57.880061+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:17.212205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870193+0000","last_change":"2024-05-10T10:34:57.865667+0000","last_active":"2024-05-10T10:34:57.870193+0000","last_peered":"2024-05-10T10:34:57.870193+0000","last_clean":"2024-05-10T10:34:57.870193+0000","last_became_active":"2024-05-10T10:34:57.865368+0000","last_became_peered":"2024-05-10T10:34:57.865368+0000","last_unstale":"2024-05-10T10:34:57.870193+0000","last_undegraded":"2024-05-10T10:34:57.870193+0000","last_fullsized":"2024-05-10T10:34:57.870193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:04.663516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870872+0000","last_change":"2024-05-10T10:34:57.866422+0000","last_active":"2024-05-10T10:34:57.870872+0000","last_peered":"2024-05-10T10:34:57.870872+0000","last_clean":"2024-05-10T10:34:57.870872+0000","last_became_active":"2024-05-10T10:34:57.866189+0000","last_became_peered":"2024-05-10T10:34:57.866189+0000","last_unstale":"2024-05-10T10:34:57.870872+0000","last_undegraded":"2024-05-10T10:34:57.870872+0000","last_fullsized":"2024-05-10T10:34:57.870872+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:01:13.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881345+0000","last_change":"2024-05-10T10:34:57.876495+0000","last_active":"2024-05-10T10:34:57.881345+0000","last_peered":"2024-05-10T10:34:57.881345+0000","last_clean":"2024-05-10T10:34:57.881345+0000","last_became_active":"2024-05-10T10:34:57.876137+0000","last_became_peered":"2024-05-10T10:34:57.876137+0000","last_unstale":"2024-05-10T10:34:57.881345+0000","last_undegraded":"2024-05-10T10:34:57.881345+0000","last_fullsized":"2024-05-10T10:34:57.881345+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880333+0000","last_change":"2024-05-10T10:34:57.875770+0000","last_active":"2024-05-10T10:34:57.880333+0000","last_peered":"2024-05-10T10:34:57.880333+0000","last_clean":"2024-05-10T10:34:57.880333+0000","last_became_active":"2024-05-10T10:34:57.875555+0000","last_became_peered":"2024-05-10T10:34:57.875555+0000","last_unstale":"2024-05-10T10:34:57.880333+0000","last_undegraded":"2024-05-10T10:34:57.880333+0000","last_fullsized":"2024-05-10T10:34:57.880333+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:38:18.842628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877339+0000","last_change":"2024-05-10T10:34:57.872086+0000","last_active":"2024-05-10T10:34:57.877339+0000","last_peered":"2024-05-10T10:34:57.877339+0000","last_clean":"2024-05-10T10:34:57.877339+0000","last_became_active":"2024-05-10T10:34:57.871777+0000","last_became_peered":"2024-05-10T10:34:57.871777+0000","last_unstale":"2024-05-10T10:34:57.877339+0000","last_undegraded":"2024-05-10T10:34:57.877339+0000","last_fullsized":"2024-05-10T10:34:57.877339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:58.298444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877772+0000","last_change":"2024-05-10T10:34:56.859545+0000","last_active":"2024-05-10T10:34:57.877772+0000","last_peered":"2024-05-10T10:34:57.877772+0000","last_clean":"2024-05-10T10:34:57.877772+0000","last_became_active":"2024-05-10T10:34:56.859226+0000","last_became_peered":"2024-05-10T10:34:56.859226+0000","last_unstale":"2024-05-10T10:34:57.877772+0000","last_undegraded":"2024-05-10T10:34:57.877772+0000","last_fullsized":"2024-05-10T10:34:57.877772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:53:59.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873839+0000","last_change":"2024-05-10T10:34:57.864324+0000","last_active":"2024-05-10T10:34:57.873839+0000","last_peered":"2024-05-10T10:34:57.873839+0000","last_clean":"2024-05-10T10:34:57.873839+0000","last_became_active":"2024-05-10T10:34:57.864022+0000","last_became_peered":"2024-05-10T10:34:57.864022+0000","last_unstale":"2024-05-10T10:34:57.873839+0000","last_undegraded":"2024-05-10T10:34:57.873839+0000","last_fullsized":"2024-05-10T10:34:57.873839+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:22.457663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884311+0000","last_change":"2024-05-10T10:34:56.867469+0000","last_active":"2024-05-10T10:34:57.884311+0000","last_peered":"2024-05-10T10:34:57.884311+0000","last_clean":"2024-05-10T10:34:57.884311+0000","last_became_active":"2024-05-10T10:34:56.867176+0000","last_became_peered":"2024-05-10T10:34:56.867176+0000","last_unstale":"2024-05-10T10:34:57.884311+0000","last_undegraded":"2024-05-10T10:34:57.884311+0000","last_fullsized":"2024-05-10T10:34:57.884311+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:36.951524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872240+0000","last_change":"2024-05-10T10:34:57.869886+0000","last_active":"2024-05-10T10:34:57.872240+0000","last_peered":"2024-05-10T10:34:57.872240+0000","last_clean":"2024-05-10T10:34:57.872240+0000","last_became_active":"2024-05-10T10:34:57.869609+0000","last_became_peered":"2024-05-10T10:34:57.869609+0000","last_unstale":"2024-05-10T10:34:57.872240+0000","last_undegraded":"2024-05-10T10:34:57.872240+0000","last_fullsized":"2024-05-10T10:34:57.872240+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:04.784690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881954+0000","last_change":"2024-05-10T10:34:56.861449+0000","last_active":"2024-05-10T10:34:57.881954+0000","last_peered":"2024-05-10T10:34:57.881954+0000","last_clean":"2024-05-10T10:34:57.881954+0000","last_became_active":"2024-05-10T10:34:56.861155+0000","last_became_peered":"2024-05-10T10:34:56.861155+0000","last_unstale":"2024-05-10T10:34:57.881954+0000","last_undegraded":"2024-05-10T10:34:57.881954+0000","last_fullsized":"2024-05-10T10:34:57.881954+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:04.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881264+0000","last_change":"2024-05-10T10:34:57.875953+0000","last_active":"2024-05-10T10:34:57.881264+0000","last_peered":"2024-05-10T10:34:57.881264+0000","last_clean":"2024-05-10T10:34:57.881264+0000","last_became_active":"2024-05-10T10:34:57.875434+0000","last_became_peered":"2024-05-10T10:34:57.875434+0000","last_unstale":"2024-05-10T10:34:57.881264+0000","last_undegraded":"2024-05-10T10:34:57.881264+0000","last_fullsized":"2024-05-10T10:34:57.881264+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:29:57.259011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.881773+0000","last_change":"2024-05-10T10:34:56.860462+0000","last_active":"2024-05-10T10:34:57.881773+0000","last_peered":"2024-05-10T10:34:57.881773+0000","last_clean":"2024-05-10T10:34:57.881773+0000","last_became_active":"2024-05-10T10:34:56.860067+0000","last_became_peered":"2024-05-10T10:34:56.860067+0000","last_unstale":"2024-05-10T10:34:57.881773+0000","last_undegraded":"2024-05-10T10:34:57.881773+0000","last_fullsized":"2024-05-10T10:34:57.881773+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:52:07.550131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877103+0000","last_change":"2024-05-10T10:34:57.870462+0000","last_active":"2024-05-10T10:34:57.877103+0000","last_peered":"2024-05-10T10:34:57.877103+0000","last_clean":"2024-05-10T10:34:57.877103+0000","last_became_active":"2024-05-10T10:34:57.868751+0000","last_became_peered":"2024-05-10T10:34:57.868751+0000","last_unstale":"2024-05-10T10:34:57.877103+0000","last_undegraded":"2024-05-10T10:34:57.877103+0000","last_fullsized":"2024-05-10T10:34:57.877103+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:08.508199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884009+0000","last_change":"2024-05-10T10:34:56.864551+0000","last_active":"2024-05-10T10:34:57.884009+0000","last_peered":"2024-05-10T10:34:57.884009+0000","last_clean":"2024-05-10T10:34:57.884009+0000","last_became_active":"2024-05-10T10:34:56.864051+0000","last_became_peered":"2024-05-10T10:34:56.864051+0000","last_unstale":"2024-05-10T10:34:57.884009+0000","last_undegraded":"2024-05-10T10:34:57.884009+0000","last_fullsized":"2024-05-10T10:34:57.884009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:50.072960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871679+0000","last_change":"2024-05-10T10:34:57.865586+0000","last_active":"2024-05-10T10:34:57.871679+0000","last_peered":"2024-05-10T10:34:57.871679+0000","last_clean":"2024-05-10T10:34:57.871679+0000","last_became_active":"2024-05-10T10:34:57.865246+0000","last_became_peered":"2024-05-10T10:34:57.865246+0000","last_unstale":"2024-05-10T10:34:57.871679+0000","last_undegraded":"2024-05-10T10:34:57.871679+0000","last_fullsized":"2024-05-10T10:34:57.871679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:49:37.243526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879905+0000","last_change":"2024-05-10T10:34:56.861850+0000","last_active":"2024-05-10T10:34:57.879905+0000","last_peered":"2024-05-10T10:34:57.879905+0000","last_clean":"2024-05-10T10:34:57.879905+0000","last_became_active":"2024-05-10T10:34:56.861284+0000","last_became_peered":"2024-05-10T10:34:56.861284+0000","last_unstale":"2024-05-10T10:34:57.879905+0000","last_undegraded":"2024-05-10T10:34:57.879905+0000","last_fullsized":"2024-05-10T10:34:57.879905+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:52:05.287263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.869252+0000","last_change":"2024-05-10T10:34:57.861124+0000","last_active":"2024-05-10T10:34:57.869252+0000","last_peered":"2024-05-10T10:34:57.869252+0000","last_clean":"2024-05-10T10:34:57.869252+0000","last_became_active":"2024-05-10T10:34:57.860834+0000","last_became_peered":"2024-05-10T10:34:57.860834+0000","last_unstale":"2024-05-10T10:34:57.869252+0000","last_undegraded":"2024-05-10T10:34:57.869252+0000","last_fullsized":"2024-05-10T10:34:57.869252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.861657+0000","last_change":"2024-05-10T10:34:56.860380+0000","last_active":"2024-05-10T10:34:57.861657+0000","last_peered":"2024-05-10T10:34:57.861657+0000","last_clean":"2024-05-10T10:34:57.861657+0000","last_became_active":"2024-05-10T10:34:56.860033+0000","last_became_peered":"2024-05-10T10:34:56.860033+0000","last_unstale":"2024-05-10T10:34:57.861657+0000","last_undegraded":"2024-05-10T10:34:57.861657+0000","last_fullsized":"2024-05-10T10:34:57.861657+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:29.174564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.876940+0000","last_change":"2024-05-10T10:34:57.871487+0000","last_active":"2024-05-10T10:34:57.876940+0000","last_peered":"2024-05-10T10:34:57.876940+0000","last_clean":"2024-05-10T10:34:57.876940+0000","last_became_active":"2024-05-10T10:34:57.871211+0000","last_became_peered":"2024-05-10T10:34:57.871211+0000","last_unstale":"2024-05-10T10:34:57.876940+0000","last_undegraded":"2024-05-10T10:34:57.876940+0000","last_fullsized":"2024-05-10T10:34:57.876940+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:09.598550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.855449+0000","last_change":"2024-05-10T10:34:56.872434+0000","last_active":"2024-05-10T10:34:57.855449+0000","last_peered":"2024-05-10T10:34:57.855449+0000","last_clean":"2024-05-10T10:34:57.855449+0000","last_became_active":"2024-05-10T10:34:56.872134+0000","last_became_peered":"2024-05-10T10:34:56.872134+0000","last_unstale":"2024-05-10T10:34:57.855449+0000","last_undegraded":"2024-05-10T10:34:57.855449+0000","last_fullsized":"2024-05-10T10:34:57.855449+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:53.787388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879960+0000","last_change":"2024-05-10T10:34:57.875401+0000","last_active":"2024-05-10T10:34:57.879960+0000","last_peered":"2024-05-10T10:34:57.879960+0000","last_clean":"2024-05-10T10:34:57.879960+0000","last_became_active":"2024-05-10T10:34:57.875099+0000","last_became_peered":"2024-05-10T10:34:57.875099+0000","last_unstale":"2024-05-10T10:34:57.879960+0000","last_undegraded":"2024-05-10T10:34:57.879960+0000","last_fullsized":"2024-05-10T10:34:57.879960+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:49.016906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.858509+0000","last_change":"2024-05-10T10:34:56.859542+0000","last_active":"2024-05-10T10:34:57.858509+0000","last_peered":"2024-05-10T10:34:57.858509+0000","last_clean":"2024-05-10T10:34:57.858509+0000","last_became_active":"2024-05-10T10:34:56.859260+0000","last_became_peered":"2024-05-10T10:34:56.859260+0000","last_unstale":"2024-05-10T10:34:57.858509+0000","last_undegraded":"2024-05-10T10:34:57.858509+0000","last_fullsized":"2024-05-10T10:34:57.858509+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:19:16.902820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.869354+0000","last_change":"2024-05-10T10:34:57.862046+0000","last_active":"2024-05-10T10:34:57.869354+0000","last_peered":"2024-05-10T10:34:57.869354+0000","last_clean":"2024-05-10T10:34:57.869354+0000","last_became_active":"2024-05-10T10:34:57.861769+0000","last_became_peered":"2024-05-10T10:34:57.861769+0000","last_unstale":"2024-05-10T10:34:57.869354+0000","last_undegraded":"2024-05-10T10:34:57.869354+0000","last_fullsized":"2024-05-10T10:34:57.869354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:01:13.097937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.861104+0000","last_change":"2024-05-10T10:34:56.863386+0000","last_active":"2024-05-10T10:34:57.861104+0000","last_peered":"2024-05-10T10:34:57.861104+0000","last_clean":"2024-05-10T10:34:57.861104+0000","last_became_active":"2024-05-10T10:34:56.863125+0000","last_became_peered":"2024-05-10T10:34:56.863125+0000","last_unstale":"2024-05-10T10:34:57.861104+0000","last_undegraded":"2024-05-10T10:34:57.861104+0000","last_fullsized":"2024-05-10T10:34:57.861104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:04.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871897+0000","last_change":"2024-05-10T10:34:57.868708+0000","last_active":"2024-05-10T10:34:57.871897+0000","last_peered":"2024-05-10T10:34:57.871897+0000","last_clean":"2024-05-10T10:34:57.871897+0000","last_became_active":"2024-05-10T10:34:57.868411+0000","last_became_peered":"2024-05-10T10:34:57.868411+0000","last_unstale":"2024-05-10T10:34:57.871897+0000","last_undegraded":"2024-05-10T10:34:57.871897+0000","last_fullsized":"2024-05-10T10:34:57.871897+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:19:17.905130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884721+0000","last_change":"2024-05-10T10:34:56.865480+0000","last_active":"2024-05-10T10:34:57.884721+0000","last_peered":"2024-05-10T10:34:57.884721+0000","last_clean":"2024-05-10T10:34:57.884721+0000","last_became_active":"2024-05-10T10:34:56.865196+0000","last_became_peered":"2024-05-10T10:34:56.865196+0000","last_unstale":"2024-05-10T10:34:57.884721+0000","last_undegraded":"2024-05-10T10:34:57.884721+0000","last_fullsized":"2024-05-10T10:34:57.884721+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:44:28.423904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877047+0000","last_change":"2024-05-10T10:34:57.870478+0000","last_active":"2024-05-10T10:34:57.877047+0000","last_peered":"2024-05-10T10:34:57.877047+0000","last_clean":"2024-05-10T10:34:57.877047+0000","last_became_active":"2024-05-10T10:34:57.869074+0000","last_became_peered":"2024-05-10T10:34:57.869074+0000","last_unstale":"2024-05-10T10:34:57.877047+0000","last_undegraded":"2024-05-10T10:34:57.877047+0000","last_fullsized":"2024-05-10T10:34:57.877047+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:54.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.885922+0000","last_change":"2024-05-10T10:34:56.867033+0000","last_active":"2024-05-10T10:34:57.885922+0000","last_peered":"2024-05-10T10:34:57.885922+0000","last_clean":"2024-05-10T10:34:57.885922+0000","last_became_active":"2024-05-10T10:34:56.866671+0000","last_became_peered":"2024-05-10T10:34:56.866671+0000","last_unstale":"2024-05-10T10:34:57.885922+0000","last_undegraded":"2024-05-10T10:34:57.885922+0000","last_fullsized":"2024-05-10T10:34:57.885922+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:51:38.241742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.869543+0000","last_change":"2024-05-10T10:34:57.862681+0000","last_active":"2024-05-10T10:34:57.869543+0000","last_peered":"2024-05-10T10:34:57.869543+0000","last_clean":"2024-05-10T10:34:57.869543+0000","last_became_active":"2024-05-10T10:34:57.862397+0000","last_became_peered":"2024-05-10T10:34:57.862397+0000","last_unstale":"2024-05-10T10:34:57.869543+0000","last_undegraded":"2024-05-10T10:34:57.869543+0000","last_fullsized":"2024-05-10T10:34:57.869543+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:21.348868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882662+0000","last_change":"2024-05-10T10:34:56.864313+0000","last_active":"2024-05-10T10:34:57.882662+0000","last_peered":"2024-05-10T10:34:57.882662+0000","last_clean":"2024-05-10T10:34:57.882662+0000","last_became_active":"2024-05-10T10:34:56.863747+0000","last_became_peered":"2024-05-10T10:34:56.863747+0000","last_unstale":"2024-05-10T10:34:57.882662+0000","last_undegraded":"2024-05-10T10:34:57.882662+0000","last_fullsized":"2024-05-10T10:34:57.882662+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:03.782381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.884256+0000","last_change":"2024-05-10T10:34:57.883318+0000","last_active":"2024-05-10T10:34:57.884256+0000","last_peered":"2024-05-10T10:34:57.884256+0000","last_clean":"2024-05-10T10:34:57.884256+0000","last_became_active":"2024-05-10T10:34:57.883052+0000","last_became_peered":"2024-05-10T10:34:57.883052+0000","last_unstale":"2024-05-10T10:34:57.884256+0000","last_undegraded":"2024-05-10T10:34:57.884256+0000","last_fullsized":"2024-05-10T10:34:57.884256+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:34:11.896217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.862375+0000","last_change":"2024-05-10T10:34:56.866513+0000","last_active":"2024-05-10T10:34:57.862375+0000","last_peered":"2024-05-10T10:34:57.862375+0000","last_clean":"2024-05-10T10:34:57.862375+0000","last_became_active":"2024-05-10T10:34:56.866212+0000","last_became_peered":"2024-05-10T10:34:56.866212+0000","last_unstale":"2024-05-10T10:34:57.862375+0000","last_undegraded":"2024-05-10T10:34:57.862375+0000","last_fullsized":"2024-05-10T10:34:57.862375+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:07.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872073+0000","last_change":"2024-05-10T10:34:57.866243+0000","last_active":"2024-05-10T10:34:57.872073+0000","last_peered":"2024-05-10T10:34:57.872073+0000","last_clean":"2024-05-10T10:34:57.872073+0000","last_became_active":"2024-05-10T10:34:57.865939+0000","last_became_peered":"2024-05-10T10:34:57.865939+0000","last_unstale":"2024-05-10T10:34:57.872073+0000","last_undegraded":"2024-05-10T10:34:57.872073+0000","last_fullsized":"2024-05-10T10:34:57.872073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:36:41.686995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857409+0000","last_change":"2024-05-10T10:34:56.861868+0000","last_active":"2024-05-10T10:34:57.857409+0000","last_peered":"2024-05-10T10:34:57.857409+0000","last_clean":"2024-05-10T10:34:57.857409+0000","last_became_active":"2024-05-10T10:34:56.861533+0000","last_became_peered":"2024-05-10T10:34:56.861533+0000","last_unstale":"2024-05-10T10:34:57.857409+0000","last_undegraded":"2024-05-10T10:34:57.857409+0000","last_fullsized":"2024-05-10T10:34:57.857409+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:03.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879003+0000","last_change":"2024-05-10T10:34:57.875798+0000","last_active":"2024-05-10T10:34:57.879003+0000","last_peered":"2024-05-10T10:34:57.879003+0000","last_clean":"2024-05-10T10:34:57.879003+0000","last_became_active":"2024-05-10T10:34:57.875524+0000","last_became_peered":"2024-05-10T10:34:57.875524+0000","last_unstale":"2024-05-10T10:34:57.879003+0000","last_undegraded":"2024-05-10T10:34:57.879003+0000","last_fullsized":"2024-05-10T10:34:57.879003+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:58.298444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879554+0000","last_change":"2024-05-10T10:34:56.861950+0000","last_active":"2024-05-10T10:34:57.879554+0000","last_peered":"2024-05-10T10:34:57.879554+0000","last_clean":"2024-05-10T10:34:57.879554+0000","last_became_active":"2024-05-10T10:34:56.861652+0000","last_became_peered":"2024-05-10T10:34:56.861652+0000","last_unstale":"2024-05-10T10:34:57.879554+0000","last_undegraded":"2024-05-10T10:34:57.879554+0000","last_fullsized":"2024-05-10T10:34:57.879554+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:20.346559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877320+0000","last_change":"2024-05-10T10:34:57.874606+0000","last_active":"2024-05-10T10:34:57.877320+0000","last_peered":"2024-05-10T10:34:57.877320+0000","last_clean":"2024-05-10T10:34:57.877320+0000","last_became_active":"2024-05-10T10:34:57.874141+0000","last_became_peered":"2024-05-10T10:34:57.874141+0000","last_unstale":"2024-05-10T10:34:57.877320+0000","last_undegraded":"2024-05-10T10:34:57.877320+0000","last_fullsized":"2024-05-10T10:34:57.877320+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:16:29.475280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.882409+0000","last_change":"2024-05-10T10:34:56.870750+0000","last_active":"2024-05-10T10:34:57.882409+0000","last_peered":"2024-05-10T10:34:57.882409+0000","last_clean":"2024-05-10T10:34:57.882409+0000","last_became_active":"2024-05-10T10:34:56.870433+0000","last_became_peered":"2024-05-10T10:34:56.870433+0000","last_unstale":"2024-05-10T10:34:57.882409+0000","last_undegraded":"2024-05-10T10:34:57.882409+0000","last_fullsized":"2024-05-10T10:34:57.882409+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:16:16.209896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.871237+0000","last_change":"2024-05-10T10:34:57.869859+0000","last_active":"2024-05-10T10:34:57.871237+0000","last_peered":"2024-05-10T10:34:57.871237+0000","last_clean":"2024-05-10T10:34:57.871237+0000","last_became_active":"2024-05-10T10:34:57.869551+0000","last_became_peered":"2024-05-10T10:34:57.869551+0000","last_unstale":"2024-05-10T10:34:57.871237+0000","last_undegraded":"2024-05-10T10:34:57.871237+0000","last_fullsized":"2024-05-10T10:34:57.871237+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:40:54.789697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879016+0000","last_change":"2024-05-10T10:34:56.859624+0000","last_active":"2024-05-10T10:34:57.879016+0000","last_peered":"2024-05-10T10:34:57.879016+0000","last_clean":"2024-05-10T10:34:57.879016+0000","last_became_active":"2024-05-10T10:34:56.859338+0000","last_became_peered":"2024-05-10T10:34:56.859338+0000","last_unstale":"2024-05-10T10:34:57.879016+0000","last_undegraded":"2024-05-10T10:34:57.879016+0000","last_fullsized":"2024-05-10T10:34:57.879016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:11.030813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877216+0000","last_change":"2024-05-10T10:34:57.870212+0000","last_active":"2024-05-10T10:34:57.877216+0000","last_peered":"2024-05-10T10:34:57.877216+0000","last_clean":"2024-05-10T10:34:57.877216+0000","last_became_active":"2024-05-10T10:34:57.868839+0000","last_became_peered":"2024-05-10T10:34:57.868839+0000","last_unstale":"2024-05-10T10:34:57.877216+0000","last_undegraded":"2024-05-10T10:34:57.877216+0000","last_fullsized":"2024-05-10T10:34:57.877216+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.176874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877015+0000","last_change":"2024-05-10T10:34:56.865559+0000","last_active":"2024-05-10T10:34:57.877015+0000","last_peered":"2024-05-10T10:34:57.877015+0000","last_clean":"2024-05-10T10:34:57.877015+0000","last_became_active":"2024-05-10T10:34:56.865287+0000","last_became_peered":"2024-05-10T10:34:56.865287+0000","last_unstale":"2024-05-10T10:34:57.877015+0000","last_undegraded":"2024-05-10T10:34:57.877015+0000","last_fullsized":"2024-05-10T10:34:57.877015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:06:36.951524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873760+0000","last_change":"2024-05-10T10:34:57.871807+0000","last_active":"2024-05-10T10:34:57.873760+0000","last_peered":"2024-05-10T10:34:57.873760+0000","last_clean":"2024-05-10T10:34:57.873760+0000","last_became_active":"2024-05-10T10:34:57.871471+0000","last_became_peered":"2024-05-10T10:34:57.871471+0000","last_unstale":"2024-05-10T10:34:57.873760+0000","last_undegraded":"2024-05-10T10:34:57.873760+0000","last_fullsized":"2024-05-10T10:34:57.873760+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:14.233463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.860224+0000","last_change":"2024-05-10T10:34:56.871293+0000","last_active":"2024-05-10T10:34:57.860224+0000","last_peered":"2024-05-10T10:34:57.860224+0000","last_clean":"2024-05-10T10:34:57.860224+0000","last_became_active":"2024-05-10T10:34:56.871022+0000","last_became_peered":"2024-05-10T10:34:56.871022+0000","last_unstale":"2024-05-10T10:34:57.860224+0000","last_undegraded":"2024-05-10T10:34:57.860224+0000","last_fullsized":"2024-05-10T10:34:57.860224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:04.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.874661+0000","last_change":"2024-05-10T10:34:57.868372+0000","last_active":"2024-05-10T10:34:57.874661+0000","last_peered":"2024-05-10T10:34:57.874661+0000","last_clean":"2024-05-10T10:34:57.874661+0000","last_became_active":"2024-05-10T10:34:57.868059+0000","last_became_peered":"2024-05-10T10:34:57.868059+0000","last_unstale":"2024-05-10T10:34:57.874661+0000","last_undegraded":"2024-05-10T10:34:57.874661+0000","last_fullsized":"2024-05-10T10:34:57.874661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:58.298444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.883498+0000","last_change":"2024-05-10T10:34:56.868593+0000","last_active":"2024-05-10T10:34:57.883498+0000","last_peered":"2024-05-10T10:34:57.883498+0000","last_clean":"2024-05-10T10:34:57.883498+0000","last_became_active":"2024-05-10T10:34:56.868271+0000","last_became_peered":"2024-05-10T10:34:56.868271+0000","last_unstale":"2024-05-10T10:34:57.883498+0000","last_undegraded":"2024-05-10T10:34:57.883498+0000","last_fullsized":"2024-05-10T10:34:57.883498+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:52:05.287263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.877092+0000","last_change":"2024-05-10T10:34:57.872953+0000","last_active":"2024-05-10T10:34:57.877092+0000","last_peered":"2024-05-10T10:34:57.877092+0000","last_clean":"2024-05-10T10:34:57.877092+0000","last_became_active":"2024-05-10T10:34:57.872677+0000","last_became_peered":"2024-05-10T10:34:57.872677+0000","last_unstale":"2024-05-10T10:34:57.877092+0000","last_undegraded":"2024-05-10T10:34:57.877092+0000","last_fullsized":"2024-05-10T10:34:57.877092+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:27:51.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857087+0000","last_change":"2024-05-10T10:34:56.864122+0000","last_active":"2024-05-10T10:34:57.857087+0000","last_peered":"2024-05-10T10:34:57.857087+0000","last_clean":"2024-05-10T10:34:57.857087+0000","last_became_active":"2024-05-10T10:34:56.863828+0000","last_became_peered":"2024-05-10T10:34:56.863828+0000","last_unstale":"2024-05-10T10:34:57.857087+0000","last_undegraded":"2024-05-10T10:34:57.857087+0000","last_fullsized":"2024-05-10T10:34:57.857087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:08.596241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870163+0000","last_change":"2024-05-10T10:34:57.865992+0000","last_active":"2024-05-10T10:34:57.870163+0000","last_peered":"2024-05-10T10:34:57.870163+0000","last_clean":"2024-05-10T10:34:57.870163+0000","last_became_active":"2024-05-10T10:34:57.865463+0000","last_became_peered":"2024-05-10T10:34:57.865463+0000","last_unstale":"2024-05-10T10:34:57.870163+0000","last_undegraded":"2024-05-10T10:34:57.870163+0000","last_fullsized":"2024-05-10T10:34:57.870163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:56:20.618702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.855880+0000","last_change":"2024-05-10T10:34:56.870819+0000","last_active":"2024-05-10T10:34:57.855880+0000","last_peered":"2024-05-10T10:34:57.855880+0000","last_clean":"2024-05-10T10:34:57.855880+0000","last_became_active":"2024-05-10T10:34:56.870473+0000","last_became_peered":"2024-05-10T10:34:56.870473+0000","last_unstale":"2024-05-10T10:34:57.855880+0000","last_undegraded":"2024-05-10T10:34:57.855880+0000","last_fullsized":"2024-05-10T10:34:57.855880+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:48.014597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872664+0000","last_change":"2024-05-10T10:34:57.870562+0000","last_active":"2024-05-10T10:34:57.872664+0000","last_peered":"2024-05-10T10:34:57.872664+0000","last_clean":"2024-05-10T10:34:57.872664+0000","last_became_active":"2024-05-10T10:34:57.870253+0000","last_became_peered":"2024-05-10T10:34:57.870253+0000","last_unstale":"2024-05-10T10:34:57.872664+0000","last_undegraded":"2024-05-10T10:34:57.872664+0000","last_fullsized":"2024-05-10T10:34:57.872664+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:49.016906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.849586+0000","last_change":"2024-05-10T10:34:56.857882+0000","last_active":"2024-05-10T10:34:57.849586+0000","last_peered":"2024-05-10T10:34:57.849586+0000","last_clean":"2024-05-10T10:34:57.849586+0000","last_became_active":"2024-05-10T10:34:56.857602+0000","last_became_peered":"2024-05-10T10:34:56.857602+0000","last_unstale":"2024-05-10T10:34:57.849586+0000","last_undegraded":"2024-05-10T10:34:57.849586+0000","last_fullsized":"2024-05-10T10:34:57.849586+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:26:09.426309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880356+0000","last_change":"2024-05-10T10:34:57.875747+0000","last_active":"2024-05-10T10:34:57.880356+0000","last_peered":"2024-05-10T10:34:57.880356+0000","last_clean":"2024-05-10T10:34:57.880356+0000","last_became_active":"2024-05-10T10:34:57.875375+0000","last_became_peered":"2024-05-10T10:34:57.875375+0000","last_unstale":"2024-05-10T10:34:57.880356+0000","last_undegraded":"2024-05-10T10:34:57.880356+0000","last_fullsized":"2024-05-10T10:34:57.880356+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:18.493297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880826+0000","last_change":"2024-05-10T10:34:56.869450+0000","last_active":"2024-05-10T10:34:57.880826+0000","last_peered":"2024-05-10T10:34:57.880826+0000","last_clean":"2024-05-10T10:34:57.880826+0000","last_became_active":"2024-05-10T10:34:56.869187+0000","last_became_peered":"2024-05-10T10:34:56.869187+0000","last_unstale":"2024-05-10T10:34:57.880826+0000","last_undegraded":"2024-05-10T10:34:57.880826+0000","last_fullsized":"2024-05-10T10:34:57.880826+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:52:05.287263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.863564+0000","last_change":"2024-05-10T10:34:57.861958+0000","last_active":"2024-05-10T10:34:57.863564+0000","last_peered":"2024-05-10T10:34:57.863564+0000","last_clean":"2024-05-10T10:34:57.863564+0000","last_became_active":"2024-05-10T10:34:57.861676+0000","last_became_peered":"2024-05-10T10:34:57.861676+0000","last_unstale":"2024-05-10T10:34:57.863564+0000","last_undegraded":"2024-05-10T10:34:57.863564+0000","last_fullsized":"2024-05-10T10:34:57.863564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:41:30.176874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.858071+0000","last_change":"2024-05-10T10:34:56.861703+0000","last_active":"2024-05-10T10:34:57.858071+0000","last_peered":"2024-05-10T10:34:57.858071+0000","last_clean":"2024-05-10T10:34:57.858071+0000","last_became_active":"2024-05-10T10:34:56.861365+0000","last_became_peered":"2024-05-10T10:34:56.861365+0000","last_unstale":"2024-05-10T10:34:57.858071+0000","last_undegraded":"2024-05-10T10:34:57.858071+0000","last_fullsized":"2024-05-10T10:34:57.858071+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:56:19.616392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.879773+0000","last_change":"2024-05-10T10:34:57.875728+0000","last_active":"2024-05-10T10:34:57.879773+0000","last_peered":"2024-05-10T10:34:57.879773+0000","last_clean":"2024-05-10T10:34:57.879773+0000","last_became_active":"2024-05-10T10:34:57.875070+0000","last_became_peered":"2024-05-10T10:34:57.875070+0000","last_unstale":"2024-05-10T10:34:57.879773+0000","last_undegraded":"2024-05-10T10:34:57.879773+0000","last_fullsized":"2024-05-10T10:34:57.879773+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:35.349340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.856832+0000","last_change":"2024-05-10T10:34:56.862064+0000","last_active":"2024-05-10T10:34:57.856832+0000","last_peered":"2024-05-10T10:34:57.856832+0000","last_clean":"2024-05-10T10:34:57.856832+0000","last_became_active":"2024-05-10T10:34:56.861795+0000","last_became_peered":"2024-05-10T10:34:56.861795+0000","last_unstale":"2024-05-10T10:34:57.856832+0000","last_undegraded":"2024-05-10T10:34:57.856832+0000","last_fullsized":"2024-05-10T10:34:57.856832+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:13.231154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.878977+0000","last_change":"2024-05-10T10:34:57.874726+0000","last_active":"2024-05-10T10:34:57.878977+0000","last_peered":"2024-05-10T10:34:57.878977+0000","last_clean":"2024-05-10T10:34:57.878977+0000","last_became_active":"2024-05-10T10:34:57.874415+0000","last_became_peered":"2024-05-10T10:34:57.874415+0000","last_unstale":"2024-05-10T10:34:57.878977+0000","last_undegraded":"2024-05-10T10:34:57.878977+0000","last_fullsized":"2024-05-10T10:34:57.878977+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:22.457663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.859095+0000","last_change":"2024-05-10T10:34:56.860754+0000","last_active":"2024-05-10T10:34:57.859095+0000","last_peered":"2024-05-10T10:34:57.859095+0000","last_clean":"2024-05-10T10:34:57.859095+0000","last_became_active":"2024-05-10T10:34:56.860381+0000","last_became_peered":"2024-05-10T10:34:56.860381+0000","last_unstale":"2024-05-10T10:34:57.859095+0000","last_undegraded":"2024-05-10T10:34:57.859095+0000","last_fullsized":"2024-05-10T10:34:57.859095+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:57.296135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.873763+0000","last_change":"2024-05-10T10:34:57.863630+0000","last_active":"2024-05-10T10:34:57.873763+0000","last_peered":"2024-05-10T10:34:57.873763+0000","last_clean":"2024-05-10T10:34:57.873763+0000","last_became_active":"2024-05-10T10:34:57.863377+0000","last_became_peered":"2024-05-10T10:34:57.863377+0000","last_unstale":"2024-05-10T10:34:57.873763+0000","last_undegraded":"2024-05-10T10:34:57.873763+0000","last_fullsized":"2024-05-10T10:34:57.873763+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:09:09.598550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857272+0000","last_change":"2024-05-10T10:34:56.868854+0000","last_active":"2024-05-10T10:34:57.857272+0000","last_peered":"2024-05-10T10:34:57.857272+0000","last_clean":"2024-05-10T10:34:57.857272+0000","last_became_active":"2024-05-10T10:34:56.868556+0000","last_became_peered":"2024-05-10T10:34:56.868556+0000","last_unstale":"2024-05-10T10:34:57.857272+0000","last_undegraded":"2024-05-10T10:34:57.857272+0000","last_fullsized":"2024-05-10T10:34:57.857272+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:21.455354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.870141+0000","last_change":"2024-05-10T10:34:57.865756+0000","last_active":"2024-05-10T10:34:57.870141+0000","last_peered":"2024-05-10T10:34:57.870141+0000","last_clean":"2024-05-10T10:34:57.870141+0000","last_became_active":"2024-05-10T10:34:57.865083+0000","last_became_peered":"2024-05-10T10:34:57.865083+0000","last_unstale":"2024-05-10T10:34:57.870141+0000","last_undegraded":"2024-05-10T10:34:57.870141+0000","last_fullsized":"2024-05-10T10:34:57.870141+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:44:29.426213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.860050+0000","last_change":"2024-05-10T10:34:56.861607+0000","last_active":"2024-05-10T10:34:57.860050+0000","last_peered":"2024-05-10T10:34:57.860050+0000","last_clean":"2024-05-10T10:34:57.860050+0000","last_became_active":"2024-05-10T10:34:56.861299+0000","last_became_peered":"2024-05-10T10:34:56.861299+0000","last_unstale":"2024-05-10T10:34:57.860050+0000","last_undegraded":"2024-05-10T10:34:57.860050+0000","last_fullsized":"2024-05-10T10:34:57.860050+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:53:59.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.872034+0000","last_change":"2024-05-10T10:34:57.866702+0000","last_active":"2024-05-10T10:34:57.872034+0000","last_peered":"2024-05-10T10:34:57.872034+0000","last_clean":"2024-05-10T10:34:57.872034+0000","last_became_active":"2024-05-10T10:34:57.866426+0000","last_became_peered":"2024-05-10T10:34:57.866426+0000","last_unstale":"2024-05-10T10:34:57.872034+0000","last_undegraded":"2024-05-10T10:34:57.872034+0000","last_fullsized":"2024-05-10T10:34:57.872034+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:39:12.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.854336+0000","last_change":"2024-05-10T10:34:56.868459+0000","last_active":"2024-05-10T10:34:57.854336+0000","last_peered":"2024-05-10T10:34:57.854336+0000","last_clean":"2024-05-10T10:34:57.854336+0000","last_became_active":"2024-05-10T10:34:56.868178+0000","last_became_peered":"2024-05-10T10:34:56.868178+0000","last_unstale":"2024-05-10T10:34:57.854336+0000","last_undegraded":"2024-05-10T10:34:57.854336+0000","last_fullsized":"2024-05-10T10:34:57.854336+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:49:36.241216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.880033+0000","last_change":"2024-05-10T10:34:57.874732+0000","last_active":"2024-05-10T10:34:57.880033+0000","last_peered":"2024-05-10T10:34:57.880033+0000","last_clean":"2024-05-10T10:34:57.880033+0000","last_became_active":"2024-05-10T10:34:57.874247+0000","last_became_peered":"2024-05-10T10:34:57.874247+0000","last_unstale":"2024-05-10T10:34:57.880033+0000","last_undegraded":"2024-05-10T10:34:57.880033+0000","last_fullsized":"2024-05-10T10:34:57.880033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:56.840371+0000","last_clean_scrub_stamp":"2024-05-10T10:34:56.840371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:50:14.233463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.854864+0000","last_change":"2024-05-10T10:34:56.871581+0000","last_active":"2024-05-10T10:34:57.854864+0000","last_peered":"2024-05-10T10:34:57.854864+0000","last_clean":"2024-05-10T10:34:57.854864+0000","last_became_active":"2024-05-10T10:34:56.871283+0000","last_became_peered":"2024-05-10T10:34:56.871283+0000","last_unstale":"2024-05-10T10:34:57.854864+0000","last_undegraded":"2024-05-10T10:34:57.854864+0000","last_fullsized":"2024-05-10T10:34:57.854864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:34:55.838062+0000","last_clean_scrub_stamp":"2024-05-10T10:34:55.838062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:58:07.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":164,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-10T10:34:57.857269+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:34:57.857269+0000","last_peered":"2024-05-10T10:34:57.857269+0000","last_clean":"2024-05-10T10:34:57.857269+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:34:57.857269+0000","last_undegraded":"2024-05-10T10:34:57.857269+0000","last_fullsized":"2024-05-10T10:34:57.857269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1138688,"data_stored":825039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":241838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":234671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":565248,"data_stored":241998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1159168,"data_stored":832262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:35:02.287 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:35:02.287 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T10:35:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.433+0000 7f8266893640 1 Processor -- start 2024-05-10T10:35:02.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.433+0000 7f8266893640 1 -- start start 2024-05-10T10:35:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f8266893640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82600d3df0 0x7f82600d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f8266893640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f82600b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f8266893640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82600d41d0 con 0x7f82600d1a10 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f8266893640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f82600cfd50 con 0x7f82600d3df0 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f8266893640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f82600b5d90 con 0x7f82600b4aa0 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f825ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82600d3df0 0x7f82600d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f825ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82600d3df0 0x7f82600d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51670/0 (socket says 172.21.15.53:51670) 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.434+0000 7f825ffff640 1 -- 172.21.15.53:0/4235634391 learned_addr learned my addr 172.21.15.53:0/4235634391 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.435+0000 7f825f7fe640 1 --2- 172.21.15.53:0/4235634391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f82600b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.435+0000 7f825ffff640 1 -- 172.21.15.53:0/4235634391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 msgr2=0x7f82600b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.435+0000 7f825ffff640 1 --2- 172.21.15.53:0/4235634391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f82600b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.435+0000 7f825ffff640 1 -- 172.21.15.53:0/4235634391 >> v1:172.21.15.135:6789/0 conn(0x7f82600d1a10 legacy=0x7f82600d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.435+0000 7f825ffff640 1 -- 172.21.15.53:0/4235634391 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600b54f0 con 0x7f82600d3df0 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.435+0000 7f825ffff640 1 --2- 172.21.15.53:0/4235634391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d3df0 0x7f82600d2d10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8250009870 tx=0x7f825002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e6706ae024af0d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f825effd640 1 -- 172.21.15.53:0/4235634391 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250037070 con 0x7f82600d3df0 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f825effd640 1 -- 172.21.15.53:0/4235634391 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8250032440 con 0x7f82600d3df0 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f825effd640 1 -- 172.21.15.53:0/4235634391 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250032810 con 0x7f82600d3df0 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 -- 172.21.15.53:0/4235634391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d3df0 msgr2=0x7f82600d2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 --2- 172.21.15.53:0/4235634391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d3df0 0x7f82600d2d10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8250009870 tx=0x7f825002a270 comp rx=0 tx=0).stop 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 -- 172.21.15.53:0/4235634391 shutdown_connections 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 --2- 172.21.15.53:0/4235634391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f82600b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 --2- 172.21.15.53:0/4235634391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d3df0 0x7f82600d2d10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 -- 172.21.15.53:0/4235634391 >> 172.21.15.53:0/4235634391 conn(0x7f826009e450 msgr2=0x7f82600d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 -- 172.21.15.53:0/4235634391 shutdown_connections 2024-05-10T10:35:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.436+0000 7f8266893640 1 -- 172.21.15.53:0/4235634391 wait complete. 2024-05-10T10:35:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.437+0000 7f8266893640 1 Processor -- start 2024-05-10T10:35:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f8266893640 1 -- start start 2024-05-10T10:35:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f8266893640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f8260150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f8266893640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d1a10 0x7f82601ea370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f8266893640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82600d3df0 0x7f826014efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f8266893640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82600cff80 con 0x7f82600d1a10 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f8266893640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82600b5ab0 con 0x7f82600d3df0 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f8266893640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f82600b6120 con 0x7f82600b4aa0 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f825f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d1a10 0x7f82601ea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f825f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d1a10 0x7f82601ea370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51676/0 (socket says 172.21.15.53:51676) 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.438+0000 7f825f7fe640 1 -- 172.21.15.53:0/3580017962 learned_addr learned my addr 172.21.15.53:0/3580017962 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825ffff640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f8260150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f8264e30640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82600d3df0 0x7f826014efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825f7fe640 1 -- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 msgr2=0x7f8260150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825f7fe640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f8260150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825f7fe640 1 -- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82600d3df0 msgr2=0x7f826014efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825f7fe640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82600d3df0 0x7f826014efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825f7fe640 1 -- 172.21.15.53:0/3580017962 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82601ea8b0 con 0x7f82600d1a10 2024-05-10T10:35:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825f7fe640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d1a10 0x7f82601ea370 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f825400d830 tx=0x7f825400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825cff9640 1 -- 172.21.15.53:0/3580017962 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825400b6c0 con 0x7f82600d1a10 2024-05-10T10:35:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825cff9640 1 -- 172.21.15.53:0/3580017962 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8254019070 con 0x7f82600d1a10 2024-05-10T10:35:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826014f860 con 0x7f82600d1a10 2024-05-10T10:35:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f825cff9640 1 -- 172.21.15.53:0/3580017962 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825400ba80 con 0x7f82600d1a10 2024-05-10T10:35:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.439+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826014fe20 con 0x7f82600d1a10 2024-05-10T10:35:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.440+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826006a220 con 0x7f82600d1a10 2024-05-10T10:35:02.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.441+0000 7f825cff9640 1 -- 172.21.15.53:0/3580017962 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f825400bc20 con 0x7f82600d1a10 2024-05-10T10:35:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.441+0000 7f825cff9640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f824c0762d0 0x7f824c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.442+0000 7f825cff9640 1 -- 172.21.15.53:0/3580017962 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f8254097d00 con 0x7f82600d1a10 2024-05-10T10:35:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.442+0000 7f825ffff640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f824c0762d0 0x7f824c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.442+0000 7f825ffff640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f824c0762d0 0x7f824c078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8250034000 tx=0x7f825002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.446+0000 7f825cff9640 1 -- 172.21.15.53:0/3580017962 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8254061440 con 0x7f82600d1a10 2024-05-10T10:35:02.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.709+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f82600d27c0 con 0x7f82600d1a10 2024-05-10T10:35:02.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.711+0000 7f825cff9640 1 -- 172.21.15.53:0/3580017962 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f8254060d40 con 0x7f82600d1a10 2024-05-10T10:35:02.712 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:35:02.712 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:35:02.712 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2024-05-10T10:35:02.712 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2024-05-10T10:35:02.712 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-10T10:35:02.713 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2024-05-10T10:35:02.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f824c0762d0 msgr2=0x7f824c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f824c0762d0 0x7f824c078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8250034000 tx=0x7f825002e000 comp rx=0 tx=0).stop 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d1a10 msgr2=0x7f82601ea370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d1a10 0x7f82601ea370 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f825400d830 tx=0x7f825400dcf0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 shutdown_connections 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f824c0762d0 0x7f824c078790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82600d3df0 0x7f826014efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82600d1a10 0x7f82601ea370 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 --2- 172.21.15.53:0/3580017962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82600b4aa0 0x7f8260150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.713+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 >> 172.21.15.53:0/3580017962 conn(0x7f826009e450 msgr2=0x7f82600d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.714+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 shutdown_connections 2024-05-10T10:35:02.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.714+0000 7f8266893640 1 -- 172.21.15.53:0/3580017962 wait complete. 2024-05-10T10:35:02.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:02.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.829+0000 7fd0539c9640 1 Processor -- start 2024-05-10T10:35:02.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.829+0000 7fd0539c9640 1 -- start start 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd0539c9640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 0x7fd04c0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd0539c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd04c0b45f0 con 0x7fd04c0d5cb0 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd0539c9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd04c0b48d0 con 0x7fd04c162b30 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd0539c9640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c0b4f20 con 0x7fd04c162750 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd050f64640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd04c0d5cb0 0x7fd04c0d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34076/0 (socket says 172.21.15.53:34076) 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd050f64640 1 -- 172.21.15.53:0/3449792074 learned_addr learned my addr 172.21.15.53:0/3449792074 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd051765640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 0x7fd04c0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136704092 0 0) 0x7fd04c0b45f0 con 0x7fd04c0d5cb0 2024-05-10T10:35:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.830+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd038003520 con 0x7fd04c0d5cb0 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 240937132 0 0) 0x7fd038003520 con 0x7fd04c0d5cb0 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 msgr2=0x7fd04c0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 0x7fd04c0d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 >> v1:172.21.15.135:6789/0 conn(0x7fd04c162b30 legacy=0x7fd04c0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c0b4bb0 con 0x7fd04c0d5cb0 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fd03c001670 con 0x7fd04c0d5cb0 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 >> v1:172.21.15.53:6789/0 conn(0x7fd04c0d5cb0 legacy=0x7fd04c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 0x7fd0380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd038003f60 0x7fd038004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038004d40 0x7fd0380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c0b4f20 con 0x7fd038003f60 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd03800bb50 con 0x7fd038004d40 2024-05-10T10:35:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd03800bd50 con 0x7fd04c162750 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd050f64640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd038003f60 0x7fd038004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd051765640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 0x7fd0380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd051f66640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038004d40 0x7fd0380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd050f64640 1 -- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 msgr2=0x7fd0380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd050f64640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c162750 0x7fd0380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd050f64640 1 -- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038004d40 msgr2=0x7fd0380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd050f64640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038004d40 0x7fd0380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.831+0000 7fd050f64640 1 -- 172.21.15.53:0/3449792074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd038003520 con 0x7fd038003f60 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.832+0000 7fd050f64640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd038003f60 0x7fd038004800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c210 tx=0x7fd038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c939eb6cd1b075 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.832+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c007d30 con 0x7fd038003f60 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.832+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03c005ad0 con 0x7fd038003f60 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.832+0000 7fd043fff640 1 -- 172.21.15.53:0/3449792074 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c007070 con 0x7fd038003f60 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.833+0000 7fd0539c9640 1 -- 172.21.15.53:0/3449792074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd038003f60 msgr2=0x7fd038004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.833+0000 7fd0539c9640 1 --2- 172.21.15.53:0/3449792074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd038003f60 0x7fd038004800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c210 tx=0x7fd038005830 comp rx=0 tx=0).stop 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.833+0000 7fd051f66640 1 -- 172.21.15.53:0/3449792074 reap_dead start 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.833+0000 7fd0539c9640 1 -- 172.21.15.53:0/3449792074 shutdown_connections 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.833+0000 7fd0539c9640 1 -- 172.21.15.53:0/3449792074 >> 172.21.15.53:0/3449792074 conn(0x7fd04c09e3b0 msgr2=0x7fd04c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.833+0000 7fd0539c9640 1 -- 172.21.15.53:0/3449792074 shutdown_connections 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.833+0000 7fd0539c9640 1 -- 172.21.15.53:0/3449792074 wait complete. 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.834+0000 7fd0539c9640 1 Processor -- start 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.834+0000 7fd0539c9640 1 -- start start 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd0539c9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038003f60 0x7fd04c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd0539c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c0b9790 0x7fd04c0b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd0539c9640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c1f4e30 0x7fd04c1f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd0539c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c0d0fe0 con 0x7fd04c0b9790 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd0539c9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c0d3f50 con 0x7fd038003f60 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd0539c9640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c0d0670 con 0x7fd04c1f4e30 2024-05-10T10:35:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c0b9790 0x7fd04c0b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c0b9790 0x7fd04c0b7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47976/0 (socket says 172.21.15.53:47976) 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 -- 172.21.15.53:0/673936090 learned_addr learned my addr 172.21.15.53:0/673936090 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd051765640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038003f60 0x7fd04c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd051f66640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c1f4e30 0x7fd04c1f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 -- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c1f4e30 msgr2=0x7fd04c1f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c1f4e30 0x7fd04c1f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 -- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038003f60 msgr2=0x7fd04c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038003f60 0x7fd04c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 -- 172.21.15.53:0/673936090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c0b8450 con 0x7fd04c0b9790 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.835+0000 7fd050f64640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c0b9790 0x7fd04c0b7f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd03c02ae20 tx=0x7fd03c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.836+0000 7fd041ffb640 1 -- 172.21.15.53:0/673936090 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c004bf0 con 0x7fd04c0b9790 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.836+0000 7fd041ffb640 1 -- 172.21.15.53:0/673936090 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03c004d90 con 0x7fd04c0b9790 2024-05-10T10:35:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.836+0000 7fd041ffb640 1 -- 172.21.15.53:0/673936090 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0072b0 con 0x7fd04c0b9790 2024-05-10T10:35:02.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.836+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c0b86f0 con 0x7fd04c0b9790 2024-05-10T10:35:02.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.836+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c0b8cb0 con 0x7fd04c0b9790 2024-05-10T10:35:02.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.837+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c06a220 con 0x7fd04c0b9790 2024-05-10T10:35:02.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.837+0000 7fd041ffb640 1 -- 172.21.15.53:0/673936090 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd03c007450 con 0x7fd04c0b9790 2024-05-10T10:35:02.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.838+0000 7fd041ffb640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0300762d0 0x7fd030078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:02.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.838+0000 7fd041ffb640 1 -- 172.21.15.53:0/673936090 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fd03c0b7bd0 con 0x7fd04c0b9790 2024-05-10T10:35:02.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.838+0000 7fd051765640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0300762d0 0x7fd030078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:02.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.839+0000 7fd051765640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0300762d0 0x7fd030078790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0ba300 tx=0x7fd048009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:02.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:02.842+0000 7fd041ffb640 1 -- 172.21.15.53:0/673936090 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd03c04c020 con 0x7fd04c0b9790 2024-05-10T10:35:03.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.071+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04c0cf480 con 0x7fd04c0b9790 2024-05-10T10:35:03.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.072+0000 7fd041ffb640 1 -- 172.21.15.53:0/673936090 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7fd03c081430 con 0x7fd04c0b9790 2024-05-10T10:35:03.073 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:03.073 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:03.073 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0300762d0 msgr2=0x7fd030078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0300762d0 0x7fd030078790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0ba300 tx=0x7fd048009190 comp rx=0 tx=0).stop 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c0b9790 msgr2=0x7fd04c0b7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c0b9790 0x7fd04c0b7f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd03c02ae20 tx=0x7fd03c001a40 comp rx=0 tx=0).stop 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 shutdown_connections 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd0300762d0 0x7fd030078790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd04c1f4e30 0x7fd04c1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c0b9790 0x7fd04c0b7f10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 --2- 172.21.15.53:0/673936090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd038003f60 0x7fd04c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 >> 172.21.15.53:0/673936090 conn(0x7fd04c09e3b0 msgr2=0x7fd04c06e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 shutdown_connections 2024-05-10T10:35:03.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.074+0000 7fd0539c9640 1 -- 172.21.15.53:0/673936090 wait complete. 2024-05-10T10:35:03.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:34:57.236131+0000', 'modified': '2024-05-10T10:35:00.256407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5232': {'gid': 5232, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/679529875', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 679529875}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 679529875}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5232, 'qdb_cluster': [5232]} 2024-05-10T10:35:03.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:35:03.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T10:35:03.494 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:35:03.494+0000 7fbcf5f8d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:35:03.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.599+0000 7fa526bc8640 1 Processor -- start 2024-05-10T10:35:03.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.599+0000 7fa526bc8640 1 -- start start 2024-05-10T10:35:03.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa526bc8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa52015f620 0x7fa52015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa526bc8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 0x7fa520070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa526bc8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa520160190 con 0x7fa52006c990 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa526bc8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa52006cd70 con 0x7fa52015f620 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa526bc8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa520160470 con 0x7fa5200704c0 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa524964640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa52006c990 0x7fa520163660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34086/0 (socket says 172.21.15.53:34086) 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa524964640 1 -- 172.21.15.53:0/1022547719 learned_addr learned my addr 172.21.15.53:0/1022547719 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa525165640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 0x7fa520070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51ffff640 1 --2- 172.21.15.53:0/1022547719 >> v2:172.21.15.135:3300/0 conn(0x7fa52015f620 0x7fa52015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210600893 0 0) 0x7fa520160190 con 0x7fa52006c990 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa510003520 con 0x7fa52006c990 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 963333493 0 0) 0x7fa510003520 con 0x7fa52006c990 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 msgr2=0x7fa520070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 0x7fa520070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 >> v2:172.21.15.135:3300/0 conn(0x7fa52015f620 msgr2=0x7fa52015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 --2- 172.21.15.53:0/1022547719 >> v2:172.21.15.135:3300/0 conn(0x7fa52015f620 0x7fa52015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa520162e60 con 0x7fa52006c990 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fa514001690 con 0x7fa52006c990 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 >> v1:172.21.15.53:6789/0 conn(0x7fa52006c990 legacy=0x7fa520163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 0x7fa5100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa510003f30 0x7fa510004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa510004d40 0x7fa5100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa52006cd70 con 0x7fa510004d40 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa520160470 con 0x7fa510003f30 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa51000b9c0 con 0x7fa5200704c0 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.600+0000 7fa525165640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa510004d40 0x7fa5100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa524964640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 0x7fa5100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa51ffff640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa510003f30 0x7fa510004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa525165640 1 -- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 msgr2=0x7fa5100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa525165640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5200704c0 0x7fa5100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa525165640 1 -- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa510003f30 msgr2=0x7fa510004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa525165640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa510003f30 0x7fa510004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa525165640 1 -- 172.21.15.53:0/1022547719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa510003520 con 0x7fa510004d40 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.601+0000 7fa525165640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa510004d40 0x7fa5100051c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa518009fc0 tx=0x7fa510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cf71a4571ebe78 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518009c10 con 0x7fa510004d40 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa518036070 con 0x7fa510004d40 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa51f7fe640 1 -- 172.21.15.53:0/1022547719 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518031670 con 0x7fa510004d40 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa526bc8640 1 -- 172.21.15.53:0/1022547719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa510004d40 msgr2=0x7fa5100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa526bc8640 1 --2- 172.21.15.53:0/1022547719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa510004d40 0x7fa5100051c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa518009fc0 tx=0x7fa510005830 comp rx=0 tx=0).stop 2024-05-10T10:35:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa525165640 1 -- 172.21.15.53:0/1022547719 reap_dead start 2024-05-10T10:35:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa526bc8640 1 -- 172.21.15.53:0/1022547719 shutdown_connections 2024-05-10T10:35:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa526bc8640 1 -- 172.21.15.53:0/1022547719 >> 172.21.15.53:0/1022547719 conn(0x7fa5200b3980 msgr2=0x7fa5200b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa526bc8640 1 -- 172.21.15.53:0/1022547719 shutdown_connections 2024-05-10T10:35:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.602+0000 7fa526bc8640 1 -- 172.21.15.53:0/1022547719 wait complete. 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.603+0000 7fa526bc8640 1 Processor -- start 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.604+0000 7fa526bc8640 1 -- start start 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.604+0000 7fa526bc8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa52015a9f0 0x7fa5201f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.604+0000 7fa526bc8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5201f5760 0x7fa5200b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.604+0000 7fa526bc8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5200b4cd0 0x7fa5200d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.604+0000 7fa526bc8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa520070920 con 0x7fa5201f5760 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.604+0000 7fa526bc8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa520073e70 con 0x7fa5200b4cd0 2024-05-10T10:35:03.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.604+0000 7fa526bc8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa520070aa0 con 0x7fa52015a9f0 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5201f5760 0x7fa5200b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5201f5760 0x7fa5200b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48000/0 (socket says 172.21.15.53:48000) 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 -- 172.21.15.53:0/2809360926 learned_addr learned my addr 172.21.15.53:0/2809360926 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 -- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa52015a9f0 msgr2=0x7fa5201f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa524964640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa52015a9f0 0x7fa5201f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa525165640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5200b4cd0 0x7fa5200d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa52015a9f0 0x7fa5201f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 -- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5200b4cd0 msgr2=0x7fa5200d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5200b4cd0 0x7fa5200d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 -- 172.21.15.53:0/2809360926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200cf380 con 0x7fa5201f5760 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa525165640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5200b4cd0 0x7fa5200d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa524964640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa52015a9f0 0x7fa5201f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:03.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51ffff640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5201f5760 0x7fa5200b4790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa50c011960 tx=0x7fa50c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51d7fa640 1 -- 172.21.15.53:0/2809360926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00fa90 con 0x7fa5201f5760 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51d7fa640 1 -- 172.21.15.53:0/2809360926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c00fc30 con 0x7fa5201f5760 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa51d7fa640 1 -- 172.21.15.53:0/2809360926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c014440 con 0x7fa5201f5760 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5200cf660 con 0x7fa5201f5760 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.605+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5200cfc20 con 0x7fa5201f5760 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.607+0000 7fa51d7fa640 1 -- 172.21.15.53:0/2809360926 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa50c012040 con 0x7fa5201f5760 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.607+0000 7fa51d7fa640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa504076300 0x7fa5040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.607+0000 7fa51d7fa640 1 -- 172.21.15.53:0/2809360926 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fa50c097d80 con 0x7fa5201f5760 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.608+0000 7fa524964640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa504076300 0x7fa5040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.608+0000 7fa524964640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa504076300 0x7fa5040787c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa5200d1290 tx=0x7fa514009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.608+0000 7fa5299be740 1 Processor -- start 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.608+0000 7fa5299be740 1 -- start start 2024-05-10T10:35:03.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9daeb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa5299be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9daebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dabbbb0 0x557e9dabf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa5299be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557e9da258d0 con 0x557e9dae9330 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9dec10 con 0x557e9dabbbb0 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557e9da21890 con 0x557e9dae8f50 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9daebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9daebca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48012/0 (socket says 172.21.15.53:48012) 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 -- 172.21.15.53:0/3248762662 learned_addr learned my addr 172.21.15.53:0/3248762662 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:03.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa524964640 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9daeb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa525165640 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dabbbb0 0x557e9dabf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 -- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 msgr2=0x557e9daeb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9daeb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 -- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dabbbb0 msgr2=0x557e9dabf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dabbbb0 0x557e9dabf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 -- 172.21.15.53:0/3248762662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9d9e65d0 con 0x557e9dae9330 2024-05-10T10:35:03.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa524964640 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9daeb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:03.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.609+0000 7fa51ffff640 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9daebca0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa50c060dd0 tx=0x7fa50c060a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee38c022e6bd399c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa50a7fc640 1 -- 172.21.15.53:0/3248762662 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c063040 con 0x557e9dae9330 2024-05-10T10:35:03.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa50a7fc640 1 -- 172.21.15.53:0/3248762662 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c069c10 con 0x557e9dae9330 2024-05-10T10:35:03.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa50a7fc640 1 -- 172.21.15.53:0/3248762662 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c069780 con 0x557e9dae9330 2024-05-10T10:35:03.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 -- 172.21.15.53:0/3248762662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 msgr2=0x557e9daebca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9daebca0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa50c060dd0 tx=0x7fa50c060a60 comp rx=0 tx=0).stop 2024-05-10T10:35:03.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 -- 172.21.15.53:0/3248762662 shutdown_connections 2024-05-10T10:35:03.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dabbbb0 0x557e9dabf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9daebca0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 --2- 172.21.15.53:0/3248762662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9daeb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 -- 172.21.15.53:0/3248762662 >> 172.21.15.53:0/3248762662 conn(0x557e9dabde20 msgr2=0x557e9db06760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 -- 172.21.15.53:0/3248762662 shutdown_connections 2024-05-10T10:35:03.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.610+0000 7fa5299be740 1 -- 172.21.15.53:0/3248762662 wait complete. 2024-05-10T10:35:03.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.611+0000 7fa5299be740 1 -- start start 2024-05-10T10:35:03.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.611+0000 7fa5299be740 1 Processor -- start 2024-05-10T10:35:03.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa5299be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae8f50 0x557e9daee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae9330 0x557e9daeebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9db08b80 0x557e9db09000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa5299be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557e9dac0fd0 con 0x557e9dae8f50 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9e2760 con 0x557e9dae9330 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557e9dac12b0 con 0x557e9db08b80 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa5299be740 10 client.0 _ll_register_callbacks cb 0x557e9da1bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae8f50 0x557e9daee6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae8f50 0x557e9daee6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48026/0 (socket says 172.21.15.53:48026) 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 -- 172.21.15.53:0/2110554554 learned_addr learned my addr 172.21.15.53:0/2110554554 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa51ffff640 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae9330 0x557e9daeebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa525165640 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9db08b80 0x557e9db09000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 -- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9db08b80 msgr2=0x557e9db09000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9db08b80 0x557e9db09000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 -- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae9330 msgr2=0x557e9daeebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae9330 0x557e9daeebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 -- 172.21.15.53:0/2110554554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9d7372d0 con 0x557e9dae8f50 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa524964640 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae8f50 0x557e9daee6b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa514015c80 tx=0x7fa520163030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18497d4120571400 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.612+0000 7fa4ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/2110554554 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514026060 con 0x557e9dae8f50 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/2110554554 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fa514015cb0 con 0x557e9dae8f50 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa50c069c10 con 0x557e9dae8f50 2024-05-10T10:35:03.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/2110554554 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa51402c050 con 0x557e9dae8f50 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 10 client.5355 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x557e9d8016b0 con 0x557e9dae8f50 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/2110554554 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c069c10 con 0x557e9dae8f50 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/2110554554 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fa51402f020 con 0x557e9dae8f50 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 10 client.5355 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 10 client.5355 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 20 client.5355 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db01660 0x557e9db01a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 4 client.5355 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9d9a9760 con 0x557e9db01660 2024-05-10T10:35:03.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.613+0000 7fa51ffff640 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db01660 0x557e9db01a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa51ffff640 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db01660 0x557e9db01a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5314c880a69d1276 server_cookie=b5d2e21978ab2f91 in_seq=0 out_seq=0 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa4ff7fe640 10 client.5355 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/2110554554 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e9d9a9760 con 0x557e9db01660 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa4ff7fe640 10 client.5355 handle_command_reply: tid=0 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa5299be740 1 client.5355 shutdown 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae8f50 msgr2=0x557e9daee6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa5299be740 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae8f50 0x557e9daee6b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa514015c80 tx=0x7fa520163030 comp rx=0 tx=0).stop 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 shutdown_connections 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa5299be740 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db01660 0x557e9db01a40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa5299be740 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9db08b80 0x557e9db09000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.614+0000 7fa5299be740 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae9330 0x557e9daeebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.615+0000 7fa5299be740 1 --2- 172.21.15.53:0/2110554554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae8f50 0x557e9daee6b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.615+0000 7fa4ff7fe640 0 client.5355 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:03.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.615+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 >> 172.21.15.53:0/2110554554 conn(0x557e9dabbbb0 msgr2=0x557e9dabcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.615+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 shutdown_connections 2024-05-10T10:35:03.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.615+0000 7fa5299be740 1 -- 172.21.15.53:0/2110554554 wait complete. 2024-05-10T10:35:03.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.615+0000 7fa5299be740 20 client.5355 trim_cache size 0 max 16384 2024-05-10T10:35:03.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.633+0000 7fa5299be740 1 Processor -- start 2024-05-10T10:35:03.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.633+0000 7fa5299be740 1 -- start start 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9db0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa5299be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9db09f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9db08b80 0x557e9db01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa5299be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa520201a70 con 0x557e9dae9330 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa520200dd0 con 0x557e9db08b80 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa520201420 con 0x557e9dae8f50 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9db09f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9db09f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48042/0 (socket says 172.21.15.53:48042) 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa51ffff640 1 -- 172.21.15.53:0/3615313638 learned_addr learned my addr 172.21.15.53:0/3615313638 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:03.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa51ffff640 1 -- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 msgr2=0x557e9db0a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.634+0000 7fa524964640 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9db0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa525165640 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9db08b80 0x557e9db01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa51ffff640 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9db0a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa51ffff640 1 -- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9db08b80 msgr2=0x557e9db01c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa51ffff640 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9db08b80 0x557e9db01c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa51ffff640 1 -- 172.21.15.53:0/3615313638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9db07890 con 0x557e9dae9330 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa51ffff640 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9db09f90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa50c05e0b0 tx=0x7fa50c067000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cea8598de4db94 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa509ffb640 1 -- 172.21.15.53:0/3615313638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c063040 con 0x557e9dae9330 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa509ffb640 1 -- 172.21.15.53:0/3615313638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c05d0d0 con 0x557e9dae9330 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa509ffb640 1 -- 172.21.15.53:0/3615313638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c069980 con 0x557e9dae9330 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 -- 172.21.15.53:0/3615313638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 msgr2=0x557e9db09f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9db09f90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa50c05e0b0 tx=0x7fa50c067000 comp rx=0 tx=0).stop 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 -- 172.21.15.53:0/3615313638 shutdown_connections 2024-05-10T10:35:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9db08b80 0x557e9db01c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9dae9330 0x557e9db09f90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 --2- 172.21.15.53:0/3615313638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae8f50 0x557e9db0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 -- 172.21.15.53:0/3615313638 >> 172.21.15.53:0/3615313638 conn(0x557e9dabbbb0 msgr2=0x557e9dac1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.635+0000 7fa5299be740 1 -- 172.21.15.53:0/3615313638 shutdown_connections 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.636+0000 7fa5299be740 1 -- 172.21.15.53:0/3615313638 wait complete. 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.636+0000 7fa5299be740 1 -- start start 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.636+0000 7fa5299be740 1 Processor -- start 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae8f50 0x557e9db6dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa5299be740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae9330 0x557e9db190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa5299be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9db08b80 0x557e9db88780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa5299be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557e9dae00d0 con 0x557e9db08b80 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557e9db02640 con 0x557e9dae8f50 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa5299be740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557e9db1a6f0 con 0x557e9dae9330 2024-05-10T10:35:03.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa5299be740 10 client.0 _ll_register_callbacks cb 0x7fa514018040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9db08b80 0x557e9db88780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9db08b80 0x557e9db88780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48058/0 (socket says 172.21.15.53:48058) 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 -- 172.21.15.53:0/498742659 learned_addr learned my addr 172.21.15.53:0/498742659 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa51ffff640 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae9330 0x557e9db190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa524964640 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae8f50 0x557e9db6dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 -- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae9330 msgr2=0x557e9db190d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae9330 0x557e9db190d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 -- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae8f50 msgr2=0x557e9db6dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae8f50 0x557e9db6dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.637+0000 7fa525165640 1 -- 172.21.15.53:0/498742659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9db87fc0 con 0x557e9db08b80 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa525165640 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9db08b80 0x557e9db88780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa51802aed0 tx=0x7fa518004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6df6ae6904b927d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa4ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:03.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/498742659 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518003fa0 con 0x557e9db08b80 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/498742659 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7fa51802e040 con 0x557e9db08b80 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa50c05d0d0 con 0x557e9db08b80 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/498742659 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa51804e050 con 0x557e9db08b80 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 10 client.5361 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x557e9dafd170 con 0x557e9db08b80 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/498742659 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c05d0d0 con 0x557e9db08b80 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/498742659 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fa518051020 con 0x557e9db08b80 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 10 client.5361 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 10 client.5361 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 20 client.5361 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db0a650 0x557e9dae2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 4 client.5361 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "status"}) -- 0x557e9dac1620 con 0x557e9db0a650 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.638+0000 7fa524964640 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db0a650 0x557e9dae2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa4ff7fe640 10 client.5361 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa524964640 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db0a650 0x557e9dae2a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92fb7cbc2d88a4a7 server_cookie=cc270f9900492f20 in_seq=0 out_seq=0 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa4ff7fe640 1 -- 172.21.15.53:0/498742659 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x557e9dac1620 con 0x557e9db0a650 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa4ff7fe640 10 client.5361 handle_command_reply: tid=0 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 client.5361 shutdown 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9db08b80 msgr2=0x557e9db88780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9db08b80 0x557e9db88780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa51802aed0 tx=0x7fa518004ab0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 shutdown_connections 2024-05-10T10:35:03.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x557e9db0a650 0x557e9dae2a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e9db08b80 0x557e9db88780 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa4ff7fe640 0 client.5361 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:03.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557e9dae9330 0x557e9db190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 --2- 172.21.15.53:0/498742659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e9dae8f50 0x557e9db6dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 >> 172.21.15.53:0/498742659 conn(0x557e9dabbbb0 msgr2=0x557e9db01c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 shutdown_connections 2024-05-10T10:35:03.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 1 -- 172.21.15.53:0/498742659 wait complete. 2024-05-10T10:35:03.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.639+0000 7fa5299be740 20 client.5361 trim_cache size 0 max 16384 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 5232, 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 5.7925925649999996, 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 26, 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 48, 2024-05-10T10:35:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 48, 2024-05-10T10:35:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 11.098633319999999 2024-05-10T10:35:03.651 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa504076300 msgr2=0x7fa5040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa504076300 0x7fa5040787c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa5200d1290 tx=0x7fa514009040 comp rx=0 tx=0).stop 2024-05-10T10:35:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5201f5760 msgr2=0x7fa5200b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5201f5760 0x7fa5200b4790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa50c011960 tx=0x7fa50c011e20 comp rx=0 tx=0).stop 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 shutdown_connections 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa504076300 0x7fa5040787c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5200b4cd0 0x7fa5200d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5201f5760 0x7fa5200b4790 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 --2- 172.21.15.53:0/2809360926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa52015a9f0 0x7fa5201f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 >> 172.21.15.53:0/2809360926 conn(0x7fa5200b3980 msgr2=0x7fa5200c7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 shutdown_connections 2024-05-10T10:35:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.640+0000 7fa526bc8640 1 -- 172.21.15.53:0/2809360926 wait complete. 2024-05-10T10:35:03.654 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:35:03.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.795+0000 7f13446a9640 1 Processor -- start 2024-05-10T10:35:03.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.795+0000 7f13446a9640 1 -- start start 2024-05-10T10:35:03.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f13446a9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f133c15f3b0 0x7f133c0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f13446a9640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 0x7f133c0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f13446a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c0d30b0 con 0x7f133c15f790 2024-05-10T10:35:03.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f13446a9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f133c0d3390 con 0x7f133c15f3b0 2024-05-10T10:35:03.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f13446a9640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f133c0c8c80 con 0x7f133c0d5d30 2024-05-10T10:35:03.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341c44640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f133c15f790 0x7f133c0c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34100/0 (socket says 172.21.15.53:34100) 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341c44640 1 -- 172.21.15.53:0/453403564 learned_addr learned my addr 172.21.15.53:0/453403564 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1342445640 1 --2- 172.21.15.53:0/453403564 >> v2:172.21.15.135:3300/0 conn(0x7f133c15f3b0 0x7f133c0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1342c46640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 0x7f133c0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982946209 0 0) 0x7f133c0d30b0 con 0x7f133c15f790 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1328003520 con 0x7f133c15f790 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 668920985 0 0) 0x7f1328003520 con 0x7f133c15f790 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 msgr2=0x7f133c0d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 0x7f133c0d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 >> v2:172.21.15.135:3300/0 conn(0x7f133c15f3b0 msgr2=0x7f133c0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 --2- 172.21.15.53:0/453403564 >> v2:172.21.15.135:3300/0 conn(0x7f133c15f3b0 0x7f133c0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c0c8f60 con 0x7f133c15f790 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f132c001670 con 0x7f133c15f790 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1342445640 1 --2- 172.21.15.53:0/453403564 >> v2:172.21.15.135:3300/0 conn(0x7f133c15f3b0 0x7f133c0d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 >> v1:172.21.15.53:6789/0 conn(0x7f133c15f790 legacy=0x7f133c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 0x7f13280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1328003f30 0x7f1328004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328004d40 0x7f13280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0d3390 con 0x7f1328003f30 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.796+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0c8c80 con 0x7f1328004d40 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f132800b9c0 con 0x7f133c0d5d30 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1341c44640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1328003f30 0x7f1328004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1342445640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 0x7f13280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1342c46640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328004d40 0x7f13280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1341c44640 1 -- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 msgr2=0x7f13280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1341c44640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 0x7f13280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1341c44640 1 -- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328004d40 msgr2=0x7f13280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1341c44640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328004d40 0x7f13280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1341c44640 1 -- 172.21.15.53:0/453403564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1328003520 con 0x7f1328003f30 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.797+0000 7f1342445640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c0d5d30 0x7f13280039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f1341c44640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1328003f30 0x7f1328004800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c210 tx=0x7f1328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16c42238f270220 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c007d30 con 0x7f1328003f30 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132c0058d0 con 0x7f1328003f30 2024-05-10T10:35:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f1341443640 1 -- 172.21.15.53:0/453403564 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c005d20 con 0x7f1328003f30 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f13446a9640 1 -- 172.21.15.53:0/453403564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1328003f30 msgr2=0x7f1328004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f13446a9640 1 --2- 172.21.15.53:0/453403564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1328003f30 0x7f1328004800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c210 tx=0x7f1328005830 comp rx=0 tx=0).stop 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f1342c46640 1 -- 172.21.15.53:0/453403564 reap_dead start 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f13446a9640 1 -- 172.21.15.53:0/453403564 shutdown_connections 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f13446a9640 1 -- 172.21.15.53:0/453403564 >> 172.21.15.53:0/453403564 conn(0x7f133c0d5130 msgr2=0x7f133c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f13446a9640 1 -- 172.21.15.53:0/453403564 shutdown_connections 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.798+0000 7f13446a9640 1 -- 172.21.15.53:0/453403564 wait complete. 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 Processor -- start 2024-05-10T10:35:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 -- start start 2024-05-10T10:35:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328003f30 0x7f133c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c1540f0 0x7f133c0ba130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f133c0ced60 0x7f133c0ba670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0d3db0 con 0x7f133c0ced60 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0d3c30 con 0x7f1328003f30 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f13446a9640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f133c0d3f30 con 0x7f133c1540f0 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.800+0000 7f1342c46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f133c0ced60 0x7f133c0ba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f133c0ced60 0x7f133c0ba670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48078/0 (socket says 172.21.15.53:48078) 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 -- 172.21.15.53:0/1645767770 learned_addr learned my addr 172.21.15.53:0/1645767770 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1341c44640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c1540f0 0x7f133c0ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342445640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328003f30 0x7f133c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 -- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c1540f0 msgr2=0x7f133c0ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c1540f0 0x7f133c0ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 -- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328003f30 msgr2=0x7f133c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328003f30 0x7f133c153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 -- 172.21.15.53:0/1645767770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c0b9530 con 0x7f133c0ced60 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1342c46640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f133c0ced60 0x7f133c0ba670 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f133c0d36a0 tx=0x7f13340115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1332ffd640 1 -- 172.21.15.53:0/1645767770 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334011db0 con 0x7f133c0ced60 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1332ffd640 1 -- 172.21.15.53:0/1645767770 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133401b070 con 0x7f133c0ced60 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c0b8f70 con 0x7f133c0ced60 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f1332ffd640 1 -- 172.21.15.53:0/1645767770 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334003990 con 0x7f133c0ced60 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.801+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c0cc330 con 0x7f133c0ced60 2024-05-10T10:35:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.802+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c06a220 con 0x7f133c0ced60 2024-05-10T10:35:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.803+0000 7f1332ffd640 1 -- 172.21.15.53:0/1645767770 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1334005d10 con 0x7f133c0ced60 2024-05-10T10:35:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.803+0000 7f1332ffd640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1320076300 0x7f13200787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.804+0000 7f1332ffd640 1 -- 172.21.15.53:0/1645767770 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f1334097a20 con 0x7f133c0ced60 2024-05-10T10:35:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.804+0000 7f1342445640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1320076300 0x7f13200787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.804+0000 7f1342445640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1320076300 0x7f13200787c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1338009ba0 tx=0x7f1338008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:03.808+0000 7f1332ffd640 1 -- 172.21.15.53:0/1645767770 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f13340611e0 con 0x7f133c0ced60 2024-05-10T10:35:04.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.038+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f133c0c9650 con 0x7f133c0ced60 2024-05-10T10:35:04.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.038+0000 7f1332ffd640 1 -- 172.21.15.53:0/1645767770 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f1334060ae0 con 0x7f133c0ced60 2024-05-10T10:35:04.039 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1320076300 msgr2=0x7f13200787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1320076300 0x7f13200787c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1338009ba0 tx=0x7f1338008040 comp rx=0 tx=0).stop 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f133c0ced60 msgr2=0x7f133c0ba670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f133c0ced60 0x7f133c0ba670 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f133c0d36a0 tx=0x7f13340115a0 comp rx=0 tx=0).stop 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 shutdown_connections 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1320076300 0x7f13200787c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f133c0ced60 0x7f133c0ba670 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f133c1540f0 0x7f133c0ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 --2- 172.21.15.53:0/1645767770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1328003f30 0x7f133c153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 >> 172.21.15.53:0/1645767770 conn(0x7f133c0d5130 msgr2=0x7f133c0d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 shutdown_connections 2024-05-10T10:35:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04.040+0000 7f13446a9640 1 -- 172.21.15.53:0/1645767770 wait complete. 2024-05-10T10:35:04.051 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:35:04.052 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:35:04.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.052 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:35:04.069 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42665sec preferred_lft 42665sec 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:04.070 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:04.071 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:35:04.072 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:35:04.072 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:35:04.072 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:04.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.072 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:35:04.134 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:35:04.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.135 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:35:04.192 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:35:04.192 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:35:04.193 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:35:04.193 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:35:04.193 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:35:04.193 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:35:04.193 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:35:04.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:35:04.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:35:04.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:35:04.194 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:35:04.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:35:04.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:35:04.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.322 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:35:04.338 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:35:04.338 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:35:04.338 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:35:04.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.339 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:35:04.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:35:04.405 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:04.460 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:35:04.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:35:04.460 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:04.516 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:35:04.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:04.601 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:04.601 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:35:04.601 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:35:04.610 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:04.610 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:35:04.610 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:35:04.610 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:35:04.610 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:35:04.615 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:35:04.616 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:35:04.616 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:35:04.616 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:35:04.616 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:35:04.616 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:35:04.617 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:35:04.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.617 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:04.643 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:35:04.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:35:04.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:35:04.747 INFO:teuthology.orchestra.run.smithi053.stdout:client5370 2024-05-10T10:35:04.747 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:35:04.747 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:04.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.764 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-10T10:35:04.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:04.825 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-10T10:35:04.883 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:35:04.883 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:35:04.884 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:35:04.884 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:35:04.884 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-10T10:35:04.884 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:35:04.991 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2024-05-10T10:35:04.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:35:05.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:05.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.144+0000 7f44ab59e640 1 Processor -- start 2024-05-10T10:35:05.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.144+0000 7f44ab59e640 1 -- start start 2024-05-10T10:35:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44ab59e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 0x7f44ac0d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44ab59e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f44ac0d5a00 0x7f44ac0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44ab59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ac0d5e20 con 0x7f44ac15f1f0 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44ab59e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac0d2ff0 con 0x7f44ac0d5a00 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44ab59e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac0d0b90 con 0x7f44ac0d3bd0 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44aa59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f44ac15f1f0 0x7f44ac0d34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34108/0 (socket says 172.21.15.53:34108) 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44aa59c640 1 -- 172.21.15.53:0/1949866775 learned_addr learned my addr 172.21.15.53:0/1949866775 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a3fff640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 0x7f44ac0d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44aad9d640 1 --2- 172.21.15.53:0/1949866775 >> v2:172.21.15.135:3300/0 conn(0x7f44ac0d5a00 0x7f44ac0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763867374 0 0) 0x7f44ac0d5e20 con 0x7f44ac15f1f0 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4498003520 con 0x7f44ac15f1f0 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 582619315 0 0) 0x7f4498003520 con 0x7f44ac15f1f0 2024-05-10T10:35:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 msgr2=0x7f44ac0d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 0x7f44ac0d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 >> v2:172.21.15.135:3300/0 conn(0x7f44ac0d5a00 msgr2=0x7f44ac0b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 --2- 172.21.15.53:0/1949866775 >> v2:172.21.15.135:3300/0 conn(0x7f44ac0d5a00 0x7f44ac0b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac0b4470 con 0x7f44ac15f1f0 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f449c001670 con 0x7f44ac15f1f0 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 >> v1:172.21.15.53:6789/0 conn(0x7f44ac15f1f0 legacy=0x7f44ac0d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44980039b0 0x7f4498003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498004800 0x7f4498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.145+0000 7f44a9d9b640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 0x7f44980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0d2ff0 con 0x7f4498004800 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0d0b90 con 0x7f44980039b0 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f449800b9f0 con 0x7f44ac0d3bd0 2024-05-10T10:35:05.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a3fff640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498004800 0x7f4498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44aa59c640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44980039b0 0x7f4498003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44aad9d640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 0x7f44980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a3fff640 1 -- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 msgr2=0x7f44980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a3fff640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0d3bd0 0x7f44980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a3fff640 1 -- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44980039b0 msgr2=0x7f4498003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a3fff640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44980039b0 0x7f4498003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.146+0000 7f44a3fff640 1 -- 172.21.15.53:0/1949866775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4498003520 con 0x7f4498004800 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44a3fff640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498004800 0x7f4498004c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4494009ac0 tx=0x7f4498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f0be1cc6e2e857 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494036070 con 0x7f4498004800 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494031400 con 0x7f4498004800 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44a9d9b640 1 -- 172.21.15.53:0/1949866775 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494009690 con 0x7f4498004800 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44ab59e640 1 -- 172.21.15.53:0/1949866775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498004800 msgr2=0x7f4498004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44ab59e640 1 --2- 172.21.15.53:0/1949866775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4498004800 0x7f4498004c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4494009ac0 tx=0x7f4498005830 comp rx=0 tx=0).stop 2024-05-10T10:35:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44aad9d640 1 -- 172.21.15.53:0/1949866775 reap_dead start 2024-05-10T10:35:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44ab59e640 1 -- 172.21.15.53:0/1949866775 shutdown_connections 2024-05-10T10:35:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44ab59e640 1 -- 172.21.15.53:0/1949866775 >> 172.21.15.53:0/1949866775 conn(0x7f44ac0d4ee0 msgr2=0x7f44ac0d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44ab59e640 1 -- 172.21.15.53:0/1949866775 shutdown_connections 2024-05-10T10:35:05.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.147+0000 7f44ab59e640 1 -- 172.21.15.53:0/1949866775 wait complete. 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.149+0000 7f44ab59e640 1 Processor -- start 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.149+0000 7f44ab59e640 1 -- start start 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4498004800 0x7f44ac0ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0b90c0 0x7f44ac0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44ac0ce320 0x7f44ac161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0d0500 con 0x7f44ac0ce320 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac15f5d0 con 0x7f4498004800 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac0d0680 con 0x7f44ac0b90c0 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44ac0ce320 0x7f44ac161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44ac0ce320 0x7f44ac161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48096/0 (socket says 172.21.15.53:48096) 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 -- 172.21.15.53:0/2684472075 learned_addr learned my addr 172.21.15.53:0/2684472075 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aa59c640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4498004800 0x7f44ac0ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44a3fff640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0b90c0 0x7f44ac0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 -- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0b90c0 msgr2=0x7f44ac0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0b90c0 0x7f44ac0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 -- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4498004800 msgr2=0x7f44ac0ba6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4498004800 0x7f44ac0ba6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 -- 172.21.15.53:0/2684472075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac0bac30 con 0x7f44ac0ce320 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44aad9d640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44ac0ce320 0x7f44ac161ee0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f44ac08d260 tx=0x7f44a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44a2ffd640 1 -- 172.21.15.53:0/2684472075 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4011e10 con 0x7f44ac0ce320 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44a2ffd640 1 -- 172.21.15.53:0/2684472075 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a401b070 con 0x7f44ac0ce320 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac0b9650 con 0x7f44ac0ce320 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44a2ffd640 1 -- 172.21.15.53:0/2684472075 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4003990 con 0x7f44ac0ce320 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.150+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac0b9c10 con 0x7f44ac0ce320 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.151+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac06a220 con 0x7f44ac0ce320 2024-05-10T10:35:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.152+0000 7f44a2ffd640 1 -- 172.21.15.53:0/2684472075 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f44a4005d10 con 0x7f44ac0ce320 2024-05-10T10:35:05.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.152+0000 7f44a2ffd640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c076300 0x7f448c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.153+0000 7f44a2ffd640 1 -- 172.21.15.53:0/2684472075 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f44a4097bd0 con 0x7f44ac0ce320 2024-05-10T10:35:05.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.153+0000 7f44aa59c640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c076300 0x7f448c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.153+0000 7f44aa59c640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c076300 0x7f448c0787c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f449c001550 tx=0x7f449c0078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.157+0000 7f44a2ffd640 1 -- 172.21.15.53:0/2684472075 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f44a40613a0 con 0x7f44ac0ce320 2024-05-10T10:35:05.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.391+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44ac0d52f0 con 0x7f44ac0ce320 2024-05-10T10:35:05.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.392+0000 7f44a2ffd640 1 -- 172.21.15.53:0/2684472075 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7f44a4060ca0 con 0x7f44ac0ce320 2024-05-10T10:35:05.392 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:05.392 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c076300 msgr2=0x7f448c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c076300 0x7f448c0787c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f449c001550 tx=0x7f449c0078f0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44ac0ce320 msgr2=0x7f44ac161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44ac0ce320 0x7f44ac161ee0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f44ac08d260 tx=0x7f44a40115a0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 shutdown_connections 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c076300 0x7f448c0787c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44ac0ce320 0x7f44ac161ee0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44ac0b90c0 0x7f44ac0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 --2- 172.21.15.53:0/2684472075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4498004800 0x7f44ac0ba6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 >> 172.21.15.53:0/2684472075 conn(0x7f44ac0d4ee0 msgr2=0x7f44ac06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 shutdown_connections 2024-05-10T10:35:05.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.394+0000 7f44ab59e640 1 -- 172.21.15.53:0/2684472075 wait complete. 2024-05-10T10:35:05.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-10T10:35:05.533 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:35:05.533 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:35:05.533 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-10T10:35:05.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.577+0000 7f2ca2f59f40 1 Processor -- start 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.577+0000 7f2ca2f59f40 1 -- start start 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca2f59f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c4998d65c0 0x55c4998d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca2f59f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998eed60 0x55c4998d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca2f59f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c49997b690 0x55c4998ea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca2f59f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c49997ba70 con 0x55c4998d65c0 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca2f59f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c49984df50 con 0x55c49997b690 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca2f59f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c4998ef260 con 0x55c4998eed60 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c4998d65c0 0x55c4998d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c4998d65c0 0x55c4998d3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48104/0 (socket says 172.21.15.53:48104) 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 -- 172.21.15.53:0/354503605 learned_addr learned my addr 172.21.15.53:0/354503605 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0252640 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998eed60 0x55c4998d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca1254640 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c49997b690 0x55c4998ea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 -- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998eed60 msgr2=0x55c4998d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998eed60 0x55c4998d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 -- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c49997b690 msgr2=0x55c4998ea850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c49997b690 0x55c4998ea850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 -- 172.21.15.53:0/354503605 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4999732b0 con 0x55c4998d65c0 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca1254640 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c49997b690 0x55c4998ea850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2ca0a53640 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c4998d65c0 0x55c4998d3ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2c90025ae0 tx=0x7f2c90051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b06144ff009e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.578+0000 7f2c9f250640 1 -- 172.21.15.53:0/354503605 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90067070 con 0x55c4998d65c0 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2c9f250640 1 -- 172.21.15.53:0/354503605 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c90058b50 con 0x55c4998d65c0 2024-05-10T10:35:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2c9f250640 1 -- 172.21.15.53:0/354503605 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90062430 con 0x55c4998d65c0 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/354503605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c4998d65c0 msgr2=0x55c4998d3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c4998d65c0 0x55c4998d3ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2c90025ae0 tx=0x7f2c90051000 comp rx=0 tx=0).stop 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/354503605 shutdown_connections 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c49997b690 0x55c4998ea850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998eed60 0x55c4998d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/354503605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c4998d65c0 0x55c4998d3ca0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/354503605 >> 172.21.15.53:0/354503605 conn(0x55c4998a6040 msgr2=0x55c4998a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/354503605 shutdown_connections 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.579+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/354503605 wait complete. 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 Processor -- start 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 -- start start 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998d65c0 0x55c499979a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4998eed60 0x55c49997c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c49997b690 0x55c499a0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c4998a47b0 con 0x55c49997b690 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c499972fd0 con 0x55c4998eed60 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca2f59f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c4998a4a90 con 0x55c4998d65c0 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca1254640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c49997b690 0x55c499a0caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca1254640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c49997b690 0x55c499a0caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48112/0 (socket says 172.21.15.53:48112) 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.580+0000 7f2ca1254640 1 -- 172.21.15.53:0/2499918257 learned_addr learned my addr 172.21.15.53:0/2499918257 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca0a53640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998d65c0 0x55c499979a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca0252640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4998eed60 0x55c49997c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca1254640 1 -- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998d65c0 msgr2=0x55c499979a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca1254640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c4998d65c0 0x55c499979a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca1254640 1 -- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4998eed60 msgr2=0x55c49997c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca1254640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4998eed60 0x55c49997c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca1254640 1 -- 172.21.15.53:0/2499918257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c499974bc0 con 0x55c49997b690 2024-05-10T10:35:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca1254640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c49997b690 0x55c499a0caa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2c98007dd0 tx=0x7f2c98006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2c9e24e640 1 -- 172.21.15.53:0/2499918257 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98006e60 con 0x55c49997b690 2024-05-10T10:35:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2c9e24e640 1 -- 172.21.15.53:0/2499918257 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c98011700 con 0x55c49997b690 2024-05-10T10:35:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2c9e24e640 1 -- 172.21.15.53:0/2499918257 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98011ad0 con 0x55c49997b690 2024-05-10T10:35:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c4998d5dd0 con 0x55c49997b690 2024-05-10T10:35:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.581+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49997c5a0 con 0x55c49997b690 2024-05-10T10:35:05.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.583+0000 7f2c9e24e640 1 -- 172.21.15.53:0/2499918257 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2c98011c70 con 0x55c49997b690 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.583+0000 7f2c9e24e640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2c80074cb0 0x7f2c80077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.583+0000 7f2c9e24e640 1 -- 172.21.15.53:0/2499918257 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f2c98095a80 con 0x55c49997b690 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.583+0000 7f2ca0a53640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2c80074cb0 0x7f2c80077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.583+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55c4998ee690 0x55c4999739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.583+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48) -- 0x55c49996fa30 con 0x55c4998ee690 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca0252640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55c4998ee690 0x55c4999739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca0a53640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2c80074cb0 0x7f2c80077170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005fd0 tx=0x7f2c90052000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca0252640 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55c4998ee690 0x55c4999739c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca0252640 1 -- 172.21.15.53:0/2499918257 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f2c8800f620 con 0x55c4998ee690 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55c4998ee690 msgr2=0x55c4999739c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55c4998ee690 0x55c4999739c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2c80074cb0 msgr2=0x7f2c80077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2c80074cb0 0x7f2c80077170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005fd0 tx=0x7f2c90052000 comp rx=0 tx=0).stop 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c49997b690 msgr2=0x55c499a0caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 --2- 172.21.15.53:0/2499918257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c49997b690 0x55c499a0caa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2c98007dd0 tx=0x7f2c98006750 comp rx=0 tx=0).stop 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca1254640 1 -- 172.21.15.53:0/2499918257 reap_dead start 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 shutdown_connections 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 >> 172.21.15.53:0/2499918257 conn(0x55c4998a6040 msgr2=0x55c4998a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 shutdown_connections 2024-05-10T10:35:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.584+0000 7f2ca2f59f40 1 -- 172.21.15.53:0/2499918257 wait complete. 2024-05-10T10:35:05.592 INFO:teuthology.orchestra.run.smithi053.stdout:Yalphaparent_a 2024-05-10T10:35:05.593 DEBUG:teuthology.orchestra.run.smithi053:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-10T10:35:05.691 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "ancestors": [ 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1099511627776, 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "alpha", 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 2 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1, 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "parent_a", 2024-05-10T10:35:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 4 2024-05-10T10:35:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-05-10T10:35:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 8, 2024-05-10T10:35:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "old_pools": [] 2024-05-10T10:35:05.693 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:05.711 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-05-10T10:35:05.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.757+0000 7f6893ab1f40 1 Processor -- start 2024-05-10T10:35:05.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.757+0000 7f6893ab1f40 1 -- start start 2024-05-10T10:35:05.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6893ab1f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b580c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6893ab1f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580dd870 0x558b580989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6893ab1f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580e0b40 0x558b580e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6893ab1f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558b5803df50 con 0x558b580dd870 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6893ab1f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558b58041f90 con 0x7f68800022f0 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6893ab1f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558b58099e30 con 0x558b580e0b40 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580dd870 0x558b580989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580dd870 0x558b580989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48124/0 (socket says 172.21.15.53:48124) 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 -- 172.21.15.53:0/4253533433 learned_addr learned my addr 172.21.15.53:0/4253533433 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6891dac640 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580e0b40 0x558b580e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f68915ab640 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b580c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 -- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580e0b40 msgr2=0x558b580e0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580e0b40 0x558b580e0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 -- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 msgr2=0x558b580c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b580c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 -- 172.21.15.53:0/4253533433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b58154c40 con 0x558b580dd870 2024-05-10T10:35:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f68915ab640 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b580c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6891dac640 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580e0b40 0x558b580e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f6890daa640 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580dd870 0x558b580989a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f687c009680 tx=0x7f687c009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3317fca00d2fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f688b7fe640 1 -- 172.21.15.53:0/4253533433 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00e960 con 0x558b580dd870 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f688b7fe640 1 -- 172.21.15.53:0/4253533433 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c013030 con 0x558b580dd870 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.758+0000 7f688b7fe640 1 -- 172.21.15.53:0/4253533433 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00ed40 con 0x558b580dd870 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 -- 172.21.15.53:0/4253533433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580dd870 msgr2=0x558b580989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580dd870 0x558b580989a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f687c009680 tx=0x7f687c009b40 comp rx=0 tx=0).stop 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 -- 172.21.15.53:0/4253533433 shutdown_connections 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580e0b40 0x558b580e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580dd870 0x558b580989a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/4253533433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b580c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 -- 172.21.15.53:0/4253533433 >> 172.21.15.53:0/4253533433 conn(0x558b580c3940 msgr2=0x558b58155870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 -- 172.21.15.53:0/4253533433 shutdown_connections 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 -- 172.21.15.53:0/4253533433 wait complete. 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.759+0000 7f6893ab1f40 1 Processor -- start 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6893ab1f40 1 -- start start 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6893ab1f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580dd870 0x558b58170920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6893ab1f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580e0b40 0x558b5816d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6893ab1f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b58155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6893ab1f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558b580c1dd0 con 0x558b580e0b40 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6893ab1f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558b580e0790 con 0x7f68800022f0 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6893ab1f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558b580c21b0 con 0x558b580dd870 2024-05-10T10:35:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6890daa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580e0b40 0x558b5816d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6890daa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580e0b40 0x558b5816d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48134/0 (socket says 172.21.15.53:48134) 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.760+0000 7f6890daa640 1 -- 172.21.15.53:0/2237778710 learned_addr learned my addr 172.21.15.53:0/2237778710 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f68915ab640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580dd870 0x558b58170920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6891dac640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b58155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6890daa640 1 -- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580dd870 msgr2=0x558b58170920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6890daa640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558b580dd870 0x558b58170920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6890daa640 1 -- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 msgr2=0x558b58155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6890daa640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68800022f0 0x558b58155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6890daa640 1 -- 172.21.15.53:0/2237778710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b5814f9f0 con 0x558b580e0b40 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6890daa640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580e0b40 0x558b5816d680 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f687c001ed0 tx=0x7f687c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f688a7fc640 1 -- 172.21.15.53:0/2237778710 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00f860 con 0x558b580e0b40 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f688a7fc640 1 -- 172.21.15.53:0/2237778710 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c00fa20 con 0x558b580e0b40 2024-05-10T10:35:05.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558b5814f610 con 0x558b580e0b40 2024-05-10T10:35:05.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f688a7fc640 1 -- 172.21.15.53:0/2237778710 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00fe10 con 0x558b580e0b40 2024-05-10T10:35:05.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.761+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b58170580 con 0x558b580e0b40 2024-05-10T10:35:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.763+0000 7f688a7fc640 1 -- 172.21.15.53:0/2237778710 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f687c024070 con 0x558b580e0b40 2024-05-10T10:35:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.763+0000 7f688a7fc640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6874074cb0 0x7f6874077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.763+0000 7f688a7fc640 1 -- 172.21.15.53:0/2237778710 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f687c098920 con 0x558b580e0b40 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.763+0000 7f68915ab640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6874074cb0 0x7f6874077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.763+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558b581fdb90 0x558b581fdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.763+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) -- 0x558b580c1890 con 0x558b581fdb90 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.764+0000 7f6891dac640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558b581fdb90 0x558b581fdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.764+0000 7f6891dac640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558b581fdb90 0x558b581fdfe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.764+0000 7f68915ab640 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6874074cb0 0x7f6874077170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6884009810 tx=0x7f6884052000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6891dac640 1 -- 172.21.15.53:0/2237778710 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v48'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f688c0112c0 con 0x558b581fdb90 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558b581fdb90 msgr2=0x558b581fdfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558b581fdb90 0x558b581fdfe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6874074cb0 msgr2=0x7f6874077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6874074cb0 0x7f6874077170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6884009810 tx=0x7f6884052000 comp rx=0 tx=0).stop 2024-05-10T10:35:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580e0b40 msgr2=0x558b5816d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 --2- 172.21.15.53:0/2237778710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b580e0b40 0x558b5816d680 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f687c001ed0 tx=0x7f687c001970 comp rx=0 tx=0).stop 2024-05-10T10:35:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6891dac640 1 -- 172.21.15.53:0/2237778710 reap_dead start 2024-05-10T10:35:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 shutdown_connections 2024-05-10T10:35:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 >> 172.21.15.53:0/2237778710 conn(0x558b580c3940 msgr2=0x558b580c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 shutdown_connections 2024-05-10T10:35:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.765+0000 7f6893ab1f40 1 -- 172.21.15.53:0/2237778710 wait complete. 2024-05-10T10:35:05.771 DEBUG:teuthology.orchestra.run.smithi053:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-10T10:35:05.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:05.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-05-10T10:35:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.883+0000 7f98e9378640 1 Processor -- start 2024-05-10T10:35:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.883+0000 7f98e9378640 1 -- start start 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.884+0000 7f98e9378640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e415ce70 0x7f98e41601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.884+0000 7f98e9378640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f98e40732e0 0x7f98e406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.884+0000 7f98e9378640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e4160730 con 0x7f98e406d810 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.884+0000 7f98e9378640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f98e4160930 con 0x7f98e40732e0 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.884+0000 7f98e9378640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f98e4073810 con 0x7f98e415ce70 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e37fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f98e406d810 0x7f98e4160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34130/0 (socket says 172.21.15.53:34130) 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e37fe640 1 -- 172.21.15.53:0/1900141467 learned_addr learned my addr 172.21.15.53:0/1900141467 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e27fc640 1 --2- 172.21.15.53:0/1900141467 >> v2:172.21.15.135:3300/0 conn(0x7f98e40732e0 0x7f98e406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e415ce70 0x7f98e41601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e1ffb640 1 -- 172.21.15.53:0/1900141467 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 119757780 0 0) 0x7f98dc001490 con 0x7f98e406d810 2024-05-10T10:35:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e1ffb640 1 -- 172.21.15.53:0/1900141467 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98cc003520 con 0x7f98e406d810 2024-05-10T10:35:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e27fc640 1 -- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e415ce70 msgr2=0x7f98e41601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e27fc640 1 --2- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e415ce70 0x7f98e41601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e27fc640 1 -- 172.21.15.53:0/1900141467 >> v1:172.21.15.53:6789/0 conn(0x7f98e406d810 legacy=0x7f98e4160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.885+0000 7f98e27fc640 1 -- 172.21.15.53:0/1900141467 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e40af340 con 0x7f98e40732e0 2024-05-10T10:35:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.886+0000 7f98e27fc640 1 --2- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e406d2d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097a0 tx=0x7f98d802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92aeff5c7344883e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.886+0000 7f98e1ffb640 1 -- 172.21.15.53:0/1900141467 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8037070 con 0x7f98e40732e0 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.886+0000 7f98e1ffb640 1 -- 172.21.15.53:0/1900141467 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d8032440 con 0x7f98e40732e0 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.886+0000 7f98e1ffb640 1 -- 172.21.15.53:0/1900141467 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d80327f0 con 0x7f98e40732e0 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 -- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 msgr2=0x7f98e406d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 --2- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e406d2d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097a0 tx=0x7f98d802a210 comp rx=0 tx=0).stop 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 -- 172.21.15.53:0/1900141467 shutdown_connections 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 --2- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e406d2d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 --2- 172.21.15.53:0/1900141467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e415ce70 0x7f98e41601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 -- 172.21.15.53:0/1900141467 >> 172.21.15.53:0/1900141467 conn(0x7f98e40727c0 msgr2=0x7f98e4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 -- 172.21.15.53:0/1900141467 shutdown_connections 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.887+0000 7f98e9378640 1 -- 172.21.15.53:0/1900141467 wait complete. 2024-05-10T10:35:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.888+0000 7f98e9378640 1 Processor -- start 2024-05-10T10:35:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.888+0000 7f98e9378640 1 -- start start 2024-05-10T10:35:05.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e9378640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e406d810 0x7f98e4145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e9378640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e4145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e9378640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e415ce70 0x7f98e41f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e9378640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4161520 con 0x7f98e415ce70 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e9378640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98e40af900 con 0x7f98e40732e0 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e9378640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f98e41616a0 con 0x7f98e406d810 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e415ce70 0x7f98e41f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e406d810 0x7f98e4145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e27fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e4145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e415ce70 0x7f98e41f8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48138/0 (socket says 172.21.15.53:48138) 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e406d810 0x7f98e4145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:42268/0 (socket says 172.21.15.53:42268) 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e37fe640 1 -- 172.21.15.53:0/261696945 learned_addr learned my addr 172.21.15.53:0/261696945 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e37fe640 1 -- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e406d810 msgr2=0x7f98e4145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e37fe640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e406d810 0x7f98e4145a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e37fe640 1 -- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 msgr2=0x7f98e4145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.889+0000 7f98e37fe640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e4145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98e37fe640 1 -- 172.21.15.53:0/261696945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e4152ae0 con 0x7f98e415ce70 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e406d810 0x7f98e4145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98e27fc640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e4145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98e37fe640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e415ce70 0x7f98e41f8510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00cad0 tx=0x7f98dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98cb7fe640 1 -- 172.21.15.53:0/261696945 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc017070 con 0x7f98e415ce70 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98cb7fe640 1 -- 172.21.15.53:0/261696945 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc00ac30 con 0x7f98e415ce70 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e415f820 con 0x7f98e415ce70 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98cb7fe640 1 -- 172.21.15.53:0/261696945 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc012400 con 0x7f98e415ce70 2024-05-10T10:35:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.890+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e4154c20 con 0x7f98e415ce70 2024-05-10T10:35:05.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.892+0000 7f98cb7fe640 1 -- 172.21.15.53:0/261696945 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f98dc01a070 con 0x7f98e415ce70 2024-05-10T10:35:05.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.893+0000 7f98ec16a740 1 Processor -- start 2024-05-10T10:35:05.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.893+0000 7f98cb7fe640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f98d00762d0 0x7f98d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.893+0000 7f98cb7fe640 1 -- 172.21.15.53:0/261696945 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f98dc097500 con 0x7f98e415ce70 2024-05-10T10:35:05.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.893+0000 7f98e27fc640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f98d00762d0 0x7f98d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.893+0000 7f98ec16a740 1 -- start start 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98e27fc640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f98d00762d0 0x7f98d0078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98d8034000 tx=0x7f98d802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5601341fcc00 0x5601341d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98ec16a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560134135cf0 con 0x5601341fd340 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601340eee10 con 0x56013421a220 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560134131cb0 con 0x5601341fcc00 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98e27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98e27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48152/0 (socket says 172.21.15.53:48152) 2024-05-10T10:35:05.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98e27fc640 1 -- 172.21.15.53:0/2197908348 learned_addr learned my addr 172.21.15.53:0/2197908348 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98e37fe640 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5601341fcc00 0x5601341d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.894+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.895+0000 7f98e27fc640 1 -- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5601341fcc00 msgr2=0x5601341d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.895+0000 7f98e27fc640 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5601341fcc00 0x5601341d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.895+0000 7f98e27fc640 1 -- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 msgr2=0x5601341ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.895+0000 7f98e27fc640 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.895+0000 7f98e27fc640 1 -- 172.21.15.53:0/2197908348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560133f0f7e0 con 0x5601341fd340 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.895+0000 7f98e27fc640 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341fc6c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009560 tx=0x7f98d802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1073b81678a70c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.896+0000 7f98c8ff9640 1 -- 172.21.15.53:0/2197908348 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8037040 con 0x5601341fd340 2024-05-10T10:35:05.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.896+0000 7f98c8ff9640 1 -- 172.21.15.53:0/2197908348 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d803c440 con 0x5601341fd340 2024-05-10T10:35:05.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.896+0000 7f98c8ff9640 1 -- 172.21.15.53:0/2197908348 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d803c7f0 con 0x5601341fd340 2024-05-10T10:35:05.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 -- 172.21.15.53:0/2197908348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 msgr2=0x5601341fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341fc6c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009560 tx=0x7f98d802c000 comp rx=0 tx=0).stop 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 -- 172.21.15.53:0/2197908348 shutdown_connections 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5601341fcc00 0x5601341d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341fc6c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2197908348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 -- 172.21.15.53:0/2197908348 >> 172.21.15.53:0/2197908348 conn(0x5601341cee20 msgr2=0x5601342176c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 -- 172.21.15.53:0/2197908348 shutdown_connections 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 -- 172.21.15.53:0/2197908348 wait complete. 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 -- start start 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.897+0000 7f98ec16a740 1 Processor -- start 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134215ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x560134218710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98ec16a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601341d0fd0 con 0x5601341fd340 2024-05-10T10:35:05.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601340f2940 con 0x56013421a220 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5601341d03c0 con 0x56013420e380 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98ec16a740 10 client.0 _ll_register_callbacks cb 0x5601340f5440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134215ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134215ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48162/0 (socket says 172.21.15.53:48162) 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98e2ffd640 1 -- 172.21.15.53:0/335986628 learned_addr learned my addr 172.21.15.53:0/335986628 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98e27fc640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.898+0000 7f98e37fe640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x560134218710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98e2ffd640 1 -- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 msgr2=0x560134218710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x560134218710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98e2ffd640 1 -- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 msgr2=0x5601341f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98e2ffd640 1 -- 172.21.15.53:0/335986628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560133f6ff70 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98e27fc640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134215ca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fd0 tx=0x7f98d4012330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f64284253c8847 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98c1ffb640 1 -- 172.21.15.53:0/335986628 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4022060 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98c1ffb640 1 -- 172.21.15.53:0/335986628 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f98d4012d70 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d803c440 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98c1ffb640 1 -- 172.21.15.53:0/335986628 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98d4028050 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98ec16a740 10 client.5409 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x560133e07170 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.899+0000 7f98c1ffb640 1 -- 172.21.15.53:0/335986628 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d803c440 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98c1ffb640 1 -- 172.21.15.53:0/335986628 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f98d402b020 con 0x5601341fd340 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98ec16a740 10 client.5409 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98ec16a740 10 client.5409 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-10T10:35:05.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98ec16a740 20 client.5409 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98ec16a740 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x56013422aed0 0x56013422b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98ec16a740 4 client.5409 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560134143900 con 0x56013422aed0 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98e27fc640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x56013422aed0 0x56013422b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98c1ffb640 10 client.5409 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.900+0000 7f98e27fc640 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x56013422aed0 0x56013422b330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cb64deaf6abfc2 server_cookie=753879a5c894dc78 in_seq=0 out_seq=0 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98c1ffb640 1 -- 172.21.15.53:0/335986628 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560134143900 con 0x56013422aed0 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98c1ffb640 10 client.5409 handle_command_reply: tid=0 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 client.5409 shutdown 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 msgr2=0x560134215ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134215ca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fd0 tx=0x7f98d4012330 comp rx=0 tx=0).stop 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 shutdown_connections 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x56013422aed0 0x56013422b330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x560134218710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 --2- 172.21.15.53:0/335986628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134215ca0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98c1ffb640 0 client.5409 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 >> 172.21.15.53:0/335986628 conn(0x5601341fcc00 msgr2=0x5601341cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 shutdown_connections 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 1 -- 172.21.15.53:0/335986628 wait complete. 2024-05-10T10:35:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.901+0000 7f98ec16a740 20 client.5409 trim_cache size 0 max 16384 2024-05-10T10:35:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.904+0000 7f7087778640 1 Processor -- start 2024-05-10T10:35:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.904+0000 7f7087778640 1 -- start start 2024-05-10T10:35:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7087778640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f70800b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7087778640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70800cc300 con 0x7f70800b3450 2024-05-10T10:35:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7087778640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70800b64d0 con 0x7f7080162810 2024-05-10T10:35:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7087778640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f70800b5c30 con 0x7f70800d5ea0 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7085d15640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f70800b3450 0x7f70800b38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34146/0 (socket says 172.21.15.53:34146) 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7085d15640 1 -- 172.21.15.53:0/3125823007 learned_addr learned my addr 172.21.15.53:0/3125823007 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7084d13640 1 --2- 172.21.15.53:0/3125823007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f70800b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7077fff640 1 -- 172.21.15.53:0/3125823007 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154127323 0 0) 0x7f7078001490 con 0x7f70800b3450 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7077fff640 1 -- 172.21.15.53:0/3125823007 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f706c003520 con 0x7f70800b3450 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7077fff640 1 -- 172.21.15.53:0/3125823007 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299837032 0 0) 0x7f70800b64d0 con 0x7f7080162810 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7077fff640 1 -- 172.21.15.53:0/3125823007 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7078001490 con 0x7f7080162810 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7084d13640 1 -- 172.21.15.53:0/3125823007 >> v1:172.21.15.135:6789/0 conn(0x7f7080162810 legacy=0x7f70800b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7084d13640 1 -- 172.21.15.53:0/3125823007 >> v1:172.21.15.53:6789/0 conn(0x7f70800b3450 legacy=0x7f70800b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.905+0000 7f7084d13640 1 -- 172.21.15.53:0/3125823007 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70800b5f10 con 0x7f70800d5ea0 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.906+0000 7f7084d13640 1 --2- 172.21.15.53:0/3125823007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f70800b2590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70700097d0 tx=0x7f707002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af55df89a038bb49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.906+0000 7f7077fff640 1 -- 172.21.15.53:0/3125823007 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070037070 con 0x7f70800d5ea0 2024-05-10T10:35:05.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.906+0000 7f7077fff640 1 -- 172.21.15.53:0/3125823007 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7070032440 con 0x7f70800d5ea0 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.906+0000 7f7077fff640 1 -- 172.21.15.53:0/3125823007 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070032810 con 0x7f70800d5ea0 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.907+0000 7f7087778640 1 -- 172.21.15.53:0/3125823007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 msgr2=0x7f70800b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.907+0000 7f7087778640 1 --2- 172.21.15.53:0/3125823007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f70800b2590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70700097d0 tx=0x7f707002a210 comp rx=0 tx=0).stop 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.907+0000 7f7087778640 1 -- 172.21.15.53:0/3125823007 shutdown_connections 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.907+0000 7f7087778640 1 --2- 172.21.15.53:0/3125823007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f70800b2590 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.907+0000 7f7087778640 1 -- 172.21.15.53:0/3125823007 >> 172.21.15.53:0/3125823007 conn(0x7f708009e590 msgr2=0x7f70800d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.907+0000 7f7087778640 1 -- 172.21.15.53:0/3125823007 shutdown_connections 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.907+0000 7f7087778640 1 -- 172.21.15.53:0/3125823007 wait complete. 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.908+0000 7f7087778640 1 Processor -- start 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.908+0000 7f7087778640 1 -- start start 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7087778640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70800b3450 0x7f70801576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7087778640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f708006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7087778640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7080162810 0x7f708006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7087778640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f70800b2f30 con 0x7f70800b3450 2024-05-10T10:35:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7087778640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70800b61f0 con 0x7f7080162810 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7087778640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f70800b58e0 con 0x7f70800d5ea0 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7085514640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70800b3450 0x7f70801576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7085514640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70800b3450 0x7f70801576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48176/0 (socket says 172.21.15.53:48176) 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7085514640 1 -- 172.21.15.53:0/2709305607 learned_addr learned my addr 172.21.15.53:0/2709305607 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7084d13640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f708006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.909+0000 7f7085d15640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7080162810 0x7f708006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7085514640 1 -- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 msgr2=0x7f708006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7085514640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70800d5ea0 0x7f708006ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7085514640 1 -- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7080162810 msgr2=0x7f708006d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7085514640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7080162810 0x7f708006d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7085514640 1 -- 172.21.15.53:0/2709305607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70800d0930 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7085514640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70800b3450 0x7f70801576d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c390 tx=0x7f707c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c017070 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707c0052f0 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c003640 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7087778640 1 -- 172.21.15.53:0/2709305607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70800d0c10 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.910+0000 7f7087778640 1 -- 172.21.15.53:0/2709305607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70800d11d0 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.911+0000 7f7087778640 1 -- 172.21.15.53:0/2709305607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f708006a220 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.912+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f707c01a070 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.912+0000 7f7075ffb640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70640762d0 0x7f7064078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.912+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f707c097240 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.912+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+508 (secure 0 0 0) 0x7f707c021070 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.912+0000 7f7084d13640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70640762d0 0x7f7064078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.912+0000 7f7087778640 1 -- 172.21.15.53:0/2709305607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f708006cbb0 con 0x7f70800b3450 2024-05-10T10:35:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.913+0000 7f7084d13640 1 --2- 172.21.15.53:0/2709305607 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70640762d0 0x7f7064078790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7070034000 tx=0x7f707002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.921+0000 7f98ec16a740 1 Processor -- start 2024-05-10T10:35:05.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.921+0000 7f98ec16a740 1 -- start start 2024-05-10T10:35:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134218710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013422a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98ec16a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4201cd0 con 0x5601341fd340 2024-05-10T10:35:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4200d10 con 0x56013421a220 2024-05-10T10:35:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f98e415ee40 con 0x56013420e380 2024-05-10T10:35:05.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134218710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134218710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48178/0 (socket says 172.21.15.53:48178) 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 -- 172.21.15.53:0/2081089779 learned_addr learned my addr 172.21.15.53:0/2081089779 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e27fc640 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013422a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e37fe640 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 -- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 msgr2=0x56013422a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013422a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 -- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 msgr2=0x5601341c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e2ffd640 1 -- 172.21.15.53:0/2081089779 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601341f9630 con 0x5601341fd340 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e27fc640 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013422a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.922+0000 7f98e37fe640 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134218710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006d40 tx=0x7f98d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49807867fcc1580e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98c3fff640 1 -- 172.21.15.53:0/2081089779 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4031070 con 0x5601341fd340 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98c3fff640 1 -- 172.21.15.53:0/2081089779 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d4005990 con 0x5601341fd340 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98c3fff640 1 -- 172.21.15.53:0/2081089779 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4005b40 con 0x5601341fd340 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98ec16a740 1 -- 172.21.15.53:0/2081089779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 msgr2=0x560134218710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134218710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006d40 tx=0x7f98d4017000 comp rx=0 tx=0).stop 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98ec16a740 1 -- 172.21.15.53:0/2081089779 shutdown_connections 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x5601341c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013422a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98ec16a740 1 --2- 172.21.15.53:0/2081089779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x560134218710 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.923+0000 7f98ec16a740 1 -- 172.21.15.53:0/2081089779 >> 172.21.15.53:0/2081089779 conn(0x5601341fcc00 msgr2=0x56013422a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.924+0000 7f98ec16a740 1 -- 172.21.15.53:0/2081089779 shutdown_connections 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.924+0000 7f98ec16a740 1 -- 172.21.15.53:0/2081089779 wait complete. 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.924+0000 7f98ec16a740 1 -- start start 2024-05-10T10:35:05.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.924+0000 7f98ec16a740 1 Processor -- start 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013428fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98ec16a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x56013421a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98ec16a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601341d02c0 con 0x5601341fd340 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98e415f490 con 0x56013421a220 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98ec16a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560134227c30 con 0x56013420e380 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98ec16a740 10 client.0 _ll_register_callbacks cb 0x7f98d4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48186/0 (socket says 172.21.15.53:48186) 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 -- 172.21.15.53:0/858158054 learned_addr learned my addr 172.21.15.53:0/858158054 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:05.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e27fc640 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013428fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e37fe640 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x56013421a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 -- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 msgr2=0x56013428fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013428fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 -- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 msgr2=0x56013421a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x56013421a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.925+0000 7f98e2ffd640 1 -- 172.21.15.53:0/858158054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560134295e30 con 0x5601341fd340 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98e2ffd640 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341d0b80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fd0 tx=0x7f98d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c60ce0b7db508 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98c1ffb640 1 -- 172.21.15.53:0/858158054 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4022070 con 0x5601341fd340 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98c1ffb640 1 -- 172.21.15.53:0/858158054 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f98d402dc30 con 0x5601341fd340 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d4005990 con 0x5601341fd340 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98c1ffb640 1 -- 172.21.15.53:0/858158054 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98d403b050 con 0x5601341fd340 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 10 client.5421 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:05.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x560134296970 con 0x5601341fd340 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98c1ffb640 1 -- 172.21.15.53:0/858158054 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d4005990 con 0x5601341fd340 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98c1ffb640 1 -- 172.21.15.53:0/858158054 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f98d403e020 con 0x5601341fd340 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 10 client.5421 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 10 client.5421 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 20 client.5421 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x5601341d3ba0 0x56013421aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 4 client.5421 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x56013413d770 con 0x5601341d3ba0 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.926+0000 7f98e27fc640 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x5601341d3ba0 0x56013421aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98c1ffb640 10 client.5421 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98e27fc640 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x5601341d3ba0 0x56013421aed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b8eee6a5b522aad server_cookie=cea266155531e729 in_seq=0 out_seq=0 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98c1ffb640 1 -- 172.21.15.53:0/858158054 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56013413d770 con 0x5601341d3ba0 2024-05-10T10:35:05.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98c1ffb640 10 client.5421 handle_command_reply: tid=0 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 client.5421 shutdown 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 msgr2=0x5601341d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341d0b80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009fd0 tx=0x7f98d40046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 shutdown_connections 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x5601341d3ba0 0x56013421aed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56013421a220 0x56013421a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56013420e380 0x56013428fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 --2- 172.21.15.53:0/858158054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601341fd340 0x5601341d0b80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98c1ffb640 0 client.5421 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.927+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 >> 172.21.15.53:0/858158054 conn(0x5601341fcc00 msgr2=0x5601341cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 shutdown_connections 2024-05-10T10:35:05.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98ec16a740 1 -- 172.21.15.53:0/858158054 wait complete. 2024-05-10T10:35:05.936 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:35:05.928+0000 7fbcf1f85640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5370=0-4194304@1} caps={5370=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5370 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b057daac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:35:05.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98ec16a740 20 client.5421 trim_cache size 0 max 16384 2024-05-10T10:35:05.936 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:35:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "907a597a-41f1-489a-aec1-01a7a43cceaa", 2024-05-10T10:35:05.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:35:05.937 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:05.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f98d00762d0 msgr2=0x7f98d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f98d00762d0 0x7f98d0078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98d8034000 tx=0x7f98d802e000 comp rx=0 tx=0).stop 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e415ce70 msgr2=0x7f98e41f8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e415ce70 0x7f98e41f8510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00cad0 tx=0x7f98dc00cf90 comp rx=0 tx=0).stop 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 shutdown_connections 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f98d00762d0 0x7f98d0078790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e415ce70 0x7f98e41f8510 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98e40732e0 0x7f98e4145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 --2- 172.21.15.53:0/261696945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f98e406d810 0x7f98e4145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 >> 172.21.15.53:0/261696945 conn(0x7f98e40727c0 msgr2=0x7f98e406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 shutdown_connections 2024-05-10T10:35:05.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:05.928+0000 7f98e9378640 1 -- 172.21.15.53:0/261696945 wait complete. 2024-05-10T10:35:05.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:05.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-10T10:35:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.044+0000 7f11ba8e1640 1 Processor -- start 2024-05-10T10:35:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.044+0000 7f11ba8e1640 1 -- start start 2024-05-10T10:35:06.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11ba8e1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f11b40d3e80 0x7f11b40b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11ba8e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d2230 0x7f11b40b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11ba8e1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11b4162fa0 con 0x7f11b4162bc0 2024-05-10T10:35:06.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11ba8e1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f11b40d1170 con 0x7f11b40d3e80 2024-05-10T10:35:06.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11ba8e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f11b40d2f70 con 0x7f11b40d2230 2024-05-10T10:35:06.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11abfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f11b40d3e80 0x7f11b40b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11b3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f11b4162bc0 0x7f11b40d3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34154/0 (socket says 172.21.15.53:34154) 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11abfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f11b40d3e80 0x7f11b40b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48198/0 (socket says 172.21.15.53:48198) 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11b3fff640 1 -- 172.21.15.53:0/1769366117 learned_addr learned my addr 172.21.15.53:0/1769366117 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.045+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1769366117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d2230 0x7f11b40b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11abfff640 1 -- 172.21.15.53:0/1769366117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d2230 msgr2=0x7f11b40b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11abfff640 1 --2- 172.21.15.53:0/1769366117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d2230 0x7f11b40b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11abfff640 1 -- 172.21.15.53:0/1769366117 >> v1:172.21.15.53:6789/0 conn(0x7f11b4162bc0 legacy=0x7f11b40d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11abfff640 1 -- 172.21.15.53:0/1769366117 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b40d3250 con 0x7f11b40d3e80 2024-05-10T10:35:06.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11b37fe640 1 -- 172.21.15.53:0/1769366117 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3500385034 0 0) 0x7f11b4162fa0 con 0x7f11b4162bc0 2024-05-10T10:35:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11abfff640 1 --2- 172.21.15.53:0/1769366117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b40d3e80 0x7f11b40b24f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f119c0097a0 tx=0x7f119c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f51c86d6736671e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11b37fe640 1 -- 172.21.15.53:0/1769366117 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c037070 con 0x7f11b40d3e80 2024-05-10T10:35:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.046+0000 7f11b37fe640 1 -- 172.21.15.53:0/1769366117 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c02ae50 con 0x7f11b40d3e80 2024-05-10T10:35:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11b37fe640 1 -- 172.21.15.53:0/1769366117 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c032660 con 0x7f11b40d3e80 2024-05-10T10:35:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 -- 172.21.15.53:0/1769366117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b40d3e80 msgr2=0x7f11b40b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/1769366117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b40d3e80 0x7f11b40b24f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f119c0097a0 tx=0x7f119c02a100 comp rx=0 tx=0).stop 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 -- 172.21.15.53:0/1769366117 shutdown_connections 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/1769366117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d2230 0x7f11b40b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/1769366117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b40d3e80 0x7f11b40b24f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 -- 172.21.15.53:0/1769366117 >> 172.21.15.53:0/1769366117 conn(0x7f11b40d5190 msgr2=0x7f11b40d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 -- 172.21.15.53:0/1769366117 shutdown_connections 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.047+0000 7f11ba8e1640 1 -- 172.21.15.53:0/1769366117 wait complete. 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.048+0000 7f11ba8e1640 1 Processor -- start 2024-05-10T10:35:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.048+0000 7f11ba8e1640 1 -- start start 2024-05-10T10:35:06.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11ba8e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11b40d2230 0x7f11b4155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11ba8e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d3e80 0x7f11b40cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11ba8e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b4162bc0 0x7f11b40cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11ba8e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11b40b1cf0 con 0x7f11b4162bc0 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11ba8e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11b40d5f90 con 0x7f11b40d2230 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11ba8e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f11b40b4030 con 0x7f11b40d3e80 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11b8e7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b4162bc0 0x7f11b40cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11b8e7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b4162bc0 0x7f11b40cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48214/0 (socket says 172.21.15.53:48214) 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11b8e7e640 1 -- 172.21.15.53:0/201628687 learned_addr learned my addr 172.21.15.53:0/201628687 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11abfff640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d3e80 0x7f11b40cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.049+0000 7f11b3fff640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11b40d2230 0x7f11b4155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b8e7e640 1 -- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d3e80 msgr2=0x7f11b40cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d3e80 0x7f11b40cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b8e7e640 1 -- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11b40d2230 msgr2=0x7f11b4155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11b40d2230 0x7f11b4155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b8e7e640 1 -- 172.21.15.53:0/201628687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b41f8b70 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b3fff640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11b40d2230 0x7f11b4155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b4162bc0 0x7f11b40cb730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00efc0 tx=0x7f11ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b17fa640 1 -- 172.21.15.53:0/201628687 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac018020 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b17fa640 1 -- 172.21.15.53:0/201628687 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac014480 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b41f8e50 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11b17fa640 1 -- 172.21.15.53:0/201628687 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac014870 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.050+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b41f9410 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.052+0000 7f11b17fa640 1 -- 172.21.15.53:0/201628687 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f11ac005d10 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.052+0000 7f11b17fa640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11940762d0 0x7f1194078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.052+0000 7f11b17fa640 1 -- 172.21.15.53:0/201628687 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f11ac098210 con 0x7f11b4162bc0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.053+0000 7f11b3fff640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11940762d0 0x7f1194078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.053+0000 7f11bd600740 1 Processor -- start 2024-05-10T10:35:06.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.053+0000 7f11bd600740 1 -- start start 2024-05-10T10:35:06.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.053+0000 7f11b3fff640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11940762d0 0x7f1194078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11a4000f60 tx=0x7f11a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1590160 0x563df1542af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df1572600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11bd600740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1572b40 0x563df1547950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11bd600740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563df150f690 con 0x563df1572b40 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563df14acf30 con 0x563df1590160 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563df1465b20 con 0x563df1573280 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1572b40 0x563df1547950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1572b40 0x563df1547950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48228/0 (socket says 172.21.15.53:48228) 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1863286057 learned_addr learned my addr 172.21.15.53:0/1863286057 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11abfff640 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df1572600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b3fff640 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1590160 0x563df1542af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 msgr2=0x563df1572600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df1572600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1590160 msgr2=0x563df1542af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1590160 0x563df1542af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1863286057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df145de60 con 0x563df1572b40 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1572b40 0x563df1547950 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f11ac05d3f0 tx=0x7f11ac098050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dea7d30e3dc19d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11a9ffb640 1 -- 172.21.15.53:0/1863286057 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac07a020 con 0x563df1572b40 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11a9ffb640 1 -- 172.21.15.53:0/1863286057 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac061370 con 0x563df1572b40 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.054+0000 7f11a9ffb640 1 -- 172.21.15.53:0/1863286057 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0607d0 con 0x563df1572b40 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 -- 172.21.15.53:0/1863286057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1572b40 msgr2=0x563df1547950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1572b40 0x563df1547950 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f11ac05d3f0 tx=0x7f11ac098050 comp rx=0 tx=0).stop 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 -- 172.21.15.53:0/1863286057 shutdown_connections 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1572b40 0x563df1547950 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df1572600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 --2- 172.21.15.53:0/1863286057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1590160 0x563df1542af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 -- 172.21.15.53:0/1863286057 >> 172.21.15.53:0/1863286057 conn(0x563df1544d60 msgr2=0x563df158d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 -- 172.21.15.53:0/1863286057 shutdown_connections 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.055+0000 7f11bd600740 1 -- 172.21.15.53:0/1863286057 wait complete. 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.056+0000 7f11bd600740 1 -- start start 2024-05-10T10:35:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.056+0000 7f11bd600740 1 Processor -- start 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1573280 0x563df1575fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df159d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df158e330 0x563df159ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563df14695e0 con 0x563df1590160 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563df14a8ef0 con 0x563df1573280 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563df1546f10 con 0x563df158e330 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 10 client.0 _ll_register_callbacks cb 0x563df1464320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df159d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df159d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48238/0 (socket says 172.21.15.53:48238) 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 -- 172.21.15.53:0/66505843 learned_addr learned my addr 172.21.15.53:0/66505843 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:06.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df158e330 0x563df159ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11b3fff640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1573280 0x563df1575fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 -- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df158e330 msgr2=0x563df159ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df158e330 0x563df159ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 -- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1573280 msgr2=0x563df1575fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1573280 0x563df1575fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 -- 172.21.15.53:0/66505843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df12887d0 con 0x563df1590160 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df158e330 0x563df159ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11b3fff640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1573280 0x563df1575fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11abfff640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df159d890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f119c035040 tx=0x7f119c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80422247eaaace45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f1192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f1192ffd640 1 -- 172.21.15.53:0/66505843 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c042070 con 0x563df1590160 2024-05-10T10:35:06.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f1192ffd640 1 -- 172.21.15.53:0/66505843 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f119c003a20 con 0x563df1590160 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.057+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ac061370 con 0x563df1590160 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f1192ffd640 1 -- 172.21.15.53:0/66505843 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f119c037040 con 0x563df1590160 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 10 client.5436 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x563df11be3f0 con 0x563df1590160 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f1192ffd640 1 -- 172.21.15.53:0/66505843 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac061370 con 0x563df1590160 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f1192ffd640 1 -- 172.21.15.53:0/66505843 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f119c003c20 con 0x563df1590160 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 10 client.5436 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 10 client.5436 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 20 client.5436 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df158c900 0x563df1573770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 4 client.5436 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563df14bab40 con 0x563df158c900 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11b3fff640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df158c900 0x563df1573770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f1192ffd640 10 client.5436 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:06.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.058+0000 7f11b3fff640 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df158c900 0x563df1573770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66bc87e3ac0a63e3 server_cookie=cd562a3b72880ad0 in_seq=0 out_seq=0 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f1192ffd640 1 -- 172.21.15.53:0/66505843 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563df14bab40 con 0x563df158c900 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f1192ffd640 10 client.5436 handle_command_reply: tid=0 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 client.5436 shutdown 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 msgr2=0x563df159d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df159d890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f119c035040 tx=0x7f119c004730 comp rx=0 tx=0).stop 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 shutdown_connections 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df158c900 0x563df1573770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df158e330 0x563df159ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df159d890 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f1192ffd640 0 client.5436 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:06.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 --2- 172.21.15.53:0/66505843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df1573280 0x563df1575fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 >> 172.21.15.53:0/66505843 conn(0x563df1572b40 msgr2=0x563df1542230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:06.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 shutdown_connections 2024-05-10T10:35:06.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 1 -- 172.21.15.53:0/66505843 wait complete. 2024-05-10T10:35:06.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.059+0000 7f11bd600740 20 client.5436 trim_cache size 0 max 16384 2024-05-10T10:35:06.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.077+0000 7f11bd600740 1 Processor -- start 2024-05-10T10:35:06.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11bd600740 1 -- start start 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df158bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df158e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11bd600740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df1574670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11bd600740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11b4201bf0 con 0x563df1590160 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11b4200ce0 con 0x563df158e330 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f11b409fe90 con 0x563df1573280 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11b8e7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df1574670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11b8e7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df1574670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48242/0 (socket says 172.21.15.53:48242) 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1533540016 learned_addr learned my addr 172.21.15.53:0/1533540016 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11b3fff640 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df158bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.078+0000 7f11abfff640 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df158e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 msgr2=0x563df158bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df158bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 msgr2=0x563df158e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df158e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11b8e7e640 1 -- 172.21.15.53:0/1533540016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df1545ea0 con 0x563df1590160 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df1574670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11ac061070 tx=0x7f11ac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ab21f7b7dd84be server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11a97fa640 1 -- 172.21.15.53:0/1533540016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac07e030 con 0x563df1590160 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11a97fa640 1 -- 172.21.15.53:0/1533540016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac065720 con 0x563df1590160 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11a97fa640 1 -- 172.21.15.53:0/1533540016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac07a040 con 0x563df1590160 2024-05-10T10:35:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 -- 172.21.15.53:0/1533540016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 msgr2=0x563df1574670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df1574670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11ac061070 tx=0x7f11ac015000 comp rx=0 tx=0).stop 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 -- 172.21.15.53:0/1533540016 shutdown_connections 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1590160 0x563df1574670 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df158e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 --2- 172.21.15.53:0/1533540016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1573280 0x563df158bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 -- 172.21.15.53:0/1533540016 >> 172.21.15.53:0/1533540016 conn(0x563df1572b40 msgr2=0x563df15765d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 -- 172.21.15.53:0/1533540016 shutdown_connections 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.079+0000 7f11bd600740 1 -- 172.21.15.53:0/1533540016 wait complete. 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.080+0000 7f11bd600740 1 -- start start 2024-05-10T10:35:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.080+0000 7f11bd600740 1 Processor -- start 2024-05-10T10:35:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11bd600740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1573280 0x563df160b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df15423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11bd600740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1590160 0x563df160c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11bd600740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563df15845b0 con 0x563df1573280 2024-05-10T10:35:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11b40a04e0 con 0x563df158e330 2024-05-10T10:35:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11bd600740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563df1549530 con 0x563df1590160 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11bd600740 10 client.0 _ll_register_callbacks cb 0x7f11ac0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1573280 0x563df160b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1573280 0x563df160b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48252/0 (socket says 172.21.15.53:48252) 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 -- 172.21.15.53:0/1287242846 learned_addr learned my addr 172.21.15.53:0/1287242846 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11abfff640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df15423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b8e7e640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1590160 0x563df160c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 -- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1590160 msgr2=0x563df160c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1590160 0x563df160c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 -- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 msgr2=0x563df15423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df15423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 -- 172.21.15.53:0/1287242846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df117e0f0 con 0x563df1573280 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11abfff640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df15423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f11b3fff640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1573280 0x563df160b850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x563df1573e70 tx=0x7f11a4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a526ccfe983996e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.081+0000 7f1192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:06.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f1192ffd640 1 -- 172.21.15.53:0/1287242846 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a40070b0 con 0x563df1573280 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f1192ffd640 1 -- 172.21.15.53:0/1287242846 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f11a401d040 con 0x563df1573280 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ac065720 con 0x563df1573280 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f1192ffd640 1 -- 172.21.15.53:0/1287242846 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11a4007250 con 0x563df1573280 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 10 client.5442 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x563df159c820 con 0x563df1573280 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f1192ffd640 1 -- 172.21.15.53:0/1287242846 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac065720 con 0x563df1573280 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f1192ffd640 1 -- 172.21.15.53:0/1287242846 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f11a40073f0 con 0x563df1573280 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 10 client.5442 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 10 client.5442 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 20 client.5442 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df160cdd0 0x563df156e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 4 client.5442 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563df14b49b0 con 0x563df160cdd0 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.082+0000 7f11abfff640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df160cdd0 0x563df156e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11abfff640 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df160cdd0 0x563df156e710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba01b060a1e9ef8 server_cookie=31afecd4e30c2ad6 in_seq=0 out_seq=0 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f1192ffd640 10 client.5442 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f1192ffd640 1 -- 172.21.15.53:0/1287242846 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x563df14b49b0 con 0x563df160cdd0 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f1192ffd640 10 client.5442 handle_command_reply: tid=0 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 client.5442 shutdown 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1573280 msgr2=0x563df160b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1573280 0x563df160b850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x563df1573e70 tx=0x7f11a4000ed0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 shutdown_connections 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563df160cdd0 0x563df156e710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563df1590160 0x563df160c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df158e330 0x563df15423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 --2- 172.21.15.53:0/1287242846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563df1573280 0x563df160b850 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f1192ffd640 0 client.5442 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 >> 172.21.15.53:0/1287242846 conn(0x563df1572b40 msgr2=0x563df158e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 shutdown_connections 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 1 -- 172.21.15.53:0/1287242846 wait complete. 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.083+0000 7f11bd600740 20 client.5442 trim_cache size 0 max 16384 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:35:06.091 INFO:teuthology.orchestra.run.smithi053.stdout: "907a597a-41f1-489a-aec1-01a7a43cceaa": { 2024-05-10T10:35:06.092 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2024-05-10T10:35:06.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "907a597a-41f1-489a-aec1-01a7a43cceaa", 2024-05-10T10:35:06.092 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair" 2024-05-10T10:35:06.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:06.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:06.092 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11940762d0 msgr2=0x7f1194078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11940762d0 0x7f1194078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11a4000f60 tx=0x7f11a4009040 comp rx=0 tx=0).stop 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b4162bc0 msgr2=0x7f11b40cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b4162bc0 0x7f11b40cb730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00efc0 tx=0x7f11ac00c490 comp rx=0 tx=0).stop 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 shutdown_connections 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f11940762d0 0x7f1194078790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11b4162bc0 0x7f11b40cb730 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f11b40d3e80 0x7f11b40cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 --2- 172.21.15.53:0/201628687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11b40d2230 0x7f11b4155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 >> 172.21.15.53:0/201628687 conn(0x7f11b40d5190 msgr2=0x7f11b406e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 shutdown_connections 2024-05-10T10:35:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.084+0000 7f11ba8e1640 1 -- 172.21.15.53:0/201628687 wait complete. 2024-05-10T10:35:06.095 INFO:tasks.cephfs.filesystem:scrub status for tag:907a597a-41f1-489a-aec1-01a7a43cceaa - {'path': '/', 'tag': '907a597a-41f1-489a-aec1-01a7a43cceaa', 'options': 'recursive,repair'} 2024-05-10T10:35:06.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:06.896+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 7 ==== log(6 entries from seq 1 at 2024-05-10T10:35:05.927814+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f707c0956f0 con 0x7f70800b3450 2024-05-10T10:35:06.897 INFO:teuthology.orchestra.run.smithi053.stdout: cluster: 2024-05-10T10:35:06.897 INFO:teuthology.orchestra.run.smithi053.stdout: id: 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:35:06.897 INFO:teuthology.orchestra.run.smithi053.stdout: health: HEALTH_OK 2024-05-10T10:35:06.897 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:06.897 INFO:teuthology.orchestra.run.smithi053.stdout: services: 2024-05-10T10:35:06.897 INFO:teuthology.orchestra.run.smithi053.stdout: mon: 3 daemons, quorum a,b,c (age 108s) 2024-05-10T10:35:06.897 INFO:teuthology.orchestra.run.smithi053.stdout: mgr: x(active, since 103s), standbys: y 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: mds: 1/1 daemons up, 3 standby 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: osd: 8 osds: 8 up (since 98s), 8 in (since 106s) 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: data: 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: volumes: 1/1 healthy 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: pools: 3 pools, 97 pgs 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: objects: 24 objects, 579 KiB 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: pgs: 97 active+clean 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: io: 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: client: 1.9 KiB/s wr, 0 op/s rd, 6 op/s wr 2024-05-10T10:35:06.898 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:06.899 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:06.899 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:35:05.927814+0000 mds.a [INF] scrub queued for path: / 2024-05-10T10:35:06.899 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:35:05.927817+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-10T10:35:06.899 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:35:05.927821+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-10T10:35:06.899 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:35:05.928554+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-05-10T10:35:06.899 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:35:05.928566+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-05-10T10:35:06.899 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:35:05.929277+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-10T10:35:09.232 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:35:07.543854+0000 mds.a [INF] scrub summary: idle 2024-05-10T10:35:09.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:09.232+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 8 ==== log(1 entries from seq 7 at 2024-05-10T10:35:07.543854+0000) ==== 189+0+0 (secure 0 0 0) 0x7f707c05c710 con 0x7f70800b3450 2024-05-10T10:35:13.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:35:13.882 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:35:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:22.254+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 9 ==== log(4 entries from seq 111 at 2024-05-10T10:35:22.170570+0000) ==== 1120+0+0 (secure 0 0 0) 0x7f707c0606e0 con 0x7f70800b3450 2024-05-10T10:35:23.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:23.267+0000 7f7075ffb640 1 -- 172.21.15.53:0/2709305607 <== mon.0 v2:172.21.15.53:3300/0 10 ==== log(2 entries from seq 394 at 2024-05-10T10:35:22.254638+0000) ==== 557+0+0 (secure 0 0 0) 0x7f707c068d10 con 0x7f70800b3450 2024-05-10T10:35:36.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:36.097 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-10T10:35:36.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.215+0000 7fe89d408640 1 Processor -- start 2024-05-10T10:35:36.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.215+0000 7fe89d408640 1 -- start start 2024-05-10T10:35:36.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe89d408640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe898160870 0x7fe8980cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe89d408640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe898160c50 con 0x7fe8980cfcc0 2024-05-10T10:35:36.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe89d408640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8980d39d0 con 0x7fe8980b3970 2024-05-10T10:35:36.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe89d408640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe8980d3cb0 con 0x7fe898160870 2024-05-10T10:35:36.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe896ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe898160870 0x7fe8980cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe8967fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe8980cfcc0 0x7fe8980cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49664/0 (socket says 172.21.15.53:49664) 2024-05-10T10:35:36.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe8967fc640 1 -- 172.21.15.53:0/2425264592 learned_addr learned my addr 172.21.15.53:0/2425264592 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:36.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe896ffd640 1 -- 172.21.15.53:0/2425264592 >> v1:172.21.15.135:6789/0 conn(0x7fe8980b3970 legacy=0x7fe8980ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe896ffd640 1 -- 172.21.15.53:0/2425264592 >> v1:172.21.15.53:6789/0 conn(0x7fe8980cfcc0 legacy=0x7fe8980cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe895ffb640 1 -- 172.21.15.53:0/2425264592 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600257645 0 0) 0x7fe898160c50 con 0x7fe8980cfcc0 2024-05-10T10:35:36.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.216+0000 7fe896ffd640 1 -- 172.21.15.53:0/2425264592 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8980d1770 con 0x7fe898160870 2024-05-10T10:35:36.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.217+0000 7fe896ffd640 1 --2- 172.21.15.53:0/2425264592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe898160870 0x7fe8980cf780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe888009d90 tx=0x7fe88802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7b530d4d29be7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.217+0000 7fe895ffb640 1 -- 172.21.15.53:0/2425264592 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888031600 con 0x7fe898160870 2024-05-10T10:35:36.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.217+0000 7fe895ffb640 1 -- 172.21.15.53:0/2425264592 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8880317a0 con 0x7fe898160870 2024-05-10T10:35:36.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.217+0000 7fe895ffb640 1 -- 172.21.15.53:0/2425264592 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888031b70 con 0x7fe898160870 2024-05-10T10:35:36.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.218+0000 7fe89d408640 1 -- 172.21.15.53:0/2425264592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe898160870 msgr2=0x7fe8980cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.218+0000 7fe89d408640 1 --2- 172.21.15.53:0/2425264592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe898160870 0x7fe8980cf780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe888009d90 tx=0x7fe88802a610 comp rx=0 tx=0).stop 2024-05-10T10:35:36.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.218+0000 7fe89d408640 1 -- 172.21.15.53:0/2425264592 shutdown_connections 2024-05-10T10:35:36.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.218+0000 7fe89d408640 1 --2- 172.21.15.53:0/2425264592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe898160870 0x7fe8980cf780 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.218+0000 7fe89d408640 1 -- 172.21.15.53:0/2425264592 >> 172.21.15.53:0/2425264592 conn(0x7fe8980d2eb0 msgr2=0x7fe8980d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.218+0000 7fe89d408640 1 -- 172.21.15.53:0/2425264592 shutdown_connections 2024-05-10T10:35:36.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.218+0000 7fe89d408640 1 -- 172.21.15.53:0/2425264592 wait complete. 2024-05-10T10:35:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.219+0000 7fe89d408640 1 Processor -- start 2024-05-10T10:35:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe89d408640 1 -- start start 2024-05-10T10:35:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe89d408640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8980b3970 0x7fe89815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe89d408640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8980cfcc0 0x7fe8980cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe89d408640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe898160870 0x7fe8980c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe89d408640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980d1460 con 0x7fe8980b3970 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe89d408640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980d1d30 con 0x7fe8980cfcc0 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe89d408640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe8980b3d80 con 0x7fe898160870 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8980b3970 0x7fe89815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8980b3970 0x7fe89815efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47714/0 (socket says 172.21.15.53:47714) 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.220+0000 7fe896ffd640 1 -- 172.21.15.53:0/4059415372 learned_addr learned my addr 172.21.15.53:0/4059415372 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe8967fc640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8980cfcc0 0x7fe8980cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe8977fe640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe898160870 0x7fe8980c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe896ffd640 1 -- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe898160870 msgr2=0x7fe8980c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe896ffd640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe898160870 0x7fe8980c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe896ffd640 1 -- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8980cfcc0 msgr2=0x7fe8980cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe896ffd640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8980cfcc0 0x7fe8980cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe896ffd640 1 -- 172.21.15.53:0/4059415372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8980ce6c0 con 0x7fe8980b3970 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe8977fe640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe898160870 0x7fe8980c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:36.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe896ffd640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8980b3970 0x7fe89815efc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe888009eb0 tx=0x7fe888004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe87f7fe640 1 -- 172.21.15.53:0/4059415372 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888040070 con 0x7fe8980b3970 2024-05-10T10:35:36.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe87f7fe640 1 -- 172.21.15.53:0/4059415372 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe888030be0 con 0x7fe8980b3970 2024-05-10T10:35:36.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8981f6160 con 0x7fe8980b3970 2024-05-10T10:35:36.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe87f7fe640 1 -- 172.21.15.53:0/4059415372 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888009450 con 0x7fe8980b3970 2024-05-10T10:35:36.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.221+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981f6720 con 0x7fe8980b3970 2024-05-10T10:35:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.223+0000 7fe87f7fe640 1 -- 172.21.15.53:0/4059415372 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe888039040 con 0x7fe8980b3970 2024-05-10T10:35:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.224+0000 7fe87f7fe640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8840762d0 0x7fe884078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.224+0000 7fe87f7fe640 1 -- 172.21.15.53:0/4059415372 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7fe8880b8220 con 0x7fe8980b3970 2024-05-10T10:35:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.224+0000 7fe8967fc640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8840762d0 0x7fe884078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.225+0000 7fe8967fc640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8840762d0 0x7fe884078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe88c007fd0 tx=0x7fe88c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.225+0000 7fe8a01f3740 1 Processor -- start 2024-05-10T10:35:36.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.225+0000 7fe8a01f3740 1 -- start start 2024-05-10T10:35:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f1eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f4e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f4eb40 0x563c93f23950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563c93eeb690 con 0x563c93f4eb40 2024-05-10T10:35:36.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563c93e88f30 con 0x563c93f4f280 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563c93e41b20 con 0x563c93f6c160 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f4eb40 0x563c93f23950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f4eb40 0x563c93f23950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47726/0 (socket says 172.21.15.53:47726) 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 -- 172.21.15.53:0/302302919 learned_addr learned my addr 172.21.15.53:0/302302919 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe896ffd640 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f1eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8967fc640 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f4e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 -- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 msgr2=0x563c93f1eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f1eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 -- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 msgr2=0x563c93f4e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f4e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 -- 172.21.15.53:0/302302919 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c93e39e60 con 0x563c93f4eb40 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.226+0000 7fe8977fe640 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f4eb40 0x563c93f23950 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe890003ee0 tx=0x7fe8900043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dae6e1536fa106 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe87cff9640 1 -- 172.21.15.53:0/302302919 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890016070 con 0x563c93f4eb40 2024-05-10T10:35:36.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe87cff9640 1 -- 172.21.15.53:0/302302919 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe890005250 con 0x563c93f4eb40 2024-05-10T10:35:36.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe87cff9640 1 -- 172.21.15.53:0/302302919 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000d940 con 0x563c93f4eb40 2024-05-10T10:35:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 -- 172.21.15.53:0/302302919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f4eb40 msgr2=0x563c93f23950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f4eb40 0x563c93f23950 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe890003ee0 tx=0x7fe8900043a0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 -- 172.21.15.53:0/302302919 shutdown_connections 2024-05-10T10:35:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f4eb40 0x563c93f23950 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f4e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/302302919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f1eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 -- 172.21.15.53:0/302302919 >> 172.21.15.53:0/302302919 conn(0x563c93f20d60 msgr2=0x563c93f69600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.227+0000 7fe8a01f3740 1 -- 172.21.15.53:0/302302919 shutdown_connections 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.228+0000 7fe8a01f3740 1 -- 172.21.15.53:0/302302919 wait complete. 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.228+0000 7fe8a01f3740 1 -- start start 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.228+0000 7fe8a01f3740 1 Processor -- start 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f795d0 0x563c93f69600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563c93e455e0 con 0x563c93f795d0 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563c93e84ef0 con 0x563c93f4f280 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563c93f22f10 con 0x563c93f6c160 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8a01f3740 10 client.0 _ll_register_callbacks cb 0x563c93e40320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f795d0 0x563c93f69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f795d0 0x563c93f69600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47738/0 (socket says 172.21.15.53:47738) 2024-05-10T10:35:36.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 -- 172.21.15.53:0/3346903612 learned_addr learned my addr 172.21.15.53:0/3346903612 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe896ffd640 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8967fc640 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 -- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 msgr2=0x563c93f79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 -- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 msgr2=0x563c93f49a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f49a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.229+0000 7fe8977fe640 1 -- 172.21.15.53:0/3346903612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c93c647d0 con 0x563c93f795d0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8977fe640 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f795d0 0x563c93f69600 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe890005000 tx=0x7fe8900036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60807d51a56fa11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe875ffb640 1 -- 172.21.15.53:0/3346903612 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890016070 con 0x563c93f795d0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe875ffb640 1 -- 172.21.15.53:0/3346903612 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7fe8900067d0 con 0x563c93f795d0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe890005250 con 0x563c93f795d0 2024-05-10T10:35:36.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe875ffb640 1 -- 172.21.15.53:0/3346903612 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe89002d050 con 0x563c93f795d0 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 10 client.5457 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x563c93b9a3f0 con 0x563c93f795d0 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe875ffb640 1 -- 172.21.15.53:0/3346903612 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe890005250 con 0x563c93f795d0 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe875ffb640 1 -- 172.21.15.53:0/3346903612 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fe890030020 con 0x563c93f795d0 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 10 client.5457 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 10 client.5457 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 20 client.5457 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f4f770 0x563c93f4fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 4 client.5457 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c93e96b40 con 0x563c93f4f770 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.230+0000 7fe896ffd640 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f4f770 0x563c93f4fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.231+0000 7fe875ffb640 10 client.5457 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.231+0000 7fe896ffd640 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f4f770 0x563c93f4fba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a6526aea4200be server_cookie=3c4c7a3c90aa547 in_seq=0 out_seq=0 2024-05-10T10:35:36.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.231+0000 7fe875ffb640 1 -- 172.21.15.53:0/3346903612 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c93e96b40 con 0x563c93f4f770 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.231+0000 7fe875ffb640 10 client.5457 handle_command_reply: tid=0 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.231+0000 7fe8a01f3740 1 client.5457 shutdown 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f795d0 msgr2=0x563c93f69600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f795d0 0x563c93f69600 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe890005000 tx=0x7fe8900036f0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 shutdown_connections 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f4f770 0x563c93f4fba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f795d0 0x563c93f69600 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f6c160 0x563c93f79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/3346903612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f4f280 0x563c93f49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe875ffb640 0 client.5457 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 >> 172.21.15.53:0/3346903612 conn(0x563c93f4eb40 msgr2=0x563c93f6b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 shutdown_connections 2024-05-10T10:35:36.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 1 -- 172.21.15.53:0/3346903612 wait complete. 2024-05-10T10:35:36.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.232+0000 7fe8a01f3740 20 client.5457 trim_cache size 0 max 16384 2024-05-10T10:35:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.250+0000 7fe8a01f3740 1 Processor -- start 2024-05-10T10:35:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.250+0000 7fe8a01f3740 1 -- start start 2024-05-10T10:35:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93f68590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f78c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f47ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ffc60 con 0x563c93f6c160 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981fec60 con 0x563c93f795d0 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe8981f3720 con 0x563c93f4f280 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f78c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f78c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47742/0 (socket says 172.21.15.53:47742) 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 -- 172.21.15.53:0/24256527 learned_addr learned my addr 172.21.15.53:0/24256527 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe896ffd640 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93f68590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8977fe640 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f47ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 -- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 msgr2=0x563c93f68590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93f68590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 -- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 msgr2=0x563c93f47ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f47ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 -- 172.21.15.53:0/24256527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c93b5a0f0 con 0x563c93f6c160 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe896ffd640 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93f68590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8977fe640 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f47ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.251+0000 7fe8967fc640 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f78c70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0075c0 tx=0x7fe88c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b4f15cfbd42d21 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe877fff640 1 -- 172.21.15.53:0/24256527 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c027070 con 0x563c93f6c160 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe877fff640 1 -- 172.21.15.53:0/24256527 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88c005c80 con 0x563c93f6c160 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe877fff640 1 -- 172.21.15.53:0/24256527 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c022640 con 0x563c93f6c160 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 -- 172.21.15.53:0/24256527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 msgr2=0x563c93f78c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f78c70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0075c0 tx=0x7fe88c010000 comp rx=0 tx=0).stop 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 -- 172.21.15.53:0/24256527 shutdown_connections 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f47ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f78c70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/24256527 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93f68590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 -- 172.21.15.53:0/24256527 >> 172.21.15.53:0/24256527 conn(0x563c93f4eb40 msgr2=0x563c93f50cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 -- 172.21.15.53:0/24256527 shutdown_connections 2024-05-10T10:35:36.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.252+0000 7fe8a01f3740 1 -- 172.21.15.53:0/24256527 wait complete. 2024-05-10T10:35:36.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.253+0000 7fe8a01f3740 1 -- start start 2024-05-10T10:35:36.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.253+0000 7fe8a01f3740 1 Processor -- start 2024-05-10T10:35:36.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93fe60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f46980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981fe610 con 0x563c93f6c160 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563c93f446f0 con 0x563c93f795d0 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563c93f513f0 con 0x563c93f4f280 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 10 client.0 _ll_register_callbacks cb 0x7fe88c001350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f50ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47750/0 (socket says 172.21.15.53:47750) 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 -- 172.21.15.53:0/211998781 learned_addr learned my addr 172.21.15.53:0/211998781 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe896ffd640 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93fe60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8977fe640 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f46980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 -- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 msgr2=0x563c93fe60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93fe60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 -- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 msgr2=0x563c93f46980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f46980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 -- 172.21.15.53:0/211998781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c93f61d40 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8967fc640 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f50ca0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe88c000f60 tx=0x7fe88c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae5c5571a6babf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe875ffb640 1 -- 172.21.15.53:0/211998781 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c031060 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe875ffb640 1 -- 172.21.15.53:0/211998781 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7fe88c022c00 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.254+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c005c80 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe875ffb640 1 -- 172.21.15.53:0/211998781 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe88c027070 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 10 client.5463 fetch_fsmap learned FSMap version 26 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x563c93f5f100 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe875ffb640 1 -- 172.21.15.53:0/211998781 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88c005c80 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe875ffb640 1 -- 172.21.15.53:0/211998781 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fe88c02c430 con 0x563c93f6c160 2024-05-10T10:35:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 10 client.5463 fetch_fsmap finished waiting for FSMap version 26 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 10 client.5463 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 20 client.5463 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f46ec0 0x563c93f5f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 4 client.5463 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 --> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c93f6a550 con 0x563c93f46ec0 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe896ffd640 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f46ec0 0x563c93f5f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe875ffb640 10 client.5463 ms_handle_connect on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.255+0000 7fe896ffd640 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f46ec0 0x563c93f5f5c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93d690dd888985 server_cookie=2104682a4c943f4e in_seq=0 out_seq=0 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe875ffb640 1 -- 172.21.15.53:0/211998781 <== mds.0 v2:172.21.15.53:6834/679529875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c93f6a550 con 0x563c93f46ec0 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe875ffb640 10 client.5463 handle_command_reply: tid=0 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 client.5463 shutdown 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 msgr2=0x563c93f50ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f50ca0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe88c000f60 tx=0x7fe88c000ed0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 shutdown_connections 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:6834/679529875,v1:172.21.15.53:6835/679529875] conn(0x563c93f46ec0 0x563c93f5f5c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c93f795d0 0x563c93f46980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe875ffb640 0 client.5463 ms_handle_reset on v2:172.21.15.53:6834/679529875 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c93f6c160 0x563c93f50ca0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 --2- 172.21.15.53:0/211998781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563c93f4f280 0x563c93fe60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 >> 172.21.15.53:0/211998781 conn(0x563c93f4eb40 msgr2=0x563c93f626b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 shutdown_connections 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 1 -- 172.21.15.53:0/211998781 wait complete. 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe8a01f3740 20 client.5463 trim_cache size 0 max 16384 2024-05-10T10:35:36.264 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:35:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:35:36.265 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8840762d0 msgr2=0x7fe884078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8840762d0 0x7fe884078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe88c007fd0 tx=0x7fe88c009040 comp rx=0 tx=0).stop 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8980b3970 msgr2=0x7fe89815efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8980b3970 0x7fe89815efc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe888009eb0 tx=0x7fe888004a40 comp rx=0 tx=0).stop 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 shutdown_connections 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8840762d0 0x7fe884078790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe898160870 0x7fe8980c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8980cfcc0 0x7fe8980cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 --2- 172.21.15.53:0/4059415372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8980b3970 0x7fe89815efc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 >> 172.21.15.53:0/4059415372 conn(0x7fe8980d2eb0 msgr2=0x7fe8980afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.256+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 shutdown_connections 2024-05-10T10:35:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.257+0000 7fe89d408640 1 -- 172.21.15.53:0/4059415372 wait complete. 2024-05-10T10:35:36.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:35:36.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:36.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.473+0000 7f4ff0ae8640 1 Processor -- start 2024-05-10T10:35:36.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.473+0000 7f4ff0ae8640 1 -- start start 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4ff0ae8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec0d5cb0 0x7f4fec0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4ff0ae8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fec0b45f0 con 0x7f4fec162b30 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4ff0ae8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fec0b48d0 con 0x7f4fec162750 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4ff0ae8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4fec0b4f20 con 0x7f4fec0d5cb0 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4fead9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4fec162b30 0x7f4fec0b53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49674/0 (socket says 172.21.15.53:49674) 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4fead9d640 1 -- 172.21.15.53:0/2009460190 learned_addr learned my addr 172.21.15.53:0/2009460190 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4fe9d9b640 1 --2- 172.21.15.53:0/2009460190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec0d5cb0 0x7f4fec0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4fe959a640 1 -- 172.21.15.53:0/2009460190 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3058352155 0 0) 0x7f4fec0b45f0 con 0x7f4fec162b30 2024-05-10T10:35:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.474+0000 7f4fe959a640 1 -- 172.21.15.53:0/2009460190 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fd8003520 con 0x7f4fec162b30 2024-05-10T10:35:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.475+0000 7f4fe9d9b640 1 -- 172.21.15.53:0/2009460190 >> v1:172.21.15.135:6789/0 conn(0x7f4fec162750 legacy=0x7f4fec0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.475+0000 7f4fe959a640 1 -- 172.21.15.53:0/2009460190 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068080352 0 0) 0x7f4fec0b48d0 con 0x7f4fec162750 2024-05-10T10:35:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.475+0000 7f4fe9d9b640 1 -- 172.21.15.53:0/2009460190 >> v1:172.21.15.53:6789/0 conn(0x7f4fec162b30 legacy=0x7f4fec0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.475+0000 7f4fe9d9b640 1 -- 172.21.15.53:0/2009460190 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec0b4bb0 con 0x7f4fec0d5cb0 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.475+0000 7f4fe9d9b640 1 --2- 172.21.15.53:0/2009460190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec0d5cb0 0x7f4fec0d3630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0097d0 tx=0x7f4fdc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d504869ce67526 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4fe959a640 1 -- 172.21.15.53:0/2009460190 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc037070 con 0x7f4fec0d5cb0 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4fe959a640 1 -- 172.21.15.53:0/2009460190 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fdc032440 con 0x7f4fec0d5cb0 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4fe959a640 1 -- 172.21.15.53:0/2009460190 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0327f0 con 0x7f4fec0d5cb0 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/2009460190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec0d5cb0 msgr2=0x7f4fec0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/2009460190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec0d5cb0 0x7f4fec0d3630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0097d0 tx=0x7f4fdc02a180 comp rx=0 tx=0).stop 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/2009460190 shutdown_connections 2024-05-10T10:35:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/2009460190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec0d5cb0 0x7f4fec0d3630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/2009460190 >> 172.21.15.53:0/2009460190 conn(0x7f4fec09e3b0 msgr2=0x7f4fec0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/2009460190 shutdown_connections 2024-05-10T10:35:36.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.476+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/2009460190 wait complete. 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.477+0000 7f4ff0ae8640 1 Processor -- start 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4ff0ae8640 1 -- start start 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4ff0ae8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec162750 0x7f4fec1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4ff0ae8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fec162b30 0x7f4fec0ba430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4ff0ae8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec0ba970 0x7f4fec0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4ff0ae8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec0d3f80 con 0x7f4fec0ba970 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4ff0ae8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec0d3e00 con 0x7f4fec162b30 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4ff0ae8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4fec0d0fe0 con 0x7f4fec162750 2024-05-10T10:35:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4fead9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec0ba970 0x7f4fec0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4fead9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec0ba970 0x7f4fec0ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47758/0 (socket says 172.21.15.53:47758) 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.478+0000 7f4fead9d640 1 -- 172.21.15.53:0/1700446375 learned_addr learned my addr 172.21.15.53:0/1700446375 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fea59c640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec162750 0x7f4fec1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fe9d9b640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fec162b30 0x7f4fec0ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fead9d640 1 -- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec162750 msgr2=0x7f4fec1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fead9d640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec162750 0x7f4fec1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fead9d640 1 -- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fec162b30 msgr2=0x7f4fec0ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fead9d640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fec162b30 0x7f4fec0ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fead9d640 1 -- 172.21.15.53:0/1700446375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec0b8930 con 0x7f4fec0ba970 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fe9d9b640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fec162b30 0x7f4fec0ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fead9d640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec0ba970 0x7f4fec0ce100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400cad0 tx=0x7f4fe400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fe2ffd640 1 -- 172.21.15.53:0/1700446375 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4017070 con 0x7f4fec0ba970 2024-05-10T10:35:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fe2ffd640 1 -- 172.21.15.53:0/1700446375 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe400ac30 con 0x7f4fec0ba970 2024-05-10T10:35:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec0b8c10 con 0x7f4fec0ba970 2024-05-10T10:35:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4fe2ffd640 1 -- 172.21.15.53:0/1700446375 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4012400 con 0x7f4fec0ba970 2024-05-10T10:35:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.479+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec0b91d0 con 0x7f4fec0ba970 2024-05-10T10:35:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.480+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fec06a220 con 0x7f4fec0ba970 2024-05-10T10:35:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.481+0000 7f4fe2ffd640 1 -- 172.21.15.53:0/1700446375 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4fe401a070 con 0x7f4fec0ba970 2024-05-10T10:35:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.481+0000 7f4fe2ffd640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4fcc0762d0 0x7f4fcc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.481+0000 7f4fe2ffd640 1 -- 172.21.15.53:0/1700446375 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7f4fe40974a0 con 0x7f4fec0ba970 2024-05-10T10:35:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.482+0000 7f4fea59c640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4fcc0762d0 0x7f4fcc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.482+0000 7f4fea59c640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4fcc0762d0 0x7f4fcc078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000f60 tx=0x7f4fd4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.486+0000 7f4fe2ffd640 1 -- 172.21.15.53:0/1700446375 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4fe4060600 con 0x7f4fec0ba970 2024-05-10T10:35:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.714+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fec160e40 con 0x7f4fec0ba970 2024-05-10T10:35:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.715+0000 7f4fe2ffd640 1 -- 172.21.15.53:0/1700446375 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7f4fe4064900 con 0x7f4fec0ba970 2024-05-10T10:35:36.716 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:36.717 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4fcc0762d0 msgr2=0x7f4fcc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4fcc0762d0 0x7f4fcc078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000f60 tx=0x7f4fd4009040 comp rx=0 tx=0).stop 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec0ba970 msgr2=0x7f4fec0ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec0ba970 0x7f4fec0ce100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400cad0 tx=0x7f4fe400cf90 comp rx=0 tx=0).stop 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 shutdown_connections 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4fcc0762d0 0x7f4fcc078790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec0ba970 0x7f4fec0ce100 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4fec162b30 0x7f4fec0ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 --2- 172.21.15.53:0/1700446375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4fec162750 0x7f4fec1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 >> 172.21.15.53:0/1700446375 conn(0x7f4fec09e3b0 msgr2=0x7f4fec0d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 shutdown_connections 2024-05-10T10:35:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.717+0000 7f4ff0ae8640 1 -- 172.21.15.53:0/1700446375 wait complete. 2024-05-10T10:35:36.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-10T10:35:36.855 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:35:36.855 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:35:36.855 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-10T10:35:36.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.899+0000 7f66c1870f40 1 Processor -- start 2024-05-10T10:35:36.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.899+0000 7f66c1870f40 1 -- start start 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66c1870f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702d6fd40 0x561702df8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66c1870f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702dfdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66c1870f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702dfec00 0x561702dfc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66c1870f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561702d872e0 con 0x561702dfec00 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66c1870f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561702ce7f50 con 0x561702d6fd40 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66c1870f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561702d875c0 con 0x561702d885e0 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702dfec00 0x561702dfc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702dfec00 0x561702dfc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47762/0 (socket says 172.21.15.53:47762) 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 -- 172.21.15.53:0/1622745448 learned_addr learned my addr 172.21.15.53:0/1622745448 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bf36a640 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702d6fd40 0x561702df8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66beb69640 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702dfdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 -- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 msgr2=0x561702dfdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702dfdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 -- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702d6fd40 msgr2=0x561702df8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702d6fd40 0x561702df8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 -- 172.21.15.53:0/1622745448 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561702e14b70 con 0x561702dfec00 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bf36a640 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702d6fd40 0x561702df8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bfb6b640 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702dfec00 0x561702dfc460 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f66b8046040 tx=0x7f66b8050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b18ff896011778 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bdb67640 1 -- 172.21.15.53:0/1622745448 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8055030 con 0x561702dfec00 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bdb67640 1 -- 172.21.15.53:0/1622745448 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b8051bc0 con 0x561702dfec00 2024-05-10T10:35:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.900+0000 7f66bdb67640 1 -- 172.21.15.53:0/1622745448 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b805b430 con 0x561702dfec00 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 -- 172.21.15.53:0/1622745448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702dfec00 msgr2=0x561702dfc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702dfec00 0x561702dfc460 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f66b8046040 tx=0x7f66b8050000 comp rx=0 tx=0).stop 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 -- 172.21.15.53:0/1622745448 shutdown_connections 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702dfec00 0x561702dfc460 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702dfdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1622745448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702d6fd40 0x561702df8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 -- 172.21.15.53:0/1622745448 >> 172.21.15.53:0/1622745448 conn(0x561702d3f7c0 msgr2=0x561702dfce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 -- 172.21.15.53:0/1622745448 shutdown_connections 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 -- 172.21.15.53:0/1622745448 wait complete. 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 Processor -- start 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.901+0000 7f66c1870f40 1 -- start start 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66c1870f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702d6fd40 0x561702e18d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66c1870f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702e0f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66c1870f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702dfec00 0x561702e19520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66c1870f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561702d3fc30 con 0x561702d6fd40 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66c1870f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561702df7d30 con 0x561702dfec00 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66c1870f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561702dfa860 con 0x561702d885e0 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66bf36a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702d6fd40 0x561702e18d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66bf36a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702d6fd40 0x561702e18d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47776/0 (socket says 172.21.15.53:47776) 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66bf36a640 1 -- 172.21.15.53:0/1165936751 learned_addr learned my addr 172.21.15.53:0/1165936751 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66beb69640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702e0f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.902+0000 7f66bfb6b640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702dfec00 0x561702e19520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bf36a640 1 -- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 msgr2=0x561702e0f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bf36a640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702e0f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bf36a640 1 -- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702dfec00 msgr2=0x561702e19520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bf36a640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702dfec00 0x561702e19520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bf36a640 1 -- 172.21.15.53:0/1165936751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561702e16660 con 0x561702d6fd40 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66beb69640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561702d885e0 0x561702e0f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:36.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bfb6b640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561702dfec00 0x561702e19520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:36.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bf36a640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702d6fd40 0x561702e18d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009d70 tx=0x7f66b0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bcb65640 1 -- 172.21.15.53:0/1165936751 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00143f0 con 0x561702d6fd40 2024-05-10T10:35:36.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bcb65640 1 -- 172.21.15.53:0/1165936751 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0019070 con 0x561702d6fd40 2024-05-10T10:35:36.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561702e16940 con 0x561702d6fd40 2024-05-10T10:35:36.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66bcb65640 1 -- 172.21.15.53:0/1165936751 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0014790 con 0x561702d6fd40 2024-05-10T10:35:36.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.903+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561702e16f00 con 0x561702d6fd40 2024-05-10T10:35:36.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.905+0000 7f66bcb65640 1 -- 172.21.15.53:0/1165936751 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f66b0014930 con 0x561702d6fd40 2024-05-10T10:35:36.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.905+0000 7f66bcb65640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66ac074ce0 0x7f66ac0771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.905+0000 7f66bcb65640 1 -- 172.21.15.53:0/1165936751 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7f66b0096100 con 0x561702d6fd40 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.905+0000 7f66beb69640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66ac074ce0 0x7f66ac0771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.905+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x561702e13550 0x561702e139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.905+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50) -- 0x561702e0fc60 con 0x561702e13550 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.905+0000 7f66bfb6b640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x561702e13550 0x561702e139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66beb69640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66ac074ce0 0x7f66ac0771a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x561702ea7420 tx=0x7f66b4003690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66bfb6b640 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x561702e13550 0x561702e139e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66bfb6b640 1 -- 172.21.15.53:0/1165936751 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f66b8065430 con 0x561702e13550 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x561702e13550 msgr2=0x561702e139e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x561702e13550 0x561702e139e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66ac074ce0 msgr2=0x7f66ac0771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f66ac074ce0 0x7f66ac0771a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x561702ea7420 tx=0x7f66b4003690 comp rx=0 tx=0).stop 2024-05-10T10:35:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702d6fd40 msgr2=0x561702e18d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 --2- 172.21.15.53:0/1165936751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561702d6fd40 0x561702e18d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009d70 tx=0x7f66b0009e60 comp rx=0 tx=0).stop 2024-05-10T10:35:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66bfb6b640 1 -- 172.21.15.53:0/1165936751 reap_dead start 2024-05-10T10:35:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 shutdown_connections 2024-05-10T10:35:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 >> 172.21.15.53:0/1165936751 conn(0x561702d3f7c0 msgr2=0x561702dfc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 shutdown_connections 2024-05-10T10:35:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:36.906+0000 7f66c1870f40 1 -- 172.21.15.53:0/1165936751 wait complete. 2024-05-10T10:35:36.914 INFO:teuthology.orchestra.run.smithi053.stdout:Yalphaparent_a 2024-05-10T10:35:36.914 DEBUG:teuthology.orchestra.run.smithi053:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-10T10:35:36.962 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "ancestors": [ 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1099511627776, 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "alpha", 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 2 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1, 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "parent_a", 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 4 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-05-10T10:35:36.963 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 8, 2024-05-10T10:35:36.964 INFO:teuthology.orchestra.run.smithi053.stdout: "old_pools": [] 2024-05-10T10:35:36.964 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:36.964 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:36.965 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:35:36.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:35:37.056 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:35:37.103 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:35:37.103 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:35:37.103 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:35:37.103 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:37.120 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:35:37.120 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:37.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:35:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.321+0000 7ff67dc09640 1 Processor -- start 2024-05-10T10:35:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.321+0000 7ff67dc09640 1 -- start start 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.321+0000 7ff67dc09640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.321+0000 7ff67dc09640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6780d5d30 0x7ff6780d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.321+0000 7ff67dc09640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6780d30b0 con 0x7ff67815f790 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.321+0000 7ff67dc09640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff6780d3390 con 0x7ff6780d5d30 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.321+0000 7ff67dc09640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff6780c8c80 con 0x7ff67815f3b0 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff677fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6780d5d30 0x7ff6780d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff677fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6780d5d30 0x7ff6780d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47778/0 (socket says 172.21.15.53:47778) 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff677fff640 1 -- 172.21.15.53:0/251045822 learned_addr learned my addr 172.21.15.53:0/251045822 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff6777fe640 1 --2- 172.21.15.53:0/251045822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff677fff640 1 -- 172.21.15.53:0/251045822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 msgr2=0x7ff6780d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff677fff640 1 --2- 172.21.15.53:0/251045822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff677fff640 1 -- 172.21.15.53:0/251045822 >> v1:172.21.15.135:6789/0 conn(0x7ff67815f790 legacy=0x7ff6780c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff677fff640 1 -- 172.21.15.53:0/251045822 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6780c8f60 con 0x7ff6780d5d30 2024-05-10T10:35:37.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.322+0000 7ff6777fe640 1 --2- 172.21.15.53:0/251045822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:37.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff677fff640 1 --2- 172.21.15.53:0/251045822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780d5d30 0x7ff6780d3c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff670009870 tx=0x7ff67002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25294f7be19dd59d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:37.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff6767fc640 1 -- 172.21.15.53:0/251045822 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670037070 con 0x7ff6780d5d30 2024-05-10T10:35:37.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff6767fc640 1 -- 172.21.15.53:0/251045822 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff670032440 con 0x7ff6780d5d30 2024-05-10T10:35:37.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff6767fc640 1 -- 172.21.15.53:0/251045822 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670032810 con 0x7ff6780d5d30 2024-05-10T10:35:37.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 -- 172.21.15.53:0/251045822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780d5d30 msgr2=0x7ff6780d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 --2- 172.21.15.53:0/251045822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780d5d30 0x7ff6780d3c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff670009870 tx=0x7ff67002a210 comp rx=0 tx=0).stop 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 -- 172.21.15.53:0/251045822 shutdown_connections 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 --2- 172.21.15.53:0/251045822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780d5d30 0x7ff6780d3c30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 --2- 172.21.15.53:0/251045822 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 -- 172.21.15.53:0/251045822 >> 172.21.15.53:0/251045822 conn(0x7ff6780d5130 msgr2=0x7ff6780d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 -- 172.21.15.53:0/251045822 shutdown_connections 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.323+0000 7ff67dc09640 1 -- 172.21.15.53:0/251045822 wait complete. 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.325+0000 7ff67dc09640 1 Processor -- start 2024-05-10T10:35:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.325+0000 7ff67dc09640 1 -- start start 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.325+0000 7ff67dc09640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6780d5d30 0x7ff6780b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.325+0000 7ff67dc09640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff67dc09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780b46a0 0x7ff67814f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff67dc09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff6780d0e60 con 0x7ff6780b46a0 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff67dc09640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6780d0ce0 con 0x7ff6780d5d30 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff67dc09640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff6780d0fe0 con 0x7ff67815f3b0 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780b46a0 0x7ff67814f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780b46a0 0x7ff67814f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47790/0 (socket says 172.21.15.53:47790) 2024-05-10T10:35:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 -- 172.21.15.53:0/896744214 learned_addr learned my addr 172.21.15.53:0/896744214 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff6777fe640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6780d5d30 0x7ff6780b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff676ffd640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 -- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 msgr2=0x7ff6780ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 -- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6780d5d30 msgr2=0x7ff6780b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6780d5d30 0x7ff6780b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 -- 172.21.15.53:0/896744214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6780ce930 con 0x7ff6780b46a0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff677fff640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780b46a0 0x7ff67814f7e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff67002a250 tx=0x7ff6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff65ffff640 1 -- 172.21.15.53:0/896744214 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670041020 con 0x7ff6780b46a0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff65ffff640 1 -- 172.21.15.53:0/896744214 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff670030990 con 0x7ff6780b46a0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6780cec10 con 0x7ff6780b46a0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff65ffff640 1 -- 172.21.15.53:0/896744214 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670037040 con 0x7ff6780b46a0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.326+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6780b4b30 con 0x7ff6780b46a0 2024-05-10T10:35:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.327+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6780b4d60 con 0x7ff6780b46a0 2024-05-10T10:35:37.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.328+0000 7ff65ffff640 1 -- 172.21.15.53:0/896744214 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff670044020 con 0x7ff6780b46a0 2024-05-10T10:35:37.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.328+0000 7ff65ffff640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6640762d0 0x7ff664078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.329+0000 7ff65ffff640 1 -- 172.21.15.53:0/896744214 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7ff6700b8190 con 0x7ff6780b46a0 2024-05-10T10:35:37.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.329+0000 7ff6777fe640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6640762d0 0x7ff664078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.329+0000 7ff6777fe640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6640762d0 0x7ff664078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff66800a990 tx=0x7ff668008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:37.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.333+0000 7ff65ffff640 1 -- 172.21.15.53:0/896744214 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff67004b030 con 0x7ff6780b46a0 2024-05-10T10:35:37.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.536+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff67806a220 con 0x7ff6780b46a0 2024-05-10T10:35:37.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.537+0000 7ff65ffff640 1 -- 172.21.15.53:0/896744214 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7ff670081920 con 0x7ff6780b46a0 2024-05-10T10:35:37.539 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:35:37.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6640762d0 msgr2=0x7ff664078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6640762d0 0x7ff664078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff66800a990 tx=0x7ff668008040 comp rx=0 tx=0).stop 2024-05-10T10:35:37.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780b46a0 msgr2=0x7ff67814f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780b46a0 0x7ff67814f7e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff67002a250 tx=0x7ff6700046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 shutdown_connections 2024-05-10T10:35:37.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6640762d0 0x7ff664078790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6780b46a0 0x7ff67814f7e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff67815f3b0 0x7ff6780ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 --2- 172.21.15.53:0/896744214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6780d5d30 0x7ff6780b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 >> 172.21.15.53:0/896744214 conn(0x7ff6780d5130 msgr2=0x7ff67806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:37.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 shutdown_connections 2024-05-10T10:35:37.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.539+0000 7ff67dc09640 1 -- 172.21.15.53:0/896744214 wait complete. 2024-05-10T10:35:37.552 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:35:37.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:37.552 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:37.592 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:35:37.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:37.592 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:35:37.648 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:35:37.649 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:35:37.649 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:37.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:35:37.668 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:35:37.668 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:35:37.668 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:37.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:35:37.706 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:35:37.706 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:35:37.706 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:37.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:35:37.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:37.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.848+0000 7f8003059640 1 Processor -- start 2024-05-10T10:35:37.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.848+0000 7f8003059640 1 -- start start 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.849+0000 7f8003059640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ffc15b6d0 0x7f7ffc0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.849+0000 7f8003059640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.849+0000 7f8003059640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ffc0c5d10 con 0x7f7ffc0cf6b0 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.849+0000 7f8003059640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7ffc0c5ff0 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.849+0000 7f8003059640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc0c62d0 con 0x7f7ffc0c6db0 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f8000df5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ffc15b6d0 0x7f7ffc0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f7ffbfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f80015f6640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7ffc0cf6b0 0x7f7ffc15f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:59640/0 (socket says 172.21.15.53:59640) 2024-05-10T10:35:37.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f80015f6640 1 -- 172.21.15.53:0/2792994326 learned_addr learned my addr 172.21.15.53:0/2792994326 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:37.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f8000df5640 1 -- 172.21.15.53:0/2792994326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 msgr2=0x7f7ffc0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:37.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f8000df5640 1 --2- 172.21.15.53:0/2792994326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc0cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f8000df5640 1 -- 172.21.15.53:0/2792994326 >> v1:172.21.15.135:6789/0 conn(0x7f7ffc0cf6b0 legacy=0x7f7ffc15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.850+0000 7f8000df5640 1 -- 172.21.15.53:0/2792994326 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc0c65b0 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f8000df5640 1 --2- 172.21.15.53:0/2792994326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0cd6d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0097a0 tx=0x7f7fec02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cd13675a359e863 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:37.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f7ffb7fe640 1 -- 172.21.15.53:0/2792994326 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec037070 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f7ffb7fe640 1 -- 172.21.15.53:0/2792994326 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec032440 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f7ffb7fe640 1 -- 172.21.15.53:0/2792994326 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec032810 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f8003059640 1 -- 172.21.15.53:0/2792994326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc15b6d0 msgr2=0x7f7ffc0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f8003059640 1 --2- 172.21.15.53:0/2792994326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0cd6d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0097a0 tx=0x7f7fec02a180 comp rx=0 tx=0).stop 2024-05-10T10:35:37.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f8003059640 1 -- 172.21.15.53:0/2792994326 shutdown_connections 2024-05-10T10:35:37.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f8003059640 1 --2- 172.21.15.53:0/2792994326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc0cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f8003059640 1 --2- 172.21.15.53:0/2792994326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0cd6d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.851+0000 7f8003059640 1 -- 172.21.15.53:0/2792994326 >> 172.21.15.53:0/2792994326 conn(0x7f7ffc0b2900 msgr2=0x7f7ffc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:37.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.852+0000 7f8003059640 1 -- 172.21.15.53:0/2792994326 shutdown_connections 2024-05-10T10:35:37.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.852+0000 7f8003059640 1 -- 172.21.15.53:0/2792994326 wait complete. 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 Processor -- start 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 -- start start 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc0cf6b0 0x7f7ffc1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc0cd2f0 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc0cd170 con 0x7f7ffc0cf6b0 2024-05-10T10:35:37.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.853+0000 7f8003059640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc0cdc10 con 0x7f7ffc0c6db0 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47802/0 (socket says 172.21.15.53:47802) 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 -- 172.21.15.53:0/2814156949 learned_addr learned my addr 172.21.15.53:0/2814156949 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f8000df5640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f7ffbfff640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc0cf6b0 0x7f7ffc1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 -- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 msgr2=0x7f7ffc154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 -- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc0cf6b0 msgr2=0x7f7ffc1f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc0cf6b0 0x7f7ffc1f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 -- 172.21.15.53:0/2814156949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc1f3b60 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f8000df5640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f7ffbfff640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc0cf6b0 0x7f7ffc1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f80015f6640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0c8f00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400cbc0 tx=0x7f7ff400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:37.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f7ff97fa640 1 -- 172.21.15.53:0/2814156949 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4017070 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f7ff97fa640 1 -- 172.21.15.53:0/2814156949 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff400ad40 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc1f3e40 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f7ff97fa640 1 -- 172.21.15.53:0/2814156949 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4012600 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.854+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1f3620 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.855+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc1f38b0 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.856+0000 7f7ff97fa640 1 -- 172.21.15.53:0/2814156949 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7ff401a070 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.857+0000 7f7ff97fa640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fe80762d0 0x7f7fe8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:37.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.857+0000 7f7ff97fa640 1 -- 172.21.15.53:0/2814156949 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7f7ff40973c0 con 0x7f7ffc15b6d0 2024-05-10T10:35:37.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.857+0000 7f8000df5640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fe80762d0 0x7f7fe8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:37.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.858+0000 7f8000df5640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fe80762d0 0x7f7fe8078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7fec034000 tx=0x7f7fec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:37.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:37.862+0000 7f7ff97fa640 1 -- 172.21.15.53:0/2814156949 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7ff40605a0 con 0x7f7ffc15b6d0 2024-05-10T10:35:38.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.093+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ffc06a220 con 0x7f7ffc15b6d0 2024-05-10T10:35:38.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.094+0000 7f7ff97fa640 1 -- 172.21.15.53:0/2814156949 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7f7ff40648a0 con 0x7f7ffc15b6d0 2024-05-10T10:35:38.095 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:38.096 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:38.096 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:38.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.095+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fe80762d0 msgr2=0x7f7fe8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.095+0000 7f8003059640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fe80762d0 0x7f7fe8078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7fec034000 tx=0x7f7fec02e000 comp rx=0 tx=0).stop 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.095+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ffc15b6d0 msgr2=0x7f7ffc0c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.095+0000 7f8003059640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0c8f00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400cbc0 tx=0x7f7ff400a430 comp rx=0 tx=0).stop 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 shutdown_connections 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fe80762d0 0x7f7fe8078790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ffc15b6d0 0x7f7ffc0c8f00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ffc0cf6b0 0x7f7ffc1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 --2- 172.21.15.53:0/2814156949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ffc0c6db0 0x7f7ffc154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 >> 172.21.15.53:0/2814156949 conn(0x7f7ffc0b2900 msgr2=0x7f7ffc0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 shutdown_connections 2024-05-10T10:35:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.096+0000 7f8003059640 1 -- 172.21.15.53:0/2814156949 wait complete. 2024-05-10T10:35:38.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:38.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.211+0000 7fdad9946640 1 Processor -- start 2024-05-10T10:35:38.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.211+0000 7fdad9946640 1 -- start start 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad9946640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdad4162750 0x7fdad40d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad9946640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad9946640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad40b45f0 con 0x7fdad40d5cb0 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad9946640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fdad40b4f20 con 0x7fdad4162750 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad9946640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40d0fe0 con 0x7fdad4162b30 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad2ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdad4162750 0x7fdad40d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad27fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdad40d5cb0 0x7fdad40d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49688/0 (socket says 172.21.15.53:49688) 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad27fc640 1 -- 172.21.15.53:0/1742613961 learned_addr learned my addr 172.21.15.53:0/1742613961 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad37fe640 1 --2- 172.21.15.53:0/1742613961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad2ffd640 1 -- 172.21.15.53:0/1742613961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 msgr2=0x7fdad40b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad2ffd640 1 --2- 172.21.15.53:0/1742613961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad2ffd640 1 -- 172.21.15.53:0/1742613961 >> v1:172.21.15.53:6789/0 conn(0x7fdad40d5cb0 legacy=0x7fdad40d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad1ffb640 1 -- 172.21.15.53:0/1742613961 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974131357 0 0) 0x7fdad40b45f0 con 0x7fdad40d5cb0 2024-05-10T10:35:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad2ffd640 1 -- 172.21.15.53:0/1742613961 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40b48d0 con 0x7fdad4162750 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad37fe640 1 --2- 172.21.15.53:0/1742613961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad2ffd640 1 --2- 172.21.15.53:0/1742613961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad4162750 0x7fdad40d0670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdac40097a0 tx=0x7fdac402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410784cca52251ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad1ffb640 1 -- 172.21.15.53:0/1742613961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4037070 con 0x7fdad4162750 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad1ffb640 1 -- 172.21.15.53:0/1742613961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac402ae50 con 0x7fdad4162750 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.213+0000 7fdad1ffb640 1 -- 172.21.15.53:0/1742613961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4032660 con 0x7fdad4162750 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 -- 172.21.15.53:0/1742613961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad4162750 msgr2=0x7fdad40d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 --2- 172.21.15.53:0/1742613961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad4162750 0x7fdad40d0670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdac40097a0 tx=0x7fdac402a100 comp rx=0 tx=0).stop 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 -- 172.21.15.53:0/1742613961 shutdown_connections 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 --2- 172.21.15.53:0/1742613961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 --2- 172.21.15.53:0/1742613961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad4162750 0x7fdad40d0670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 -- 172.21.15.53:0/1742613961 >> 172.21.15.53:0/1742613961 conn(0x7fdad409e3b0 msgr2=0x7fdad40d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 -- 172.21.15.53:0/1742613961 shutdown_connections 2024-05-10T10:35:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.214+0000 7fdad9946640 1 -- 172.21.15.53:0/1742613961 wait complete. 2024-05-10T10:35:38.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.215+0000 7fdad9946640 1 Processor -- start 2024-05-10T10:35:38.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.215+0000 7fdad9946640 1 -- start start 2024-05-10T10:35:38.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad9946640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad4162750 0x7fdad4157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad9946640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad9946640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad40ba190 0x7fdad40ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad9946640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40d0d30 con 0x7fdad40ba190 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad9946640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40d0bb0 con 0x7fdad4162750 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad9946640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40d3d40 con 0x7fdad4162b30 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad40ba190 0x7fdad40ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad40ba190 0x7fdad40ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47810/0 (socket says 172.21.15.53:47810) 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 -- 172.21.15.53:0/324934727 learned_addr learned my addr 172.21.15.53:0/324934727 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad27fc640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad2ffd640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad4162750 0x7fdad4157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 -- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 msgr2=0x7fdad40b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 -- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad4162750 msgr2=0x7fdad4157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad4162750 0x7fdad4157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 -- 172.21.15.53:0/324934727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40cf240 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad27fc640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.216+0000 7fdad37fe640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad40ba190 0x7fdad40ce530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00cd50 tx=0x7fdacc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.217+0000 7fdabbfff640 1 -- 172.21.15.53:0/324934727 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00f680 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.217+0000 7fdabbfff640 1 -- 172.21.15.53:0/324934727 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc00f840 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.217+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad40cd210 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.217+0000 7fdabbfff640 1 -- 172.21.15.53:0/324934727 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00fc30 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.217+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad40cd7d0 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.218+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad406a220 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.218+0000 7fdabbfff640 1 -- 172.21.15.53:0/324934727 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdacc00e3e0 con 0x7fdad40ba190 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.219+0000 7fdabbfff640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdac00762d0 0x7fdac0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.219+0000 7fdabbfff640 1 -- 172.21.15.53:0/324934727 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7fdacc098650 con 0x7fdad40ba190 2024-05-10T10:35:38.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.219+0000 7fdad2ffd640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdac00762d0 0x7fdac0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.220+0000 7fdad2ffd640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdac00762d0 0x7fdac0078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdac4034000 tx=0x7fdac402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.223+0000 7fdabbfff640 1 -- 172.21.15.53:0/324934727 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdacc014030 con 0x7fdad40ba190 2024-05-10T10:35:38.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.453+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdad40b6420 con 0x7fdad40ba190 2024-05-10T10:35:38.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.453+0000 7fdabbfff640 1 -- 172.21.15.53:0/324934727 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7fdacc061d70 con 0x7fdad40ba190 2024-05-10T10:35:38.455 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:38.455 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:38.455 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdac00762d0 msgr2=0x7fdac0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdac00762d0 0x7fdac0078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdac4034000 tx=0x7fdac402e000 comp rx=0 tx=0).stop 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad40ba190 msgr2=0x7fdad40ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad40ba190 0x7fdad40ce530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00cd50 tx=0x7fdacc00a650 comp rx=0 tx=0).stop 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 shutdown_connections 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdac00762d0 0x7fdac0078790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdad40ba190 0x7fdad40ce530 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdad4162b30 0x7fdad40b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 --2- 172.21.15.53:0/324934727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdad4162750 0x7fdad4157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 >> 172.21.15.53:0/324934727 conn(0x7fdad409e3b0 msgr2=0x7fdad40b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:38.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 shutdown_connections 2024-05-10T10:35:38.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.455+0000 7fdad9946640 1 -- 172.21.15.53:0/324934727 wait complete. 2024-05-10T10:35:38.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:35:38.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.569+0000 7ff3ec33f640 1 Processor -- start 2024-05-10T10:35:38.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.569+0000 7ff3ec33f640 1 -- start start 2024-05-10T10:35:38.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ec33f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff3e40d3d80 0x7ff3e40b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ec33f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3e40d59d0 0x7ff3e40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ec33f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e40d2d20 con 0x7ff3e4162940 2024-05-10T10:35:38.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ec33f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e4162d20 con 0x7ff3e40d59d0 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ec33f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e40d3000 con 0x7ff3e40d3d80 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ea8dc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3e40d59d0 0x7ff3e40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ea0db640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff3e4162940 0x7ff3e40d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49694/0 (socket says 172.21.15.53:49694) 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.570+0000 7ff3ea0db640 1 -- 172.21.15.53:0/996102747 learned_addr learned my addr 172.21.15.53:0/996102747 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3ea8dc640 1 -- 172.21.15.53:0/996102747 >> v2:172.21.15.135:3300/0 conn(0x7ff3e40d3d80 msgr2=0x7ff3e40b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3e98da640 1 --2- 172.21.15.53:0/996102747 >> v2:172.21.15.135:3300/0 conn(0x7ff3e40d3d80 0x7ff3e40b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3ea8dc640 1 --2- 172.21.15.53:0/996102747 >> v2:172.21.15.135:3300/0 conn(0x7ff3e40d3d80 0x7ff3e40b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3ea8dc640 1 -- 172.21.15.53:0/996102747 >> v1:172.21.15.53:6789/0 conn(0x7ff3e4162940 legacy=0x7ff3e40d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3e90d9640 1 -- 172.21.15.53:0/996102747 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3738731460 0 0) 0x7ff3e40d2d20 con 0x7ff3e4162940 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3ea8dc640 1 -- 172.21.15.53:0/996102747 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e40d32e0 con 0x7ff3e40d59d0 2024-05-10T10:35:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3e98da640 1 --2- 172.21.15.53:0/996102747 >> v2:172.21.15.135:3300/0 conn(0x7ff3e40d3d80 0x7ff3e40b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.571+0000 7ff3ea8dc640 1 --2- 172.21.15.53:0/996102747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e40d01e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff3e409eab0 tx=0x7ff3d402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f577add3328f4790 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3e90d9640 1 -- 172.21.15.53:0/996102747 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4007740 con 0x7ff3e40d59d0 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3e90d9640 1 -- 172.21.15.53:0/996102747 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d40078e0 con 0x7ff3e40d59d0 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3e90d9640 1 -- 172.21.15.53:0/996102747 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4007cb0 con 0x7ff3e40d59d0 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 -- 172.21.15.53:0/996102747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 msgr2=0x7ff3e40d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/996102747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e40d01e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff3e409eab0 tx=0x7ff3d402a5f0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 -- 172.21.15.53:0/996102747 shutdown_connections 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/996102747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e40d01e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/996102747 >> v2:172.21.15.135:3300/0 conn(0x7ff3e40d3d80 0x7ff3e40b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 -- 172.21.15.53:0/996102747 >> 172.21.15.53:0/996102747 conn(0x7ff3e409e380 msgr2=0x7ff3e40d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 -- 172.21.15.53:0/996102747 shutdown_connections 2024-05-10T10:35:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.572+0000 7ff3ec33f640 1 -- 172.21.15.53:0/996102747 wait complete. 2024-05-10T10:35:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 Processor -- start 2024-05-10T10:35:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 -- start start 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3e40d3d80 0x7ff3e415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e41f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3e4162940 0x7ff3e40b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40d2390 con 0x7ff3e40d59d0 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40cfc50 con 0x7ff3e4162940 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e40d2510 con 0x7ff3e40d3d80 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e41f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e41f4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47820/0 (socket says 172.21.15.53:47820) 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 -- 172.21.15.53:0/2761330618 learned_addr learned my addr 172.21.15.53:0/2761330618 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ea8dc640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3e4162940 0x7ff3e40b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ea0db640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3e40d3d80 0x7ff3e415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 -- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3e40d3d80 msgr2=0x7ff3e415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3e40d3d80 0x7ff3e415d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 -- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3e4162940 msgr2=0x7ff3e40b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3e4162940 0x7ff3e40b9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 -- 172.21.15.53:0/2761330618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e41f5680 con 0x7ff3e40d59d0 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e98da640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e41f4b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c7f0 tx=0x7ff3d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e2ffd640 1 -- 172.21.15.53:0/2761330618 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8015070 con 0x7ff3e40d59d0 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e2ffd640 1 -- 172.21.15.53:0/2761330618 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d8007820 con 0x7ff3e40d59d0 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e41f5960 con 0x7ff3e40d59d0 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3e2ffd640 1 -- 172.21.15.53:0/2761330618 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8007c10 con 0x7ff3e40d59d0 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.575+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e40cba00 con 0x7ff3e40d59d0 2024-05-10T10:35:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.576+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e406a220 con 0x7ff3e40d59d0 2024-05-10T10:35:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.577+0000 7ff3e2ffd640 1 -- 172.21.15.53:0/2761330618 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff3d8018070 con 0x7ff3e40d59d0 2024-05-10T10:35:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.577+0000 7ff3e2ffd640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3d00762d0 0x7ff3d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.579+0000 7ff3e2ffd640 1 -- 172.21.15.53:0/2761330618 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7ff3d8097820 con 0x7ff3e40d59d0 2024-05-10T10:35:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.579+0000 7ff3ea0db640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3d00762d0 0x7ff3d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.579+0000 7ff3ea0db640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3d00762d0 0x7ff3d0078790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc007fd0 tx=0x7ff3dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.582+0000 7ff3e2ffd640 1 -- 172.21.15.53:0/2761330618 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff3d8061050 con 0x7ff3e40d59d0 2024-05-10T10:35:38.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.776+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3e40d0b90 con 0x7ff3e40d59d0 2024-05-10T10:35:38.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.777+0000 7ff3e2ffd640 1 -- 172.21.15.53:0/2761330618 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17037 (secure 0 0 0) 0x7ff3d8060950 con 0x7ff3e40d59d0 2024-05-10T10:35:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:38.779 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":50,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:35:23.254735+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:54.908375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:56.185211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.17","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3d00762d0 msgr2=0x7ff3d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3d00762d0 0x7ff3d0078790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc007fd0 tx=0x7ff3dc009040 comp rx=0 tx=0).stop 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 msgr2=0x7ff3e41f4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e41f4b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c7f0 tx=0x7ff3d800ccb0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 shutdown_connections 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3d00762d0 0x7ff3d0078790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3e4162940 0x7ff3e40b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e40d59d0 0x7ff3e41f4b90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 --2- 172.21.15.53:0/2761330618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3e40d3d80 0x7ff3e415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 >> 172.21.15.53:0/2761330618 conn(0x7ff3e409e380 msgr2=0x7ff3e406e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:38.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 shutdown_connections 2024-05-10T10:35:38.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.779+0000 7ff3ec33f640 1 -- 172.21.15.53:0/2761330618 wait complete. 2024-05-10T10:35:38.792 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:35:38.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:38.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.895+0000 7f6f0eeda640 1 Processor -- start 2024-05-10T10:35:38.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.895+0000 7f6f0eeda640 1 -- start start 2024-05-10T10:35:38.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f0eeda640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6f080d5cb0 0x7f6f080d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f0eeda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f08162b30 0x7f6f080d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f0eeda640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f080b45f0 con 0x7f6f08162750 2024-05-10T10:35:38.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f0eeda640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6f080b4f20 con 0x7f6f080d5cb0 2024-05-10T10:35:38.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f0eeda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6f080d0fe0 con 0x7f6f08162b30 2024-05-10T10:35:38.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6f080d5cb0 0x7f6f080d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6f080d5cb0 0x7f6f080d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47824/0 (socket says 172.21.15.53:47824) 2024-05-10T10:35:38.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f07fff640 1 -- 172.21.15.53:0/2045801343 learned_addr learned my addr 172.21.15.53:0/2045801343 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:38.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f0d477640 1 --2- 172.21.15.53:0/2045801343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f08162b30 0x7f6f080d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f07fff640 1 -- 172.21.15.53:0/2045801343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f08162b30 msgr2=0x7f6f080d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:38.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f07fff640 1 --2- 172.21.15.53:0/2045801343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f08162b30 0x7f6f080d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f07fff640 1 -- 172.21.15.53:0/2045801343 >> v1:172.21.15.135:6789/0 conn(0x7f6f08162750 legacy=0x7f6f080d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.896+0000 7f6f07fff640 1 -- 172.21.15.53:0/2045801343 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f080b48d0 con 0x7f6f080d5cb0 2024-05-10T10:35:38.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.897+0000 7f6f07fff640 1 --2- 172.21.15.53:0/2045801343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f080d5cb0 0x7f6f080d3630 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009870 tx=0x7f6efc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80621a5c4fdb01d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.897+0000 7f6f077fe640 1 -- 172.21.15.53:0/2045801343 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc037070 con 0x7f6f080d5cb0 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.897+0000 7f6f077fe640 1 -- 172.21.15.53:0/2045801343 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc032440 con 0x7f6f080d5cb0 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.897+0000 7f6f077fe640 1 -- 172.21.15.53:0/2045801343 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc032810 con 0x7f6f080d5cb0 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 -- 172.21.15.53:0/2045801343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f080d5cb0 msgr2=0x7f6f080d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/2045801343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f080d5cb0 0x7f6f080d3630 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009870 tx=0x7f6efc02a1c0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 -- 172.21.15.53:0/2045801343 shutdown_connections 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/2045801343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f08162b30 0x7f6f080d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/2045801343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f080d5cb0 0x7f6f080d3630 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 -- 172.21.15.53:0/2045801343 >> 172.21.15.53:0/2045801343 conn(0x7f6f0809e3b0 msgr2=0x7f6f080d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 -- 172.21.15.53:0/2045801343 shutdown_connections 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.898+0000 7f6f0eeda640 1 -- 172.21.15.53:0/2045801343 wait complete. 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.899+0000 7f6f0eeda640 1 Processor -- start 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.899+0000 7f6f0eeda640 1 -- start start 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0eeda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f08162750 0x7f6f080c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0eeda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f08162b30 0x7f6f080cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0eeda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f080ce520 0x7f6f080ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0eeda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f080b53b0 con 0x7f6f08162b30 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0eeda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6f080d0d80 con 0x7f6f08162750 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0eeda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6f080b5530 con 0x7f6f080ce520 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f08162b30 0x7f6f080cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f08162b30 0x7f6f080cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47826/0 (socket says 172.21.15.53:47826) 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f07fff640 1 -- 172.21.15.53:0/3253490376 learned_addr learned my addr 172.21.15.53:0/3253490376 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0d477640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f080ce520 0x7f6f080ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.900+0000 7f6f0cc76640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f08162750 0x7f6f080c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f07fff640 1 -- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f080ce520 msgr2=0x7f6f080ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f07fff640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f080ce520 0x7f6f080ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f07fff640 1 -- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f08162750 msgr2=0x7f6f080c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f07fff640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f08162750 0x7f6f080c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:38.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f07fff640 1 -- 172.21.15.53:0/3253490376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f08157910 con 0x7f6f08162b30 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f0cc76640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f08162750 0x7f6f080c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f0d477640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f080ce520 0x7f6f080ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f07fff640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f08162b30 0x7f6f080cdfe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000c00 tx=0x7f6efc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f057fa640 1 -- 172.21.15.53:0/3253490376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc037040 con 0x7f6f08162b30 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f057fa640 1 -- 172.21.15.53:0/3253490376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc042070 con 0x7f6f08162b30 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f080ceaa0 con 0x7f6f08162b30 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f057fa640 1 -- 172.21.15.53:0/3253490376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc030880 con 0x7f6f08162b30 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.901+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f080b99f0 con 0x7f6f08162b30 2024-05-10T10:35:38.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.902+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0806a220 con 0x7f6f08162b30 2024-05-10T10:35:38.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.903+0000 7f6f057fa640 1 -- 172.21.15.53:0/3253490376 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6efc02ad10 con 0x7f6f08162b30 2024-05-10T10:35:38.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.903+0000 7f6f057fa640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6ef40762d0 0x7f6ef4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:38.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.903+0000 7f6f057fa640 1 -- 172.21.15.53:0/3253490376 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7f6efc0b7da0 con 0x7f6f08162b30 2024-05-10T10:35:38.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.903+0000 7f6f0cc76640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6ef40762d0 0x7f6ef4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:38.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.904+0000 7f6f0cc76640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6ef40762d0 0x7f6ef4078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f080ca710 tx=0x7f6ef8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:38.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:38.907+0000 7f6f057fa640 1 -- 172.21.15.53:0/3253490376 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6efc081540 con 0x7f6f08162b30 2024-05-10T10:35:39.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.139+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f080d1690 con 0x7f6f08162b30 2024-05-10T10:35:39.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.139+0000 7f6f057fa640 1 -- 172.21.15.53:0/3253490376 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7f6efc080e40 con 0x7f6f08162b30 2024-05-10T10:35:39.140 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:39.140 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:39.140 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:39.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6ef40762d0 msgr2=0x7f6ef4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6ef40762d0 0x7f6ef4078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f080ca710 tx=0x7f6ef8009040 comp rx=0 tx=0).stop 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f08162b30 msgr2=0x7f6f080cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f08162b30 0x7f6f080cdfe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000c00 tx=0x7f6efc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 shutdown_connections 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6ef40762d0 0x7f6ef4078790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f080ce520 0x7f6f080ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f08162b30 0x7f6f080cdfe0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 --2- 172.21.15.53:0/3253490376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f08162750 0x7f6f080c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 >> 172.21.15.53:0/3253490376 conn(0x7f6f0809e3b0 msgr2=0x7f6f0806dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 shutdown_connections 2024-05-10T10:35:39.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.142+0000 7f6f0eeda640 1 -- 172.21.15.53:0/3253490376 wait complete. 2024-05-10T10:35:39.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:39.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.256+0000 7f6899c1c640 1 Processor -- start 2024-05-10T10:35:39.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.256+0000 7f6899c1c640 1 -- start start 2024-05-10T10:35:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f6899c1c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68940d3450 0x7f68940d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f6899c1c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68940702e0 0x7f689406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f6899c1c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689406f3c0 con 0x7f689415f290 2024-05-10T10:35:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f6899c1c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f68940d14c0 con 0x7f68940d3450 2024-05-10T10:35:39.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f6899c1c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f68940d2300 con 0x7f68940702e0 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f6892ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68940d3450 0x7f68940d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f68937fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f689415f290 0x7f68940d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49696/0 (socket says 172.21.15.53:49696) 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.257+0000 7f68937fe640 1 -- 172.21.15.53:0/2687582256 learned_addr learned my addr 172.21.15.53:0/2687582256 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.258+0000 7f6893fff640 1 --2- 172.21.15.53:0/2687582256 >> v2:172.21.15.135:3300/0 conn(0x7f68940702e0 0x7f689406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.258+0000 7f6892ffd640 1 -- 172.21.15.53:0/2687582256 >> v2:172.21.15.135:3300/0 conn(0x7f68940702e0 msgr2=0x7f689406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.258+0000 7f6892ffd640 1 --2- 172.21.15.53:0/2687582256 >> v2:172.21.15.135:3300/0 conn(0x7f68940702e0 0x7f689406d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.258+0000 7f6892ffd640 1 -- 172.21.15.53:0/2687582256 >> v1:172.21.15.53:6789/0 conn(0x7f689415f290 legacy=0x7f68940d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.258+0000 7f68927fc640 1 -- 172.21.15.53:0/2687582256 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 702329108 0 0) 0x7f689406f3c0 con 0x7f689415f290 2024-05-10T10:35:39.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.258+0000 7f6892ffd640 1 -- 172.21.15.53:0/2687582256 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68940d16f0 con 0x7f68940d3450 2024-05-10T10:35:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.258+0000 7f6892ffd640 1 --2- 172.21.15.53:0/2687582256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940d0f80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f68940b5a80 tx=0x7f688802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8b4b9eddf6e8ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f68927fc640 1 -- 172.21.15.53:0/2687582256 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888037070 con 0x7f68940d3450 2024-05-10T10:35:39.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f68927fc640 1 -- 172.21.15.53:0/2687582256 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6888032440 con 0x7f68940d3450 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f68927fc640 1 -- 172.21.15.53:0/2687582256 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888032810 con 0x7f68940d3450 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 -- 172.21.15.53:0/2687582256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 msgr2=0x7f68940d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 --2- 172.21.15.53:0/2687582256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940d0f80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f68940b5a80 tx=0x7f688802a230 comp rx=0 tx=0).stop 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 -- 172.21.15.53:0/2687582256 shutdown_connections 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 --2- 172.21.15.53:0/2687582256 >> v2:172.21.15.135:3300/0 conn(0x7f68940702e0 0x7f689406d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 --2- 172.21.15.53:0/2687582256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940d0f80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 -- 172.21.15.53:0/2687582256 >> 172.21.15.53:0/2687582256 conn(0x7f689409ba20 msgr2=0x7f689409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 -- 172.21.15.53:0/2687582256 shutdown_connections 2024-05-10T10:35:39.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.259+0000 7f6899c1c640 1 -- 172.21.15.53:0/2687582256 wait complete. 2024-05-10T10:35:39.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.260+0000 7f6899c1c640 1 Processor -- start 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6899c1c640 1 -- start start 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6899c1c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68940702e0 0x7f689409d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6899c1c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6899c1c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689415f290 0x7f6894154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6899c1c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f68940d1f90 con 0x7f68940d3450 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6899c1c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68940d1cb0 con 0x7f68940702e0 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6899c1c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f689415f670 con 0x7f689415f290 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6892ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6892ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47846/0 (socket says 172.21.15.53:47846) 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.261+0000 7f6892ffd640 1 -- 172.21.15.53:0/914165959 learned_addr learned my addr 172.21.15.53:0/914165959 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f68937fe640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68940702e0 0x7f689409d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6893fff640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689415f290 0x7f6894154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6892ffd640 1 -- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689415f290 msgr2=0x7f6894154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6892ffd640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689415f290 0x7f6894154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6892ffd640 1 -- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68940702e0 msgr2=0x7f689409d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6892ffd640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68940702e0 0x7f689409d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6892ffd640 1 -- 172.21.15.53:0/914165959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689409e510 con 0x7f68940d3450 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f68937fe640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68940702e0 0x7f689409d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6892ffd640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940c8e60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6888000c00 tx=0x7f68880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f687bfff640 1 -- 172.21.15.53:0/914165959 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888037050 con 0x7f68940d3450 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f687bfff640 1 -- 172.21.15.53:0/914165959 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6888038040 con 0x7f68940d3450 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689409e7f0 con 0x7f68940d3450 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f687bfff640 1 -- 172.21.15.53:0/914165959 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888004470 con 0x7f68940d3450 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.262+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689409edb0 con 0x7f68940d3450 2024-05-10T10:35:39.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.263+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68940693f0 con 0x7f68940d3450 2024-05-10T10:35:39.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.264+0000 7f687bfff640 1 -- 172.21.15.53:0/914165959 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6888030900 con 0x7f68940d3450 2024-05-10T10:35:39.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.264+0000 7f687bfff640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68800762d0 0x7f6880078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.264+0000 7f687bfff640 1 -- 172.21.15.53:0/914165959 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7f68880b8310 con 0x7f68940d3450 2024-05-10T10:35:39.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.265+0000 7f68937fe640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68800762d0 0x7f6880078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.265+0000 7f68937fe640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68800762d0 0x7f6880078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6884000f60 tx=0x7f6884009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.269+0000 7f687bfff640 1 -- 172.21.15.53:0/914165959 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6888081a20 con 0x7f68940d3450 2024-05-10T10:35:39.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.502+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68940ca290 con 0x7f68940d3450 2024-05-10T10:35:39.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.502+0000 7f687bfff640 1 -- 172.21.15.53:0/914165959 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5178 (secure 0 0 0) 0x7f6888081320 con 0x7f68940d3450 2024-05-10T10:35:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:39.503 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:34:57.236131+0000","modified":"2024-05-10T10:35:00.256407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/679529875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":679529875},{"type":"v1","addr":"172.21.15.53:6835","nonce":679529875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5232,"qdb_cluster":[5232]},"id":3}]} 2024-05-10T10:35:39.504 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 26 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68800762d0 msgr2=0x7f6880078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68800762d0 0x7f6880078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6884000f60 tx=0x7f6884009040 comp rx=0 tx=0).stop 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 msgr2=0x7f68940c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940c8e60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6888000c00 tx=0x7f68880046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 shutdown_connections 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f68800762d0 0x7f6880078790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689415f290 0x7f6894154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68940d3450 0x7f68940c8e60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 --2- 172.21.15.53:0/914165959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68940702e0 0x7f689409d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 >> 172.21.15.53:0/914165959 conn(0x7f689409ba20 msgr2=0x7f68940b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:39.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 shutdown_connections 2024-05-10T10:35:39.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.505+0000 7f6899c1c640 1 -- 172.21.15.53:0/914165959 wait complete. 2024-05-10T10:35:39.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:35:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.619+0000 7f8b3d919640 1 Processor -- start 2024-05-10T10:35:39.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.619+0000 7f8b3d919640 1 -- start start 2024-05-10T10:35:39.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b3d919640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b38146750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b3d919640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8b3806c6a0 0x7f8b38070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b3d919640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8b38073090 0x7f8b38073520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b3d919640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8b380af4a0 con 0x7f8b38073090 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b3d919640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8b380719d0 con 0x7f8b3806c6a0 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b3d919640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8b38071150 con 0x7f8b3806c2c0 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b377fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8b38073090 0x7f8b38073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b377fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8b38073090 0x7f8b38073520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47862/0 (socket says 172.21.15.53:47862) 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.620+0000 7f8b377fe640 1 -- 172.21.15.53:0/958379043 learned_addr learned my addr 172.21.15.53:0/958379043 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.621+0000 7f8b36ffd640 1 --2- 172.21.15.53:0/958379043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b38146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.621+0000 7f8b367fc640 1 --2- 172.21.15.53:0/958379043 >> v2:172.21.15.135:3300/0 conn(0x7f8b3806c6a0 0x7f8b38070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.621+0000 7f8b377fe640 1 -- 172.21.15.53:0/958379043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 msgr2=0x7f8b38146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.621+0000 7f8b377fe640 1 --2- 172.21.15.53:0/958379043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b38146750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.621+0000 7f8b377fe640 1 -- 172.21.15.53:0/958379043 >> v2:172.21.15.135:3300/0 conn(0x7f8b3806c6a0 msgr2=0x7f8b38070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:39.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.621+0000 7f8b377fe640 1 --2- 172.21.15.53:0/958379043 >> v2:172.21.15.135:3300/0 conn(0x7f8b3806c6a0 0x7f8b38070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.621+0000 7f8b377fe640 1 -- 172.21.15.53:0/958379043 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b38071410 con 0x7f8b38073090 2024-05-10T10:35:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b377fe640 1 --2- 172.21.15.53:0/958379043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b38073090 0x7f8b38073520 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b380b49d0 tx=0x7f8b2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7222bf0ac6073d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b35ffb640 1 -- 172.21.15.53:0/958379043 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28037070 con 0x7f8b38073090 2024-05-10T10:35:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b35ffb640 1 -- 172.21.15.53:0/958379043 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b28032440 con 0x7f8b38073090 2024-05-10T10:35:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b35ffb640 1 -- 172.21.15.53:0/958379043 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28032810 con 0x7f8b38073090 2024-05-10T10:35:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 -- 172.21.15.53:0/958379043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b38073090 msgr2=0x7f8b38073520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 --2- 172.21.15.53:0/958379043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b38073090 0x7f8b38073520 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b380b49d0 tx=0x7f8b2802a1e0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 -- 172.21.15.53:0/958379043 shutdown_connections 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 --2- 172.21.15.53:0/958379043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b38073090 0x7f8b38073520 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 --2- 172.21.15.53:0/958379043 >> v2:172.21.15.135:3300/0 conn(0x7f8b3806c6a0 0x7f8b38070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 --2- 172.21.15.53:0/958379043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b38146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 -- 172.21.15.53:0/958379043 >> 172.21.15.53:0/958379043 conn(0x7f8b380d1d70 msgr2=0x7f8b380d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 -- 172.21.15.53:0/958379043 shutdown_connections 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.622+0000 7f8b3d919640 1 -- 172.21.15.53:0/958379043 wait complete. 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.623+0000 7f8b3d919640 1 Processor -- start 2024-05-10T10:35:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.624+0000 7f8b3d919640 1 -- start start 2024-05-10T10:35:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.624+0000 7f8b3d919640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b381e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.624+0000 7f8b3d919640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b3806c6a0 0x7f8b380b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.624+0000 7f8b3d919640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b38073090 0x7f8b380d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.624+0000 7f8b3d919640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3806cb00 con 0x7f8b3806c6a0 2024-05-10T10:35:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.624+0000 7f8b3d919640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b380af130 con 0x7f8b38073090 2024-05-10T10:35:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.624+0000 7f8b3d919640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380b1530 con 0x7f8b3806c2c0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b3806c6a0 0x7f8b380b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b3806c6a0 0x7f8b380b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47878/0 (socket says 172.21.15.53:47878) 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 -- 172.21.15.53:0/2131394910 learned_addr learned my addr 172.21.15.53:0/2131394910 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b377fe640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b38073090 0x7f8b380d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b36ffd640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b381e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 -- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 msgr2=0x7f8b381e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b381e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 -- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b38073090 msgr2=0x7f8b380d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b38073090 0x7f8b380d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 -- 172.21.15.53:0/2131394910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b381e6cc0 con 0x7f8b3806c6a0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b367fc640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b3806c6a0 0x7f8b380b6930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000d800 tx=0x7f8b2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b2ffff640 1 -- 172.21.15.53:0/2131394910 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2000b6c0 con 0x7f8b3806c6a0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b2ffff640 1 -- 172.21.15.53:0/2131394910 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b20019070 con 0x7f8b3806c6a0 2024-05-10T10:35:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b381e6fa0 con 0x7f8b3806c6a0 2024-05-10T10:35:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b2ffff640 1 -- 172.21.15.53:0/2131394910 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2000ba80 con 0x7f8b3806c6a0 2024-05-10T10:35:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.625+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381e7560 con 0x7f8b3806c6a0 2024-05-10T10:35:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.626+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b380693f0 con 0x7f8b3806c6a0 2024-05-10T10:35:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.627+0000 7f8b2ffff640 1 -- 172.21.15.53:0/2131394910 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8b2000bc20 con 0x7f8b3806c6a0 2024-05-10T10:35:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.627+0000 7f8b2ffff640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8b180762d0 0x7f8b18078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.627+0000 7f8b2ffff640 1 -- 172.21.15.53:0/2131394910 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7f8b20097c10 con 0x7f8b3806c6a0 2024-05-10T10:35:39.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.628+0000 7f8b36ffd640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8b180762d0 0x7f8b18078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.630 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.628+0000 7f8b36ffd640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8b180762d0 0x7f8b18078790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b3815c460 tx=0x7f8b30003a10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.632+0000 7f8b2ffff640 1 -- 172.21.15.53:0/2131394910 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8b20061440 con 0x7f8b3806c6a0 2024-05-10T10:35:39.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.823+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b38145dc0 con 0x7f8b3806c6a0 2024-05-10T10:35:39.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.825+0000 7f8b2ffff640 1 -- 172.21.15.53:0/2131394910 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17037 (secure 0 0 0) 0x7f8b200942a0 con 0x7f8b3806c6a0 2024-05-10T10:35:39.825 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:39.825 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":50,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:35:23.254735+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:34:54.908375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T10:34:56.185211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.17","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8b180762d0 msgr2=0x7f8b18078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8b180762d0 0x7f8b18078790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b3815c460 tx=0x7f8b30003a10 comp rx=0 tx=0).stop 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b3806c6a0 msgr2=0x7f8b380b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b3806c6a0 0x7f8b380b6930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000d800 tx=0x7f8b2000dcc0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 shutdown_connections 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8b180762d0 0x7f8b18078790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b38073090 0x7f8b380d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b3806c6a0 0x7f8b380b6930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 --2- 172.21.15.53:0/2131394910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8b3806c2c0 0x7f8b381e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 >> 172.21.15.53:0/2131394910 conn(0x7f8b380d1d70 msgr2=0x7f8b380b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 shutdown_connections 2024-05-10T10:35:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.827+0000 7f8b3d919640 1 -- 172.21.15.53:0/2131394910 wait complete. 2024-05-10T10:35:39.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:35:39.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.943+0000 7f514c740640 1 Processor -- start 2024-05-10T10:35:39.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.943+0000 7f514c740640 1 -- start start 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514c740640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51441611e0 0x7f51440cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514c740640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51440c65f0 0x7f514415d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514c740640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51440c6a80 con 0x7f514415ce60 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514c740640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f51440cd090 con 0x7f51440c65f0 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514c740640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f51440cd370 con 0x7f51441611e0 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514acdd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51440c65f0 0x7f514415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514acdd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51440c65f0 0x7f514415d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47884/0 (socket says 172.21.15.53:47884) 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514acdd640 1 -- 172.21.15.53:0/1757860679 learned_addr learned my addr 172.21.15.53:0/1757860679 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f5149cdb640 1 --2- 172.21.15.53:0/1757860679 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51441611e0 0x7f51440cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514acdd640 1 -- 172.21.15.53:0/1757860679 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51441611e0 msgr2=0x7f51440cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514acdd640 1 --2- 172.21.15.53:0/1757860679 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51441611e0 0x7f51440cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514acdd640 1 -- 172.21.15.53:0/1757860679 >> v1:172.21.15.135:6789/0 conn(0x7f514415ce60 legacy=0x7f5144160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:39.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f514acdd640 1 -- 172.21.15.53:0/1757860679 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51440cd650 con 0x7f51440c65f0 2024-05-10T10:35:39.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.944+0000 7f5149cdb640 1 --2- 172.21.15.53:0/1757860679 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51441611e0 0x7f51440cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:39.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.945+0000 7f514acdd640 1 --2- 172.21.15.53:0/1757860679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51440c65f0 0x7f514415d240 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f513c009870 tx=0x7f513c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5904cbab2fd5b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.945+0000 7f51494da640 1 -- 172.21.15.53:0/1757860679 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c037070 con 0x7f51440c65f0 2024-05-10T10:35:39.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.945+0000 7f51494da640 1 -- 172.21.15.53:0/1757860679 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c032440 con 0x7f51440c65f0 2024-05-10T10:35:39.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.945+0000 7f51494da640 1 -- 172.21.15.53:0/1757860679 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c032810 con 0x7f51440c65f0 2024-05-10T10:35:39.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 -- 172.21.15.53:0/1757860679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51440c65f0 msgr2=0x7f514415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 --2- 172.21.15.53:0/1757860679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51440c65f0 0x7f514415d240 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f513c009870 tx=0x7f513c02a180 comp rx=0 tx=0).stop 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 -- 172.21.15.53:0/1757860679 shutdown_connections 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 --2- 172.21.15.53:0/1757860679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51440c65f0 0x7f514415d240 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 --2- 172.21.15.53:0/1757860679 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51441611e0 0x7f51440cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 -- 172.21.15.53:0/1757860679 >> 172.21.15.53:0/1757860679 conn(0x7f51440b2900 msgr2=0x7f51440b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 -- 172.21.15.53:0/1757860679 shutdown_connections 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.946+0000 7f514c740640 1 -- 172.21.15.53:0/1757860679 wait complete. 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.947+0000 7f514c740640 1 Processor -- start 2024-05-10T10:35:39.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.947+0000 7f514c740640 1 -- start start 2024-05-10T10:35:39.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514c740640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51440c65f0 0x7f51441502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514c740640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f514415ce60 0x7f51440aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514c740640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51441611e0 0x7f51440c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514c740640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51440cde50 con 0x7f514415ce60 2024-05-10T10:35:39.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514c740640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51440ccd60 con 0x7f51441611e0 2024-05-10T10:35:39.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514c740640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f51440cdfd0 con 0x7f51440c65f0 2024-05-10T10:35:39.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f514415ce60 0x7f51440aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f514415ce60 0x7f51440aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47894/0 (socket says 172.21.15.53:47894) 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 -- 172.21.15.53:0/3963976466 learned_addr learned my addr 172.21.15.53:0/3963976466 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514a4dc640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51440c65f0 0x7f51441502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 -- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51440c65f0 msgr2=0x7f51441502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51440c65f0 0x7f51441502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 -- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51441611e0 msgr2=0x7f51440c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51441611e0 0x7f51440c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 -- 172.21.15.53:0/3963976466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51441507e0 con 0x7f514415ce60 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f5149cdb640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f514415ce60 0x7f51440aedf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f513400c7f0 tx=0x7f513400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f513affd640 1 -- 172.21.15.53:0/3963976466 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134015070 con 0x7f514415ce60 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f513affd640 1 -- 172.21.15.53:0/3963976466 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5134007820 con 0x7f514415ce60 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f513affd640 1 -- 172.21.15.53:0/3963976466 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134007bd0 con 0x7f514415ce60 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.948+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51440af6d0 con 0x7f514415ce60 2024-05-10T10:35:39.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.949+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51440c8200 con 0x7f514415ce60 2024-05-10T10:35:39.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.949+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514406a220 con 0x7f514415ce60 2024-05-10T10:35:39.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.950+0000 7f513affd640 1 -- 172.21.15.53:0/3963976466 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5134018070 con 0x7f514415ce60 2024-05-10T10:35:39.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.951+0000 7f513affd640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51280762d0 0x7f5128078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:39.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.951+0000 7f513affd640 1 -- 172.21.15.53:0/3963976466 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6159+0+0 (secure 0 0 0) 0x7f5134097830 con 0x7f514415ce60 2024-05-10T10:35:39.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.951+0000 7f514a4dc640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51280762d0 0x7f5128078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:39.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.951+0000 7f514a4dc640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51280762d0 0x7f5128078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5140007fd0 tx=0x7f5140009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:39.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:39.955+0000 7f513affd640 1 -- 172.21.15.53:0/3963976466 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5134061060 con 0x7f514415ce60 2024-05-10T10:35:40.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.186+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f51440c7300 con 0x7f514415ce60 2024-05-10T10:35:40.187 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:35:40.186+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:35:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.187+0000 7f513affd640 1 -- 172.21.15.53:0/3963976466 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f5134060960 con 0x7f514415ce60 2024-05-10T10:35:40.188 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51280762d0 msgr2=0x7f5128078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51280762d0 0x7f5128078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5140007fd0 tx=0x7f5140009040 comp rx=0 tx=0).stop 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f514415ce60 msgr2=0x7f51440aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f514415ce60 0x7f51440aedf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f513400c7f0 tx=0x7f513400ccb0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 shutdown_connections 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51280762d0 0x7f5128078790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51441611e0 0x7f51440c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f514415ce60 0x7f51440aedf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 --2- 172.21.15.53:0/3963976466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51440c65f0 0x7f51441502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 >> 172.21.15.53:0/3963976466 conn(0x7f51440b2900 msgr2=0x7f5144071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 shutdown_connections 2024-05-10T10:35:40.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.190+0000 7f514c740640 1 -- 172.21.15.53:0/3963976466 wait complete. 2024-05-10T10:35:40.194 INFO:tasks.ceph.mds.a.smithi053.stderr: -5812> 2024-05-10T10:35:03.494+0000 7fbcf5f8d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:35:40.197 INFO:tasks.ceph.mds.a.smithi053.stderr: -3053> 2024-05-10T10:35:05.928+0000 7fbcf1f85640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5370=0-4194304@1} caps={5370=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5370 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b057daac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:35:40.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:35:40.226 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.310+0000 7f2916312640 1 Processor -- start 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.310+0000 7f2916312640 1 -- start start 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f2916312640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f29100b52a0 0x7f291006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f2916312640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f29100730b0 0x7f2910073520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f2916312640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29100b68c0 con 0x7f291015d600 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f2916312640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2910073a60 con 0x7f29100b52a0 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f2916312640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f291006f650 con 0x7f29100730b0 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f290f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f29100b52a0 0x7f291006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f290ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f291015d600 0x7f29100b61b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49714/0 (socket says 172.21.15.53:49714) 2024-05-10T10:35:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f290ffff640 1 -- 172.21.15.53:0/2990424057 learned_addr learned my addr 172.21.15.53:0/2990424057 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f29148af640 1 --2- 172.21.15.53:0/2990424057 >> v2:172.21.15.135:3300/0 conn(0x7f29100730b0 0x7f2910073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f290f7fe640 1 -- 172.21.15.53:0/2990424057 >> v2:172.21.15.135:3300/0 conn(0x7f29100730b0 msgr2=0x7f2910073520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f290f7fe640 1 --2- 172.21.15.53:0/2990424057 >> v2:172.21.15.135:3300/0 conn(0x7f29100730b0 0x7f2910073520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f290f7fe640 1 -- 172.21.15.53:0/2990424057 >> v1:172.21.15.53:6789/0 conn(0x7f291015d600 legacy=0x7f29100b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.311+0000 7f290f7fe640 1 -- 172.21.15.53:0/2990424057 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29100af980 con 0x7f29100b52a0 2024-05-10T10:35:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f290f7fe640 1 --2- 172.21.15.53:0/2990424057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100b52a0 0x7f291006f110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f291006f010 tx=0x7f290402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2d253b6b1d01dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f290effd640 1 -- 172.21.15.53:0/2990424057 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904031710 con 0x7f29100b52a0 2024-05-10T10:35:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f290effd640 1 -- 172.21.15.53:0/2990424057 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29040318b0 con 0x7f29100b52a0 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f290effd640 1 -- 172.21.15.53:0/2990424057 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904031c80 con 0x7f29100b52a0 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f2916312640 1 -- 172.21.15.53:0/2990424057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100b52a0 msgr2=0x7f291006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f2916312640 1 --2- 172.21.15.53:0/2990424057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100b52a0 0x7f291006f110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f291006f010 tx=0x7f290402c7e0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f2916312640 1 -- 172.21.15.53:0/2990424057 shutdown_connections 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f2916312640 1 --2- 172.21.15.53:0/2990424057 >> v2:172.21.15.135:3300/0 conn(0x7f29100730b0 0x7f2910073520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f2916312640 1 --2- 172.21.15.53:0/2990424057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100b52a0 0x7f291006f110 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.312+0000 7f2916312640 1 -- 172.21.15.53:0/2990424057 >> 172.21.15.53:0/2990424057 conn(0x7f29100d1ed0 msgr2=0x7f29100d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.313+0000 7f2916312640 1 -- 172.21.15.53:0/2990424057 shutdown_connections 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.313+0000 7f2916312640 1 -- 172.21.15.53:0/2990424057 wait complete. 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.314+0000 7f2916312640 1 Processor -- start 2024-05-10T10:35:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.314+0000 7f2916312640 1 -- start start 2024-05-10T10:35:40.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f2916312640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100730b0 0x7f291014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:40.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f2916312640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29100b52a0 0x7f291015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:40.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f2916312640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f291015d600 0x7f2910154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:40.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f2916312640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29100b01e0 con 0x7f29100730b0 2024-05-10T10:35:40.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f2916312640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29100aff00 con 0x7f291015d600 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f2916312640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29100b04c0 con 0x7f29100b52a0 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100730b0 0x7f291014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100730b0 0x7f291014c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47932/0 (socket says 172.21.15.53:47932) 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 -- 172.21.15.53:0/2291026017 learned_addr learned my addr 172.21.15.53:0/2291026017 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290f7fe640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29100b52a0 0x7f291015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f29148af640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f291015d600 0x7f2910154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 -- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29100b52a0 msgr2=0x7f291015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29100b52a0 0x7f291015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 -- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f291015d600 msgr2=0x7f2910154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f291015d600 0x7f2910154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 -- 172.21.15.53:0/2291026017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2910158d00 con 0x7f29100730b0 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290ffff640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100730b0 0x7f291014c110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f290000c400 tx=0x7f290000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.315+0000 7f290cff9640 1 -- 172.21.15.53:0/2291026017 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900017070 con 0x7f29100730b0 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.316+0000 7f290cff9640 1 -- 172.21.15.53:0/2291026017 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2900005ce0 con 0x7f29100730b0 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.316+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2910158fe0 con 0x7f29100730b0 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.316+0000 7f290cff9640 1 -- 172.21.15.53:0/2291026017 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000072f0 con 0x7f29100730b0 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.316+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29101595a0 con 0x7f29100730b0 2024-05-10T10:35:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.317+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2910069b00 con 0x7f29100730b0 2024-05-10T10:35:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.317+0000 7f290cff9640 1 -- 172.21.15.53:0/2291026017 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f290001a070 con 0x7f29100730b0 2024-05-10T10:35:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.318+0000 7f290cff9640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28fc0762d0 0x7f28fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.318+0000 7f290cff9640 1 -- 172.21.15.53:0/2291026017 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6245+0+0 (secure 0 0 0) 0x7f2900096b80 con 0x7f29100730b0 2024-05-10T10:35:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.318+0000 7f290f7fe640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28fc0762d0 0x7f28fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.319+0000 7f290f7fe640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28fc0762d0 0x7f28fc078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2904036000 tx=0x7f2904031000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:40.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.323+0000 7f290cff9640 1 -- 172.21.15.53:0/2291026017 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2910069b00 con 0x7f29100730b0 2024-05-10T10:35:40.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:40.554+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f29100693f0 con 0x7f29100730b0 2024-05-10T10:35:41.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.189+0000 7f290cff9640 1 -- 172.21.15.53:0/2291026017 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f290005fc70 con 0x7f29100730b0 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28fc0762d0 msgr2=0x7f28fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28fc0762d0 0x7f28fc078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2904036000 tx=0x7f2904031000 comp rx=0 tx=0).stop 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100730b0 msgr2=0x7f291014c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100730b0 0x7f291014c110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f290000c400 tx=0x7f290000c8c0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 shutdown_connections 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f28fc0762d0 0x7f28fc078790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f291015d600 0x7f2910154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29100b52a0 0x7f291015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 --2- 172.21.15.53:0/2291026017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29100730b0 0x7f291014c110 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 >> 172.21.15.53:0/2291026017 conn(0x7f29100d1ed0 msgr2=0x7f291006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 shutdown_connections 2024-05-10T10:35:41.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.192+0000 7f2916312640 1 -- 172.21.15.53:0/2291026017 wait complete. 2024-05-10T10:35:41.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:35:41.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.363+0000 7f63e68a7640 1 Processor -- start 2024-05-10T10:35:41.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.363+0000 7f63e68a7640 1 -- start start 2024-05-10T10:35:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e68a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00d3610 0x7f63e00d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e68a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f63e00b5750 0x7f63e00b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e68a7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e00b46b0 con 0x7f63e015f600 2024-05-10T10:35:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e68a7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f63e00d3fb0 con 0x7f63e00b5750 2024-05-10T10:35:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e68a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f63e00b4990 con 0x7f63e00d3610 2024-05-10T10:35:41.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e4e44640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f63e00b5750 0x7f63e00b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e4e44640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f63e00b5750 0x7f63e00b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47940/0 (socket says 172.21.15.53:47940) 2024-05-10T10:35:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e4e44640 1 -- 172.21.15.53:0/195697593 learned_addr learned my addr 172.21.15.53:0/195697593 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63df7fe640 1 --2- 172.21.15.53:0/195697593 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00d3610 0x7f63e00d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e4e44640 1 -- 172.21.15.53:0/195697593 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00d3610 msgr2=0x7f63e00d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e4e44640 1 --2- 172.21.15.53:0/195697593 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00d3610 0x7f63e00d3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e4e44640 1 -- 172.21.15.53:0/195697593 >> v1:172.21.15.135:6789/0 conn(0x7f63e015f600 legacy=0x7f63e00d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:41.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.364+0000 7f63e4e44640 1 -- 172.21.15.53:0/195697593 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e00b4c70 con 0x7f63e00b5750 2024-05-10T10:35:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.365+0000 7f63e4e44640 1 --2- 172.21.15.53:0/195697593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00b5750 0x7f63e00b5be0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009e50 tx=0x7f63d802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2871b2ef5f6ef484 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.365+0000 7f63deffd640 1 -- 172.21.15.53:0/195697593 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8036070 con 0x7f63e00b5750 2024-05-10T10:35:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.365+0000 7f63deffd640 1 -- 172.21.15.53:0/195697593 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d8031600 con 0x7f63e00b5750 2024-05-10T10:35:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.365+0000 7f63deffd640 1 -- 172.21.15.53:0/195697593 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80319d0 con 0x7f63e00b5750 2024-05-10T10:35:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 -- 172.21.15.53:0/195697593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00b5750 msgr2=0x7f63e00b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:41.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 --2- 172.21.15.53:0/195697593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00b5750 0x7f63e00b5be0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009e50 tx=0x7f63d802a630 comp rx=0 tx=0).stop 2024-05-10T10:35:41.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 -- 172.21.15.53:0/195697593 shutdown_connections 2024-05-10T10:35:41.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 --2- 172.21.15.53:0/195697593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00b5750 0x7f63e00b5be0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 --2- 172.21.15.53:0/195697593 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00d3610 0x7f63e00d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 -- 172.21.15.53:0/195697593 >> 172.21.15.53:0/195697593 conn(0x7f63e009e390 msgr2=0x7f63e00d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:41.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 -- 172.21.15.53:0/195697593 shutdown_connections 2024-05-10T10:35:41.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.366+0000 7f63e68a7640 1 -- 172.21.15.53:0/195697593 wait complete. 2024-05-10T10:35:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.367+0000 7f63e68a7640 1 Processor -- start 2024-05-10T10:35:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.367+0000 7f63e68a7640 1 -- start start 2024-05-10T10:35:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63e68a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00b5750 0x7f63e00b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63e68a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00d3610 0x7f63e01620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63e68a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63e015f600 0x7f63e01590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63e68a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00d0d60 con 0x7f63e00d3610 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63e68a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00b5230 con 0x7f63e015f600 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63e68a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f63e00d0ee0 con 0x7f63e00b5750 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63df7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00d3610 0x7f63e01620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63df7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00d3610 0x7f63e01620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47954/0 (socket says 172.21.15.53:47954) 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.368+0000 7f63df7fe640 1 -- 172.21.15.53:0/2286741481 learned_addr learned my addr 172.21.15.53:0/2286741481 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63e4e44640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63e015f600 0x7f63e01590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63dffff640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00b5750 0x7f63e00b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63df7fe640 1 -- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00b5750 msgr2=0x7f63e00b9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63df7fe640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00b5750 0x7f63e00b9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63df7fe640 1 -- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63e015f600 msgr2=0x7f63e01590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63df7fe640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63e015f600 0x7f63e01590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63df7fe640 1 -- 172.21.15.53:0/2286741481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e01f99b0 con 0x7f63e00d3610 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63e4e44640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63e015f600 0x7f63e01590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63df7fe640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00d3610 0x7f63e01620c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f63d400d5f0 tx=0x7f63d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:41.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63dcff9640 1 -- 172.21.15.53:0/2286741481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d40125f0 con 0x7f63e00d3610 2024-05-10T10:35:41.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63dcff9640 1 -- 172.21.15.53:0/2286741481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d4019070 con 0x7f63e00d3610 2024-05-10T10:35:41.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e00b9480 con 0x7f63e00d3610 2024-05-10T10:35:41.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63dcff9640 1 -- 172.21.15.53:0/2286741481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d40129d0 con 0x7f63e00d3610 2024-05-10T10:35:41.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.369+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e00b9a40 con 0x7f63e00d3610 2024-05-10T10:35:41.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.370+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e006a220 con 0x7f63e00d3610 2024-05-10T10:35:41.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.371+0000 7f63dcff9640 1 -- 172.21.15.53:0/2286741481 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f63d4012b70 con 0x7f63e00d3610 2024-05-10T10:35:41.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.371+0000 7f63dcff9640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f63cc0762d0 0x7f63cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:41.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.372+0000 7f63dcff9640 1 -- 172.21.15.53:0/2286741481 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6245+0+0 (secure 0 0 0) 0x7f63d4097bc0 con 0x7f63e00d3610 2024-05-10T10:35:41.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.372+0000 7f63dffff640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f63cc0762d0 0x7f63cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:41.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.372+0000 7f63dffff640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f63cc0762d0 0x7f63cc078790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f63d0004ea0 tx=0x7f63d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:41.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.376+0000 7f63dcff9640 1 -- 172.21.15.53:0/2286741481 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f63d4061390 con 0x7f63e00d3610 2024-05-10T10:35:41.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:41.568+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63e00b4220 con 0x7f63e00d3610 2024-05-10T10:35:42.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.311+0000 7f63dcff9640 1 -- 172.21.15.53:0/2286741481 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) ==== 181+0+0 (secure 0 0 0) 0x7f63d4060c90 con 0x7f63e00d3610 2024-05-10T10:35:42.311 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f63cc0762d0 msgr2=0x7f63cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f63cc0762d0 0x7f63cc078790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f63d0004ea0 tx=0x7f63d0009040 comp rx=0 tx=0).stop 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00d3610 msgr2=0x7f63e01620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00d3610 0x7f63e01620c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f63d400d5f0 tx=0x7f63d400dab0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 shutdown_connections 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f63cc0762d0 0x7f63cc078790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63e015f600 0x7f63e01590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e00d3610 0x7f63e01620c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 --2- 172.21.15.53:0/2286741481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63e00b5750 0x7f63e00b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 >> 172.21.15.53:0/2286741481 conn(0x7f63e009e390 msgr2=0x7f63e00b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:42.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 shutdown_connections 2024-05-10T10:35:42.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.314+0000 7f63e68a7640 1 -- 172.21.15.53:0/2286741481 wait complete. 2024-05-10T10:35:42.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:35:42.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.441+0000 7f674fff0640 1 Processor -- start 2024-05-10T10:35:42.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.441+0000 7f674fff0640 1 -- start start 2024-05-10T10:35:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674fff0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67480d1540 0x7f674815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674fff0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674815fa80 con 0x7f6748072ef0 2024-05-10T10:35:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674fff0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674815fd60 con 0x7f6748159920 2024-05-10T10:35:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674fff0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6748160040 con 0x7f67480d1540 2024-05-10T10:35:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674d58b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6748072ef0 0x7f67480d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49724/0 (socket says 172.21.15.53:49724) 2024-05-10T10:35:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674d58b640 1 -- 172.21.15.53:0/3045949839 learned_addr learned my addr 172.21.15.53:0/3045949839 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674e58d640 1 --2- 172.21.15.53:0/3045949839 >> v2:172.21.15.53:3300/0 conn(0x7f67480d1540 0x7f674815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674cd8a640 1 -- 172.21.15.53:0/3045949839 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400708154 0 0) 0x7f674815fa80 con 0x7f6748072ef0 2024-05-10T10:35:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.442+0000 7f674cd8a640 1 -- 172.21.15.53:0/3045949839 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6730003520 con 0x7f6748072ef0 2024-05-10T10:35:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.443+0000 7f674e58d640 1 -- 172.21.15.53:0/3045949839 >> v1:172.21.15.135:6789/0 conn(0x7f6748159920 legacy=0x7f67480b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.443+0000 7f674e58d640 1 -- 172.21.15.53:0/3045949839 >> v1:172.21.15.53:6789/0 conn(0x7f6748072ef0 legacy=0x7f67480d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.443+0000 7f674e58d640 1 -- 172.21.15.53:0/3045949839 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6748160320 con 0x7f67480d1540 2024-05-10T10:35:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.443+0000 7f674e58d640 1 --2- 172.21.15.53:0/3045949839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67480d1540 0x7f674815c7e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f67400097d0 tx=0x7f674002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7bc14d9bb344f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.443+0000 7f674cd8a640 1 -- 172.21.15.53:0/3045949839 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740037070 con 0x7f67480d1540 2024-05-10T10:35:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674cd8a640 1 -- 172.21.15.53:0/3045949839 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6740032440 con 0x7f67480d1540 2024-05-10T10:35:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674cd8a640 1 -- 172.21.15.53:0/3045949839 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67400327f0 con 0x7f67480d1540 2024-05-10T10:35:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674fff0640 1 -- 172.21.15.53:0/3045949839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67480d1540 msgr2=0x7f674815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:42.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674fff0640 1 --2- 172.21.15.53:0/3045949839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67480d1540 0x7f674815c7e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f67400097d0 tx=0x7f674002a1c0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674fff0640 1 -- 172.21.15.53:0/3045949839 shutdown_connections 2024-05-10T10:35:42.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674fff0640 1 --2- 172.21.15.53:0/3045949839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67480d1540 0x7f674815c7e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674fff0640 1 -- 172.21.15.53:0/3045949839 >> 172.21.15.53:0/3045949839 conn(0x7f674806c4a0 msgr2=0x7f674806c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:42.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674fff0640 1 -- 172.21.15.53:0/3045949839 shutdown_connections 2024-05-10T10:35:42.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.444+0000 7f674fff0640 1 -- 172.21.15.53:0/3045949839 wait complete. 2024-05-10T10:35:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.445+0000 7f674fff0640 1 Processor -- start 2024-05-10T10:35:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.445+0000 7f674fff0640 1 -- start start 2024-05-10T10:35:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674fff0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6748072ef0 0x7f674815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674fff0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67480d1540 0x7f67480b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674fff0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6748159920 0x7f674809dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674fff0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67480b3db0 con 0x7f6748159920 2024-05-10T10:35:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674fff0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67481608e0 con 0x7f67480d1540 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674fff0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6748159d00 con 0x7f6748072ef0 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674e58d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6748159920 0x7f674809dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674e58d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6748159920 0x7f674809dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47962/0 (socket says 172.21.15.53:47962) 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674e58d640 1 -- 172.21.15.53:0/4147760228 learned_addr learned my addr 172.21.15.53:0/4147760228 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674dd8c640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6748072ef0 0x7f674815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.446+0000 7f674d58b640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67480d1540 0x7f67480b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674e58d640 1 -- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6748072ef0 msgr2=0x7f674815c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674e58d640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6748072ef0 0x7f674815c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674e58d640 1 -- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67480d1540 msgr2=0x7f67480b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674e58d640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67480d1540 0x7f67480b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674e58d640 1 -- 172.21.15.53:0/4147760228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67480ce4a0 con 0x7f6748159920 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674e58d640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6748159920 0x7f674809dc40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f67400097d0 tx=0x7f67400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f67467fc640 1 -- 172.21.15.53:0/4147760228 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740037050 con 0x7f6748159920 2024-05-10T10:35:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f67467fc640 1 -- 172.21.15.53:0/4147760228 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6740038040 con 0x7f6748159920 2024-05-10T10:35:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67480ce780 con 0x7f6748159920 2024-05-10T10:35:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f67467fc640 1 -- 172.21.15.53:0/4147760228 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740004450 con 0x7f6748159920 2024-05-10T10:35:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.447+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67480ced40 con 0x7f6748159920 2024-05-10T10:35:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.448+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674806a220 con 0x7f6748159920 2024-05-10T10:35:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.449+0000 7f67467fc640 1 -- 172.21.15.53:0/4147760228 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f674003c6d0 con 0x7f6748159920 2024-05-10T10:35:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.449+0000 7f67467fc640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67340762d0 0x7f6734078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.449+0000 7f67467fc640 1 -- 172.21.15.53:0/4147760228 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5807+0+0 (secure 0 0 0) 0x7f67400b7f80 con 0x7f6748159920 2024-05-10T10:35:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.450+0000 7f674dd8c640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67340762d0 0x7f6734078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.450+0000 7f674dd8c640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67340762d0 0x7f6734078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6738001ca0 tx=0x7f6738009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:42.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.453+0000 7f67467fc640 1 -- 172.21.15.53:0/4147760228 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6740081910 con 0x7f6748159920 2024-05-10T10:35:42.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:42.647+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f674806cb00 con 0x7f6748159920 2024-05-10T10:35:43.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.312+0000 7f67467fc640 1 -- 172.21.15.53:0/4147760228 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) ==== 169+0+0 (secure 0 0 0) 0x7f6740081210 con 0x7f6748159920 2024-05-10T10:35:43.313 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67340762d0 msgr2=0x7f6734078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67340762d0 0x7f6734078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6738001ca0 tx=0x7f6738009040 comp rx=0 tx=0).stop 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6748159920 msgr2=0x7f674809dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6748159920 0x7f674809dc40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f67400097d0 tx=0x7f67400046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 shutdown_connections 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67340762d0 0x7f6734078790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6748159920 0x7f674809dc40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67480d1540 0x7f67480b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 --2- 172.21.15.53:0/4147760228 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6748072ef0 0x7f674815c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 >> 172.21.15.53:0/4147760228 conn(0x7f674806c4a0 msgr2=0x7f67480af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:43.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 shutdown_connections 2024-05-10T10:35:43.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.316+0000 7f674fff0640 1 -- 172.21.15.53:0/4147760228 wait complete. 2024-05-10T10:35:43.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T10:35:43.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.440+0000 7f523c6da640 1 Processor -- start 2024-05-10T10:35:43.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.440+0000 7f523c6da640 1 -- start start 2024-05-10T10:35:43.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523c6da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52340d3df0 0x7f52340d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523c6da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f52340d2120 0x7f52340d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523c6da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52340d41d0 con 0x7f52340b4aa0 2024-05-10T10:35:43.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523c6da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f52340cfd50 con 0x7f52340d3df0 2024-05-10T10:35:43.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523c6da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f52340b5d90 con 0x7f52340d2120 2024-05-10T10:35:43.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523a476640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52340d3df0 0x7f52340d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523a476640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52340d3df0 0x7f52340d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50188/0 (socket says 172.21.15.53:50188) 2024-05-10T10:35:43.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523a476640 1 -- 172.21.15.53:0/2553482543 learned_addr learned my addr 172.21.15.53:0/2553482543 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:43.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523ac77640 1 --2- 172.21.15.53:0/2553482543 >> v2:172.21.15.135:3300/0 conn(0x7f52340d2120 0x7f52340d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523a476640 1 -- 172.21.15.53:0/2553482543 >> v2:172.21.15.135:3300/0 conn(0x7f52340d2120 msgr2=0x7f52340d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:43.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523a476640 1 --2- 172.21.15.53:0/2553482543 >> v2:172.21.15.135:3300/0 conn(0x7f52340d2120 0x7f52340d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523a476640 1 -- 172.21.15.53:0/2553482543 >> v1:172.21.15.135:6789/0 conn(0x7f52340b4aa0 legacy=0x7f52340d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:43.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.441+0000 7f523a476640 1 -- 172.21.15.53:0/2553482543 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52340b54f0 con 0x7f52340d3df0 2024-05-10T10:35:43.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.442+0000 7f523a476640 1 --2- 172.21.15.53:0/2553482543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340d3df0 0x7f52340d2d10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f52340d2ca0 tx=0x7f523002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcde9337bdafa93c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.442+0000 7f5239474640 1 -- 172.21.15.53:0/2553482543 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230037070 con 0x7f52340d3df0 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.442+0000 7f5239474640 1 -- 172.21.15.53:0/2553482543 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5230032440 con 0x7f52340d3df0 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.442+0000 7f5239474640 1 -- 172.21.15.53:0/2553482543 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230032810 con 0x7f52340d3df0 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 -- 172.21.15.53:0/2553482543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340d3df0 msgr2=0x7f52340d2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 --2- 172.21.15.53:0/2553482543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340d3df0 0x7f52340d2d10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f52340d2ca0 tx=0x7f523002a230 comp rx=0 tx=0).stop 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 -- 172.21.15.53:0/2553482543 shutdown_connections 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 --2- 172.21.15.53:0/2553482543 >> v2:172.21.15.135:3300/0 conn(0x7f52340d2120 0x7f52340d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 --2- 172.21.15.53:0/2553482543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340d3df0 0x7f52340d2d10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 -- 172.21.15.53:0/2553482543 >> 172.21.15.53:0/2553482543 conn(0x7f523409e450 msgr2=0x7f52340d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 -- 172.21.15.53:0/2553482543 shutdown_connections 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.443+0000 7f523c6da640 1 -- 172.21.15.53:0/2553482543 wait complete. 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.444+0000 7f523c6da640 1 Processor -- start 2024-05-10T10:35:43.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.444+0000 7f523c6da640 1 -- start start 2024-05-10T10:35:43.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523c6da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340b4aa0 0x7f52340b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523c6da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52340d2120 0x7f52340c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523c6da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52340d3df0 0x7f523415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523c6da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52340cff80 con 0x7f52340b4aa0 2024-05-10T10:35:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523c6da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52340b5ab0 con 0x7f52340d2120 2024-05-10T10:35:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523c6da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f52340b6120 con 0x7f52340d3df0 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340b4aa0 0x7f52340b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340b4aa0 0x7f52340b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50196/0 (socket says 172.21.15.53:50196) 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 -- 172.21.15.53:0/2242903388 learned_addr learned my addr 172.21.15.53:0/2242903388 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f5239c75640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52340d2120 0x7f52340c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523ac77640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52340d3df0 0x7f523415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 -- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52340d3df0 msgr2=0x7f523415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52340d3df0 0x7f523415ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 -- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52340d2120 msgr2=0x7f52340c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52340d2120 0x7f52340c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 -- 172.21.15.53:0/2242903388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52340c9380 con 0x7f52340b4aa0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523a476640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340b4aa0 0x7f52340b3510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f523002a270 tx=0x7f52300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f522affd640 1 -- 172.21.15.53:0/2242903388 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230041020 con 0x7f52340b4aa0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f522affd640 1 -- 172.21.15.53:0/2242903388 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52300306c0 con 0x7f52340b4aa0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52340c9660 con 0x7f52340b4aa0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.445+0000 7f522affd640 1 -- 172.21.15.53:0/2242903388 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230037040 con 0x7f52340b4aa0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.446+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52340c9c20 con 0x7f52340b4aa0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.447+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523406a220 con 0x7f52340b4aa0 2024-05-10T10:35:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.447+0000 7f522affd640 1 -- 172.21.15.53:0/2242903388 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5230044020 con 0x7f52340b4aa0 2024-05-10T10:35:43.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.447+0000 7f522affd640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52180762d0 0x7f5218078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.448+0000 7f522affd640 1 -- 172.21.15.53:0/2242903388 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f52300b7d30 con 0x7f52340b4aa0 2024-05-10T10:35:43.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.448+0000 7f5239c75640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52180762d0 0x7f5218078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.448+0000 7f5239c75640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52180762d0 0x7f5218078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f52340b3e20 tx=0x7f5224009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.452+0000 7f522affd640 1 -- 172.21.15.53:0/2242903388 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f523004b030 con 0x7f52340b4aa0 2024-05-10T10:35:43.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.641+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f52340b9160 con 0x7f52340b4aa0 2024-05-10T10:35:43.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.644+0000 7f522affd640 1 -- 172.21.15.53:0/2242903388 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f5230081760 con 0x7f52340b4aa0 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52180762d0 msgr2=0x7f5218078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52180762d0 0x7f5218078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f52340b3e20 tx=0x7f5224009040 comp rx=0 tx=0).stop 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340b4aa0 msgr2=0x7f52340b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340b4aa0 0x7f52340b3510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f523002a270 tx=0x7f52300046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 shutdown_connections 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f52180762d0 0x7f5218078790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52340d3df0 0x7f523415ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52340d2120 0x7f52340c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 --2- 172.21.15.53:0/2242903388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52340b4aa0 0x7f52340b3510 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 >> 172.21.15.53:0/2242903388 conn(0x7f523409e450 msgr2=0x7f52340d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 shutdown_connections 2024-05-10T10:35:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.646+0000 7f523c6da640 1 -- 172.21.15.53:0/2242903388 wait complete. 2024-05-10T10:35:43.658 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-10T10:35:43.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.761+0000 7fd5831da640 1 Processor -- start 2024-05-10T10:35:43.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.761+0000 7fd5831da640 1 -- start start 2024-05-10T10:35:43.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd5831da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd57c15d650 0x7fd57c0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd5831da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd57c0b43a0 0x7fd57c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd5831da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57c0ce590 con 0x7fd57c0d3980 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd5831da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c15da30 con 0x7fd57c15d650 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd5831da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c0d0500 con 0x7fd57c0b43a0 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd580f76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd57c15d650 0x7fd57c0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd57bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd57c0d3980 0x7fd57c0cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46286/0 (socket says 172.21.15.53:46286) 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd57bfff640 1 -- 172.21.15.53:0/828630003 learned_addr learned my addr 172.21.15.53:0/828630003 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.762+0000 7fd581777640 1 --2- 172.21.15.53:0/828630003 >> v2:172.21.15.135:3300/0 conn(0x7fd57c0b43a0 0x7fd57c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.763+0000 7fd580f76640 1 -- 172.21.15.53:0/828630003 >> v2:172.21.15.135:3300/0 conn(0x7fd57c0b43a0 msgr2=0x7fd57c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.763+0000 7fd580f76640 1 --2- 172.21.15.53:0/828630003 >> v2:172.21.15.135:3300/0 conn(0x7fd57c0b43a0 0x7fd57c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.763+0000 7fd580f76640 1 -- 172.21.15.53:0/828630003 >> v1:172.21.15.53:6789/0 conn(0x7fd57c0d3980 legacy=0x7fd57c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.763+0000 7fd580f76640 1 -- 172.21.15.53:0/828630003 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c0ce810 con 0x7fd57c15d650 2024-05-10T10:35:43.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.763+0000 7fd57b7fe640 1 -- 172.21.15.53:0/828630003 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464943056 0 0) 0x7fd56c001490 con 0x7fd57c0d3980 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.763+0000 7fd580f76640 1 --2- 172.21.15.53:0/828630003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c15d650 0x7fd57c0cf540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06d200 tx=0x7fd57402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c383acd0e599d82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd57b7fe640 1 -- 172.21.15.53:0/828630003 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574037070 con 0x7fd57c15d650 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd57b7fe640 1 -- 172.21.15.53:0/828630003 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd574032440 con 0x7fd57c15d650 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd57b7fe640 1 -- 172.21.15.53:0/828630003 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574032810 con 0x7fd57c15d650 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 -- 172.21.15.53:0/828630003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c15d650 msgr2=0x7fd57c0cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 --2- 172.21.15.53:0/828630003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c15d650 0x7fd57c0cf540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06d200 tx=0x7fd57402a1a0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 -- 172.21.15.53:0/828630003 shutdown_connections 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 --2- 172.21.15.53:0/828630003 >> v2:172.21.15.135:3300/0 conn(0x7fd57c0b43a0 0x7fd57c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 --2- 172.21.15.53:0/828630003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c15d650 0x7fd57c0cf540 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 -- 172.21.15.53:0/828630003 >> 172.21.15.53:0/828630003 conn(0x7fd57c0d2df0 msgr2=0x7fd57c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 -- 172.21.15.53:0/828630003 shutdown_connections 2024-05-10T10:35:43.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.764+0000 7fd5831da640 1 -- 172.21.15.53:0/828630003 wait complete. 2024-05-10T10:35:43.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.765+0000 7fd5831da640 1 Processor -- start 2024-05-10T10:35:43.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.765+0000 7fd5831da640 1 -- start start 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd5831da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd57c0b43a0 0x7fd57c0b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd5831da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c0d3980 0x7fd57c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd5831da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd57c15d650 0x7fd57c0c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd5831da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c0d0190 con 0x7fd57c0d3980 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd5831da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c0cedd0 con 0x7fd57c15d650 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd5831da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c0b26a0 con 0x7fd57c0b43a0 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd57bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c0d3980 0x7fd57c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd57bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c0d3980 0x7fd57c1592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50216/0 (socket says 172.21.15.53:50216) 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.766+0000 7fd57bfff640 1 -- 172.21.15.53:0/2331367967 learned_addr learned my addr 172.21.15.53:0/2331367967 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd57bfff640 1 -- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd57c0b43a0 msgr2=0x7fd57c0b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd580f76640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd57c0b43a0 0x7fd57c0b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd581777640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd57c15d650 0x7fd57c0c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd57bfff640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd57c0b43a0 0x7fd57c0b7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd57bfff640 1 -- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd57c15d650 msgr2=0x7fd57c0c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd57bfff640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd57c15d650 0x7fd57c0c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd57bfff640 1 -- 172.21.15.53:0/2331367967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c0b5d90 con 0x7fd57c0d3980 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd57bfff640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c0d3980 0x7fd57c1592d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00c850 tx=0x7fd56c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd5797fa640 1 -- 172.21.15.53:0/2331367967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c018070 con 0x7fd57c0d3980 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd5797fa640 1 -- 172.21.15.53:0/2331367967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c00a640 con 0x7fd57c0d3980 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c0b8240 con 0x7fd57c0d3980 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd5797fa640 1 -- 172.21.15.53:0/2331367967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00aa70 con 0x7fd57c0d3980 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.767+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c0b8aa0 con 0x7fd57c0d3980 2024-05-10T10:35:43.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.768+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c06a220 con 0x7fd57c0d3980 2024-05-10T10:35:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.769+0000 7fd5797fa640 1 -- 172.21.15.53:0/2331367967 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd56c01b070 con 0x7fd57c0d3980 2024-05-10T10:35:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.769+0000 7fd5797fa640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd55c076300 0x7fd55c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.769+0000 7fd5797fa640 1 -- 172.21.15.53:0/2331367967 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7fd56c096ac0 con 0x7fd57c0d3980 2024-05-10T10:35:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.770+0000 7fd580f76640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd55c076300 0x7fd55c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:43.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.770+0000 7fd580f76640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd55c076300 0x7fd55c0787c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd574034000 tx=0x7fd57402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:43.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:43.774+0000 7fd5797fa640 1 -- 172.21.15.53:0/2331367967 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd57c0b5d90 con 0x7fd57c0d3980 2024-05-10T10:35:44.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:35:44.024 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:35:44.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.048+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fd57c0b49b0 con 0x7fd57c0d3980 2024-05-10T10:35:44.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.323+0000 7fd5797fa640 1 -- 172.21.15.53:0/2331367967 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v126) ==== 149+0+0 (secure 0 0 0) 0x7fd57c0ce590 con 0x7fd57c0d3980 2024-05-10T10:35:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd55c076300 msgr2=0x7fd55c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd55c076300 0x7fd55c0787c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd574034000 tx=0x7fd57402e000 comp rx=0 tx=0).stop 2024-05-10T10:35:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c0d3980 msgr2=0x7fd57c1592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c0d3980 0x7fd57c1592d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00c850 tx=0x7fd56c00cd10 comp rx=0 tx=0).stop 2024-05-10T10:35:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 shutdown_connections 2024-05-10T10:35:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd55c076300 0x7fd55c0787c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd57c15d650 0x7fd57c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd57c0d3980 0x7fd57c1592d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 --2- 172.21.15.53:0/2331367967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd57c0b43a0 0x7fd57c0b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 >> 172.21.15.53:0/2331367967 conn(0x7fd57c0d2df0 msgr2=0x7fd57c070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 shutdown_connections 2024-05-10T10:35:44.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.325+0000 7fd5831da640 1 -- 172.21.15.53:0/2331367967 wait complete. 2024-05-10T10:35:44.338 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-05-10T10:35:44.338 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-10T10:35:44.338 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-10T10:35:44.338 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:35:44.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.442+0000 7f1227dc8640 1 Processor -- start 2024-05-10T10:35:44.443 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.442+0000 7f1227dc8640 1 -- start start 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1227dc8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12200d5cb0 0x7f12200d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1227dc8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1220162b30 0x7f12200d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1227dc8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12200b45f0 con 0x7f1220162750 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1227dc8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f12200b4f20 con 0x7f12200d5cb0 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1227dc8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f12200d0fe0 con 0x7f1220162b30 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1225363640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12200d5cb0 0x7f12200d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1225363640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12200d5cb0 0x7f12200d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50220/0 (socket says 172.21.15.53:50220) 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1225363640 1 -- 172.21.15.53:0/4110216053 learned_addr learned my addr 172.21.15.53:0/4110216053 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.443+0000 7f1226365640 1 --2- 172.21.15.53:0/4110216053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1220162b30 0x7f12200d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.444+0000 7f1225363640 1 -- 172.21.15.53:0/4110216053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1220162b30 msgr2=0x7f12200d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.444+0000 7f1225363640 1 --2- 172.21.15.53:0/4110216053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1220162b30 0x7f12200d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.444+0000 7f1225363640 1 -- 172.21.15.53:0/4110216053 >> v1:172.21.15.135:6789/0 conn(0x7f1220162750 legacy=0x7f12200d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:44.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.444+0000 7f1225363640 1 -- 172.21.15.53:0/4110216053 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200b48d0 con 0x7f12200d5cb0 2024-05-10T10:35:44.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.444+0000 7f1226365640 1 --2- 172.21.15.53:0/4110216053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1220162b30 0x7f12200d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:44.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.444+0000 7f1225363640 1 --2- 172.21.15.53:0/4110216053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200d5cb0 0x7f12200d3630 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1214009870 tx=0x7f121402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1835fc081f938e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:44.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.444+0000 7f1224b62640 1 -- 172.21.15.53:0/4110216053 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214037070 con 0x7f12200d5cb0 2024-05-10T10:35:44.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1224b62640 1 -- 172.21.15.53:0/4110216053 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1214032440 con 0x7f12200d5cb0 2024-05-10T10:35:44.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1224b62640 1 -- 172.21.15.53:0/4110216053 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214032810 con 0x7f12200d5cb0 2024-05-10T10:35:44.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 -- 172.21.15.53:0/4110216053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200d5cb0 msgr2=0x7f12200d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4110216053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200d5cb0 0x7f12200d3630 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1214009870 tx=0x7f121402a180 comp rx=0 tx=0).stop 2024-05-10T10:35:44.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 -- 172.21.15.53:0/4110216053 shutdown_connections 2024-05-10T10:35:44.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4110216053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1220162b30 0x7f12200d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4110216053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200d5cb0 0x7f12200d3630 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 -- 172.21.15.53:0/4110216053 >> 172.21.15.53:0/4110216053 conn(0x7f122009e3b0 msgr2=0x7f12200d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:44.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 -- 172.21.15.53:0/4110216053 shutdown_connections 2024-05-10T10:35:44.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.445+0000 7f1227dc8640 1 -- 172.21.15.53:0/4110216053 wait complete. 2024-05-10T10:35:44.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.446+0000 7f1227dc8640 1 Processor -- start 2024-05-10T10:35:44.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.446+0000 7f1227dc8640 1 -- start start 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1227dc8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220162750 0x7f12200ba780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1227dc8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1220162b30 0x7f122015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1227dc8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f122015a9f0 0x7f12200ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1227dc8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f12200b53b0 con 0x7f1220162750 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1227dc8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12200d0d80 con 0x7f1220162b30 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1227dc8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f12200b5530 con 0x7f122015a9f0 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1225b64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220162750 0x7f12200ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1225b64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220162750 0x7f12200ba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50234/0 (socket says 172.21.15.53:50234) 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1225b64640 1 -- 172.21.15.53:0/4286036940 learned_addr learned my addr 172.21.15.53:0/4286036940 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1226365640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f122015a9f0 0x7f12200ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.447+0000 7f1225363640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1220162b30 0x7f122015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1225b64640 1 -- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f122015a9f0 msgr2=0x7f12200ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1225b64640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f122015a9f0 0x7f12200ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1225b64640 1 -- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1220162b30 msgr2=0x7f122015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1225b64640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1220162b30 0x7f122015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1225b64640 1 -- 172.21.15.53:0/4286036940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200cb5c0 con 0x7f1220162750 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1225363640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1220162b30 0x7f122015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1226365640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f122015a9f0 0x7f12200ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1225b64640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220162750 0x7f12200ba780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f121800c430 tx=0x7f121800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f121e7fc640 1 -- 172.21.15.53:0/4286036940 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218011790 con 0x7f1220162750 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f121e7fc640 1 -- 172.21.15.53:0/4286036940 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1218017070 con 0x7f1220162750 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12200cb8a0 con 0x7f1220162750 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f121e7fc640 1 -- 172.21.15.53:0/4286036940 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218011bb0 con 0x7f1220162750 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.448+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12200ba260 con 0x7f1220162750 2024-05-10T10:35:44.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.449+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122006a220 con 0x7f1220162750 2024-05-10T10:35:44.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.449+0000 7f121e7fc640 1 -- 172.21.15.53:0/4286036940 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f121801c460 con 0x7f1220162750 2024-05-10T10:35:44.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.450+0000 7f121e7fc640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f120c0762d0 0x7f120c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.450+0000 7f121e7fc640 1 -- 172.21.15.53:0/4286036940 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f1218097310 con 0x7f1220162750 2024-05-10T10:35:44.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.450+0000 7f1225363640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f120c0762d0 0x7f120c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.451+0000 7f1225363640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f120c0762d0 0x7f120c078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1214034000 tx=0x7f121402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:44.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.454+0000 7f121e7fc640 1 -- 172.21.15.53:0/4286036940 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1218060de0 con 0x7f1220162750 2024-05-10T10:35:44.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.646+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12200cf380 con 0x7f1220162750 2024-05-10T10:35:44.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.647+0000 7f121e7fc640 1 -- 172.21.15.53:0/4286036940 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+13291 (secure 0 0 0) 0x7f12180606e0 con 0x7f1220162750 2024-05-10T10:35:44.648 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:44.648 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:35:43.311602+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/679529875":"2024-05-11T10:35:40.186991+0000","172.21.15.53:6834/679529875":"2024-05-11T10:35:40.186991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:35:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f120c0762d0 msgr2=0x7f120c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f120c0762d0 0x7f120c078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1214034000 tx=0x7f121402e000 comp rx=0 tx=0).stop 2024-05-10T10:35:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220162750 msgr2=0x7f12200ba780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220162750 0x7f12200ba780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f121800c430 tx=0x7f121800c8f0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 shutdown_connections 2024-05-10T10:35:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f120c0762d0 0x7f120c078790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f122015a9f0 0x7f12200ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1220162b30 0x7f122015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 --2- 172.21.15.53:0/4286036940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220162750 0x7f12200ba780 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 >> 172.21.15.53:0/4286036940 conn(0x7f122009e3b0 msgr2=0x7f1220161a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.649+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 shutdown_connections 2024-05-10T10:35:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.650+0000 7f1227dc8640 1 -- 172.21.15.53:0/4286036940 wait complete. 2024-05-10T10:35:44.661 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:33:28.171977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T10:35:44.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:35:44.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.767+0000 7fd1496fc640 1 Processor -- start 2024-05-10T10:35:44.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.767+0000 7fd1496fc640 1 -- start start 2024-05-10T10:35:44.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd1496fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd144162ce0 0x7fd1440b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd1496fc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1440d4190 con 0x7fd1440b4e20 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd1496fc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1440cfc40 con 0x7fd144162900 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd1496fc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd1440d3eb0 con 0x7fd144162ce0 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd1437fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd1440b4e20 0x7fd1440d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46290/0 (socket says 172.21.15.53:46290) 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd1437fe640 1 -- 172.21.15.53:0/2602979964 learned_addr learned my addr 172.21.15.53:0/2602979964 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd1427fc640 1 --2- 172.21.15.53:0/2602979964 >> v2:172.21.15.135:3300/0 conn(0x7fd144162ce0 0x7fd1440b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2583881346 0 0) 0x7fd13c001490 con 0x7fd1440b4e20 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.768+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd12c003520 con 0x7fd1440b4e20 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1810497196 0 0) 0x7fd12c003520 con 0x7fd1440b4e20 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 >> v2:172.21.15.135:3300/0 conn(0x7fd144162ce0 msgr2=0x7fd1440b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 --2- 172.21.15.53:0/2602979964 >> v2:172.21.15.135:3300/0 conn(0x7fd144162ce0 0x7fd1440b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 >> v1:172.21.15.135:6789/0 conn(0x7fd144162900 legacy=0x7fd1440d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1440cfee0 con 0x7fd1440b4e20 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fd13c001bc0 con 0x7fd1440b4e20 2024-05-10T10:35:44.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 >> v1:172.21.15.53:6789/0 conn(0x7fd1440b4e20 legacy=0x7fd1440d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd12c004800 0x7fd12c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd12c0051c0 0x7fd12c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd1440d3eb0 con 0x7fd12c0051c0 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c00c160 con 0x7fd12c004800 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c00c360 con 0x7fd12c0039b0 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1437fe640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd12c0051c0 0x7fd12c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1427fc640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd12c004800 0x7fd12c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd142ffd640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1437fe640 1 -- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0039b0 msgr2=0x7fd12c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1437fe640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1437fe640 1 -- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd12c004800 msgr2=0x7fd12c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1437fe640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd12c004800 0x7fd12c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1437fe640 1 -- 172.21.15.53:0/2602979964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c003520 con 0x7fd12c0051c0 2024-05-10T10:35:44.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.769+0000 7fd1427fc640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd12c004800 0x7fd12c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:44.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.770+0000 7fd1437fe640 1 -- 172.21.15.53:0/2602979964 reap_dead start 2024-05-10T10:35:44.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.770+0000 7fd1437fe640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd12c0051c0 0x7fd12c00b9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00aa30 tx=0x7fd12c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126baedced9bf76e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:44.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.770+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00a650 con 0x7fd12c0051c0 2024-05-10T10:35:44.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.770+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c02f070 con 0x7fd12c0051c0 2024-05-10T10:35:44.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.770+0000 7fd141ffb640 1 -- 172.21.15.53:0/2602979964 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c034470 con 0x7fd12c0051c0 2024-05-10T10:35:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.771+0000 7fd1496fc640 1 -- 172.21.15.53:0/2602979964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd12c0051c0 msgr2=0x7fd12c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.771+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd12c0051c0 0x7fd12c00b9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00aa30 tx=0x7fd12c005830 comp rx=0 tx=0).stop 2024-05-10T10:35:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.771+0000 7fd1496fc640 1 -- 172.21.15.53:0/2602979964 shutdown_connections 2024-05-10T10:35:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.771+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2602979964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd12c0051c0 0x7fd12c00b9c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.771+0000 7fd1496fc640 1 -- 172.21.15.53:0/2602979964 >> 172.21.15.53:0/2602979964 conn(0x7fd14409e430 msgr2=0x7fd1440d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:44.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.771+0000 7fd1496fc640 1 -- 172.21.15.53:0/2602979964 shutdown_connections 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.771+0000 7fd1496fc640 1 -- 172.21.15.53:0/2602979964 wait complete. 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.772+0000 7fd1496fc640 1 Processor -- start 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.772+0000 7fd1496fc640 1 -- start start 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1496fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0051c0 0x7fd1441534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1496fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415d430 0x7fd1440cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1496fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1441f9300 0x7fd144160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1496fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd1440b4300 con 0x7fd1441f9300 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1496fc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd1440b4180 con 0x7fd14415d430 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1496fc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd1440b48e0 con 0x7fd12c0051c0 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1441f9300 0x7fd144160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1441f9300 0x7fd144160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50246/0 (socket says 172.21.15.53:50246) 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 -- 172.21.15.53:0/2835952761 learned_addr learned my addr 172.21.15.53:0/2835952761 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd142ffd640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0051c0 0x7fd1441534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1427fc640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415d430 0x7fd1440cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 -- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0051c0 msgr2=0x7fd1441534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0051c0 0x7fd1441534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 -- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415d430 msgr2=0x7fd1440cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415d430 0x7fd1440cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 -- 172.21.15.53:0/2835952761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1440c8c90 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd142ffd640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0051c0 0x7fd1441534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.773+0000 7fd1437fe640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1441f9300 0x7fd144160220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00ab20 tx=0x7fd1440d00b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.774+0000 7fd12b7fe640 1 -- 172.21.15.53:0/2835952761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00a650 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.774+0000 7fd12b7fe640 1 -- 172.21.15.53:0/2835952761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c001490 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.774+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1440c8f70 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.774+0000 7fd12b7fe640 1 -- 172.21.15.53:0/2835952761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c02be60 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.774+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1441f8de0 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.775+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14406d980 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.775+0000 7fd12b7fe640 1 -- 172.21.15.53:0/2835952761 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd13c034670 con 0x7fd1441f9300 2024-05-10T10:35:44.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.776+0000 7fd12b7fe640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1300762d0 0x7fd130078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:44.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.776+0000 7fd12b7fe640 1 -- 172.21.15.53:0/2835952761 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7fd13c0b73a0 con 0x7fd1441f9300 2024-05-10T10:35:44.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.776+0000 7fd142ffd640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1300762d0 0x7fd130078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:44.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.776+0000 7fd142ffd640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1300762d0 0x7fd130078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd134000c00 tx=0x7fd134009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:44.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.780+0000 7fd12b7fe640 1 -- 172.21.15.53:0/2835952761 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd14406d980 con 0x7fd1441f9300 2024-05-10T10:35:44.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.968+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd14409e840 con 0x7fd1441f9300 2024-05-10T10:35:44.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.969+0000 7fd12b7fe640 1 -- 172.21.15.53:0/2835952761 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7fd1440d4190 con 0x7fd1441f9300 2024-05-10T10:35:44.969 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1300762d0 msgr2=0x7fd130078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1300762d0 0x7fd130078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd134000c00 tx=0x7fd134009040 comp rx=0 tx=0).stop 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1441f9300 msgr2=0x7fd144160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1441f9300 0x7fd144160220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00ab20 tx=0x7fd1440d00b0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 shutdown_connections 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1300762d0 0x7fd130078790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1441f9300 0x7fd144160220 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415d430 0x7fd1440cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 --2- 172.21.15.53:0/2835952761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd12c0051c0 0x7fd1441534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 >> 172.21.15.53:0/2835952761 conn(0x7fd14409e430 msgr2=0x7fd1441612c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 shutdown_connections 2024-05-10T10:35:44.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:44.971+0000 7fd1496fc640 1 -- 172.21.15.53:0/2835952761 wait complete. 2024-05-10T10:35:44.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-10T10:35:45.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.086+0000 7f621e483640 1 Processor -- start 2024-05-10T10:35:45.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.086+0000 7f621e483640 1 -- start start 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621e483640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62180d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621e483640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f62180d5cb0 0x7f62180d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621e483640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6218162b30 0x7f62180d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621e483640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f62180b4f20 con 0x7f6218162b30 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621e483640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f62180d0fe0 con 0x7f62180d5cb0 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621e483640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f62180b45f0 con 0x7f6218162750 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6218162b30 0x7f62180d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6218162b30 0x7f62180d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50256/0 (socket says 172.21.15.53:50256) 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 -- 172.21.15.53:0/2242985111 learned_addr learned my addr 172.21.15.53:0/2242985111 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f62177fe640 1 --2- 172.21.15.53:0/2242985111 >> v2:172.21.15.135:3300/0 conn(0x7f62180d5cb0 0x7f62180d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f6217fff640 1 --2- 172.21.15.53:0/2242985111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62180d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 -- 172.21.15.53:0/2242985111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 msgr2=0x7f62180d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 --2- 172.21.15.53:0/2242985111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62180d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 -- 172.21.15.53:0/2242985111 >> v2:172.21.15.135:3300/0 conn(0x7f62180d5cb0 msgr2=0x7f62180d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 --2- 172.21.15.53:0/2242985111 >> v2:172.21.15.135:3300/0 conn(0x7f62180d5cb0 0x7f62180d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f621ca20640 1 -- 172.21.15.53:0/2242985111 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62180b48d0 con 0x7f6218162b30 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f6217fff640 1 --2- 172.21.15.53:0/2242985111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62180d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.087+0000 7f62177fe640 1 --2- 172.21.15.53:0/2242985111 >> v2:172.21.15.135:3300/0 conn(0x7f62180d5cb0 0x7f62180d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.088+0000 7f621ca20640 1 --2- 172.21.15.53:0/2242985111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180d3b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f621809eae0 tx=0x7f621002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622ed870da4dcef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:45.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.088+0000 7f6216ffd640 1 -- 172.21.15.53:0/2242985111 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210036070 con 0x7f6218162b30 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.088+0000 7f6216ffd640 1 -- 172.21.15.53:0/2242985111 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6210031600 con 0x7f6218162b30 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.088+0000 7f6216ffd640 1 -- 172.21.15.53:0/2242985111 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62100319d0 con 0x7f6218162b30 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 -- 172.21.15.53:0/2242985111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 msgr2=0x7f62180d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 --2- 172.21.15.53:0/2242985111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180d3b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f621809eae0 tx=0x7f621002a610 comp rx=0 tx=0).stop 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 -- 172.21.15.53:0/2242985111 shutdown_connections 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 --2- 172.21.15.53:0/2242985111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180d3b70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 --2- 172.21.15.53:0/2242985111 >> v2:172.21.15.135:3300/0 conn(0x7f62180d5cb0 0x7f62180d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 --2- 172.21.15.53:0/2242985111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62180d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 -- 172.21.15.53:0/2242985111 >> 172.21.15.53:0/2242985111 conn(0x7f621809e3b0 msgr2=0x7f62180d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 -- 172.21.15.53:0/2242985111 shutdown_connections 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.089+0000 7f621e483640 1 -- 172.21.15.53:0/2242985111 wait complete. 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.090+0000 7f621e483640 1 Processor -- start 2024-05-10T10:35:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.090+0000 7f621e483640 1 -- start start 2024-05-10T10:35:45.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f621e483640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62181f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f621e483640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180ba430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f621e483640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62180c93e0 0x7f62180caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f621e483640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62180d0d30 con 0x7f6218162b30 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f621e483640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62180d0bb0 con 0x7f62180c93e0 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f621e483640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f62180d1660 con 0x7f6218162750 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50264/0 (socket says 172.21.15.53:50264) 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 -- 172.21.15.53:0/2071477172 learned_addr learned my addr 172.21.15.53:0/2071477172 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f6217fff640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62181f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f621ca20640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62180c93e0 0x7f62180caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 -- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 msgr2=0x7f62181f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62181f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 -- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62180c93e0 msgr2=0x7f62180caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62180c93e0 0x7f62180caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 -- 172.21.15.53:0/2071477172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62181f4870 con 0x7f6218162b30 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.091+0000 7f62177fe640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180ba430 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f620c00e810 tx=0x7f620c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.092+0000 7f6214ff9640 1 -- 172.21.15.53:0/2071477172 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f040 con 0x7f6218162b30 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.092+0000 7f6214ff9640 1 -- 172.21.15.53:0/2071477172 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c004030 con 0x7f6218162b30 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.092+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62180c98e0 con 0x7f6218162b30 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.092+0000 7f6214ff9640 1 -- 172.21.15.53:0/2071477172 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c004400 con 0x7f6218162b30 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.092+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62180c9ea0 con 0x7f6218162b30 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.093+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62180b7c40 con 0x7f6218162b30 2024-05-10T10:35:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.093+0000 7f6214ff9640 1 -- 172.21.15.53:0/2071477172 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f620c00c670 con 0x7f6218162b30 2024-05-10T10:35:45.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.094+0000 7f6214ff9640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f62040762d0 0x7f6204078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:45.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.094+0000 7f6214ff9640 1 -- 172.21.15.53:0/2071477172 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f620c097050 con 0x7f6218162b30 2024-05-10T10:35:45.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.094+0000 7f6217fff640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f62040762d0 0x7f6204078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:45.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.094+0000 7f6217fff640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f62040762d0 0x7f6204078790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62180cf710 tx=0x7f6208008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:45.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.098+0000 7f6214ff9640 1 -- 172.21.15.53:0/2071477172 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f620c017020 con 0x7f6218162b30 2024-05-10T10:35:45.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:45.363+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f62180d4920 con 0x7f6218162b30 2024-05-10T10:35:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.330+0000 7f6214ff9640 1 -- 172.21.15.53:0/2071477172 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v128) ==== 172+0+0 (secure 0 0 0) 0x7f620c0604d0 con 0x7f6218162b30 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f62040762d0 msgr2=0x7f6204078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f62040762d0 0x7f6204078790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62180cf710 tx=0x7f6208008040 comp rx=0 tx=0).stop 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 msgr2=0x7f62180ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180ba430 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f620c00e810 tx=0x7f620c00ecd0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 shutdown_connections 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f62040762d0 0x7f6204078790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62180c93e0 0x7f62180caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6218162b30 0x7f62180ba430 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 --2- 172.21.15.53:0/2071477172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6218162750 0x7f62181f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 >> 172.21.15.53:0/2071477172 conn(0x7f621809e3b0 msgr2=0x7f62180d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 shutdown_connections 2024-05-10T10:35:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.333+0000 7f621e483640 1 -- 172.21.15.53:0/2071477172 wait complete. 2024-05-10T10:35:46.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T10:35:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.464+0000 7f778e009640 1 Processor -- start 2024-05-10T10:35:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.464+0000 7f778e009640 1 -- start start 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f778e009640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f77880cffd0 0x7f77880b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f778e009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 0x7f77880d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f778e009640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778806e720 con 0x7f77880cf3e0 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f778e009640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f778806de80 con 0x7f77880cffd0 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f778e009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f778806e160 con 0x7f77880cd500 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f77877fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f77880cf3e0 0x7f77880cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46300/0 (socket says 172.21.15.53:46300) 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f77877fe640 1 -- 172.21.15.53:0/866106015 learned_addr learned my addr 172.21.15.53:0/866106015 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f7787fff640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 0x7f77880d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f7786ffd640 1 --2- 172.21.15.53:0/866106015 >> v2:172.21.15.135:3300/0 conn(0x7f77880cffd0 0x7f77880b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.465+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428662340 0 0) 0x7f778806e720 con 0x7f77880cf3e0 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7770003520 con 0x7f77880cf3e0 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3849134742 0 0) 0x7f7770003520 con 0x7f77880cf3e0 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 msgr2=0x7f77880d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 0x7f77880d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 >> v2:172.21.15.135:3300/0 conn(0x7f77880cffd0 msgr2=0x7f77880b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 --2- 172.21.15.53:0/866106015 >> v2:172.21.15.135:3300/0 conn(0x7f77880cffd0 0x7f77880b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778806e440 con 0x7f77880cf3e0 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f7778001670 con 0x7f77880cf3e0 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 >> v1:172.21.15.53:6789/0 conn(0x7f77880cf3e0 legacy=0x7f77880cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77700039b0 0x7f7770003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 0x7f7770004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7770004d40 0x7f77700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f778806de80 con 0x7f7770004d40 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f778806e160 con 0x7f77700039b0 2024-05-10T10:35:46.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f777000b9c0 con 0x7f77880cd500 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f7787fff640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7770004d40 0x7f77700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f77877fe640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77700039b0 0x7f7770003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f7786ffd640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 0x7f7770004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f7787fff640 1 -- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 msgr2=0x7f7770004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f7787fff640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77880cd500 0x7f7770004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f7787fff640 1 -- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77700039b0 msgr2=0x7f7770003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f7787fff640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77700039b0 0x7f7770003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.466+0000 7f7787fff640 1 -- 172.21.15.53:0/866106015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7770003520 con 0x7f7770004d40 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.467+0000 7f7787fff640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7770004d40 0x7f77700051c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7788073ea0 tx=0x7f7770005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc3e641cd76b417 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.467+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780037070 con 0x7f7770004d40 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.467+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778002e050 con 0x7f7770004d40 2024-05-10T10:35:46.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.467+0000 7f77867fc640 1 -- 172.21.15.53:0/866106015 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780009c10 con 0x7f7770004d40 2024-05-10T10:35:46.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.468+0000 7f778e009640 1 -- 172.21.15.53:0/866106015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7770004d40 msgr2=0x7f77700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.468+0000 7f778e009640 1 --2- 172.21.15.53:0/866106015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7770004d40 0x7f77700051c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7788073ea0 tx=0x7f7770005830 comp rx=0 tx=0).stop 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.468+0000 7f7787fff640 1 -- 172.21.15.53:0/866106015 reap_dead start 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.468+0000 7f778e009640 1 -- 172.21.15.53:0/866106015 shutdown_connections 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.468+0000 7f778e009640 1 -- 172.21.15.53:0/866106015 >> 172.21.15.53:0/866106015 conn(0x7f77880d2eb0 msgr2=0x7f77880d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.468+0000 7f778e009640 1 -- 172.21.15.53:0/866106015 shutdown_connections 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.468+0000 7f778e009640 1 -- 172.21.15.53:0/866106015 wait complete. 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.469+0000 7f778e009640 1 Processor -- start 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.469+0000 7f778e009640 1 -- start start 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f778e009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7770004d40 0x7f778815f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f778e009640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77880c9790 0x7f7788153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f778e009640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77880d1600 0x7f77880d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f778e009640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77880d1180 con 0x7f77880d1600 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f778e009640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77880b4510 con 0x7f77880c9790 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f778e009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f77880d1300 con 0x7f7770004d40 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f7787fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77880d1600 0x7f77880d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f7787fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77880d1600 0x7f77880d1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50276/0 (socket says 172.21.15.53:50276) 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.470+0000 7f7787fff640 1 -- 172.21.15.53:0/2164493087 learned_addr learned my addr 172.21.15.53:0/2164493087 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7786ffd640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77880c9790 0x7f7788153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f77877fe640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7770004d40 0x7f778815f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7787fff640 1 -- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7770004d40 msgr2=0x7f778815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7787fff640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7770004d40 0x7f778815f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7787fff640 1 -- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77880c9790 msgr2=0x7f7788153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7787fff640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77880c9790 0x7f7788153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7787fff640 1 -- 172.21.15.53:0/2164493087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7788153890 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7786ffd640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77880c9790 0x7f7788153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f77877fe640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7770004d40 0x7f778815f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f7787fff640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77880d1600 0x7f77880d1ab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7780002230 tx=0x7f7780009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f776ffff640 1 -- 172.21.15.53:0/2164493087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780037070 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f776ffff640 1 -- 172.21.15.53:0/2164493087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7780042070 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77880d2800 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f776ffff640 1 -- 172.21.15.53:0/2164493087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780009c10 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.471+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77880b4930 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.472+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778806a220 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.473+0000 7f776ffff640 1 -- 172.21.15.53:0/2164493087 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f778003ca50 con 0x7f77880d1600 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.473+0000 7f776ffff640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7774076300 0x7f77740787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.473+0000 7f776ffff640 1 -- 172.21.15.53:0/2164493087 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f77800b7e90 con 0x7f77880d1600 2024-05-10T10:35:46.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.474+0000 7f77877fe640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7774076300 0x7f77740787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.474+0000 7f77877fe640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7774076300 0x7f77740787c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7778000c00 tx=0x7f7778009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:46.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.478+0000 7f776ffff640 1 -- 172.21.15.53:0/2164493087 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f778004c020 con 0x7f77880d1600 2024-05-10T10:35:46.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.669+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f77880c9b70 con 0x7f77880d1600 2024-05-10T10:35:46.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.672+0000 7f776ffff640 1 -- 172.21.15.53:0/2164493087 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f77800b90c0 con 0x7f77880d1600 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7774076300 msgr2=0x7f77740787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7774076300 0x7f77740787c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7778000c00 tx=0x7f7778009040 comp rx=0 tx=0).stop 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77880d1600 msgr2=0x7f77880d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77880d1600 0x7f77880d1ab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7780002230 tx=0x7f7780009750 comp rx=0 tx=0).stop 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 shutdown_connections 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7774076300 0x7f77740787c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77880d1600 0x7f77880d1ab0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77880c9790 0x7f7788153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 --2- 172.21.15.53:0/2164493087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7770004d40 0x7f778815f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 >> 172.21.15.53:0/2164493087 conn(0x7f77880d2eb0 msgr2=0x7f77880b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:46.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 shutdown_connections 2024-05-10T10:35:46.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.674+0000 7f778e009640 1 -- 172.21.15.53:0/2164493087 wait complete. 2024-05-10T10:35:46.686 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:35:46.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:35:46.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:35:46.764 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:35:46.764 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:35:46.764 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:46.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:35:46.781 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:35:46.781 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:46.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:35:46.823 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:35:46.823 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:46.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:35:46.839 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:35:46.839 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:46.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:35:46.881 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T10:35:46.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:46.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.984+0000 7f9962904640 1 Processor -- start 2024-05-10T10:35:46.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.984+0000 7f9962904640 1 -- start start 2024-05-10T10:35:46.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f9962904640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f995c0b8860 0x7f995c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f9962904640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 0x7f995c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f9962904640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995c15fc90 con 0x7f995c0d0230 2024-05-10T10:35:46.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f9962904640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f995c06ee50 con 0x7f995c0b8860 2024-05-10T10:35:46.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f9962904640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f995c0b8c40 con 0x7f995c0cfd80 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f995c0d0230 0x7f995c0cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46314/0 (socket says 172.21.15.53:46314) 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995b7fe640 1 -- 172.21.15.53:0/3603010563 learned_addr learned my addr 172.21.15.53:0/3603010563 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995bfff640 1 --2- 172.21.15.53:0/3603010563 >> v2:172.21.15.135:3300/0 conn(0x7f995c0b8860 0x7f995c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f9960ea1640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 0x7f995c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637893963 0 0) 0x7f995c15fc90 con 0x7f995c0d0230 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9944003520 con 0x7f995c0d0230 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2414085801 0 0) 0x7f9944003520 con 0x7f995c0d0230 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 msgr2=0x7f995c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 0x7f995c0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 >> v2:172.21.15.135:3300/0 conn(0x7f995c0b8860 msgr2=0x7f995c06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 --2- 172.21.15.53:0/3603010563 >> v2:172.21.15.135:3300/0 conn(0x7f995c0b8860 0x7f995c06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c0cd570 con 0x7f995c0d0230 2024-05-10T10:35:46.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.985+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f9950001670 con 0x7f995c0d0230 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 >> v1:172.21.15.53:6789/0 conn(0x7f995c0d0230 legacy=0x7f995c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995affd640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 0x7f99440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995affd640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944003f30 0x7f9944004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995affd640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9944004d40 0x7f99440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f995c06ee50 con 0x7f9944004d40 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f995c0b8c40 con 0x7f9944003f30 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f994400ba10 con 0x7f995c0cfd80 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f9960ea1640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9944004d40 0x7f99440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995bfff640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 0x7f99440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995b7fe640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944003f30 0x7f9944004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f9960ea1640 1 -- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 msgr2=0x7f99440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f9960ea1640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c0cfd80 0x7f99440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f9960ea1640 1 -- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944003f30 msgr2=0x7f9944004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f9960ea1640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944003f30 0x7f9944004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f9960ea1640 1 -- 172.21.15.53:0/3603010563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9944003520 con 0x7f9944004d40 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.986+0000 7f995b7fe640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944003f30 0x7f9944004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9960ea1640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9944004d40 0x7f99440051c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9954009f90 tx=0x7f9944005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46c53f2f648d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954009c10 con 0x7f9944004d40 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9954036070 con 0x7f9944004d40 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f995affd640 1 -- 172.21.15.53:0/3603010563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954031810 con 0x7f9944004d40 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9962904640 1 -- 172.21.15.53:0/3603010563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9944004d40 msgr2=0x7f99440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9962904640 1 --2- 172.21.15.53:0/3603010563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9944004d40 0x7f99440051c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9954009f90 tx=0x7f9944005830 comp rx=0 tx=0).stop 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9960ea1640 1 -- 172.21.15.53:0/3603010563 reap_dead start 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9962904640 1 -- 172.21.15.53:0/3603010563 shutdown_connections 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9962904640 1 -- 172.21.15.53:0/3603010563 >> 172.21.15.53:0/3603010563 conn(0x7f995c0d2df0 msgr2=0x7f995c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9962904640 1 -- 172.21.15.53:0/3603010563 shutdown_connections 2024-05-10T10:35:46.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.987+0000 7f9962904640 1 -- 172.21.15.53:0/3603010563 wait complete. 2024-05-10T10:35:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.989+0000 7f9962904640 1 Processor -- start 2024-05-10T10:35:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.989+0000 7f9962904640 1 -- start start 2024-05-10T10:35:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944004d40 0x7f995c157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f995c0cbed0 0x7f995c0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c1f4c30 0x7f995c1f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f995c0cde10 con 0x7f995c0cbed0 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f995c0cdb30 con 0x7f9944004d40 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f995c06ea60 con 0x7f995c1f4c30 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f995c0cbed0 0x7f995c0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f995c0cbed0 0x7f995c0cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50296/0 (socket says 172.21.15.53:50296) 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 -- 172.21.15.53:0/616336034 learned_addr learned my addr 172.21.15.53:0/616336034 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9960ea1640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c1f4c30 0x7f995c1f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 -- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c1f4c30 msgr2=0x7f995c1f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995bfff640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944004d40 0x7f995c157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c1f4c30 0x7f995c1f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 -- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944004d40 msgr2=0x7f995c157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944004d40 0x7f995c157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 -- 172.21.15.53:0/616336034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c1f5a20 con 0x7f995c0cbed0 2024-05-10T10:35:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f995b7fe640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f995c0cbed0 0x7f995c0cc350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f995000e8c0 tx=0x7f995000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9958ff9640 1 -- 172.21.15.53:0/616336034 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995000c870 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9958ff9640 1 -- 172.21.15.53:0/616336034 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9950018070 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c1f84a0 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9958ff9640 1 -- 172.21.15.53:0/616336034 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995000cc10 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.990+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c1f8a60 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.992+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995c1f8c60 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.992+0000 7f9958ff9640 1 -- 172.21.15.53:0/616336034 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9950013420 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.993+0000 7f9958ff9640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9948076300 0x7f99480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.993+0000 7f9958ff9640 1 -- 172.21.15.53:0/616336034 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f9950097050 con 0x7f995c0cbed0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.993+0000 7f995bfff640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9948076300 0x7f99480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:46.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.993+0000 7f995bfff640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9948076300 0x7f99480787c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f994c009a60 tx=0x7f994c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:46.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:46.997+0000 7f9958ff9640 1 -- 172.21.15.53:0/616336034 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f995c1f8c60 con 0x7f995c0cbed0 2024-05-10T10:35:47.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.227+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f995c06a220 con 0x7f995c0cbed0 2024-05-10T10:35:47.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.228+0000 7f9958ff9640 1 -- 172.21.15.53:0/616336034 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3869 (secure 0 0 0) 0x7f9950060bb0 con 0x7f995c0cbed0 2024-05-10T10:35:47.229 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:47.229 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4841,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/454314849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":454314849},{"type":"v1","addr":"172.21.15.53:6833","nonce":454314849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1610428704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1610428704},{"type":"v1","addr":"172.21.15.135:6836","nonce":1610428704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/288247062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":288247062},{"type":"v1","addr":"172.21.15.135:6837","nonce":288247062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3240060075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3240060075},{"type":"v1","addr":"172.21.15.53:6835","nonce":3240060075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-05-10T10:35:47.229 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 28 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9948076300 msgr2=0x7f99480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9948076300 0x7f99480787c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f994c009a60 tx=0x7f994c008040 comp rx=0 tx=0).stop 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f995c0cbed0 msgr2=0x7f995c0cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f995c0cbed0 0x7f995c0cc350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f995000e8c0 tx=0x7f995000ed80 comp rx=0 tx=0).stop 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 shutdown_connections 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9948076300 0x7f99480787c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f995c1f4c30 0x7f995c1f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f995c0cbed0 0x7f995c0cc350 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 --2- 172.21.15.53:0/616336034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9944004d40 0x7f995c157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 >> 172.21.15.53:0/616336034 conn(0x7f995c0d2df0 msgr2=0x7f995c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:47.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 shutdown_connections 2024-05-10T10:35:47.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.230+0000 7f9962904640 1 -- 172.21.15.53:0/616336034 wait complete. 2024-05-10T10:35:47.242 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:35:47.242 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T10:35:47.242 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:35:47.243 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:35:47.243 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:35:47.243 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T10:35:47.243 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:35:47.243 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:35:47.243 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:35:47.243 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T10:35:47.244 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:35:47.244 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:35:47.244 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:35:47.244 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T10:35:47.244 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:35:47.244 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:35:47.252 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:35:47.252 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:35:47.253 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:35:47.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:35:47.258 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:35:47.258 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:35:47.259 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:35:47.259 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:35:47.295 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:35:47.296 DEBUG:teuthology.parallel:result is None 2024-05-10T10:35:47.296 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:35:47.296 DEBUG:teuthology.parallel:result is None 2024-05-10T10:35:47.301 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:35:47.301 DEBUG:teuthology.parallel:result is None 2024-05-10T10:35:47.302 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:35:47.302 DEBUG:teuthology.parallel:result is None 2024-05-10T10:35:47.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T10:35:47.375 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:35:47.375 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:35:47.383 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:35:47.384 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:35:47.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.411+0000 7f16a61da640 1 Processor -- start 2024-05-10T10:35:47.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.411+0000 7f16a61da640 1 -- start start 2024-05-10T10:35:47.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f16a61da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f16a015f290 0x7f16a00d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:47.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f16a61da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:47.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f16a61da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16a006f3c0 con 0x7f16a00d3280 2024-05-10T10:35:47.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f16a61da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f16a00d14c0 con 0x7f16a015f290 2024-05-10T10:35:47.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f16a61da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f16a00d2300 con 0x7f16a00702e0 2024-05-10T10:35:47.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f169f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f16a015f290 0x7f16a00d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:47.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f169f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f16a015f290 0x7f16a00d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50308/0 (socket says 172.21.15.53:50308) 2024-05-10T10:35:47.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f169f7fe640 1 -- 172.21.15.53:0/1061889439 learned_addr learned my addr 172.21.15.53:0/1061889439 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:47.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.412+0000 7f169effd640 1 --2- 172.21.15.53:0/1061889439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:47.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.413+0000 7f169f7fe640 1 -- 172.21.15.53:0/1061889439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 msgr2=0x7f16a00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:47.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.413+0000 7f169f7fe640 1 --2- 172.21.15.53:0/1061889439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a00d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.413+0000 7f169f7fe640 1 -- 172.21.15.53:0/1061889439 >> v1:172.21.15.135:6789/0 conn(0x7f16a00d3280 legacy=0x7f16a00cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:47.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.413+0000 7f169f7fe640 1 -- 172.21.15.53:0/1061889439 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a00d16f0 con 0x7f16a015f290 2024-05-10T10:35:47.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.413+0000 7f169f7fe640 1 --2- 172.21.15.53:0/1061889439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a015f290 0x7f16a00d0f80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1690009870 tx=0x7f169002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd13dd03d1f04f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:47.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f169e7fc640 1 -- 172.21.15.53:0/1061889439 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690037070 con 0x7f16a015f290 2024-05-10T10:35:47.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f169e7fc640 1 -- 172.21.15.53:0/1061889439 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169002ae50 con 0x7f16a015f290 2024-05-10T10:35:47.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f169e7fc640 1 -- 172.21.15.53:0/1061889439 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690032860 con 0x7f16a015f290 2024-05-10T10:35:47.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 -- 172.21.15.53:0/1061889439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a015f290 msgr2=0x7f16a00d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:47.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 --2- 172.21.15.53:0/1061889439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a015f290 0x7f16a00d0f80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1690009870 tx=0x7f169002a300 comp rx=0 tx=0).stop 2024-05-10T10:35:47.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 -- 172.21.15.53:0/1061889439 shutdown_connections 2024-05-10T10:35:47.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 --2- 172.21.15.53:0/1061889439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a00d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 --2- 172.21.15.53:0/1061889439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a015f290 0x7f16a00d0f80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 -- 172.21.15.53:0/1061889439 >> 172.21.15.53:0/1061889439 conn(0x7f16a009ba20 msgr2=0x7f16a009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:47.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 -- 172.21.15.53:0/1061889439 shutdown_connections 2024-05-10T10:35:47.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.414+0000 7f16a61da640 1 -- 172.21.15.53:0/1061889439 wait complete. 2024-05-10T10:35:47.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.415+0000 7f16a61da640 1 Processor -- start 2024-05-10T10:35:47.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f16a61da640 1 -- start start 2024-05-10T10:35:47.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f16a61da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:47.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f16a61da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a00d3280 0x7f16a01e8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:47.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f16a61da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16a015f290 0x7f16a00c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f16a61da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f16a00d1f90 con 0x7f16a00d3280 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f16a61da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16a00d1cb0 con 0x7f16a015f290 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f16a61da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f16a015f670 con 0x7f16a00702e0 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f169effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a00d3280 0x7f16a01e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f169effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a00d3280 0x7f16a01e8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50312/0 (socket says 172.21.15.53:50312) 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f169effd640 1 -- 172.21.15.53:0/814840205 learned_addr learned my addr 172.21.15.53:0/814840205 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.416+0000 7f169ffff640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16a015f290 0x7f16a00c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169f7fe640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169effd640 1 -- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 msgr2=0x7f16a014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169effd640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a014d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169effd640 1 -- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16a015f290 msgr2=0x7f16a00c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169effd640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16a015f290 0x7f16a00c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:47.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169effd640 1 -- 172.21.15.53:0/814840205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a014db70 con 0x7f16a00d3280 2024-05-10T10:35:47.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169f7fe640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:47.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f169effd640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a00d3280 0x7f16a01e8000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f169400e8a0 tx=0x7f169400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:47.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f1687fff640 1 -- 172.21.15.53:0/814840205 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694017070 con 0x7f16a00d3280 2024-05-10T10:35:47.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f1687fff640 1 -- 172.21.15.53:0/814840205 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169400c940 con 0x7f16a00d3280 2024-05-10T10:35:47.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f1687fff640 1 -- 172.21.15.53:0/814840205 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169400cd30 con 0x7f16a00d3280 2024-05-10T10:35:47.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a01f8340 con 0x7f16a00d3280 2024-05-10T10:35:47.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.417+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01f8900 con 0x7f16a00d3280 2024-05-10T10:35:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.418+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a00693f0 con 0x7f16a00d3280 2024-05-10T10:35:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.419+0000 7f1687fff640 1 -- 172.21.15.53:0/814840205 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f169401a070 con 0x7f16a00d3280 2024-05-10T10:35:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.419+0000 7f1687fff640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f168c0762d0 0x7f168c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.419+0000 7f1687fff640 1 -- 172.21.15.53:0/814840205 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f1694097630 con 0x7f16a00d3280 2024-05-10T10:35:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.420+0000 7f169f7fe640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f168c0762d0 0x7f168c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:47.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.420+0000 7f169f7fe640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f168c0762d0 0x7f168c078790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1690034000 tx=0x7f169002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:47.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.424+0000 7f1687fff640 1 -- 172.21.15.53:0/814840205 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1694060ab0 con 0x7f16a00d3280 2024-05-10T10:35:47.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:47.626+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f16a0075220 con 0x7f16a00d3280 2024-05-10T10:35:48.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.341+0000 7f1687fff640 1 -- 172.21.15.53:0/814840205 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) ==== 97+0+0 (secure 0 0 0) 0x7f1694064db0 con 0x7f16a00d3280 2024-05-10T10:35:48.342 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2024-05-10T10:35:48.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.344+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f168c0762d0 msgr2=0x7f168c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.344+0000 7f16a61da640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f168c0762d0 0x7f168c078790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1690034000 tx=0x7f169002e000 comp rx=0 tx=0).stop 2024-05-10T10:35:48.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.344+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a00d3280 msgr2=0x7f16a01e8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.344+0000 7f16a61da640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a00d3280 0x7f16a01e8000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f169400e8a0 tx=0x7f169400ed60 comp rx=0 tx=0).stop 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 shutdown_connections 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f168c0762d0 0x7f168c078790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16a015f290 0x7f16a00c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16a00d3280 0x7f16a01e8000 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 --2- 172.21.15.53:0/814840205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16a00702e0 0x7f16a014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 >> 172.21.15.53:0/814840205 conn(0x7f16a009ba20 msgr2=0x7f16a00d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 shutdown_connections 2024-05-10T10:35:48.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.345+0000 7f16a61da640 1 -- 172.21.15.53:0/814840205 wait complete. 2024-05-10T10:35:48.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T10:35:48.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.463+0000 7fca248e1640 1 Processor -- start 2024-05-10T10:35:48.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.463+0000 7fca248e1640 1 -- start start 2024-05-10T10:35:48.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.464+0000 7fca248e1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fca2015f3b0 0x7fca200d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.464+0000 7fca248e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca2015f790 0x7fca200d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.464+0000 7fca248e1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fca200d5d30 0x7fca200c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.464+0000 7fca248e1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fca200d30b0 con 0x7fca2015f3b0 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.464+0000 7fca248e1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fca200d3390 con 0x7fca200d5d30 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.464+0000 7fca248e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fca200c8c80 con 0x7fca2015f790 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fca2015f3b0 0x7fca200d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fca2015f3b0 0x7fca200d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50322/0 (socket says 172.21.15.53:50322) 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 -- 172.21.15.53:0/366505466 learned_addr learned my addr 172.21.15.53:0/366505466 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 -- 172.21.15.53:0/366505466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca2015f790 msgr2=0x7fca200d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 --2- 172.21.15.53:0/366505466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca2015f790 0x7fca200d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 -- 172.21.15.53:0/366505466 >> v2:172.21.15.135:3300/0 conn(0x7fca200d5d30 msgr2=0x7fca200c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:48.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 --2- 172.21.15.53:0/366505466 >> v2:172.21.15.135:3300/0 conn(0x7fca200d5d30 0x7fca200c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.465+0000 7fca1e59c640 1 -- 172.21.15.53:0/366505466 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca200c8f60 con 0x7fca2015f3b0 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca1e59c640 1 --2- 172.21.15.53:0/366505466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200d36f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca200d36a0 tx=0x7fca1002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13082fa330be834a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca1d59a640 1 -- 172.21.15.53:0/366505466 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10036070 con 0x7fca2015f3b0 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca1d59a640 1 -- 172.21.15.53:0/366505466 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca10031600 con 0x7fca2015f3b0 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca1d59a640 1 -- 172.21.15.53:0/366505466 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca100319d0 con 0x7fca2015f3b0 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 -- 172.21.15.53:0/366505466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 msgr2=0x7fca200d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 --2- 172.21.15.53:0/366505466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200d36f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca200d36a0 tx=0x7fca1002a6a0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 -- 172.21.15.53:0/366505466 shutdown_connections 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 --2- 172.21.15.53:0/366505466 >> v2:172.21.15.135:3300/0 conn(0x7fca200d5d30 0x7fca200c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 --2- 172.21.15.53:0/366505466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca2015f790 0x7fca200d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 --2- 172.21.15.53:0/366505466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200d36f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 -- 172.21.15.53:0/366505466 >> 172.21.15.53:0/366505466 conn(0x7fca200d5130 msgr2=0x7fca200d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:48.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 -- 172.21.15.53:0/366505466 shutdown_connections 2024-05-10T10:35:48.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.466+0000 7fca248e1640 1 -- 172.21.15.53:0/366505466 wait complete. 2024-05-10T10:35:48.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.468+0000 7fca248e1640 1 Processor -- start 2024-05-10T10:35:48.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.468+0000 7fca248e1640 1 -- start start 2024-05-10T10:35:48.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca248e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca200d5d30 0x7fca2015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca248e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200b9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca248e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca20158720 0x7fca200cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca248e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca200c9e20 con 0x7fca2015f3b0 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca248e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca200c9ca0 con 0x7fca20158720 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca248e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fca200c9fa0 con 0x7fca200d5d30 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200b9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200b9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50330/0 (socket says 172.21.15.53:50330) 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 -- 172.21.15.53:0/407031618 learned_addr learned my addr 172.21.15.53:0/407031618 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1e59c640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca200d5d30 0x7fca2015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1ed9d640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca20158720 0x7fca200cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 -- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca200d5d30 msgr2=0x7fca2015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca200d5d30 0x7fca2015bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 -- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca20158720 msgr2=0x7fca200cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca20158720 0x7fca200cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 -- 172.21.15.53:0/407031618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2015c1b0 con 0x7fca2015f3b0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1e59c640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca200d5d30 0x7fca2015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1ed9d640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca20158720 0x7fca200cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca1dd9b640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200b9f80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b440 tx=0x7fca0800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca16ffd640 1 -- 172.21.15.53:0/407031618 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08015020 con 0x7fca2015f3b0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca16ffd640 1 -- 172.21.15.53:0/407031618 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca080118b0 con 0x7fca2015f3b0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca200b8730 con 0x7fca2015f3b0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.469+0000 7fca16ffd640 1 -- 172.21.15.53:0/407031618 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08011ca0 con 0x7fca2015f3b0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.470+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca200b8cf0 con 0x7fca2015f3b0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.471+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2006a220 con 0x7fca2015f3b0 2024-05-10T10:35:48.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.471+0000 7fca16ffd640 1 -- 172.21.15.53:0/407031618 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fca08018020 con 0x7fca2015f3b0 2024-05-10T10:35:48.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.472+0000 7fca16ffd640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fca000762d0 0x7fca00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.472+0000 7fca16ffd640 1 -- 172.21.15.53:0/407031618 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7fca08097550 con 0x7fca2015f3b0 2024-05-10T10:35:48.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.472+0000 7fca1e59c640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fca000762d0 0x7fca00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.472+0000 7fca1e59c640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fca000762d0 0x7fca00078790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca10033000 tx=0x7fca10009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:48.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.476+0000 7fca16ffd640 1 -- 172.21.15.53:0/407031618 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fca0801f020 con 0x7fca2015f3b0 2024-05-10T10:35:48.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.740+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fca200cb370 con 0x7fca2015f3b0 2024-05-10T10:35:48.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.741+0000 7fca16ffd640 1 -- 172.21.15.53:0/407031618 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7fca2015c1b0 con 0x7fca2015f3b0 2024-05-10T10:35:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:48.741 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg==", 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2024-05-10T10:35:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ==", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA==", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:35:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew==", 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA==", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA==", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw==", 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA==", 2024-05-10T10:35:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g==", 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A==", 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA==", 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg==", 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2024-05-10T10:35:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg==", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g==", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:35:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw==", 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2024-05-10T10:35:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw==", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1mKY2tIhAAzPSpOQLOFy1GEKdl5bPBEw==", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw==", 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ==", 2024-05-10T10:35:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:35:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:35:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:35:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:35:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:35:48.755 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:48.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fca000762d0 msgr2=0x7fca00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fca000762d0 0x7fca00078790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca10033000 tx=0x7fca10009000 comp rx=0 tx=0).stop 2024-05-10T10:35:48.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 msgr2=0x7fca200b9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200b9f80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b440 tx=0x7fca0800b900 comp rx=0 tx=0).stop 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 shutdown_connections 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fca000762d0 0x7fca00078790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca20158720 0x7fca200cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca2015f3b0 0x7fca200b9f80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 --2- 172.21.15.53:0/407031618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fca200d5d30 0x7fca2015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 >> 172.21.15.53:0/407031618 conn(0x7fca200d5130 msgr2=0x7fca200d35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 shutdown_connections 2024-05-10T10:35:48.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.743+0000 7fca248e1640 1 -- 172.21.15.53:0/407031618 wait complete. 2024-05-10T10:35:48.758 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:35:48.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.860+0000 7f2f2179d640 1 Processor -- start 2024-05-10T10:35:48.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.860+0000 7f2f2179d640 1 -- start start 2024-05-10T10:35:48.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f2179d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2f1c162750 0x7f2f1c0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f2179d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2f1c162b30 0x7f2f1c0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f2179d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f1c0b45f0 con 0x7f2f1c0d5cb0 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f2179d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2f1c0b4f20 con 0x7f2f1c162750 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f2179d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2f1c0d0fe0 con 0x7f2f1c162b30 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f1affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2f1c162750 0x7f2f1c0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f1affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2f1c162750 0x7f2f1c0d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50336/0 (socket says 172.21.15.53:50336) 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f1a7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2f1c0d5cb0 0x7f2f1c0d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46350/0 (socket says 172.21.15.53:46350) 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f1affd640 1 -- 172.21.15.53:0/2953617154 learned_addr learned my addr 172.21.15.53:0/2953617154 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.861+0000 7f2f1b7fe640 1 --2- 172.21.15.53:0/2953617154 >> v2:172.21.15.135:3300/0 conn(0x7f2f1c162b30 0x7f2f1c0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.862+0000 7f2f1affd640 1 -- 172.21.15.53:0/2953617154 >> v2:172.21.15.135:3300/0 conn(0x7f2f1c162b30 msgr2=0x7f2f1c0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.862+0000 7f2f1affd640 1 --2- 172.21.15.53:0/2953617154 >> v2:172.21.15.135:3300/0 conn(0x7f2f1c162b30 0x7f2f1c0b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.862+0000 7f2f1affd640 1 -- 172.21.15.53:0/2953617154 >> v1:172.21.15.53:6789/0 conn(0x7f2f1c0d5cb0 legacy=0x7f2f1c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.862+0000 7f2f1affd640 1 -- 172.21.15.53:0/2953617154 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c0b48d0 con 0x7f2f1c162750 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.862+0000 7f2f19ffb640 1 -- 172.21.15.53:0/2953617154 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532419867 0 0) 0x7f2f1c0b45f0 con 0x7f2f1c0d5cb0 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.862+0000 7f2f1affd640 1 --2- 172.21.15.53:0/2953617154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c162750 0x7f2f1c0d0670 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0d32c0 tx=0x7f2f0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef63989f29ea14d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.862+0000 7f2f19ffb640 1 -- 172.21.15.53:0/2953617154 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c036070 con 0x7f2f1c162750 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f19ffb640 1 -- 172.21.15.53:0/2953617154 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c031600 con 0x7f2f1c162750 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f19ffb640 1 -- 172.21.15.53:0/2953617154 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0319d0 con 0x7f2f1c162750 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 -- 172.21.15.53:0/2953617154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c162750 msgr2=0x7f2f1c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 --2- 172.21.15.53:0/2953617154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c162750 0x7f2f1c0d0670 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0d32c0 tx=0x7f2f0c02a6a0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 -- 172.21.15.53:0/2953617154 shutdown_connections 2024-05-10T10:35:48.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 --2- 172.21.15.53:0/2953617154 >> v2:172.21.15.135:3300/0 conn(0x7f2f1c162b30 0x7f2f1c0b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 --2- 172.21.15.53:0/2953617154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c162750 0x7f2f1c0d0670 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 -- 172.21.15.53:0/2953617154 >> 172.21.15.53:0/2953617154 conn(0x7f2f1c09e3b0 msgr2=0x7f2f1c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:48.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 -- 172.21.15.53:0/2953617154 shutdown_connections 2024-05-10T10:35:48.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.863+0000 7f2f2179d640 1 -- 172.21.15.53:0/2953617154 wait complete. 2024-05-10T10:35:48.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.864+0000 7f2f2179d640 1 Processor -- start 2024-05-10T10:35:48.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f2179d640 1 -- start start 2024-05-10T10:35:48.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f2179d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f1c162750 0x7f2f1c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f2179d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f1c162b30 0x7f2f1c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f2179d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c0ce810 0x7f2f1c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f2179d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c0d0d30 con 0x7f2f1c0ce810 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f2179d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c0d0bb0 con 0x7f2f1c162b30 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f2179d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1c0d3d40 con 0x7f2f1c162750 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f1b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c0ce810 0x7f2f1c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f1b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c0ce810 0x7f2f1c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50344/0 (socket says 172.21.15.53:50344) 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f1affd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f1c162750 0x7f2f1c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.865+0000 7f2f1b7fe640 1 -- 172.21.15.53:0/1573877563 learned_addr learned my addr 172.21.15.53:0/1573877563 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f1a7fc640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f1c162b30 0x7f2f1c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f1b7fe640 1 -- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f1c162750 msgr2=0x7f2f1c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f1b7fe640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f1c162750 0x7f2f1c156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f1b7fe640 1 -- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f1c162b30 msgr2=0x7f2f1c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f1b7fe640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f1c162b30 0x7f2f1c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f1b7fe640 1 -- 172.21.15.53:0/1573877563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c0cf430 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f1b7fe640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c0ce810 0x7f2f1c0b3260 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400d600 tx=0x7f2f1400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f137fe640 1 -- 172.21.15.53:0/1573877563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140125f0 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f137fe640 1 -- 172.21.15.53:0/1573877563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f14019070 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1c0cee70 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f137fe640 1 -- 172.21.15.53:0/1573877563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140129f0 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.866+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1c0b8960 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.867+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1c06a220 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.868+0000 7f2f137fe640 1 -- 172.21.15.53:0/1573877563 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2f14012b90 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.869+0000 7f2f137fe640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2efc0762d0 0x7f2efc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.869+0000 7f2f137fe640 1 -- 172.21.15.53:0/1573877563 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f2f140976d0 con 0x7f2f1c0ce810 2024-05-10T10:35:48.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.869+0000 7f2f1affd640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2efc0762d0 0x7f2efc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:48.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.869+0000 7f2f1affd640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2efc0762d0 0x7f2efc078790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c033000 tx=0x7f2f0c009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:48.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:48.873+0000 7f2f137fe640 1 -- 172.21.15.53:0/1573877563 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2f14060bb0 con 0x7f2f1c0ce810 2024-05-10T10:35:49.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.065+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2f1c1f5240 con 0x7f2f1c0ce810 2024-05-10T10:35:49.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.066+0000 7f2f137fe640 1 -- 172.21.15.53:0/1573877563 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) ==== 84+0+65 (secure 0 0 0) 0x7f2f14064eb0 con 0x7f2f1c0ce810 2024-05-10T10:35:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:49.066 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:35:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:35:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:35:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:35:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:35:49.067 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2efc0762d0 msgr2=0x7f2efc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2efc0762d0 0x7f2efc078790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c033000 tx=0x7f2f0c009000 comp rx=0 tx=0).stop 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c0ce810 msgr2=0x7f2f1c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c0ce810 0x7f2f1c0b3260 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400d600 tx=0x7f2f1400dac0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 shutdown_connections 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2efc0762d0 0x7f2efc078790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f1c0ce810 0x7f2f1c0b3260 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f1c162b30 0x7f2f1c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.067+0000 7f2f2179d640 1 --2- 172.21.15.53:0/1573877563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f1c162750 0x7f2f1c156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.068+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 >> 172.21.15.53:0/1573877563 conn(0x7f2f1c09e3b0 msgr2=0x7f2f1c0b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:49.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.068+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 shutdown_connections 2024-05-10T10:35:49.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.068+0000 7f2f2179d640 1 -- 172.21.15.53:0/1573877563 wait complete. 2024-05-10T10:35:49.079 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:35:49.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:35:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.186+0000 7fafce9c8640 1 Processor -- start 2024-05-10T10:35:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.187+0000 7fafce9c8640 1 -- start start 2024-05-10T10:35:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.187+0000 7fafce9c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc80d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.187+0000 7fafce9c8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fafc815f6e0 0x7fafc80d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.187+0000 7fafce9c8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc80d3fa0 con 0x7fafc80d5cf0 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.187+0000 7fafce9c8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fafc80d1010 con 0x7fafc815f6e0 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.187+0000 7fafce9c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fafc80b26c0 con 0x7fafc80d3680 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafc77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fafc815f6e0 0x7fafc80d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafccf65640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fafc80d5cf0 0x7fafc80d1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46362/0 (socket says 172.21.15.53:46362) 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafccf65640 1 -- 172.21.15.53:0/746201811 learned_addr learned my addr 172.21.15.53:0/746201811 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafc7fff640 1 --2- 172.21.15.53:0/746201811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc80d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafc77fe640 1 -- 172.21.15.53:0/746201811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 msgr2=0x7fafc80d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafc77fe640 1 --2- 172.21.15.53:0/746201811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc80d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafc77fe640 1 -- 172.21.15.53:0/746201811 >> v1:172.21.15.53:6789/0 conn(0x7fafc80d5cf0 legacy=0x7fafc80d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafc77fe640 1 -- 172.21.15.53:0/746201811 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc80b1d90 con 0x7fafc815f6e0 2024-05-10T10:35:49.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.188+0000 7fafc6ffd640 1 -- 172.21.15.53:0/746201811 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 957671825 0 0) 0x7fafc80d3fa0 con 0x7fafc80d5cf0 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafc77fe640 1 --2- 172.21.15.53:0/746201811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80d0750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0097a0 tx=0x7fafbc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3bf73fc2ddb5bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafc6ffd640 1 -- 172.21.15.53:0/746201811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc037070 con 0x7fafc815f6e0 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafc6ffd640 1 -- 172.21.15.53:0/746201811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc02ae50 con 0x7fafc815f6e0 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafc6ffd640 1 -- 172.21.15.53:0/746201811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc032660 con 0x7fafc815f6e0 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 -- 172.21.15.53:0/746201811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 msgr2=0x7fafc80d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 --2- 172.21.15.53:0/746201811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80d0750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0097a0 tx=0x7fafbc02a100 comp rx=0 tx=0).stop 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 -- 172.21.15.53:0/746201811 shutdown_connections 2024-05-10T10:35:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 --2- 172.21.15.53:0/746201811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80d0750 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 --2- 172.21.15.53:0/746201811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc80d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 -- 172.21.15.53:0/746201811 >> 172.21.15.53:0/746201811 conn(0x7fafc809e370 msgr2=0x7fafc80d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 -- 172.21.15.53:0/746201811 shutdown_connections 2024-05-10T10:35:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.189+0000 7fafce9c8640 1 -- 172.21.15.53:0/746201811 wait complete. 2024-05-10T10:35:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.191+0000 7fafce9c8640 1 Processor -- start 2024-05-10T10:35:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.191+0000 7fafce9c8640 1 -- start start 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc81f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc80d5cf0 0x7fafc80ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80d0c90 con 0x7fafc815f6e0 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80b2350 con 0x7fafc80d5cf0 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fafc80b3b60 con 0x7fafc80d3680 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50352/0 (socket says 172.21.15.53:50352) 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 -- 172.21.15.53:0/145098970 learned_addr learned my addr 172.21.15.53:0/145098970 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafc77fe640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc80d5cf0 0x7fafc80ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 -- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 msgr2=0x7fafc81f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafc7fff640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc81f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc81f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 -- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc80d5cf0 msgr2=0x7fafc80ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc80d5cf0 0x7fafc80ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 -- 172.21.15.53:0/145098970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc80cd360 con 0x7fafc815f6e0 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafc77fe640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc80d5cf0 0x7fafc80ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafc7fff640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc81f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafccf65640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80c9a90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fafc0001560 tx=0x7fafc000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafc4ff9640 1 -- 172.21.15.53:0/145098970 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0018070 con 0x7fafc815f6e0 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafc4ff9640 1 -- 172.21.15.53:0/145098970 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc000a640 con 0x7fafc815f6e0 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc80cd640 con 0x7fafc815f6e0 2024-05-10T10:35:49.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafc4ff9640 1 -- 172.21.15.53:0/145098970 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc000aa10 con 0x7fafc815f6e0 2024-05-10T10:35:49.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.192+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc80cdc00 con 0x7fafc815f6e0 2024-05-10T10:35:49.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.193+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc8068d80 con 0x7fafc815f6e0 2024-05-10T10:35:49.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.194+0000 7fafc4ff9640 1 -- 172.21.15.53:0/145098970 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fafc001b070 con 0x7fafc815f6e0 2024-05-10T10:35:49.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.195+0000 7fafc4ff9640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fafb4076300 0x7fafb40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:49.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.195+0000 7fafc4ff9640 1 -- 172.21.15.53:0/145098970 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7fafc0096a40 con 0x7fafc815f6e0 2024-05-10T10:35:49.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.195+0000 7fafc7fff640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fafb4076300 0x7fafb40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:49.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.195+0000 7fafc7fff640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fafb4076300 0x7fafb40787c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006fd0 tx=0x7fafb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:49.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.199+0000 7fafc4ff9640 1 -- 172.21.15.53:0/145098970 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fafc0060550 con 0x7fafc815f6e0 2024-05-10T10:35:49.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:49.392+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fafc80d3460 con 0x7fafc815f6e0 2024-05-10T10:35:50.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.350+0000 7fafc4ff9640 1 -- 172.21.15.53:0/145098970 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) ==== 138+0+0 (secure 0 0 0) 0x7fafc005fe50 con 0x7fafc815f6e0 2024-05-10T10:35:50.351 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:35:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fafb4076300 msgr2=0x7fafb40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fafb4076300 0x7fafb40787c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006fd0 tx=0x7fafb8008040 comp rx=0 tx=0).stop 2024-05-10T10:35:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 msgr2=0x7fafc80c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80c9a90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fafc0001560 tx=0x7fafc000cc70 comp rx=0 tx=0).stop 2024-05-10T10:35:50.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 shutdown_connections 2024-05-10T10:35:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fafb4076300 0x7fafb40787c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafc815f6e0 0x7fafc80c9a90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafc80d5cf0 0x7fafc80ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 --2- 172.21.15.53:0/145098970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafc80d3680 0x7fafc81f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 >> 172.21.15.53:0/145098970 conn(0x7fafc809e370 msgr2=0x7fafc806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 shutdown_connections 2024-05-10T10:35:50.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.353+0000 7fafce9c8640 1 -- 172.21.15.53:0/145098970 wait complete. 2024-05-10T10:35:50.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:35:50.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.486+0000 7f3864332640 1 Processor -- start 2024-05-10T10:35:50.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.487+0000 7f3864332640 1 -- start start 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.487+0000 7f3864332640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f385c15f410 0x7f385c0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.487+0000 7f3864332640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0d0f10 0x7f385c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.487+0000 7f3864332640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f385c0b54d0 0x7f385c0d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.487+0000 7f3864332640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f385c0d2800 con 0x7f385c15f410 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.487+0000 7f3864332640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f385c0d2ae0 con 0x7f385c0b54d0 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.487+0000 7f3864332640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f385c0d01d0 con 0x7f385c0d0f10 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f385c15f410 0x7f385c0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:50.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f385c15f410 0x7f385c0d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50354/0 (socket says 172.21.15.53:50354) 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 -- 172.21.15.53:0/794220123 learned_addr learned my addr 172.21.15.53:0/794220123 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38628cf640 1 --2- 172.21.15.53:0/794220123 >> v2:172.21.15.135:3300/0 conn(0x7f385c0b54d0 0x7f385c0d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38618cd640 1 --2- 172.21.15.53:0/794220123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0d0f10 0x7f385c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 -- 172.21.15.53:0/794220123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0d0f10 msgr2=0x7f385c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 --2- 172.21.15.53:0/794220123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0d0f10 0x7f385c0b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 -- 172.21.15.53:0/794220123 >> v2:172.21.15.135:3300/0 conn(0x7f385c0b54d0 msgr2=0x7f385c0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 --2- 172.21.15.53:0/794220123 >> v2:172.21.15.135:3300/0 conn(0x7f385c0b54d0 0x7f385c0d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38620ce640 1 -- 172.21.15.53:0/794220123 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c0d04b0 con 0x7f385c15f410 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.488+0000 7f38628cf640 1 --2- 172.21.15.53:0/794220123 >> v2:172.21.15.135:3300/0 conn(0x7f385c0b54d0 0x7f385c0d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.489+0000 7f38620ce640 1 --2- 172.21.15.53:0/794220123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c15f410 0x7f385c0d09d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f385c06c840 tx=0x7f385802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd6da1031b4feb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.489+0000 7f38610cc640 1 -- 172.21.15.53:0/794220123 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858037070 con 0x7f385c15f410 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.489+0000 7f38610cc640 1 -- 172.21.15.53:0/794220123 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3858032440 con 0x7f385c15f410 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.489+0000 7f38610cc640 1 -- 172.21.15.53:0/794220123 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858032810 con 0x7f385c15f410 2024-05-10T10:35:50.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.489+0000 7f3864332640 1 -- 172.21.15.53:0/794220123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c15f410 msgr2=0x7f385c0d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.489+0000 7f3864332640 1 --2- 172.21.15.53:0/794220123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c15f410 0x7f385c0d09d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f385c06c840 tx=0x7f385802a140 comp rx=0 tx=0).stop 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.490+0000 7f3864332640 1 -- 172.21.15.53:0/794220123 shutdown_connections 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.490+0000 7f3864332640 1 --2- 172.21.15.53:0/794220123 >> v2:172.21.15.135:3300/0 conn(0x7f385c0b54d0 0x7f385c0d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.490+0000 7f3864332640 1 --2- 172.21.15.53:0/794220123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0d0f10 0x7f385c0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.490+0000 7f3864332640 1 --2- 172.21.15.53:0/794220123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c15f410 0x7f385c0d09d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.490+0000 7f3864332640 1 -- 172.21.15.53:0/794220123 >> 172.21.15.53:0/794220123 conn(0x7f385c08ed30 msgr2=0x7f385c09e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.490+0000 7f3864332640 1 -- 172.21.15.53:0/794220123 shutdown_connections 2024-05-10T10:35:50.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.490+0000 7f3864332640 1 -- 172.21.15.53:0/794220123 wait complete. 2024-05-10T10:35:50.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.491+0000 7f3864332640 1 Processor -- start 2024-05-10T10:35:50.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.491+0000 7f3864332640 1 -- start start 2024-05-10T10:35:50.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.491+0000 7f3864332640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0b54d0 0x7f385c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.491+0000 7f3864332640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c0d0f10 0x7f385c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3864332640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c15f410 0x7f385c0cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3864332640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f385c0cffb0 con 0x7f385c0d0f10 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3864332640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f385c0cfe30 con 0x7f385c15f410 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3864332640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f385c0d3c10 con 0x7f385c0b54d0 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c0d0f10 0x7f385c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c0d0f10 0x7f385c152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50362/0 (socket says 172.21.15.53:50362) 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 -- 172.21.15.53:0/2218260637 learned_addr learned my addr 172.21.15.53:0/2218260637 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38628cf640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c15f410 0x7f385c0cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38620ce640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0b54d0 0x7f385c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 -- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0b54d0 msgr2=0x7f385c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0b54d0 0x7f385c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 -- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c15f410 msgr2=0x7f385c0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:50.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c15f410 0x7f385c0cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 -- 172.21.15.53:0/2218260637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c0cd340 con 0x7f385c0d0f10 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f38618cd640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c0d0f10 0x7f385c152640 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c7f0 tx=0x7f384c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3852ffd640 1 -- 172.21.15.53:0/2218260637 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c015070 con 0x7f385c0d0f10 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3852ffd640 1 -- 172.21.15.53:0/2218260637 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384c004240 con 0x7f385c0d0f10 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c0cdb40 con 0x7f385c0d0f10 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3852ffd640 1 -- 172.21.15.53:0/2218260637 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c007620 con 0x7f385c0d0f10 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.492+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c0ce3e0 con 0x7f385c0d0f10 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.493+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c06e420 con 0x7f385c0d0f10 2024-05-10T10:35:50.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.494+0000 7f3852ffd640 1 -- 172.21.15.53:0/2218260637 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f384c0079d0 con 0x7f385c0d0f10 2024-05-10T10:35:50.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.494+0000 7f3852ffd640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f38400762d0 0x7f3840078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:50.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.495+0000 7f3852ffd640 1 -- 172.21.15.53:0/2218260637 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5619+0+0 (secure 0 0 0) 0x7f384c097a10 con 0x7f385c0d0f10 2024-05-10T10:35:50.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.495+0000 7f38620ce640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f38400762d0 0x7f3840078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:50.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.495+0000 7f38620ce640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f38400762d0 0x7f3840078790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3858034000 tx=0x7f385802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:50.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.499+0000 7f3852ffd640 1 -- 172.21.15.53:0/2218260637 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f385c06e420 con 0x7f385c0d0f10 2024-05-10T10:35:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:50.691+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f385c06a220 con 0x7f385c0d0f10 2024-05-10T10:35:51.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.352+0000 7f3852ffd640 1 -- 172.21.15.53:0/2218260637 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) ==== 170+0+0 (secure 0 0 0) 0x7f384c061290 con 0x7f385c0d0f10 2024-05-10T10:35:51.354 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:35:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f38400762d0 msgr2=0x7f3840078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f38400762d0 0x7f3840078790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3858034000 tx=0x7f385802e000 comp rx=0 tx=0).stop 2024-05-10T10:35:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c0d0f10 msgr2=0x7f385c152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c0d0f10 0x7f385c152640 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c7f0 tx=0x7f384c00ccb0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 shutdown_connections 2024-05-10T10:35:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f38400762d0 0x7f3840078790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f385c15f410 0x7f385c0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f385c0d0f10 0x7f385c152640 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 --2- 172.21.15.53:0/2218260637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f385c0b54d0 0x7f385c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 >> 172.21.15.53:0/2218260637 conn(0x7f385c08ed30 msgr2=0x7f385c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 shutdown_connections 2024-05-10T10:35:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.355+0000 7f3864332640 1 -- 172.21.15.53:0/2218260637 wait complete. 2024-05-10T10:35:51.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:35:51.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.494+0000 7f5ac7cfc640 1 Processor -- start 2024-05-10T10:35:51.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.494+0000 7f5ac7cfc640 1 -- start start 2024-05-10T10:35:51.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac7cfc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:51.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac7cfc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5ac00cd500 0x7f5ac00d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:51.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac7cfc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac006e720 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac7cfc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac006de80 con 0x7f5ac00cd500 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac7cfc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac006e160 con 0x7f5ac00cffd0 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac5297640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac5297640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac00b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:35868/0 (socket says 172.21.15.53:35868) 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.495+0000 7f5ac5297640 1 -- 172.21.15.53:0/3924407488 learned_addr learned my addr 172.21.15.53:0/3924407488 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.496+0000 7f5ac6299640 1 --2- 172.21.15.53:0/3924407488 >> v2:172.21.15.135:3300/0 conn(0x7f5ac00cd500 0x7f5ac00d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.496+0000 7f5ac5297640 1 -- 172.21.15.53:0/3924407488 >> v2:172.21.15.135:3300/0 conn(0x7f5ac00cd500 msgr2=0x7f5ac00d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.496+0000 7f5ac5297640 1 --2- 172.21.15.53:0/3924407488 >> v2:172.21.15.135:3300/0 conn(0x7f5ac00cd500 0x7f5ac00d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.496+0000 7f5ac5297640 1 -- 172.21.15.53:0/3924407488 >> v1:172.21.15.135:6789/0 conn(0x7f5ac00cf3e0 legacy=0x7f5ac00cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.496+0000 7f5ac4a96640 1 -- 172.21.15.53:0/3924407488 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983539107 0 0) 0x7f5ac006e720 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.496+0000 7f5ac5297640 1 -- 172.21.15.53:0/3924407488 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac006e440 con 0x7f5ac00cffd0 2024-05-10T10:35:51.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac5297640 1 --2- 172.21.15.53:0/3924407488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac00b3de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009870 tx=0x7f5ab002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf040edaeada2ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac4a96640 1 -- 172.21.15.53:0/3924407488 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0037070 con 0x7f5ac00cffd0 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac4a96640 1 -- 172.21.15.53:0/3924407488 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0032440 con 0x7f5ac00cffd0 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac4a96640 1 -- 172.21.15.53:0/3924407488 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0032810 con 0x7f5ac00cffd0 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/3924407488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 msgr2=0x7f5ac00b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/3924407488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac00b3de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009870 tx=0x7f5ab002a270 comp rx=0 tx=0).stop 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/3924407488 shutdown_connections 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/3924407488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac00b3de0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/3924407488 >> v2:172.21.15.135:3300/0 conn(0x7f5ac00cd500 0x7f5ac00d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.497+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/3924407488 >> 172.21.15.53:0/3924407488 conn(0x7f5ac00d2eb0 msgr2=0x7f5ac00d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.498+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/3924407488 shutdown_connections 2024-05-10T10:35:51.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.498+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/3924407488 wait complete. 2024-05-10T10:35:51.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 Processor -- start 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 -- start start 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ac00cd500 0x7f5ac00b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac00cf3e0 0x7f5ac014f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac01609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00d16c0 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00b4510 con 0x7f5ac00cd500 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.499+0000 7f5ac7cfc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac00d1840 con 0x7f5ac00cffd0 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac00cf3e0 0x7f5ac014f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac00cf3e0 0x7f5ac014f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50372/0 (socket says 172.21.15.53:50372) 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 -- 172.21.15.53:0/2374265958 learned_addr learned my addr 172.21.15.53:0/2374265958 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac6299640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac01609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5a98640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ac00cd500 0x7f5ac00b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 -- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 msgr2=0x7f5ac01609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:51.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac01609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 -- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ac00cd500 msgr2=0x7f5ac00b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ac00cd500 0x7f5ac00b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 -- 172.21.15.53:0/2374265958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac00c8920 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac5297640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac00cf3e0 0x7f5ac014f980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0000c00 tx=0x7f5ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ab67fc640 1 -- 172.21.15.53:0/2374265958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0037040 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac014fec0 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.500+0000 7f5ab67fc640 1 -- 172.21.15.53:0/2374265958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0042070 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.501+0000 7f5ab67fc640 1 -- 172.21.15.53:0/2374265958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0030880 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.501+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01f5410 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.501+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac006a220 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.502+0000 7f5ab67fc640 1 -- 172.21.15.53:0/2374265958 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ab0030a20 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.503+0000 7f5ab67fc640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5aa40762d0 0x7f5aa4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:51.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.503+0000 7f5ab67fc640 1 -- 172.21.15.53:0/2374265958 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6001+0+0 (secure 0 0 0) 0x7f5ab00b7f70 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.503+0000 7f5ac5a98640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5aa40762d0 0x7f5aa4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:51.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.504+0000 7f5ac5a98640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5aa40762d0 0x7f5aa4078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5abc000f60 tx=0x7f5abc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:51.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.507+0000 7f5ab67fc640 1 -- 172.21.15.53:0/2374265958 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5ac014fec0 con 0x7f5ac00cf3e0 2024-05-10T10:35:51.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:51.740+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5ac00d2a50 con 0x7f5ac00cf3e0 2024-05-10T10:35:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 9 and data pool 10 2024-05-10T10:35:52.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.382+0000 7f5ab67fc640 1 -- 172.21.15.53:0/2374265958 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f5ac00c8920 con 0x7f5ac00cf3e0 2024-05-10T10:35:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.388+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5aa40762d0 msgr2=0x7f5aa4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.388+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5aa40762d0 0x7f5aa4078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5abc000f60 tx=0x7f5abc009040 comp rx=0 tx=0).stop 2024-05-10T10:35:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.388+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac00cf3e0 msgr2=0x7f5ac014f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.388+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac00cf3e0 0x7f5ac014f980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0000c00 tx=0x7f5ab00046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.391+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 shutdown_connections 2024-05-10T10:35:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.391+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5aa40762d0 0x7f5aa4078790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.391+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ac00cffd0 0x7f5ac01609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.391+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac00cf3e0 0x7f5ac014f980 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.391+0000 7f5ac7cfc640 1 --2- 172.21.15.53:0/2374265958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ac00cd500 0x7f5ac00b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.391+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 >> 172.21.15.53:0/2374265958 conn(0x7f5ac00d2eb0 msgr2=0x7f5ac00cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:52.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.395+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 shutdown_connections 2024-05-10T10:35:52.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.395+0000 7f5ac7cfc640 1 -- 172.21.15.53:0/2374265958 wait complete. 2024-05-10T10:35:52.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:35:52.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.522+0000 7f9ee01c1640 1 Processor -- start 2024-05-10T10:35:52.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.522+0000 7f9ee01c1640 1 -- start start 2024-05-10T10:35:52.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ee01c1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ed806c990 0x7f9ed8163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ee01c1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed8070bd0 0x7f9ed815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ee01c1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed8163ba0 con 0x7f9ed815f620 2024-05-10T10:35:52.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ee01c1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed806cd70 con 0x7f9ed806c990 2024-05-10T10:35:52.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ee01c1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed8160190 con 0x7f9ed8070bd0 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ede75e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed8070bd0 0x7f9ed815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9eddf5d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ed806c990 0x7f9ed8163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9edd75c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9ed815f620 0x7f9ed80704c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:60240/0 (socket says 172.21.15.53:60240) 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ede75e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed8070bd0 0x7f9ed815fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:35884/0 (socket says 172.21.15.53:35884) 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9edd75c640 1 -- 172.21.15.53:0/754441895 learned_addr learned my addr 172.21.15.53:0/754441895 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ede75e640 1 -- 172.21.15.53:0/754441895 >> v2:172.21.15.135:3300/0 conn(0x7f9ed806c990 msgr2=0x7f9ed8163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ede75e640 1 --2- 172.21.15.53:0/754441895 >> v2:172.21.15.135:3300/0 conn(0x7f9ed806c990 0x7f9ed8163660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.523+0000 7f9ede75e640 1 -- 172.21.15.53:0/754441895 >> v1:172.21.15.135:6789/0 conn(0x7f9ed815f620 legacy=0x7f9ed80704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.524+0000 7f9ede75e640 1 -- 172.21.15.53:0/754441895 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed8160470 con 0x7f9ed8070bd0 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.524+0000 7f9ede75e640 1 --2- 172.21.15.53:0/754441895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed8070bd0 0x7f9ed815fab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5efdb0b2c6c214c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.524+0000 7f9edcf5b640 1 -- 172.21.15.53:0/754441895 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0037070 con 0x7f9ed8070bd0 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.524+0000 7f9edcf5b640 1 -- 172.21.15.53:0/754441895 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed0032440 con 0x7f9ed8070bd0 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.524+0000 7f9edcf5b640 1 -- 172.21.15.53:0/754441895 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0032810 con 0x7f9ed8070bd0 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 -- 172.21.15.53:0/754441895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed8070bd0 msgr2=0x7f9ed815fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/754441895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed8070bd0 0x7f9ed815fab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed002a180 comp rx=0 tx=0).stop 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 -- 172.21.15.53:0/754441895 shutdown_connections 2024-05-10T10:35:52.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/754441895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed8070bd0 0x7f9ed815fab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/754441895 >> v2:172.21.15.135:3300/0 conn(0x7f9ed806c990 0x7f9ed8163660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 -- 172.21.15.53:0/754441895 >> 172.21.15.53:0/754441895 conn(0x7f9ed80b3980 msgr2=0x7f9ed80b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 -- 172.21.15.53:0/754441895 shutdown_connections 2024-05-10T10:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.525+0000 7f9ee01c1640 1 -- 172.21.15.53:0/754441895 wait complete. 2024-05-10T10:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.526+0000 7f9ee01c1640 1 Processor -- start 2024-05-10T10:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.526+0000 7f9ee01c1640 1 -- start start 2024-05-10T10:35:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9ee01c1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed806c990 0x7f9ed815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9ee01c1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed8070bd0 0x7f9ed81e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9ee01c1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed815f620 0x7f9ed81496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9ee01c1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8073e70 con 0x7f9ed806c990 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9ee01c1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8163140 con 0x7f9ed8070bd0 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9ee01c1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed80cf920 con 0x7f9ed815f620 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9eddf5d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed806c990 0x7f9ed815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9eddf5d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed806c990 0x7f9ed815d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50388/0 (socket says 172.21.15.53:50388) 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9eddf5d640 1 -- 172.21.15.53:0/1845718106 learned_addr learned my addr 172.21.15.53:0/1845718106 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9edd75c640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed8070bd0 0x7f9ed81e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.527+0000 7f9ede75e640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed815f620 0x7f9ed81496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9eddf5d640 1 -- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed815f620 msgr2=0x7f9ed81496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9eddf5d640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed815f620 0x7f9ed81496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9eddf5d640 1 -- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed8070bd0 msgr2=0x7f9ed81e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9eddf5d640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed8070bd0 0x7f9ed81e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9eddf5d640 1 -- 172.21.15.53:0/1845718106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed815d8c0 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9eddf5d640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed806c990 0x7f9ed815d1c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009db0 tx=0x7f9ed400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9ece7fc640 1 -- 172.21.15.53:0/1845718106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4018020 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9ece7fc640 1 -- 172.21.15.53:0/1845718106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed4014480 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed815dba0 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9ece7fc640 1 -- 172.21.15.53:0/1845718106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4014850 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.528+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed815a9a0 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.529+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed806a220 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.530+0000 7f9ece7fc640 1 -- 172.21.15.53:0/1845718106 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9ed4005d10 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.530+0000 7f9ece7fc640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9ebc0762d0 0x7f9ebc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.530+0000 7f9ece7fc640 1 -- 172.21.15.53:0/1845718106 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f9ed40980d0 con 0x7f9ed806c990 2024-05-10T10:35:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.530+0000 7f9edd75c640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9ebc0762d0 0x7f9ebc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.531+0000 7f9edd75c640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9ebc0762d0 0x7f9ebc078790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8149ff0 tx=0x7f9ec8007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.535+0000 7f9ece7fc640 1 -- 172.21.15.53:0/1845718106 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9ed4022070 con 0x7f9ed806c990 2024-05-10T10:35:52.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.733+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ed8073540 con 0x7f9ed806c990 2024-05-10T10:35:52.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.734+0000 7f9ece7fc640 1 -- 172.21.15.53:0/1845718106 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16897 (secure 0 0 0) 0x7f9ed4010040 con 0x7f9ed806c990 2024-05-10T10:35:52.735 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:52.735 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:35:51.741264+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:49.396339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:50.695199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9ebc0762d0 msgr2=0x7f9ebc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9ebc0762d0 0x7f9ebc078790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8149ff0 tx=0x7f9ec8007e80 comp rx=0 tx=0).stop 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed806c990 msgr2=0x7f9ed815d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed806c990 0x7f9ed815d1c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009db0 tx=0x7f9ed400c490 comp rx=0 tx=0).stop 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 shutdown_connections 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9ebc0762d0 0x7f9ebc078790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed815f620 0x7f9ed81496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed8070bd0 0x7f9ed81e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 --2- 172.21.15.53:0/1845718106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ed806c990 0x7f9ed815d1c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 >> 172.21.15.53:0/1845718106 conn(0x7f9ed80b3980 msgr2=0x7f9ed80c6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:52.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 shutdown_connections 2024-05-10T10:35:52.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.736+0000 7f9ee01c1640 1 -- 172.21.15.53:0/1845718106 wait complete. 2024-05-10T10:35:52.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:35:52.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.851+0000 7f32b1389640 1 Processor -- start 2024-05-10T10:35:52.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.851+0000 7f32b1389640 1 -- start start 2024-05-10T10:35:52.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32b1389640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32ac15ce60 0x7f32ac0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32b1389640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f32ac1611e0 0x7f32ac15d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32b1389640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ac0c6a60 con 0x7f32ac0c65f0 2024-05-10T10:35:52.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32b1389640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f32ac0cd090 con 0x7f32ac15ce60 2024-05-10T10:35:52.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32b1389640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f32ac0cd370 con 0x7f32ac1611e0 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32aaffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32ac15ce60 0x7f32ac0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32aaffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32ac15ce60 0x7f32ac0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56466/0 (socket says 172.21.15.53:56466) 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32aaffd640 1 -- 172.21.15.53:0/1620374855 learned_addr learned my addr 172.21.15.53:0/1620374855 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32aaffd640 1 -- 172.21.15.53:0/1620374855 >> v2:172.21.15.135:3300/0 conn(0x7f32ac1611e0 msgr2=0x7f32ac15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32ab7fe640 1 --2- 172.21.15.53:0/1620374855 >> v2:172.21.15.135:3300/0 conn(0x7f32ac1611e0 0x7f32ac15d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32aaffd640 1 --2- 172.21.15.53:0/1620374855 >> v2:172.21.15.135:3300/0 conn(0x7f32ac1611e0 0x7f32ac15d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32aaffd640 1 -- 172.21.15.53:0/1620374855 >> v1:172.21.15.135:6789/0 conn(0x7f32ac0c65f0 legacy=0x7f32ac160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32aaffd640 1 -- 172.21.15.53:0/1620374855 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac0cd650 con 0x7f32ac15ce60 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.852+0000 7f32ab7fe640 1 --2- 172.21.15.53:0/1620374855 >> v2:172.21.15.135:3300/0 conn(0x7f32ac1611e0 0x7f32ac15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:52.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.853+0000 7f32aaffd640 1 --2- 172.21.15.53:0/1620374855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac15ce60 0x7f32ac0cb300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0cc8a0 tx=0x7f329c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a45e6b2fd43fdff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.853+0000 7f32a9ffb640 1 -- 172.21.15.53:0/1620374855 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c037070 con 0x7f32ac15ce60 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.853+0000 7f32a9ffb640 1 -- 172.21.15.53:0/1620374855 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c032440 con 0x7f32ac15ce60 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.853+0000 7f32a9ffb640 1 -- 172.21.15.53:0/1620374855 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c032810 con 0x7f32ac15ce60 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 -- 172.21.15.53:0/1620374855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac15ce60 msgr2=0x7f32ac0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 --2- 172.21.15.53:0/1620374855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac15ce60 0x7f32ac0cb300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0cc8a0 tx=0x7f329c02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 -- 172.21.15.53:0/1620374855 shutdown_connections 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 --2- 172.21.15.53:0/1620374855 >> v2:172.21.15.135:3300/0 conn(0x7f32ac1611e0 0x7f32ac15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 --2- 172.21.15.53:0/1620374855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac15ce60 0x7f32ac0cb300 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 -- 172.21.15.53:0/1620374855 >> 172.21.15.53:0/1620374855 conn(0x7f32ac0b2900 msgr2=0x7f32ac0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 -- 172.21.15.53:0/1620374855 shutdown_connections 2024-05-10T10:35:52.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.854+0000 7f32b1389640 1 -- 172.21.15.53:0/1620374855 wait complete. 2024-05-10T10:35:52.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.855+0000 7f32b1389640 1 Processor -- start 2024-05-10T10:35:52.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.855+0000 7f32b1389640 1 -- start start 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32b1389640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac0c65f0 0x7f32ac06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32b1389640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32ac15ce60 0x7f32ac152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32b1389640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32ac1611e0 0x7f32ac0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32b1389640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0cde50 con 0x7f32ac0c65f0 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32b1389640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0ccd60 con 0x7f32ac1611e0 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32b1389640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac0cdfd0 con 0x7f32ac15ce60 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac0c65f0 0x7f32ac06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac0c65f0 0x7f32ac06e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56472/0 (socket says 172.21.15.53:56472) 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 -- 172.21.15.53:0/3888085165 learned_addr learned my addr 172.21.15.53:0/3888085165 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aa7fc640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32ac15ce60 0x7f32ac152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32ab7fe640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32ac1611e0 0x7f32ac0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 -- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32ac15ce60 msgr2=0x7f32ac152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32ac15ce60 0x7f32ac152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 -- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32ac1611e0 msgr2=0x7f32ac0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32ac1611e0 0x7f32ac0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.856+0000 7f32aaffd640 1 -- 172.21.15.53:0/3888085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac0b0ff0 con 0x7f32ac0c65f0 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.857+0000 7f32aaffd640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac0c65f0 0x7f32ac06e930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f329c000c00 tx=0x7f329c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.857+0000 7f32937fe640 1 -- 172.21.15.53:0/3888085165 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c041020 con 0x7f32ac0c65f0 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.857+0000 7f32937fe640 1 -- 172.21.15.53:0/3888085165 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c030970 con 0x7f32ac0c65f0 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.857+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac0b12d0 con 0x7f32ac0c65f0 2024-05-10T10:35:52.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.857+0000 7f32937fe640 1 -- 172.21.15.53:0/3888085165 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c037040 con 0x7f32ac0c65f0 2024-05-10T10:35:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.857+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac0b1890 con 0x7f32ac0c65f0 2024-05-10T10:35:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.858+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac06a220 con 0x7f32ac0c65f0 2024-05-10T10:35:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.858+0000 7f32937fe640 1 -- 172.21.15.53:0/3888085165 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f329c044020 con 0x7f32ac0c65f0 2024-05-10T10:35:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.859+0000 7f32937fe640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32980762d0 0x7f3298078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.859+0000 7f32937fe640 1 -- 172.21.15.53:0/3888085165 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f329c0b8140 con 0x7f32ac0c65f0 2024-05-10T10:35:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.859+0000 7f32aa7fc640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32980762d0 0x7f3298078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.860+0000 7f32aa7fc640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32980762d0 0x7f3298078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b2860 tx=0x7f32a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:52.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:52.863+0000 7f32937fe640 1 -- 172.21.15.53:0/3888085165 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f329c04b030 con 0x7f32ac0c65f0 2024-05-10T10:35:53.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.057+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32ac0c7300 con 0x7f32ac0c65f0 2024-05-10T10:35:53.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.058+0000 7f32937fe640 1 -- 172.21.15.53:0/3888085165 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16897 (secure 0 0 0) 0x7f329c0819a0 con 0x7f32ac0c65f0 2024-05-10T10:35:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:53.059 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:35:51.741264+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:49.396339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:50.695199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:35:53.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32980762d0 msgr2=0x7f3298078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:53.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32980762d0 0x7f3298078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b2860 tx=0x7f32a0009040 comp rx=0 tx=0).stop 2024-05-10T10:35:53.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac0c65f0 msgr2=0x7f32ac06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:53.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac0c65f0 0x7f32ac06e930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f329c000c00 tx=0x7f329c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 shutdown_connections 2024-05-10T10:35:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32980762d0 0x7f3298078790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32ac1611e0 0x7f32ac0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32ac15ce60 0x7f32ac152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 --2- 172.21.15.53:0/3888085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32ac0c65f0 0x7f32ac06e930 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 >> 172.21.15.53:0/3888085165 conn(0x7f32ac0b2900 msgr2=0x7f32ac073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 shutdown_connections 2024-05-10T10:35:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.060+0000 7f32b1389640 1 -- 172.21.15.53:0/3888085165 wait complete. 2024-05-10T10:35:53.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:35:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.178+0000 7f764c933640 1 Processor -- start 2024-05-10T10:35:53.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.178+0000 7f764c933640 1 -- start start 2024-05-10T10:35:53.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f764c933640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f76480cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:53.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f764c933640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f76480d2190 0x7f76480cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:53.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f764c933640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764806e720 con 0x7f76480cd500 2024-05-10T10:35:53.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f764c933640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f764806de80 con 0x7f76480d2190 2024-05-10T10:35:53.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f764c933640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f764806e160 con 0x7f76480cf3e0 2024-05-10T10:35:53.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f7645d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f76480cd500 0x7f76480b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48900/0 (socket says 172.21.15.53:48900) 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f7645d9b640 1 -- 172.21.15.53:0/2646733435 learned_addr learned my addr 172.21.15.53:0/2646733435 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f7646d9d640 1 --2- 172.21.15.53:0/2646733435 >> v2:172.21.15.53:3300/0 conn(0x7f76480d2190 0x7f76480cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f764659c640 1 --2- 172.21.15.53:0/2646733435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f76480cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f7646d9d640 1 -- 172.21.15.53:0/2646733435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 msgr2=0x7f76480cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f7646d9d640 1 --2- 172.21.15.53:0/2646733435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f76480cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f7646d9d640 1 -- 172.21.15.53:0/2646733435 >> v1:172.21.15.53:6789/0 conn(0x7f76480cd500 legacy=0x7f76480b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f7646d9d640 1 -- 172.21.15.53:0/2646733435 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764806e440 con 0x7f76480d2190 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.179+0000 7f764559a640 1 -- 172.21.15.53:0/2646733435 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160732716 0 0) 0x7f764806e720 con 0x7f76480cd500 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.180+0000 7f7646d9d640 1 --2- 172.21.15.53:0/2646733435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f76480cfe00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7640009d90 tx=0x7f764002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f870decb92bb618 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.180+0000 7f764559a640 1 -- 172.21.15.53:0/2646733435 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640007740 con 0x7f76480d2190 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.180+0000 7f764559a640 1 -- 172.21.15.53:0/2646733435 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76400078e0 con 0x7f76480d2190 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.180+0000 7f764559a640 1 -- 172.21.15.53:0/2646733435 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640007cb0 con 0x7f76480d2190 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 -- 172.21.15.53:0/2646733435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 msgr2=0x7f76480cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:53.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 --2- 172.21.15.53:0/2646733435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f76480cfe00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7640009d90 tx=0x7f764002a660 comp rx=0 tx=0).stop 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 -- 172.21.15.53:0/2646733435 shutdown_connections 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 --2- 172.21.15.53:0/2646733435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f76480cfe00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 --2- 172.21.15.53:0/2646733435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f76480cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 -- 172.21.15.53:0/2646733435 >> 172.21.15.53:0/2646733435 conn(0x7f76480d2eb0 msgr2=0x7f76480d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 -- 172.21.15.53:0/2646733435 shutdown_connections 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.181+0000 7f764c933640 1 -- 172.21.15.53:0/2646733435 wait complete. 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.182+0000 7f764c933640 1 Processor -- start 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764c933640 1 -- start start 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764c933640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76480cd500 0x7f76480ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764c933640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f7648158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764c933640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f7648070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764c933640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76480b34e0 con 0x7f76480d2190 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764c933640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76480b4510 con 0x7f76480cd500 2024-05-10T10:35:53.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764c933640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f76480b3660 con 0x7f76480cf3e0 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f7646d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f7648070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f7646d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f7648070d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56486/0 (socket says 172.21.15.53:56486) 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f7646d9d640 1 -- 172.21.15.53:0/3440269867 learned_addr learned my addr 172.21.15.53:0/3440269867 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f7645d9b640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f7648158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.183+0000 7f764659c640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76480cd500 0x7f76480ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7646d9d640 1 -- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 msgr2=0x7f7648158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7646d9d640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f7648158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7646d9d640 1 -- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76480cd500 msgr2=0x7f76480ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7646d9d640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76480cd500 0x7f76480ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:53.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7646d9d640 1 -- 172.21.15.53:0/3440269867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481f80b0 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7646d9d640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f7648070d70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7640009eb0 tx=0x7f7640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7636ffd640 1 -- 172.21.15.53:0/3440269867 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764003f070 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7636ffd640 1 -- 172.21.15.53:0/3440269867 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7640031bc0 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481f8390 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f7636ffd640 1 -- 172.21.15.53:0/3440269867 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640009450 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.184+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481f8950 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.185+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764806a220 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.186+0000 7f7636ffd640 1 -- 172.21.15.53:0/3440269867 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7640044460 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.186+0000 7f7636ffd640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76280762d0 0x7f7628078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.186+0000 7f7636ffd640 1 -- 172.21.15.53:0/3440269867 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f76400b8d50 con 0x7f76480d2190 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.187+0000 7f764659c640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76280762d0 0x7f7628078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:53.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.187+0000 7f764659c640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76280762d0 0x7f7628078790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7638009b40 tx=0x7f7638009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:53.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.191+0000 7f7636ffd640 1 -- 172.21.15.53:0/3440269867 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f76400bc030 con 0x7f76480d2190 2024-05-10T10:35:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:53.423+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f76480751b0 con 0x7f76480d2190 2024-05-10T10:35:54.350 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:35:54.350+0000 7f9df95b9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:35:54.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.392+0000 7f7636ffd640 1 -- 172.21.15.53:0/3440269867 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7f7640081f60 con 0x7f76480d2190 2024-05-10T10:35:54.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76280762d0 msgr2=0x7f7628078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:54.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76280762d0 0x7f7628078790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7638009b40 tx=0x7f7638009190 comp rx=0 tx=0).stop 2024-05-10T10:35:54.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 msgr2=0x7f7648070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f7648070d70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7640009eb0 tx=0x7f7640004a40 comp rx=0 tx=0).stop 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 shutdown_connections 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76280762d0 0x7f7628078790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76480d2190 0x7f7648070d70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76480cf3e0 0x7f7648158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 --2- 172.21.15.53:0/3440269867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76480cd500 0x7f76480ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.395+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 >> 172.21.15.53:0/3440269867 conn(0x7f76480d2eb0 msgr2=0x7f76480705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.396+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 shutdown_connections 2024-05-10T10:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.396+0000 7f764c933640 1 -- 172.21.15.53:0/3440269867 wait complete. 2024-05-10T10:35:54.410 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T10:35:54.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:35:54.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.522+0000 7fb43c714640 1 Processor -- start 2024-05-10T10:35:54.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.522+0000 7fb43c714640 1 -- start start 2024-05-10T10:35:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb43c714640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4340b4050 0x7fb4340b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb43c714640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4340b44e0 con 0x7fb43415f330 2024-05-10T10:35:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb43c714640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4340b47c0 con 0x7fb4340b2c80 2024-05-10T10:35:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb43c714640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb4340b50f0 con 0x7fb4340b4050 2024-05-10T10:35:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb439caf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4340b4050 0x7fb4340b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb43a4b0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb43415f330 0x7fb4340d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48914/0 (socket says 172.21.15.53:48914) 2024-05-10T10:35:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb43a4b0640 1 -- 172.21.15.53:0/1899025243 learned_addr learned my addr 172.21.15.53:0/1899025243 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:54.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb439caf640 1 -- 172.21.15.53:0/1899025243 >> v1:172.21.15.135:6789/0 conn(0x7fb4340b2c80 legacy=0x7fb4340cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:54.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb439caf640 1 -- 172.21.15.53:0/1899025243 >> v1:172.21.15.53:6789/0 conn(0x7fb43415f330 legacy=0x7fb4340d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:54.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb4394ae640 1 -- 172.21.15.53:0/1899025243 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1626474787 0 0) 0x7fb4340b44e0 con 0x7fb43415f330 2024-05-10T10:35:54.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.523+0000 7fb439caf640 1 -- 172.21.15.53:0/1899025243 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340b4aa0 con 0x7fb4340b4050 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.524+0000 7fb439caf640 1 --2- 172.21.15.53:0/1899025243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b4050 0x7fb4340b5580 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097a0 tx=0x7fb42402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacd4b02ecbdb94 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.524+0000 7fb4394ae640 1 -- 172.21.15.53:0/1899025243 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424037070 con 0x7fb4340b4050 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.524+0000 7fb4394ae640 1 -- 172.21.15.53:0/1899025243 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb424032440 con 0x7fb4340b4050 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.524+0000 7fb4394ae640 1 -- 172.21.15.53:0/1899025243 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424032810 con 0x7fb4340b4050 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.525+0000 7fb43c714640 1 -- 172.21.15.53:0/1899025243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b4050 msgr2=0x7fb4340b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.525+0000 7fb43c714640 1 --2- 172.21.15.53:0/1899025243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b4050 0x7fb4340b5580 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097a0 tx=0x7fb42402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.525+0000 7fb43c714640 1 -- 172.21.15.53:0/1899025243 shutdown_connections 2024-05-10T10:35:54.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.525+0000 7fb43c714640 1 --2- 172.21.15.53:0/1899025243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b4050 0x7fb4340b5580 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.525+0000 7fb43c714640 1 -- 172.21.15.53:0/1899025243 >> 172.21.15.53:0/1899025243 conn(0x7fb4340cdc80 msgr2=0x7fb4340ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:54.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.525+0000 7fb43c714640 1 -- 172.21.15.53:0/1899025243 shutdown_connections 2024-05-10T10:35:54.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.525+0000 7fb43c714640 1 -- 172.21.15.53:0/1899025243 wait complete. 2024-05-10T10:35:54.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.526+0000 7fb43c714640 1 Processor -- start 2024-05-10T10:35:54.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.526+0000 7fb43c714640 1 -- start start 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43c714640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b2c80 0x7fb4340ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43c714640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb43415f330 0x7fb4340c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43c714640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4340c8d40 0x7fb4340ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43c714640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340b30b0 con 0x7fb4340b2c80 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43c714640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb43415f710 con 0x7fb4340c8d40 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43c714640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340b3230 con 0x7fb43415f330 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b2c80 0x7fb4340ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b2c80 0x7fb4340ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56518/0 (socket says 172.21.15.53:56518) 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 -- 172.21.15.53:0/2752768503 learned_addr learned my addr 172.21.15.53:0/2752768503 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb439caf640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb43415f330 0x7fb4340c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:54.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43acb1640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4340c8d40 0x7fb4340ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 -- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb43415f330 msgr2=0x7fb4340c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb43415f330 0x7fb4340c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 -- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4340c8d40 msgr2=0x7fb4340ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4340c8d40 0x7fb4340ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43a4b0640 1 -- 172.21.15.53:0/2752768503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340af9a0 con 0x7fb4340b2c80 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb439caf640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb43415f330 0x7fb4340c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.527+0000 7fb43acb1640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4340c8d40 0x7fb4340ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.528+0000 7fb43a4b0640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b2c80 0x7fb4340ae200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c460 tx=0x7fb43000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.528+0000 7fb42affd640 1 -- 172.21.15.53:0/2752768503 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4300074e0 con 0x7fb4340b2c80 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.528+0000 7fb42affd640 1 -- 172.21.15.53:0/2752768503 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4300047e0 con 0x7fb4340b2c80 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.528+0000 7fb42affd640 1 -- 172.21.15.53:0/2752768503 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430004bd0 con 0x7fb4340b2c80 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.528+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4340ae900 con 0x7fb4340b2c80 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.528+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4340aeec0 con 0x7fb4340b2c80 2024-05-10T10:35:54.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.529+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43406a220 con 0x7fb4340b2c80 2024-05-10T10:35:54.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.529+0000 7fb42affd640 1 -- 172.21.15.53:0/2752768503 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb430011070 con 0x7fb4340b2c80 2024-05-10T10:35:54.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.530+0000 7fb42affd640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb418076300 0x7fb4180787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:54.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.530+0000 7fb42affd640 1 -- 172.21.15.53:0/2752768503 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fb4300973a0 con 0x7fb4340b2c80 2024-05-10T10:35:54.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.530+0000 7fb439caf640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb418076300 0x7fb4180787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:54.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.531+0000 7fb439caf640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb418076300 0x7fb4180787c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb424034000 tx=0x7fb42402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:54.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.534+0000 7fb42affd640 1 -- 172.21.15.53:0/2752768503 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb430060b60 con 0x7fb4340b2c80 2024-05-10T10:35:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:54.765+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb4340b3a40 con 0x7fb4340b2c80 2024-05-10T10:35:55.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.393+0000 7fb42affd640 1 -- 172.21.15.53:0/2752768503 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fb430060460 con 0x7fb4340b2c80 2024-05-10T10:35:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb418076300 msgr2=0x7fb4180787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb418076300 0x7fb4180787c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb424034000 tx=0x7fb42402e000 comp rx=0 tx=0).stop 2024-05-10T10:35:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b2c80 msgr2=0x7fb4340ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b2c80 0x7fb4340ae200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c460 tx=0x7fb43000c920 comp rx=0 tx=0).stop 2024-05-10T10:35:55.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 shutdown_connections 2024-05-10T10:35:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb418076300 0x7fb4180787c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4340c8d40 0x7fb4340ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb43415f330 0x7fb4340c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 --2- 172.21.15.53:0/2752768503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4340b2c80 0x7fb4340ae200 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 >> 172.21.15.53:0/2752768503 conn(0x7fb4340cdc80 msgr2=0x7fb43415e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.396+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 shutdown_connections 2024-05-10T10:35:55.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.397+0000 7fb43c714640 1 -- 172.21.15.53:0/2752768503 wait complete. 2024-05-10T10:35:55.411 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:35:55.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:55.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.520+0000 7f8eb56e1640 1 Processor -- start 2024-05-10T10:35:55.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.520+0000 7f8eb56e1640 1 -- start start 2024-05-10T10:35:55.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.520+0000 7f8eb56e1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8eb00b8860 0x7f8eb006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.520+0000 7f8eb56e1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb015fc90 con 0x7f8eb00d0230 2024-05-10T10:35:55.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.520+0000 7f8eb56e1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb00b8c40 con 0x7f8eb00cfd80 2024-05-10T10:35:55.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.520+0000 7f8eb56e1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb006ee50 con 0x7f8eb00b8860 2024-05-10T10:35:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eae7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8eb00d0230 0x7f8eb00cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48930/0 (socket says 172.21.15.53:48930) 2024-05-10T10:35:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eae7fc640 1 -- 172.21.15.53:0/1624771518 learned_addr learned my addr 172.21.15.53:0/1624771518 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eaeffd640 1 --2- 172.21.15.53:0/1624771518 >> v2:172.21.15.135:3300/0 conn(0x7f8eb00b8860 0x7f8eb006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eadffb640 1 -- 172.21.15.53:0/1624771518 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1026506349 0 0) 0x7f8eb015fc90 con 0x7f8eb00d0230 2024-05-10T10:35:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eadffb640 1 -- 172.21.15.53:0/1624771518 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e98003520 con 0x7f8eb00d0230 2024-05-10T10:35:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eadffb640 1 -- 172.21.15.53:0/1624771518 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3567787473 0 0) 0x7f8e98003520 con 0x7f8eb00d0230 2024-05-10T10:35:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eaeffd640 1 -- 172.21.15.53:0/1624771518 >> v1:172.21.15.135:6789/0 conn(0x7f8eb00cfd80 legacy=0x7f8eb00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eaeffd640 1 -- 172.21.15.53:0/1624771518 >> v1:172.21.15.53:6789/0 conn(0x7f8eb00d0230 legacy=0x7f8eb00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.521+0000 7f8eaeffd640 1 -- 172.21.15.53:0/1624771518 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb00cd570 con 0x7f8eb00b8860 2024-05-10T10:35:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.522+0000 7f8eaeffd640 1 --2- 172.21.15.53:0/1624771518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00b8860 0x7f8eb006e520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00097a0 tx=0x7f8ea002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcef41b78bcdbb20 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.522+0000 7f8eadffb640 1 -- 172.21.15.53:0/1624771518 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0037070 con 0x7f8eb00b8860 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.522+0000 7f8eadffb640 1 -- 172.21.15.53:0/1624771518 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea002aad0 con 0x7f8eb00b8860 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.522+0000 7f8eadffb640 1 -- 172.21.15.53:0/1624771518 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4001670 con 0x7f8eb00b8860 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.523+0000 7f8eb56e1640 1 -- 172.21.15.53:0/1624771518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00b8860 msgr2=0x7f8eb006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.523+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/1624771518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00b8860 0x7f8eb006e520 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00097a0 tx=0x7f8ea002a040 comp rx=0 tx=0).stop 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.523+0000 7f8eb56e1640 1 -- 172.21.15.53:0/1624771518 shutdown_connections 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.523+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/1624771518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00b8860 0x7f8eb006e520 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.523+0000 7f8eb56e1640 1 -- 172.21.15.53:0/1624771518 >> 172.21.15.53:0/1624771518 conn(0x7f8eb00d2df0 msgr2=0x7f8eb00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.523+0000 7f8eb56e1640 1 -- 172.21.15.53:0/1624771518 shutdown_connections 2024-05-10T10:35:55.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.523+0000 7f8eb56e1640 1 -- 172.21.15.53:0/1624771518 wait complete. 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.524+0000 7f8eb56e1640 1 Processor -- start 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.524+0000 7f8eb56e1640 1 -- start start 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eb56e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b8860 0x7f8eb009cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eb56e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00cfd80 0x7f8eb015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eb56e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00d0230 0x7f8eb00cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eb56e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb00cde10 con 0x7f8eb00d0230 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eb56e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb00cdb30 con 0x7f8eb00cfd80 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eb56e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb006ea60 con 0x7f8eb00b8860 2024-05-10T10:35:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eaf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00d0230 0x7f8eb00cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eaf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00d0230 0x7f8eb00cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56522/0 (socket says 172.21.15.53:56522) 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eaf7fe640 1 -- 172.21.15.53:0/134144666 learned_addr learned my addr 172.21.15.53:0/134144666 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eae7fc640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00cfd80 0x7f8eb015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eaeffd640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b8860 0x7f8eb009cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eaf7fe640 1 -- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b8860 msgr2=0x7f8eb009cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.525+0000 7f8eaf7fe640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b8860 0x7f8eb009cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8eaf7fe640 1 -- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00cfd80 msgr2=0x7f8eb015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8eaf7fe640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00cfd80 0x7f8eb015ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8eaf7fe640 1 -- 172.21.15.53:0/134144666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb00c9f60 con 0x7f8eb00d0230 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8eaf7fe640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00d0230 0x7f8eb00cc0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c810 tx=0x7f8ea800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8e977fe640 1 -- 172.21.15.53:0/134144666 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003da0 con 0x7f8eb00d0230 2024-05-10T10:35:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8e977fe640 1 -- 172.21.15.53:0/134144666 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea8005070 con 0x7f8eb00d0230 2024-05-10T10:35:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb00cc5e0 con 0x7f8eb00d0230 2024-05-10T10:35:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8e977fe640 1 -- 172.21.15.53:0/134144666 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8005460 con 0x7f8eb00d0230 2024-05-10T10:35:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.526+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb009d4c0 con 0x7f8eb00d0230 2024-05-10T10:35:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.527+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb006a220 con 0x7f8eb00d0230 2024-05-10T10:35:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.528+0000 7f8e977fe640 1 -- 172.21.15.53:0/134144666 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ea8005600 con 0x7f8eb00d0230 2024-05-10T10:35:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.528+0000 7f8e977fe640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e9c0762d0 0x7f8e9c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.528+0000 7f8e977fe640 1 -- 172.21.15.53:0/134144666 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f8ea8097610 con 0x7f8eb00d0230 2024-05-10T10:35:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.528+0000 7f8eaeffd640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e9c0762d0 0x7f8e9c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.529+0000 7f8eaeffd640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e9c0762d0 0x7f8e9c078790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0034000 tx=0x7f8ea002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.533+0000 7f8e977fe640 1 -- 172.21.15.53:0/134144666 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8ea8060820 con 0x7f8eb00d0230 2024-05-10T10:35:55.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.765+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8eb006f2a0 con 0x7f8eb00d0230 2024-05-10T10:35:55.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.766+0000 7f8e977fe640 1 -- 172.21.15.53:0/134144666 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5188 (secure 0 0 0) 0x7f8ea8064b20 con 0x7f8eb00d0230 2024-05-10T10:35:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:55.767 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:35:54.766365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:35:55.767 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 34 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e9c0762d0 msgr2=0x7f8e9c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e9c0762d0 0x7f8e9c078790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0034000 tx=0x7f8ea002e000 comp rx=0 tx=0).stop 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00d0230 msgr2=0x7f8eb00cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00d0230 0x7f8eb00cc0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c810 tx=0x7f8ea800ccd0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 shutdown_connections 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8e9c0762d0 0x7f8e9c078790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00d0230 0x7f8eb00cc0a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb00cfd80 0x7f8eb015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 --2- 172.21.15.53:0/134144666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b8860 0x7f8eb009cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 >> 172.21.15.53:0/134144666 conn(0x7f8eb00d2df0 msgr2=0x7f8eb00b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:55.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 shutdown_connections 2024-05-10T10:35:55.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.768+0000 7f8eb56e1640 1 -- 172.21.15.53:0/134144666 wait complete. 2024-05-10T10:35:55.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:35:55.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.883+0000 7f9fbf4e1640 1 Processor -- start 2024-05-10T10:35:55.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.883+0000 7f9fbf4e1640 1 -- start start 2024-05-10T10:35:55.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbf4e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb81608b0 0x7f9fb80d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbf4e1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fb80d1230 0x7f9fb80d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbf4e1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb8160c90 con 0x7f9fb80b2900 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbf4e1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb80ce5f0 con 0x7f9fb80d1230 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbf4e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb80ce8d0 con 0x7f9fb81608b0 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbca7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fb80d1230 0x7f9fb80d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbda7e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9fb80b2900 0x7f9fb80b2d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48932/0 (socket says 172.21.15.53:48932) 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbca7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fb80d1230 0x7f9fb80d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56530/0 (socket says 172.21.15.53:56530) 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbda7e640 1 -- 172.21.15.53:0/4024677006 learned_addr learned my addr 172.21.15.53:0/4024677006 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbd27d640 1 --2- 172.21.15.53:0/4024677006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb81608b0 0x7f9fb80d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbca7c640 1 -- 172.21.15.53:0/4024677006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb81608b0 msgr2=0x7f9fb80d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbca7c640 1 --2- 172.21.15.53:0/4024677006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb81608b0 0x7f9fb80d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbca7c640 1 -- 172.21.15.53:0/4024677006 >> v1:172.21.15.53:6789/0 conn(0x7f9fb80b2900 legacy=0x7f9fb80b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9faffff640 1 -- 172.21.15.53:0/4024677006 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096595647 0 0) 0x7f9fb0001490 con 0x7f9fb80b2900 2024-05-10T10:35:55.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.884+0000 7f9fbca7c640 1 -- 172.21.15.53:0/4024677006 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80cebb0 con 0x7f9fb80d1230 2024-05-10T10:35:55.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.885+0000 7f9fbca7c640 1 --2- 172.21.15.53:0/4024677006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb80d1230 0x7f9fb80d16b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80097a0 tx=0x7f9fa802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3db3405291eade server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.885+0000 7f9faffff640 1 -- 172.21.15.53:0/4024677006 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8037070 con 0x7f9fb80d1230 2024-05-10T10:35:55.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.885+0000 7f9faffff640 1 -- 172.21.15.53:0/4024677006 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa8032440 con 0x7f9fb80d1230 2024-05-10T10:35:55.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.885+0000 7f9faffff640 1 -- 172.21.15.53:0/4024677006 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8032810 con 0x7f9fb80d1230 2024-05-10T10:35:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/4024677006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb80d1230 msgr2=0x7f9fb80d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/4024677006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb80d1230 0x7f9fb80d16b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80097a0 tx=0x7f9fa802a180 comp rx=0 tx=0).stop 2024-05-10T10:35:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/4024677006 shutdown_connections 2024-05-10T10:35:55.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/4024677006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb80d1230 0x7f9fb80d16b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/4024677006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb81608b0 0x7f9fb80d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/4024677006 >> 172.21.15.53:0/4024677006 conn(0x7f9fb80d2df0 msgr2=0x7f9fb80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/4024677006 shutdown_connections 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.886+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/4024677006 wait complete. 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.887+0000 7f9fbf4e1640 1 Processor -- start 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.887+0000 7f9fbf4e1640 1 -- start start 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbf4e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb80b2900 0x7f9fb815c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbf4e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb80d1230 0x7f9fb806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbf4e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81608b0 0x7f9fb80b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbf4e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80d1f10 con 0x7f9fb81608b0 2024-05-10T10:35:55.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbf4e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80d1d90 con 0x7f9fb80d1230 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbf4e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb80d03c0 con 0x7f9fb80b2900 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81608b0 0x7f9fb80b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81608b0 0x7f9fb80b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56546/0 (socket says 172.21.15.53:56546) 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 -- 172.21.15.53:0/3923775582 learned_addr learned my addr 172.21.15.53:0/3923775582 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbca7c640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb80d1230 0x7f9fb806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbd27d640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb80b2900 0x7f9fb815c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 -- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb80b2900 msgr2=0x7f9fb815c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb80b2900 0x7f9fb815c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 -- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb80d1230 msgr2=0x7f9fb806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb80d1230 0x7f9fb806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:55.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 -- 172.21.15.53:0/3923775582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb806dd40 con 0x7f9fb81608b0 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbd27d640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb80b2900 0x7f9fb815c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbca7c640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb80d1230 0x7f9fb806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbda7e640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81608b0 0x7f9fb80b71a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c7f0 tx=0x7f9fb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fadffb640 1 -- 172.21.15.53:0/3923775582 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0018070 con 0x7f9fb81608b0 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fadffb640 1 -- 172.21.15.53:0/3923775582 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb000a5e0 con 0x7f9fb81608b0 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb806e020 con 0x7f9fb81608b0 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.888+0000 7f9fadffb640 1 -- 172.21.15.53:0/3923775582 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000aa10 con 0x7f9fb81608b0 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.889+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb806e5e0 con 0x7f9fb81608b0 2024-05-10T10:35:55.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.890+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb806a220 con 0x7f9fb81608b0 2024-05-10T10:35:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.890+0000 7f9fadffb640 1 -- 172.21.15.53:0/3923775582 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9fb001b070 con 0x7f9fb81608b0 2024-05-10T10:35:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.891+0000 7f9fadffb640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f9c076300 0x7f9f9c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.891+0000 7f9fadffb640 1 -- 172.21.15.53:0/3923775582 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f9fb0096cd0 con 0x7f9fb81608b0 2024-05-10T10:35:55.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.891+0000 7f9fbd27d640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f9c076300 0x7f9f9c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.892+0000 7f9fbd27d640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f9c076300 0x7f9f9c0787c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:55.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:55.895+0000 7f9fadffb640 1 -- 172.21.15.53:0/3923775582 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9fb8160c90 con 0x7f9fb81608b0 2024-05-10T10:35:56.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.087+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fb80d39f0 con 0x7f9fb81608b0 2024-05-10T10:35:56.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.088+0000 7f9fadffb640 1 -- 172.21.15.53:0/3923775582 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16884 (secure 0 0 0) 0x7f9fb0060530 con 0x7f9fb81608b0 2024-05-10T10:35:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:56.089 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:35:53.380821+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:49.396339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:50.695199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f9c076300 msgr2=0x7f9f9c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f9c076300 0x7f9f9c0787c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4008040 comp rx=0 tx=0).stop 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81608b0 msgr2=0x7f9fb80b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81608b0 0x7f9fb80b71a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c7f0 tx=0x7f9fb000ccb0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 shutdown_connections 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9f9c076300 0x7f9f9c0787c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb81608b0 0x7f9fb80b71a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb80d1230 0x7f9fb806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 --2- 172.21.15.53:0/3923775582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb80b2900 0x7f9fb815c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 >> 172.21.15.53:0/3923775582 conn(0x7f9fb80d2df0 msgr2=0x7f9fb80cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:56.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 shutdown_connections 2024-05-10T10:35:56.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.091+0000 7f9fbf4e1640 1 -- 172.21.15.53:0/3923775582 wait complete. 2024-05-10T10:35:56.103 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:35:56.104 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:35:56.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.207+0000 7fbb54854640 1 Processor -- start 2024-05-10T10:35:56.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.207+0000 7fbb54854640 1 -- start start 2024-05-10T10:35:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb54854640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbb501598c0 0x7fbb5006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb54854640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb5006ec40 0x7fbb500c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb54854640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5006f1c0 con 0x7fbb500c71c0 2024-05-10T10:35:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb54854640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5006f4a0 con 0x7fbb501598c0 2024-05-10T10:35:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb54854640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5006f780 con 0x7fbb5006ec40 2024-05-10T10:35:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb4e59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbb501598c0 0x7fbb5006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb4dd9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb5006ec40 0x7fbb500c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb4ed9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbb500c71c0 0x7fbb5015fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:59108/0 (socket says 172.21.15.53:59108) 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb4ed9d640 1 -- 172.21.15.53:0/3165102358 learned_addr learned my addr 172.21.15.53:0/3165102358 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.208+0000 7fbb4e59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbb501598c0 0x7fbb5006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:34794/0 (socket says 172.21.15.53:34794) 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.209+0000 7fbb4e59c640 1 -- 172.21.15.53:0/3165102358 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb5006ec40 msgr2=0x7fbb500c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.209+0000 7fbb4e59c640 1 --2- 172.21.15.53:0/3165102358 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb5006ec40 0x7fbb500c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.209+0000 7fbb4e59c640 1 -- 172.21.15.53:0/3165102358 >> v1:172.21.15.135:6789/0 conn(0x7fbb500c71c0 legacy=0x7fbb5015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.209+0000 7fbb4d59a640 1 -- 172.21.15.53:0/3165102358 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133299204 0 0) 0x7fbb5006f1c0 con 0x7fbb500c71c0 2024-05-10T10:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.209+0000 7fbb4e59c640 1 -- 172.21.15.53:0/3165102358 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5006fa60 con 0x7fbb501598c0 2024-05-10T10:35:56.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.209+0000 7fbb4e59c640 1 --2- 172.21.15.53:0/3165102358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb501598c0 0x7fbb5006e700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb400097a0 tx=0x7fbb4002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab42fbc0c86c33a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb4d59a640 1 -- 172.21.15.53:0/3165102358 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40037070 con 0x7fbb501598c0 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb4d59a640 1 -- 172.21.15.53:0/3165102358 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb40032440 con 0x7fbb501598c0 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb4d59a640 1 -- 172.21.15.53:0/3165102358 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40032810 con 0x7fbb501598c0 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 -- 172.21.15.53:0/3165102358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb501598c0 msgr2=0x7fbb5006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 --2- 172.21.15.53:0/3165102358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb501598c0 0x7fbb5006e700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb400097a0 tx=0x7fbb4002a180 comp rx=0 tx=0).stop 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 -- 172.21.15.53:0/3165102358 shutdown_connections 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 --2- 172.21.15.53:0/3165102358 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb5006ec40 0x7fbb500c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 --2- 172.21.15.53:0/3165102358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb501598c0 0x7fbb5006e700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 -- 172.21.15.53:0/3165102358 >> 172.21.15.53:0/3165102358 conn(0x7fbb500b2900 msgr2=0x7fbb500b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 -- 172.21.15.53:0/3165102358 shutdown_connections 2024-05-10T10:35:56.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.210+0000 7fbb54854640 1 -- 172.21.15.53:0/3165102358 wait complete. 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.212+0000 7fbb54854640 1 Processor -- start 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.212+0000 7fbb54854640 1 -- start start 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.212+0000 7fbb54854640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb5006ec40 0x7fbb5014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb54854640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb500c71c0 0x7fbb50158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb54854640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb501598c0 0x7fbb500aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb54854640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50159ca0 con 0x7fbb500c71c0 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb54854640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5015c6e0 con 0x7fbb5006ec40 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb54854640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbb50159e20 con 0x7fbb501598c0 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb500c71c0 0x7fbb50158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb500c71c0 0x7fbb50158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56558/0 (socket says 172.21.15.53:56558) 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 -- 172.21.15.53:0/2867573882 learned_addr learned my addr 172.21.15.53:0/2867573882 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4e59c640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb5006ec40 0x7fbb5014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4ed9d640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb501598c0 0x7fbb500aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 -- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb501598c0 msgr2=0x7fbb500aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb501598c0 0x7fbb500aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 -- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb5006ec40 msgr2=0x7fbb5014f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb5006ec40 0x7fbb5014f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 -- 172.21.15.53:0/2867573882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb501f8a60 con 0x7fbb500c71c0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb4dd9b640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb500c71c0 0x7fbb50158430 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e870 tx=0x7fbb4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb3effd640 1 -- 172.21.15.53:0/2867573882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44017070 con 0x7fbb500c71c0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb3effd640 1 -- 172.21.15.53:0/2867573882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4400c940 con 0x7fbb500c71c0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb501f8d40 con 0x7fbb500c71c0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb3effd640 1 -- 172.21.15.53:0/2867573882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400cd30 con 0x7fbb500c71c0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.213+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb501f9300 con 0x7fbb500c71c0 2024-05-10T10:35:56.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.214+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5006a220 con 0x7fbb500c71c0 2024-05-10T10:35:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.215+0000 7fbb3effd640 1 -- 172.21.15.53:0/2867573882 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbb4401a070 con 0x7fbb500c71c0 2024-05-10T10:35:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.216+0000 7fbb3effd640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb300762d0 0x7fbb30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.216+0000 7fbb3effd640 1 -- 172.21.15.53:0/2867573882 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fbb440979e0 con 0x7fbb500c71c0 2024-05-10T10:35:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.216+0000 7fbb4e59c640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb300762d0 0x7fbb30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.216+0000 7fbb4e59c640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb300762d0 0x7fbb30078790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.220+0000 7fbb3effd640 1 -- 172.21.15.53:0/2867573882 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbb440611b0 con 0x7fbb500c71c0 2024-05-10T10:35:56.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.406+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb5006e4e0 con 0x7fbb300762d0 2024-05-10T10:35:56.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.413+0000 7fbb3effd640 1 -- 172.21.15.53:0/2867573882 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251938 (secure 0 0 0) 0x7fbb5006e4e0 con 0x7fbb300762d0 2024-05-10T10:35:56.416 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:56.418 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:35:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb300762d0 msgr2=0x7fbb30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb300762d0 0x7fbb30078790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).stop 2024-05-10T10:35:56.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb500c71c0 msgr2=0x7fbb50158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb500c71c0 0x7fbb50158430 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e870 tx=0x7fbb4400ed30 comp rx=0 tx=0).stop 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 shutdown_connections 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb300762d0 0x7fbb30078790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb501598c0 0x7fbb500aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb500c71c0 0x7fbb50158430 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 --2- 172.21.15.53:0/2867573882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb5006ec40 0x7fbb5014f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 >> 172.21.15.53:0/2867573882 conn(0x7fbb500b2900 msgr2=0x7fbb500c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 shutdown_connections 2024-05-10T10:35:56.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.416+0000 7fbb54854640 1 -- 172.21.15.53:0/2867573882 wait complete. 2024-05-10T10:35:56.428 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-05-10T10:35:56.188670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239660,"kb_used_data":6836,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735060,"statfs":{"total":773094113280,"available":772848701440,"internally_reserved":0,"allocated":7000064,"data_stored":3747106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.834374"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393491+0000","last_change":"2024-05-10T10:35:52.390729+0000","last_active":"2024-05-10T10:35:52.393491+0000","last_peered":"2024-05-10T10:35:52.393491+0000","last_clean":"2024-05-10T10:35:52.393491+0000","last_became_active":"2024-05-10T10:35:52.390434+0000","last_became_peered":"2024-05-10T10:35:52.390434+0000","last_unstale":"2024-05-10T10:35:52.393491+0000","last_undegraded":"2024-05-10T10:35:52.393491+0000","last_fullsized":"2024-05-10T10:35:52.393491+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:48.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388572+0000","last_change":"2024-05-10T10:35:52.385215+0000","last_active":"2024-05-10T10:35:52.388572+0000","last_peered":"2024-05-10T10:35:52.388572+0000","last_clean":"2024-05-10T10:35:52.388572+0000","last_became_active":"2024-05-10T10:35:52.384753+0000","last_became_peered":"2024-05-10T10:35:52.384753+0000","last_unstale":"2024-05-10T10:35:52.388572+0000","last_undegraded":"2024-05-10T10:35:52.388572+0000","last_fullsized":"2024-05-10T10:35:52.388572+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:26.783046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393935+0000","last_change":"2024-05-10T10:35:52.389134+0000","last_active":"2024-05-10T10:35:52.393935+0000","last_peered":"2024-05-10T10:35:52.393935+0000","last_clean":"2024-05-10T10:35:52.393935+0000","last_became_active":"2024-05-10T10:35:52.388847+0000","last_became_peered":"2024-05-10T10:35:52.388847+0000","last_unstale":"2024-05-10T10:35:52.393935+0000","last_undegraded":"2024-05-10T10:35:52.393935+0000","last_fullsized":"2024-05-10T10:35:52.393935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:35.101141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388387+0000","last_change":"2024-05-10T10:35:52.382733+0000","last_active":"2024-05-10T10:35:52.388387+0000","last_peered":"2024-05-10T10:35:52.388387+0000","last_clean":"2024-05-10T10:35:52.388387+0000","last_became_active":"2024-05-10T10:35:52.382374+0000","last_became_peered":"2024-05-10T10:35:52.382374+0000","last_unstale":"2024-05-10T10:35:52.388387+0000","last_undegraded":"2024-05-10T10:35:52.388387+0000","last_fullsized":"2024-05-10T10:35:52.388387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:41.527270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.397655+0000","last_change":"2024-05-10T10:35:52.395323+0000","last_active":"2024-05-10T10:35:52.397655+0000","last_peered":"2024-05-10T10:35:52.397655+0000","last_clean":"2024-05-10T10:35:52.397655+0000","last_became_active":"2024-05-10T10:35:52.395039+0000","last_became_peered":"2024-05-10T10:35:52.395039+0000","last_unstale":"2024-05-10T10:35:52.397655+0000","last_undegraded":"2024-05-10T10:35:52.397655+0000","last_fullsized":"2024-05-10T10:35:52.397655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:30:49.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392062+0000","last_change":"2024-05-10T10:35:52.387021+0000","last_active":"2024-05-10T10:35:52.392062+0000","last_peered":"2024-05-10T10:35:52.392062+0000","last_clean":"2024-05-10T10:35:52.392062+0000","last_became_active":"2024-05-10T10:35:52.386736+0000","last_became_peered":"2024-05-10T10:35:52.386736+0000","last_unstale":"2024-05-10T10:35:52.392062+0000","last_undegraded":"2024-05-10T10:35:52.392062+0000","last_fullsized":"2024-05-10T10:35:52.392062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:18.972647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.397323+0000","last_change":"2024-05-10T10:35:52.394532+0000","last_active":"2024-05-10T10:35:52.397323+0000","last_peered":"2024-05-10T10:35:52.397323+0000","last_clean":"2024-05-10T10:35:52.397323+0000","last_became_active":"2024-05-10T10:35:52.394073+0000","last_became_peered":"2024-05-10T10:35:52.394073+0000","last_unstale":"2024-05-10T10:35:52.397323+0000","last_undegraded":"2024-05-10T10:35:52.397323+0000","last_fullsized":"2024-05-10T10:35:52.397323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:59:03.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394813+0000","last_change":"2024-05-10T10:35:52.392977+0000","last_active":"2024-05-10T10:35:52.394813+0000","last_peered":"2024-05-10T10:35:52.394813+0000","last_clean":"2024-05-10T10:35:52.394813+0000","last_became_active":"2024-05-10T10:35:52.392646+0000","last_became_peered":"2024-05-10T10:35:52.392646+0000","last_unstale":"2024-05-10T10:35:52.394813+0000","last_undegraded":"2024-05-10T10:35:52.394813+0000","last_fullsized":"2024-05-10T10:35:52.394813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:12:03.431670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398086+0000","last_change":"2024-05-10T10:35:52.395098+0000","last_active":"2024-05-10T10:35:52.398086+0000","last_peered":"2024-05-10T10:35:52.398086+0000","last_clean":"2024-05-10T10:35:52.398086+0000","last_became_active":"2024-05-10T10:35:52.394824+0000","last_became_peered":"2024-05-10T10:35:52.394824+0000","last_unstale":"2024-05-10T10:35:52.398086+0000","last_undegraded":"2024-05-10T10:35:52.398086+0000","last_fullsized":"2024-05-10T10:35:52.398086+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:37:17.187253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391940+0000","last_change":"2024-05-10T10:35:52.389825+0000","last_active":"2024-05-10T10:35:52.391940+0000","last_peered":"2024-05-10T10:35:52.391940+0000","last_clean":"2024-05-10T10:35:52.391940+0000","last_became_active":"2024-05-10T10:35:52.389470+0000","last_became_peered":"2024-05-10T10:35:52.389470+0000","last_unstale":"2024-05-10T10:35:52.391940+0000","last_undegraded":"2024-05-10T10:35:52.391940+0000","last_fullsized":"2024-05-10T10:35:52.391940+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:30:00.573188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395385+0000","last_change":"2024-05-10T10:35:52.392597+0000","last_active":"2024-05-10T10:35:52.395385+0000","last_peered":"2024-05-10T10:35:52.395385+0000","last_clean":"2024-05-10T10:35:52.395385+0000","last_became_active":"2024-05-10T10:35:52.391875+0000","last_became_peered":"2024-05-10T10:35:52.391875+0000","last_unstale":"2024-05-10T10:35:52.395385+0000","last_undegraded":"2024-05-10T10:35:52.395385+0000","last_fullsized":"2024-05-10T10:35:52.395385+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:06.891450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395393+0000","last_change":"2024-05-10T10:35:52.392914+0000","last_active":"2024-05-10T10:35:52.395393+0000","last_peered":"2024-05-10T10:35:52.395393+0000","last_clean":"2024-05-10T10:35:52.395393+0000","last_became_active":"2024-05-10T10:35:52.392605+0000","last_became_peered":"2024-05-10T10:35:52.392605+0000","last_unstale":"2024-05-10T10:35:52.395393+0000","last_undegraded":"2024-05-10T10:35:52.395393+0000","last_fullsized":"2024-05-10T10:35:52.395393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:45:33.469087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388734+0000","last_change":"2024-05-10T10:35:52.382317+0000","last_active":"2024-05-10T10:35:52.388734+0000","last_peered":"2024-05-10T10:35:52.388734+0000","last_clean":"2024-05-10T10:35:52.388734+0000","last_became_active":"2024-05-10T10:35:52.382042+0000","last_became_peered":"2024-05-10T10:35:52.382042+0000","last_unstale":"2024-05-10T10:35:52.388734+0000","last_undegraded":"2024-05-10T10:35:52.388734+0000","last_fullsized":"2024-05-10T10:35:52.388734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:59.312394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394521+0000","last_change":"2024-05-10T10:35:52.390761+0000","last_active":"2024-05-10T10:35:52.394521+0000","last_peered":"2024-05-10T10:35:52.394521+0000","last_clean":"2024-05-10T10:35:52.394521+0000","last_became_active":"2024-05-10T10:35:52.390444+0000","last_became_peered":"2024-05-10T10:35:52.390444+0000","last_unstale":"2024-05-10T10:35:52.394521+0000","last_undegraded":"2024-05-10T10:35:52.394521+0000","last_fullsized":"2024-05-10T10:35:52.394521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:12:58.694304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392326+0000","last_change":"2024-05-10T10:35:52.391281+0000","last_active":"2024-05-10T10:35:52.392326+0000","last_peered":"2024-05-10T10:35:52.392326+0000","last_clean":"2024-05-10T10:35:52.392326+0000","last_became_active":"2024-05-10T10:35:52.390834+0000","last_became_peered":"2024-05-10T10:35:52.390834+0000","last_unstale":"2024-05-10T10:35:52.392326+0000","last_undegraded":"2024-05-10T10:35:52.392326+0000","last_fullsized":"2024-05-10T10:35:52.392326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:39:26.071931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389970+0000","last_change":"2024-05-10T10:35:52.387019+0000","last_active":"2024-05-10T10:35:52.389970+0000","last_peered":"2024-05-10T10:35:52.389970+0000","last_clean":"2024-05-10T10:35:52.389970+0000","last_became_active":"2024-05-10T10:35:52.386736+0000","last_became_peered":"2024-05-10T10:35:52.386736+0000","last_unstale":"2024-05-10T10:35:52.389970+0000","last_undegraded":"2024-05-10T10:35:52.389970+0000","last_fullsized":"2024-05-10T10:35:52.389970+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:53:00.799654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394222+0000","last_change":"2024-05-10T10:35:52.392230+0000","last_active":"2024-05-10T10:35:52.394222+0000","last_peered":"2024-05-10T10:35:52.394222+0000","last_clean":"2024-05-10T10:35:52.394222+0000","last_became_active":"2024-05-10T10:35:52.391539+0000","last_became_peered":"2024-05-10T10:35:52.391539+0000","last_unstale":"2024-05-10T10:35:52.394222+0000","last_undegraded":"2024-05-10T10:35:52.394222+0000","last_fullsized":"2024-05-10T10:35:52.394222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:03.491778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388353+0000","last_change":"2024-05-10T10:35:52.382491+0000","last_active":"2024-05-10T10:35:52.388353+0000","last_peered":"2024-05-10T10:35:52.388353+0000","last_clean":"2024-05-10T10:35:52.388353+0000","last_became_active":"2024-05-10T10:35:52.382096+0000","last_became_peered":"2024-05-10T10:35:52.382096+0000","last_unstale":"2024-05-10T10:35:52.388353+0000","last_undegraded":"2024-05-10T10:35:52.388353+0000","last_fullsized":"2024-05-10T10:35:52.388353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:57.321844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.396981+0000","last_change":"2024-05-10T10:35:52.395413+0000","last_active":"2024-05-10T10:35:52.396981+0000","last_peered":"2024-05-10T10:35:52.396981+0000","last_clean":"2024-05-10T10:35:52.396981+0000","last_became_active":"2024-05-10T10:35:52.394773+0000","last_became_peered":"2024-05-10T10:35:52.394773+0000","last_unstale":"2024-05-10T10:35:52.396981+0000","last_undegraded":"2024-05-10T10:35:52.396981+0000","last_fullsized":"2024-05-10T10:35:52.396981+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:18:49.471589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399586+0000","last_change":"2024-05-10T10:35:52.395738+0000","last_active":"2024-05-10T10:35:52.399586+0000","last_peered":"2024-05-10T10:35:52.399586+0000","last_clean":"2024-05-10T10:35:52.399586+0000","last_became_active":"2024-05-10T10:35:52.395466+0000","last_became_peered":"2024-05-10T10:35:52.395466+0000","last_unstale":"2024-05-10T10:35:52.399586+0000","last_undegraded":"2024-05-10T10:35:52.399586+0000","last_fullsized":"2024-05-10T10:35:52.399586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:57.565541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.390766+0000","last_change":"2024-05-10T10:35:52.383055+0000","last_active":"2024-05-10T10:35:52.390766+0000","last_peered":"2024-05-10T10:35:52.390766+0000","last_clean":"2024-05-10T10:35:52.390766+0000","last_became_active":"2024-05-10T10:35:52.382633+0000","last_became_peered":"2024-05-10T10:35:52.382633+0000","last_unstale":"2024-05-10T10:35:52.390766+0000","last_undegraded":"2024-05-10T10:35:52.390766+0000","last_fullsized":"2024-05-10T10:35:52.390766+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:35.101141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399550+0000","last_change":"2024-05-10T10:35:52.396692+0000","last_active":"2024-05-10T10:35:52.399550+0000","last_peered":"2024-05-10T10:35:52.399550+0000","last_clean":"2024-05-10T10:35:52.399550+0000","last_became_active":"2024-05-10T10:35:52.396428+0000","last_became_peered":"2024-05-10T10:35:52.396428+0000","last_unstale":"2024-05-10T10:35:52.399550+0000","last_undegraded":"2024-05-10T10:35:52.399550+0000","last_fullsized":"2024-05-10T10:35:52.399550+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:54:29.102508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387551+0000","last_change":"2024-05-10T10:35:52.385788+0000","last_active":"2024-05-10T10:35:52.387551+0000","last_peered":"2024-05-10T10:35:52.387551+0000","last_clean":"2024-05-10T10:35:52.387551+0000","last_became_active":"2024-05-10T10:35:52.385540+0000","last_became_peered":"2024-05-10T10:35:52.385540+0000","last_unstale":"2024-05-10T10:35:52.387551+0000","last_undegraded":"2024-05-10T10:35:52.387551+0000","last_fullsized":"2024-05-10T10:35:52.387551+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:25.736922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391920+0000","last_change":"2024-05-10T10:35:52.389157+0000","last_active":"2024-05-10T10:35:52.391920+0000","last_peered":"2024-05-10T10:35:52.391920+0000","last_clean":"2024-05-10T10:35:52.391920+0000","last_became_active":"2024-05-10T10:35:52.388861+0000","last_became_peered":"2024-05-10T10:35:52.388861+0000","last_unstale":"2024-05-10T10:35:52.391920+0000","last_undegraded":"2024-05-10T10:35:52.391920+0000","last_fullsized":"2024-05-10T10:35:52.391920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:17:05.283765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.390727+0000","last_change":"2024-05-10T10:35:52.383540+0000","last_active":"2024-05-10T10:35:52.390727+0000","last_peered":"2024-05-10T10:35:52.390727+0000","last_clean":"2024-05-10T10:35:52.390727+0000","last_became_active":"2024-05-10T10:35:52.383257+0000","last_became_peered":"2024-05-10T10:35:52.383257+0000","last_unstale":"2024-05-10T10:35:52.390727+0000","last_undegraded":"2024-05-10T10:35:52.390727+0000","last_fullsized":"2024-05-10T10:35:52.390727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:17:05.283765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.390804+0000","last_change":"2024-05-10T10:35:52.383092+0000","last_active":"2024-05-10T10:35:52.390804+0000","last_peered":"2024-05-10T10:35:52.390804+0000","last_clean":"2024-05-10T10:35:52.390804+0000","last_became_active":"2024-05-10T10:35:52.382692+0000","last_became_peered":"2024-05-10T10:35:52.382692+0000","last_unstale":"2024-05-10T10:35:52.390804+0000","last_undegraded":"2024-05-10T10:35:52.390804+0000","last_fullsized":"2024-05-10T10:35:52.390804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:10.396371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388241+0000","last_change":"2024-05-10T10:35:52.383237+0000","last_active":"2024-05-10T10:35:52.388241+0000","last_peered":"2024-05-10T10:35:52.388241+0000","last_clean":"2024-05-10T10:35:52.388241+0000","last_became_active":"2024-05-10T10:35:52.382089+0000","last_became_peered":"2024-05-10T10:35:52.382089+0000","last_unstale":"2024-05-10T10:35:52.388241+0000","last_undegraded":"2024-05-10T10:35:52.388241+0000","last_fullsized":"2024-05-10T10:35:52.388241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:30:49.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394631+0000","last_change":"2024-05-10T10:35:52.393215+0000","last_active":"2024-05-10T10:35:52.394631+0000","last_peered":"2024-05-10T10:35:52.394631+0000","last_clean":"2024-05-10T10:35:52.394631+0000","last_became_active":"2024-05-10T10:35:52.392938+0000","last_became_peered":"2024-05-10T10:35:52.392938+0000","last_unstale":"2024-05-10T10:35:52.394631+0000","last_undegraded":"2024-05-10T10:35:52.394631+0000","last_fullsized":"2024-05-10T10:35:52.394631+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:07:01.564919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.396332+0000","last_change":"2024-05-10T10:35:52.394140+0000","last_active":"2024-05-10T10:35:52.396332+0000","last_peered":"2024-05-10T10:35:52.396332+0000","last_clean":"2024-05-10T10:35:52.396332+0000","last_became_active":"2024-05-10T10:35:52.393829+0000","last_became_peered":"2024-05-10T10:35:52.393829+0000","last_unstale":"2024-05-10T10:35:52.396332+0000","last_undegraded":"2024-05-10T10:35:52.396332+0000","last_fullsized":"2024-05-10T10:35:52.396332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:18.972647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391168+0000","last_change":"2024-05-10T10:35:52.384727+0000","last_active":"2024-05-10T10:35:52.391168+0000","last_peered":"2024-05-10T10:35:52.391168+0000","last_clean":"2024-05-10T10:35:52.391168+0000","last_became_active":"2024-05-10T10:35:52.384431+0000","last_became_peered":"2024-05-10T10:35:52.384431+0000","last_unstale":"2024-05-10T10:35:52.391168+0000","last_undegraded":"2024-05-10T10:35:52.391168+0000","last_fullsized":"2024-05-10T10:35:52.391168+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:30:00.573188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389427+0000","last_change":"2024-05-10T10:35:52.383091+0000","last_active":"2024-05-10T10:35:52.389427+0000","last_peered":"2024-05-10T10:35:52.389427+0000","last_clean":"2024-05-10T10:35:52.389427+0000","last_became_active":"2024-05-10T10:35:52.382807+0000","last_became_peered":"2024-05-10T10:35:52.382807+0000","last_unstale":"2024-05-10T10:35:52.389427+0000","last_undegraded":"2024-05-10T10:35:52.389427+0000","last_fullsized":"2024-05-10T10:35:52.389427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:36:49.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387627+0000","last_change":"2024-05-10T10:35:52.382139+0000","last_active":"2024-05-10T10:35:52.387627+0000","last_peered":"2024-05-10T10:35:52.387627+0000","last_clean":"2024-05-10T10:35:52.387627+0000","last_became_active":"2024-05-10T10:35:52.381390+0000","last_became_peered":"2024-05-10T10:35:52.381390+0000","last_unstale":"2024-05-10T10:35:52.387627+0000","last_undegraded":"2024-05-10T10:35:52.387627+0000","last_fullsized":"2024-05-10T10:35:52.387627+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:12:03.431670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395499+0000","last_change":"2024-05-10T10:35:52.393311+0000","last_active":"2024-05-10T10:35:52.395499+0000","last_peered":"2024-05-10T10:35:52.395499+0000","last_clean":"2024-05-10T10:35:52.395499+0000","last_became_active":"2024-05-10T10:35:52.393041+0000","last_became_peered":"2024-05-10T10:35:52.393041+0000","last_unstale":"2024-05-10T10:35:52.395499+0000","last_undegraded":"2024-05-10T10:35:52.395499+0000","last_fullsized":"2024-05-10T10:35:52.395499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:41.527270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404242+0000","last_change":"2024-05-10T10:35:51.377776+0000","last_active":"2024-05-10T10:35:52.404242+0000","last_peered":"2024-05-10T10:35:52.404242+0000","last_clean":"2024-05-10T10:35:52.404242+0000","last_became_active":"2024-05-10T10:35:51.377199+0000","last_became_peered":"2024-05-10T10:35:51.377199+0000","last_unstale":"2024-05-10T10:35:52.404242+0000","last_undegraded":"2024-05-10T10:35:52.404242+0000","last_fullsized":"2024-05-10T10:35:52.404242+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:14.693162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395405+0000","last_change":"2024-05-10T10:35:52.392825+0000","last_active":"2024-05-10T10:35:52.395405+0000","last_peered":"2024-05-10T10:35:52.395405+0000","last_clean":"2024-05-10T10:35:52.395405+0000","last_became_active":"2024-05-10T10:35:52.392503+0000","last_became_peered":"2024-05-10T10:35:52.392503+0000","last_unstale":"2024-05-10T10:35:52.395405+0000","last_undegraded":"2024-05-10T10:35:52.395405+0000","last_fullsized":"2024-05-10T10:35:52.395405+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:09.843423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.402496+0000","last_change":"2024-05-10T10:35:51.375812+0000","last_active":"2024-05-10T10:35:52.402496+0000","last_peered":"2024-05-10T10:35:52.402496+0000","last_clean":"2024-05-10T10:35:52.402496+0000","last_became_active":"2024-05-10T10:35:51.375516+0000","last_became_peered":"2024-05-10T10:35:51.375516+0000","last_unstale":"2024-05-10T10:35:52.402496+0000","last_undegraded":"2024-05-10T10:35:52.402496+0000","last_fullsized":"2024-05-10T10:35:52.402496+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:29:59.568532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389607+0000","last_change":"2024-05-10T10:35:52.385630+0000","last_active":"2024-05-10T10:35:52.389607+0000","last_peered":"2024-05-10T10:35:52.389607+0000","last_clean":"2024-05-10T10:35:52.389607+0000","last_became_active":"2024-05-10T10:35:52.385363+0000","last_became_peered":"2024-05-10T10:35:52.385363+0000","last_unstale":"2024-05-10T10:35:52.389607+0000","last_undegraded":"2024-05-10T10:35:52.389607+0000","last_fullsized":"2024-05-10T10:35:52.389607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:39:26.071931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.400912+0000","last_change":"2024-05-10T10:35:51.372181+0000","last_active":"2024-05-10T10:35:52.400912+0000","last_peered":"2024-05-10T10:35:52.400912+0000","last_clean":"2024-05-10T10:35:52.400912+0000","last_became_active":"2024-05-10T10:35:51.371918+0000","last_became_peered":"2024-05-10T10:35:51.371918+0000","last_unstale":"2024-05-10T10:35:52.400912+0000","last_undegraded":"2024-05-10T10:35:52.400912+0000","last_fullsized":"2024-05-10T10:35:52.400912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:58.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391952+0000","last_change":"2024-05-10T10:35:52.390462+0000","last_active":"2024-05-10T10:35:52.391952+0000","last_peered":"2024-05-10T10:35:52.391952+0000","last_clean":"2024-05-10T10:35:52.391952+0000","last_became_active":"2024-05-10T10:35:52.390206+0000","last_became_peered":"2024-05-10T10:35:52.390206+0000","last_unstale":"2024-05-10T10:35:52.391952+0000","last_undegraded":"2024-05-10T10:35:52.391952+0000","last_fullsized":"2024-05-10T10:35:52.391952+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:12:03.431670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.403389+0000","last_change":"2024-05-10T10:35:51.371881+0000","last_active":"2024-05-10T10:35:52.403389+0000","last_peered":"2024-05-10T10:35:52.403389+0000","last_clean":"2024-05-10T10:35:52.403389+0000","last_became_active":"2024-05-10T10:35:51.371594+0000","last_became_peered":"2024-05-10T10:35:51.371594+0000","last_unstale":"2024-05-10T10:35:52.403389+0000","last_undegraded":"2024-05-10T10:35:52.403389+0000","last_fullsized":"2024-05-10T10:35:52.403389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:17:04.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389992+0000","last_change":"2024-05-10T10:35:52.387883+0000","last_active":"2024-05-10T10:35:52.389992+0000","last_peered":"2024-05-10T10:35:52.389992+0000","last_clean":"2024-05-10T10:35:52.389992+0000","last_became_active":"2024-05-10T10:35:52.387602+0000","last_became_peered":"2024-05-10T10:35:52.387602+0000","last_unstale":"2024-05-10T10:35:52.389992+0000","last_undegraded":"2024-05-10T10:35:52.389992+0000","last_fullsized":"2024-05-10T10:35:52.389992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:59.536959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.402345+0000","last_change":"2024-05-10T10:35:51.366216+0000","last_active":"2024-05-10T10:35:52.402345+0000","last_peered":"2024-05-10T10:35:52.402345+0000","last_clean":"2024-05-10T10:35:52.402345+0000","last_became_active":"2024-05-10T10:35:51.365925+0000","last_became_peered":"2024-05-10T10:35:51.365925+0000","last_unstale":"2024-05-10T10:35:52.402345+0000","last_undegraded":"2024-05-10T10:35:52.402345+0000","last_fullsized":"2024-05-10T10:35:52.402345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:58.453878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394182+0000","last_change":"2024-05-10T10:35:52.391112+0000","last_active":"2024-05-10T10:35:52.394182+0000","last_peered":"2024-05-10T10:35:52.394182+0000","last_clean":"2024-05-10T10:35:52.394182+0000","last_became_active":"2024-05-10T10:35:52.390825+0000","last_became_peered":"2024-05-10T10:35:52.390825+0000","last_unstale":"2024-05-10T10:35:52.394182+0000","last_undegraded":"2024-05-10T10:35:52.394182+0000","last_fullsized":"2024-05-10T10:35:52.394182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:16.718296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.402430+0000","last_change":"2024-05-10T10:35:51.369615+0000","last_active":"2024-05-10T10:35:52.402430+0000","last_peered":"2024-05-10T10:35:52.402430+0000","last_clean":"2024-05-10T10:35:52.402430+0000","last_became_active":"2024-05-10T10:35:51.369313+0000","last_became_peered":"2024-05-10T10:35:51.369313+0000","last_unstale":"2024-05-10T10:35:52.402430+0000","last_undegraded":"2024-05-10T10:35:52.402430+0000","last_fullsized":"2024-05-10T10:35:52.402430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:06.094811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399276+0000","last_change":"2024-05-10T10:35:52.398093+0000","last_active":"2024-05-10T10:35:52.399276+0000","last_peered":"2024-05-10T10:35:52.399276+0000","last_clean":"2024-05-10T10:35:52.399276+0000","last_became_active":"2024-05-10T10:35:52.397712+0000","last_became_peered":"2024-05-10T10:35:52.397712+0000","last_unstale":"2024-05-10T10:35:52.399276+0000","last_undegraded":"2024-05-10T10:35:52.399276+0000","last_fullsized":"2024-05-10T10:35:52.399276+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:33.621733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.376098+0000","last_change":"2024-05-10T10:35:51.372855+0000","last_active":"2024-05-10T10:35:52.376098+0000","last_peered":"2024-05-10T10:35:52.376098+0000","last_clean":"2024-05-10T10:35:52.376098+0000","last_became_active":"2024-05-10T10:35:51.372435+0000","last_became_peered":"2024-05-10T10:35:51.372435+0000","last_unstale":"2024-05-10T10:35:52.376098+0000","last_undegraded":"2024-05-10T10:35:52.376098+0000","last_fullsized":"2024-05-10T10:35:52.376098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:37:16.182598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387854+0000","last_change":"2024-05-10T10:35:52.386299+0000","last_active":"2024-05-10T10:35:52.387854+0000","last_peered":"2024-05-10T10:35:52.387854+0000","last_clean":"2024-05-10T10:35:52.387854+0000","last_became_active":"2024-05-10T10:35:52.385979+0000","last_became_peered":"2024-05-10T10:35:52.385979+0000","last_unstale":"2024-05-10T10:35:52.387854+0000","last_undegraded":"2024-05-10T10:35:52.387854+0000","last_fullsized":"2024-05-10T10:35:52.387854+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:31.285698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.384892+0000","last_change":"2024-05-10T10:35:51.369856+0000","last_active":"2024-05-10T10:35:52.384892+0000","last_peered":"2024-05-10T10:35:52.384892+0000","last_clean":"2024-05-10T10:35:52.384892+0000","last_became_active":"2024-05-10T10:35:51.369567+0000","last_became_peered":"2024-05-10T10:35:51.369567+0000","last_unstale":"2024-05-10T10:35:52.384892+0000","last_undegraded":"2024-05-10T10:35:52.384892+0000","last_fullsized":"2024-05-10T10:35:52.384892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:50.764438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392087+0000","last_change":"2024-05-10T10:35:52.386058+0000","last_active":"2024-05-10T10:35:52.392087+0000","last_peered":"2024-05-10T10:35:52.392087+0000","last_clean":"2024-05-10T10:35:52.392087+0000","last_became_active":"2024-05-10T10:35:52.385784+0000","last_became_peered":"2024-05-10T10:35:52.385784+0000","last_unstale":"2024-05-10T10:35:52.392087+0000","last_undegraded":"2024-05-10T10:35:52.392087+0000","last_fullsized":"2024-05-10T10:35:52.392087+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:01:54.716038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.381212+0000","last_change":"2024-05-10T10:35:51.378284+0000","last_active":"2024-05-10T10:35:52.381212+0000","last_peered":"2024-05-10T10:35:52.381212+0000","last_clean":"2024-05-10T10:35:52.381212+0000","last_became_active":"2024-05-10T10:35:51.377947+0000","last_became_peered":"2024-05-10T10:35:51.377947+0000","last_unstale":"2024-05-10T10:35:52.381212+0000","last_undegraded":"2024-05-10T10:35:52.381212+0000","last_fullsized":"2024-05-10T10:35:52.381212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:33.677845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387352+0000","last_change":"2024-05-10T10:35:52.385204+0000","last_active":"2024-05-10T10:35:52.387352+0000","last_peered":"2024-05-10T10:35:52.387352+0000","last_clean":"2024-05-10T10:35:52.387352+0000","last_became_active":"2024-05-10T10:35:52.384909+0000","last_became_peered":"2024-05-10T10:35:52.384909+0000","last_unstale":"2024-05-10T10:35:52.387352+0000","last_undegraded":"2024-05-10T10:35:52.387352+0000","last_fullsized":"2024-05-10T10:35:52.387352+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:22.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.378709+0000","last_change":"2024-05-10T10:35:51.369750+0000","last_active":"2024-05-10T10:35:52.378709+0000","last_peered":"2024-05-10T10:35:52.378709+0000","last_clean":"2024-05-10T10:35:52.378709+0000","last_became_active":"2024-05-10T10:35:51.369433+0000","last_became_peered":"2024-05-10T10:35:51.369433+0000","last_unstale":"2024-05-10T10:35:52.378709+0000","last_undegraded":"2024-05-10T10:35:52.378709+0000","last_fullsized":"2024-05-10T10:35:52.378709+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:28.854767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398393+0000","last_change":"2024-05-10T10:35:52.397095+0000","last_active":"2024-05-10T10:35:52.398393+0000","last_peered":"2024-05-10T10:35:52.398393+0000","last_clean":"2024-05-10T10:35:52.398393+0000","last_became_active":"2024-05-10T10:35:52.396616+0000","last_became_peered":"2024-05-10T10:35:52.396616+0000","last_unstale":"2024-05-10T10:35:52.398393+0000","last_undegraded":"2024-05-10T10:35:52.398393+0000","last_fullsized":"2024-05-10T10:35:52.398393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:13:51.776442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404760+0000","last_change":"2024-05-10T10:35:51.378197+0000","last_active":"2024-05-10T10:35:52.404760+0000","last_peered":"2024-05-10T10:35:52.404760+0000","last_clean":"2024-05-10T10:35:52.404760+0000","last_became_active":"2024-05-10T10:35:51.377859+0000","last_became_peered":"2024-05-10T10:35:51.377859+0000","last_unstale":"2024-05-10T10:35:52.404760+0000","last_undegraded":"2024-05-10T10:35:52.404760+0000","last_fullsized":"2024-05-10T10:35:52.404760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:07:00.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.386290+0000","last_change":"2024-05-10T10:35:52.380971+0000","last_active":"2024-05-10T10:35:52.386290+0000","last_peered":"2024-05-10T10:35:52.386290+0000","last_clean":"2024-05-10T10:35:52.386290+0000","last_became_active":"2024-05-10T10:35:52.380556+0000","last_became_peered":"2024-05-10T10:35:52.380556+0000","last_unstale":"2024-05-10T10:35:52.386290+0000","last_undegraded":"2024-05-10T10:35:52.386290+0000","last_fullsized":"2024-05-10T10:35:52.386290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:51.769093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.406439+0000","last_change":"2024-05-10T10:35:51.375371+0000","last_active":"2024-05-10T10:35:52.406439+0000","last_peered":"2024-05-10T10:35:52.406439+0000","last_clean":"2024-05-10T10:35:52.406439+0000","last_became_active":"2024-05-10T10:35:51.375080+0000","last_became_peered":"2024-05-10T10:35:51.375080+0000","last_unstale":"2024-05-10T10:35:52.406439+0000","last_undegraded":"2024-05-10T10:35:52.406439+0000","last_fullsized":"2024-05-10T10:35:52.406439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:36:13.020212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389488+0000","last_change":"2024-05-10T10:35:52.385172+0000","last_active":"2024-05-10T10:35:52.389488+0000","last_peered":"2024-05-10T10:35:52.389488+0000","last_clean":"2024-05-10T10:35:52.389488+0000","last_became_active":"2024-05-10T10:35:52.384900+0000","last_became_peered":"2024-05-10T10:35:52.384900+0000","last_unstale":"2024-05-10T10:35:52.389488+0000","last_undegraded":"2024-05-10T10:35:52.389488+0000","last_fullsized":"2024-05-10T10:35:52.389488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:18:49.471589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404939+0000","last_change":"2024-05-10T10:35:51.377189+0000","last_active":"2024-05-10T10:35:52.404939+0000","last_peered":"2024-05-10T10:35:52.404939+0000","last_clean":"2024-05-10T10:35:52.404939+0000","last_became_active":"2024-05-10T10:35:51.376890+0000","last_became_peered":"2024-05-10T10:35:51.376890+0000","last_unstale":"2024-05-10T10:35:52.404939+0000","last_undegraded":"2024-05-10T10:35:52.404939+0000","last_fullsized":"2024-05-10T10:35:52.404939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:25.672830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388917+0000","last_change":"2024-05-10T10:35:52.379711+0000","last_active":"2024-05-10T10:35:52.388917+0000","last_peered":"2024-05-10T10:35:52.388917+0000","last_clean":"2024-05-10T10:35:52.388917+0000","last_became_active":"2024-05-10T10:35:52.379427+0000","last_became_peered":"2024-05-10T10:35:52.379427+0000","last_unstale":"2024-05-10T10:35:52.388917+0000","last_undegraded":"2024-05-10T10:35:52.388917+0000","last_fullsized":"2024-05-10T10:35:52.388917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:37:31.887382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.374279+0000","last_change":"2024-05-10T10:35:51.373731+0000","last_active":"2024-05-10T10:35:52.374279+0000","last_peered":"2024-05-10T10:35:52.374279+0000","last_clean":"2024-05-10T10:35:52.374279+0000","last_became_active":"2024-05-10T10:35:51.373466+0000","last_became_peered":"2024-05-10T10:35:51.373466+0000","last_unstale":"2024-05-10T10:35:52.374279+0000","last_undegraded":"2024-05-10T10:35:52.374279+0000","last_fullsized":"2024-05-10T10:35:52.374279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:15.713640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398991+0000","last_change":"2024-05-10T10:35:52.397347+0000","last_active":"2024-05-10T10:35:52.398991+0000","last_peered":"2024-05-10T10:35:52.398991+0000","last_clean":"2024-05-10T10:35:52.398991+0000","last_became_active":"2024-05-10T10:35:52.397075+0000","last_became_peered":"2024-05-10T10:35:52.397075+0000","last_unstale":"2024-05-10T10:35:52.398991+0000","last_undegraded":"2024-05-10T10:35:52.398991+0000","last_fullsized":"2024-05-10T10:35:52.398991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:39.477272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.381186+0000","last_change":"2024-05-10T10:35:51.377851+0000","last_active":"2024-05-10T10:35:52.381186+0000","last_peered":"2024-05-10T10:35:52.381186+0000","last_clean":"2024-05-10T10:35:52.381186+0000","last_became_active":"2024-05-10T10:35:51.377560+0000","last_became_peered":"2024-05-10T10:35:51.377560+0000","last_unstale":"2024-05-10T10:35:52.381186+0000","last_undegraded":"2024-05-10T10:35:52.381186+0000","last_fullsized":"2024-05-10T10:35:52.381186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:37:30.882726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394135+0000","last_change":"2024-05-10T10:35:52.391948+0000","last_active":"2024-05-10T10:35:52.394135+0000","last_peered":"2024-05-10T10:35:52.394135+0000","last_clean":"2024-05-10T10:35:52.394135+0000","last_became_active":"2024-05-10T10:35:52.391664+0000","last_became_peered":"2024-05-10T10:35:52.391664+0000","last_unstale":"2024-05-10T10:35:52.394135+0000","last_undegraded":"2024-05-10T10:35:52.394135+0000","last_fullsized":"2024-05-10T10:35:52.394135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:21:59.559549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.405024+0000","last_change":"2024-05-10T10:35:51.370598+0000","last_active":"2024-05-10T10:35:52.405024+0000","last_peered":"2024-05-10T10:35:52.405024+0000","last_clean":"2024-05-10T10:35:52.405024+0000","last_became_active":"2024-05-10T10:35:51.370317+0000","last_became_peered":"2024-05-10T10:35:51.370317+0000","last_unstale":"2024-05-10T10:35:52.405024+0000","last_undegraded":"2024-05-10T10:35:52.405024+0000","last_fullsized":"2024-05-10T10:35:52.405024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:17:22.980706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392049+0000","last_change":"2024-05-10T10:35:52.389879+0000","last_active":"2024-05-10T10:35:52.392049+0000","last_peered":"2024-05-10T10:35:52.392049+0000","last_clean":"2024-05-10T10:35:52.392049+0000","last_became_active":"2024-05-10T10:35:52.389104+0000","last_became_peered":"2024-05-10T10:35:52.389104+0000","last_unstale":"2024-05-10T10:35:52.392049+0000","last_undegraded":"2024-05-10T10:35:52.392049+0000","last_fullsized":"2024-05-10T10:35:52.392049+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:45:33.469087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399095+0000","last_change":"2024-05-10T10:35:51.372708+0000","last_active":"2024-05-10T10:35:52.399095+0000","last_peered":"2024-05-10T10:35:52.399095+0000","last_clean":"2024-05-10T10:35:52.399095+0000","last_became_active":"2024-05-10T10:35:51.372437+0000","last_became_peered":"2024-05-10T10:35:51.372437+0000","last_unstale":"2024-05-10T10:35:52.399095+0000","last_undegraded":"2024-05-10T10:35:52.399095+0000","last_fullsized":"2024-05-10T10:35:52.399095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:07:00.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398365+0000","last_change":"2024-05-10T10:35:52.396853+0000","last_active":"2024-05-10T10:35:52.398365+0000","last_peered":"2024-05-10T10:35:52.398365+0000","last_clean":"2024-05-10T10:35:52.398365+0000","last_became_active":"2024-05-10T10:35:52.396319+0000","last_became_peered":"2024-05-10T10:35:52.396319+0000","last_unstale":"2024-05-10T10:35:52.398365+0000","last_undegraded":"2024-05-10T10:35:52.398365+0000","last_fullsized":"2024-05-10T10:35:52.398365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:06.891450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404299+0000","last_change":"2024-05-10T10:35:51.375956+0000","last_active":"2024-05-10T10:35:52.404299+0000","last_peered":"2024-05-10T10:35:52.404299+0000","last_clean":"2024-05-10T10:35:52.404299+0000","last_became_active":"2024-05-10T10:35:51.375658+0000","last_became_peered":"2024-05-10T10:35:51.375658+0000","last_unstale":"2024-05-10T10:35:52.404299+0000","last_undegraded":"2024-05-10T10:35:52.404299+0000","last_fullsized":"2024-05-10T10:35:52.404299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:58.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395468+0000","last_change":"2024-05-10T10:35:52.392895+0000","last_active":"2024-05-10T10:35:52.395468+0000","last_peered":"2024-05-10T10:35:52.395468+0000","last_clean":"2024-05-10T10:35:52.395468+0000","last_became_active":"2024-05-10T10:35:52.392615+0000","last_became_peered":"2024-05-10T10:35:52.392615+0000","last_unstale":"2024-05-10T10:35:52.395468+0000","last_undegraded":"2024-05-10T10:35:52.395468+0000","last_fullsized":"2024-05-10T10:35:52.395468+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:13:51.776442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404655+0000","last_change":"2024-05-10T10:35:51.378399+0000","last_active":"2024-05-10T10:35:52.404655+0000","last_peered":"2024-05-10T10:35:52.404655+0000","last_clean":"2024-05-10T10:35:52.404655+0000","last_became_active":"2024-05-10T10:35:51.378093+0000","last_became_peered":"2024-05-10T10:35:51.378093+0000","last_unstale":"2024-05-10T10:35:52.404655+0000","last_undegraded":"2024-05-10T10:35:52.404655+0000","last_fullsized":"2024-05-10T10:35:52.404655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:09.391715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399817+0000","last_change":"2024-05-10T10:35:52.398895+0000","last_active":"2024-05-10T10:35:52.399817+0000","last_peered":"2024-05-10T10:35:52.399817+0000","last_clean":"2024-05-10T10:35:52.399817+0000","last_became_active":"2024-05-10T10:35:52.398596+0000","last_became_peered":"2024-05-10T10:35:52.398596+0000","last_unstale":"2024-05-10T10:35:52.399817+0000","last_undegraded":"2024-05-10T10:35:52.399817+0000","last_fullsized":"2024-05-10T10:35:52.399817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:58:12.841201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.377208+0000","last_change":"2024-05-10T10:35:51.371359+0000","last_active":"2024-05-10T10:35:52.377208+0000","last_peered":"2024-05-10T10:35:52.377208+0000","last_clean":"2024-05-10T10:35:52.377208+0000","last_became_active":"2024-05-10T10:35:51.371084+0000","last_became_peered":"2024-05-10T10:35:51.371084+0000","last_unstale":"2024-05-10T10:35:52.377208+0000","last_undegraded":"2024-05-10T10:35:52.377208+0000","last_fullsized":"2024-05-10T10:35:52.377208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:05.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387586+0000","last_change":"2024-05-10T10:35:52.381017+0000","last_active":"2024-05-10T10:35:52.387586+0000","last_peered":"2024-05-10T10:35:52.387586+0000","last_clean":"2024-05-10T10:35:52.387586+0000","last_became_active":"2024-05-10T10:35:52.380578+0000","last_became_peered":"2024-05-10T10:35:52.380578+0000","last_unstale":"2024-05-10T10:35:52.387586+0000","last_undegraded":"2024-05-10T10:35:52.387586+0000","last_fullsized":"2024-05-10T10:35:52.387586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:45:33.469087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.403920+0000","last_change":"2024-05-10T10:35:51.370738+0000","last_active":"2024-05-10T10:35:52.403920+0000","last_peered":"2024-05-10T10:35:52.403920+0000","last_clean":"2024-05-10T10:35:52.403920+0000","last_became_active":"2024-05-10T10:35:51.370244+0000","last_became_peered":"2024-05-10T10:35:51.370244+0000","last_unstale":"2024-05-10T10:35:52.403920+0000","last_undegraded":"2024-05-10T10:35:52.403920+0000","last_fullsized":"2024-05-10T10:35:52.403920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:09.391715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388942+0000","last_change":"2024-05-10T10:35:52.381305+0000","last_active":"2024-05-10T10:35:52.388942+0000","last_peered":"2024-05-10T10:35:52.388942+0000","last_clean":"2024-05-10T10:35:52.388942+0000","last_became_active":"2024-05-10T10:35:52.381050+0000","last_became_peered":"2024-05-10T10:35:52.381050+0000","last_unstale":"2024-05-10T10:35:52.388942+0000","last_undegraded":"2024-05-10T10:35:52.388942+0000","last_fullsized":"2024-05-10T10:35:52.388942+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:34.682501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.374339+0000","last_change":"2024-05-10T10:35:51.370997+0000","last_active":"2024-05-10T10:35:52.374339+0000","last_peered":"2024-05-10T10:35:52.374339+0000","last_clean":"2024-05-10T10:35:52.374339+0000","last_became_active":"2024-05-10T10:35:51.370595+0000","last_became_peered":"2024-05-10T10:35:51.370595+0000","last_unstale":"2024-05-10T10:35:52.374339+0000","last_undegraded":"2024-05-10T10:35:52.374339+0000","last_fullsized":"2024-05-10T10:35:52.374339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:21:58.554894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398349+0000","last_change":"2024-05-10T10:35:52.396926+0000","last_active":"2024-05-10T10:35:52.398349+0000","last_peered":"2024-05-10T10:35:52.398349+0000","last_clean":"2024-05-10T10:35:52.398349+0000","last_became_active":"2024-05-10T10:35:52.396644+0000","last_became_peered":"2024-05-10T10:35:52.396644+0000","last_unstale":"2024-05-10T10:35:52.398349+0000","last_undegraded":"2024-05-10T10:35:52.398349+0000","last_fullsized":"2024-05-10T10:35:52.398349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:09.843423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.372584+0000","last_change":"2024-05-10T10:35:51.375022+0000","last_active":"2024-05-10T10:35:52.372584+0000","last_peered":"2024-05-10T10:35:52.372584+0000","last_clean":"2024-05-10T10:35:52.372584+0000","last_became_active":"2024-05-10T10:35:51.374665+0000","last_became_peered":"2024-05-10T10:35:51.374665+0000","last_unstale":"2024-05-10T10:35:52.372584+0000","last_undegraded":"2024-05-10T10:35:52.372584+0000","last_fullsized":"2024-05-10T10:35:52.372584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:33.677845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387974+0000","last_change":"2024-05-10T10:35:52.383509+0000","last_active":"2024-05-10T10:35:52.387974+0000","last_peered":"2024-05-10T10:35:52.387974+0000","last_clean":"2024-05-10T10:35:52.387974+0000","last_became_active":"2024-05-10T10:35:52.383189+0000","last_became_peered":"2024-05-10T10:35:52.383189+0000","last_unstale":"2024-05-10T10:35:52.387974+0000","last_undegraded":"2024-05-10T10:35:52.387974+0000","last_fullsized":"2024-05-10T10:35:52.387974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:12:58.694304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.400838+0000","last_change":"2024-05-10T10:35:51.375850+0000","last_active":"2024-05-10T10:35:52.400838+0000","last_peered":"2024-05-10T10:35:52.400838+0000","last_clean":"2024-05-10T10:35:52.400838+0000","last_became_active":"2024-05-10T10:35:51.375566+0000","last_became_peered":"2024-05-10T10:35:51.375566+0000","last_unstale":"2024-05-10T10:35:52.400838+0000","last_undegraded":"2024-05-10T10:35:52.400838+0000","last_fullsized":"2024-05-10T10:35:52.400838+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:02.487122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391339+0000","last_change":"2024-05-10T10:35:52.389183+0000","last_active":"2024-05-10T10:35:52.391339+0000","last_peered":"2024-05-10T10:35:52.391339+0000","last_clean":"2024-05-10T10:35:52.391339+0000","last_became_active":"2024-05-10T10:35:52.388753+0000","last_became_peered":"2024-05-10T10:35:52.388753+0000","last_unstale":"2024-05-10T10:35:52.391339+0000","last_undegraded":"2024-05-10T10:35:52.391339+0000","last_fullsized":"2024-05-10T10:35:52.391339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:32.463915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.374995+0000","last_change":"2024-05-10T10:35:51.374976+0000","last_active":"2024-05-10T10:35:52.374995+0000","last_peered":"2024-05-10T10:35:52.374995+0000","last_clean":"2024-05-10T10:35:52.374995+0000","last_became_active":"2024-05-10T10:35:51.374647+0000","last_became_peered":"2024-05-10T10:35:51.374647+0000","last_unstale":"2024-05-10T10:35:52.374995+0000","last_undegraded":"2024-05-10T10:35:52.374995+0000","last_fullsized":"2024-05-10T10:35:52.374995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:14.372647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388835+0000","last_change":"2024-05-10T10:35:52.379131+0000","last_active":"2024-05-10T10:35:52.388835+0000","last_peered":"2024-05-10T10:35:52.388835+0000","last_clean":"2024-05-10T10:35:52.388835+0000","last_became_active":"2024-05-10T10:35:52.378848+0000","last_became_peered":"2024-05-10T10:35:52.378848+0000","last_unstale":"2024-05-10T10:35:52.388835+0000","last_undegraded":"2024-05-10T10:35:52.388835+0000","last_fullsized":"2024-05-10T10:35:52.388835+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:26.677485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.372782+0000","last_change":"2024-05-10T10:35:51.371489+0000","last_active":"2024-05-10T10:35:52.372782+0000","last_peered":"2024-05-10T10:35:52.372782+0000","last_clean":"2024-05-10T10:35:52.372782+0000","last_became_active":"2024-05-10T10:35:51.371217+0000","last_became_peered":"2024-05-10T10:35:51.371217+0000","last_unstale":"2024-05-10T10:35:52.372782+0000","last_undegraded":"2024-05-10T10:35:52.372782+0000","last_fullsized":"2024-05-10T10:35:52.372782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:34.096486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391549+0000","last_change":"2024-05-10T10:35:52.390182+0000","last_active":"2024-05-10T10:35:52.391549+0000","last_peered":"2024-05-10T10:35:52.391549+0000","last_clean":"2024-05-10T10:35:52.391549+0000","last_became_active":"2024-05-10T10:35:52.389901+0000","last_became_peered":"2024-05-10T10:35:52.389901+0000","last_unstale":"2024-05-10T10:35:52.391549+0000","last_undegraded":"2024-05-10T10:35:52.391549+0000","last_fullsized":"2024-05-10T10:35:52.391549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:57.321844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.377634+0000","last_change":"2024-05-10T10:35:51.374675+0000","last_active":"2024-05-10T10:35:52.377634+0000","last_peered":"2024-05-10T10:35:52.377634+0000","last_clean":"2024-05-10T10:35:52.377634+0000","last_became_active":"2024-05-10T10:35:51.374386+0000","last_became_peered":"2024-05-10T10:35:51.374386+0000","last_unstale":"2024-05-10T10:35:52.377634+0000","last_undegraded":"2024-05-10T10:35:52.377634+0000","last_fullsized":"2024-05-10T10:35:52.377634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:28.094091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.396189+0000","last_change":"2024-05-10T10:35:52.393422+0000","last_active":"2024-05-10T10:35:52.396189+0000","last_peered":"2024-05-10T10:35:52.396189+0000","last_clean":"2024-05-10T10:35:52.396189+0000","last_became_active":"2024-05-10T10:35:52.393034+0000","last_became_peered":"2024-05-10T10:35:52.393034+0000","last_unstale":"2024-05-10T10:35:52.396189+0000","last_undegraded":"2024-05-10T10:35:52.396189+0000","last_fullsized":"2024-05-10T10:35:52.396189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:36:49.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.380655+0000","last_change":"2024-05-10T10:35:51.376564+0000","last_active":"2024-05-10T10:35:52.380655+0000","last_peered":"2024-05-10T10:35:52.380655+0000","last_clean":"2024-05-10T10:35:52.380655+0000","last_became_active":"2024-05-10T10:35:51.376237+0000","last_became_peered":"2024-05-10T10:35:51.376237+0000","last_unstale":"2024-05-10T10:35:52.380655+0000","last_undegraded":"2024-05-10T10:35:52.380655+0000","last_fullsized":"2024-05-10T10:35:52.380655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:14.372647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389450+0000","last_change":"2024-05-10T10:35:52.385985+0000","last_active":"2024-05-10T10:35:52.389450+0000","last_peered":"2024-05-10T10:35:52.389450+0000","last_clean":"2024-05-10T10:35:52.389450+0000","last_became_active":"2024-05-10T10:35:52.385712+0000","last_became_peered":"2024-05-10T10:35:52.385712+0000","last_unstale":"2024-05-10T10:35:52.389450+0000","last_undegraded":"2024-05-10T10:35:52.389450+0000","last_fullsized":"2024-05-10T10:35:52.389450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:43.953297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.376743+0000","last_change":"2024-05-10T10:35:51.368388+0000","last_active":"2024-05-10T10:35:52.376743+0000","last_peered":"2024-05-10T10:35:52.376743+0000","last_clean":"2024-05-10T10:35:52.376743+0000","last_became_active":"2024-05-10T10:35:51.368118+0000","last_became_peered":"2024-05-10T10:35:51.368118+0000","last_unstale":"2024-05-10T10:35:52.376743+0000","last_undegraded":"2024-05-10T10:35:52.376743+0000","last_fullsized":"2024-05-10T10:35:52.376743+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:39:46.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393455+0000","last_change":"2024-05-10T10:35:52.390378+0000","last_active":"2024-05-10T10:35:52.393455+0000","last_peered":"2024-05-10T10:35:52.393455+0000","last_clean":"2024-05-10T10:35:52.393455+0000","last_became_active":"2024-05-10T10:35:52.390085+0000","last_became_peered":"2024-05-10T10:35:52.390085+0000","last_unstale":"2024-05-10T10:35:52.393455+0000","last_undegraded":"2024-05-10T10:35:52.393455+0000","last_fullsized":"2024-05-10T10:35:52.393455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:05:17.155890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.376270+0000","last_change":"2024-05-10T10:35:51.373906+0000","last_active":"2024-05-10T10:35:52.376270+0000","last_peered":"2024-05-10T10:35:52.376270+0000","last_clean":"2024-05-10T10:35:52.376270+0000","last_became_active":"2024-05-10T10:35:51.373166+0000","last_became_peered":"2024-05-10T10:35:51.373166+0000","last_unstale":"2024-05-10T10:35:52.376270+0000","last_undegraded":"2024-05-10T10:35:52.376270+0000","last_fullsized":"2024-05-10T10:35:52.376270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:17:25.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393460+0000","last_change":"2024-05-10T10:35:52.391115+0000","last_active":"2024-05-10T10:35:52.393460+0000","last_peered":"2024-05-10T10:35:52.393460+0000","last_clean":"2024-05-10T10:35:52.393460+0000","last_became_active":"2024-05-10T10:35:52.390395+0000","last_became_peered":"2024-05-10T10:35:52.390395+0000","last_unstale":"2024-05-10T10:35:52.393460+0000","last_undegraded":"2024-05-10T10:35:52.393460+0000","last_fullsized":"2024-05-10T10:35:52.393460+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:42:24.686955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.375963+0000","last_change":"2024-05-10T10:35:51.372214+0000","last_active":"2024-05-10T10:35:52.375963+0000","last_peered":"2024-05-10T10:35:52.375963+0000","last_clean":"2024-05-10T10:35:52.375963+0000","last_became_active":"2024-05-10T10:35:51.371912+0000","last_became_peered":"2024-05-10T10:35:51.371912+0000","last_unstale":"2024-05-10T10:35:52.375963+0000","last_undegraded":"2024-05-10T10:35:52.375963+0000","last_fullsized":"2024-05-10T10:35:52.375963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:50.184685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.397680+0000","last_change":"2024-05-10T10:35:52.396014+0000","last_active":"2024-05-10T10:35:52.397680+0000","last_peered":"2024-05-10T10:35:52.397680+0000","last_clean":"2024-05-10T10:35:52.397680+0000","last_became_active":"2024-05-10T10:35:52.395752+0000","last_became_peered":"2024-05-10T10:35:52.395752+0000","last_unstale":"2024-05-10T10:35:52.397680+0000","last_undegraded":"2024-05-10T10:35:52.397680+0000","last_fullsized":"2024-05-10T10:35:52.397680+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:43.953297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.375606+0000","last_change":"2024-05-10T10:35:51.365519+0000","last_active":"2024-05-10T10:35:52.375606+0000","last_peered":"2024-05-10T10:35:52.375606+0000","last_clean":"2024-05-10T10:35:52.375606+0000","last_became_active":"2024-05-10T10:35:51.365224+0000","last_became_peered":"2024-05-10T10:35:51.365224+0000","last_unstale":"2024-05-10T10:35:52.375606+0000","last_undegraded":"2024-05-10T10:35:52.375606+0000","last_fullsized":"2024-05-10T10:35:52.375606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:17.967992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":191,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.379799+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:35:52.379799+0000","last_peered":"2024-05-10T10:35:52.379799+0000","last_clean":"2024-05-10T10:35:52.379799+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:35:52.379799+0000","last_undegraded":"2024-05-10T10:35:52.379799+0000","last_fullsized":"2024-05-10T10:35:52.379799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1306624,"data_stored":905485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":322318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":322306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":315117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":724992,"data_stored":322284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":724992,"data_stored":322284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":753664,"data_stored":324660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1318912,"data_stored":912652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:35:56.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:35:56.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.571+0000 7fdef570a640 1 Processor -- start 2024-05-10T10:35:56.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.571+0000 7fdef570a640 1 -- start start 2024-05-10T10:35:56.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdef570a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdef00702e0 0x7fdef00d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdef570a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef006f3c0 con 0x7fdef015f290 2024-05-10T10:35:56.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdef570a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef00d16f0 con 0x7fdef00d3450 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdef570a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fdef00d14c0 con 0x7fdef00702e0 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdeef7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdef00702e0 0x7fdef00d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdeeeffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdef015f290 0x7fdef00d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48944/0 (socket says 172.21.15.53:48944) 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdeeeffd640 1 -- 172.21.15.53:0/4099563002 learned_addr learned my addr 172.21.15.53:0/4099563002 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.572+0000 7fdeef7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdef00702e0 0x7fdef00d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56574/0 (socket says 172.21.15.53:56574) 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.573+0000 7fdeef7fe640 1 -- 172.21.15.53:0/4099563002 >> v1:172.21.15.135:6789/0 conn(0x7fdef00d3450 legacy=0x7fdef006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.573+0000 7fdeef7fe640 1 -- 172.21.15.53:0/4099563002 >> v1:172.21.15.53:6789/0 conn(0x7fdef015f290 legacy=0x7fdef00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.573+0000 7fdeedffb640 1 -- 172.21.15.53:0/4099563002 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571149622 0 0) 0x7fdef006f3c0 con 0x7fdef015f290 2024-05-10T10:35:56.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.573+0000 7fdeef7fe640 1 -- 172.21.15.53:0/4099563002 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef00d19d0 con 0x7fdef00702e0 2024-05-10T10:35:56.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.573+0000 7fdeef7fe640 1 --2- 172.21.15.53:0/4099563002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef00d0f80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdee00097a0 tx=0x7fdee002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4985bef6b162a8c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdeedffb640 1 -- 172.21.15.53:0/4099563002 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0037070 con 0x7fdef00702e0 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdeedffb640 1 -- 172.21.15.53:0/4099563002 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee0032640 con 0x7fdef00702e0 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdeedffb640 1 -- 172.21.15.53:0/4099563002 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0032a10 con 0x7fdef00702e0 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdef570a640 1 -- 172.21.15.53:0/4099563002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 msgr2=0x7fdef00d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdef570a640 1 --2- 172.21.15.53:0/4099563002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef00d0f80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdee00097a0 tx=0x7fdee002a2c0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdef570a640 1 -- 172.21.15.53:0/4099563002 shutdown_connections 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdef570a640 1 --2- 172.21.15.53:0/4099563002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef00d0f80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdef570a640 1 -- 172.21.15.53:0/4099563002 >> 172.21.15.53:0/4099563002 conn(0x7fdef009ba20 msgr2=0x7fdef009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdef570a640 1 -- 172.21.15.53:0/4099563002 shutdown_connections 2024-05-10T10:35:56.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.574+0000 7fdef570a640 1 -- 172.21.15.53:0/4099563002 wait complete. 2024-05-10T10:35:56.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 Processor -- start 2024-05-10T10:35:56.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 -- start start 2024-05-10T10:35:56.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef01f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdef00d3450 0x7fdef00b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdef015f290 0x7fdef00b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdef00d1f90 con 0x7fdef00702e0 2024-05-10T10:35:56.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdef00d2300 con 0x7fdef00d3450 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdef570a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdef015f670 con 0x7fdef015f290 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.576+0000 7fdeeeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef01f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef01f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56576/0 (socket says 172.21.15.53:56576) 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 -- 172.21.15.53:0/1494241449 learned_addr learned my addr 172.21.15.53:0/1494241449 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 -- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdef015f290 msgr2=0x7fdef00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdef015f290 0x7fdef00b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 -- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdef00d3450 msgr2=0x7fdef00b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdef00d3450 0x7fdef00b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 -- 172.21.15.53:0/1494241449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef009ff60 con 0x7fdef00702e0 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdeeeffd640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef01f5480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c8c0 tx=0x7fdee800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdee77fe640 1 -- 172.21.15.53:0/1494241449 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8018070 con 0x7fdef00702e0 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdee77fe640 1 -- 172.21.15.53:0/1494241449 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee800a640 con 0x7fdef00702e0 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdee77fe640 1 -- 172.21.15.53:0/1494241449 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee800aa70 con 0x7fdef00702e0 2024-05-10T10:35:56.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef00a0240 con 0x7fdef00702e0 2024-05-10T10:35:56.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.577+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef00a0800 con 0x7fdef00702e0 2024-05-10T10:35:56.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.578+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef00693f0 con 0x7fdef00702e0 2024-05-10T10:35:56.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.579+0000 7fdee77fe640 1 -- 172.21.15.53:0/1494241449 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdee801b070 con 0x7fdef00702e0 2024-05-10T10:35:56.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.579+0000 7fdee77fe640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fded00762d0 0x7fded0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.580+0000 7fdee77fe640 1 -- 172.21.15.53:0/1494241449 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fdee8096dd0 con 0x7fdef00702e0 2024-05-10T10:35:56.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.580+0000 7fdeee7fc640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fded00762d0 0x7fded0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.580+0000 7fdeee7fc640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fded00762d0 0x7fded0078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fded8000f60 tx=0x7fded80079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.584+0000 7fdee77fe640 1 -- 172.21.15.53:0/1494241449 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdee8060630 con 0x7fdef00702e0 2024-05-10T10:35:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.774+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdef0070c20 con 0x7fded00762d0 2024-05-10T10:35:56.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.780+0000 7fdee77fe640 1 -- 172.21.15.53:0/1494241449 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251938 (secure 0 0 0) 0x7fdef0070c20 con 0x7fded00762d0 2024-05-10T10:35:56.783 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:35:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.783+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fded00762d0 msgr2=0x7fded0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.783+0000 7fdef570a640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fded00762d0 0x7fded0078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fded8000f60 tx=0x7fded80079b0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.783+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 msgr2=0x7fdef01f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.783+0000 7fdef570a640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef01f5480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c8c0 tx=0x7fdee800cd80 comp rx=0 tx=0).stop 2024-05-10T10:35:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 shutdown_connections 2024-05-10T10:35:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fded00762d0 0x7fded0078790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdef015f290 0x7fdef00b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdef00d3450 0x7fdef00b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 --2- 172.21.15.53:0/1494241449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00702e0 0x7fdef01f5480 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 >> 172.21.15.53:0/1494241449 conn(0x7fdef009ba20 msgr2=0x7fdef00b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 shutdown_connections 2024-05-10T10:35:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.784+0000 7fdef570a640 1 -- 172.21.15.53:0/1494241449 wait complete. 2024-05-10T10:35:56.795 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-05-10T10:35:56.188670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239660,"kb_used_data":6836,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735060,"statfs":{"total":773094113280,"available":772848701440,"internally_reserved":0,"allocated":7000064,"data_stored":3747106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.834374"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393491+0000","last_change":"2024-05-10T10:35:52.390729+0000","last_active":"2024-05-10T10:35:52.393491+0000","last_peered":"2024-05-10T10:35:52.393491+0000","last_clean":"2024-05-10T10:35:52.393491+0000","last_became_active":"2024-05-10T10:35:52.390434+0000","last_became_peered":"2024-05-10T10:35:52.390434+0000","last_unstale":"2024-05-10T10:35:52.393491+0000","last_undegraded":"2024-05-10T10:35:52.393491+0000","last_fullsized":"2024-05-10T10:35:52.393491+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:48.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388572+0000","last_change":"2024-05-10T10:35:52.385215+0000","last_active":"2024-05-10T10:35:52.388572+0000","last_peered":"2024-05-10T10:35:52.388572+0000","last_clean":"2024-05-10T10:35:52.388572+0000","last_became_active":"2024-05-10T10:35:52.384753+0000","last_became_peered":"2024-05-10T10:35:52.384753+0000","last_unstale":"2024-05-10T10:35:52.388572+0000","last_undegraded":"2024-05-10T10:35:52.388572+0000","last_fullsized":"2024-05-10T10:35:52.388572+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:26.783046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393935+0000","last_change":"2024-05-10T10:35:52.389134+0000","last_active":"2024-05-10T10:35:52.393935+0000","last_peered":"2024-05-10T10:35:52.393935+0000","last_clean":"2024-05-10T10:35:52.393935+0000","last_became_active":"2024-05-10T10:35:52.388847+0000","last_became_peered":"2024-05-10T10:35:52.388847+0000","last_unstale":"2024-05-10T10:35:52.393935+0000","last_undegraded":"2024-05-10T10:35:52.393935+0000","last_fullsized":"2024-05-10T10:35:52.393935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:35.101141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388387+0000","last_change":"2024-05-10T10:35:52.382733+0000","last_active":"2024-05-10T10:35:52.388387+0000","last_peered":"2024-05-10T10:35:52.388387+0000","last_clean":"2024-05-10T10:35:52.388387+0000","last_became_active":"2024-05-10T10:35:52.382374+0000","last_became_peered":"2024-05-10T10:35:52.382374+0000","last_unstale":"2024-05-10T10:35:52.388387+0000","last_undegraded":"2024-05-10T10:35:52.388387+0000","last_fullsized":"2024-05-10T10:35:52.388387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:41.527270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.397655+0000","last_change":"2024-05-10T10:35:52.395323+0000","last_active":"2024-05-10T10:35:52.397655+0000","last_peered":"2024-05-10T10:35:52.397655+0000","last_clean":"2024-05-10T10:35:52.397655+0000","last_became_active":"2024-05-10T10:35:52.395039+0000","last_became_peered":"2024-05-10T10:35:52.395039+0000","last_unstale":"2024-05-10T10:35:52.397655+0000","last_undegraded":"2024-05-10T10:35:52.397655+0000","last_fullsized":"2024-05-10T10:35:52.397655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:30:49.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392062+0000","last_change":"2024-05-10T10:35:52.387021+0000","last_active":"2024-05-10T10:35:52.392062+0000","last_peered":"2024-05-10T10:35:52.392062+0000","last_clean":"2024-05-10T10:35:52.392062+0000","last_became_active":"2024-05-10T10:35:52.386736+0000","last_became_peered":"2024-05-10T10:35:52.386736+0000","last_unstale":"2024-05-10T10:35:52.392062+0000","last_undegraded":"2024-05-10T10:35:52.392062+0000","last_fullsized":"2024-05-10T10:35:52.392062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:18.972647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.397323+0000","last_change":"2024-05-10T10:35:52.394532+0000","last_active":"2024-05-10T10:35:52.397323+0000","last_peered":"2024-05-10T10:35:52.397323+0000","last_clean":"2024-05-10T10:35:52.397323+0000","last_became_active":"2024-05-10T10:35:52.394073+0000","last_became_peered":"2024-05-10T10:35:52.394073+0000","last_unstale":"2024-05-10T10:35:52.397323+0000","last_undegraded":"2024-05-10T10:35:52.397323+0000","last_fullsized":"2024-05-10T10:35:52.397323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:59:03.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394813+0000","last_change":"2024-05-10T10:35:52.392977+0000","last_active":"2024-05-10T10:35:52.394813+0000","last_peered":"2024-05-10T10:35:52.394813+0000","last_clean":"2024-05-10T10:35:52.394813+0000","last_became_active":"2024-05-10T10:35:52.392646+0000","last_became_peered":"2024-05-10T10:35:52.392646+0000","last_unstale":"2024-05-10T10:35:52.394813+0000","last_undegraded":"2024-05-10T10:35:52.394813+0000","last_fullsized":"2024-05-10T10:35:52.394813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:12:03.431670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398086+0000","last_change":"2024-05-10T10:35:52.395098+0000","last_active":"2024-05-10T10:35:52.398086+0000","last_peered":"2024-05-10T10:35:52.398086+0000","last_clean":"2024-05-10T10:35:52.398086+0000","last_became_active":"2024-05-10T10:35:52.394824+0000","last_became_peered":"2024-05-10T10:35:52.394824+0000","last_unstale":"2024-05-10T10:35:52.398086+0000","last_undegraded":"2024-05-10T10:35:52.398086+0000","last_fullsized":"2024-05-10T10:35:52.398086+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:37:17.187253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391940+0000","last_change":"2024-05-10T10:35:52.389825+0000","last_active":"2024-05-10T10:35:52.391940+0000","last_peered":"2024-05-10T10:35:52.391940+0000","last_clean":"2024-05-10T10:35:52.391940+0000","last_became_active":"2024-05-10T10:35:52.389470+0000","last_became_peered":"2024-05-10T10:35:52.389470+0000","last_unstale":"2024-05-10T10:35:52.391940+0000","last_undegraded":"2024-05-10T10:35:52.391940+0000","last_fullsized":"2024-05-10T10:35:52.391940+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:30:00.573188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395385+0000","last_change":"2024-05-10T10:35:52.392597+0000","last_active":"2024-05-10T10:35:52.395385+0000","last_peered":"2024-05-10T10:35:52.395385+0000","last_clean":"2024-05-10T10:35:52.395385+0000","last_became_active":"2024-05-10T10:35:52.391875+0000","last_became_peered":"2024-05-10T10:35:52.391875+0000","last_unstale":"2024-05-10T10:35:52.395385+0000","last_undegraded":"2024-05-10T10:35:52.395385+0000","last_fullsized":"2024-05-10T10:35:52.395385+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:06.891450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395393+0000","last_change":"2024-05-10T10:35:52.392914+0000","last_active":"2024-05-10T10:35:52.395393+0000","last_peered":"2024-05-10T10:35:52.395393+0000","last_clean":"2024-05-10T10:35:52.395393+0000","last_became_active":"2024-05-10T10:35:52.392605+0000","last_became_peered":"2024-05-10T10:35:52.392605+0000","last_unstale":"2024-05-10T10:35:52.395393+0000","last_undegraded":"2024-05-10T10:35:52.395393+0000","last_fullsized":"2024-05-10T10:35:52.395393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:45:33.469087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388734+0000","last_change":"2024-05-10T10:35:52.382317+0000","last_active":"2024-05-10T10:35:52.388734+0000","last_peered":"2024-05-10T10:35:52.388734+0000","last_clean":"2024-05-10T10:35:52.388734+0000","last_became_active":"2024-05-10T10:35:52.382042+0000","last_became_peered":"2024-05-10T10:35:52.382042+0000","last_unstale":"2024-05-10T10:35:52.388734+0000","last_undegraded":"2024-05-10T10:35:52.388734+0000","last_fullsized":"2024-05-10T10:35:52.388734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:59.312394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394521+0000","last_change":"2024-05-10T10:35:52.390761+0000","last_active":"2024-05-10T10:35:52.394521+0000","last_peered":"2024-05-10T10:35:52.394521+0000","last_clean":"2024-05-10T10:35:52.394521+0000","last_became_active":"2024-05-10T10:35:52.390444+0000","last_became_peered":"2024-05-10T10:35:52.390444+0000","last_unstale":"2024-05-10T10:35:52.394521+0000","last_undegraded":"2024-05-10T10:35:52.394521+0000","last_fullsized":"2024-05-10T10:35:52.394521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:12:58.694304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392326+0000","last_change":"2024-05-10T10:35:52.391281+0000","last_active":"2024-05-10T10:35:52.392326+0000","last_peered":"2024-05-10T10:35:52.392326+0000","last_clean":"2024-05-10T10:35:52.392326+0000","last_became_active":"2024-05-10T10:35:52.390834+0000","last_became_peered":"2024-05-10T10:35:52.390834+0000","last_unstale":"2024-05-10T10:35:52.392326+0000","last_undegraded":"2024-05-10T10:35:52.392326+0000","last_fullsized":"2024-05-10T10:35:52.392326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:39:26.071931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389970+0000","last_change":"2024-05-10T10:35:52.387019+0000","last_active":"2024-05-10T10:35:52.389970+0000","last_peered":"2024-05-10T10:35:52.389970+0000","last_clean":"2024-05-10T10:35:52.389970+0000","last_became_active":"2024-05-10T10:35:52.386736+0000","last_became_peered":"2024-05-10T10:35:52.386736+0000","last_unstale":"2024-05-10T10:35:52.389970+0000","last_undegraded":"2024-05-10T10:35:52.389970+0000","last_fullsized":"2024-05-10T10:35:52.389970+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:53:00.799654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394222+0000","last_change":"2024-05-10T10:35:52.392230+0000","last_active":"2024-05-10T10:35:52.394222+0000","last_peered":"2024-05-10T10:35:52.394222+0000","last_clean":"2024-05-10T10:35:52.394222+0000","last_became_active":"2024-05-10T10:35:52.391539+0000","last_became_peered":"2024-05-10T10:35:52.391539+0000","last_unstale":"2024-05-10T10:35:52.394222+0000","last_undegraded":"2024-05-10T10:35:52.394222+0000","last_fullsized":"2024-05-10T10:35:52.394222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:03.491778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388353+0000","last_change":"2024-05-10T10:35:52.382491+0000","last_active":"2024-05-10T10:35:52.388353+0000","last_peered":"2024-05-10T10:35:52.388353+0000","last_clean":"2024-05-10T10:35:52.388353+0000","last_became_active":"2024-05-10T10:35:52.382096+0000","last_became_peered":"2024-05-10T10:35:52.382096+0000","last_unstale":"2024-05-10T10:35:52.388353+0000","last_undegraded":"2024-05-10T10:35:52.388353+0000","last_fullsized":"2024-05-10T10:35:52.388353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:57.321844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.396981+0000","last_change":"2024-05-10T10:35:52.395413+0000","last_active":"2024-05-10T10:35:52.396981+0000","last_peered":"2024-05-10T10:35:52.396981+0000","last_clean":"2024-05-10T10:35:52.396981+0000","last_became_active":"2024-05-10T10:35:52.394773+0000","last_became_peered":"2024-05-10T10:35:52.394773+0000","last_unstale":"2024-05-10T10:35:52.396981+0000","last_undegraded":"2024-05-10T10:35:52.396981+0000","last_fullsized":"2024-05-10T10:35:52.396981+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:18:49.471589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399586+0000","last_change":"2024-05-10T10:35:52.395738+0000","last_active":"2024-05-10T10:35:52.399586+0000","last_peered":"2024-05-10T10:35:52.399586+0000","last_clean":"2024-05-10T10:35:52.399586+0000","last_became_active":"2024-05-10T10:35:52.395466+0000","last_became_peered":"2024-05-10T10:35:52.395466+0000","last_unstale":"2024-05-10T10:35:52.399586+0000","last_undegraded":"2024-05-10T10:35:52.399586+0000","last_fullsized":"2024-05-10T10:35:52.399586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:57:57.565541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.390766+0000","last_change":"2024-05-10T10:35:52.383055+0000","last_active":"2024-05-10T10:35:52.390766+0000","last_peered":"2024-05-10T10:35:52.390766+0000","last_clean":"2024-05-10T10:35:52.390766+0000","last_became_active":"2024-05-10T10:35:52.382633+0000","last_became_peered":"2024-05-10T10:35:52.382633+0000","last_unstale":"2024-05-10T10:35:52.390766+0000","last_undegraded":"2024-05-10T10:35:52.390766+0000","last_fullsized":"2024-05-10T10:35:52.390766+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:35.101141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399550+0000","last_change":"2024-05-10T10:35:52.396692+0000","last_active":"2024-05-10T10:35:52.399550+0000","last_peered":"2024-05-10T10:35:52.399550+0000","last_clean":"2024-05-10T10:35:52.399550+0000","last_became_active":"2024-05-10T10:35:52.396428+0000","last_became_peered":"2024-05-10T10:35:52.396428+0000","last_unstale":"2024-05-10T10:35:52.399550+0000","last_undegraded":"2024-05-10T10:35:52.399550+0000","last_fullsized":"2024-05-10T10:35:52.399550+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:54:29.102508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387551+0000","last_change":"2024-05-10T10:35:52.385788+0000","last_active":"2024-05-10T10:35:52.387551+0000","last_peered":"2024-05-10T10:35:52.387551+0000","last_clean":"2024-05-10T10:35:52.387551+0000","last_became_active":"2024-05-10T10:35:52.385540+0000","last_became_peered":"2024-05-10T10:35:52.385540+0000","last_unstale":"2024-05-10T10:35:52.387551+0000","last_undegraded":"2024-05-10T10:35:52.387551+0000","last_fullsized":"2024-05-10T10:35:52.387551+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:25.736922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391920+0000","last_change":"2024-05-10T10:35:52.389157+0000","last_active":"2024-05-10T10:35:52.391920+0000","last_peered":"2024-05-10T10:35:52.391920+0000","last_clean":"2024-05-10T10:35:52.391920+0000","last_became_active":"2024-05-10T10:35:52.388861+0000","last_became_peered":"2024-05-10T10:35:52.388861+0000","last_unstale":"2024-05-10T10:35:52.391920+0000","last_undegraded":"2024-05-10T10:35:52.391920+0000","last_fullsized":"2024-05-10T10:35:52.391920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:17:05.283765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.390727+0000","last_change":"2024-05-10T10:35:52.383540+0000","last_active":"2024-05-10T10:35:52.390727+0000","last_peered":"2024-05-10T10:35:52.390727+0000","last_clean":"2024-05-10T10:35:52.390727+0000","last_became_active":"2024-05-10T10:35:52.383257+0000","last_became_peered":"2024-05-10T10:35:52.383257+0000","last_unstale":"2024-05-10T10:35:52.390727+0000","last_undegraded":"2024-05-10T10:35:52.390727+0000","last_fullsized":"2024-05-10T10:35:52.390727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:17:05.283765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.390804+0000","last_change":"2024-05-10T10:35:52.383092+0000","last_active":"2024-05-10T10:35:52.390804+0000","last_peered":"2024-05-10T10:35:52.390804+0000","last_clean":"2024-05-10T10:35:52.390804+0000","last_became_active":"2024-05-10T10:35:52.382692+0000","last_became_peered":"2024-05-10T10:35:52.382692+0000","last_unstale":"2024-05-10T10:35:52.390804+0000","last_undegraded":"2024-05-10T10:35:52.390804+0000","last_fullsized":"2024-05-10T10:35:52.390804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:10.396371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388241+0000","last_change":"2024-05-10T10:35:52.383237+0000","last_active":"2024-05-10T10:35:52.388241+0000","last_peered":"2024-05-10T10:35:52.388241+0000","last_clean":"2024-05-10T10:35:52.388241+0000","last_became_active":"2024-05-10T10:35:52.382089+0000","last_became_peered":"2024-05-10T10:35:52.382089+0000","last_unstale":"2024-05-10T10:35:52.388241+0000","last_undegraded":"2024-05-10T10:35:52.388241+0000","last_fullsized":"2024-05-10T10:35:52.388241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:30:49.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394631+0000","last_change":"2024-05-10T10:35:52.393215+0000","last_active":"2024-05-10T10:35:52.394631+0000","last_peered":"2024-05-10T10:35:52.394631+0000","last_clean":"2024-05-10T10:35:52.394631+0000","last_became_active":"2024-05-10T10:35:52.392938+0000","last_became_peered":"2024-05-10T10:35:52.392938+0000","last_unstale":"2024-05-10T10:35:52.394631+0000","last_undegraded":"2024-05-10T10:35:52.394631+0000","last_fullsized":"2024-05-10T10:35:52.394631+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:07:01.564919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.396332+0000","last_change":"2024-05-10T10:35:52.394140+0000","last_active":"2024-05-10T10:35:52.396332+0000","last_peered":"2024-05-10T10:35:52.396332+0000","last_clean":"2024-05-10T10:35:52.396332+0000","last_became_active":"2024-05-10T10:35:52.393829+0000","last_became_peered":"2024-05-10T10:35:52.393829+0000","last_unstale":"2024-05-10T10:35:52.396332+0000","last_undegraded":"2024-05-10T10:35:52.396332+0000","last_fullsized":"2024-05-10T10:35:52.396332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:18.972647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391168+0000","last_change":"2024-05-10T10:35:52.384727+0000","last_active":"2024-05-10T10:35:52.391168+0000","last_peered":"2024-05-10T10:35:52.391168+0000","last_clean":"2024-05-10T10:35:52.391168+0000","last_became_active":"2024-05-10T10:35:52.384431+0000","last_became_peered":"2024-05-10T10:35:52.384431+0000","last_unstale":"2024-05-10T10:35:52.391168+0000","last_undegraded":"2024-05-10T10:35:52.391168+0000","last_fullsized":"2024-05-10T10:35:52.391168+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:30:00.573188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389427+0000","last_change":"2024-05-10T10:35:52.383091+0000","last_active":"2024-05-10T10:35:52.389427+0000","last_peered":"2024-05-10T10:35:52.389427+0000","last_clean":"2024-05-10T10:35:52.389427+0000","last_became_active":"2024-05-10T10:35:52.382807+0000","last_became_peered":"2024-05-10T10:35:52.382807+0000","last_unstale":"2024-05-10T10:35:52.389427+0000","last_undegraded":"2024-05-10T10:35:52.389427+0000","last_fullsized":"2024-05-10T10:35:52.389427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:36:49.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387627+0000","last_change":"2024-05-10T10:35:52.382139+0000","last_active":"2024-05-10T10:35:52.387627+0000","last_peered":"2024-05-10T10:35:52.387627+0000","last_clean":"2024-05-10T10:35:52.387627+0000","last_became_active":"2024-05-10T10:35:52.381390+0000","last_became_peered":"2024-05-10T10:35:52.381390+0000","last_unstale":"2024-05-10T10:35:52.387627+0000","last_undegraded":"2024-05-10T10:35:52.387627+0000","last_fullsized":"2024-05-10T10:35:52.387627+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:12:03.431670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395499+0000","last_change":"2024-05-10T10:35:52.393311+0000","last_active":"2024-05-10T10:35:52.395499+0000","last_peered":"2024-05-10T10:35:52.395499+0000","last_clean":"2024-05-10T10:35:52.395499+0000","last_became_active":"2024-05-10T10:35:52.393041+0000","last_became_peered":"2024-05-10T10:35:52.393041+0000","last_unstale":"2024-05-10T10:35:52.395499+0000","last_undegraded":"2024-05-10T10:35:52.395499+0000","last_fullsized":"2024-05-10T10:35:52.395499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:43:41.527270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404242+0000","last_change":"2024-05-10T10:35:51.377776+0000","last_active":"2024-05-10T10:35:52.404242+0000","last_peered":"2024-05-10T10:35:52.404242+0000","last_clean":"2024-05-10T10:35:52.404242+0000","last_became_active":"2024-05-10T10:35:51.377199+0000","last_became_peered":"2024-05-10T10:35:51.377199+0000","last_unstale":"2024-05-10T10:35:52.404242+0000","last_undegraded":"2024-05-10T10:35:52.404242+0000","last_fullsized":"2024-05-10T10:35:52.404242+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:14.693162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395405+0000","last_change":"2024-05-10T10:35:52.392825+0000","last_active":"2024-05-10T10:35:52.395405+0000","last_peered":"2024-05-10T10:35:52.395405+0000","last_clean":"2024-05-10T10:35:52.395405+0000","last_became_active":"2024-05-10T10:35:52.392503+0000","last_became_peered":"2024-05-10T10:35:52.392503+0000","last_unstale":"2024-05-10T10:35:52.395405+0000","last_undegraded":"2024-05-10T10:35:52.395405+0000","last_fullsized":"2024-05-10T10:35:52.395405+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:09.843423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.402496+0000","last_change":"2024-05-10T10:35:51.375812+0000","last_active":"2024-05-10T10:35:52.402496+0000","last_peered":"2024-05-10T10:35:52.402496+0000","last_clean":"2024-05-10T10:35:52.402496+0000","last_became_active":"2024-05-10T10:35:51.375516+0000","last_became_peered":"2024-05-10T10:35:51.375516+0000","last_unstale":"2024-05-10T10:35:52.402496+0000","last_undegraded":"2024-05-10T10:35:52.402496+0000","last_fullsized":"2024-05-10T10:35:52.402496+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:29:59.568532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389607+0000","last_change":"2024-05-10T10:35:52.385630+0000","last_active":"2024-05-10T10:35:52.389607+0000","last_peered":"2024-05-10T10:35:52.389607+0000","last_clean":"2024-05-10T10:35:52.389607+0000","last_became_active":"2024-05-10T10:35:52.385363+0000","last_became_peered":"2024-05-10T10:35:52.385363+0000","last_unstale":"2024-05-10T10:35:52.389607+0000","last_undegraded":"2024-05-10T10:35:52.389607+0000","last_fullsized":"2024-05-10T10:35:52.389607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:39:26.071931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.400912+0000","last_change":"2024-05-10T10:35:51.372181+0000","last_active":"2024-05-10T10:35:52.400912+0000","last_peered":"2024-05-10T10:35:52.400912+0000","last_clean":"2024-05-10T10:35:52.400912+0000","last_became_active":"2024-05-10T10:35:51.371918+0000","last_became_peered":"2024-05-10T10:35:51.371918+0000","last_unstale":"2024-05-10T10:35:52.400912+0000","last_undegraded":"2024-05-10T10:35:52.400912+0000","last_fullsized":"2024-05-10T10:35:52.400912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:58.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391952+0000","last_change":"2024-05-10T10:35:52.390462+0000","last_active":"2024-05-10T10:35:52.391952+0000","last_peered":"2024-05-10T10:35:52.391952+0000","last_clean":"2024-05-10T10:35:52.391952+0000","last_became_active":"2024-05-10T10:35:52.390206+0000","last_became_peered":"2024-05-10T10:35:52.390206+0000","last_unstale":"2024-05-10T10:35:52.391952+0000","last_undegraded":"2024-05-10T10:35:52.391952+0000","last_fullsized":"2024-05-10T10:35:52.391952+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:12:03.431670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.403389+0000","last_change":"2024-05-10T10:35:51.371881+0000","last_active":"2024-05-10T10:35:52.403389+0000","last_peered":"2024-05-10T10:35:52.403389+0000","last_clean":"2024-05-10T10:35:52.403389+0000","last_became_active":"2024-05-10T10:35:51.371594+0000","last_became_peered":"2024-05-10T10:35:51.371594+0000","last_unstale":"2024-05-10T10:35:52.403389+0000","last_undegraded":"2024-05-10T10:35:52.403389+0000","last_fullsized":"2024-05-10T10:35:52.403389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:17:04.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389992+0000","last_change":"2024-05-10T10:35:52.387883+0000","last_active":"2024-05-10T10:35:52.389992+0000","last_peered":"2024-05-10T10:35:52.389992+0000","last_clean":"2024-05-10T10:35:52.389992+0000","last_became_active":"2024-05-10T10:35:52.387602+0000","last_became_peered":"2024-05-10T10:35:52.387602+0000","last_unstale":"2024-05-10T10:35:52.389992+0000","last_undegraded":"2024-05-10T10:35:52.389992+0000","last_fullsized":"2024-05-10T10:35:52.389992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:49:59.536959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.402345+0000","last_change":"2024-05-10T10:35:51.366216+0000","last_active":"2024-05-10T10:35:52.402345+0000","last_peered":"2024-05-10T10:35:52.402345+0000","last_clean":"2024-05-10T10:35:52.402345+0000","last_became_active":"2024-05-10T10:35:51.365925+0000","last_became_peered":"2024-05-10T10:35:51.365925+0000","last_unstale":"2024-05-10T10:35:52.402345+0000","last_undegraded":"2024-05-10T10:35:52.402345+0000","last_fullsized":"2024-05-10T10:35:52.402345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:22:58.453878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394182+0000","last_change":"2024-05-10T10:35:52.391112+0000","last_active":"2024-05-10T10:35:52.394182+0000","last_peered":"2024-05-10T10:35:52.394182+0000","last_clean":"2024-05-10T10:35:52.394182+0000","last_became_active":"2024-05-10T10:35:52.390825+0000","last_became_peered":"2024-05-10T10:35:52.390825+0000","last_unstale":"2024-05-10T10:35:52.394182+0000","last_undegraded":"2024-05-10T10:35:52.394182+0000","last_fullsized":"2024-05-10T10:35:52.394182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:16.718296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.402430+0000","last_change":"2024-05-10T10:35:51.369615+0000","last_active":"2024-05-10T10:35:52.402430+0000","last_peered":"2024-05-10T10:35:52.402430+0000","last_clean":"2024-05-10T10:35:52.402430+0000","last_became_active":"2024-05-10T10:35:51.369313+0000","last_became_peered":"2024-05-10T10:35:51.369313+0000","last_unstale":"2024-05-10T10:35:52.402430+0000","last_undegraded":"2024-05-10T10:35:52.402430+0000","last_fullsized":"2024-05-10T10:35:52.402430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:06.094811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399276+0000","last_change":"2024-05-10T10:35:52.398093+0000","last_active":"2024-05-10T10:35:52.399276+0000","last_peered":"2024-05-10T10:35:52.399276+0000","last_clean":"2024-05-10T10:35:52.399276+0000","last_became_active":"2024-05-10T10:35:52.397712+0000","last_became_peered":"2024-05-10T10:35:52.397712+0000","last_unstale":"2024-05-10T10:35:52.399276+0000","last_undegraded":"2024-05-10T10:35:52.399276+0000","last_fullsized":"2024-05-10T10:35:52.399276+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:33.621733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.376098+0000","last_change":"2024-05-10T10:35:51.372855+0000","last_active":"2024-05-10T10:35:52.376098+0000","last_peered":"2024-05-10T10:35:52.376098+0000","last_clean":"2024-05-10T10:35:52.376098+0000","last_became_active":"2024-05-10T10:35:51.372435+0000","last_became_peered":"2024-05-10T10:35:51.372435+0000","last_unstale":"2024-05-10T10:35:52.376098+0000","last_undegraded":"2024-05-10T10:35:52.376098+0000","last_fullsized":"2024-05-10T10:35:52.376098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:37:16.182598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387854+0000","last_change":"2024-05-10T10:35:52.386299+0000","last_active":"2024-05-10T10:35:52.387854+0000","last_peered":"2024-05-10T10:35:52.387854+0000","last_clean":"2024-05-10T10:35:52.387854+0000","last_became_active":"2024-05-10T10:35:52.385979+0000","last_became_peered":"2024-05-10T10:35:52.385979+0000","last_unstale":"2024-05-10T10:35:52.387854+0000","last_undegraded":"2024-05-10T10:35:52.387854+0000","last_fullsized":"2024-05-10T10:35:52.387854+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:31.285698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.384892+0000","last_change":"2024-05-10T10:35:51.369856+0000","last_active":"2024-05-10T10:35:52.384892+0000","last_peered":"2024-05-10T10:35:52.384892+0000","last_clean":"2024-05-10T10:35:52.384892+0000","last_became_active":"2024-05-10T10:35:51.369567+0000","last_became_peered":"2024-05-10T10:35:51.369567+0000","last_unstale":"2024-05-10T10:35:52.384892+0000","last_undegraded":"2024-05-10T10:35:52.384892+0000","last_fullsized":"2024-05-10T10:35:52.384892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:50.764438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392087+0000","last_change":"2024-05-10T10:35:52.386058+0000","last_active":"2024-05-10T10:35:52.392087+0000","last_peered":"2024-05-10T10:35:52.392087+0000","last_clean":"2024-05-10T10:35:52.392087+0000","last_became_active":"2024-05-10T10:35:52.385784+0000","last_became_peered":"2024-05-10T10:35:52.385784+0000","last_unstale":"2024-05-10T10:35:52.392087+0000","last_undegraded":"2024-05-10T10:35:52.392087+0000","last_fullsized":"2024-05-10T10:35:52.392087+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:01:54.716038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.381212+0000","last_change":"2024-05-10T10:35:51.378284+0000","last_active":"2024-05-10T10:35:52.381212+0000","last_peered":"2024-05-10T10:35:52.381212+0000","last_clean":"2024-05-10T10:35:52.381212+0000","last_became_active":"2024-05-10T10:35:51.377947+0000","last_became_peered":"2024-05-10T10:35:51.377947+0000","last_unstale":"2024-05-10T10:35:52.381212+0000","last_undegraded":"2024-05-10T10:35:52.381212+0000","last_fullsized":"2024-05-10T10:35:52.381212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:33.677845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387352+0000","last_change":"2024-05-10T10:35:52.385204+0000","last_active":"2024-05-10T10:35:52.387352+0000","last_peered":"2024-05-10T10:35:52.387352+0000","last_clean":"2024-05-10T10:35:52.387352+0000","last_became_active":"2024-05-10T10:35:52.384909+0000","last_became_peered":"2024-05-10T10:35:52.384909+0000","last_unstale":"2024-05-10T10:35:52.387352+0000","last_undegraded":"2024-05-10T10:35:52.387352+0000","last_fullsized":"2024-05-10T10:35:52.387352+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:22.132928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.378709+0000","last_change":"2024-05-10T10:35:51.369750+0000","last_active":"2024-05-10T10:35:52.378709+0000","last_peered":"2024-05-10T10:35:52.378709+0000","last_clean":"2024-05-10T10:35:52.378709+0000","last_became_active":"2024-05-10T10:35:51.369433+0000","last_became_peered":"2024-05-10T10:35:51.369433+0000","last_unstale":"2024-05-10T10:35:52.378709+0000","last_undegraded":"2024-05-10T10:35:52.378709+0000","last_fullsized":"2024-05-10T10:35:52.378709+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:28.854767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398393+0000","last_change":"2024-05-10T10:35:52.397095+0000","last_active":"2024-05-10T10:35:52.398393+0000","last_peered":"2024-05-10T10:35:52.398393+0000","last_clean":"2024-05-10T10:35:52.398393+0000","last_became_active":"2024-05-10T10:35:52.396616+0000","last_became_peered":"2024-05-10T10:35:52.396616+0000","last_unstale":"2024-05-10T10:35:52.398393+0000","last_undegraded":"2024-05-10T10:35:52.398393+0000","last_fullsized":"2024-05-10T10:35:52.398393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:13:51.776442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404760+0000","last_change":"2024-05-10T10:35:51.378197+0000","last_active":"2024-05-10T10:35:52.404760+0000","last_peered":"2024-05-10T10:35:52.404760+0000","last_clean":"2024-05-10T10:35:52.404760+0000","last_became_active":"2024-05-10T10:35:51.377859+0000","last_became_peered":"2024-05-10T10:35:51.377859+0000","last_unstale":"2024-05-10T10:35:52.404760+0000","last_undegraded":"2024-05-10T10:35:52.404760+0000","last_fullsized":"2024-05-10T10:35:52.404760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:07:00.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.386290+0000","last_change":"2024-05-10T10:35:52.380971+0000","last_active":"2024-05-10T10:35:52.386290+0000","last_peered":"2024-05-10T10:35:52.386290+0000","last_clean":"2024-05-10T10:35:52.386290+0000","last_became_active":"2024-05-10T10:35:52.380556+0000","last_became_peered":"2024-05-10T10:35:52.380556+0000","last_unstale":"2024-05-10T10:35:52.386290+0000","last_undegraded":"2024-05-10T10:35:52.386290+0000","last_fullsized":"2024-05-10T10:35:52.386290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:51.769093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.406439+0000","last_change":"2024-05-10T10:35:51.375371+0000","last_active":"2024-05-10T10:35:52.406439+0000","last_peered":"2024-05-10T10:35:52.406439+0000","last_clean":"2024-05-10T10:35:52.406439+0000","last_became_active":"2024-05-10T10:35:51.375080+0000","last_became_peered":"2024-05-10T10:35:51.375080+0000","last_unstale":"2024-05-10T10:35:52.406439+0000","last_undegraded":"2024-05-10T10:35:52.406439+0000","last_fullsized":"2024-05-10T10:35:52.406439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:36:13.020212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389488+0000","last_change":"2024-05-10T10:35:52.385172+0000","last_active":"2024-05-10T10:35:52.389488+0000","last_peered":"2024-05-10T10:35:52.389488+0000","last_clean":"2024-05-10T10:35:52.389488+0000","last_became_active":"2024-05-10T10:35:52.384900+0000","last_became_peered":"2024-05-10T10:35:52.384900+0000","last_unstale":"2024-05-10T10:35:52.389488+0000","last_undegraded":"2024-05-10T10:35:52.389488+0000","last_fullsized":"2024-05-10T10:35:52.389488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:18:49.471589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404939+0000","last_change":"2024-05-10T10:35:51.377189+0000","last_active":"2024-05-10T10:35:52.404939+0000","last_peered":"2024-05-10T10:35:52.404939+0000","last_clean":"2024-05-10T10:35:52.404939+0000","last_became_active":"2024-05-10T10:35:51.376890+0000","last_became_peered":"2024-05-10T10:35:51.376890+0000","last_unstale":"2024-05-10T10:35:52.404939+0000","last_undegraded":"2024-05-10T10:35:52.404939+0000","last_fullsized":"2024-05-10T10:35:52.404939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:25.672830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388917+0000","last_change":"2024-05-10T10:35:52.379711+0000","last_active":"2024-05-10T10:35:52.388917+0000","last_peered":"2024-05-10T10:35:52.388917+0000","last_clean":"2024-05-10T10:35:52.388917+0000","last_became_active":"2024-05-10T10:35:52.379427+0000","last_became_peered":"2024-05-10T10:35:52.379427+0000","last_unstale":"2024-05-10T10:35:52.388917+0000","last_undegraded":"2024-05-10T10:35:52.388917+0000","last_fullsized":"2024-05-10T10:35:52.388917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:37:31.887382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.374279+0000","last_change":"2024-05-10T10:35:51.373731+0000","last_active":"2024-05-10T10:35:52.374279+0000","last_peered":"2024-05-10T10:35:52.374279+0000","last_clean":"2024-05-10T10:35:52.374279+0000","last_became_active":"2024-05-10T10:35:51.373466+0000","last_became_peered":"2024-05-10T10:35:51.373466+0000","last_unstale":"2024-05-10T10:35:52.374279+0000","last_undegraded":"2024-05-10T10:35:52.374279+0000","last_fullsized":"2024-05-10T10:35:52.374279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:15.713640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398991+0000","last_change":"2024-05-10T10:35:52.397347+0000","last_active":"2024-05-10T10:35:52.398991+0000","last_peered":"2024-05-10T10:35:52.398991+0000","last_clean":"2024-05-10T10:35:52.398991+0000","last_became_active":"2024-05-10T10:35:52.397075+0000","last_became_peered":"2024-05-10T10:35:52.397075+0000","last_unstale":"2024-05-10T10:35:52.398991+0000","last_undegraded":"2024-05-10T10:35:52.398991+0000","last_fullsized":"2024-05-10T10:35:52.398991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:39.477272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.381186+0000","last_change":"2024-05-10T10:35:51.377851+0000","last_active":"2024-05-10T10:35:52.381186+0000","last_peered":"2024-05-10T10:35:52.381186+0000","last_clean":"2024-05-10T10:35:52.381186+0000","last_became_active":"2024-05-10T10:35:51.377560+0000","last_became_peered":"2024-05-10T10:35:51.377560+0000","last_unstale":"2024-05-10T10:35:52.381186+0000","last_undegraded":"2024-05-10T10:35:52.381186+0000","last_fullsized":"2024-05-10T10:35:52.381186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:37:30.882726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.394135+0000","last_change":"2024-05-10T10:35:52.391948+0000","last_active":"2024-05-10T10:35:52.394135+0000","last_peered":"2024-05-10T10:35:52.394135+0000","last_clean":"2024-05-10T10:35:52.394135+0000","last_became_active":"2024-05-10T10:35:52.391664+0000","last_became_peered":"2024-05-10T10:35:52.391664+0000","last_unstale":"2024-05-10T10:35:52.394135+0000","last_undegraded":"2024-05-10T10:35:52.394135+0000","last_fullsized":"2024-05-10T10:35:52.394135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:21:59.559549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.405024+0000","last_change":"2024-05-10T10:35:51.370598+0000","last_active":"2024-05-10T10:35:52.405024+0000","last_peered":"2024-05-10T10:35:52.405024+0000","last_clean":"2024-05-10T10:35:52.405024+0000","last_became_active":"2024-05-10T10:35:51.370317+0000","last_became_peered":"2024-05-10T10:35:51.370317+0000","last_unstale":"2024-05-10T10:35:52.405024+0000","last_undegraded":"2024-05-10T10:35:52.405024+0000","last_fullsized":"2024-05-10T10:35:52.405024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:17:22.980706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.392049+0000","last_change":"2024-05-10T10:35:52.389879+0000","last_active":"2024-05-10T10:35:52.392049+0000","last_peered":"2024-05-10T10:35:52.392049+0000","last_clean":"2024-05-10T10:35:52.392049+0000","last_became_active":"2024-05-10T10:35:52.389104+0000","last_became_peered":"2024-05-10T10:35:52.389104+0000","last_unstale":"2024-05-10T10:35:52.392049+0000","last_undegraded":"2024-05-10T10:35:52.392049+0000","last_fullsized":"2024-05-10T10:35:52.392049+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:45:33.469087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399095+0000","last_change":"2024-05-10T10:35:51.372708+0000","last_active":"2024-05-10T10:35:52.399095+0000","last_peered":"2024-05-10T10:35:52.399095+0000","last_clean":"2024-05-10T10:35:52.399095+0000","last_became_active":"2024-05-10T10:35:51.372437+0000","last_became_peered":"2024-05-10T10:35:51.372437+0000","last_unstale":"2024-05-10T10:35:52.399095+0000","last_undegraded":"2024-05-10T10:35:52.399095+0000","last_fullsized":"2024-05-10T10:35:52.399095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:07:00.560263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398365+0000","last_change":"2024-05-10T10:35:52.396853+0000","last_active":"2024-05-10T10:35:52.398365+0000","last_peered":"2024-05-10T10:35:52.398365+0000","last_clean":"2024-05-10T10:35:52.398365+0000","last_became_active":"2024-05-10T10:35:52.396319+0000","last_became_peered":"2024-05-10T10:35:52.396319+0000","last_unstale":"2024-05-10T10:35:52.398365+0000","last_undegraded":"2024-05-10T10:35:52.398365+0000","last_fullsized":"2024-05-10T10:35:52.398365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:06.891450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404299+0000","last_change":"2024-05-10T10:35:51.375956+0000","last_active":"2024-05-10T10:35:52.404299+0000","last_peered":"2024-05-10T10:35:52.404299+0000","last_clean":"2024-05-10T10:35:52.404299+0000","last_became_active":"2024-05-10T10:35:51.375658+0000","last_became_peered":"2024-05-10T10:35:51.375658+0000","last_unstale":"2024-05-10T10:35:52.404299+0000","last_undegraded":"2024-05-10T10:35:52.404299+0000","last_fullsized":"2024-05-10T10:35:52.404299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:58.307738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.395468+0000","last_change":"2024-05-10T10:35:52.392895+0000","last_active":"2024-05-10T10:35:52.395468+0000","last_peered":"2024-05-10T10:35:52.395468+0000","last_clean":"2024-05-10T10:35:52.395468+0000","last_became_active":"2024-05-10T10:35:52.392615+0000","last_became_peered":"2024-05-10T10:35:52.392615+0000","last_unstale":"2024-05-10T10:35:52.395468+0000","last_undegraded":"2024-05-10T10:35:52.395468+0000","last_fullsized":"2024-05-10T10:35:52.395468+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:13:51.776442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.404655+0000","last_change":"2024-05-10T10:35:51.378399+0000","last_active":"2024-05-10T10:35:52.404655+0000","last_peered":"2024-05-10T10:35:52.404655+0000","last_clean":"2024-05-10T10:35:52.404655+0000","last_became_active":"2024-05-10T10:35:51.378093+0000","last_became_peered":"2024-05-10T10:35:51.378093+0000","last_unstale":"2024-05-10T10:35:52.404655+0000","last_undegraded":"2024-05-10T10:35:52.404655+0000","last_fullsized":"2024-05-10T10:35:52.404655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:09.391715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.399817+0000","last_change":"2024-05-10T10:35:52.398895+0000","last_active":"2024-05-10T10:35:52.399817+0000","last_peered":"2024-05-10T10:35:52.399817+0000","last_clean":"2024-05-10T10:35:52.399817+0000","last_became_active":"2024-05-10T10:35:52.398596+0000","last_became_peered":"2024-05-10T10:35:52.398596+0000","last_unstale":"2024-05-10T10:35:52.399817+0000","last_undegraded":"2024-05-10T10:35:52.399817+0000","last_fullsized":"2024-05-10T10:35:52.399817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:58:12.841201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.377208+0000","last_change":"2024-05-10T10:35:51.371359+0000","last_active":"2024-05-10T10:35:52.377208+0000","last_peered":"2024-05-10T10:35:52.377208+0000","last_clean":"2024-05-10T10:35:52.377208+0000","last_became_active":"2024-05-10T10:35:51.371084+0000","last_became_peered":"2024-05-10T10:35:51.371084+0000","last_unstale":"2024-05-10T10:35:52.377208+0000","last_undegraded":"2024-05-10T10:35:52.377208+0000","last_fullsized":"2024-05-10T10:35:52.377208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:19:05.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387586+0000","last_change":"2024-05-10T10:35:52.381017+0000","last_active":"2024-05-10T10:35:52.387586+0000","last_peered":"2024-05-10T10:35:52.387586+0000","last_clean":"2024-05-10T10:35:52.387586+0000","last_became_active":"2024-05-10T10:35:52.380578+0000","last_became_peered":"2024-05-10T10:35:52.380578+0000","last_unstale":"2024-05-10T10:35:52.387586+0000","last_undegraded":"2024-05-10T10:35:52.387586+0000","last_fullsized":"2024-05-10T10:35:52.387586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:45:33.469087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.403920+0000","last_change":"2024-05-10T10:35:51.370738+0000","last_active":"2024-05-10T10:35:52.403920+0000","last_peered":"2024-05-10T10:35:52.403920+0000","last_clean":"2024-05-10T10:35:52.403920+0000","last_became_active":"2024-05-10T10:35:51.370244+0000","last_became_peered":"2024-05-10T10:35:51.370244+0000","last_unstale":"2024-05-10T10:35:52.403920+0000","last_undegraded":"2024-05-10T10:35:52.403920+0000","last_fullsized":"2024-05-10T10:35:52.403920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:09.391715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388942+0000","last_change":"2024-05-10T10:35:52.381305+0000","last_active":"2024-05-10T10:35:52.388942+0000","last_peered":"2024-05-10T10:35:52.388942+0000","last_clean":"2024-05-10T10:35:52.388942+0000","last_became_active":"2024-05-10T10:35:52.381050+0000","last_became_peered":"2024-05-10T10:35:52.381050+0000","last_unstale":"2024-05-10T10:35:52.388942+0000","last_undegraded":"2024-05-10T10:35:52.388942+0000","last_fullsized":"2024-05-10T10:35:52.388942+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:34.682501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.374339+0000","last_change":"2024-05-10T10:35:51.370997+0000","last_active":"2024-05-10T10:35:52.374339+0000","last_peered":"2024-05-10T10:35:52.374339+0000","last_clean":"2024-05-10T10:35:52.374339+0000","last_became_active":"2024-05-10T10:35:51.370595+0000","last_became_peered":"2024-05-10T10:35:51.370595+0000","last_unstale":"2024-05-10T10:35:52.374339+0000","last_undegraded":"2024-05-10T10:35:52.374339+0000","last_fullsized":"2024-05-10T10:35:52.374339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:21:58.554894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.398349+0000","last_change":"2024-05-10T10:35:52.396926+0000","last_active":"2024-05-10T10:35:52.398349+0000","last_peered":"2024-05-10T10:35:52.398349+0000","last_clean":"2024-05-10T10:35:52.398349+0000","last_became_active":"2024-05-10T10:35:52.396644+0000","last_became_peered":"2024-05-10T10:35:52.396644+0000","last_unstale":"2024-05-10T10:35:52.398349+0000","last_undegraded":"2024-05-10T10:35:52.398349+0000","last_fullsized":"2024-05-10T10:35:52.398349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:00:09.843423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.372584+0000","last_change":"2024-05-10T10:35:51.375022+0000","last_active":"2024-05-10T10:35:52.372584+0000","last_peered":"2024-05-10T10:35:52.372584+0000","last_clean":"2024-05-10T10:35:52.372584+0000","last_became_active":"2024-05-10T10:35:51.374665+0000","last_became_peered":"2024-05-10T10:35:51.374665+0000","last_unstale":"2024-05-10T10:35:52.372584+0000","last_undegraded":"2024-05-10T10:35:52.372584+0000","last_fullsized":"2024-05-10T10:35:52.372584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:49:33.677845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.387974+0000","last_change":"2024-05-10T10:35:52.383509+0000","last_active":"2024-05-10T10:35:52.387974+0000","last_peered":"2024-05-10T10:35:52.387974+0000","last_clean":"2024-05-10T10:35:52.387974+0000","last_became_active":"2024-05-10T10:35:52.383189+0000","last_became_peered":"2024-05-10T10:35:52.383189+0000","last_unstale":"2024-05-10T10:35:52.387974+0000","last_undegraded":"2024-05-10T10:35:52.387974+0000","last_fullsized":"2024-05-10T10:35:52.387974+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:12:58.694304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.400838+0000","last_change":"2024-05-10T10:35:51.375850+0000","last_active":"2024-05-10T10:35:52.400838+0000","last_peered":"2024-05-10T10:35:52.400838+0000","last_clean":"2024-05-10T10:35:52.400838+0000","last_became_active":"2024-05-10T10:35:51.375566+0000","last_became_peered":"2024-05-10T10:35:51.375566+0000","last_unstale":"2024-05-10T10:35:52.400838+0000","last_undegraded":"2024-05-10T10:35:52.400838+0000","last_fullsized":"2024-05-10T10:35:52.400838+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:02.487122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391339+0000","last_change":"2024-05-10T10:35:52.389183+0000","last_active":"2024-05-10T10:35:52.391339+0000","last_peered":"2024-05-10T10:35:52.391339+0000","last_clean":"2024-05-10T10:35:52.391339+0000","last_became_active":"2024-05-10T10:35:52.388753+0000","last_became_peered":"2024-05-10T10:35:52.388753+0000","last_unstale":"2024-05-10T10:35:52.391339+0000","last_undegraded":"2024-05-10T10:35:52.391339+0000","last_fullsized":"2024-05-10T10:35:52.391339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:32.463915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.374995+0000","last_change":"2024-05-10T10:35:51.374976+0000","last_active":"2024-05-10T10:35:52.374995+0000","last_peered":"2024-05-10T10:35:52.374995+0000","last_clean":"2024-05-10T10:35:52.374995+0000","last_became_active":"2024-05-10T10:35:51.374647+0000","last_became_peered":"2024-05-10T10:35:51.374647+0000","last_unstale":"2024-05-10T10:35:52.374995+0000","last_undegraded":"2024-05-10T10:35:52.374995+0000","last_fullsized":"2024-05-10T10:35:52.374995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:14.372647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.388835+0000","last_change":"2024-05-10T10:35:52.379131+0000","last_active":"2024-05-10T10:35:52.388835+0000","last_peered":"2024-05-10T10:35:52.388835+0000","last_clean":"2024-05-10T10:35:52.388835+0000","last_became_active":"2024-05-10T10:35:52.378848+0000","last_became_peered":"2024-05-10T10:35:52.378848+0000","last_unstale":"2024-05-10T10:35:52.388835+0000","last_undegraded":"2024-05-10T10:35:52.388835+0000","last_fullsized":"2024-05-10T10:35:52.388835+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:26.677485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.372782+0000","last_change":"2024-05-10T10:35:51.371489+0000","last_active":"2024-05-10T10:35:52.372782+0000","last_peered":"2024-05-10T10:35:52.372782+0000","last_clean":"2024-05-10T10:35:52.372782+0000","last_became_active":"2024-05-10T10:35:51.371217+0000","last_became_peered":"2024-05-10T10:35:51.371217+0000","last_unstale":"2024-05-10T10:35:52.372782+0000","last_undegraded":"2024-05-10T10:35:52.372782+0000","last_fullsized":"2024-05-10T10:35:52.372782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:44:34.096486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.391549+0000","last_change":"2024-05-10T10:35:52.390182+0000","last_active":"2024-05-10T10:35:52.391549+0000","last_peered":"2024-05-10T10:35:52.391549+0000","last_clean":"2024-05-10T10:35:52.391549+0000","last_became_active":"2024-05-10T10:35:52.389901+0000","last_became_peered":"2024-05-10T10:35:52.389901+0000","last_unstale":"2024-05-10T10:35:52.391549+0000","last_undegraded":"2024-05-10T10:35:52.391549+0000","last_fullsized":"2024-05-10T10:35:52.391549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:23:57.321844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.377634+0000","last_change":"2024-05-10T10:35:51.374675+0000","last_active":"2024-05-10T10:35:52.377634+0000","last_peered":"2024-05-10T10:35:52.377634+0000","last_clean":"2024-05-10T10:35:52.377634+0000","last_became_active":"2024-05-10T10:35:51.374386+0000","last_became_peered":"2024-05-10T10:35:51.374386+0000","last_unstale":"2024-05-10T10:35:52.377634+0000","last_undegraded":"2024-05-10T10:35:52.377634+0000","last_fullsized":"2024-05-10T10:35:52.377634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:28.094091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.396189+0000","last_change":"2024-05-10T10:35:52.393422+0000","last_active":"2024-05-10T10:35:52.396189+0000","last_peered":"2024-05-10T10:35:52.396189+0000","last_clean":"2024-05-10T10:35:52.396189+0000","last_became_active":"2024-05-10T10:35:52.393034+0000","last_became_peered":"2024-05-10T10:35:52.393034+0000","last_unstale":"2024-05-10T10:35:52.396189+0000","last_undegraded":"2024-05-10T10:35:52.396189+0000","last_fullsized":"2024-05-10T10:35:52.396189+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:36:49.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.380655+0000","last_change":"2024-05-10T10:35:51.376564+0000","last_active":"2024-05-10T10:35:52.380655+0000","last_peered":"2024-05-10T10:35:52.380655+0000","last_clean":"2024-05-10T10:35:52.380655+0000","last_became_active":"2024-05-10T10:35:51.376237+0000","last_became_peered":"2024-05-10T10:35:51.376237+0000","last_unstale":"2024-05-10T10:35:52.380655+0000","last_undegraded":"2024-05-10T10:35:52.380655+0000","last_fullsized":"2024-05-10T10:35:52.380655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:14.372647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.389450+0000","last_change":"2024-05-10T10:35:52.385985+0000","last_active":"2024-05-10T10:35:52.389450+0000","last_peered":"2024-05-10T10:35:52.389450+0000","last_clean":"2024-05-10T10:35:52.389450+0000","last_became_active":"2024-05-10T10:35:52.385712+0000","last_became_peered":"2024-05-10T10:35:52.385712+0000","last_unstale":"2024-05-10T10:35:52.389450+0000","last_undegraded":"2024-05-10T10:35:52.389450+0000","last_fullsized":"2024-05-10T10:35:52.389450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:43.953297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.376743+0000","last_change":"2024-05-10T10:35:51.368388+0000","last_active":"2024-05-10T10:35:52.376743+0000","last_peered":"2024-05-10T10:35:52.376743+0000","last_clean":"2024-05-10T10:35:52.376743+0000","last_became_active":"2024-05-10T10:35:51.368118+0000","last_became_peered":"2024-05-10T10:35:51.368118+0000","last_unstale":"2024-05-10T10:35:52.376743+0000","last_undegraded":"2024-05-10T10:35:52.376743+0000","last_fullsized":"2024-05-10T10:35:52.376743+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:39:46.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393455+0000","last_change":"2024-05-10T10:35:52.390378+0000","last_active":"2024-05-10T10:35:52.393455+0000","last_peered":"2024-05-10T10:35:52.393455+0000","last_clean":"2024-05-10T10:35:52.393455+0000","last_became_active":"2024-05-10T10:35:52.390085+0000","last_became_peered":"2024-05-10T10:35:52.390085+0000","last_unstale":"2024-05-10T10:35:52.393455+0000","last_undegraded":"2024-05-10T10:35:52.393455+0000","last_fullsized":"2024-05-10T10:35:52.393455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:05:17.155890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.376270+0000","last_change":"2024-05-10T10:35:51.373906+0000","last_active":"2024-05-10T10:35:52.376270+0000","last_peered":"2024-05-10T10:35:52.376270+0000","last_clean":"2024-05-10T10:35:52.376270+0000","last_became_active":"2024-05-10T10:35:51.373166+0000","last_became_peered":"2024-05-10T10:35:51.373166+0000","last_unstale":"2024-05-10T10:35:52.376270+0000","last_undegraded":"2024-05-10T10:35:52.376270+0000","last_fullsized":"2024-05-10T10:35:52.376270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:17:25.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.393460+0000","last_change":"2024-05-10T10:35:52.391115+0000","last_active":"2024-05-10T10:35:52.393460+0000","last_peered":"2024-05-10T10:35:52.393460+0000","last_clean":"2024-05-10T10:35:52.393460+0000","last_became_active":"2024-05-10T10:35:52.390395+0000","last_became_peered":"2024-05-10T10:35:52.390395+0000","last_unstale":"2024-05-10T10:35:52.393460+0000","last_undegraded":"2024-05-10T10:35:52.393460+0000","last_fullsized":"2024-05-10T10:35:52.393460+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:42:24.686955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.375963+0000","last_change":"2024-05-10T10:35:51.372214+0000","last_active":"2024-05-10T10:35:52.375963+0000","last_peered":"2024-05-10T10:35:52.375963+0000","last_clean":"2024-05-10T10:35:52.375963+0000","last_became_active":"2024-05-10T10:35:51.371912+0000","last_became_peered":"2024-05-10T10:35:51.371912+0000","last_unstale":"2024-05-10T10:35:52.375963+0000","last_undegraded":"2024-05-10T10:35:52.375963+0000","last_fullsized":"2024-05-10T10:35:52.375963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:50.184685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.397680+0000","last_change":"2024-05-10T10:35:52.396014+0000","last_active":"2024-05-10T10:35:52.397680+0000","last_peered":"2024-05-10T10:35:52.397680+0000","last_clean":"2024-05-10T10:35:52.397680+0000","last_became_active":"2024-05-10T10:35:52.395752+0000","last_became_peered":"2024-05-10T10:35:52.395752+0000","last_unstale":"2024-05-10T10:35:52.397680+0000","last_undegraded":"2024-05-10T10:35:52.397680+0000","last_fullsized":"2024-05-10T10:35:52.397680+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:51.350453+0000","last_clean_scrub_stamp":"2024-05-10T10:35:51.350453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:43.953297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.375606+0000","last_change":"2024-05-10T10:35:51.365519+0000","last_active":"2024-05-10T10:35:52.375606+0000","last_peered":"2024-05-10T10:35:52.375606+0000","last_clean":"2024-05-10T10:35:52.375606+0000","last_became_active":"2024-05-10T10:35:51.365224+0000","last_became_peered":"2024-05-10T10:35:51.365224+0000","last_unstale":"2024-05-10T10:35:52.375606+0000","last_undegraded":"2024-05-10T10:35:52.375606+0000","last_fullsized":"2024-05-10T10:35:52.375606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:35:50.345797+0000","last_clean_scrub_stamp":"2024-05-10T10:35:50.345797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:22:17.967992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":191,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-10T10:35:52.379799+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:35:52.379799+0000","last_peered":"2024-05-10T10:35:52.379799+0000","last_clean":"2024-05-10T10:35:52.379799+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:35:52.379799+0000","last_undegraded":"2024-05-10T10:35:52.379799+0000","last_fullsized":"2024-05-10T10:35:52.379799+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1306624,"data_stored":905485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":322318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":322306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":315117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":724992,"data_stored":322284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":724992,"data_stored":322284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":753664,"data_stored":324660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1318912,"data_stored":912652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:35:56.799 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:35:56.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T10:35:56.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.945+0000 7faab8cc4640 1 Processor -- start 2024-05-10T10:35:56.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.945+0000 7faab8cc4640 1 -- start start 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.945+0000 7faab8cc4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faab4160530 0x7faab415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.945+0000 7faab8cc4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faab415c9e0 0x7faab40cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.945+0000 7faab8cc4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab415cfb0 con 0x7faab406f030 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.945+0000 7faab8cc4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7faab40ccb60 con 0x7faab4160530 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.945+0000 7faab8cc4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7faab4160990 con 0x7faab415c9e0 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faab4160530 0x7faab415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faaaa59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faab406f030 0x7faab406f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48952/0 (socket says 172.21.15.53:48952) 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faab4160530 0x7faab415c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56588/0 (socket says 172.21.15.53:56588) 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faaaa59c640 1 -- 172.21.15.53:0/3444950673 learned_addr learned my addr 172.21.15.53:0/3444950673 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab2d9d640 1 --2- 172.21.15.53:0/3444950673 >> v2:172.21.15.135:3300/0 conn(0x7faab415c9e0 0x7faab40cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab259c640 1 -- 172.21.15.53:0/3444950673 >> v2:172.21.15.135:3300/0 conn(0x7faab415c9e0 msgr2=0x7faab40cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab259c640 1 --2- 172.21.15.53:0/3444950673 >> v2:172.21.15.135:3300/0 conn(0x7faab415c9e0 0x7faab40cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab259c640 1 -- 172.21.15.53:0/3444950673 >> v1:172.21.15.53:6789/0 conn(0x7faab406f030 legacy=0x7faab406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab259c640 1 -- 172.21.15.53:0/3444950673 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab415d290 con 0x7faab4160530 2024-05-10T10:35:56.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.946+0000 7faab1d9b640 1 -- 172.21.15.53:0/3444950673 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546371708 0 0) 0x7faab415cfb0 con 0x7faab406f030 2024-05-10T10:35:56.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab259c640 1 --2- 172.21.15.53:0/3444950673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab415c4a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faab40d07e0 tx=0x7faa9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa82555d9b5f9a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab1d9b640 1 -- 172.21.15.53:0/3444950673 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c037070 con 0x7faab4160530 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab1d9b640 1 -- 172.21.15.53:0/3444950673 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa9c032440 con 0x7faab4160530 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab1d9b640 1 -- 172.21.15.53:0/3444950673 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c032810 con 0x7faab4160530 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 -- 172.21.15.53:0/3444950673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 msgr2=0x7faab415c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 --2- 172.21.15.53:0/3444950673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab415c4a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faab40d07e0 tx=0x7faa9c02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 -- 172.21.15.53:0/3444950673 shutdown_connections 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 --2- 172.21.15.53:0/3444950673 >> v2:172.21.15.135:3300/0 conn(0x7faab415c9e0 0x7faab40cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 --2- 172.21.15.53:0/3444950673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab415c4a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 -- 172.21.15.53:0/3444950673 >> 172.21.15.53:0/3444950673 conn(0x7faab40d0c90 msgr2=0x7faab40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 -- 172.21.15.53:0/3444950673 shutdown_connections 2024-05-10T10:35:56.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.947+0000 7faab8cc4640 1 -- 172.21.15.53:0/3444950673 wait complete. 2024-05-10T10:35:56.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.949+0000 7faab8cc4640 1 Processor -- start 2024-05-10T10:35:56.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.949+0000 7faab8cc4640 1 -- start start 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faab406f030 0x7faab40c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faab415c9e0 0x7faab414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faab415fa40 con 0x7faab4160530 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faab4160010 con 0x7faab415c9e0 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7faab40cb910 con 0x7faab406f030 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab414b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56600/0 (socket says 172.21.15.53:56600) 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 -- 172.21.15.53:0/2890582727 learned_addr learned my addr 172.21.15.53:0/2890582727 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab259c640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faab415c9e0 0x7faab414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faaaa59c640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faab406f030 0x7faab40c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 -- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faab406f030 msgr2=0x7faab40c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faab406f030 0x7faab40c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 -- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faab415c9e0 msgr2=0x7faab414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faab415c9e0 0x7faab414e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 -- 172.21.15.53:0/2890582727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab414ee40 con 0x7faab4160530 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab259c640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faab415c9e0 0x7faab414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faaaa59c640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faab406f030 0x7faab40c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab2d9d640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab414b5b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faaac00efc0 tx=0x7faaac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faaab7fe640 1 -- 172.21.15.53:0/2890582727 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac018020 con 0x7faab4160530 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faaab7fe640 1 -- 172.21.15.53:0/2890582727 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaac009b10 con 0x7faab4160530 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab40b6360 con 0x7faab4160530 2024-05-10T10:35:56.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faaab7fe640 1 -- 172.21.15.53:0/2890582727 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0146c0 con 0x7faab4160530 2024-05-10T10:35:56.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.950+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab40c6400 con 0x7faab4160530 2024-05-10T10:35:56.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.952+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab406a220 con 0x7faab4160530 2024-05-10T10:35:56.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.952+0000 7faaab7fe640 1 -- 172.21.15.53:0/2890582727 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faaac005d10 con 0x7faab4160530 2024-05-10T10:35:56.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.953+0000 7faaab7fe640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa940762d0 0x7faa94078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:56.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.953+0000 7faaab7fe640 1 -- 172.21.15.53:0/2890582727 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7faaac0981b0 con 0x7faab4160530 2024-05-10T10:35:56.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.953+0000 7faaaa59c640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa940762d0 0x7faa94078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:56.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.954+0000 7faaaa59c640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa940762d0 0x7faa94078790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000f60 tx=0x7faaa4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:56.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:56.957+0000 7faaab7fe640 1 -- 172.21.15.53:0/2890582727 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7faaac022070 con 0x7faab4160530 2024-05-10T10:35:57.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.220+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faab40cf020 con 0x7faab4160530 2024-05-10T10:35:57.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.222+0000 7faaab7fe640 1 -- 172.21.15.53:0/2890582727 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7faab414ee40 con 0x7faab4160530 2024-05-10T10:35:57.223 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:35:57.223 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:35:57.223 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2024-05-10T10:35:57.223 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2024-05-10T10:35:57.223 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-10T10:35:57.223 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2024-05-10T10:35:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa940762d0 msgr2=0x7faa94078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa940762d0 0x7faa94078790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000f60 tx=0x7faaa4009040 comp rx=0 tx=0).stop 2024-05-10T10:35:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 msgr2=0x7faab414b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab414b5b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faaac00efc0 tx=0x7faaac00c490 comp rx=0 tx=0).stop 2024-05-10T10:35:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 shutdown_connections 2024-05-10T10:35:57.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa940762d0 0x7faa94078790 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faab4160530 0x7faab414b5b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faab415c9e0 0x7faab414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 --2- 172.21.15.53:0/2890582727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faab406f030 0x7faab40c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 >> 172.21.15.53:0/2890582727 conn(0x7faab40d0c90 msgr2=0x7faab40aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:57.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 shutdown_connections 2024-05-10T10:35:57.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.224+0000 7faab8cc4640 1 -- 172.21.15.53:0/2890582727 wait complete. 2024-05-10T10:35:57.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:57.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.339+0000 7f3309158640 1 Processor -- start 2024-05-10T10:35:57.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.339+0000 7f3309158640 1 -- start start 2024-05-10T10:35:57.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f3309158640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f330415f3b0 0x7f33040d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f3309158640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f330415f790 0x7f33040d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f3309158640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33040d30b0 con 0x7f33040d5d30 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f3309158640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f33040d3390 con 0x7f330415f790 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f3309158640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f33040c8c80 con 0x7f330415f3b0 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f330259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f330415f790 0x7f33040d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f330359e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f33040d5d30 0x7f33040c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48962/0 (socket says 172.21.15.53:48962) 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f330359e640 1 -- 172.21.15.53:0/2584376710 learned_addr learned my addr 172.21.15.53:0/2584376710 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f3302d9d640 1 --2- 172.21.15.53:0/2584376710 >> v2:172.21.15.135:3300/0 conn(0x7f330415f3b0 0x7f33040d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f330259c640 1 -- 172.21.15.53:0/2584376710 >> v2:172.21.15.135:3300/0 conn(0x7f330415f3b0 msgr2=0x7f33040d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:57.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f330259c640 1 --2- 172.21.15.53:0/2584376710 >> v2:172.21.15.135:3300/0 conn(0x7f330415f3b0 0x7f33040d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f330259c640 1 -- 172.21.15.53:0/2584376710 >> v1:172.21.15.53:6789/0 conn(0x7f33040d5d30 legacy=0x7f33040c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f330259c640 1 -- 172.21.15.53:0/2584376710 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33040c8f60 con 0x7f330415f790 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.340+0000 7f3301d9b640 1 -- 172.21.15.53:0/2584376710 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162611903 0 0) 0x7f33040d30b0 con 0x7f33040d5d30 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.341+0000 7f330259c640 1 --2- 172.21.15.53:0/2584376710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330415f790 0x7f33040d3c30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f330408e820 tx=0x7f32f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaa330fcd544fa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.341+0000 7f3301d9b640 1 -- 172.21.15.53:0/2584376710 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8037070 con 0x7f330415f790 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.341+0000 7f3301d9b640 1 -- 172.21.15.53:0/2584376710 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f8032440 con 0x7f330415f790 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.341+0000 7f3301d9b640 1 -- 172.21.15.53:0/2584376710 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8032810 con 0x7f330415f790 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.341+0000 7f3309158640 1 -- 172.21.15.53:0/2584376710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330415f790 msgr2=0x7f33040d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.341+0000 7f3309158640 1 --2- 172.21.15.53:0/2584376710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330415f790 0x7f33040d3c30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f330408e820 tx=0x7f32f802a1a0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.342+0000 7f3309158640 1 -- 172.21.15.53:0/2584376710 shutdown_connections 2024-05-10T10:35:57.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.342+0000 7f3309158640 1 --2- 172.21.15.53:0/2584376710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330415f790 0x7f33040d3c30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.342+0000 7f3309158640 1 --2- 172.21.15.53:0/2584376710 >> v2:172.21.15.135:3300/0 conn(0x7f330415f3b0 0x7f33040d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.342+0000 7f3309158640 1 -- 172.21.15.53:0/2584376710 >> 172.21.15.53:0/2584376710 conn(0x7f33040d5130 msgr2=0x7f33040d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:57.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.342+0000 7f3309158640 1 -- 172.21.15.53:0/2584376710 shutdown_connections 2024-05-10T10:35:57.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.342+0000 7f3309158640 1 -- 172.21.15.53:0/2584376710 wait complete. 2024-05-10T10:35:57.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.343+0000 7f3309158640 1 Processor -- start 2024-05-10T10:35:57.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.343+0000 7f3309158640 1 -- start start 2024-05-10T10:35:57.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f3309158640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33040d5d30 0x7f330406e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:57.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f3309158640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f330415f3b0 0x7f33041521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:57.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f3309158640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33041f5080 0x7f33041f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:57.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f3309158640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f33040d0e60 con 0x7f33041f5080 2024-05-10T10:35:57.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f3309158640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33040d0ce0 con 0x7f330415f3b0 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f3309158640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f33040d0fe0 con 0x7f33040d5d30 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33041f5080 0x7f33041f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33041f5080 0x7f33041f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56616/0 (socket says 172.21.15.53:56616) 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 -- 172.21.15.53:0/48205390 learned_addr learned my addr 172.21.15.53:0/48205390 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f3302d9d640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33040d5d30 0x7f330406e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330259c640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f330415f3b0 0x7f33041521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 -- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33040d5d30 msgr2=0x7f330406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33040d5d30 0x7f330406e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 -- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f330415f3b0 msgr2=0x7f33041521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f330415f3b0 0x7f33041521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 -- 172.21.15.53:0/48205390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33040ccd40 con 0x7f33041f5080 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.344+0000 7f330359e640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33041f5080 0x7f33041f5530 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c7a0 tx=0x7f32fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.345+0000 7f32eb7fe640 1 -- 172.21.15.53:0/48205390 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc018070 con 0x7f33041f5080 2024-05-10T10:35:57.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.345+0000 7f32eb7fe640 1 -- 172.21.15.53:0/48205390 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc00a640 con 0x7f33041f5080 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.345+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33040cd020 con 0x7f33041f5080 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.345+0000 7f32eb7fe640 1 -- 172.21.15.53:0/48205390 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00aab0 con 0x7f33041f5080 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.345+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33040cd5e0 con 0x7f33041f5080 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.346+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330406a220 con 0x7f33041f5080 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.347+0000 7f32eb7fe640 1 -- 172.21.15.53:0/48205390 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f32fc01b070 con 0x7f33041f5080 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.347+0000 7f32eb7fe640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32f00762d0 0x7f32f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.347+0000 7f32eb7fe640 1 -- 172.21.15.53:0/48205390 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f32fc096d20 con 0x7f33041f5080 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.347+0000 7f3302d9d640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32f00762d0 0x7f32f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:57.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.348+0000 7f3302d9d640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32f00762d0 0x7f32f0078790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f32f4006fd0 tx=0x7f32f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:57.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.352+0000 7f32eb7fe640 1 -- 172.21.15.53:0/48205390 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f32fc060580 con 0x7f33041f5080 2024-05-10T10:35:57.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.583+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33040cab50 con 0x7f33041f5080 2024-05-10T10:35:57.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.583+0000 7f32eb7fe640 1 -- 172.21.15.53:0/48205390 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5188 (secure 0 0 0) 0x7f32fc063460 con 0x7f33041f5080 2024-05-10T10:35:57.584 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:57.584 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:35:54.766365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:35:57.584 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 34 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32f00762d0 msgr2=0x7f32f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32f00762d0 0x7f32f0078790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f32f4006fd0 tx=0x7f32f4008040 comp rx=0 tx=0).stop 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33041f5080 msgr2=0x7f33041f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33041f5080 0x7f33041f5530 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c7a0 tx=0x7f32fc00cc60 comp rx=0 tx=0).stop 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 shutdown_connections 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32f00762d0 0x7f32f0078790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33041f5080 0x7f33041f5530 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f330415f3b0 0x7f33041521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 --2- 172.21.15.53:0/48205390 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33040d5d30 0x7f330406e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 >> 172.21.15.53:0/48205390 conn(0x7f33040d5130 msgr2=0x7f330415e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 shutdown_connections 2024-05-10T10:35:57.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:57.585+0000 7f3309158640 1 -- 172.21.15.53:0/48205390 wait complete. 2024-05-10T10:35:58.000 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:35:51.741227+0000', 'modified': '2024-05-10T10:35:54.766365+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5622}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5622': {'gid': 5622, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/1934136965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 1934136965}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1934136965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5622, 'qdb_cluster': [5622]} 2024-05-10T10:35:58.000 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:35:58.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T10:35:58.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.111+0000 7f3f9dca0640 1 Processor -- start 2024-05-10T10:35:58.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.111+0000 7f3f9dca0640 1 -- start start 2024-05-10T10:35:58.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f9dca0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3f98162750 0x7f3f980d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f9dca0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f980d5cb0 0x7f3f980d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f9dca0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f980b45f0 con 0x7f3f98162b30 2024-05-10T10:35:58.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f9dca0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3f980b4f20 con 0x7f3f98162750 2024-05-10T10:35:58.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f9dca0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3f980d0fe0 con 0x7f3f980d5cb0 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f97fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3f98162b30 0x7f3f980d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48972/0 (socket says 172.21.15.53:48972) 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f97fff640 1 -- 172.21.15.53:0/3529008603 learned_addr learned my addr 172.21.15.53:0/3529008603 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f977fe640 1 --2- 172.21.15.53:0/3529008603 >> v2:172.21.15.135:3300/0 conn(0x7f3f98162750 0x7f3f980d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f980d5cb0 0x7f3f980d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f967fc640 1 -- 172.21.15.53:0/3529008603 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579130507 0 0) 0x7f3f980b45f0 con 0x7f3f98162b30 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.112+0000 7f3f967fc640 1 -- 172.21.15.53:0/3529008603 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f80003520 con 0x7f3f98162b30 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.113+0000 7f3f977fe640 1 -- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f980d5cb0 msgr2=0x7f3f980d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.113+0000 7f3f977fe640 1 --2- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f980d5cb0 0x7f3f980d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.113+0000 7f3f977fe640 1 -- 172.21.15.53:0/3529008603 >> v1:172.21.15.53:6789/0 conn(0x7f3f98162b30 legacy=0x7f3f980d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.113+0000 7f3f977fe640 1 -- 172.21.15.53:0/3529008603 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f980b48d0 con 0x7f3f98162750 2024-05-10T10:35:58.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.113+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f980d5cb0 0x7f3f980d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:35:58.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.113+0000 7f3f977fe640 1 --2- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 0x7f3f980d0670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f880097a0 tx=0x7f3f8802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3526cb2bb72ef88a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f967fc640 1 -- 172.21.15.53:0/3529008603 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88037070 con 0x7f3f98162750 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f967fc640 1 -- 172.21.15.53:0/3529008603 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8802ae50 con 0x7f3f98162750 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f967fc640 1 -- 172.21.15.53:0/3529008603 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88032660 con 0x7f3f98162750 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 -- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 msgr2=0x7f3f980d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 0x7f3f980d0670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f880097a0 tx=0x7f3f8802a100 comp rx=0 tx=0).stop 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 -- 172.21.15.53:0/3529008603 shutdown_connections 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f980d5cb0 0x7f3f980d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/3529008603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 0x7f3f980d0670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 -- 172.21.15.53:0/3529008603 >> 172.21.15.53:0/3529008603 conn(0x7f3f9809e3b0 msgr2=0x7f3f980d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 -- 172.21.15.53:0/3529008603 shutdown_connections 2024-05-10T10:35:58.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.114+0000 7f3f9dca0640 1 -- 172.21.15.53:0/3529008603 wait complete. 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.116+0000 7f3f9dca0640 1 Processor -- start 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.116+0000 7f3f9dca0640 1 -- start start 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.116+0000 7f3f9dca0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 0x7f3f98159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.116+0000 7f3f9dca0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f98162b30 0x7f3f980cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.116+0000 7f3f9dca0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f980ba830 0x7f3f98157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f9dca0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980d0d30 con 0x7f3f980ba830 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f9dca0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980d0bb0 con 0x7f3f98162750 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f9dca0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3f980b63c0 con 0x7f3f98162b30 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f980ba830 0x7f3f98157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f980ba830 0x7f3f98157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56618/0 (socket says 172.21.15.53:56618) 2024-05-10T10:35:58.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 -- 172.21.15.53:0/402010140 learned_addr learned my addr 172.21.15.53:0/402010140 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f977fe640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 0x7f3f98159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f98162b30 0x7f3f980cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 -- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f98162b30 msgr2=0x7f3f980cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f98162b30 0x7f3f980cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 -- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 msgr2=0x7f3f98159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 0x7f3f98159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 -- 172.21.15.53:0/402010140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9815a3a0 con 0x7f3f980ba830 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f97fff640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f980ba830 0x7f3f98157580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000cad0 tx=0x7f3f9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f7ffff640 1 -- 172.21.15.53:0/402010140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90017070 con 0x7f3f980ba830 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f7ffff640 1 -- 172.21.15.53:0/402010140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9000ac30 con 0x7f3f980ba830 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f7ffff640 1 -- 172.21.15.53:0/402010140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90012400 con 0x7f3f980ba830 2024-05-10T10:35:58.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9815a640 con 0x7f3f980ba830 2024-05-10T10:35:58.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.117+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f980b7f20 con 0x7f3f980ba830 2024-05-10T10:35:58.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.119+0000 7f3f7ffff640 1 -- 172.21.15.53:0/402010140 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f9001a070 con 0x7f3f980ba830 2024-05-10T10:35:58.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.120+0000 7f3f7ffff640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f84076300 0x7f3f840787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.120+0000 7f3f7ffff640 1 -- 172.21.15.53:0/402010140 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f3f900975b0 con 0x7f3f980ba830 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.120+0000 7f3f977fe640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f84076300 0x7f3f840787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.120+0000 7f3fa09c3740 1 Processor -- start 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.120+0000 7f3fa09c3740 1 -- start start 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.120+0000 7f3f977fe640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f84076300 0x7f3f840787c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f88034000 tx=0x7f3f8802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d82028b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82028bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82025bbb0 0x55d82025f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d8201c58d0 con 0x55d820289330 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d82017ec10 con 0x55d820288f50 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d8201c1890 con 0x55d82025bbb0 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3f96ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82028bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3f96ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82028bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56626/0 (socket says 172.21.15.53:56626) 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3f96ffd640 1 -- 172.21.15.53:0/1568394930 learned_addr learned my addr 172.21.15.53:0/1568394930 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3f977fe640 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d82028b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.121+0000 7f3f97fff640 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82025bbb0 0x55d82025f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f96ffd640 1 -- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82025bbb0 msgr2=0x55d82025f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82025bbb0 0x55d82025f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f96ffd640 1 -- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 msgr2=0x55d82028b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d82028b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f96ffd640 1 -- 172.21.15.53:0/1568394930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8201865d0 con 0x55d820289330 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82028bca0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009b20 tx=0x7f3f8c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a56e53e9473823 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f7d7fa640 1 -- 172.21.15.53:0/1568394930 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c019070 con 0x55d820289330 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f7d7fa640 1 -- 172.21.15.53:0/1568394930 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c004220 con 0x55d820289330 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.122+0000 7f3f7d7fa640 1 -- 172.21.15.53:0/1568394930 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0119c0 con 0x55d820289330 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1568394930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 msgr2=0x55d82028bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82028bca0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009b20 tx=0x7f3f8c009190 comp rx=0 tx=0).stop 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1568394930 shutdown_connections 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82025bbb0 0x55d82025f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82028bca0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1568394930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d82028b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1568394930 >> 172.21.15.53:0/1568394930 conn(0x55d82025de20 msgr2=0x55d8202a6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1568394930 shutdown_connections 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.123+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1568394930 wait complete. 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 -- start start 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 Processor -- start 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d820288f50 0x55d8202a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820289330 0x55d82027f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d82027fcd0 0x55d82027f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d820260fd0 con 0x55d82027fcd0 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d820182760 con 0x55d820289330 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.124+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d8202612b0 con 0x55d820288f50 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3fa09c3740 10 client.0 _ll_register_callbacks cb 0x55d8201bbcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d82027fcd0 0x55d82027f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d82027fcd0 0x55d82027f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56638/0 (socket says 172.21.15.53:56638) 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 -- 172.21.15.53:0/1885650965 learned_addr learned my addr 172.21.15.53:0/1885650965 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820289330 0x55d82027f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f977fe640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d820288f50 0x55d8202a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 -- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d820288f50 msgr2=0x55d8202a9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d820288f50 0x55d8202a9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 -- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820289330 msgr2=0x55d82027f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820289330 0x55d82027f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 -- 172.21.15.53:0/1885650965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81fed72d0 con 0x55d82027fcd0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f977fe640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d820288f50 0x55d8202a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f97fff640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d82027fcd0 0x55d82027f5c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000caa0 tx=0x7f3f90060c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e214d0cc016407 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f767fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f767fc640 1 -- 172.21.15.53:0/1885650965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90064990 con 0x55d82027fcd0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f767fc640 1 -- 172.21.15.53:0/1885650965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f3f9005f840 con 0x55d82027fcd0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f8c004220 con 0x55d82027fcd0 2024-05-10T10:35:58.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3f767fc640 1 -- 172.21.15.53:0/1885650965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f90095050 con 0x55d82027fcd0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3fa09c3740 10 client.5760 fetch_fsmap learned FSMap version 34 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.125+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55d81ffa16b0 con 0x55d82027fcd0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3f767fc640 1 -- 172.21.15.53:0/1885650965 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c004220 con 0x55d82027fcd0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3f767fc640 1 -- 172.21.15.53:0/1885650965 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f3f90065b20 con 0x55d82027fcd0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3fa09c3740 10 client.5760 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3fa09c3740 10 client.5760 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3fa09c3740 20 client.5760 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d8202a13f0 0x55d8202a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3fa09c3740 4 client.5760 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d820149760 con 0x55d8202a13f0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3f977fe640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d8202a13f0 0x55d8202a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3f767fc640 10 client.5760 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.126+0000 7f3f977fe640 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d8202a13f0 0x55d8202a17d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca04623e238d7fa1 server_cookie=ffe82f6fc967df02 in_seq=0 out_seq=0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3f767fc640 1 -- 172.21.15.53:0/1885650965 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d820149760 con 0x55d8202a13f0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3f767fc640 10 client.5760 handle_command_reply: tid=0 2024-05-10T10:35:58.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 client.5760 shutdown 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d82027fcd0 msgr2=0x55d82027f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d82027fcd0 0x55d82027f5c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000caa0 tx=0x7f3f90060c00 comp rx=0 tx=0).stop 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 shutdown_connections 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d8202a13f0 0x55d8202a17d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d82027fcd0 0x55d82027f5c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820289330 0x55d82027f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3f767fc640 0 client.5760 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/1885650965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d820288f50 0x55d8202a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 >> 172.21.15.53:0/1885650965 conn(0x55d82025bbb0 msgr2=0x55d82025c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 shutdown_connections 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 1 -- 172.21.15.53:0/1885650965 wait complete. 2024-05-10T10:35:58.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.127+0000 7f3fa09c3740 20 client.5760 trim_cache size 0 max 16384 2024-05-10T10:35:58.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.146+0000 7f3fa09c3740 1 Processor -- start 2024-05-10T10:35:58.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.146+0000 7f3fa09c3740 1 -- start start 2024-05-10T10:35:58.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d8202a3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d8202a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82029ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98200b80 con 0x55d820289330 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981ffef0 con 0x55d820288f50 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3f98200510 con 0x55d82027fcd0 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82029ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82029ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56640/0 (socket says 172.21.15.53:56640) 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 -- 172.21.15.53:0/2188218457 learned_addr learned my addr 172.21.15.53:0/2188218457 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f977fe640 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d8202a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d8202a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 -- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 msgr2=0x55d8202a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d8202a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 -- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 msgr2=0x55d8202a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d8202a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f97fff640 1 -- 172.21.15.53:0/2188218457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8202a1b40 con 0x55d820289330 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f977fe640 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d8202a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.147+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d8202a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3f97fff640 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82029ff90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f9005d850 tx=0x7f3f9001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6f5d63e8fdeb06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3f7cff9640 1 -- 172.21.15.53:0/2188218457 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90064990 con 0x55d820289330 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3f7cff9640 1 -- 172.21.15.53:0/2188218457 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f90097300 con 0x55d820289330 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3f7cff9640 1 -- 172.21.15.53:0/2188218457 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9005f840 con 0x55d820289330 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 -- 172.21.15.53:0/2188218457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 msgr2=0x55d82029ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82029ff90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f9005d850 tx=0x7f3f9001e000 comp rx=0 tx=0).stop 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 -- 172.21.15.53:0/2188218457 shutdown_connections 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82029ff90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d8202a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/2188218457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d8202a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 -- 172.21.15.53:0/2188218457 >> 172.21.15.53:0/2188218457 conn(0x55d82025bbb0 msgr2=0x55d820261be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 -- 172.21.15.53:0/2188218457 shutdown_connections 2024-05-10T10:35:58.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.148+0000 7f3fa09c3740 1 -- 172.21.15.53:0/2188218457 wait complete. 2024-05-10T10:35:58.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.149+0000 7f3fa09c3740 1 -- start start 2024-05-10T10:35:58.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.149+0000 7f3fa09c3740 1 Processor -- start 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d820286820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d820323f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3fa09c3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82025f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d82029e250 con 0x55d820289330 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8202a7e50 con 0x55d820288f50 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3fa09c3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d8202a8e50 con 0x55d82027fcd0 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3fa09c3740 10 client.0 _ll_register_callbacks cb 0x7f3f90060450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82025f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82025f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56654/0 (socket says 172.21.15.53:56654) 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 -- 172.21.15.53:0/3604659193 learned_addr learned my addr 172.21.15.53:0/3604659193 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f96ffd640 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d820323f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f977fe640 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d820286820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 -- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 msgr2=0x55d820286820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d820286820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 -- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 msgr2=0x55d820323f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d820323f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 -- 172.21.15.53:0/3604659193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8202aee30 con 0x55d820289330 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f97fff640 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82025f550 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3f90060520 tx=0x7f3f90001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532d3d0b3151a218 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.150+0000 7f3f767fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f767fc640 1 -- 172.21.15.53:0/3604659193 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9006e060 con 0x55d820289330 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f767fc640 1 -- 172.21.15.53:0/3604659193 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f3f9005fa50 con 0x55d820289330 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f90097300 con 0x55d820289330 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f767fc640 1 -- 172.21.15.53:0/3604659193 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f90074050 con 0x55d820289330 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 10 client.5766 fetch_fsmap learned FSMap version 34 2024-05-10T10:35:58.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55d8202aee30 con 0x55d820289330 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f767fc640 1 -- 172.21.15.53:0/3604659193 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f90097300 con 0x55d820289330 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f767fc640 1 -- 172.21.15.53:0/3604659193 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f3f9005bf50 con 0x55d820289330 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 10 client.5766 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 10 client.5766 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 20 client.5766 populate_metadata read hostname 'smithi053' 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d820282ac0 0x55d820282ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 4 client.5766 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "status"}) -- 0x55d8201d34e0 con 0x55d820282ac0 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f977fe640 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d820282ac0 0x55d820282ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f767fc640 10 client.5766 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.151+0000 7f3f977fe640 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d820282ac0 0x55d820282ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6467b8a219a0ba3 server_cookie=ea0f07b1594bd008 in_seq=0 out_seq=0 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f767fc640 1 -- 172.21.15.53:0/3604659193 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55d8201d34e0 con 0x55d820282ac0 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f767fc640 10 client.5766 handle_command_reply: tid=0 2024-05-10T10:35:58.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 client.5766 shutdown 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 msgr2=0x55d82025f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82025f550 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3f90060520 tx=0x7f3f90001250 comp rx=0 tx=0).stop 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 shutdown_connections 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55d820282ac0 0x55d820282ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d820289330 0x55d82025f550 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f767fc640 0 client.5766 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d820288f50 0x55d820323f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 --2- 172.21.15.53:0/3604659193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d82027fcd0 0x55d820286820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 >> 172.21.15.53:0/3604659193 conn(0x55d82025bbb0 msgr2=0x55d820261be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 shutdown_connections 2024-05-10T10:35:58.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 1 -- 172.21.15.53:0/3604659193 wait complete. 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3fa09c3740 20 client.5766 trim_cache size 0 max 16384 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 5622, 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 5.768858882, 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 34, 2024-05-10T10:35:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 61, 2024-05-10T10:35:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 61, 2024-05-10T10:35:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 10.775645454999999 2024-05-10T10:35:58.161 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:35:58.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f84076300 msgr2=0x7f3f840787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f84076300 0x7f3f840787c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f88034000 tx=0x7f3f8802e000 comp rx=0 tx=0).stop 2024-05-10T10:35:58.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f980ba830 msgr2=0x7f3f98157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f980ba830 0x7f3f98157580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000cad0 tx=0x7f3f9000cf90 comp rx=0 tx=0).stop 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 shutdown_connections 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f84076300 0x7f3f840787c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f980ba830 0x7f3f98157580 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f98162b30 0x7f3f980cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.152+0000 7f3f9dca0640 1 --2- 172.21.15.53:0/402010140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f98162750 0x7f3f98159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.153+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 >> 172.21.15.53:0/402010140 conn(0x7f3f9809e3b0 msgr2=0x7f3f980d4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.153+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 shutdown_connections 2024-05-10T10:35:58.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.153+0000 7f3f9dca0640 1 -- 172.21.15.53:0/402010140 wait complete. 2024-05-10T10:35:58.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:35:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.267+0000 7f2959f6a640 1 Processor -- start 2024-05-10T10:35:58.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.268+0000 7f2959f6a640 1 -- start start 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.268+0000 7f2959f6a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 0x7f29540d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.268+0000 7f2959f6a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f29540d5d30 0x7f29540d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.268+0000 7f2959f6a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29540d30b0 con 0x7f295415f790 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.268+0000 7f2959f6a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f29540d3390 con 0x7f29540d5d30 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.268+0000 7f2959f6a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29540c8c80 con 0x7f295415f3b0 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f2952ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f295415f790 0x7f29540c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48980/0 (socket says 172.21.15.53:48980) 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f2952ffd640 1 -- 172.21.15.53:0/406875975 learned_addr learned my addr 172.21.15.53:0/406875975 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f2953fff640 1 --2- 172.21.15.53:0/406875975 >> v2:172.21.15.135:3300/0 conn(0x7f29540d5d30 0x7f29540d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29537fe640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 0x7f29540d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3089081894 0 0) 0x7f29540d30b0 con 0x7f295415f790 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f293c003520 con 0x7f295415f790 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2710298176 0 0) 0x7f293c003520 con 0x7f295415f790 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 msgr2=0x7f29540d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 0x7f29540d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 >> v2:172.21.15.135:3300/0 conn(0x7f29540d5d30 msgr2=0x7f29540d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 --2- 172.21.15.53:0/406875975 >> v2:172.21.15.135:3300/0 conn(0x7f29540d5d30 0x7f29540d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29540c8f60 con 0x7f295415f790 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f2948001670 con 0x7f295415f790 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 >> v1:172.21.15.53:6789/0 conn(0x7f295415f790 legacy=0x7f29540c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c0039b0 0x7f293c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c004800 0x7f293c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 0x7f293c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29540d3390 con 0x7f293c004800 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29540c8c80 con 0x7f293c0039b0 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f293c00b9c0 con 0x7f295415f3b0 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f2952ffd640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c004800 0x7f293c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f29537fe640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c0039b0 0x7f293c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.269+0000 7f2953fff640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 0x7f293c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f2952ffd640 1 -- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 msgr2=0x7f293c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f2952ffd640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295415f3b0 0x7f293c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f2952ffd640 1 -- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c0039b0 msgr2=0x7f293c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f2952ffd640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c0039b0 0x7f293c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f2952ffd640 1 -- 172.21.15.53:0/406875975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c003520 con 0x7f293c004800 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f2952ffd640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c004800 0x7f293c004c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f294800c210 tx=0x7f293c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77764eca12c01d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948007d30 con 0x7f293c004800 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2948005ad0 con 0x7f293c004800 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.270+0000 7f29527fc640 1 -- 172.21.15.53:0/406875975 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948007070 con 0x7f293c004800 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.271+0000 7f2959f6a640 1 -- 172.21.15.53:0/406875975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c004800 msgr2=0x7f293c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.271+0000 7f2959f6a640 1 --2- 172.21.15.53:0/406875975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c004800 0x7f293c004c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f294800c210 tx=0x7f293c005830 comp rx=0 tx=0).stop 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.271+0000 7f2953fff640 1 -- 172.21.15.53:0/406875975 reap_dead start 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.271+0000 7f2959f6a640 1 -- 172.21.15.53:0/406875975 shutdown_connections 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.271+0000 7f2959f6a640 1 -- 172.21.15.53:0/406875975 >> 172.21.15.53:0/406875975 conn(0x7f29540d5130 msgr2=0x7f29540d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.271+0000 7f2959f6a640 1 -- 172.21.15.53:0/406875975 shutdown_connections 2024-05-10T10:35:58.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.271+0000 7f2959f6a640 1 -- 172.21.15.53:0/406875975 wait complete. 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.272+0000 7f2959f6a640 1 Processor -- start 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.272+0000 7f2959f6a640 1 -- start start 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2959f6a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c004800 0x7f29541f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2959f6a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295406d350 0x7f295406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2959f6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29540b91d0 0x7f29540b9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2959f6a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29540d3870 con 0x7f29540b91d0 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2959f6a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29540d36f0 con 0x7f293c004800 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2959f6a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29540d39f0 con 0x7f295406d350 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2953fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29540b91d0 0x7f29540b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2953fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29540b91d0 0x7f29540b9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56666/0 (socket says 172.21.15.53:56666) 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f2953fff640 1 -- 172.21.15.53:0/3590770130 learned_addr learned my addr 172.21.15.53:0/3590770130 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.273+0000 7f29537fe640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c004800 0x7f29541f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2952ffd640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295406d350 0x7f295406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2953fff640 1 -- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295406d350 msgr2=0x7f295406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2953fff640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295406d350 0x7f295406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2953fff640 1 -- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c004800 msgr2=0x7f29541f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2953fff640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c004800 0x7f29541f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2953fff640 1 -- 172.21.15.53:0/3590770130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29540b9b90 con 0x7f29540b91d0 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f29537fe640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c004800 0x7f29541f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2953fff640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29540b91d0 0x7f29540b9650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f29540cfa80 tx=0x7f294c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f293bfff640 1 -- 172.21.15.53:0/3590770130 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c015430 con 0x7f29540b91d0 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f293bfff640 1 -- 172.21.15.53:0/3590770130 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294c0155f0 con 0x7f29540b91d0 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29540b9e20 con 0x7f29540b91d0 2024-05-10T10:35:58.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f293bfff640 1 -- 172.21.15.53:0/3590770130 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c0159e0 con 0x7f29540b91d0 2024-05-10T10:35:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.274+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29540ba3e0 con 0x7f29540b91d0 2024-05-10T10:35:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.275+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295406a220 con 0x7f29540b91d0 2024-05-10T10:35:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.276+0000 7f293bfff640 1 -- 172.21.15.53:0/3590770130 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f294c015b80 con 0x7f29540b91d0 2024-05-10T10:35:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.276+0000 7f293bfff640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29400762d0 0x7f2940078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.276+0000 7f293bfff640 1 -- 172.21.15.53:0/3590770130 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f294c0985d0 con 0x7f29540b91d0 2024-05-10T10:35:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.276+0000 7f29537fe640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29400762d0 0x7f2940078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.277+0000 7f29537fe640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29400762d0 0x7f2940078790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2944009a60 tx=0x7f2944008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.281+0000 7f293bfff640 1 -- 172.21.15.53:0/3590770130 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f294c019030 con 0x7f29540b91d0 2024-05-10T10:35:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.512+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f29540c99e0 con 0x7f29540b91d0 2024-05-10T10:35:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.512+0000 7f293bfff640 1 -- 172.21.15.53:0/3590770130 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f294c010040 con 0x7f29540b91d0 2024-05-10T10:35:58.513 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:35:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29400762d0 msgr2=0x7f2940078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29400762d0 0x7f2940078790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2944009a60 tx=0x7f2944008040 comp rx=0 tx=0).stop 2024-05-10T10:35:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29540b91d0 msgr2=0x7f29540b9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29540b91d0 0x7f29540b9650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f29540cfa80 tx=0x7f294c00f580 comp rx=0 tx=0).stop 2024-05-10T10:35:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 shutdown_connections 2024-05-10T10:35:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29400762d0 0x7f2940078790 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29540b91d0 0x7f29540b9650 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f295406d350 0x7f295406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 --2- 172.21.15.53:0/3590770130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f293c004800 0x7f29541f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 >> 172.21.15.53:0/3590770130 conn(0x7f29540d5130 msgr2=0x7f29540cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 shutdown_connections 2024-05-10T10:35:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58.514+0000 7f2959f6a640 1 -- 172.21.15.53:0/3590770130 wait complete. 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:35:58.526 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:35:58.527 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:35:58.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:58.527 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:35:58.543 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:35:58.543 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:35:58.543 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:35:58.543 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42611sec preferred_lft 42611sec 2024-05-10T10:35:58.544 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:35:58.545 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:35:58.546 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:35:58.546 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:35:58.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:58.546 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:35:58.607 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:35:58.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:58.609 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:35:58.665 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:35:58.666 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:35:58.666 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:35:58.666 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:35:58.666 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:35:58.666 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:35:58.666 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:35:58.666 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:35:58.666 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:35:58.666 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:35:58.666 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:35:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:35:58.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:35:58.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:58.798 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:35:58.814 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:35:58.814 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:35:58.814 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:35:58.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:58.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:35:58.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:35:58.880 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:58.936 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:35:58.936 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:35:58.937 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:58.991 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:35:58.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:58.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:59.076 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:59.076 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:35:59.076 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:35:59.084 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:59.085 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:35:59.085 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:35:59.085 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:35:59.085 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:35:59.091 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:35:59.092 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:35:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:35:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:35:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:35:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:35:59.094 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:35:59.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:59.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:35:59.120 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:35:59.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:35:59.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:35:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:client5165 2024-05-10T10:35:59.223 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:35:59.223 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:35:59.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:59.239 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-05-10T10:35:59.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:35:59.300 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-05-10T10:35:59.358 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:35:59.358 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:35:59.358 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:35:59.358 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:35:59.358 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-05-10T10:35:59.358 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:35:59.465 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2024-05-10T10:35:59.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:35:59.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:35:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.618+0000 7f9335659640 1 Processor -- start 2024-05-10T10:35:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.619+0000 7f9335659640 1 -- start start 2024-05-10T10:35:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.619+0000 7f9335659640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f93300b5520 0x7f93300d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.619+0000 7f9335659640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93300b9180 0x7f93300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.619+0000 7f9335659640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93300d41f0 con 0x7f93300b5900 2024-05-10T10:35:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.619+0000 7f9335659640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f93300d5c60 con 0x7f93300b9180 2024-05-10T10:35:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.619+0000 7f9335659640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f93300d5eb0 con 0x7f93300b5520 2024-05-10T10:35:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.620+0000 7f932f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93300b9180 0x7f93300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.620+0000 7f932f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93300b9180 0x7f93300d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56668/0 (socket says 172.21.15.53:56668) 2024-05-10T10:35:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.620+0000 7f932f7fe640 1 -- 172.21.15.53:0/79946339 learned_addr learned my addr 172.21.15.53:0/79946339 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.620+0000 7f932f7fe640 1 -- 172.21.15.53:0/79946339 >> v2:172.21.15.135:3300/0 conn(0x7f93300b5520 msgr2=0x7f93300d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.620+0000 7f932f7fe640 1 --2- 172.21.15.53:0/79946339 >> v2:172.21.15.135:3300/0 conn(0x7f93300b5520 0x7f93300d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.620+0000 7f932f7fe640 1 -- 172.21.15.53:0/79946339 >> v1:172.21.15.135:6789/0 conn(0x7f93300b5900 legacy=0x7f93300d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:35:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.620+0000 7f932f7fe640 1 -- 172.21.15.53:0/79946339 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93300d0a30 con 0x7f93300b9180 2024-05-10T10:35:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f932f7fe640 1 --2- 172.21.15.53:0/79946339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b9180 0x7f93300d1de0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9330074950 tx=0x7f932802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3d93fc00888494 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f932e7fc640 1 -- 172.21.15.53:0/79946339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328036070 con 0x7f93300b9180 2024-05-10T10:35:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f932e7fc640 1 -- 172.21.15.53:0/79946339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9328031600 con 0x7f93300b9180 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f932e7fc640 1 -- 172.21.15.53:0/79946339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280319d0 con 0x7f93300b9180 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 -- 172.21.15.53:0/79946339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b9180 msgr2=0x7f93300d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 --2- 172.21.15.53:0/79946339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b9180 0x7f93300d1de0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9330074950 tx=0x7f932802a6f0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 -- 172.21.15.53:0/79946339 shutdown_connections 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 --2- 172.21.15.53:0/79946339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b9180 0x7f93300d1de0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 --2- 172.21.15.53:0/79946339 >> v2:172.21.15.135:3300/0 conn(0x7f93300b5520 0x7f93300d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 -- 172.21.15.53:0/79946339 >> 172.21.15.53:0/79946339 conn(0x7f93300d5170 msgr2=0x7f93300d5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 -- 172.21.15.53:0/79946339 shutdown_connections 2024-05-10T10:35:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.621+0000 7f9335659640 1 -- 172.21.15.53:0/79946339 wait complete. 2024-05-10T10:35:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 Processor -- start 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 -- start start 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93300b5520 0x7f933015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b5900 0x7f93301591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93300b9180 0x7f93301596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93300b6460 con 0x7f93300b5900 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93300b62e0 con 0x7f93300b5520 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.623+0000 7f9335659640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f93300b65e0 con 0x7f93300b9180 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b5900 0x7f93301591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b5900 0x7f93301591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56684/0 (socket says 172.21.15.53:56684) 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 -- 172.21.15.53:0/3709128835 learned_addr learned my addr 172.21.15.53:0/3709128835 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f932effd640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93300b5520 0x7f933015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f932f7fe640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93300b9180 0x7f93301596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 -- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93300b9180 msgr2=0x7f93301596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93300b9180 0x7f93301596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 -- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93300b5520 msgr2=0x7f933015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93300b5520 0x7f933015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 -- 172.21.15.53:0/3709128835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933015b930 con 0x7f93300b5900 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f932effd640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93300b5520 0x7f933015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9327fff640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b5900 0x7f93301591b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f931800c440 tx=0x7f931800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f93277fe640 1 -- 172.21.15.53:0/3709128835 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318017070 con 0x7f93300b5900 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f93277fe640 1 -- 172.21.15.53:0/3709128835 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93180035f0 con 0x7f93300b5900 2024-05-10T10:35:59.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f93277fe640 1 -- 172.21.15.53:0/3709128835 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93180039e0 con 0x7f93300b5900 2024-05-10T10:35:59.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933015bc10 con 0x7f93300b5900 2024-05-10T10:35:59.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.624+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933015c1d0 con 0x7f93300b5900 2024-05-10T10:35:59.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.625+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330069d40 con 0x7f93300b5900 2024-05-10T10:35:59.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.626+0000 7f93277fe640 1 -- 172.21.15.53:0/3709128835 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f931801a070 con 0x7f93300b5900 2024-05-10T10:35:59.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.626+0000 7f93277fe640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93100762d0 0x7f9310078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:35:59.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.626+0000 7f93277fe640 1 -- 172.21.15.53:0/3709128835 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f9318096b60 con 0x7f93300b5900 2024-05-10T10:35:59.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.627+0000 7f932effd640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93100762d0 0x7f9310078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:35:59.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.627+0000 7f932effd640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93100762d0 0x7f9310078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9320009930 tx=0x7f9320008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:35:59.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.631+0000 7f93277fe640 1 -- 172.21.15.53:0/3709128835 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9330069d40 con 0x7f93300b5900 2024-05-10T10:35:59.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.862+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93300693f0 con 0x7f93300b5900 2024-05-10T10:35:59.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.863+0000 7f93277fe640 1 -- 172.21.15.53:0/3709128835 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5188 (secure 0 0 0) 0x7f9318060290 con 0x7f93300b5900 2024-05-10T10:35:59.864 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:35:59.864 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:35:54.766365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:35:59.864 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 34 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93100762d0 msgr2=0x7f9310078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93100762d0 0x7f9310078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9320009930 tx=0x7f9320008040 comp rx=0 tx=0).stop 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b5900 msgr2=0x7f93301591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b5900 0x7f93301591b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f931800c440 tx=0x7f931800c900 comp rx=0 tx=0).stop 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 shutdown_connections 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93100762d0 0x7f9310078790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93300b9180 0x7f93301596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300b5900 0x7f93301591b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 --2- 172.21.15.53:0/3709128835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93300b5520 0x7f933015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:35:59.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 >> 172.21.15.53:0/3709128835 conn(0x7f93300d5170 msgr2=0x7f93300d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:35:59.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 shutdown_connections 2024-05-10T10:35:59.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:35:59.865+0000 7f9335659640 1 -- 172.21.15.53:0/3709128835 wait complete. 2024-05-10T10:35:59.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-10T10:36:00.009 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:36:00.009 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:36:00.010 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-10T10:36:00.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.054+0000 7f1642d1ff40 1 Processor -- start 2024-05-10T10:36:00.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.054+0000 7f1642d1ff40 1 -- start start 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f1642d1ff40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 0x55b04bec84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f1642d1ff40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04be42900 0x55b04be405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f1642d1ff40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04becd3b0 0x55b04be5bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f1642d1ff40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b04be42ce0 con 0x55b04be42900 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f1642d1ff40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b04bdb9f50 con 0x55b04becd3b0 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f1642d1ff40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b04be5b1d0 con 0x55b04be10830 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04be42900 0x55b04be405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04be42900 0x55b04be405c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56690/0 (socket says 172.21.15.53:56690) 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 -- 172.21.15.53:0/1972219832 learned_addr learned my addr 172.21.15.53:0/1972219832 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f1640819640 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 0x55b04bec84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f164101a640 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04becd3b0 0x55b04be5bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 -- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 msgr2=0x55b04bec84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 0x55b04bec84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 -- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04becd3b0 msgr2=0x55b04be5bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04becd3b0 0x55b04be5bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.055+0000 7f163bfff640 1 -- 172.21.15.53:0/1972219832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04bee9a40 con 0x55b04be42900 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f164101a640 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04becd3b0 0x55b04be5bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f163bfff640 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04be42900 0x55b04be405c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f162c001ce0 tx=0x7f162c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdfa82449b409a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f163affd640 1 -- 172.21.15.53:0/1972219832 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c00bd80 con 0x55b04be42900 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f163affd640 1 -- 172.21.15.53:0/1972219832 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f162c00f400 con 0x55b04be42900 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f163affd640 1 -- 172.21.15.53:0/1972219832 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c00f7f0 con 0x55b04be42900 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 -- 172.21.15.53:0/1972219832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04be42900 msgr2=0x55b04be405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04be42900 0x55b04be405c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f162c001ce0 tx=0x7f162c00b5b0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 -- 172.21.15.53:0/1972219832 shutdown_connections 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04becd3b0 0x55b04be5bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04be42900 0x55b04be405c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/1972219832 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 0x55b04bec84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 -- 172.21.15.53:0/1972219832 >> 172.21.15.53:0/1972219832 conn(0x55b04be12040 msgr2=0x55b04becdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 -- 172.21.15.53:0/1972219832 shutdown_connections 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.056+0000 7f1642d1ff40 1 -- 172.21.15.53:0/1972219832 wait complete. 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.057+0000 7f1642d1ff40 1 Processor -- start 2024-05-10T10:36:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.057+0000 7f1642d1ff40 1 -- start start 2024-05-10T10:36:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f1642d1ff40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 0x55b04bec7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f1642d1ff40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04be42900 0x55b04bee8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f1642d1ff40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04becd3b0 0x55b04be5c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f1642d1ff40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b04bee97a0 con 0x55b04becd3b0 2024-05-10T10:36:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f1642d1ff40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b04be40130 con 0x55b04be42900 2024-05-10T10:36:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f1642d1ff40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b04beed020 con 0x55b04be10830 2024-05-10T10:36:00.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04becd3b0 0x55b04be5c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04becd3b0 0x55b04be5c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56696/0 (socket says 172.21.15.53:56696) 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 -- 172.21.15.53:0/137787653 learned_addr learned my addr 172.21.15.53:0/137787653 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f1640819640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 0x55b04bec7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f163bfff640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04be42900 0x55b04bee8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 -- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 msgr2=0x55b04bec7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b04be10830 0x55b04bec7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 -- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04be42900 msgr2=0x55b04bee8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04be42900 0x55b04bee8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 -- 172.21.15.53:0/137787653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04bee9090 con 0x55b04becd3b0 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.058+0000 7f164101a640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04becd3b0 0x55b04be5c0f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f163c00f870 tx=0x7f163c00fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.059+0000 7f1639ffb640 1 -- 172.21.15.53:0/137787653 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c019020 con 0x55b04becd3b0 2024-05-10T10:36:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.059+0000 7f1639ffb640 1 -- 172.21.15.53:0/137787653 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c00fe60 con 0x55b04becd3b0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.059+0000 7f1639ffb640 1 -- 172.21.15.53:0/137787653 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c00dbc0 con 0x55b04becd3b0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.059+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b04becfc40 con 0x55b04becd3b0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.059+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04be5c630 con 0x55b04becd3b0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.060+0000 7f1639ffb640 1 -- 172.21.15.53:0/137787653 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f163c01c020 con 0x55b04becd3b0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.061+0000 7f1639ffb640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1624074ce0 0x7f16240771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.061+0000 7f1639ffb640 1 -- 172.21.15.53:0/137787653 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f163c096a10 con 0x55b04becd3b0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.061+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b04bed0720 0x55b04bed0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.061+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61) -- 0x55b04bed1040 con 0x55b04bed0720 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.061+0000 7f1640819640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1624074ce0 0x7f16240771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.061+0000 7f163bfff640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b04bed0720 0x55b04bed0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1640819640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1624074ce0 0x7f16240771a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1634056000 tx=0x7f1634050000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f163bfff640 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b04bed0720 0x55b04bed0b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f163bfff640 1 -- 172.21.15.53:0/137787653 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f162c00f5b0 con 0x55b04bed0720 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b04bed0720 msgr2=0x55b04bed0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b04bed0720 0x55b04bed0b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1624074ce0 msgr2=0x7f16240771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1624074ce0 0x7f16240771a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1634056000 tx=0x7f1634050000 comp rx=0 tx=0).stop 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04becd3b0 msgr2=0x55b04be5c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 --2- 172.21.15.53:0/137787653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b04becd3b0 0x55b04be5c0f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f163c00f870 tx=0x7f163c00fd30 comp rx=0 tx=0).stop 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f164101a640 1 -- 172.21.15.53:0/137787653 reap_dead start 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 shutdown_connections 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 >> 172.21.15.53:0/137787653 conn(0x55b04be12040 msgr2=0x55b04beec580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 shutdown_connections 2024-05-10T10:36:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.062+0000 7f1642d1ff40 1 -- 172.21.15.53:0/137787653 wait complete. 2024-05-10T10:36:00.068 INFO:teuthology.orchestra.run.smithi053.stdout:Xfiledir_test 2024-05-10T10:36:00.068 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:36:00.069 DEBUG:teuthology.orchestra.run.smithi053:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-10T10:36:00.116 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:36:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2024-05-10T10:36:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ancestors": [ 2024-05-10T10:36:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:36:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1099511627776, 2024-05-10T10:36:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "file", 2024-05-10T10:36:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 2 2024-05-10T10:36:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:36:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:36:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: "dirino": 1, 2024-05-10T10:36:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "dir_test", 2024-05-10T10:36:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 4 2024-05-10T10:36:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:36:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-05-10T10:36:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 10, 2024-05-10T10:36:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: "old_pools": [] 2024-05-10T10:36:00.119 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:36:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:36:00.120 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-05-10T10:36:00.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 Processor -- start 2024-05-10T10:36:00.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 -- start start 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688055d0 0x5649688aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688a4b00 0x5649688a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688ab850 0x5649688a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56496877cf50 con 0x5649688ab850 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564968780f90 con 0x5649688055d0 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eb0d28f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5649688ab050 con 0x5649688a4b00 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eae5be640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688ab850 0x5649688a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eae5be640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688ab850 0x5649688a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56700/0 (socket says 172.21.15.53:56700) 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.206+0000 7f1eae5be640 1 -- 172.21.15.53:0/1484594761 learned_addr learned my addr 172.21.15.53:0/1484594761 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eaddbd640 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688055d0 0x5649688aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eae5be640 1 -- 172.21.15.53:0/1484594761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688a4b00 msgr2=0x5649688a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eae5be640 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688a4b00 0x5649688a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eae5be640 1 -- 172.21.15.53:0/1484594761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688055d0 msgr2=0x5649688aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eae5be640 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688055d0 0x5649688aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eae5be640 1 -- 172.21.15.53:0/1484594761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649688016c0 con 0x5649688ab850 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eae5be640 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688ab850 0x5649688a26a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001620 tx=0x7f1ea0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5bb939204f7d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1ea7fff640 1 -- 172.21.15.53:0/1484594761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0017070 con 0x5649688ab850 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1ea7fff640 1 -- 172.21.15.53:0/1484594761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea000f710 con 0x5649688ab850 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1ea7fff640 1 -- 172.21.15.53:0/1484594761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000fb00 con 0x5649688ab850 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/1484594761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688ab850 msgr2=0x5649688a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.207+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688ab850 0x5649688a26a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001620 tx=0x7f1ea0001ae0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/1484594761 shutdown_connections 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688ab850 0x5649688a26a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688a4b00 0x5649688a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/1484594761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688055d0 0x5649688aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/1484594761 >> 172.21.15.53:0/1484594761 conn(0x5649687d5050 msgr2=0x564968804ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/1484594761 shutdown_connections 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/1484594761 wait complete. 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 Processor -- start 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.208+0000 7f1eb0d28f40 1 -- start start 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eb0d28f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688055d0 0x5649688a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eb0d28f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688a4b00 0x564968934cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eb0d28f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688ab850 0x5649688a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eb0d28f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5649688aa970 con 0x5649688055d0 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eb0d28f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564968899f60 con 0x5649688a4b00 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eb0d28f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5649687d3f80 con 0x5649688ab850 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688055d0 0x5649688a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688055d0 0x5649688a9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56702/0 (socket says 172.21.15.53:56702) 2024-05-10T10:36:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 -- 172.21.15.53:0/639834597 learned_addr learned my addr 172.21.15.53:0/639834597 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eae5be640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688ab850 0x5649688a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1ead5bc640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688a4b00 0x564968934cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 -- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688ab850 msgr2=0x5649688a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5649688ab850 0x5649688a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 -- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688a4b00 msgr2=0x564968934cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688a4b00 0x564968934cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1eaddbd640 1 -- 172.21.15.53:0/639834597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649688a3f80 con 0x5649688055d0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.209+0000 7f1ead5bc640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649688a4b00 0x564968934cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.210+0000 7f1eaddbd640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688055d0 0x5649688a9ad0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8000c00 tx=0x7f1ea8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.210+0000 7f1ea6ffd640 1 -- 172.21.15.53:0/639834597 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea806b070 con 0x5649688055d0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.210+0000 7f1ea6ffd640 1 -- 172.21.15.53:0/639834597 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea8052b50 con 0x5649688055d0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.210+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5649689341f0 con 0x5649688055d0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.210+0000 7f1ea6ffd640 1 -- 172.21.15.53:0/639834597 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8056040 con 0x5649688055d0 2024-05-10T10:36:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.210+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649689347b0 con 0x5649688055d0 2024-05-10T10:36:00.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.211+0000 7f1ea6ffd640 1 -- 172.21.15.53:0/639834597 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ea806e070 con 0x5649688055d0 2024-05-10T10:36:00.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.212+0000 7f1ea6ffd640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1e90074ce0 0x7f1e900771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.212+0000 7f1ea6ffd640 1 -- 172.21.15.53:0/639834597 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f1ea80da2c0 con 0x5649688055d0 2024-05-10T10:36:00.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.212+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x5649688a2dd0 0x564968936c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.212+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61) -- 0x564968937150 con 0x5649688a2dd0 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.212+0000 7f1ead5bc640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1e90074ce0 0x7f1e900771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.212+0000 7f1eae5be640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x5649688a2dd0 0x564968936c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.213+0000 7f1ead5bc640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1e90074ce0 0x7f1e900771a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e98000f60 tx=0x7f1e980015e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.213+0000 7f1eae5be640 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x5649688a2dd0 0x564968936c10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eae5be640 1 -- 172.21.15.53:0/639834597 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v61'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1ea0003030 con 0x5649688a2dd0 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x5649688a2dd0 msgr2=0x564968936c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x5649688a2dd0 0x564968936c10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1e90074ce0 msgr2=0x7f1e900771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1e90074ce0 0x7f1e900771a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e98000f60 tx=0x7f1e980015e0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688055d0 msgr2=0x5649688a9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 --2- 172.21.15.53:0/639834597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649688055d0 0x5649688a9ad0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8000c00 tx=0x7f1ea8002fb0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eae5be640 1 -- 172.21.15.53:0/639834597 reap_dead start 2024-05-10T10:36:00.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 shutdown_connections 2024-05-10T10:36:00.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 >> 172.21.15.53:0/639834597 conn(0x5649687d5050 msgr2=0x564968804ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 shutdown_connections 2024-05-10T10:36:00.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.214+0000 7f1eb0d28f40 1 -- 172.21.15.53:0/639834597 wait complete. 2024-05-10T10:36:00.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:00.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-05-10T10:36:00.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.326+0000 7f3139f40640 1 Processor -- start 2024-05-10T10:36:00.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.326+0000 7f3139f40640 1 -- start start 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.327+0000 7f3139f40640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f313415f600 0x7f31340d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.327+0000 7f3139f40640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31340d3d20 0x7f31340b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.327+0000 7f3139f40640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31340b46b0 con 0x7f31340d0b90 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.327+0000 7f3139f40640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f31340b4990 con 0x7f31340d3d20 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.327+0000 7f3139f40640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31340b4c70 con 0x7f313415f600 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f31337fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f313415f600 0x7f31340d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f3133fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31340d3d20 0x7f31340b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f31337fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f313415f600 0x7f31340d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:58338/0 (socket says 172.21.15.53:58338) 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f31337fe640 1 -- 172.21.15.53:0/4092691870 learned_addr learned my addr 172.21.15.53:0/4092691870 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f3132ffd640 1 --1- 172.21.15.53:0/4092691870 >> v1:172.21.15.135:6789/0 conn(0x7f31340d0b90 0x7f31340d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:59144/0 (socket says 172.21.15.53:59144) 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f31337fe640 1 -- 172.21.15.53:0/4092691870 >> v2:172.21.15.135:3300/0 conn(0x7f31340d3d20 msgr2=0x7f31340b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f31337fe640 1 --2- 172.21.15.53:0/4092691870 >> v2:172.21.15.135:3300/0 conn(0x7f31340d3d20 0x7f31340b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f31337fe640 1 -- 172.21.15.53:0/4092691870 >> v1:172.21.15.135:6789/0 conn(0x7f31340d0b90 legacy=0x7f31340d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.328+0000 7f31337fe640 1 -- 172.21.15.53:0/4092691870 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31340b4f50 con 0x7f313415f600 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f31337fe640 1 --2- 172.21.15.53:0/4092691870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f313415f600 0x7f31340d0650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3124009870 tx=0x7f312402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7019b24f0dadb311 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f31327fc640 1 -- 172.21.15.53:0/4092691870 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124037070 con 0x7f313415f600 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f31327fc640 1 -- 172.21.15.53:0/4092691870 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3124032440 con 0x7f313415f600 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f31327fc640 1 -- 172.21.15.53:0/4092691870 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124032810 con 0x7f313415f600 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 -- 172.21.15.53:0/4092691870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f313415f600 msgr2=0x7f31340d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 --2- 172.21.15.53:0/4092691870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f313415f600 0x7f31340d0650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3124009870 tx=0x7f312402a1c0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 -- 172.21.15.53:0/4092691870 shutdown_connections 2024-05-10T10:36:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 --2- 172.21.15.53:0/4092691870 >> v2:172.21.15.135:3300/0 conn(0x7f31340d3d20 0x7f31340b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 --2- 172.21.15.53:0/4092691870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f313415f600 0x7f31340d0650 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 -- 172.21.15.53:0/4092691870 >> 172.21.15.53:0/4092691870 conn(0x7f313409e390 msgr2=0x7f31340d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 -- 172.21.15.53:0/4092691870 shutdown_connections 2024-05-10T10:36:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.329+0000 7f3139f40640 1 -- 172.21.15.53:0/4092691870 wait complete. 2024-05-10T10:36:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.331+0000 7f3139f40640 1 Processor -- start 2024-05-10T10:36:00.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.331+0000 7f3139f40640 1 -- start start 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.331+0000 7f3139f40640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31340d0b90 0x7f31341f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3139f40640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31340d3d20 0x7f31340ba200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3139f40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f313415f600 0x7f31340cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3139f40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31340b5e10 con 0x7f313415f600 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3139f40640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31340b5c90 con 0x7f31340d0b90 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3139f40640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31340b5f90 con 0x7f31340d3d20 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f313415f600 0x7f31340cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f313415f600 0x7f31340cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56712/0 (socket says 172.21.15.53:56712) 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 -- 172.21.15.53:0/1690296810 learned_addr learned my addr 172.21.15.53:0/1690296810 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f31337fe640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31340d0b90 0x7f31341f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3132ffd640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31340d3d20 0x7f31340ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 -- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31340d3d20 msgr2=0x7f31340ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31340d3d20 0x7f31340ba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 -- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31340d0b90 msgr2=0x7f31341f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31340d0b90 0x7f31341f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 -- 172.21.15.53:0/1690296810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31341f5c70 con 0x7f313415f600 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3132ffd640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31340d3d20 0x7f31340ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3133fff640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f313415f600 0x7f31340cb2d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f312c009dd0 tx=0x7f312c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f311bfff640 1 -- 172.21.15.53:0/1690296810 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c018020 con 0x7f313415f600 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f311bfff640 1 -- 172.21.15.53:0/1690296810 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c014480 con 0x7f313415f600 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31341f9290 con 0x7f313415f600 2024-05-10T10:36:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.332+0000 7f311bfff640 1 -- 172.21.15.53:0/1690296810 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c014870 con 0x7f313415f600 2024-05-10T10:36:00.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.333+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31340b9160 con 0x7f313415f600 2024-05-10T10:36:00.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.335+0000 7f311bfff640 1 -- 172.21.15.53:0/1690296810 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f312c005d10 con 0x7f313415f600 2024-05-10T10:36:00.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.335+0000 7f311bfff640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3120076300 0x7f31200787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.335+0000 7f311bfff640 1 -- 172.21.15.53:0/1690296810 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f312c0983c0 con 0x7f313415f600 2024-05-10T10:36:00.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.335+0000 7f31337fe640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3120076300 0x7f31200787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.336+0000 7f313cc5a740 1 Processor -- start 2024-05-10T10:36:00.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.336+0000 7f313cc5a740 1 -- start start 2024-05-10T10:36:00.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.336+0000 7f31337fe640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3120076300 0x7f31200787c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3124034000 tx=0x7f312402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efbe50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efbe55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbb54f0 0x55c9efbb8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f313cc5a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9efb1ebc0 con 0x55c9efbe2c70 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9efad7d30 con 0x55c9efbe2890 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c9efb1ab80 con 0x55c9efbb54f0 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efbe55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efbe55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56724/0 (socket says 172.21.15.53:56724) 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 -- 172.21.15.53:0/1757438663 learned_addr learned my addr 172.21.15.53:0/1757438663 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f31337fe640 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efbe50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3133fff640 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbb54f0 0x55c9efbb8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 -- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbb54f0 msgr2=0x55c9efbb8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbb54f0 0x55c9efbb8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 -- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 msgr2=0x55c9efbe50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efbe50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 -- 172.21.15.53:0/1757438663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9efabc100 con 0x55c9efbe2c70 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f3132ffd640 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efbe55e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3128005d70 tx=0x7f3128007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0f68cf931929cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f31197fa640 1 -- 172.21.15.53:0/1757438663 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128019070 con 0x55c9efbe2c70 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f31197fa640 1 -- 172.21.15.53:0/1757438663 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31280016a0 con 0x55c9efbe2c70 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.337+0000 7f31197fa640 1 -- 172.21.15.53:0/1757438663 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128001a90 con 0x55c9efbe2c70 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 -- 172.21.15.53:0/1757438663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 msgr2=0x55c9efbe55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efbe55e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3128005d70 tx=0x7f3128007420 comp rx=0 tx=0).stop 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 -- 172.21.15.53:0/1757438663 shutdown_connections 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbb54f0 0x55c9efbb8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efbe55e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 --2- 172.21.15.53:0/1757438663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efbe50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 -- 172.21.15.53:0/1757438663 >> 172.21.15.53:0/1757438663 conn(0x55c9efbb7760 msgr2=0x55c9efc000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 -- 172.21.15.53:0/1757438663 shutdown_connections 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.338+0000 7f313cc5a740 1 -- 172.21.15.53:0/1757438663 wait complete. 2024-05-10T10:36:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.339+0000 7f313cc5a740 1 -- start start 2024-05-10T10:36:00.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.339+0000 7f313cc5a740 1 Processor -- start 2024-05-10T10:36:00.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2890 0x55c9efbe2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2c70 0x55c9efbe7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efc011d0 0x55c9efbfc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9efbba910 con 0x55c9efbe2890 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9efadb870 con 0x55c9efbe2c70 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c9efbbabf0 con 0x55c9efc011d0 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 10 client.0 _ll_register_callbacks cb 0x55c9efb2bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2890 0x55c9efbe2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2890 0x55c9efbe2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56728/0 (socket says 172.21.15.53:56728) 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 -- 172.21.15.53:0/2992895392 learned_addr learned my addr 172.21.15.53:0/2992895392 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f3132ffd640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2c70 0x55c9efbe7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f3133fff640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efc011d0 0x55c9efbfc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 -- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efc011d0 msgr2=0x55c9efbfc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efc011d0 0x55c9efbfc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 -- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2c70 msgr2=0x55c9efbe7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2c70 0x55c9efbe7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 -- 172.21.15.53:0/2992895392 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ef8fc7b0 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f3133fff640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efc011d0 0x55c9efbfc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31337fe640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2890 0x55c9efbe2080 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db075cb61495a16a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31127fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31127fc640 1 -- 172.21.15.53:0/2992895392 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312404f020 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f31127fc640 1 -- 172.21.15.53:0/2992895392 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f312404b870 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.340+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31280016a0 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f31127fc640 1 -- 172.21.15.53:0/2992895392 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3124037040 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 10 client.5805 fetch_fsmap learned FSMap version 34 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55c9ef95cf40 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f31127fc640 1 -- 172.21.15.53:0/2992895392 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31280016a0 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f31127fc640 1 -- 172.21.15.53:0/2992895392 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f312404ba70 con 0x55c9efbe2890 2024-05-10T10:36:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 10 client.5805 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 10 client.5805 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 20 client.5805 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efc03140 0x55c9efc03550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 4 client.5805 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9efb2c7d0 con 0x55c9efc03140 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f3132ffd640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efc03140 0x55c9efc03550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f3132ffd640 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efc03140 0x55c9efc03550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b6d9587014d9f8 server_cookie=96689f9a03ec5657 in_seq=0 out_seq=0 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.341+0000 7f31127fc640 10 client.5805 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f31127fc640 1 -- 172.21.15.53:0/2992895392 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9efb2c7d0 con 0x55c9efc03140 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f31127fc640 10 client.5805 handle_command_reply: tid=0 2024-05-10T10:36:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 client.5805 shutdown 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2890 msgr2=0x55c9efbe2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2890 0x55c9efbe2080 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124004730 comp rx=0 tx=0).stop 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 shutdown_connections 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efc03140 0x55c9efc03550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efc011d0 0x55c9efbfc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2c70 0x55c9efbe7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2992895392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2890 0x55c9efbe2080 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f31127fc640 0 client.5805 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 >> 172.21.15.53:0/2992895392 conn(0x55c9efbb54f0 msgr2=0x55c9efbb8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 shutdown_connections 2024-05-10T10:36:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 1 -- 172.21.15.53:0/2992895392 wait complete. 2024-05-10T10:36:00.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.342+0000 7f313cc5a740 20 client.5805 trim_cache size 0 max 16384 2024-05-10T10:36:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.360+0000 7f313cc5a740 1 Processor -- start 2024-05-10T10:36:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.361+0000 7f313cc5a740 1 -- start start 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.361+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efc000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.361+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2c70 0x55c9efc10bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.361+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efc011d0 0x55c9efbb59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.361+0000 7f313cc5a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3134201ec0 con 0x55c9efc011d0 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.361+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3134200f10 con 0x55c9efbe2890 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.361+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f313414efc0 con 0x55c9efbe2c70 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efc011d0 0x55c9efbb59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efc011d0 0x55c9efbb59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56734/0 (socket says 172.21.15.53:56734) 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 -- 172.21.15.53:0/3653673500 learned_addr learned my addr 172.21.15.53:0/3653673500 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f31337fe640 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efc000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3132ffd640 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2c70 0x55c9efc10bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 -- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2c70 msgr2=0x55c9efc10bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2c70 0x55c9efc10bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 -- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 msgr2=0x55c9efc000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efc000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 -- 172.21.15.53:0/3653673500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ef7f00a0 con 0x55c9efc011d0 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3132ffd640 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2c70 0x55c9efc10bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3133fff640 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efc011d0 0x55c9efbb59d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f312c009a40 tx=0x7f312c05dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8797e16dedabeb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3118ff9640 1 -- 172.21.15.53:0/3653673500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c07a020 con 0x55c9efc011d0 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3118ff9640 1 -- 172.21.15.53:0/3653673500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c06a2f0 con 0x55c9efc011d0 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.362+0000 7f3118ff9640 1 -- 172.21.15.53:0/3653673500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c069e60 con 0x55c9efc011d0 2024-05-10T10:36:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 -- 172.21.15.53:0/3653673500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efc011d0 msgr2=0x55c9efbb59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efc011d0 0x55c9efbb59d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f312c009a40 tx=0x7f312c05dbf0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 -- 172.21.15.53:0/3653673500 shutdown_connections 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efc011d0 0x55c9efbb59d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2c70 0x55c9efc10bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 --2- 172.21.15.53:0/3653673500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efbe2890 0x55c9efc000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 -- 172.21.15.53:0/3653673500 >> 172.21.15.53:0/3653673500 conn(0x55c9efbb54f0 msgr2=0x55c9efc240b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 -- 172.21.15.53:0/3653673500 shutdown_connections 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 -- 172.21.15.53:0/3653673500 wait complete. 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.363+0000 7f313cc5a740 1 -- start start 2024-05-10T10:36:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 1 Processor -- start 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2890 0x55c9efbe1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efc7dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efc011d0 0x55c9efbfed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31342008c0 con 0x55c9efbe2c70 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9efc7e550 con 0x55c9efc011d0 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c9efbe6400 con 0x55c9efbe2890 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f313cc5a740 10 client.0 _ll_register_callbacks cb 0x7f31240096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efc7dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efc7dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56746/0 (socket says 172.21.15.53:56746) 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.364+0000 7f3132ffd640 1 -- 172.21.15.53:0/2449276924 learned_addr learned my addr 172.21.15.53:0/2449276924 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f31337fe640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2890 0x55c9efbe1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3133fff640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efc011d0 0x55c9efbfed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3132ffd640 1 -- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2890 msgr2=0x55c9efbe1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3132ffd640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2890 0x55c9efbe1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3132ffd640 1 -- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efc011d0 msgr2=0x55c9efbfed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3132ffd640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efc011d0 0x55c9efbfed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3132ffd640 1 -- 172.21.15.53:0/2449276924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9efbe66e0 con 0x55c9efbe2c70 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3133fff640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efc011d0 0x55c9efbfed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f3132ffd640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efc7dad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3128013000 tx=0x7f3128000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea92f5450203c9e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f31127fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f31127fc640 1 -- 172.21.15.53:0/2449276924 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128029020 con 0x55c9efbe2c70 2024-05-10T10:36:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f31127fc640 1 -- 172.21.15.53:0/2449276924 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f31280016a0 con 0x55c9efbe2c70 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f312c06a2f0 con 0x55c9efbe2c70 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f31127fc640 1 -- 172.21.15.53:0/2449276924 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3128033050 con 0x55c9efbe2c70 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f313cc5a740 10 client.5811 fetch_fsmap learned FSMap version 34 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55c9efbff2f0 con 0x55c9efbe2c70 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f31127fc640 1 -- 172.21.15.53:0/2449276924 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c06a2f0 con 0x55c9efbe2c70 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.365+0000 7f31127fc640 1 -- 172.21.15.53:0/2449276924 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f3128036020 con 0x55c9efbe2c70 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f313cc5a740 10 client.5811 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f313cc5a740 10 client.5811 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f313cc5a740 20 client.5811 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efbe5e80 0x55c9efbe6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f313cc5a740 4 client.5811 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55c9efbe8ab0 con 0x55c9efbe5e80 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f31337fe640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efbe5e80 0x55c9efbe6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f31127fc640 10 client.5811 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.366+0000 7f31337fe640 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efbe5e80 0x55c9efbe6ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c015367461070cc server_cookie=a529eae33152c539 in_seq=0 out_seq=0 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f31127fc640 1 -- 172.21.15.53:0/2449276924 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9efbe8ab0 con 0x55c9efbe5e80 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f31127fc640 10 client.5811 handle_command_reply: tid=0 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 client.5811 shutdown 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 msgr2=0x55c9efc7dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efc7dad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3128013000 tx=0x7f3128000bc0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 shutdown_connections 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55c9efbe5e80 0x55c9efbe6ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9efc011d0 0x55c9efbfed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9efbe2c70 0x55c9efc7dad0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 --2- 172.21.15.53:0/2449276924 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c9efbe2890 0x55c9efbe1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 >> 172.21.15.53:0/2449276924 conn(0x55c9efbb54f0 msgr2=0x55c9efc240b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f31127fc640 0 client.5811 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.374 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f9df55b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5165=0-4194304@1} caps={5165=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5165 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55ebd1e6cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:36:00.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 shutdown_connections 2024-05-10T10:36:00.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 1 -- 172.21.15.53:0/2449276924 wait complete. 2024-05-10T10:36:00.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.367+0000 7f313cc5a740 20 client.5811 trim_cache size 0 max 16384 2024-05-10T10:36:00.375 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:36:00.375 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:36:00.375 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "5085360d-21b0-49c5-8ac2-915409e451b2", 2024-05-10T10:36:00.375 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:36:00.375 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3120076300 msgr2=0x7f31200787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3120076300 0x7f31200787c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3124034000 tx=0x7f312402e000 comp rx=0 tx=0).stop 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f313415f600 msgr2=0x7f31340cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f313415f600 0x7f31340cb2d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f312c009dd0 tx=0x7f312c00c490 comp rx=0 tx=0).stop 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 shutdown_connections 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3120076300 0x7f31200787c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f313415f600 0x7f31340cb2d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31340d3d20 0x7f31340ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 --2- 172.21.15.53:0/1690296810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31340d0b90 0x7f31341f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 >> 172.21.15.53:0/1690296810 conn(0x7f313409e390 msgr2=0x7f313406dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 shutdown_connections 2024-05-10T10:36:00.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.368+0000 7f3139f40640 1 -- 172.21.15.53:0/1690296810 wait complete. 2024-05-10T10:36:00.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:00.379 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-10T10:36:00.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.486+0000 7f470e5ef640 1 Processor -- start 2024-05-10T10:36:00.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.486+0000 7f470e5ef640 1 -- start start 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.486+0000 7f470e5ef640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f470815f410 0x7f47080d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.486+0000 7f470e5ef640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080b56a0 0x7f47080d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.486+0000 7f470e5ef640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47080d2800 con 0x7f47080d0f10 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.486+0000 7f470e5ef640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f47080d2ae0 con 0x7f470815f410 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.486+0000 7f470e5ef640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f47080d01d0 con 0x7f47080b56a0 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f4707fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f470815f410 0x7f47080d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f4707fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f470815f410 0x7f47080d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56748/0 (socket says 172.21.15.53:56748) 2024-05-10T10:36:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f4707fff640 1 -- 172.21.15.53:0/3016990547 learned_addr learned my addr 172.21.15.53:0/3016990547 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f470cb8c640 1 --2- 172.21.15.53:0/3016990547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080b56a0 0x7f47080d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f4707fff640 1 -- 172.21.15.53:0/3016990547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080b56a0 msgr2=0x7f47080d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f4707fff640 1 --2- 172.21.15.53:0/3016990547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080b56a0 0x7f47080d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f4707fff640 1 -- 172.21.15.53:0/3016990547 >> v1:172.21.15.135:6789/0 conn(0x7f47080d0f10 legacy=0x7f47080b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f4707fff640 1 -- 172.21.15.53:0/3016990547 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47080d04b0 con 0x7f470815f410 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.487+0000 7f470cb8c640 1 --2- 172.21.15.53:0/3016990547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080b56a0 0x7f47080d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f4707fff640 1 --2- 172.21.15.53:0/3016990547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080d09d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f76b05df4a71f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f4706ffd640 1 -- 172.21.15.53:0/3016990547 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8037070 con 0x7f470815f410 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f4706ffd640 1 -- 172.21.15.53:0/3016990547 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f8032440 con 0x7f470815f410 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f4706ffd640 1 -- 172.21.15.53:0/3016990547 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8032810 con 0x7f470815f410 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 -- 172.21.15.53:0/3016990547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 msgr2=0x7f47080d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 --2- 172.21.15.53:0/3016990547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080d09d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802a180 comp rx=0 tx=0).stop 2024-05-10T10:36:00.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 -- 172.21.15.53:0/3016990547 shutdown_connections 2024-05-10T10:36:00.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 --2- 172.21.15.53:0/3016990547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080b56a0 0x7f47080d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 --2- 172.21.15.53:0/3016990547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080d09d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 -- 172.21.15.53:0/3016990547 >> 172.21.15.53:0/3016990547 conn(0x7f470808ed30 msgr2=0x7f470809e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 -- 172.21.15.53:0/3016990547 shutdown_connections 2024-05-10T10:36:00.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.488+0000 7f470e5ef640 1 -- 172.21.15.53:0/3016990547 wait complete. 2024-05-10T10:36:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 Processor -- start 2024-05-10T10:36:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 -- start start 2024-05-10T10:36:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080d0f10 0x7f4708158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080b8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080ba9b0 0x7f47080c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f47080cffb0 con 0x7f470815f410 2024-05-10T10:36:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47080cfe30 con 0x7f47080ba9b0 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.490+0000 7f470e5ef640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f47080d3c10 con 0x7f47080d0f10 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080b8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56750/0 (socket says 172.21.15.53:56750) 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 -- 172.21.15.53:0/2701621778 learned_addr learned my addr 172.21.15.53:0/2701621778 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f4707fff640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080d0f10 0x7f4708158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f470cb8c640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080ba9b0 0x7f47080c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 -- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080d0f10 msgr2=0x7f4708158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080d0f10 0x7f4708158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 -- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080ba9b0 msgr2=0x7f47080c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080ba9b0 0x7f47080c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 -- 172.21.15.53:0/2701621778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47080ca090 con 0x7f470815f410 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f4707fff640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080d0f10 0x7f4708158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f470cb8c640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080ba9b0 0x7f47080c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f47077fe640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080b8150 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00c430 tx=0x7f46fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f4704ff9640 1 -- 172.21.15.53:0/2701621778 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc011790 con 0x7f470815f410 2024-05-10T10:36:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f4704ff9640 1 -- 172.21.15.53:0/2701621778 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc017070 con 0x7f470815f410 2024-05-10T10:36:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47080ca370 con 0x7f470815f410 2024-05-10T10:36:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f4704ff9640 1 -- 172.21.15.53:0/2701621778 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc011bb0 con 0x7f470815f410 2024-05-10T10:36:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.491+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47080bae60 con 0x7f470815f410 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.493+0000 7f4704ff9640 1 -- 172.21.15.53:0/2701621778 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f46fc01c460 con 0x7f470815f410 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.494+0000 7f4704ff9640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46f40762d0 0x7f46f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.494+0000 7f4704ff9640 1 -- 172.21.15.53:0/2701621778 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f46fc097600 con 0x7f470815f410 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.494+0000 7f4707fff640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46f40762d0 0x7f46f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.494+0000 7f47113df740 1 Processor -- start 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f47113df740 1 -- start start 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f4707fff640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46f40762d0 0x7f46f4078790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f46f8034000 tx=0x7f46f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x55809212eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f47113df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215eb40 0x558092133950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f47113df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5580920fb690 con 0x55809215eb40 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558092098f30 con 0x55809215f280 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558092051b20 con 0x55809217c160 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f470cb8c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215eb40 0x558092133950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f470cb8c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215eb40 0x558092133950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56754/0 (socket says 172.21.15.53:56754) 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.495+0000 7f470cb8c640 1 -- 172.21.15.53:0/4011149771 learned_addr learned my addr 172.21.15.53:0/4011149771 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f47077fe640 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f4707fff640 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x55809212eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f470cb8c640 1 -- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 msgr2=0x55809212eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f470cb8c640 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x55809212eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f470cb8c640 1 -- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809215f280 msgr2=0x55809215e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f470cb8c640 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f470cb8c640 1 -- 172.21.15.53:0/4011149771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558092049e60 con 0x55809215eb40 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f4707fff640 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x55809212eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f470cb8c640 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215eb40 0x558092133950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f470000d4e0 tx=0x7f470000d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7f105c2c1b8727 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f46edffb640 1 -- 172.21.15.53:0/4011149771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470001a070 con 0x55809215eb40 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f46edffb640 1 -- 172.21.15.53:0/4011149771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470000b7c0 con 0x55809215eb40 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f46edffb640 1 -- 172.21.15.53:0/4011149771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470000bb90 con 0x55809215eb40 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f47113df740 1 -- 172.21.15.53:0/4011149771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215eb40 msgr2=0x558092133950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.496+0000 7f47113df740 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215eb40 0x558092133950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f470000d4e0 tx=0x7f470000d9a0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 -- 172.21.15.53:0/4011149771 shutdown_connections 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215eb40 0x558092133950 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 --2- 172.21.15.53:0/4011149771 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x55809212eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 -- 172.21.15.53:0/4011149771 >> 172.21.15.53:0/4011149771 conn(0x558092130d60 msgr2=0x558092179600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 -- 172.21.15.53:0/4011149771 shutdown_connections 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 -- 172.21.15.53:0/4011149771 wait complete. 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 -- start start 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.497+0000 7f47113df740 1 Processor -- start 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x55809214bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47113df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809217c160 0x558092161920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x55809217b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47113df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5580920555e0 con 0x55809217c160 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558092094ef0 con 0x558092160180 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558092132f10 con 0x55809215f280 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47113df740 10 client.0 _ll_register_callbacks cb 0x558092050320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809217c160 0x558092161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809217c160 0x558092161920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56756/0 (socket says 172.21.15.53:56756) 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f47077fe640 1 -- 172.21.15.53:0/1717453255 learned_addr learned my addr 172.21.15.53:0/1717453255 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f4707fff640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x55809214bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.498+0000 7f470cb8c640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x55809217b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47077fe640 1 -- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 msgr2=0x55809214bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47077fe640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x55809214bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47077fe640 1 -- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 msgr2=0x55809217b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47077fe640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x55809217b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47077fe640 1 -- 172.21.15.53:0/1717453255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558091e747d0 con 0x55809217c160 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f470cb8c640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x55809217b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47077fe640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809217c160 0x558092161920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f46fc060670 tx=0x7f46fc060330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d2abab14c0d39c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f46e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f46e6ffd640 1 -- 172.21.15.53:0/1717453255 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc05fb40 con 0x55809217c160 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f46e6ffd640 1 -- 172.21.15.53:0/1717453255 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f46fc05c050 con 0x55809217c160 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470000b7c0 con 0x55809217c160 2024-05-10T10:36:00.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f46e6ffd640 1 -- 172.21.15.53:0/1717453255 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f46fc064a90 con 0x55809217c160 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 10 client.5823 fetch_fsmap learned FSMap version 34 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x558091daa3f0 con 0x55809217c160 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f46e6ffd640 1 -- 172.21.15.53:0/1717453255 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470000b7c0 con 0x55809217c160 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f46e6ffd640 1 -- 172.21.15.53:0/1717453255 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f46fc05c250 con 0x55809217c160 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 10 client.5823 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 10 client.5823 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 20 client.5823 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809214cb00 0x55809215d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 4 client.5823 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.499+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580920a6b40 con 0x55809214cb00 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.500+0000 7f4707fff640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809214cb00 0x55809215d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.500+0000 7f4707fff640 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809214cb00 0x55809215d610 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04df4829aaadb61 server_cookie=b248c2edc4cae6dd in_seq=0 out_seq=0 2024-05-10T10:36:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.500+0000 7f46e6ffd640 10 client.5823 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.500+0000 7f46e6ffd640 1 -- 172.21.15.53:0/1717453255 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580920a6b40 con 0x55809214cb00 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.500+0000 7f46e6ffd640 10 client.5823 handle_command_reply: tid=0 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.500+0000 7f47113df740 1 client.5823 shutdown 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809217c160 msgr2=0x558092161920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809217c160 0x558092161920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f46fc060670 tx=0x7f46fc060330 comp rx=0 tx=0).stop 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 shutdown_connections 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809214cb00 0x55809215d610 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x55809217b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f46e6ffd640 0 client.5823 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809217c160 0x558092161920 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 --2- 172.21.15.53:0/1717453255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x55809214bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 >> 172.21.15.53:0/1717453255 conn(0x55809215eb40 msgr2=0x5580921326a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 shutdown_connections 2024-05-10T10:36:00.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 1 -- 172.21.15.53:0/1717453255 wait complete. 2024-05-10T10:36:00.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.501+0000 7f47113df740 20 client.5823 trim_cache size 0 max 16384 2024-05-10T10:36:00.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.519+0000 7f47113df740 1 Processor -- start 2024-05-10T10:36:00.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.519+0000 7f47113df740 1 -- start start 2024-05-10T10:36:00.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x558092161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47113df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558092160180 0x558092132300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809217c160 0x558092132840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47113df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f47081f9ae0 con 0x558092160180 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f470815bda0 con 0x55809217c160 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f470815b130 con 0x55809215f280 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558092160180 0x558092132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558092160180 0x558092132300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56768/0 (socket says 172.21.15.53:56768) 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 -- 172.21.15.53:0/3361996526 learned_addr learned my addr 172.21.15.53:0/3361996526 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f4707fff640 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x558092161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f470cb8c640 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809217c160 0x558092132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 -- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 msgr2=0x558092161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x558092161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 -- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809217c160 msgr2=0x558092132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809217c160 0x558092132840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 -- 172.21.15.53:0/3361996526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558091d6a0f0 con 0x558092160180 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f470cb8c640 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809217c160 0x558092132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f4707fff640 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x558092161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f47077fe640 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558092160180 0x558092132300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0603e0 tx=0x7f46fc097050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82759981638db3a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f46ed7fa640 1 -- 172.21.15.53:0/3361996526 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc069500 con 0x558092160180 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f46ed7fa640 1 -- 172.21.15.53:0/3361996526 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc080590 con 0x558092160180 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.520+0000 7f46ed7fa640 1 -- 172.21.15.53:0/3361996526 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc05cd30 con 0x558092160180 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 -- 172.21.15.53:0/3361996526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558092160180 msgr2=0x558092132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558092160180 0x558092132300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0603e0 tx=0x7f46fc097050 comp rx=0 tx=0).stop 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 -- 172.21.15.53:0/3361996526 shutdown_connections 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55809217c160 0x558092132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558092160180 0x558092132300 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 --2- 172.21.15.53:0/3361996526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809215f280 0x558092161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 -- 172.21.15.53:0/3361996526 >> 172.21.15.53:0/3361996526 conn(0x55809215eb40 msgr2=0x5580921625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 -- 172.21.15.53:0/3361996526 shutdown_connections 2024-05-10T10:36:00.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.521+0000 7f47113df740 1 -- 172.21.15.53:0/3361996526 wait complete. 2024-05-10T10:36:00.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.522+0000 7f47113df740 1 -- start start 2024-05-10T10:36:00.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.522+0000 7f47113df740 1 Processor -- start 2024-05-10T10:36:00.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x558092161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x558092162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558092159ff0 con 0x55809215f280 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f470815b750 con 0x558092160180 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558092132ed0 con 0x55809217c160 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 10 client.0 _ll_register_callbacks cb 0x7f46fc05c5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56778/0 (socket says 172.21.15.53:56778) 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 -- 172.21.15.53:0/2056376984 learned_addr learned my addr 172.21.15.53:0/2056376984 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47077fe640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x558092161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f470cb8c640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x558092162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 -- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 msgr2=0x558092162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x558092162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 -- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 msgr2=0x558092161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x558092161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 -- 172.21.15.53:0/2056376984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558092132c60 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47077fe640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x558092161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f4707fff640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215f280 0x55809215e600 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009630 tx=0x7f46f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5460ef9dfea47ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f46e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f46e6ffd640 1 -- 172.21.15.53:0/2056376984 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8037040 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f46e6ffd640 1 -- 172.21.15.53:0/2056376984 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f46f8004240 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46fc080590 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f46e6ffd640 1 -- 172.21.15.53:0/2056376984 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f46f805b050 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 10 client.5829 fetch_fsmap learned FSMap version 34 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.523+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5580921f9c10 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f46e6ffd640 1 -- 172.21.15.53:0/2056376984 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc080590 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f46e6ffd640 1 -- 172.21.15.53:0/2056376984 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f46f805e020 con 0x55809215f280 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 10 client.5829 fetch_fsmap finished waiting for FSMap version 34 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 10 client.5829 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 20 client.5829 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809212dfd0 0x55809219bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 4 client.5829 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5580921707d0 con 0x55809212dfd0 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47077fe640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809212dfd0 0x55809219bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47077fe640 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809212dfd0 0x55809219bca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91f0ebc47873102 server_cookie=245c3e1e3673f744 in_seq=0 out_seq=0 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f46e6ffd640 10 client.5829 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f46e6ffd640 1 -- 172.21.15.53:0/2056376984 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x5580921707d0 con 0x55809212dfd0 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f46e6ffd640 10 client.5829 handle_command_reply: tid=0 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 1 client.5829 shutdown 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215f280 msgr2=0x55809215e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215f280 0x55809215e600 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009630 tx=0x7f46f80046c0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.524+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 shutdown_connections 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55809212dfd0 0x55809219bca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55809217c160 0x558092162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f46e6ffd640 0 client.5829 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558092160180 0x558092161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 1 --2- 172.21.15.53:0/2056376984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55809215f280 0x55809215e600 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 >> 172.21.15.53:0/2056376984 conn(0x55809215eb40 msgr2=0x55809215dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 shutdown_connections 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 1 -- 172.21.15.53:0/2056376984 wait complete. 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f47113df740 20 client.5829 trim_cache size 0 max 16384 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:36:00.532 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:36:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "5085360d-21b0-49c5-8ac2-915409e451b2": { 2024-05-10T10:36:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2024-05-10T10:36:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "5085360d-21b0-49c5-8ac2-915409e451b2", 2024-05-10T10:36:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive" 2024-05-10T10:36:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:36:00.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:36:00.533 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46f40762d0 msgr2=0x7f46f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46f40762d0 0x7f46f4078790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f46f8034000 tx=0x7f46f802e000 comp rx=0 tx=0).stop 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 msgr2=0x7f47080b8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080b8150 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00c430 tx=0x7f46fc00c8f0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 shutdown_connections 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f46f40762d0 0x7f46f4078790 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080ba9b0 0x7f47080c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f470815f410 0x7f47080b8150 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 --2- 172.21.15.53:0/2701621778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f47080d0f10 0x7f4708158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:00.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 >> 172.21.15.53:0/2701621778 conn(0x7f470808ed30 msgr2=0x7f47080d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:00.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 shutdown_connections 2024-05-10T10:36:00.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:00.525+0000 7f470e5ef640 1 -- 172.21.15.53:0/2701621778 wait complete. 2024-05-10T10:36:00.537 INFO:tasks.cephfs.filesystem:scrub status for tag:5085360d-21b0-49c5-8ac2-915409e451b2 - {'path': '/', 'tag': '5085360d-21b0-49c5-8ac2-915409e451b2', 'options': 'recursive'} 2024-05-10T10:36:04.415 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:36:04.414+0000 7f39f976c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-10T10:36:14.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:36:14.072 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:36:30.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:30.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.656+0000 7f89bffda640 1 Processor -- start 2024-05-10T10:36:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.656+0000 7f89bffda640 1 -- start start 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bffda640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89b80cf3e0 0x7f89b80cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bffda640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f89b80d2190 0x7f89b80cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bffda640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b806e720 con 0x7f89b80cd500 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bffda640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f89b806de80 con 0x7f89b80cf3e0 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bffda640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f89b806e160 con 0x7f89b80d2190 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bdd76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89b80cf3e0 0x7f89b80cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bdd76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89b80cf3e0 0x7f89b80cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40266/0 (socket says 172.21.15.53:40266) 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.657+0000 7f89bdd76640 1 -- 172.21.15.53:0/3141462795 learned_addr learned my addr 172.21.15.53:0/3141462795 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.658+0000 7f89be577640 1 --2- 172.21.15.53:0/3141462795 >> v2:172.21.15.135:3300/0 conn(0x7f89b80d2190 0x7f89b80cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.658+0000 7f89bdd76640 1 -- 172.21.15.53:0/3141462795 >> v2:172.21.15.135:3300/0 conn(0x7f89b80d2190 msgr2=0x7f89b80cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.658+0000 7f89bdd76640 1 --2- 172.21.15.53:0/3141462795 >> v2:172.21.15.135:3300/0 conn(0x7f89b80d2190 0x7f89b80cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.658+0000 7f89bdd76640 1 -- 172.21.15.53:0/3141462795 >> v1:172.21.15.135:6789/0 conn(0x7f89b80cd500 legacy=0x7f89b80b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.658+0000 7f89bdd76640 1 -- 172.21.15.53:0/3141462795 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b806e440 con 0x7f89b80cf3e0 2024-05-10T10:36:30.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.658+0000 7f89be577640 1 --2- 172.21.15.53:0/3141462795 >> v2:172.21.15.135:3300/0 conn(0x7f89b80d2190 0x7f89b80cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.658+0000 7f89bdd76640 1 --2- 172.21.15.53:0/3141462795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b80cf8c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89b8073e80 tx=0x7f89b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63d700f5e1dbf3a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bcd74640 1 -- 172.21.15.53:0/3141462795 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4037070 con 0x7f89b80cf3e0 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bcd74640 1 -- 172.21.15.53:0/3141462795 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b4032440 con 0x7f89b80cf3e0 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bcd74640 1 -- 172.21.15.53:0/3141462795 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4032810 con 0x7f89b80cf3e0 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 -- 172.21.15.53:0/3141462795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 msgr2=0x7f89b80cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 --2- 172.21.15.53:0/3141462795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b80cf8c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89b8073e80 tx=0x7f89b402a230 comp rx=0 tx=0).stop 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 -- 172.21.15.53:0/3141462795 shutdown_connections 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 --2- 172.21.15.53:0/3141462795 >> v2:172.21.15.135:3300/0 conn(0x7f89b80d2190 0x7f89b80cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 --2- 172.21.15.53:0/3141462795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b80cf8c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 -- 172.21.15.53:0/3141462795 >> 172.21.15.53:0/3141462795 conn(0x7f89b80d2eb0 msgr2=0x7f89b80d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 -- 172.21.15.53:0/3141462795 shutdown_connections 2024-05-10T10:36:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.659+0000 7f89bffda640 1 -- 172.21.15.53:0/3141462795 wait complete. 2024-05-10T10:36:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.660+0000 7f89bffda640 1 Processor -- start 2024-05-10T10:36:30.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bffda640 1 -- start start 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bffda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89b80cd500 0x7f89b815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bffda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bffda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b80d2190 0x7f89b815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bffda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89b80b34e0 con 0x7f89b80cf3e0 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bffda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89b80b4510 con 0x7f89b80d2190 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bffda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f89b80b3660 con 0x7f89b80cd500 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bd575640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bd575640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40268/0 (socket says 172.21.15.53:40268) 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.661+0000 7f89bd575640 1 -- 172.21.15.53:0/458594264 learned_addr learned my addr 172.21.15.53:0/458594264 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bdd76640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89b80cd500 0x7f89b815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89be577640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b80d2190 0x7f89b815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bd575640 1 -- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89b80cd500 msgr2=0x7f89b815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bd575640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89b80cd500 0x7f89b815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bd575640 1 -- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b80d2190 msgr2=0x7f89b815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bd575640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b80d2190 0x7f89b815f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bd575640 1 -- 172.21.15.53:0/458594264 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b80c7220 con 0x7f89b80cf3e0 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bd575640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b815efd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c400 tx=0x7f89a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89ae7fc640 1 -- 172.21.15.53:0/458594264 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8011790 con 0x7f89b80cf3e0 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89ae7fc640 1 -- 172.21.15.53:0/458594264 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a8017070 con 0x7f89b80cf3e0 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89ae7fc640 1 -- 172.21.15.53:0/458594264 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8011bb0 con 0x7f89b80cf3e0 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b80c7500 con 0x7f89b80cf3e0 2024-05-10T10:36:30.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.662+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b8159880 con 0x7f89b80cf3e0 2024-05-10T10:36:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.664+0000 7f89ae7fc640 1 -- 172.21.15.53:0/458594264 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f89a801c460 con 0x7f89b80cf3e0 2024-05-10T10:36:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.665+0000 7f89ae7fc640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f899c0762d0 0x7f899c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.665+0000 7f89ae7fc640 1 -- 172.21.15.53:0/458594264 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f89a80975e0 con 0x7f89b80cf3e0 2024-05-10T10:36:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.665+0000 7f89bdd76640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f899c0762d0 0x7f899c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.666+0000 7f89c2dcf740 1 Processor -- start 2024-05-10T10:36:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.666+0000 7f89c2dcf740 1 -- start start 2024-05-10T10:36:30.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.666+0000 7f89bdd76640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f899c0762d0 0x7f899c078790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f89b4034000 tx=0x7f89b402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d100baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d103b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d103bb40 0x5647d1010950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5647d0fd8690 con 0x5647d103c280 2024-05-10T10:36:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647d0f75f30 con 0x5647d1059160 2024-05-10T10:36:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5647d0f2eb20 con 0x5647d103bb40 2024-05-10T10:36:30.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d103b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d103b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40284/0 (socket says 172.21.15.53:40284) 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 -- 172.21.15.53:0/1465504268 learned_addr learned my addr 172.21.15.53:0/1465504268 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89be577640 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d103bb40 0x5647d1010950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bdd76640 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d100baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 -- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d103bb40 msgr2=0x5647d1010950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d103bb40 0x5647d1010950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 -- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 msgr2=0x5647d100baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d100baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 -- 172.21.15.53:0/1465504268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d0f26e60 con 0x5647d103c280 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f89bd575640 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d103b600 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89a80605c0 tx=0x7f89a8060140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d276d27924102a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f899b7fe640 1 -- 172.21.15.53:0/1465504268 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8064930 con 0x5647d103c280 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.667+0000 7f899b7fe640 1 -- 172.21.15.53:0/1465504268 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a8069350 con 0x5647d103c280 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f899b7fe640 1 -- 172.21.15.53:0/1465504268 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a805f890 con 0x5647d103c280 2024-05-10T10:36:30.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1465504268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 msgr2=0x5647d103b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d103b600 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89a80605c0 tx=0x7f89a8060140 comp rx=0 tx=0).stop 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1465504268 shutdown_connections 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d103bb40 0x5647d1010950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d103b600 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1465504268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d100baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1465504268 >> 172.21.15.53:0/1465504268 conn(0x5647d100dd60 msgr2=0x5647d1056600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1465504268 shutdown_connections 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.668+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1465504268 wait complete. 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.669+0000 7f89c2dcf740 1 -- start start 2024-05-10T10:36:30.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.669+0000 7f89c2dcf740 1 Processor -- start 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d103c280 0x5647d106a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1037080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d1058690 0x5647d103c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5647d0f325e0 con 0x5647d1058690 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647d0f71ef0 con 0x5647d103c280 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5647d100ff10 con 0x5647d1059160 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89c2dcf740 10 client.0 _ll_register_callbacks cb 0x5647d0f2d320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:30.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d1058690 0x5647d103c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d1058690 0x5647d103c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40300/0 (socket says 172.21.15.53:40300) 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 -- 172.21.15.53:0/1669597967 learned_addr learned my addr 172.21.15.53:0/1669597967 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89bdd76640 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d103c280 0x5647d106a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89bd575640 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1037080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 -- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 msgr2=0x5647d1037080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1037080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 -- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d103c280 msgr2=0x5647d106a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d103c280 0x5647d106a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.670+0000 7f89be577640 1 -- 172.21.15.53:0/1669597967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d0d517d0 con 0x5647d1058690 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89be577640 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d1058690 0x5647d103c770 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f89b000fbf0 tx=0x7f89b000d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3e13b30e09089c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f8998ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f8998ff9640 1 -- 172.21.15.53:0/1669597967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0003990 con 0x5647d1058690 2024-05-10T10:36:30.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f8998ff9640 1 -- 172.21.15.53:0/1669597967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f89b0003b30 con 0x5647d1058690 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a8069350 con 0x5647d1058690 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f8998ff9640 1 -- 172.21.15.53:0/1669597967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f89b0003d30 con 0x5647d1058690 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 10 client.5841 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5647d0c873f0 con 0x5647d1058690 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f8998ff9640 1 -- 172.21.15.53:0/1669597967 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a8069350 con 0x5647d1058690 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f8998ff9640 1 -- 172.21.15.53:0/1669597967 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f89b0025080 con 0x5647d1058690 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 10 client.5841 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 10 client.5841 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 20 client.5841 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d103ccb0 0x5647d103add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 4 client.5841 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647d0f83b40 con 0x5647d103ccb0 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.671+0000 7f89bdd76640 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d103ccb0 0x5647d103add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.672+0000 7f8998ff9640 10 client.5841 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.672+0000 7f89bdd76640 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d103ccb0 0x5647d103add0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a5e7e23fa1b8fb server_cookie=2f8fd7d75b085ee1 in_seq=0 out_seq=0 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.672+0000 7f8998ff9640 1 -- 172.21.15.53:0/1669597967 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647d0f83b40 con 0x5647d103ccb0 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.672+0000 7f8998ff9640 10 client.5841 handle_command_reply: tid=0 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.672+0000 7f89c2dcf740 1 client.5841 shutdown 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.672+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d1058690 msgr2=0x5647d103c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.672+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d1058690 0x5647d103c770 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f89b000fbf0 tx=0x7f89b000d740 comp rx=0 tx=0).stop 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 shutdown_connections 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d103ccb0 0x5647d103add0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d1058690 0x5647d103c770 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1037080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f8998ff9640 0 client.5841 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/1669597967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d103c280 0x5647d106a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 >> 172.21.15.53:0/1669597967 conn(0x5647d103bb40 msgr2=0x5647d100b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 shutdown_connections 2024-05-10T10:36:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 1 -- 172.21.15.53:0/1669597967 wait complete. 2024-05-10T10:36:30.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.673+0000 7f89c2dcf740 20 client.5841 trim_cache size 0 max 16384 2024-05-10T10:36:30.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.691+0000 7f89c2dcf740 1 Processor -- start 2024-05-10T10:36:30.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.691+0000 7f89c2dcf740 1 -- start start 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d107ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1058690 0x5647d103abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89b82020a0 con 0x5647d103c280 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8201080 con 0x5647d1058690 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f89b814c930 con 0x5647d1059160 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d107ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d107ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40308/0 (socket says 172.21.15.53:40308) 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 -- 172.21.15.53:0/2447898790 learned_addr learned my addr 172.21.15.53:0/2447898790 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bd575640 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1058690 0x5647d103abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89be577640 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 -- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 msgr2=0x5647d1069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 -- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1058690 msgr2=0x5647d103abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1058690 0x5647d103abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 -- 172.21.15.53:0/2447898790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d0c470f0 con 0x5647d103c280 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f89bdd76640 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d107ace0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f89b40090b0 tx=0x7f89b402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5571f8c785415e69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f899affd640 1 -- 172.21.15.53:0/2447898790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4037040 con 0x5647d103c280 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f899affd640 1 -- 172.21.15.53:0/2447898790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b403c700 con 0x5647d103c280 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.692+0000 7f899affd640 1 -- 172.21.15.53:0/2447898790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b403cab0 con 0x5647d103c280 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 -- 172.21.15.53:0/2447898790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 msgr2=0x5647d107ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d107ace0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f89b40090b0 tx=0x7f89b402c000 comp rx=0 tx=0).stop 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 -- 172.21.15.53:0/2447898790 shutdown_connections 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1058690 0x5647d103abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/2447898790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d107ace0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 -- 172.21.15.53:0/2447898790 >> 172.21.15.53:0/2447898790 conn(0x5647d103bb40 msgr2=0x5647d1059a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 -- 172.21.15.53:0/2447898790 shutdown_connections 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.693+0000 7f89c2dcf740 1 -- 172.21.15.53:0/2447898790 wait complete. 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.694+0000 7f89c2dcf740 1 -- start start 2024-05-10T10:36:30.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.694+0000 7f89c2dcf740 1 Processor -- start 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d10530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1058690 0x5647d1056e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5647d1039100 con 0x5647d103c280 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8200a20 con 0x5647d1059160 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5647d1038f80 con 0x5647d1058690 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 10 client.0 _ll_register_callbacks cb 0x7f89b0007a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d10530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d10530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40324/0 (socket says 172.21.15.53:40324) 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 -- 172.21.15.53:0/778954092 learned_addr learned my addr 172.21.15.53:0/778954092 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bd575640 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1058690 0x5647d1056e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89be577640 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 -- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1058690 msgr2=0x5647d1056e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1058690 0x5647d1056e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 -- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 msgr2=0x5647d1069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 -- 172.21.15.53:0/778954092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d103d2c0 con 0x5647d103c280 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89bdd76640 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d10530c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009960 tx=0x7f89b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c054763550eaa017 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f8998ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f8998ff9640 1 -- 172.21.15.53:0/778954092 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40042b0 con 0x5647d103c280 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f8998ff9640 1 -- 172.21.15.53:0/778954092 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f89b4004450 con 0x5647d103c280 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.695+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b403c700 con 0x5647d103c280 2024-05-10T10:36:30.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f8998ff9640 1 -- 172.21.15.53:0/778954092 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f89b403c8b0 con 0x5647d103c280 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 10 client.5847 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5647d10360c0 con 0x5647d103c280 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f8998ff9640 1 -- 172.21.15.53:0/778954092 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b403c700 con 0x5647d103c280 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f8998ff9640 1 -- 172.21.15.53:0/778954092 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f89b403ca50 con 0x5647d103c280 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 10 client.5847 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 10 client.5847 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 20 client.5847 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d1057360 0x5647d106a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 4 client.5847 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:36:30.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647d0f7d9b0 con 0x5647d1057360 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89bd575640 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d1057360 0x5647d106a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f8998ff9640 10 client.5847 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89bd575640 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d1057360 0x5647d106a270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49126eb58dfc3284 server_cookie=95a70e42be98cd6b in_seq=0 out_seq=0 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f8998ff9640 1 -- 172.21.15.53:0/778954092 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5647d0f7d9b0 con 0x5647d1057360 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f8998ff9640 10 client.5847 handle_command_reply: tid=0 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.696+0000 7f89c2dcf740 1 client.5847 shutdown 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 msgr2=0x5647d10530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d10530c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009960 tx=0x7f89b4004730 comp rx=0 tx=0).stop 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 shutdown_connections 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x5647d1057360 0x5647d106a270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d1059160 0x5647d1069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f8998ff9640 0 client.5847 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:30.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5647d1058690 0x5647d1056e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 --2- 172.21.15.53:0/778954092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5647d103c280 0x5647d10530c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 >> 172.21.15.53:0/778954092 conn(0x5647d103bb40 msgr2=0x5647d1011080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 shutdown_connections 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 1 -- 172.21.15.53:0/778954092 wait complete. 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89c2dcf740 20 client.5847 trim_cache size 0 max 16384 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:36:30.704 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:36:30.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f899c0762d0 msgr2=0x7f899c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f899c0762d0 0x7f899c078790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f89b4034000 tx=0x7f89b402e000 comp rx=0 tx=0).stop 2024-05-10T10:36:30.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 msgr2=0x7f89b815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b815efd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c400 tx=0x7f89a800c8c0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 shutdown_connections 2024-05-10T10:36:30.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f899c0762d0 0x7f899c078790 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b80d2190 0x7f89b815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89b80cf3e0 0x7f89b815efd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.697+0000 7f89bffda640 1 --2- 172.21.15.53:0/458594264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89b80cd500 0x7f89b815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.698+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 >> 172.21.15.53:0/458594264 conn(0x7f89b80d2eb0 msgr2=0x7f89b80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:30.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.698+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 shutdown_connections 2024-05-10T10:36:30.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.698+0000 7f89bffda640 1 -- 172.21.15.53:0/458594264 wait complete. 2024-05-10T10:36:30.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:36:30.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:36:30.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.812+0000 7f489ef71640 1 Processor -- start 2024-05-10T10:36:30.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.812+0000 7f489ef71640 1 -- start start 2024-05-10T10:36:30.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489ef71640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48980d5cb0 0x7f48980d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489ef71640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4898162b30 0x7f48980d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489ef71640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48980b45f0 con 0x7f4898162750 2024-05-10T10:36:30.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489ef71640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f48980b4f20 con 0x7f48980d5cb0 2024-05-10T10:36:30.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489ef71640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f48980d0fe0 con 0x7f4898162b30 2024-05-10T10:36:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489d50e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4898162b30 0x7f48980d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f4897fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48980d5cb0 0x7f48980d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489cd0d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4898162750 0x7f48980d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:53738/0 (socket says 172.21.15.53:53738) 2024-05-10T10:36:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.813+0000 7f489cd0d640 1 -- 172.21.15.53:0/346137246 learned_addr learned my addr 172.21.15.53:0/346137246 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:36:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f48977fe640 1 -- 172.21.15.53:0/346137246 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3532995513 0 0) 0x7f48980b45f0 con 0x7f4898162750 2024-05-10T10:36:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f48977fe640 1 -- 172.21.15.53:0/346137246 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4880003520 con 0x7f4898162750 2024-05-10T10:36:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f489d50e640 1 -- 172.21.15.53:0/346137246 >> v2:172.21.15.135:3300/0 conn(0x7f48980d5cb0 msgr2=0x7f48980d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f489d50e640 1 --2- 172.21.15.53:0/346137246 >> v2:172.21.15.135:3300/0 conn(0x7f48980d5cb0 0x7f48980d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f489d50e640 1 -- 172.21.15.53:0/346137246 >> v1:172.21.15.135:6789/0 conn(0x7f4898162750 legacy=0x7f48980d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f489d50e640 1 -- 172.21.15.53:0/346137246 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48980b48d0 con 0x7f4898162b30 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f4897fff640 1 --2- 172.21.15.53:0/346137246 >> v2:172.21.15.135:3300/0 conn(0x7f48980d5cb0 0x7f48980d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.814+0000 7f489d50e640 1 --2- 172.21.15.53:0/346137246 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4898162b30 0x7f48980d3b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48900097a0 tx=0x7f489002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd9037a19851c51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f48977fe640 1 -- 172.21.15.53:0/346137246 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890037070 con 0x7f4898162b30 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f48977fe640 1 -- 172.21.15.53:0/346137246 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489002ae50 con 0x7f4898162b30 2024-05-10T10:36:30.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f48977fe640 1 -- 172.21.15.53:0/346137246 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890032660 con 0x7f4898162b30 2024-05-10T10:36:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 -- 172.21.15.53:0/346137246 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4898162b30 msgr2=0x7f48980d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 --2- 172.21.15.53:0/346137246 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4898162b30 0x7f48980d3b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48900097a0 tx=0x7f489002a100 comp rx=0 tx=0).stop 2024-05-10T10:36:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 -- 172.21.15.53:0/346137246 shutdown_connections 2024-05-10T10:36:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 --2- 172.21.15.53:0/346137246 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4898162b30 0x7f48980d3b70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 --2- 172.21.15.53:0/346137246 >> v2:172.21.15.135:3300/0 conn(0x7f48980d5cb0 0x7f48980d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 -- 172.21.15.53:0/346137246 >> 172.21.15.53:0/346137246 conn(0x7f489809e3b0 msgr2=0x7f48980d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:30.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 -- 172.21.15.53:0/346137246 shutdown_connections 2024-05-10T10:36:30.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.815+0000 7f489ef71640 1 -- 172.21.15.53:0/346137246 wait complete. 2024-05-10T10:36:30.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.816+0000 7f489ef71640 1 Processor -- start 2024-05-10T10:36:30.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f489ef71640 1 -- start start 2024-05-10T10:36:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f489ef71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4898162750 0x7f48981601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f489ef71640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898162b30 0x7f48980b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f489ef71640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48980ba050 0x7f489814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f489ef71640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f48980b53b0 con 0x7f4898162b30 2024-05-10T10:36:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f489ef71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48980d0d80 con 0x7f4898162750 2024-05-10T10:36:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f489ef71640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f48980b5530 con 0x7f48980ba050 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.817+0000 7f4897fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898162b30 0x7f48980b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898162b30 0x7f48980b9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40340/0 (socket says 172.21.15.53:40340) 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 -- 172.21.15.53:0/1142302732 learned_addr learned my addr 172.21.15.53:0/1142302732 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f489d50e640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48980ba050 0x7f489814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f489cd0d640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4898162750 0x7f48981601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 -- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48980ba050 msgr2=0x7f489814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48980ba050 0x7f489814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 -- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4898162750 msgr2=0x7f48981601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4898162750 0x7f48981601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 -- 172.21.15.53:0/1142302732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48980b9420 con 0x7f4898162b30 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f489d50e640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48980ba050 0x7f489814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f489cd0d640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4898162750 0x7f48981601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f4897fff640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898162b30 0x7f48980b9b10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f488c00e980 tx=0x7f488c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f48957fa640 1 -- 172.21.15.53:0/1142302732 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c017070 con 0x7f4898162b30 2024-05-10T10:36:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f48957fa640 1 -- 172.21.15.53:0/1142302732 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488c00ca00 con 0x7f4898162b30 2024-05-10T10:36:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48980b8b80 con 0x7f4898162b30 2024-05-10T10:36:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f48957fa640 1 -- 172.21.15.53:0/1142302732 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c00cdf0 con 0x7f4898162b30 2024-05-10T10:36:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.818+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48980ccaf0 con 0x7f4898162b30 2024-05-10T10:36:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.819+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489806a220 con 0x7f4898162b30 2024-05-10T10:36:30.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.820+0000 7f48957fa640 1 -- 172.21.15.53:0/1142302732 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f488c01a070 con 0x7f4898162b30 2024-05-10T10:36:30.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.820+0000 7f48957fa640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f48840762d0 0x7f4884078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:30.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.821+0000 7f48957fa640 1 -- 172.21.15.53:0/1142302732 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f488c097370 con 0x7f4898162b30 2024-05-10T10:36:30.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.821+0000 7f489cd0d640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f48840762d0 0x7f4884078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:30.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.821+0000 7f489cd0d640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f48840762d0 0x7f4884078790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4888005c00 tx=0x7f4888009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:30.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:30.825+0000 7f48957fa640 1 -- 172.21.15.53:0/1142302732 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f488c060ab0 con 0x7f4898162b30 2024-05-10T10:36:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.056+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48980d5520 con 0x7f4898162b30 2024-05-10T10:36:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.057+0000 7f48957fa640 1 -- 172.21.15.53:0/1142302732 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5188 (secure 0 0 0) 0x7f488c0603b0 con 0x7f4898162b30 2024-05-10T10:36:31.058 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:36:31.058 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:36:04.407737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:36:31.058 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 35 2024-05-10T10:36:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f48840762d0 msgr2=0x7f4884078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f48840762d0 0x7f4884078790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4888005c00 tx=0x7f4888009040 comp rx=0 tx=0).stop 2024-05-10T10:36:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898162b30 msgr2=0x7f48980b9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898162b30 0x7f48980b9b10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f488c00e980 tx=0x7f488c00ee40 comp rx=0 tx=0).stop 2024-05-10T10:36:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 shutdown_connections 2024-05-10T10:36:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f48840762d0 0x7f4884078790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48980ba050 0x7f489814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898162b30 0x7f48980b9b10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 --2- 172.21.15.53:0/1142302732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4898162750 0x7f48981601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.059+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 >> 172.21.15.53:0/1142302732 conn(0x7f489809e3b0 msgr2=0x7f48980b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.060+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 shutdown_connections 2024-05-10T10:36:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.060+0000 7f489ef71640 1 -- 172.21.15.53:0/1142302732 wait complete. 2024-05-10T10:36:31.071 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-10T10:36:31.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.176+0000 7f1455e46640 1 Processor -- start 2024-05-10T10:36:31.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.176+0000 7f1455e46640 1 -- start start 2024-05-10T10:36:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f1455e46640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14500b4f90 0x7f14500b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f1455e46640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f14500d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f1455e46640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14500d2800 con 0x7f145015f410 2024-05-10T10:36:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f1455e46640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f14500d2ae0 con 0x7f14500b4f90 2024-05-10T10:36:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f1455e46640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f14500d01d0 con 0x7f14500d36d0 2024-05-10T10:36:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14500b4f90 0x7f14500b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f145015f410 0x7f14500d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58446/0 (socket says 172.21.15.53:58446) 2024-05-10T10:36:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144f7fe640 1 -- 172.21.15.53:0/3946977539 learned_addr learned my addr 172.21.15.53:0/3946977539 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:36:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144ffff640 1 --2- 172.21.15.53:0/3946977539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f14500d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144effd640 1 -- 172.21.15.53:0/3946977539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 msgr2=0x7f14500d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144effd640 1 --2- 172.21.15.53:0/3946977539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f14500d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144effd640 1 -- 172.21.15.53:0/3946977539 >> v1:172.21.15.53:6789/0 conn(0x7f145015f410 legacy=0x7f14500d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.177+0000 7f144effd640 1 -- 172.21.15.53:0/3946977539 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14500d04b0 con 0x7f14500b4f90 2024-05-10T10:36:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.178+0000 7f144effd640 1 --2- 172.21.15.53:0/3946977539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500b5410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f14440097a0 tx=0x7f144402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eb02c6705ab4e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.178+0000 7f144e7fc640 1 -- 172.21.15.53:0/3946977539 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444037070 con 0x7f14500b4f90 2024-05-10T10:36:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.178+0000 7f144e7fc640 1 -- 172.21.15.53:0/3946977539 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144402ae50 con 0x7f14500b4f90 2024-05-10T10:36:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.178+0000 7f144e7fc640 1 -- 172.21.15.53:0/3946977539 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444032660 con 0x7f14500b4f90 2024-05-10T10:36:31.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 -- 172.21.15.53:0/3946977539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 msgr2=0x7f14500b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 --2- 172.21.15.53:0/3946977539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500b5410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f14440097a0 tx=0x7f144402a100 comp rx=0 tx=0).stop 2024-05-10T10:36:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 -- 172.21.15.53:0/3946977539 shutdown_connections 2024-05-10T10:36:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 --2- 172.21.15.53:0/3946977539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f14500d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 --2- 172.21.15.53:0/3946977539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500b5410 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 -- 172.21.15.53:0/3946977539 >> 172.21.15.53:0/3946977539 conn(0x7f145008ed30 msgr2=0x7f145009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 -- 172.21.15.53:0/3946977539 shutdown_connections 2024-05-10T10:36:31.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.179+0000 7f1455e46640 1 -- 172.21.15.53:0/3946977539 wait complete. 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.180+0000 7f1455e46640 1 Processor -- start 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.180+0000 7f1455e46640 1 -- start start 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f1455e46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500ba340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f1455e46640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f1450161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f1455e46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f145015f410 0x7f14501623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f1455e46640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14500cffb0 con 0x7f14500b4f90 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f1455e46640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14500cfe30 con 0x7f145015f410 2024-05-10T10:36:31.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f1455e46640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f14500d10e0 con 0x7f14500d36d0 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500ba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40344/0 (socket says 172.21.15.53:40344) 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.181+0000 7f144f7fe640 1 -- 172.21.15.53:0/3786909331 learned_addr learned my addr 172.21.15.53:0/3786909331 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144effd640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f1450161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144ffff640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f145015f410 0x7f14501623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144f7fe640 1 -- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 msgr2=0x7f1450161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144f7fe640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f1450161cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144f7fe640 1 -- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f145015f410 msgr2=0x7f14501623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144f7fe640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f145015f410 0x7f14501623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144f7fe640 1 -- 172.21.15.53:0/3786909331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1450162910 con 0x7f14500b4f90 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f144f7fe640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500ba340 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f144000c3c0 tx=0x7f144000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f1437fff640 1 -- 172.21.15.53:0/3786909331 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440017070 con 0x7f14500b4f90 2024-05-10T10:36:31.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f1437fff640 1 -- 172.21.15.53:0/3786909331 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1440005ce0 con 0x7f14500b4f90 2024-05-10T10:36:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14500ba920 con 0x7f14500b4f90 2024-05-10T10:36:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f1437fff640 1 -- 172.21.15.53:0/3786909331 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400072f0 con 0x7f14500b4f90 2024-05-10T10:36:31.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.182+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145015fc70 con 0x7f14500b4f90 2024-05-10T10:36:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.184+0000 7f1437fff640 1 -- 172.21.15.53:0/3786909331 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f144001a070 con 0x7f14500b4f90 2024-05-10T10:36:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.184+0000 7f1437fff640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f143c0762d0 0x7f143c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.184+0000 7f1437fff640 1 -- 172.21.15.53:0/3786909331 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f1440096b50 con 0x7f14500b4f90 2024-05-10T10:36:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.185+0000 7f144effd640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f143c0762d0 0x7f143c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.185+0000 7f1458c34740 1 Processor -- start 2024-05-10T10:36:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.185+0000 7f144effd640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f143c0762d0 0x7f143c078790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1444034000 tx=0x7f144402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.185+0000 7f1458c34740 1 -- start start 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f1458c34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5fecf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feff7f0 0x563b5fefeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff0b0 0x563b5fed3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f1458c34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563b5fdbf920 con 0x563b5ff1c6d0 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b5fe34dd0 con 0x563b5feff0b0 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563b5fde53e0 con 0x563b5feff7f0 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5fecf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5fecf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40360/0 (socket says 172.21.15.53:40360) 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 -- 172.21.15.53:0/1856562575 learned_addr learned my addr 172.21.15.53:0/1856562575 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144effd640 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feff7f0 0x563b5fefeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144ffff640 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff0b0 0x563b5fed3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 -- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feff7f0 msgr2=0x563b5fefeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feff7f0 0x563b5fefeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 -- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff0b0 msgr2=0x563b5fed3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff0b0 0x563b5fed3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144f7fe640 1 -- 172.21.15.53:0/1856562575 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5fc167c0 con 0x563b5ff1c6d0 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.186+0000 7f144ffff640 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff0b0 0x563b5fed3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.187+0000 7f144f7fe640 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5fecf2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1440094e60 tx=0x7f14400682a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690ab2bb568f0f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.187+0000 7f14357fa640 1 -- 172.21.15.53:0/1856562575 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440079070 con 0x563b5ff1c6d0 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.187+0000 7f14357fa640 1 -- 172.21.15.53:0/1856562575 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144005fb90 con 0x563b5ff1c6d0 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.187+0000 7f14357fa640 1 -- 172.21.15.53:0/1856562575 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400076b0 con 0x563b5ff1c6d0 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.187+0000 7f1458c34740 1 -- 172.21.15.53:0/1856562575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 msgr2=0x563b5fecf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.187+0000 7f1458c34740 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5fecf2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1440094e60 tx=0x7f14400682a0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.187+0000 7f1458c34740 1 -- 172.21.15.53:0/1856562575 shutdown_connections 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff0b0 0x563b5fed3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feff7f0 0x563b5fefeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 --2- 172.21.15.53:0/1856562575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5fecf2f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 -- 172.21.15.53:0/1856562575 >> 172.21.15.53:0/1856562575 conn(0x563b5fed1560 msgr2=0x563b5ff19b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 -- 172.21.15.53:0/1856562575 shutdown_connections 2024-05-10T10:36:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 -- 172.21.15.53:0/1856562575 wait complete. 2024-05-10T10:36:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 -- start start 2024-05-10T10:36:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.188+0000 7f1458c34740 1 Processor -- start 2024-05-10T10:36:31.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff2cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5ff1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f1458c34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feef640 0x563b5ff1b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f1458c34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563b5fe3d440 con 0x563b5feef640 2024-05-10T10:36:31.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b5fe38120 con 0x563b5feff7f0 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563b5fe46210 con 0x563b5ff1c6d0 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f1458c34740 10 client.0 _ll_register_callbacks cb 0x563b5fddd690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feef640 0x563b5ff1b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feef640 0x563b5ff1b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40376/0 (socket says 172.21.15.53:40376) 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 -- 172.21.15.53:0/2843345044 learned_addr learned my addr 172.21.15.53:0/2843345044 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144f7fe640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff2cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144effd640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5ff1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 -- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 msgr2=0x563b5ff1aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5ff1aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 -- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 msgr2=0x563b5ff2cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff2cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 -- 172.21.15.53:0/2843345044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5fc76f50 con 0x563b5feef640 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144effd640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5ff1aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f144ffff640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feef640 0x563b5ff1b1e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f144800ea50 tx=0x7f144800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b0cd99cc539c7a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.189+0000 7f142e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f142e7fc640 1 -- 172.21.15.53:0/2843345044 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144800cb60 con 0x563b5feef640 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f142e7fc640 1 -- 172.21.15.53:0/2843345044 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f1448019cf0 con 0x563b5feef640 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f144005fb90 con 0x563b5feef640 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f142e7fc640 1 -- 172.21.15.53:0/2843345044 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f144800cd00 con 0x563b5feef640 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 10 client.5865 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x563b5fb0a0b0 con 0x563b5feef640 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f142e7fc640 1 -- 172.21.15.53:0/2843345044 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144005fb90 con 0x563b5feef640 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f142e7fc640 1 -- 172.21.15.53:0/2843345044 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f1448020050 con 0x563b5feef640 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 10 client.5865 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 10 client.5865 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 20 client.5865 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fefa5d0 0x563b5ff01aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 4 client.5865 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b5fe46a20 con 0x563b5fefa5d0 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.190+0000 7f144f7fe640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fefa5d0 0x563b5ff01aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f144f7fe640 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fefa5d0 0x563b5ff01aa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e01fa2ecf991114 server_cookie=adac015fb86f0370 in_seq=0 out_seq=0 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f142e7fc640 10 client.5865 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f142e7fc640 1 -- 172.21.15.53:0/2843345044 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563b5fe46a20 con 0x563b5fefa5d0 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f142e7fc640 10 client.5865 handle_command_reply: tid=0 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 client.5865 shutdown 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feef640 msgr2=0x563b5ff1b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feef640 0x563b5ff1b1e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f144800ea50 tx=0x7f144800ef10 comp rx=0 tx=0).stop 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 shutdown_connections 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fefa5d0 0x563b5ff01aa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feef640 0x563b5ff1b1e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5ff1aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 --2- 172.21.15.53:0/2843345044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff2cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 >> 172.21.15.53:0/2843345044 conn(0x563b5feff0b0 msgr2=0x563b5fecea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.191+0000 7f142e7fc640 0 client.5865 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.192+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 shutdown_connections 2024-05-10T10:36:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.192+0000 7f1458c34740 1 -- 172.21.15.53:0/2843345044 wait complete. 2024-05-10T10:36:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.192+0000 7f1458c34740 20 client.5865 trim_cache size 0 max 16384 2024-05-10T10:36:31.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.205+0000 7f1458c34740 1 Processor -- start 2024-05-10T10:36:31.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.205+0000 7f1458c34740 1 -- start start 2024-05-10T10:36:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feef640 0x563b5ff02480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff1b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f1458c34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5ff10930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f1458c34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1450201210 con 0x563b5ff1c6d0 2024-05-10T10:36:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1450161120 con 0x563b5feff7f0 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1450161770 con 0x563b5feef640 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5ff10930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5ff10930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40392/0 (socket says 172.21.15.53:40392) 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 -- 172.21.15.53:0/3527717378 learned_addr learned my addr 172.21.15.53:0/3527717378 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 -- 172.21.15.53:0/3527717378 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feef640 msgr2=0x563b5ff02480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feef640 0x563b5ff02480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 -- 172.21.15.53:0/3527717378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 msgr2=0x563b5ff1b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144effd640 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff1b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff1b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.206+0000 7f144ffff640 1 -- 172.21.15.53:0/3527717378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5ff01410 con 0x563b5ff1c6d0 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f144ffff640 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5ff10930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1448002230 tx=0x7f1448013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84982e9b33d91d5b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1434ff9640 1 -- 172.21.15.53:0/3527717378 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144800cb60 con 0x563b5ff1c6d0 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1434ff9640 1 -- 172.21.15.53:0/3527717378 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1448022690 con 0x563b5ff1c6d0 2024-05-10T10:36:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1434ff9640 1 -- 172.21.15.53:0/3527717378 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144800cea0 con 0x563b5ff1c6d0 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 -- 172.21.15.53:0/3527717378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 msgr2=0x563b5ff10930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5ff10930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1448002230 tx=0x7f1448013000 comp rx=0 tx=0).stop 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 -- 172.21.15.53:0/3527717378 shutdown_connections 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5ff1c6d0 0x563b5ff10930 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feff7f0 0x563b5ff1b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 --2- 172.21.15.53:0/3527717378 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5feef640 0x563b5ff02480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 -- 172.21.15.53:0/3527717378 >> 172.21.15.53:0/3527717378 conn(0x563b5feff0b0 msgr2=0x563b5ff1bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 -- 172.21.15.53:0/3527717378 shutdown_connections 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.207+0000 7f1458c34740 1 -- 172.21.15.53:0/3527717378 wait complete. 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.208+0000 7f1458c34740 1 -- start start 2024-05-10T10:36:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.208+0000 7f1458c34740 1 Processor -- start 2024-05-10T10:36:31.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feef640 0x563b5ff98720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f1458c34740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feff7f0 0x563b5ff0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f1458c34740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5fed2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f1458c34740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563b5ff02a10 con 0x563b5feff7f0 2024-05-10T10:36:31.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b5fdeea20 con 0x563b5feef640 2024-05-10T10:36:31.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f1458c34740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563b5ff00200 con 0x563b5ff1c6d0 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f1458c34740 10 client.0 _ll_register_callbacks cb 0x7f1448008660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feff7f0 0x563b5ff0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feff7f0 0x563b5ff0f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40394/0 (socket says 172.21.15.53:40394) 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 -- 172.21.15.53:0/536139483 learned_addr learned my addr 172.21.15.53:0/536139483 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144f7fe640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feef640 0x563b5ff98720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144ffff640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5fed2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 -- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 msgr2=0x563b5fed2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5fed2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 -- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feef640 msgr2=0x563b5ff98720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feef640 0x563b5ff98720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 -- 172.21.15.53:0/536139483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5fec9d40 con 0x563b5feff7f0 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144ffff640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5fed2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:31.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f144effd640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feff7f0 0x563b5ff0f500 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f144402a6f0 tx=0x7f1444004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388dffb682e70b90 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.209+0000 7f142e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f142e7fc640 1 -- 172.21.15.53:0/536139483 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144404f060 con 0x563b5feff7f0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f142e7fc640 1 -- 172.21.15.53:0/536139483 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f1444003810 con 0x563b5feff7f0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1448022690 con 0x563b5feff7f0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f142e7fc640 1 -- 172.21.15.53:0/536139483 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1444055050 con 0x563b5feff7f0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 10 client.5871 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x563b5fed2410 con 0x563b5feff7f0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f142e7fc640 1 -- 172.21.15.53:0/536139483 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1448022690 con 0x563b5feff7f0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f142e7fc640 1 -- 172.21.15.53:0/536139483 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f1444058020 con 0x563b5feff7f0 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 10 client.5871 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 10 client.5871 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 20 client.5871 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fed2db0 0x563b5fefaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 4 client.5871 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x563b5fe40890 con 0x563b5fed2db0 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.210+0000 7f144f7fe640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fed2db0 0x563b5fefaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f142e7fc640 10 client.5871 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f144f7fe640 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fed2db0 0x563b5fefaf30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced8d3423ecb3bbd server_cookie=d916becbe438fa23 in_seq=0 out_seq=0 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f142e7fc640 1 -- 172.21.15.53:0/536139483 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+140 (crc 0 0 0) 0x563b5fe40890 con 0x563b5fed2db0 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f142e7fc640 10 client.5871 handle_command_reply: tid=0 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 client.5871 shutdown 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feff7f0 msgr2=0x563b5ff0f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feff7f0 0x563b5ff0f500 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f144402a6f0 tx=0x7f1444004730 comp rx=0 tx=0).stop 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 shutdown_connections 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x563b5fed2db0 0x563b5fefaf30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563b5ff1c6d0 0x563b5fed2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b5feff7f0 0x563b5ff0f500 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 --2- 172.21.15.53:0/536139483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b5feef640 0x563b5ff98720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f142e7fc640 0 client.5871 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 >> 172.21.15.53:0/536139483 conn(0x563b5feff0b0 msgr2=0x563b5feeea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 shutdown_connections 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 1 -- 172.21.15.53:0/536139483 wait complete. 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.211+0000 7f1458c34740 20 client.5871 trim_cache size 0 max 16384 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:36:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:36:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "damage_type": "backtrace", 2024-05-10T10:36:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 812241800, 2024-05-10T10:36:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2024-05-10T10:36:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/dir_test/file" 2024-05-10T10:36:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:36:31.219 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f143c0762d0 msgr2=0x7f143c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f143c0762d0 0x7f143c078790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1444034000 tx=0x7f144402e000 comp rx=0 tx=0).stop 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 msgr2=0x7f14500ba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500ba340 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f144000c3c0 tx=0x7f144000c880 comp rx=0 tx=0).stop 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 shutdown_connections 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f143c0762d0 0x7f143c078790 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f145015f410 0x7f14501623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14500d36d0 0x7f1450161cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 --2- 172.21.15.53:0/3786909331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14500b4f90 0x7f14500ba340 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 >> 172.21.15.53:0/3786909331 conn(0x7f145008ed30 msgr2=0x7f14500d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 shutdown_connections 2024-05-10T10:36:31.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.212+0000 7f1455e46640 1 -- 172.21.15.53:0/3786909331 wait complete. 2024-05-10T10:36:31.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T10:36:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.324+0000 7fc419062640 1 Processor -- start 2024-05-10T10:36:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.325+0000 7fc419062640 1 -- start start 2024-05-10T10:36:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.325+0000 7fc419062640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc414162750 0x7fc4140d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.325+0000 7fc419062640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4140d5cb0 0x7fc4140d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.325+0000 7fc419062640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4140b45f0 con 0x7fc414162b30 2024-05-10T10:36:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.325+0000 7fc419062640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc4140b4f20 con 0x7fc4140d5cb0 2024-05-10T10:36:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.325+0000 7fc419062640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc4140d0fe0 con 0x7fc414162750 2024-05-10T10:36:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.325+0000 7fc41259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4140d5cb0 0x7fc4140d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc41359e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc414162b30 0x7fc4140d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58454/0 (socket says 172.21.15.53:58454) 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc41359e640 1 -- 172.21.15.53:0/2922008498 learned_addr learned my addr 172.21.15.53:0/2922008498 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc412d9d640 1 --2- 172.21.15.53:0/2922008498 >> v2:172.21.15.135:3300/0 conn(0x7fc414162750 0x7fc4140d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc41259c640 1 -- 172.21.15.53:0/2922008498 >> v2:172.21.15.135:3300/0 conn(0x7fc414162750 msgr2=0x7fc4140d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc41259c640 1 --2- 172.21.15.53:0/2922008498 >> v2:172.21.15.135:3300/0 conn(0x7fc414162750 0x7fc4140d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc41259c640 1 -- 172.21.15.53:0/2922008498 >> v1:172.21.15.53:6789/0 conn(0x7fc414162b30 legacy=0x7fc4140d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc41259c640 1 -- 172.21.15.53:0/2922008498 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140b48d0 con 0x7fc4140d5cb0 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.326+0000 7fc411d9b640 1 -- 172.21.15.53:0/2922008498 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461355029 0 0) 0x7fc4140b45f0 con 0x7fc414162b30 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc41259c640 1 --2- 172.21.15.53:0/2922008498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140d5cb0 0x7fc4140d3630 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4140b7c40 tx=0x7fc40802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fef64607b9230de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc411d9b640 1 -- 172.21.15.53:0/2922008498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408037070 con 0x7fc4140d5cb0 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc411d9b640 1 -- 172.21.15.53:0/2922008498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc408032440 con 0x7fc4140d5cb0 2024-05-10T10:36:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc411d9b640 1 -- 172.21.15.53:0/2922008498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40802ae60 con 0x7fc4140d5cb0 2024-05-10T10:36:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 -- 172.21.15.53:0/2922008498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140d5cb0 msgr2=0x7fc4140d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 --2- 172.21.15.53:0/2922008498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140d5cb0 0x7fc4140d3630 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4140b7c40 tx=0x7fc40802a110 comp rx=0 tx=0).stop 2024-05-10T10:36:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 -- 172.21.15.53:0/2922008498 shutdown_connections 2024-05-10T10:36:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 --2- 172.21.15.53:0/2922008498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140d5cb0 0x7fc4140d3630 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 --2- 172.21.15.53:0/2922008498 >> v2:172.21.15.135:3300/0 conn(0x7fc414162750 0x7fc4140d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 -- 172.21.15.53:0/2922008498 >> 172.21.15.53:0/2922008498 conn(0x7fc41409e3b0 msgr2=0x7fc4140d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 -- 172.21.15.53:0/2922008498 shutdown_connections 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.327+0000 7fc419062640 1 -- 172.21.15.53:0/2922008498 wait complete. 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.329+0000 7fc419062640 1 Processor -- start 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.329+0000 7fc419062640 1 -- start start 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.329+0000 7fc419062640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc414162750 0x7fc41414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.329+0000 7fc419062640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc414162b30 0x7fc4140ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc419062640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140b9fe0 0x7fc4140ba470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc419062640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4140d0d30 con 0x7fc4140b9fe0 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc419062640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4140d0bb0 con 0x7fc414162750 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc419062640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc4140b63c0 con 0x7fc414162b30 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140b9fe0 0x7fc4140ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140b9fe0 0x7fc4140ba470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40420/0 (socket says 172.21.15.53:40420) 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 -- 172.21.15.53:0/641405795 learned_addr learned my addr 172.21.15.53:0/641405795 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc412d9d640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc414162750 0x7fc41414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41259c640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc414162b30 0x7fc4140ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 -- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc414162b30 msgr2=0x7fc4140ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc414162b30 0x7fc4140ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 -- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc414162750 msgr2=0x7fc41414e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc414162750 0x7fc41414e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 -- 172.21.15.53:0/641405795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41414e9c0 con 0x7fc4140b9fe0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc41359e640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140b9fe0 0x7fc4140ba470 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c700 tx=0x7fc40c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc3fb7fe640 1 -- 172.21.15.53:0/641405795 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c018070 con 0x7fc4140b9fe0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc3fb7fe640 1 -- 172.21.15.53:0/641405795 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c00ce10 con 0x7fc4140b9fe0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4140cd410 con 0x7fc4140b9fe0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc3fb7fe640 1 -- 172.21.15.53:0/641405795 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00a860 con 0x7fc4140b9fe0 2024-05-10T10:36:31.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.330+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4140cc650 con 0x7fc4140b9fe0 2024-05-10T10:36:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.331+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41408f140 con 0x7fc4140b9fe0 2024-05-10T10:36:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.332+0000 7fc3fb7fe640 1 -- 172.21.15.53:0/641405795 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc40c01b070 con 0x7fc4140b9fe0 2024-05-10T10:36:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.332+0000 7fc3fb7fe640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4000762d0 0x7fc400078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.333+0000 7fc3fb7fe640 1 -- 172.21.15.53:0/641405795 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fc40c096cb0 con 0x7fc4140b9fe0 2024-05-10T10:36:31.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.333+0000 7fc412d9d640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4000762d0 0x7fc400078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.333+0000 7fc412d9d640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4000762d0 0x7fc400078790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc404006fd0 tx=0x7fc404008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.337+0000 7fc3fb7fe640 1 -- 172.21.15.53:0/641405795 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc41408f140 con 0x7fc4140b9fe0 2024-05-10T10:36:31.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.575+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc41406a220 con 0x7fc4140b9fe0 2024-05-10T10:36:31.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.575+0000 7fc3fb7fe640 1 -- 172.21.15.53:0/641405795 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fc41414e9c0 con 0x7fc4140b9fe0 2024-05-10T10:36:31.576 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:36:31.576 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-10T10:36:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4000762d0 msgr2=0x7fc400078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4000762d0 0x7fc400078790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc404006fd0 tx=0x7fc404008040 comp rx=0 tx=0).stop 2024-05-10T10:36:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140b9fe0 msgr2=0x7fc4140ba470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140b9fe0 0x7fc4140ba470 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c700 tx=0x7fc40c00cbc0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 shutdown_connections 2024-05-10T10:36:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4000762d0 0x7fc400078790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4140b9fe0 0x7fc4140ba470 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc414162b30 0x7fc4140ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 --2- 172.21.15.53:0/641405795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc414162750 0x7fc41414e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 >> 172.21.15.53:0/641405795 conn(0x7fc41409e3b0 msgr2=0x7fc4140d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 shutdown_connections 2024-05-10T10:36:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.577+0000 7fc419062640 1 -- 172.21.15.53:0/641405795 wait complete. 2024-05-10T10:36:31.589 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T10:36:31.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:31.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-05-10T10:36:31.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.696+0000 7fc2ffa83640 1 Processor -- start 2024-05-10T10:36:31.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.696+0000 7fc2ffa83640 1 -- start start 2024-05-10T10:36:31.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.697+0000 7fc2ffa83640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2f815b3d0 0x7fc2f80b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.697+0000 7fc2ffa83640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815b7b0 0x7fc2f80b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.697+0000 7fc2ffa83640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2f806ec80 0x7fc2f806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.697+0000 7fc2ffa83640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f8072cc0 con 0x7fc2f806ec80 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.697+0000 7fc2ffa83640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f80720f0 con 0x7fc2f815b3d0 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.697+0000 7fc2ffa83640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f8072390 con 0x7fc2f815b7b0 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.697+0000 7fc2fe020640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2f806ec80 0x7fc2f806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fe020640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2f806ec80 0x7fc2f806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40434/0 (socket says 172.21.15.53:40434) 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fe020640 1 -- 172.21.15.53:0/1067974022 learned_addr learned my addr 172.21.15.53:0/1067974022 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/1067974022 >> v2:172.21.15.135:3300/0 conn(0x7fc2f815b3d0 0x7fc2f80b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/1067974022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815b7b0 0x7fc2f80b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fe020640 1 -- 172.21.15.53:0/1067974022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815b7b0 msgr2=0x7fc2f80b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fe020640 1 --2- 172.21.15.53:0/1067974022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815b7b0 0x7fc2f80b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fe020640 1 -- 172.21.15.53:0/1067974022 >> v2:172.21.15.135:3300/0 conn(0x7fc2f815b3d0 msgr2=0x7fc2f80b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fe020640 1 --2- 172.21.15.53:0/1067974022 >> v2:172.21.15.135:3300/0 conn(0x7fc2f815b3d0 0x7fc2f80b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.698+0000 7fc2fe020640 1 -- 172.21.15.53:0/1067974022 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f8072670 con 0x7fc2f806ec80 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2fe020640 1 --2- 172.21.15.53:0/1067974022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f806ec80 0x7fc2f806f100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8070220 tx=0x7fc2f002c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b511de187e19e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2fc81d640 1 -- 172.21.15.53:0/1067974022 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0035400 con 0x7fc2f806ec80 2024-05-10T10:36:31.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2fc81d640 1 -- 172.21.15.53:0/1067974022 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f00355a0 con 0x7fc2f806ec80 2024-05-10T10:36:31.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2fc81d640 1 -- 172.21.15.53:0/1067974022 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0035970 con 0x7fc2f806ec80 2024-05-10T10:36:31.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 -- 172.21.15.53:0/1067974022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f806ec80 msgr2=0x7fc2f806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/1067974022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f806ec80 0x7fc2f806f100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8070220 tx=0x7fc2f002c8a0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 -- 172.21.15.53:0/1067974022 shutdown_connections 2024-05-10T10:36:31.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/1067974022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f806ec80 0x7fc2f806f100 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/1067974022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815b7b0 0x7fc2f80b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/1067974022 >> v2:172.21.15.135:3300/0 conn(0x7fc2f815b3d0 0x7fc2f80b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 -- 172.21.15.53:0/1067974022 >> 172.21.15.53:0/1067974022 conn(0x7fc2f806c4a0 msgr2=0x7fc2f806c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 -- 172.21.15.53:0/1067974022 shutdown_connections 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.699+0000 7fc2ffa83640 1 -- 172.21.15.53:0/1067974022 wait complete. 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 Processor -- start 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 -- start start 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f806ec80 0x7fc2f809e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f815b3d0 0x7fc2f8153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815b7b0 0x7fc2f81535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80ade90 con 0x7fc2f815b7b0 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f806f640 con 0x7fc2f815b3d0 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2ffa83640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f80ae010 con 0x7fc2f806ec80 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2fe020640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815b7b0 0x7fc2f81535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2fe020640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815b7b0 0x7fc2f81535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40440/0 (socket says 172.21.15.53:40440) 2024-05-10T10:36:31.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.701+0000 7fc2fe020640 1 -- 172.21.15.53:0/3602569103 learned_addr learned my addr 172.21.15.53:0/3602569103 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f806ec80 0x7fc2f809e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fe020640 1 -- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f806ec80 msgr2=0x7fc2f809e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f815b3d0 0x7fc2f8153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fe020640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f806ec80 0x7fc2f809e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fe020640 1 -- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f815b3d0 msgr2=0x7fc2f8153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fe020640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f815b3d0 0x7fc2f8153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fe020640 1 -- 172.21.15.53:0/3602569103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f80cd5b0 con 0x7fc2f815b7b0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2fe020640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815b7b0 0x7fc2f81535b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc2f002ce90 tx=0x7fc2f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2ee7fc640 1 -- 172.21.15.53:0/3602569103 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0045070 con 0x7fc2f815b7b0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2ee7fc640 1 -- 172.21.15.53:0/3602569103 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f0034960 con 0x7fc2f815b7b0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f80cd850 con 0x7fc2f815b7b0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2ee7fc640 1 -- 172.21.15.53:0/3602569103 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0035400 con 0x7fc2f815b7b0 2024-05-10T10:36:31.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.702+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f80cde10 con 0x7fc2f815b7b0 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.704+0000 7fc2ee7fc640 1 -- 172.21.15.53:0/3602569103 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc2f003e430 con 0x7fc2f815b7b0 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.704+0000 7fc2ee7fc640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2dc0762d0 0x7fc2dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.704+0000 7fc2ee7fc640 1 -- 172.21.15.53:0/3602569103 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fc2f00b9eb0 con 0x7fc2f815b7b0 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.705+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2dc0762d0 0x7fc2dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.705+0000 7fc3027a1740 1 Processor -- start 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.705+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2dc0762d0 0x7fc2dc078790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009a10 tx=0x7fc2f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.705+0000 7fc3027a1740 1 -- start start 2024-05-10T10:36:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c490c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c493c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493c860 0x55a0c4911670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc3027a1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a0c4876bc0 con 0x55a0c493cfa0 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0c482fd00 con 0x55a0c4959e80 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a0c4872b80 con 0x55a0c493c860 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c493c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c493c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40446/0 (socket says 172.21.15.53:40446) 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 -- 172.21.15.53:0/2125776539 learned_addr learned my addr 172.21.15.53:0/2125776539 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c490c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fe020640 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493c860 0x55a0c4911670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 -- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493c860 msgr2=0x55a0c4911670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493c860 0x55a0c4911670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 -- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 msgr2=0x55a0c490c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c490c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fd01e640 1 -- 172.21.15.53:0/2125776539 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c4654940 con 0x55a0c493cfa0 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.706+0000 7fc2fe020640 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493c860 0x55a0c4911670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c493c320 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80099c0 tx=0x7fc2e8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9828a992ca89a51d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc2db7fe640 1 -- 172.21.15.53:0/2125776539 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8018070 con 0x55a0c493cfa0 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc2db7fe640 1 -- 172.21.15.53:0/2125776539 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8010ca0 con 0x55a0c493cfa0 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc2db7fe640 1 -- 172.21.15.53:0/2125776539 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e800f040 con 0x55a0c493cfa0 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 -- 172.21.15.53:0/2125776539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 msgr2=0x55a0c493c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c493c320 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80099c0 tx=0x7fc2e8009190 comp rx=0 tx=0).stop 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 -- 172.21.15.53:0/2125776539 shutdown_connections 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493c860 0x55a0c4911670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c493c320 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2125776539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c490c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 -- 172.21.15.53:0/2125776539 >> 172.21.15.53:0/2125776539 conn(0x55a0c490ea80 msgr2=0x55a0c4957320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 -- 172.21.15.53:0/2125776539 shutdown_connections 2024-05-10T10:36:31.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.707+0000 7fc3027a1740 1 -- 172.21.15.53:0/2125776539 wait complete. 2024-05-10T10:36:31.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.708+0000 7fc3027a1740 1 -- start start 2024-05-10T10:36:31.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.708+0000 7fc3027a1740 1 Processor -- start 2024-05-10T10:36:31.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c492d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c493f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4950070 0x55a0c4957320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc3027a1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a0c4910c30 con 0x55a0c493cfa0 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0c4833830 con 0x55a0c4959e80 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a0c4910020 con 0x55a0c4950070 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc3027a1740 10 client.0 _ll_register_callbacks cb 0x55a0c4870750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c492d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c492d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40462/0 (socket says 172.21.15.53:40462) 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2334879747 learned_addr learned my addr 172.21.15.53:0/2334879747 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c493f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fe020640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4950070 0x55a0c4957320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4950070 msgr2=0x55a0c4957320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4950070 0x55a0c4957320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 msgr2=0x55a0c493f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c493f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2334879747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c46b50d0 con 0x55a0c493cfa0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.709+0000 7fc2fe020640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4950070 0x55a0c4957320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c492d130 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4015a20 tx=0x7fc2f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb5bc1c5fcb56e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/2334879747 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4026070 con 0x55a0c493cfa0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/2334879747 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fc2f4016db0 con 0x55a0c493cfa0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e8010ca0 con 0x55a0c493cfa0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/2334879747 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc2f4031050 con 0x55a0c493cfa0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 10 client.5892 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55a0c4548230 con 0x55a0c493cfa0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/2334879747 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8010ca0 con 0x55a0c493cfa0 2024-05-10T10:36:31.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/2334879747 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fc2f4034020 con 0x55a0c493cfa0 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 10 client.5892 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 10 client.5892 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 20 client.5892 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c492a5f0 0x55a0c492aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 4 client.5892 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c48847d0 con 0x55a0c492a5f0 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.710+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c492a5f0 0x55a0c492aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c492a5f0 0x55a0c492aa30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503aa462b31344dd server_cookie=7d9db46e82ec7c89 in_seq=0 out_seq=0 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc2d8ff9640 10 client.5892 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/2334879747 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0c48847d0 con 0x55a0c492a5f0 2024-05-10T10:36:31.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc2d8ff9640 10 client.5892 handle_command_reply: tid=0 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc3027a1740 1 client.5892 shutdown 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 msgr2=0x55a0c492d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c492d130 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4015a20 tx=0x7fc2f40046c0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 shutdown_connections 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c492a5f0 0x55a0c492aa30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4950070 0x55a0c4957320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.711+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4959e80 0x55a0c493f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.712+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2334879747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c492d130 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.712+0000 7fc2d8ff9640 0 client.5892 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.712+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 >> 172.21.15.53:0/2334879747 conn(0x55a0c493c860 msgr2=0x55a0c4958380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.712+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 shutdown_connections 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.712+0000 7fc3027a1740 1 -- 172.21.15.53:0/2334879747 wait complete. 2024-05-10T10:36:31.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.712+0000 7fc3027a1740 20 client.5892 trim_cache size 0 max 16384 2024-05-10T10:36:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.730+0000 7fc3027a1740 1 Processor -- start 2024-05-10T10:36:31.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.730+0000 7fc3027a1740 1 -- start start 2024-05-10T10:36:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c494e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c490b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4959e80 0x55a0c490bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc3027a1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f81fdc40 con 0x55a0c493cfa0 2024-05-10T10:36:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f81fcc60 con 0x55a0c4950070 2024-05-10T10:36:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f81fc010 con 0x55a0c4959e80 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c494e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c494e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40476/0 (socket says 172.21.15.53:40476) 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2640020652 learned_addr learned my addr 172.21.15.53:0/2640020652 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fe020640 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4959e80 0x55a0c490bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c490b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4959e80 msgr2=0x55a0c490bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4959e80 0x55a0c490bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 msgr2=0x55a0c490b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c490b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 -- 172.21.15.53:0/2640020652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c494e660 con 0x55a0c493cfa0 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c494e0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000c00 tx=0x7fc2f4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b930ecfb9171610 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.731+0000 7fc2daffd640 1 -- 172.21.15.53:0/2640020652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4026050 con 0x55a0c493cfa0 2024-05-10T10:36:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc2daffd640 1 -- 172.21.15.53:0/2640020652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f401d040 con 0x55a0c493cfa0 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc2daffd640 1 -- 172.21.15.53:0/2640020652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f401c040 con 0x55a0c493cfa0 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 -- 172.21.15.53:0/2640020652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 msgr2=0x55a0c494e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c494e0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000c00 tx=0x7fc2f4022000 comp rx=0 tx=0).stop 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 -- 172.21.15.53:0/2640020652 shutdown_connections 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c4959e80 0x55a0c490bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c490b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 --2- 172.21.15.53:0/2640020652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c493cfa0 0x55a0c494e0a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 -- 172.21.15.53:0/2640020652 >> 172.21.15.53:0/2640020652 conn(0x55a0c493c860 msgr2=0x55a0c4950a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 -- 172.21.15.53:0/2640020652 shutdown_connections 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.732+0000 7fc3027a1740 1 -- 172.21.15.53:0/2640020652 wait complete. 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.733+0000 7fc3027a1740 1 -- start start 2024-05-10T10:36:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.733+0000 7fc3027a1740 1 Processor -- start 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493cfa0 0x55a0c49388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c49586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c4959e80 0x55a0c490b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f81fc590 con 0x55a0c4959e80 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0c4939300 con 0x55a0c4950070 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a0c49666b0 con 0x55a0c493cfa0 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 10 client.0 _ll_register_callbacks cb 0x7fc2f4010950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c4959e80 0x55a0c490b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c4959e80 0x55a0c490b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40484/0 (socket says 172.21.15.53:40484) 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 -- 172.21.15.53:0/95226186 learned_addr learned my addr 172.21.15.53:0/95226186 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493cfa0 0x55a0c49388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fd01e640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c49586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 -- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493cfa0 msgr2=0x55a0c49388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493cfa0 0x55a0c49388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 -- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 msgr2=0x55a0c49586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c49586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 -- 172.21.15.53:0/95226186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c4938e00 con 0x55a0c4959e80 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493cfa0 0x55a0c49388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2fe020640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c4959e80 0x55a0c490b6a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00801f0 tx=0x7fc2f0082a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953a2a00e6510be7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/95226186 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f004e030 con 0x55a0c4959e80 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/95226186 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fc2f008bac0 con 0x55a0c4959e80 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.734+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f401d040 con 0x55a0c4959e80 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/95226186 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc2f00b6050 con 0x55a0c4959e80 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 10 client.5898 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55a0c493d570 con 0x55a0c4959e80 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/95226186 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f401d040 con 0x55a0c4959e80 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/95226186 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fc2f0051060 con 0x55a0c4959e80 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 10 client.5898 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 10 client.5898 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 20 client.5898 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c493e980 0x55a0c490bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 4 client.5898 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55a0c487e640 con 0x55a0c493e980 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c493e980 0x55a0c490bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc2d8ff9640 10 client.5898 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.735+0000 7fc2fd81f640 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c493e980 0x55a0c490bbe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dde70cc963ee0f server_cookie=d69ffaade788ecf0 in_seq=0 out_seq=0 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc2d8ff9640 1 -- 172.21.15.53:0/95226186 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0c487e640 con 0x55a0c493e980 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc2d8ff9640 10 client.5898 handle_command_reply: tid=0 2024-05-10T10:36:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 client.5898 shutdown 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c4959e80 msgr2=0x55a0c490b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c4959e80 0x55a0c490b6a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00801f0 tx=0x7fc2f0082a90 comp rx=0 tx=0).stop 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 shutdown_connections 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x55a0c493e980 0x55a0c490bbe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a0c4959e80 0x55a0c490b6a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0c4950070 0x55a0c49586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 --2- 172.21.15.53:0/95226186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a0c493cfa0 0x55a0c49388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 >> 172.21.15.53:0/95226186 conn(0x55a0c493c860 msgr2=0x55a0c4970c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.736+0000 7fc2d8ff9640 0 client.5898 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.742 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:36:31.736+0000 7f9df55b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5165=0-4194304@1} caps={5165=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5165 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ebd1e6cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:36:31.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 shutdown_connections 2024-05-10T10:36:31.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc3027a1740 1 -- 172.21.15.53:0/95226186 wait complete. 2024-05-10T10:36:31.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc3027a1740 20 client.5898 trim_cache size 0 max 16384 2024-05-10T10:36:31.743 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:36:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:36:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "084f7b87-a003-4fc6-bb83-a0539944cc23", 2024-05-10T10:36:31.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:36:31.744 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:36:31.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2dc0762d0 msgr2=0x7fc2dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2dc0762d0 0x7fc2dc078790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009a10 tx=0x7fc2f4008040 comp rx=0 tx=0).stop 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815b7b0 msgr2=0x7fc2f81535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815b7b0 0x7fc2f81535b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc2f002ce90 tx=0x7fc2f0004a40 comp rx=0 tx=0).stop 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 shutdown_connections 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2dc0762d0 0x7fc2dc078790 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815b7b0 0x7fc2f81535b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f815b3d0 0x7fc2f8153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 --2- 172.21.15.53:0/3602569103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f806ec80 0x7fc2f809e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 >> 172.21.15.53:0/3602569103 conn(0x7fc2f806c4a0 msgr2=0x7fc2f80b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 shutdown_connections 2024-05-10T10:36:31.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.737+0000 7fc2ffa83640 1 -- 172.21.15.53:0/3602569103 wait complete. 2024-05-10T10:36:31.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:36:31.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-10T10:36:31.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.853+0000 7f95a36b0640 1 Processor -- start 2024-05-10T10:36:31.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.853+0000 7f95a36b0640 1 -- start start 2024-05-10T10:36:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a36b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f959c0cffd0 0x7f959c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a36b0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cd500 0x7f959c0d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a36b0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959c06e720 con 0x7f959c0cf3e0 2024-05-10T10:36:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a36b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f959c06de80 con 0x7f959c0cffd0 2024-05-10T10:36:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a36b0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f959c06e160 con 0x7f959c0cd500 2024-05-10T10:36:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a0c4b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f959c0cffd0 0x7f959c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a0c4b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f959c0cffd0 0x7f959c0b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40486/0 (socket says 172.21.15.53:40486) 2024-05-10T10:36:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2593051776 learned_addr learned my addr 172.21.15.53:0/2593051776 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.854+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/2593051776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cd500 0x7f959c0d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2593051776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cd500 msgr2=0x7f959c0d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2593051776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cd500 0x7f959c0d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2593051776 >> v1:172.21.15.135:6789/0 conn(0x7f959c0cf3e0 legacy=0x7f959c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2593051776 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c06e440 con 0x7f959c0cffd0 2024-05-10T10:36:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2593051776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cffd0 0x7f959c0b3de0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f958c009870 tx=0x7f958c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec34838126a9ab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f9593fff640 1 -- 172.21.15.53:0/2593051776 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c037070 con 0x7f959c0cffd0 2024-05-10T10:36:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f9593fff640 1 -- 172.21.15.53:0/2593051776 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c032440 con 0x7f959c0cffd0 2024-05-10T10:36:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.855+0000 7f9593fff640 1 -- 172.21.15.53:0/2593051776 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c032810 con 0x7f959c0cffd0 2024-05-10T10:36:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 -- 172.21.15.53:0/2593051776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cffd0 msgr2=0x7f959c0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2593051776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cffd0 0x7f959c0b3de0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f958c009870 tx=0x7f958c02a270 comp rx=0 tx=0).stop 2024-05-10T10:36:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 -- 172.21.15.53:0/2593051776 shutdown_connections 2024-05-10T10:36:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2593051776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cd500 0x7f959c0d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2593051776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cffd0 0x7f959c0b3de0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 -- 172.21.15.53:0/2593051776 >> 172.21.15.53:0/2593051776 conn(0x7f959c0d2eb0 msgr2=0x7f959c0d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 -- 172.21.15.53:0/2593051776 shutdown_connections 2024-05-10T10:36:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.856+0000 7f95a36b0640 1 -- 172.21.15.53:0/2593051776 wait complete. 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.857+0000 7f95a36b0640 1 Processor -- start 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.857+0000 7f95a36b0640 1 -- start start 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a36b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cd500 0x7f959c0ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a36b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f959c0cf3e0 0x7f959c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a36b0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cffd0 0x7f959c1f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a36b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f959c0d16c0 con 0x7f959c0cd500 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a36b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f959c0b4510 con 0x7f959c0cf3e0 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a36b0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f959c0d1840 con 0x7f959c0cffd0 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cd500 0x7f959c0ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cd500 0x7f959c0ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40492/0 (socket says 172.21.15.53:40492) 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 -- 172.21.15.53:0/2971356895 learned_addr learned my addr 172.21.15.53:0/2971356895 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cffd0 0x7f959c1f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f959c0cf3e0 0x7f959c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 -- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cffd0 msgr2=0x7f959c1f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cffd0 0x7f959c1f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 -- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f959c0cf3e0 msgr2=0x7f959c15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f959c0cf3e0 0x7f959c15f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.858+0000 7f95a144c640 1 -- 172.21.15.53:0/2971356895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c15f980 con 0x7f959c0cd500 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.859+0000 7f95a144c640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cd500 0x7f959c0ccd10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f959800c430 tx=0x7f959800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.859+0000 7f9591ffb640 1 -- 172.21.15.53:0/2971356895 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980052d0 con 0x7f959c0cd500 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.859+0000 7f9591ffb640 1 -- 172.21.15.53:0/2971356895 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9598005490 con 0x7f959c0cd500 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.859+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c15fbf0 con 0x7f959c0cd500 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.859+0000 7f9591ffb640 1 -- 172.21.15.53:0/2971356895 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598007260 con 0x7f959c0cd500 2024-05-10T10:36:31.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.859+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c0c7ad0 con 0x7f959c0cd500 2024-05-10T10:36:31.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.860+0000 7f9591ffb640 1 -- 172.21.15.53:0/2971356895 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9598007400 con 0x7f959c0cd500 2024-05-10T10:36:31.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.861+0000 7f9591ffb640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95800762d0 0x7f9580078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.861+0000 7f9591ffb640 1 -- 172.21.15.53:0/2971356895 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f9598096c90 con 0x7f959c0cd500 2024-05-10T10:36:31.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.861+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95800762d0 0x7f9580078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.862+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95800762d0 0x7f9580078790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f958c034000 tx=0x7f958c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.862+0000 7f95a63cc740 1 Processor -- start 2024-05-10T10:36:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.862+0000 7f95a63cc740 1 -- start start 2024-05-10T10:36:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d29af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d59600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d59b40 0x560c03d2e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a63cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c03cf6690 con 0x560c03d59b40 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560c03c93f30 con 0x560c03d5a280 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560c03c4cb20 con 0x560c03d77160 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d59b40 0x560c03d2e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d59b40 0x560c03d2e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40504/0 (socket says 172.21.15.53:40504) 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 -- 172.21.15.53:0/1959692825 learned_addr learned my addr 172.21.15.53:0/1959692825 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a144c640 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d29af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 -- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 msgr2=0x560c03d29af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d29af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 -- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 msgr2=0x560c03d59600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d59600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 -- 172.21.15.53:0/1959692825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c03c44e60 con 0x560c03d59b40 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d59600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d59b40 0x560c03d2e950 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9594009fa0 tx=0x7f95940077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2255cb9b53fdd252 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f9586ffd640 1 -- 172.21.15.53:0/1959692825 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594013780 con 0x560c03d59b40 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f9586ffd640 1 -- 172.21.15.53:0/1959692825 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594019070 con 0x560c03d59b40 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.863+0000 7f9586ffd640 1 -- 172.21.15.53:0/1959692825 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594013b60 con 0x560c03d59b40 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 -- 172.21.15.53:0/1959692825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d59b40 msgr2=0x560c03d2e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d59b40 0x560c03d2e950 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9594009fa0 tx=0x7f95940077c0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 -- 172.21.15.53:0/1959692825 shutdown_connections 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d59b40 0x560c03d2e950 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d59600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 --2- 172.21.15.53:0/1959692825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d29af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 -- 172.21.15.53:0/1959692825 >> 172.21.15.53:0/1959692825 conn(0x560c03d2bd60 msgr2=0x560c03d74600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 -- 172.21.15.53:0/1959692825 shutdown_connections 2024-05-10T10:36:31.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.864+0000 7f95a63cc740 1 -- 172.21.15.53:0/1959692825 wait complete. 2024-05-10T10:36:31.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.865+0000 7f95a63cc740 1 -- start start 2024-05-10T10:36:31.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.865+0000 7f95a63cc740 1 Processor -- start 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.865+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c03c505e0 con 0x560c03d77160 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560c03c8fef0 con 0x560c03d5a280 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560c03d2df10 con 0x560c03d5b530 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 10 client.0 _ll_register_callbacks cb 0x560c03c4b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d56080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40512/0 (socket says 172.21.15.53:40512) 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2513852741 learned_addr learned my addr 172.21.15.53:0/2513852741 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a144c640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 msgr2=0x560c03d49a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d49a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 msgr2=0x560c03d6b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d6b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 -- 172.21.15.53:0/2513852741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c03a6f7d0 con 0x560c03d77160 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d56080 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f958c02a860 tx=0x7f958c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43e34712ed56c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f957bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f957bfff640 1 -- 172.21.15.53:0/2513852741 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c04d060 con 0x560c03d77160 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f957bfff640 1 -- 172.21.15.53:0/2513852741 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f958c03ca70 con 0x560c03d77160 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9594019070 con 0x560c03d77160 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f957bfff640 1 -- 172.21.15.53:0/2513852741 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f958c053050 con 0x560c03d77160 2024-05-10T10:36:31.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 10 client.5910 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.866+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x560c039a53f0 con 0x560c03d77160 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f957bfff640 1 -- 172.21.15.53:0/2513852741 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594019070 con 0x560c03d77160 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f957bfff640 1 -- 172.21.15.53:0/2513852741 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f958c047070 con 0x560c03d77160 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a63cc740 10 client.5910 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a63cc740 10 client.5910 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a63cc740 20 client.5910 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a63cc740 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d5c550 0x560c03d5c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a63cc740 4 client.5910 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c03ca1b40 con 0x560c03d5c550 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a144c640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d5c550 0x560c03d5c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f95a144c640 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d5c550 0x560c03d5c960 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a30f675e072fdf server_cookie=e22cbc46c54a3a03 in_seq=0 out_seq=0 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.867+0000 7f957bfff640 10 client.5910 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f957bfff640 1 -- 172.21.15.53:0/2513852741 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c03ca1b40 con 0x560c03d5c550 2024-05-10T10:36:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f957bfff640 10 client.5910 handle_command_reply: tid=0 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 client.5910 shutdown 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 msgr2=0x560c03d56080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d56080 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f958c02a860 tx=0x7f958c02c070 comp rx=0 tx=0).stop 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 shutdown_connections 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d5c550 0x560c03d5c960 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d56080 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 --2- 172.21.15.53:0/2513852741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d6b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f957bfff640 0 client.5910 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 >> 172.21.15.53:0/2513852741 conn(0x560c03d59b40 msgr2=0x560c03d29230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 shutdown_connections 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 1 -- 172.21.15.53:0/2513852741 wait complete. 2024-05-10T10:36:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.868+0000 7f95a63cc740 20 client.5910 trim_cache size 0 max 16384 2024-05-10T10:36:31.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 Processor -- start 2024-05-10T10:36:31.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 -- start start 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d73780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d69f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c03d55790 con 0x560c03d77160 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f959c202050 con 0x560c03d5a280 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f959c201030 con 0x560c03d5b530 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a1c4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d69f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a1c4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d69f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40524/0 (socket says 172.21.15.53:40524) 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.887+0000 7f95a1c4d640 1 -- 172.21.15.53:0/3775532223 learned_addr learned my addr 172.21.15.53:0/3775532223 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a144c640 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d73780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a1c4d640 1 -- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 msgr2=0x560c03d56080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d56080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a1c4d640 1 -- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 msgr2=0x560c03d73780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d73780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a1c4d640 1 -- 172.21.15.53:0/3775532223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c039650f0 con 0x560c03d77160 2024-05-10T10:36:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d69f90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9594016040 tx=0x7f9594013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72395b9a5eb4568 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95867fc640 1 -- 172.21.15.53:0/3775532223 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594003b90 con 0x560c03d77160 2024-05-10T10:36:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95867fc640 1 -- 172.21.15.53:0/3775532223 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594019050 con 0x560c03d77160 2024-05-10T10:36:31.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.888+0000 7f95867fc640 1 -- 172.21.15.53:0/3775532223 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594004040 con 0x560c03d77160 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 -- 172.21.15.53:0/3775532223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 msgr2=0x560c03d69f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d69f90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9594016040 tx=0x7f9594013000 comp rx=0 tx=0).stop 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 -- 172.21.15.53:0/3775532223 shutdown_connections 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d77160 0x560c03d69f90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d5b530 0x560c03d56080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3775532223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03d73780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 -- 172.21.15.53:0/3775532223 >> 172.21.15.53:0/3775532223 conn(0x560c03d59b40 msgr2=0x560c03d2a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 -- 172.21.15.53:0/3775532223 shutdown_connections 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 -- 172.21.15.53:0/3775532223 wait complete. 2024-05-10T10:36:31.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 -- start start 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.889+0000 7f95a63cc740 1 Processor -- start 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03df1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d5b530 0x560c03d82d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a63cc740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d83260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a63cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f959c200960 con 0x560c03d5b530 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f959c200310 con 0x560c03d5a280 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a63cc740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560c03d55790 con 0x560c03d77160 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a63cc740 10 client.0 _ll_register_callbacks cb 0x7f958c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a0c4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d5b530 0x560c03d82d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a0c4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d5b530 0x560c03d82d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40528/0 (socket says 172.21.15.53:40528) 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a0c4b640 1 -- 172.21.15.53:0/3282230730 learned_addr learned my addr 172.21.15.53:0/3282230730 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a144c640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03df1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.890+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d83260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a0c4b640 1 -- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 msgr2=0x560c03d83260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d83260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a0c4b640 1 -- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 msgr2=0x560c03df1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03df1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a0c4b640 1 -- 172.21.15.53:0/3282230730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c03d74a90 con 0x560c03d5b530 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a144c640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03df1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a1c4d640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d83260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a0c4b640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d5b530 0x560c03d82d20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f958c03cfd0 tx=0x7f958c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1529026e1a195527 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f957bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f957bfff640 1 -- 172.21.15.53:0/3282230730 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c037040 con 0x560c03d5b530 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f957bfff640 1 -- 172.21.15.53:0/3282230730 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f958c004450 con 0x560c03d5b530 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9594019050 con 0x560c03d5b530 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f957bfff640 1 -- 172.21.15.53:0/3282230730 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f958c032490 con 0x560c03d5b530 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 10 client.5916 fetch_fsmap learned FSMap version 35 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x560c03d6ac20 con 0x560c03d5b530 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f957bfff640 1 -- 172.21.15.53:0/3282230730 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594019050 con 0x560c03d5b530 2024-05-10T10:36:31.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f957bfff640 1 -- 172.21.15.53:0/3282230730 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f958c047040 con 0x560c03d5b530 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 10 client.5916 fetch_fsmap finished waiting for FSMap version 35 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 10 client.5916 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 20 client.5916 populate_metadata read hostname 'smithi053' 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d6d8a0 0x560c03d96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 4 client.5916 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.891+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c03c9b9b0 con 0x560c03d6d8a0 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a144c640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d6d8a0 0x560c03d96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f957bfff640 10 client.5916 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a144c640 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d6d8a0 0x560c03d96d00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3128dd639f1a9efc server_cookie=4ba8c704bca22901 in_seq=0 out_seq=0 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f957bfff640 1 -- 172.21.15.53:0/3282230730 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x560c03c9b9b0 con 0x560c03d6d8a0 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f957bfff640 10 client.5916 handle_command_reply: tid=0 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 client.5916 shutdown 2024-05-10T10:36:31.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d5b530 msgr2=0x560c03d82d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d5b530 0x560c03d82d20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f958c03cfd0 tx=0x7f958c004730 comp rx=0 tx=0).stop 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 shutdown_connections 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x560c03d6d8a0 0x560c03d96d00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560c03d77160 0x560c03d83260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c03d5b530 0x560c03d82d20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f957bfff640 0 client.5916 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 --2- 172.21.15.53:0/3282230730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c03d5a280 0x560c03df1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 >> 172.21.15.53:0/3282230730 conn(0x560c03d59b40 msgr2=0x560c03d69f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 shutdown_connections 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.892+0000 7f95a63cc740 1 -- 172.21.15.53:0/3282230730 wait complete. 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a63cc740 20 client.5916 trim_cache size 0 max 16384 2024-05-10T10:36:31.899 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "084f7b87-a003-4fc6-bb83-a0539944cc23": { 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "084f7b87-a003-4fc6-bb83-a0539944cc23", 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair,force" 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:36:31.900 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95800762d0 msgr2=0x7f9580078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95800762d0 0x7f9580078790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f958c034000 tx=0x7f958c02e000 comp rx=0 tx=0).stop 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cd500 msgr2=0x7f959c0ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cd500 0x7f959c0ccd10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f959800c430 tx=0x7f959800c8f0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 shutdown_connections 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95800762d0 0x7f9580078790 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f959c0cffd0 0x7f959c1f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f959c0cf3e0 0x7f959c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 --2- 172.21.15.53:0/2971356895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c0cd500 0x7f959c0ccd10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:36:31.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 >> 172.21.15.53:0/2971356895 conn(0x7f959c0d2eb0 msgr2=0x7f959c071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:36:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 shutdown_connections 2024-05-10T10:36:31.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:36:31.893+0000 7f95a36b0640 1 -- 172.21.15.53:0/2971356895 wait complete. 2024-05-10T10:36:31.904 INFO:tasks.cephfs.filesystem:scrub status for tag:084f7b87-a003-4fc6-bb83-a0539944cc23 - {'path': '/', 'tag': '084f7b87-a003-4fc6-bb83-a0539944cc23', 'options': 'recursive,repair,force'} 2024-05-10T10:36:44.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:36:44.203 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:37:01.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:01.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-10T10:37:02.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.022+0000 7f701c6fc640 1 Processor -- start 2024-05-10T10:37:02.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.022+0000 7f701c6fc640 1 -- start start 2024-05-10T10:37:02.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.023+0000 7f701c6fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70140cf3e0 0x7f70140cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.023+0000 7f701c6fc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701406e720 con 0x7f70140d2190 2024-05-10T10:37:02.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.023+0000 7f701c6fc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701406de80 con 0x7f70140cd500 2024-05-10T10:37:02.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.023+0000 7f701c6fc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f701406e160 con 0x7f70140cf3e0 2024-05-10T10:37:02.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.023+0000 7f701ac99640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f70140d2190 0x7f70140b34e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57464/0 (socket says 172.21.15.53:57464) 2024-05-10T10:37:02.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.023+0000 7f701ac99640 1 -- 172.21.15.53:0/1412455233 learned_addr learned my addr 172.21.15.53:0/1412455233 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1821000910 0 0) 0x7f701406e720 con 0x7f70140d2190 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ffc003520 con 0x7f70140d2190 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f701a498640 1 --2- 172.21.15.53:0/1412455233 >> v2:172.21.15.135:3300/0 conn(0x7f70140cf3e0 0x7f70140cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382702872 0 0) 0x7f6ffc003520 con 0x7f70140d2190 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 >> v2:172.21.15.135:3300/0 conn(0x7f70140cf3e0 msgr2=0x7f70140cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 --2- 172.21.15.53:0/1412455233 >> v2:172.21.15.135:3300/0 conn(0x7f70140cf3e0 0x7f70140cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 >> v1:172.21.15.135:6789/0 conn(0x7f70140cd500 legacy=0x7f70140b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701406e440 con 0x7f70140d2190 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f700c001670 con 0x7f70140d2190 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 >> v1:172.21.15.53:6789/0 conn(0x7f70140d2190 legacy=0x7f70140b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ffc0039b0 0x7f6ffc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc004800 0x7f6ffc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ffc0051c0 0x7f6ffc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f701406e160 con 0x7f6ffc0051c0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00c060 con 0x7f6ffc0039b0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6ffc00c260 con 0x7f6ffc004800 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f701ac99640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ffc0051c0 0x7f6ffc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f701a498640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ffc0039b0 0x7f6ffc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.024+0000 7f7019c97640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc004800 0x7f6ffc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.025+0000 7f701ac99640 1 -- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc004800 msgr2=0x7f6ffc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.025+0000 7f701ac99640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc004800 0x7f6ffc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.025+0000 7f701ac99640 1 -- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ffc0039b0 msgr2=0x7f6ffc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.025+0000 7f701ac99640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ffc0039b0 0x7f6ffc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.025+0000 7f701ac99640 1 -- 172.21.15.53:0/1412455233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc003520 con 0x7f6ffc0051c0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.025+0000 7f701ac99640 1 -- 172.21.15.53:0/1412455233 reap_dead start 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.025+0000 7f701ac99640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ffc0051c0 0x7f6ffc00b9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c210 tx=0x7f6ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94104e9325267bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c007d30 con 0x7f6ffc0051c0 2024-05-10T10:37:02.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c005af0 con 0x7f6ffc0051c0 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f7019496640 1 -- 172.21.15.53:0/1412455233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c007070 con 0x7f6ffc0051c0 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f701c6fc640 1 -- 172.21.15.53:0/1412455233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ffc0051c0 msgr2=0x7f6ffc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f701c6fc640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ffc0051c0 0x7f6ffc00b9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c210 tx=0x7f6ffc005830 comp rx=0 tx=0).stop 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f701c6fc640 1 -- 172.21.15.53:0/1412455233 shutdown_connections 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f701c6fc640 1 --2- 172.21.15.53:0/1412455233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ffc0051c0 0x7f6ffc00b9c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f701c6fc640 1 -- 172.21.15.53:0/1412455233 >> 172.21.15.53:0/1412455233 conn(0x7f70140d2eb0 msgr2=0x7f70140d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f701c6fc640 1 -- 172.21.15.53:0/1412455233 shutdown_connections 2024-05-10T10:37:02.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.026+0000 7f701c6fc640 1 -- 172.21.15.53:0/1412455233 wait complete. 2024-05-10T10:37:02.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.027+0000 7f701c6fc640 1 Processor -- start 2024-05-10T10:37:02.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.027+0000 7f701c6fc640 1 -- start start 2024-05-10T10:37:02.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701c6fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc0051c0 0x7f70141f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701c6fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70141f8a20 0x7f701414edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701c6fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70140cd950 0x7f70140cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701c6fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f70140cf8c0 con 0x7f70140cd950 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701c6fc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b4510 con 0x7f70141f8a20 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701c6fc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f70140cfa40 con 0x7f6ffc0051c0 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701ac99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70140cd950 0x7f70140cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701ac99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70140cd950 0x7f70140cddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59552/0 (socket says 172.21.15.53:59552) 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.028+0000 7f701ac99640 1 -- 172.21.15.53:0/4092706742 learned_addr learned my addr 172.21.15.53:0/4092706742 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f7019c97640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70141f8a20 0x7f701414edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701a498640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc0051c0 0x7f70141f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701ac99640 1 -- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc0051c0 msgr2=0x7f70141f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701ac99640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc0051c0 0x7f70141f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701ac99640 1 -- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70141f8a20 msgr2=0x7f701414edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701ac99640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70141f8a20 0x7f701414edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701ac99640 1 -- 172.21.15.53:0/4092706742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701414fcf0 con 0x7f70140cd950 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f7019c97640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70141f8a20 0x7f701414edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:02.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701a498640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc0051c0 0x7f70141f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:02.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701ac99640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70140cd950 0x7f70140cddd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c330 tx=0x7f700c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f7012ffd640 1 -- 172.21.15.53:0/4092706742 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c040020 con 0x7f70140cd950 2024-05-10T10:37:02.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f7012ffd640 1 -- 172.21.15.53:0/4092706742 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c007270 con 0x7f70140cd950 2024-05-10T10:37:02.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f7012ffd640 1 -- 172.21.15.53:0/4092706742 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c007640 con 0x7f70140cd950 2024-05-10T10:37:02.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701414f730 con 0x7f70140cd950 2024-05-10T10:37:02.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.029+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701415e370 con 0x7f70140cd950 2024-05-10T10:37:02.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.031+0000 7f7012ffd640 1 -- 172.21.15.53:0/4092706742 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f700c043020 con 0x7f70140cd950 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.031+0000 7f7012ffd640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7000076300 0x7f70000787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.031+0000 7f7012ffd640 1 -- 172.21.15.53:0/4092706742 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f700c0b7970 con 0x7f70140cd950 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.032+0000 7f701a498640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7000076300 0x7f70000787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.032+0000 7f701f414740 1 Processor -- start 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.032+0000 7f701a498640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7000076300 0x7f70000787c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f70141f7d70 tx=0x7f700400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.032+0000 7f701f414740 1 -- start start 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadcab160 0x559cadc5daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadc8e280 0x559cadc8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701f414740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8db40 0x559cadc62950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701f414740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559cadc2a690 con 0x559cadc8db40 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cadbc7f30 con 0x559cadc8e280 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x559cadb80b20 con 0x559cadcab160 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701ac99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8db40 0x559cadc62950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701ac99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8db40 0x559cadc62950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59562/0 (socket says 172.21.15.53:59562) 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701ac99640 1 -- 172.21.15.53:0/3795350119 learned_addr learned my addr 172.21.15.53:0/3795350119 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f701a498640 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadcab160 0x559cadc5daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.033+0000 7f7019c97640 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadc8e280 0x559cadc8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f701ac99640 1 -- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadcab160 msgr2=0x559cadc5daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f701ac99640 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadcab160 0x559cadc5daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f701ac99640 1 -- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadc8e280 msgr2=0x559cadc8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f701ac99640 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadc8e280 0x559cadc8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f701ac99640 1 -- 172.21.15.53:0/3795350119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cadb78e60 con 0x559cadc8db40 2024-05-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f701ac99640 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8db40 0x559cadc62950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f700c001970 tx=0x7f700c07d1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fb9abc7aea7e5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f6ffbfff640 1 -- 172.21.15.53:0/3795350119 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c080470 con 0x559cadc8db40 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f6ffbfff640 1 -- 172.21.15.53:0/3795350119 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c09a070 con 0x559cadc8db40 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.034+0000 7f6ffbfff640 1 -- 172.21.15.53:0/3795350119 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c080290 con 0x559cadc8db40 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 -- 172.21.15.53:0/3795350119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8db40 msgr2=0x559cadc62950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8db40 0x559cadc62950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f700c001970 tx=0x7f700c07d1f0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 -- 172.21.15.53:0/3795350119 shutdown_connections 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8db40 0x559cadc62950 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadc8e280 0x559cadc8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 --2- 172.21.15.53:0/3795350119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadcab160 0x559cadc5daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 -- 172.21.15.53:0/3795350119 >> 172.21.15.53:0/3795350119 conn(0x559cadc5fd60 msgr2=0x559cadca8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 -- 172.21.15.53:0/3795350119 shutdown_connections 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 -- 172.21.15.53:0/3795350119 wait complete. 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 -- start start 2024-05-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.035+0000 7f701f414740 1 Processor -- start 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.036+0000 7f701f414740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8e280 0x559cadc89080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.036+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadc8e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.036+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8a160 0x559cadc8a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.036+0000 7f701f414740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559cadb845e0 con 0x559cadc8e280 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.036+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cadbc3ef0 con 0x559cadcab160 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.036+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x559cadb80b20 con 0x559cadc8a160 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.036+0000 7f701f414740 10 client.0 _ll_register_callbacks cb 0x559cadb7f320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8e280 0x559cadc89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8e280 0x559cadc89080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59564/0 (socket says 172.21.15.53:59564) 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 -- 172.21.15.53:0/603139148 learned_addr learned my addr 172.21.15.53:0/603139148 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f7019c97640 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadc8e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701ac99640 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8a160 0x559cadc8a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 -- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8a160 msgr2=0x559cadc8a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8a160 0x559cadc8a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 -- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 msgr2=0x559cadc8e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadc8e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 -- 172.21.15.53:0/603139148 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cad9a37d0 con 0x559cadc8e280 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701a498640 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8e280 0x559cadc89080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7004016460 tx=0x7f7004004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb4cff488b1ac07 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f6ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f6ff97fa640 1 -- 172.21.15.53:0/603139148 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004027070 con 0x559cadc8e280 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f6ff97fa640 1 -- 172.21.15.53:0/603139148 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f7004023820 con 0x559cadc8e280 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f700c09a070 con 0x559cadc8e280 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f6ff97fa640 1 -- 172.21.15.53:0/603139148 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7004032050 con 0x559cadc8e280 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701f414740 10 client.5928 fetch_fsmap learned FSMap version 36 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.037+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x559cad8d93f0 con 0x559cadc8e280 2024-05-10T10:37:02.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f6ff97fa640 1 -- 172.21.15.53:0/603139148 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c09a070 con 0x559cadc8e280 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f6ff97fa640 1 -- 172.21.15.53:0/603139148 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f7004035020 con 0x559cadc8e280 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f701f414740 10 client.5928 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f701f414740 10 client.5928 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f701f414740 20 client.5928 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f701f414740 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadc8ba90 0x559cadc8bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f701f414740 4 client.5928 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cadbd5b40 con 0x559cadc8ba90 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f7019c97640 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadc8ba90 0x559cadc8bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f6ff97fa640 10 client.5928 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.038+0000 7f7019c97640 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadc8ba90 0x559cadc8bee0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c27076b3c3ddff server_cookie=3699b93df23b013a in_seq=0 out_seq=0 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f6ff97fa640 1 -- 172.21.15.53:0/603139148 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559cadbd5b40 con 0x559cadc8ba90 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f6ff97fa640 10 client.5928 handle_command_reply: tid=0 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 client.5928 shutdown 2024-05-10T10:37:02.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8e280 msgr2=0x559cadc89080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8e280 0x559cadc89080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7004016460 tx=0x7f7004004ab0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 shutdown_connections 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadc8ba90 0x559cadc8bee0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8a160 0x559cadc8a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadc8e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f6ff97fa640 0 client.5928 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 --2- 172.21.15.53:0/603139148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8e280 0x559cadc89080 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 >> 172.21.15.53:0/603139148 conn(0x559cadc8db40 msgr2=0x559cadc61650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 shutdown_connections 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 1 -- 172.21.15.53:0/603139148 wait complete. 2024-05-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.039+0000 7f701f414740 20 client.5928 trim_cache size 0 max 16384 2024-05-10T10:37:02.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 Processor -- start 2024-05-10T10:37:02.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 -- start start 2024-05-10T10:37:02.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadccbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadc5e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadc5ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7014202740 con 0x559cadc8a160 2024-05-10T10:37:02.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7014201720 con 0x559cadcab160 2024-05-10T10:37:02.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7014200a90 con 0x559cadc8e280 2024-05-10T10:37:02.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701a498640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadccbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701a498640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadccbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59574/0 (socket says 172.21.15.53:59574) 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.058+0000 7f701a498640 1 -- 172.21.15.53:0/683309127 learned_addr learned my addr 172.21.15.53:0/683309127 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f701a498640 1 -- 172.21.15.53:0/683309127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 msgr2=0x559cadc5e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f701a498640 1 --2- 172.21.15.53:0/683309127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadc5e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f701a498640 1 -- 172.21.15.53:0/683309127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 msgr2=0x559cadc5ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f701a498640 1 --2- 172.21.15.53:0/683309127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadc5ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f701a498640 1 -- 172.21.15.53:0/683309127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cadc8eae0 con 0x559cadc8a160 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f701a498640 1 --2- 172.21.15.53:0/683309127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadccbcd0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7004016430 tx=0x7f7004023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85143665cfdb962 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f6ffb7fe640 1 -- 172.21.15.53:0/683309127 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004027070 con 0x559cadc8a160 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f6ffb7fe640 1 -- 172.21.15.53:0/683309127 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700401c040 con 0x559cadc8a160 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.059+0000 7f6ffb7fe640 1 -- 172.21.15.53:0/683309127 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004023a20 con 0x559cadc8a160 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 -- 172.21.15.53:0/683309127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 msgr2=0x559cadccbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 --2- 172.21.15.53:0/683309127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadccbcd0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7004016430 tx=0x7f7004023000 comp rx=0 tx=0).stop 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 -- 172.21.15.53:0/683309127 shutdown_connections 2024-05-10T10:37:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 --2- 172.21.15.53:0/683309127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadc5ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 --2- 172.21.15.53:0/683309127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadc5e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 --2- 172.21.15.53:0/683309127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadccbcd0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 -- 172.21.15.53:0/683309127 >> 172.21.15.53:0/683309127 conn(0x559cadc8db40 msgr2=0x559cadc909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 -- 172.21.15.53:0/683309127 shutdown_connections 2024-05-10T10:37:02.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 -- 172.21.15.53:0/683309127 wait complete. 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 -- start start 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.060+0000 7f701f414740 1 Processor -- start 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701f414740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadd24cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadccb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701f414740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadccb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701f414740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559cadc2a690 con 0x559cadc8a160 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7014201090 con 0x559cadcab160 2024-05-10T10:37:02.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701f414740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x559cadbc7f30 con 0x559cadc8e280 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701f414740 10 client.0 _ll_register_callbacks cb 0x7f7004015450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701a498640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadd24cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701a498640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadd24cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59590/0 (socket says 172.21.15.53:59590) 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.061+0000 7f701a498640 1 -- 172.21.15.53:0/3193667610 learned_addr learned my addr 172.21.15.53:0/3193667610 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f7019c97640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadccb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701a498640 1 -- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 msgr2=0x559cadccb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701a498640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadccb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701a498640 1 -- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 msgr2=0x559cadccb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701ac99640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadccb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701a498640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadccb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701a498640 1 -- 172.21.15.53:0/3193667610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cadc8a950 con 0x559cadc8a160 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f7019c97640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadccb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701ac99640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadccb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701a498640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadd24cf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7004007d80 tx=0x7f7004019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c001cc8a6c8293 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f6ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f6ff97fa640 1 -- 172.21.15.53:0/3193667610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004027040 con 0x559cadc8a160 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f6ff97fa640 1 -- 172.21.15.53:0/3193667610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f7004042d50 con 0x559cadc8a160 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f700401c040 con 0x559cadc8a160 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f6ff97fa640 1 -- 172.21.15.53:0/3193667610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7004023230 con 0x559cadc8a160 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 10 client.5934 fetch_fsmap learned FSMap version 36 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x559cad8990f0 con 0x559cadc8a160 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f6ff97fa640 1 -- 172.21.15.53:0/3193667610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700401c040 con 0x559cadc8a160 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f6ff97fa640 1 -- 172.21.15.53:0/3193667610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f7004035070 con 0x559cadc8a160 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 10 client.5934 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 10 client.5934 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 20 client.5934 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadcb7150 0x559cadcb7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 4 client.5934 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.062+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559cadbcf9b0 con 0x559cadcb7150 2024-05-10T10:37:02.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f7019c97640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadcb7150 0x559cadcb7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f6ff97fa640 10 client.5934 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f7019c97640 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadcb7150 0x559cadcb7530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f6455d4ab04240 server_cookie=60efed7ecbe7a4e in_seq=0 out_seq=0 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f6ff97fa640 1 -- 172.21.15.53:0/3193667610 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559cadbcf9b0 con 0x559cadcb7150 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f6ff97fa640 10 client.5934 handle_command_reply: tid=0 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 client.5934 shutdown 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 msgr2=0x559cadd24cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadd24cf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7004007d80 tx=0x7f7004019000 comp rx=0 tx=0).stop 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 shutdown_connections 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x559cadcb7150 0x559cadcb7530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cadcab160 0x559cadccb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x559cadc8e280 0x559cadccb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 --2- 172.21.15.53:0/3193667610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559cadc8a160 0x559cadd24cf0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f6ff97fa640 0 client.5934 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 >> 172.21.15.53:0/3193667610 conn(0x559cadc8db40 msgr2=0x559cadc5e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 shutdown_connections 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 1 -- 172.21.15.53:0/3193667610 wait complete. 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.063+0000 7f701f414740 20 client.5934 trim_cache size 0 max 16384 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:37:02.071 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:37:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7000076300 msgr2=0x7f70000787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7000076300 0x7f70000787c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f70141f7d70 tx=0x7f700400b040 comp rx=0 tx=0).stop 2024-05-10T10:37:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70140cd950 msgr2=0x7f70140cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70140cd950 0x7f70140cddd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c330 tx=0x7f700c001a40 comp rx=0 tx=0).stop 2024-05-10T10:37:02.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 shutdown_connections 2024-05-10T10:37:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7000076300 0x7f70000787c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70140cd950 0x7f70140cddd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70141f8a20 0x7f701414edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 --2- 172.21.15.53:0/4092706742 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ffc0051c0 0x7f70141f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 >> 172.21.15.53:0/4092706742 conn(0x7f70140d2eb0 msgr2=0x7f70140757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 shutdown_connections 2024-05-10T10:37:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.064+0000 7f701c6fc640 1 -- 172.21.15.53:0/4092706742 wait complete. 2024-05-10T10:37:02.076 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:37:02.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:02.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.178+0000 7f1d6c5e7640 1 Processor -- start 2024-05-10T10:37:02.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.178+0000 7f1d6c5e7640 1 -- start start 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6c5e7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d640b8860 0x7f1d6406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6c5e7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d640cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6c5e7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d6415fc90 con 0x7f1d640cfbb0 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6c5e7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6406ee50 con 0x7f1d640b8860 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6c5e7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640b8c40 con 0x7f1d640d0230 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6a383640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d640b8860 0x7f1d6406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6a383640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d640b8860 0x7f1d6406e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59600/0 (socket says 172.21.15.53:59600) 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6a383640 1 -- 172.21.15.53:0/2339446355 learned_addr learned my addr 172.21.15.53:0/2339446355 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d69b82640 1 --2- 172.21.15.53:0/2339446355 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d640cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6a383640 1 -- 172.21.15.53:0/2339446355 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 msgr2=0x7f1d640cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6a383640 1 --2- 172.21.15.53:0/2339446355 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d640cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6a383640 1 -- 172.21.15.53:0/2339446355 >> v1:172.21.15.135:6789/0 conn(0x7f1d640cfbb0 legacy=0x7f1d640d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.179+0000 7f1d6a383640 1 -- 172.21.15.53:0/2339446355 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d640cd570 con 0x7f1d640b8860 2024-05-10T10:37:02.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.180+0000 7f1d6a383640 1 --2- 172.21.15.53:0/2339446355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640b8860 0x7f1d6406e520 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009870 tx=0x7f1d6002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c900ab99721bdb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.180+0000 7f1d69381640 1 -- 172.21.15.53:0/2339446355 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60037070 con 0x7f1d640b8860 2024-05-10T10:37:02.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.180+0000 7f1d69381640 1 -- 172.21.15.53:0/2339446355 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60032440 con 0x7f1d640b8860 2024-05-10T10:37:02.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.180+0000 7f1d69381640 1 -- 172.21.15.53:0/2339446355 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60032810 con 0x7f1d640b8860 2024-05-10T10:37:02.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/2339446355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640b8860 msgr2=0x7f1d6406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/2339446355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640b8860 0x7f1d6406e520 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009870 tx=0x7f1d6002a180 comp rx=0 tx=0).stop 2024-05-10T10:37:02.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/2339446355 shutdown_connections 2024-05-10T10:37:02.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/2339446355 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d640cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/2339446355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640b8860 0x7f1d6406e520 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/2339446355 >> 172.21.15.53:0/2339446355 conn(0x7f1d640d2df0 msgr2=0x7f1d640d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/2339446355 shutdown_connections 2024-05-10T10:37:02.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.181+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/2339446355 wait complete. 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.182+0000 7f1d6c5e7640 1 Processor -- start 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.182+0000 7f1d6c5e7640 1 -- start start 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6c5e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d640b8860 0x7f1d640c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6c5e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640cfbb0 0x7f1d6414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6c5e7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d6414c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6c5e7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640cde10 con 0x7f1d640cfbb0 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6c5e7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640cdb30 con 0x7f1d640b8860 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6c5e7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6406ea60 con 0x7f1d640d0230 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d69b82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640cfbb0 0x7f1d6414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d69b82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640cfbb0 0x7f1d6414c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59610/0 (socket says 172.21.15.53:59610) 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d69b82640 1 -- 172.21.15.53:0/1060193886 learned_addr learned my addr 172.21.15.53:0/1060193886 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6a383640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d640b8860 0x7f1d640c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.183+0000 7f1d6ab84640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d6414c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d69b82640 1 -- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 msgr2=0x7f1d6414c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d69b82640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d6414c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d69b82640 1 -- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d640b8860 msgr2=0x7f1d640c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d69b82640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d640b8860 0x7f1d640c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d69b82640 1 -- 172.21.15.53:0/1060193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d640c8380 con 0x7f1d640cfbb0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d69b82640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640cfbb0 0x7f1d6414c100 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c820 tx=0x7f1d5400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d5affd640 1 -- 172.21.15.53:0/1060193886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54015070 con 0x7f1d640cfbb0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d5affd640 1 -- 172.21.15.53:0/1060193886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d54004240 con 0x7f1d640cfbb0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d5affd640 1 -- 172.21.15.53:0/1060193886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400ce10 con 0x7f1d640cfbb0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d640c8660 con 0x7f1d640cfbb0 2024-05-10T10:37:02.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.184+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d640c8f00 con 0x7f1d640cfbb0 2024-05-10T10:37:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.185+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640c8a50 con 0x7f1d640cfbb0 2024-05-10T10:37:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.186+0000 7f1d5affd640 1 -- 172.21.15.53:0/1060193886 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1d54007830 con 0x7f1d640cfbb0 2024-05-10T10:37:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.186+0000 7f1d5affd640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.186+0000 7f1d5affd640 1 -- 172.21.15.53:0/1060193886 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f1d54097620 con 0x7f1d640cfbb0 2024-05-10T10:37:02.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.186+0000 7f1d6a383640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.187+0000 7f1d6a383640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d60034000 tx=0x7f1d6002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.191+0000 7f1d5affd640 1 -- 172.21.15.53:0/1060193886 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1d54060710 con 0x7f1d640cfbb0 2024-05-10T10:37:02.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.427+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d6406a220 con 0x7f1d640cfbb0 2024-05-10T10:37:02.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.428+0000 7f1d5affd640 1 -- 172.21.15.53:0/1060193886 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5189 (secure 0 0 0) 0x7f1d54064a10 con 0x7f1d640cfbb0 2024-05-10T10:37:02.429 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:02.429 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:36:32.408868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":13,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:37:02.430 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 msgr2=0x7f1d48078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d60034000 tx=0x7f1d6002e000 comp rx=0 tx=0).stop 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640cfbb0 msgr2=0x7f1d6414c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640cfbb0 0x7f1d6414c100 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c820 tx=0x7f1d5400cce0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 shutdown_connections 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d640d0230 0x7f1d6414c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d640cfbb0 0x7f1d6414c100 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 --2- 172.21.15.53:0/1060193886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d640b8860 0x7f1d640c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 >> 172.21.15.53:0/1060193886 conn(0x7f1d640d2df0 msgr2=0x7f1d640ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 shutdown_connections 2024-05-10T10:37:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.430+0000 7f1d6c5e7640 1 -- 172.21.15.53:0/1060193886 wait complete. 2024-05-10T10:37:02.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-10T10:37:02.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.547+0000 7f3b81064640 1 Processor -- start 2024-05-10T10:37:02.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.547+0000 7f3b81064640 1 -- start start 2024-05-10T10:37:02.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b81064640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b7c0cf3e0 0x7f3b7c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b81064640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b7c0cd500 0x7f3b7c0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b81064640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b81064640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7c06e720 con 0x7f3b7c0cd500 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b81064640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7c06de80 con 0x7f3b7c0cf3e0 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b81064640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7c06e160 con 0x7f3b7c0b2dd0 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b7c0cd500 0x7f3b7c0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b7c0cd500 0x7f3b7c0cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59624/0 (socket says 172.21.15.53:59624) 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 -- 172.21.15.53:0/2267673414 learned_addr learned my addr 172.21.15.53:0/2267673414 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/2267673414 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/2267673414 >> v2:172.21.15.135:3300/0 conn(0x7f3b7c0cf3e0 0x7f3b7c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 -- 172.21.15.53:0/2267673414 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 msgr2=0x7f3b7c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 --2- 172.21.15.53:0/2267673414 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 -- 172.21.15.53:0/2267673414 >> v2:172.21.15.135:3300/0 conn(0x7f3b7c0cf3e0 msgr2=0x7f3b7c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 --2- 172.21.15.53:0/2267673414 >> v2:172.21.15.135:3300/0 conn(0x7f3b7c0cf3e0 0x7f3b7c0cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7259c640 1 -- 172.21.15.53:0/2267673414 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7c06e440 con 0x7f3b7c0cd500 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/2267673414 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:02.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.548+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/2267673414 >> v2:172.21.15.135:3300/0 conn(0x7f3b7c0cf3e0 0x7f3b7c0cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.549+0000 7f3b7259c640 1 --2- 172.21.15.53:0/2267673414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cfe00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c06fed0 tx=0x7f3b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bce1c3be2143f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.549+0000 7f3b7a59c640 1 -- 172.21.15.53:0/2267673414 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64037070 con 0x7f3b7c0cd500 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.549+0000 7f3b7a59c640 1 -- 172.21.15.53:0/2267673414 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64032440 con 0x7f3b7c0cd500 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.549+0000 7f3b7a59c640 1 -- 172.21.15.53:0/2267673414 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64032810 con 0x7f3b7c0cd500 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 -- 172.21.15.53:0/2267673414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 msgr2=0x7f3b7c0cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 --2- 172.21.15.53:0/2267673414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cfe00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c06fed0 tx=0x7f3b6402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 -- 172.21.15.53:0/2267673414 shutdown_connections 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 --2- 172.21.15.53:0/2267673414 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 --2- 172.21.15.53:0/2267673414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cfe00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 --2- 172.21.15.53:0/2267673414 >> v2:172.21.15.135:3300/0 conn(0x7f3b7c0cf3e0 0x7f3b7c0cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 -- 172.21.15.53:0/2267673414 >> 172.21.15.53:0/2267673414 conn(0x7f3b7c0d2eb0 msgr2=0x7f3b7c0d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 -- 172.21.15.53:0/2267673414 shutdown_connections 2024-05-10T10:37:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.550+0000 7f3b81064640 1 -- 172.21.15.53:0/2267673414 wait complete. 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.551+0000 7f3b81064640 1 Processor -- start 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.551+0000 7f3b81064640 1 -- start start 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b81064640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b81064640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b81064640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b7c0cf3e0 0x7f3b7c154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b81064640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c0b37a0 con 0x7f3b7c0cd500 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b81064640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c0b4510 con 0x7f3b7c0cf3e0 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b81064640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7c0b3920 con 0x7f3b7c0b2dd0 2024-05-10T10:37:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59638/0 (socket says 172.21.15.53:59638) 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 -- 172.21.15.53:0/610961550 learned_addr learned my addr 172.21.15.53:0/610961550 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b7c0cf3e0 0x7f3b7c154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 -- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 msgr2=0x7f3b7c0b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 -- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b7c0cf3e0 msgr2=0x7f3b7c154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b7c0cf3e0 0x7f3b7c154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.552+0000 7f3b7259c640 1 -- 172.21.15.53:0/610961550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7c0cc4b0 con 0x7f3b7c0cd500 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.553+0000 7f3b7259c640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cbf70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3b6402a220 tx=0x7f3b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.553+0000 7f3b73fff640 1 -- 172.21.15.53:0/610961550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64037040 con 0x7f3b7c0cd500 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.553+0000 7f3b73fff640 1 -- 172.21.15.53:0/610961550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64042070 con 0x7f3b7c0cd500 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.553+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7c1549a0 con 0x7f3b7c0cd500 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.553+0000 7f3b73fff640 1 -- 172.21.15.53:0/610961550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64030bb0 con 0x7f3b7c0cd500 2024-05-10T10:37:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.553+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b7c0b83f0 con 0x7f3b7c0cd500 2024-05-10T10:37:02.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.554+0000 7f3b73fff640 1 -- 172.21.15.53:0/610961550 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b64047460 con 0x7f3b7c0cd500 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.555+0000 7f3b73fff640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b5c0762d0 0x7f3b5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.555+0000 7f3b73fff640 1 -- 172.21.15.53:0/610961550 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3b640b84d0 con 0x7f3b7c0cd500 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.555+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b5c0762d0 0x7f3b5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.556+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b5c0762d0 0x7f3b5c078790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00a950 tx=0x7f3b6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.556+0000 7f3b83d7d740 1 Processor -- start 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.556+0000 7f3b83d7d740 1 -- start start 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41ca86d0 0x561f41c5b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c8b7f0 0x561f41c8ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b0b0 0x561f41c5fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561f41b4b920 con 0x561f41c8b0b0 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f41bc0dd0 con 0x561f41ca86d0 2024-05-10T10:37:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561f41b713e0 con 0x561f41c8b7f0 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b0b0 0x561f41c5fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b0b0 0x561f41c5fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59654/0 (socket says 172.21.15.53:59654) 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 -- 172.21.15.53:0/845543396 learned_addr learned my addr 172.21.15.53:0/845543396 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7259c640 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c8b7f0 0x561f41c8ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41ca86d0 0x561f41c5b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 -- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c8b7f0 msgr2=0x561f41c8ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c8b7f0 0x561f41c8ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 -- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41ca86d0 msgr2=0x561f41c5b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41ca86d0 0x561f41c5b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.557+0000 7f3b7b59e640 1 -- 172.21.15.53:0/845543396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f419a27c0 con 0x561f41c8b0b0 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b0b0 0x561f41c5fec0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400e520 tx=0x7f3b7400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8209bc1c921546ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b70d99640 1 -- 172.21.15.53:0/845543396 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7400c9f0 con 0x561f41c8b0b0 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b70d99640 1 -- 172.21.15.53:0/845543396 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7400cb90 con 0x561f41c8b0b0 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b70d99640 1 -- 172.21.15.53:0/845543396 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7400a410 con 0x561f41c8b0b0 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b83d7d740 1 -- 172.21.15.53:0/845543396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b0b0 msgr2=0x561f41c5fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b0b0 0x561f41c5fec0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400e520 tx=0x7f3b7400e9e0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b83d7d740 1 -- 172.21.15.53:0/845543396 shutdown_connections 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b0b0 0x561f41c5fec0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c8b7f0 0x561f41c8ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/845543396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41ca86d0 0x561f41c5b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.558+0000 7f3b83d7d740 1 -- 172.21.15.53:0/845543396 >> 172.21.15.53:0/845543396 conn(0x561f41c5d560 msgr2=0x561f41ca5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.559+0000 7f3b83d7d740 1 -- 172.21.15.53:0/845543396 shutdown_connections 2024-05-10T10:37:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.559+0000 7f3b83d7d740 1 -- 172.21.15.53:0/845543396 wait complete. 2024-05-10T10:37:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.559+0000 7f3b83d7d740 1 -- start start 2024-05-10T10:37:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.559+0000 7f3b83d7d740 1 Processor -- start 2024-05-10T10:37:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41c89f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41ca86d0 0x561f41ca6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c7afa0 0x561f41c7b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561f41bc9440 con 0x561f41ca86d0 2024-05-10T10:37:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f41bc4120 con 0x561f41c8b7f0 2024-05-10T10:37:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561f41bd2210 con 0x561f41c7afa0 2024-05-10T10:37:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b83d7d740 10 client.0 _ll_register_callbacks cb 0x561f41b69690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41ca86d0 0x561f41ca6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41ca86d0 0x561f41ca6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59656/0 (socket says 172.21.15.53:59656) 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 -- 172.21.15.53:0/3405883428 learned_addr learned my addr 172.21.15.53:0/3405883428 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c7afa0 0x561f41c7b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41c89f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 -- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c7afa0 msgr2=0x561f41c7b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c7afa0 0x561f41c7b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 -- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 msgr2=0x561f41c89f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41c89f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 -- 172.21.15.53:0/3405883428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f41a02f50 con 0x561f41ca86d0 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41c89f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b7259c640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41ca86d0 0x561f41ca6580 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3b6407da60 tx=0x7f3b64080f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702a0fb47233a2f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.560+0000 7f3b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b59ffb640 1 -- 172.21.15.53:0/3405883428 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b640818d0 con 0x561f41ca86d0 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b59ffb640 1 -- 172.21.15.53:0/3405883428 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3b64089db0 con 0x561f41ca86d0 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b7400cb90 con 0x561f41ca86d0 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b59ffb640 1 -- 172.21.15.53:0/3405883428 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b640b82c0 con 0x561f41ca86d0 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 10 client.5952 fetch_fsmap learned FSMap version 36 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x561f418960b0 con 0x561f41ca86d0 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b59ffb640 1 -- 172.21.15.53:0/3405883428 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7400cb90 con 0x561f41ca86d0 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b59ffb640 1 -- 172.21.15.53:0/3405883428 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f3b6403a030 con 0x561f41ca86d0 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 10 client.5952 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 10 client.5952 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 20 client.5952 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41c9d5f0 0x561f41c9d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 4 client.5952 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f41bd2a20 con 0x561f41c9d5f0 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.561+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41c9d5f0 0x561f41c9d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b59ffb640 10 client.5952 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41c9d5f0 0x561f41c9d9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c6f7481a1113b8 server_cookie=d746ec3f0567f485 in_seq=0 out_seq=0 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b59ffb640 1 -- 172.21.15.53:0/3405883428 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f41bd2a20 con 0x561f41c9d5f0 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b59ffb640 10 client.5952 handle_command_reply: tid=0 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 client.5952 shutdown 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41ca86d0 msgr2=0x561f41ca6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41ca86d0 0x561f41ca6580 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3b6407da60 tx=0x7f3b64080f80 comp rx=0 tx=0).stop 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 shutdown_connections 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41c9d5f0 0x561f41c9d9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41c7afa0 0x561f41c7b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41ca86d0 0x561f41ca6580 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3405883428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41c89f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 >> 172.21.15.53:0/3405883428 conn(0x561f41c8b0b0 msgr2=0x561f41c5aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b59ffb640 0 client.5952 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 shutdown_connections 2024-05-10T10:37:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.562+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3405883428 wait complete. 2024-05-10T10:37:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.563+0000 7f3b83d7d740 20 client.5952 trim_cache size 0 max 16384 2024-05-10T10:37:02.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.577+0000 7f3b83d7d740 1 Processor -- start 2024-05-10T10:37:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.577+0000 7f3b83d7d740 1 -- start start 2024-05-10T10:37:02.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c7afa0 0x561f41c7b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b7f0 0x561f41ca7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41ca4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c201050 con 0x561f41c8b7f0 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c0b6360 con 0x561f41c7afa0 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7c2009b0 con 0x561f41ca86d0 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b7f0 0x561f41ca7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b7f0 0x561f41ca7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59660/0 (socket says 172.21.15.53:59660) 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 -- 172.21.15.53:0/3481738549 learned_addr learned my addr 172.21.15.53:0/3481738549 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41ca4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c7afa0 0x561f41c7b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 -- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 msgr2=0x561f41ca4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41ca4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 -- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c7afa0 msgr2=0x561f41c7b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c7afa0 0x561f41c7b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 -- 172.21.15.53:0/3481738549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f41c55d40 con 0x561f41c8b7f0 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.578+0000 7f3b7259c640 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b7f0 0x561f41ca7780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b6402a7d0 tx=0x7f3b64049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad17521277bfef96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b5bfff640 1 -- 172.21.15.53:0/3481738549 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b640b82c0 con 0x561f41c8b7f0 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b5bfff640 1 -- 172.21.15.53:0/3481738549 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6408a240 con 0x561f41c8b7f0 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b5bfff640 1 -- 172.21.15.53:0/3481738549 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b640b63f0 con 0x561f41c8b7f0 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3481738549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b7f0 msgr2=0x561f41ca7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b7f0 0x561f41ca7780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b6402a7d0 tx=0x7f3b64049000 comp rx=0 tx=0).stop 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3481738549 shutdown_connections 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41ca4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c8b7f0 0x561f41ca7780 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/3481738549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c7afa0 0x561f41c7b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3481738549 >> 172.21.15.53:0/3481738549 conn(0x561f41c8b0b0 msgr2=0x561f41c8a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3481738549 shutdown_connections 2024-05-10T10:37:02.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.579+0000 7f3b83d7d740 1 -- 172.21.15.53:0/3481738549 wait complete. 2024-05-10T10:37:02.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.580+0000 7f3b83d7d740 1 -- start start 2024-05-10T10:37:02.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.580+0000 7f3b83d7d740 1 Processor -- start 2024-05-10T10:37:02.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c7afa0 0x561f41c61220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41cbef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b83d7d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41c9d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561f41c8ca10 con 0x561f41c7afa0 2024-05-10T10:37:02.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f41b7aa20 con 0x561f41c8b7f0 2024-05-10T10:37:02.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b83d7d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561f41c9e740 con 0x561f41ca86d0 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b83d7d740 10 client.0 _ll_register_callbacks cb 0x7f3b6407d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c7afa0 0x561f41c61220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c7afa0 0x561f41c61220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59662/0 (socket says 172.21.15.53:59662) 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 -- 172.21.15.53:0/1298802061 learned_addr learned my addr 172.21.15.53:0/1298802061 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7b59e640 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41c9d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7259c640 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41cbef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 -- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 msgr2=0x561f41c9d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41c9d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 -- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 msgr2=0x561f41cbef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41cbef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 -- 172.21.15.53:0/1298802061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f41cbe700 con 0x561f41c7afa0 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b7ad9d640 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c7afa0 0x561f41c61220 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x561f41c8cbc0 tx=0x7f3b6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592a619b76342b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.581+0000 7f3b59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b59ffb640 1 -- 172.21.15.53:0/1298802061 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c01a040 con 0x561f41c7afa0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b59ffb640 1 -- 172.21.15.53:0/1298802061 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3b6c0283c0 con 0x561f41c7afa0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6408a240 con 0x561f41c7afa0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b59ffb640 1 -- 172.21.15.53:0/1298802061 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b6c038050 con 0x561f41c7afa0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 10 client.5958 fetch_fsmap learned FSMap version 36 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x561f41d23bb0 con 0x561f41c7afa0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b59ffb640 1 -- 172.21.15.53:0/1298802061 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6408a240 con 0x561f41c7afa0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b59ffb640 1 -- 172.21.15.53:0/1298802061 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f3b6c020040 con 0x561f41c7afa0 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 10 client.5958 fetch_fsmap finished waiting for FSMap version 36 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 10 client.5958 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 20 client.5958 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41d23710 0x561f41c86fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 4 client.5958 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 --> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x561f41bcc890 con 0x561f41d23710 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.582+0000 7f3b7259c640 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41d23710 0x561f41c86fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b7259c640 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41d23710 0x561f41c86fd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8ca9764f630dee server_cookie=692210ef5284e62b in_seq=0 out_seq=0 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b59ffb640 10 client.5958 ms_handle_connect on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b59ffb640 1 -- 172.21.15.53:0/1298802061 <== mds.0 v2:172.21.15.53:6833/1934136965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f41bcc890 con 0x561f41d23710 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b59ffb640 10 client.5958 handle_command_reply: tid=0 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 client.5958 shutdown 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c7afa0 msgr2=0x561f41c61220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c7afa0 0x561f41c61220 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x561f41c8cbc0 tx=0x7f3b6c004730 comp rx=0 tx=0).stop 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 shutdown_connections 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:6833/1934136965,v1:172.21.15.53:6835/1934136965] conn(0x561f41d23710 0x561f41c86fd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x561f41ca86d0 0x561f41c9d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f41c8b7f0 0x561f41cbef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b59ffb640 0 client.5958 ms_handle_reset on v2:172.21.15.53:6833/1934136965 2024-05-10T10:37:02.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 --2- 172.21.15.53:0/1298802061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f41c7afa0 0x561f41c61220 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 >> 172.21.15.53:0/1298802061 conn(0x561f41c8b0b0 msgr2=0x561f41cc9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 shutdown_connections 2024-05-10T10:37:02.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 1 -- 172.21.15.53:0/1298802061 wait complete. 2024-05-10T10:37:02.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.583+0000 7f3b83d7d740 20 client.5958 trim_cache size 0 max 16384 2024-05-10T10:37:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:02.592 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2024-05-10T10:37:02.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b5c0762d0 msgr2=0x7f3b5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b5c0762d0 0x7f3b5c078790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00a950 tx=0x7f3b6c008040 comp rx=0 tx=0).stop 2024-05-10T10:37:02.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 msgr2=0x7f3b7c0cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cbf70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3b6402a220 tx=0x7f3b640046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 shutdown_connections 2024-05-10T10:37:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b5c0762d0 0x7f3b5c078790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b7c0cf3e0 0x7f3b7c154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b7c0cd500 0x7f3b7c0cbf70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 --2- 172.21.15.53:0/610961550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b7c0b2dd0 0x7f3b7c0b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 >> 172.21.15.53:0/610961550 conn(0x7f3b7c0d2eb0 msgr2=0x7f3b7c1624f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 shutdown_connections 2024-05-10T10:37:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.584+0000 7f3b81064640 1 -- 172.21.15.53:0/610961550 wait complete. 2024-05-10T10:37:02.596 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T10:37:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.697+0000 7fe863f32640 1 Processor -- start 2024-05-10T10:37:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.697+0000 7fe863f32640 1 -- start start 2024-05-10T10:37:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe863f32640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe85c162750 0x7fe85c0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe863f32640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe85c0d5cb0 0x7fe85c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe863f32640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe863f32640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe85c0b4f20 con 0x7fe85c162750 2024-05-10T10:37:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe863f32640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe85c0d0fe0 con 0x7fe85c0d5cb0 2024-05-10T10:37:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe863f32640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe85c0b45f0 con 0x7fe85c162b30 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe85c162750 0x7fe85c0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe85c162750 0x7fe85c0d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54486/0 (socket says 172.21.15.53:54486) 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 -- 172.21.15.53:0/1993453495 learned_addr learned my addr 172.21.15.53:0/1993453495 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe8614cd640 1 --2- 172.21.15.53:0/1993453495 >> v2:172.21.15.135:3300/0 conn(0x7fe85c0d5cb0 0x7fe85c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe8624cf640 1 --2- 172.21.15.53:0/1993453495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 -- 172.21.15.53:0/1993453495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 msgr2=0x7fe85c0d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 --2- 172.21.15.53:0/1993453495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 -- 172.21.15.53:0/1993453495 >> v2:172.21.15.135:3300/0 conn(0x7fe85c0d5cb0 msgr2=0x7fe85c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 --2- 172.21.15.53:0/1993453495 >> v2:172.21.15.135:3300/0 conn(0x7fe85c0d5cb0 0x7fe85c0d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe861cce640 1 -- 172.21.15.53:0/1993453495 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c0b48d0 con 0x7fe85c162750 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe8614cd640 1 --2- 172.21.15.53:0/1993453495 >> v2:172.21.15.135:3300/0 conn(0x7fe85c0d5cb0 0x7fe85c0d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:02.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.698+0000 7fe8624cf640 1 --2- 172.21.15.53:0/1993453495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.699+0000 7fe861cce640 1 --2- 172.21.15.53:0/1993453495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c162750 0x7fe85c0d0670 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0b4e90 tx=0x7fe85802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370a27809b26e866 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.699+0000 7fe860ccc640 1 -- 172.21.15.53:0/1993453495 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858037070 con 0x7fe85c162750 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.699+0000 7fe860ccc640 1 -- 172.21.15.53:0/1993453495 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe858032440 con 0x7fe85c162750 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.699+0000 7fe860ccc640 1 -- 172.21.15.53:0/1993453495 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858032810 con 0x7fe85c162750 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 -- 172.21.15.53:0/1993453495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c162750 msgr2=0x7fe85c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 --2- 172.21.15.53:0/1993453495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c162750 0x7fe85c0d0670 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0b4e90 tx=0x7fe85802a140 comp rx=0 tx=0).stop 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 -- 172.21.15.53:0/1993453495 shutdown_connections 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 --2- 172.21.15.53:0/1993453495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 --2- 172.21.15.53:0/1993453495 >> v2:172.21.15.135:3300/0 conn(0x7fe85c0d5cb0 0x7fe85c0d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 --2- 172.21.15.53:0/1993453495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c162750 0x7fe85c0d0670 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 -- 172.21.15.53:0/1993453495 >> 172.21.15.53:0/1993453495 conn(0x7fe85c09e3b0 msgr2=0x7fe85c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 -- 172.21.15.53:0/1993453495 shutdown_connections 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.700+0000 7fe863f32640 1 -- 172.21.15.53:0/1993453495 wait complete. 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.701+0000 7fe863f32640 1 Processor -- start 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.701+0000 7fe863f32640 1 -- start start 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe863f32640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c162750 0x7fe85c0b9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe863f32640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe863f32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c15cae0 0x7fe85c150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe863f32640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c0d0d30 con 0x7fe85c15cae0 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe863f32640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c0d0bb0 con 0x7fe85c162750 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe863f32640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe85c0d1660 con 0x7fe85c162b30 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c15cae0 0x7fe85c150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c15cae0 0x7fe85c150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54490/0 (socket says 172.21.15.53:54490) 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 -- 172.21.15.53:0/125910231 learned_addr learned my addr 172.21.15.53:0/125910231 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8614cd640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe861cce640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c162750 0x7fe85c0b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 -- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 msgr2=0x7fe85c0ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 -- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c162750 msgr2=0x7fe85c0b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c162750 0x7fe85c0b9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 -- 172.21.15.53:0/125910231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c0b9600 con 0x7fe85c15cae0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8614cd640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe861cce640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c162750 0x7fe85c0b9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8624cf640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c15cae0 0x7fe85c150430 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe85400cd50 tx=0x7fe85400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8527fc640 1 -- 172.21.15.53:0/125910231 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85400f680 con 0x7fe85c15cae0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.702+0000 7fe8527fc640 1 -- 172.21.15.53:0/125910231 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85400f840 con 0x7fe85c15cae0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.703+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85c0b98e0 con 0x7fe85c15cae0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.703+0000 7fe8527fc640 1 -- 172.21.15.53:0/125910231 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85400fc30 con 0x7fe85c15cae0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.703+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85c0b9ea0 con 0x7fe85c15cae0 2024-05-10T10:37:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.703+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe85c06a220 con 0x7fe85c15cae0 2024-05-10T10:37:02.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.704+0000 7fe8527fc640 1 -- 172.21.15.53:0/125910231 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe8540213f0 con 0x7fe85c15cae0 2024-05-10T10:37:02.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.705+0000 7fe8527fc640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8400762d0 0x7fe840078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:02.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.705+0000 7fe8527fc640 1 -- 172.21.15.53:0/125910231 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fe854098580 con 0x7fe85c15cae0 2024-05-10T10:37:02.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.705+0000 7fe861cce640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8400762d0 0x7fe840078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:02.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.706+0000 7fe861cce640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8400762d0 0x7fe840078790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe858034000 tx=0x7fe85802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:02.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.709+0000 7fe8527fc640 1 -- 172.21.15.53:0/125910231 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe854014030 con 0x7fe85c15cae0 2024-05-10T10:37:02.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.942+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe85c161d70 con 0x7fe85c15cae0 2024-05-10T10:37:02.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.943+0000 7fe8527fc640 1 -- 172.21.15.53:0/125910231 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe854061cc0 con 0x7fe85c15cae0 2024-05-10T10:37:02.945 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:02.945 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8400762d0 msgr2=0x7fe840078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8400762d0 0x7fe840078790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe858034000 tx=0x7fe85802e000 comp rx=0 tx=0).stop 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c15cae0 msgr2=0x7fe85c150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c15cae0 0x7fe85c150430 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe85400cd50 tx=0x7fe85400a650 comp rx=0 tx=0).stop 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 shutdown_connections 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8400762d0 0x7fe840078790 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c15cae0 0x7fe85c150430 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe85c162b30 0x7fe85c0ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 --2- 172.21.15.53:0/125910231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c162750 0x7fe85c0b9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 >> 172.21.15.53:0/125910231 conn(0x7fe85c09e3b0 msgr2=0x7fe85c0d4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 shutdown_connections 2024-05-10T10:37:02.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:02.945+0000 7fe863f32640 1 -- 172.21.15.53:0/125910231 wait complete. 2024-05-10T10:37:02.957 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T10:37:02.958 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:37:02.958 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:37:03.009 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:37:03.055 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:37:03.055 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:37:03.056 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:03.056 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:03.073 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:03.073 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:03.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:37:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.273+0000 7f2e9b558640 1 Processor -- start 2024-05-10T10:37:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.273+0000 7f2e9b558640 1 -- start start 2024-05-10T10:37:03.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e9b558640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e9415f3b0 0x7f2e940d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e9b558640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e9415f790 0x7f2e940d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e9b558640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2e940d5d30 0x7f2e940c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e9b558640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940d30b0 con 0x7f2e9415f790 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e9b558640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940d3390 con 0x7f2e940d5d30 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e9b558640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2e940c8c80 con 0x7f2e9415f3b0 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e98af3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e9415f790 0x7f2e940d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e98af3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e9415f790 0x7f2e940d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54492/0 (socket says 172.21.15.53:54492) 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.274+0000 7f2e98af3640 1 -- 172.21.15.53:0/3035138970 learned_addr learned my addr 172.21.15.53:0/3035138970 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e99af5640 1 --2- 172.21.15.53:0/3035138970 >> v2:172.21.15.135:3300/0 conn(0x7f2e940d5d30 0x7f2e940c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e992f4640 1 --2- 172.21.15.53:0/3035138970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e9415f3b0 0x7f2e940d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e98af3640 1 -- 172.21.15.53:0/3035138970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e9415f3b0 msgr2=0x7f2e940d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e98af3640 1 --2- 172.21.15.53:0/3035138970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e9415f3b0 0x7f2e940d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e98af3640 1 -- 172.21.15.53:0/3035138970 >> v2:172.21.15.135:3300/0 conn(0x7f2e940d5d30 msgr2=0x7f2e940c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e98af3640 1 --2- 172.21.15.53:0/3035138970 >> v2:172.21.15.135:3300/0 conn(0x7f2e940d5d30 0x7f2e940c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e98af3640 1 -- 172.21.15.53:0/3035138970 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e940c8f60 con 0x7f2e9415f790 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.275+0000 7f2e98af3640 1 --2- 172.21.15.53:0/3035138970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f790 0x7f2e940d3c30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2e940d36a0 tx=0x7f2e8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dea1835f00f01f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e8bfff640 1 -- 172.21.15.53:0/3035138970 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84037070 con 0x7f2e9415f790 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e8bfff640 1 -- 172.21.15.53:0/3035138970 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e84032440 con 0x7f2e9415f790 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e8bfff640 1 -- 172.21.15.53:0/3035138970 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84032810 con 0x7f2e9415f790 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 -- 172.21.15.53:0/3035138970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f790 msgr2=0x7f2e940d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 --2- 172.21.15.53:0/3035138970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f790 0x7f2e940d3c30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2e940d36a0 tx=0x7f2e8402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 -- 172.21.15.53:0/3035138970 shutdown_connections 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 --2- 172.21.15.53:0/3035138970 >> v2:172.21.15.135:3300/0 conn(0x7f2e940d5d30 0x7f2e940c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 --2- 172.21.15.53:0/3035138970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f790 0x7f2e940d3c30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 --2- 172.21.15.53:0/3035138970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e9415f3b0 0x7f2e940d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 -- 172.21.15.53:0/3035138970 >> 172.21.15.53:0/3035138970 conn(0x7f2e940d5130 msgr2=0x7f2e940d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 -- 172.21.15.53:0/3035138970 shutdown_connections 2024-05-10T10:37:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.276+0000 7f2e9b558640 1 -- 172.21.15.53:0/3035138970 wait complete. 2024-05-10T10:37:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.277+0000 7f2e9b558640 1 Processor -- start 2024-05-10T10:37:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e9b558640 1 -- start start 2024-05-10T10:37:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e9b558640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e940d5d30 0x7f2e9415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e9b558640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f3b0 0x7f2e940b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e9b558640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e940b9230 0x7f2e9409fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e9b558640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940c9e20 con 0x7f2e9415f3b0 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e9b558640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940c9ca0 con 0x7f2e940d5d30 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e9b558640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2e940c9fa0 con 0x7f2e940b9230 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e98af3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f3b0 0x7f2e940b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e98af3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f3b0 0x7f2e940b8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54508/0 (socket says 172.21.15.53:54508) 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.278+0000 7f2e98af3640 1 -- 172.21.15.53:0/2719331538 learned_addr learned my addr 172.21.15.53:0/2719331538 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e99af5640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e940b9230 0x7f2e9409fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e992f4640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e940d5d30 0x7f2e9415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e98af3640 1 -- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e940b9230 msgr2=0x7f2e9409fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e98af3640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e940b9230 0x7f2e9409fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e98af3640 1 -- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e940d5d30 msgr2=0x7f2e9415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e98af3640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e940d5d30 0x7f2e9415cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e98af3640 1 -- 172.21.15.53:0/2719331538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e940b9b50 con 0x7f2e9415f3b0 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e98af3640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f3b0 0x7f2e940b8cf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2e8402a220 tx=0x7f2e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e89ffb640 1 -- 172.21.15.53:0/2719331538 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84037040 con 0x7f2e9415f3b0 2024-05-10T10:37:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e89ffb640 1 -- 172.21.15.53:0/2719331538 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e84042070 con 0x7f2e9415f3b0 2024-05-10T10:37:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9415d140 con 0x7f2e9415f3b0 2024-05-10T10:37:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e89ffb640 1 -- 172.21.15.53:0/2719331538 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84030bb0 con 0x7f2e9415f3b0 2024-05-10T10:37:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.279+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e940ce150 con 0x7f2e9415f3b0 2024-05-10T10:37:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.280+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e940b9640 con 0x7f2e9415f3b0 2024-05-10T10:37:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.281+0000 7f2e89ffb640 1 -- 172.21.15.53:0/2719331538 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e84047460 con 0x7f2e9415f3b0 2024-05-10T10:37:03.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.281+0000 7f2e89ffb640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e78076300 0x7f2e780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.281+0000 7f2e89ffb640 1 -- 172.21.15.53:0/2719331538 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f2e840b83b0 con 0x7f2e9415f3b0 2024-05-10T10:37:03.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.282+0000 7f2e992f4640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e78076300 0x7f2e780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.282+0000 7f2e992f4640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e78076300 0x7f2e780787c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006fd0 tx=0x7f2e8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.286+0000 7f2e89ffb640 1 -- 172.21.15.53:0/2719331538 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2e940b9640 con 0x7f2e9415f3b0 2024-05-10T10:37:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.490+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e9406a220 con 0x7f2e9415f3b0 2024-05-10T10:37:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.490+0000 7f2e89ffb640 1 -- 172.21.15.53:0/2719331538 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7f2e940b9b50 con 0x7f2e9415f3b0 2024-05-10T10:37:03.492 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:37:03.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.492+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e78076300 msgr2=0x7f2e780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.492+0000 7f2e9b558640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e78076300 0x7f2e780787c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006fd0 tx=0x7f2e8c008040 comp rx=0 tx=0).stop 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.492+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f3b0 msgr2=0x7f2e940b8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.492+0000 7f2e9b558640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f3b0 0x7f2e940b8cf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2e8402a220 tx=0x7f2e840046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.492+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 shutdown_connections 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.492+0000 7f2e9b558640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e78076300 0x7f2e780787c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.493+0000 7f2e9b558640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2e940b9230 0x7f2e9409fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.493+0000 7f2e9b558640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9415f3b0 0x7f2e940b8cf0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.493+0000 7f2e9b558640 1 --2- 172.21.15.53:0/2719331538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e940d5d30 0x7f2e9415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.493+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 >> 172.21.15.53:0/2719331538 conn(0x7f2e940d5130 msgr2=0x7f2e94075930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.493+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 shutdown_connections 2024-05-10T10:37:03.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.493+0000 7f2e9b558640 1 -- 172.21.15.53:0/2719331538 wait complete. 2024-05-10T10:37:03.505 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:37:03.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:03.505 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:03.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:37:03.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:03.547 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:37:03.604 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:37:03.604 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:37:03.605 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:03.623 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:37:03.623 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:37:03.623 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:03.623 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:03.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:37:03.660 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:37:03.661 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:37:03.661 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:03.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:37:03.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.802+0000 7f3c883d4640 1 Processor -- start 2024-05-10T10:37:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.802+0000 7f3c883d4640 1 -- start start 2024-05-10T10:37:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c883d4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c8015f790 0x7f3c800cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c883d4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3c800d5d30 0x7f3c800d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c883d4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c800d30b0 con 0x7f3c8015f3b0 2024-05-10T10:37:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c883d4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3c800d3390 con 0x7f3c800d5d30 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c883d4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800c8c80 con 0x7f3c8015f790 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c86971640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3c800d5d30 0x7f3c800d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c8596f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c8015f790 0x7f3c800cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c86170640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3c8015f3b0 0x7f3c800d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:44566/0 (socket says 172.21.15.53:44566) 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.803+0000 7f3c86170640 1 -- 172.21.15.53:0/1658145884 learned_addr learned my addr 172.21.15.53:0/1658145884 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.804+0000 7f3c86971640 1 -- 172.21.15.53:0/1658145884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c8015f790 msgr2=0x7f3c800cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.804+0000 7f3c86971640 1 --2- 172.21.15.53:0/1658145884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c8015f790 0x7f3c800cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.804+0000 7f3c86971640 1 -- 172.21.15.53:0/1658145884 >> v1:172.21.15.135:6789/0 conn(0x7f3c8015f3b0 legacy=0x7f3c800d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.804+0000 7f3c8516e640 1 -- 172.21.15.53:0/1658145884 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199452296 0 0) 0x7f3c800d30b0 con 0x7f3c8015f3b0 2024-05-10T10:37:03.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.804+0000 7f3c86971640 1 -- 172.21.15.53:0/1658145884 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c800c8f60 con 0x7f3c800d5d30 2024-05-10T10:37:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.804+0000 7f3c86971640 1 --2- 172.21.15.53:0/1658145884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c800d5d30 0x7f3c800d0210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3c780097a0 tx=0x7f3c7802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9954906f185cf5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c8516e640 1 -- 172.21.15.53:0/1658145884 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78037070 con 0x7f3c800d5d30 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c8516e640 1 -- 172.21.15.53:0/1658145884 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c78032440 con 0x7f3c800d5d30 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c8516e640 1 -- 172.21.15.53:0/1658145884 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78032810 con 0x7f3c800d5d30 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 -- 172.21.15.53:0/1658145884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c800d5d30 msgr2=0x7f3c800d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 --2- 172.21.15.53:0/1658145884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c800d5d30 0x7f3c800d0210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3c780097a0 tx=0x7f3c7802a180 comp rx=0 tx=0).stop 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 -- 172.21.15.53:0/1658145884 shutdown_connections 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 --2- 172.21.15.53:0/1658145884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c8015f790 0x7f3c800cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 --2- 172.21.15.53:0/1658145884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c800d5d30 0x7f3c800d0210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 -- 172.21.15.53:0/1658145884 >> 172.21.15.53:0/1658145884 conn(0x7f3c800d5130 msgr2=0x7f3c800d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 -- 172.21.15.53:0/1658145884 shutdown_connections 2024-05-10T10:37:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.805+0000 7f3c883d4640 1 -- 172.21.15.53:0/1658145884 wait complete. 2024-05-10T10:37:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.806+0000 7f3c883d4640 1 Processor -- start 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.806+0000 7f3c883d4640 1 -- start start 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c883d4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c800d5d30 0x7f3c80152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c883d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c8015f3b0 0x7f3c80154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c883d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c80155340 0x7f3c801570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c883d4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c800d3f80 con 0x7f3c8015f3b0 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c883d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3c800d3e00 con 0x7f3c80155340 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c883d4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800ca770 con 0x7f3c800d5d30 2024-05-10T10:37:03.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c8596f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c8015f3b0 0x7f3c80154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c8596f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c8015f3b0 0x7f3c80154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54510/0 (socket says 172.21.15.53:54510) 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c8596f640 1 -- 172.21.15.53:0/891157498 learned_addr learned my addr 172.21.15.53:0/891157498 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c86170640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c800d5d30 0x7f3c80152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.807+0000 7f3c86971640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c80155340 0x7f3c801570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c8596f640 1 -- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c800d5d30 msgr2=0x7f3c80152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c8596f640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c800d5d30 0x7f3c80152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c8596f640 1 -- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c80155340 msgr2=0x7f3c801570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c8596f640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c80155340 0x7f3c801570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c8596f640 1 -- 172.21.15.53:0/891157498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c80155970 con 0x7f3c8015f3b0 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c8596f640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c8015f3b0 0x7f3c80154e00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000e840 tx=0x7f3c7000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c76ffd640 1 -- 172.21.15.53:0/891157498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70017070 con 0x7f3c8015f3b0 2024-05-10T10:37:03.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c76ffd640 1 -- 172.21.15.53:0/891157498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7000c940 con 0x7f3c8015f3b0 2024-05-10T10:37:03.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c80155c50 con 0x7f3c8015f3b0 2024-05-10T10:37:03.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c76ffd640 1 -- 172.21.15.53:0/891157498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000cd10 con 0x7f3c8015f3b0 2024-05-10T10:37:03.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.808+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8015fd20 con 0x7f3c8015f3b0 2024-05-10T10:37:03.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.809+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8006a220 con 0x7f3c8015f3b0 2024-05-10T10:37:03.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.810+0000 7f3c76ffd640 1 -- 172.21.15.53:0/891157498 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3c7001a070 con 0x7f3c8015f3b0 2024-05-10T10:37:03.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.811+0000 7f3c76ffd640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c640762d0 0x7f3c64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:03.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.811+0000 7f3c76ffd640 1 -- 172.21.15.53:0/891157498 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3c70097990 con 0x7f3c8015f3b0 2024-05-10T10:37:03.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.811+0000 7f3c86170640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c640762d0 0x7f3c64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:03.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.811+0000 7f3c86170640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c640762d0 0x7f3c64078790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0012e0 tx=0x7f3c7c001700 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:03.815+0000 7f3c76ffd640 1 -- 172.21.15.53:0/891157498 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3c70060a80 con 0x7f3c8015f3b0 2024-05-10T10:37:04.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.050+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c8006d7f0 con 0x7f3c8015f3b0 2024-05-10T10:37:04.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.051+0000 7f3c76ffd640 1 -- 172.21.15.53:0/891157498 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5189 (secure 0 0 0) 0x7f3c70064d80 con 0x7f3c8015f3b0 2024-05-10T10:37:04.053 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:04.053 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:36:32.408868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":13,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:37:04.053 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2024-05-10T10:37:04.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c640762d0 msgr2=0x7f3c64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c640762d0 0x7f3c64078790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0012e0 tx=0x7f3c7c001700 comp rx=0 tx=0).stop 2024-05-10T10:37:04.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c8015f3b0 msgr2=0x7f3c80154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c8015f3b0 0x7f3c80154e00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000e840 tx=0x7f3c7000ed00 comp rx=0 tx=0).stop 2024-05-10T10:37:04.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 shutdown_connections 2024-05-10T10:37:04.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c640762d0 0x7f3c64078790 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c80155340 0x7f3c801570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c8015f3b0 0x7f3c80154e00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 --2- 172.21.15.53:0/891157498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c800d5d30 0x7f3c80152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 >> 172.21.15.53:0/891157498 conn(0x7f3c800d5130 msgr2=0x7f3c800caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:04.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 shutdown_connections 2024-05-10T10:37:04.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.053+0000 7f3c883d4640 1 -- 172.21.15.53:0/891157498 wait complete. 2024-05-10T10:37:04.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.168+0000 7f1d6e932640 1 Processor -- start 2024-05-10T10:37:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.168+0000 7f1d6e932640 1 -- start start 2024-05-10T10:37:04.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d6e932640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806c2c0 0x7f1d6806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d6e932640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d680703c0 0x7f1d68070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d6e932640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d6806cbe0 con 0x7f1d6806edc0 2024-05-10T10:37:04.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d6e932640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1d68070d60 con 0x7f1d680703c0 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d6e932640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1d680ce8e0 con 0x7f1d6806c2c0 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d5ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d680703c0 0x7f1d68070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d5ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d680703c0 0x7f1d68070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54516/0 (socket says 172.21.15.53:54516) 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d5ffff640 1 -- 172.21.15.53:0/4255785024 learned_addr learned my addr 172.21.15.53:0/4255785024 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.169+0000 7f1d67fff640 1 --2- 172.21.15.53:0/4255785024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806c2c0 0x7f1d6806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.170+0000 7f1d5ffff640 1 -- 172.21.15.53:0/4255785024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806c2c0 msgr2=0x7f1d6806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.170+0000 7f1d5ffff640 1 --2- 172.21.15.53:0/4255785024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806c2c0 0x7f1d6806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.170+0000 7f1d5ffff640 1 -- 172.21.15.53:0/4255785024 >> v1:172.21.15.135:6789/0 conn(0x7f1d6806edc0 legacy=0x7f1d6806f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.170+0000 7f1d5ffff640 1 -- 172.21.15.53:0/4255785024 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680cebc0 con 0x7f1d680703c0 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.170+0000 7f1d5ffff640 1 --2- 172.21.15.53:0/4255785024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d68070820 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e63b9dfd0f1bb93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.170+0000 7f1d677fe640 1 -- 172.21.15.53:0/4255785024 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50037070 con 0x7f1d680703c0 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d677fe640 1 -- 172.21.15.53:0/4255785024 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50032440 con 0x7f1d680703c0 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d677fe640 1 -- 172.21.15.53:0/4255785024 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50032810 con 0x7f1d680703c0 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 -- 172.21.15.53:0/4255785024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 msgr2=0x7f1d68070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4255785024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d68070820 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d5002a270 comp rx=0 tx=0).stop 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 -- 172.21.15.53:0/4255785024 shutdown_connections 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4255785024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d68070820 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4255785024 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806c2c0 0x7f1d6806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 -- 172.21.15.53:0/4255785024 >> 172.21.15.53:0/4255785024 conn(0x7f1d680b3980 msgr2=0x7f1d680b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 -- 172.21.15.53:0/4255785024 shutdown_connections 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.171+0000 7f1d6e932640 1 -- 172.21.15.53:0/4255785024 wait complete. 2024-05-10T10:37:04.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.172+0000 7f1d6e932640 1 Processor -- start 2024-05-10T10:37:04.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.172+0000 7f1d6e932640 1 -- start start 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6e932640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d6806c2c0 0x7f1d6814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6e932640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806edc0 0x7f1d6815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6e932640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d6815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6e932640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680cfc70 con 0x7f1d680703c0 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6e932640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680cf180 con 0x7f1d6806c2c0 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6e932640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6815d2a0 con 0x7f1d6806edc0 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6cecf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d6815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6cecf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d6815bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54520/0 (socket says 172.21.15.53:54520) 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.173+0000 7f1d6cecf640 1 -- 172.21.15.53:0/4215682115 learned_addr learned my addr 172.21.15.53:0/4215682115 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d5ffff640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806edc0 0x7f1d6815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d67fff640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d6806c2c0 0x7f1d6814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6cecf640 1 -- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806edc0 msgr2=0x7f1d6815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6cecf640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806edc0 0x7f1d6815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6cecf640 1 -- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d6806c2c0 msgr2=0x7f1d6814fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6cecf640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d6806c2c0 0x7f1d6814fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6cecf640 1 -- 172.21.15.53:0/4215682115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680c7d30 con 0x7f1d680703c0 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d67fff640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d6806c2c0 0x7f1d6814fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d5ffff640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806edc0 0x7f1d6815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6cecf640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d6815bd50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000c7d0 tx=0x7f1d6000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d657fa640 1 -- 172.21.15.53:0/4215682115 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60003b70 con 0x7f1d680703c0 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d657fa640 1 -- 172.21.15.53:0/4215682115 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60003d30 con 0x7f1d680703c0 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d657fa640 1 -- 172.21.15.53:0/4215682115 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000aa90 con 0x7f1d680703c0 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d680c8010 con 0x7f1d680703c0 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.174+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d680b4d80 con 0x7f1d680703c0 2024-05-10T10:37:04.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.175+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6806a220 con 0x7f1d680703c0 2024-05-10T10:37:04.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.176+0000 7f1d657fa640 1 -- 172.21.15.53:0/4215682115 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1d6000ac30 con 0x7f1d680703c0 2024-05-10T10:37:04.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.176+0000 7f1d657fa640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.176+0000 7f1d657fa640 1 -- 172.21.15.53:0/4215682115 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f1d60097740 con 0x7f1d680703c0 2024-05-10T10:37:04.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.176+0000 7f1d67fff640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.177+0000 7f1d67fff640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009ac0 tx=0x7f1d58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.180+0000 7f1d657fa640 1 -- 172.21.15.53:0/4215682115 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1d60060e80 con 0x7f1d680703c0 2024-05-10T10:37:04.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.412+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d68073820 con 0x7f1d680703c0 2024-05-10T10:37:04.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.413+0000 7f1d657fa640 1 -- 172.21.15.53:0/4215682115 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5189 (secure 0 0 0) 0x7f1d60067e20 con 0x7f1d680703c0 2024-05-10T10:37:04.415 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:04.415 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:36:32.408868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":13,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:37:04.415 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 msgr2=0x7f1d48078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009ac0 tx=0x7f1d58008040 comp rx=0 tx=0).stop 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 msgr2=0x7f1d6815bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d6815bd50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000c7d0 tx=0x7f1d6000cc90 comp rx=0 tx=0).stop 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 shutdown_connections 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1d480762d0 0x7f1d48078790 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d680703c0 0x7f1d6815bd50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d6806edc0 0x7f1d6815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 --2- 172.21.15.53:0/4215682115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d6806c2c0 0x7f1d6814fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 >> 172.21.15.53:0/4215682115 conn(0x7f1d680b3980 msgr2=0x7f1d680b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 shutdown_connections 2024-05-10T10:37:04.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.415+0000 7f1d6e932640 1 -- 172.21.15.53:0/4215682115 wait complete. 2024-05-10T10:37:04.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:37:04.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.531+0000 7ff49971c640 1 Processor -- start 2024-05-10T10:37:04.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.531+0000 7ff49971c640 1 -- start start 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff49971c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff494162750 0x7ff4940d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff49971c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff494162b30 0x7ff4940b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff49971c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4940b45f0 con 0x7ff4940d5cb0 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff49971c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff4940b4f20 con 0x7ff494162b30 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff49971c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff4940d0fe0 con 0x7ff494162750 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff4927fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff4940d5cb0 0x7ff4940d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43662/0 (socket says 172.21.15.53:43662) 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff4927fc640 1 -- 172.21.15.53:0/2198445894 learned_addr learned my addr 172.21.15.53:0/2198445894 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:04.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff492ffd640 1 --2- 172.21.15.53:0/2198445894 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff494162750 0x7ff4940d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff4937fe640 1 --2- 172.21.15.53:0/2198445894 >> v2:172.21.15.135:3300/0 conn(0x7ff494162b30 0x7ff4940b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff491ffb640 1 -- 172.21.15.53:0/2198445894 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244638538 0 0) 0x7ff4940b45f0 con 0x7ff4940d5cb0 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff491ffb640 1 -- 172.21.15.53:0/2198445894 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff480003520 con 0x7ff4940d5cb0 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff492ffd640 1 -- 172.21.15.53:0/2198445894 >> v2:172.21.15.135:3300/0 conn(0x7ff494162b30 msgr2=0x7ff4940b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff492ffd640 1 --2- 172.21.15.53:0/2198445894 >> v2:172.21.15.135:3300/0 conn(0x7ff494162b30 0x7ff4940b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff492ffd640 1 -- 172.21.15.53:0/2198445894 >> v1:172.21.15.53:6789/0 conn(0x7ff4940d5cb0 legacy=0x7ff4940d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff492ffd640 1 -- 172.21.15.53:0/2198445894 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4940b48d0 con 0x7ff494162750 2024-05-10T10:37:04.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.532+0000 7ff4937fe640 1 --2- 172.21.15.53:0/2198445894 >> v2:172.21.15.135:3300/0 conn(0x7ff494162b30 0x7ff4940b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.533+0000 7ff492ffd640 1 --2- 172.21.15.53:0/2198445894 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff494162750 0x7ff4940d0670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097a0 tx=0x7ff47c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a47ecb446ece16a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.533+0000 7ff491ffb640 1 -- 172.21.15.53:0/2198445894 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c037070 con 0x7ff494162750 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.533+0000 7ff491ffb640 1 -- 172.21.15.53:0/2198445894 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c02ae50 con 0x7ff494162750 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.533+0000 7ff491ffb640 1 -- 172.21.15.53:0/2198445894 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c032640 con 0x7ff494162750 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 -- 172.21.15.53:0/2198445894 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff494162750 msgr2=0x7ff4940d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 --2- 172.21.15.53:0/2198445894 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff494162750 0x7ff4940d0670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097a0 tx=0x7ff47c02a100 comp rx=0 tx=0).stop 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 -- 172.21.15.53:0/2198445894 shutdown_connections 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 --2- 172.21.15.53:0/2198445894 >> v2:172.21.15.135:3300/0 conn(0x7ff494162b30 0x7ff4940b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 --2- 172.21.15.53:0/2198445894 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff494162750 0x7ff4940d0670 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 -- 172.21.15.53:0/2198445894 >> 172.21.15.53:0/2198445894 conn(0x7ff49409e3b0 msgr2=0x7ff4940d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:04.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 -- 172.21.15.53:0/2198445894 shutdown_connections 2024-05-10T10:37:04.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.534+0000 7ff49971c640 1 -- 172.21.15.53:0/2198445894 wait complete. 2024-05-10T10:37:04.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.535+0000 7ff49971c640 1 Processor -- start 2024-05-10T10:37:04.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.535+0000 7ff49971c640 1 -- start start 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff49971c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff494162750 0x7ff494158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff49971c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff494162b30 0x7ff4940ba860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff49971c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff49414f620 0x7ff49415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff49971c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4940d0d30 con 0x7ff494162750 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff49971c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4940d0bb0 con 0x7ff494162b30 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff49971c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff4940d3d40 con 0x7ff49414f620 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff494162750 0x7ff494158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff492ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff494162750 0x7ff494158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54530/0 (socket says 172.21.15.53:54530) 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff492ffd640 1 -- 172.21.15.53:0/188474515 learned_addr learned my addr 172.21.15.53:0/188474515 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:04.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff4927fc640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff494162b30 0x7ff4940ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff4937fe640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff49414f620 0x7ff49415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff492ffd640 1 -- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff49414f620 msgr2=0x7ff49415e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff492ffd640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff49414f620 0x7ff49415e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff492ffd640 1 -- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff494162b30 msgr2=0x7ff4940ba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.536+0000 7ff492ffd640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff494162b30 0x7ff4940ba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff492ffd640 1 -- 172.21.15.53:0/188474515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49414fab0 con 0x7ff494162750 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff4927fc640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff494162b30 0x7ff4940ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:04.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff492ffd640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff494162750 0x7ff494158650 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff47c035040 tx=0x7ff47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff48b7fe640 1 -- 172.21.15.53:0/188474515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c037050 con 0x7ff494162750 2024-05-10T10:37:04.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff48b7fe640 1 -- 172.21.15.53:0/188474515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c038040 con 0x7ff494162750 2024-05-10T10:37:04.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49415ce00 con 0x7ff494162750 2024-05-10T10:37:04.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff48b7fe640 1 -- 172.21.15.53:0/188474515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0044c0 con 0x7ff494162750 2024-05-10T10:37:04.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.537+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4940b8d50 con 0x7ff494162750 2024-05-10T10:37:04.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.538+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49406a220 con 0x7ff494162750 2024-05-10T10:37:04.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.539+0000 7ff48b7fe640 1 -- 172.21.15.53:0/188474515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff47c045020 con 0x7ff494162750 2024-05-10T10:37:04.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.539+0000 7ff48b7fe640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.540+0000 7ff48b7fe640 1 -- 172.21.15.53:0/188474515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7ff47c0b77a0 con 0x7ff494162750 2024-05-10T10:37:04.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.540+0000 7ff4927fc640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.540+0000 7ff4927fc640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff484007fd0 tx=0x7ff484009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.544+0000 7ff48b7fe640 1 -- 172.21.15.53:0/188474515 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff47c080f70 con 0x7ff494162750 2024-05-10T10:37:04.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.741+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4940cb9b0 con 0x7ff494162750 2024-05-10T10:37:04.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.742+0000 7ff48b7fe640 1 -- 172.21.15.53:0/188474515 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17123 (secure 0 0 0) 0x7ff47c080870 con 0x7ff494162750 2024-05-10T10:37:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:04.744 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":63,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:36:23.496667+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:49.396339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:50.695199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 msgr2=0x7ff474078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff484007fd0 tx=0x7ff484009040 comp rx=0 tx=0).stop 2024-05-10T10:37:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff494162750 msgr2=0x7ff494158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff494162750 0x7ff494158650 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff47c035040 tx=0x7ff47c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 shutdown_connections 2024-05-10T10:37:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff49414f620 0x7ff49415e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff494162b30 0x7ff4940ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 --2- 172.21.15.53:0/188474515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff494162750 0x7ff494158650 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 >> 172.21.15.53:0/188474515 conn(0x7ff49409e3b0 msgr2=0x7ff49406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 shutdown_connections 2024-05-10T10:37:04.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.744+0000 7ff49971c640 1 -- 172.21.15.53:0/188474515 wait complete. 2024-05-10T10:37:04.757 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:37:04.757 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.859+0000 7fd7f8db6640 1 Processor -- start 2024-05-10T10:37:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.859+0000 7fd7f8db6640 1 -- start start 2024-05-10T10:37:04.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.860+0000 7fd7f8db6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd7f415f790 0x7fd7f40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.860+0000 7fd7f8db6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd7f40d5d30 0x7fd7f40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.860+0000 7fd7f8db6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f40d30b0 con 0x7fd7f415f3b0 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.860+0000 7fd7f8db6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f40d3390 con 0x7fd7f415f790 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.860+0000 7fd7f8db6640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f40c8c80 con 0x7fd7f40d5d30 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.860+0000 7fd7f1d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd7f415f790 0x7fd7f40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f1d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd7f415f790 0x7fd7f40cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54532/0 (socket says 172.21.15.53:54532) 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f259c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd7f415f3b0 0x7fd7f40d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43668/0 (socket says 172.21.15.53:43668) 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f1d9b640 1 -- 172.21.15.53:0/1158014749 learned_addr learned my addr 172.21.15.53:0/1158014749 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f2d9d640 1 --2- 172.21.15.53:0/1158014749 >> v2:172.21.15.135:3300/0 conn(0x7fd7f40d5d30 0x7fd7f40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f1d9b640 1 -- 172.21.15.53:0/1158014749 >> v2:172.21.15.135:3300/0 conn(0x7fd7f40d5d30 msgr2=0x7fd7f40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f1d9b640 1 --2- 172.21.15.53:0/1158014749 >> v2:172.21.15.135:3300/0 conn(0x7fd7f40d5d30 0x7fd7f40d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f1d9b640 1 -- 172.21.15.53:0/1158014749 >> v1:172.21.15.53:6789/0 conn(0x7fd7f415f3b0 legacy=0x7fd7f40d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.861+0000 7fd7f1d9b640 1 -- 172.21.15.53:0/1158014749 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f40c8f60 con 0x7fd7f415f790 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f1d9b640 1 --2- 172.21.15.53:0/1158014749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f415f790 0x7fd7f40cfcd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd7f408d220 tx=0x7fd7e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18dceff43ee0c35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f159a640 1 -- 172.21.15.53:0/1158014749 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4037070 con 0x7fd7f415f790 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f159a640 1 -- 172.21.15.53:0/1158014749 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e4032440 con 0x7fd7f415f790 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f159a640 1 -- 172.21.15.53:0/1158014749 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4032810 con 0x7fd7f415f790 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 -- 172.21.15.53:0/1158014749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f415f790 msgr2=0x7fd7f40cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/1158014749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f415f790 0x7fd7f40cfcd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd7f408d220 tx=0x7fd7e402a270 comp rx=0 tx=0).stop 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 -- 172.21.15.53:0/1158014749 shutdown_connections 2024-05-10T10:37:04.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/1158014749 >> v2:172.21.15.135:3300/0 conn(0x7fd7f40d5d30 0x7fd7f40d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/1158014749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f415f790 0x7fd7f40cfcd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 -- 172.21.15.53:0/1158014749 >> 172.21.15.53:0/1158014749 conn(0x7fd7f40d5130 msgr2=0x7fd7f40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:04.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 -- 172.21.15.53:0/1158014749 shutdown_connections 2024-05-10T10:37:04.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.862+0000 7fd7f8db6640 1 -- 172.21.15.53:0/1158014749 wait complete. 2024-05-10T10:37:04.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.864+0000 7fd7f8db6640 1 Processor -- start 2024-05-10T10:37:04.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.864+0000 7fd7f8db6640 1 -- start start 2024-05-10T10:37:04.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f8db6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7f40d5d30 0x7fd7f4155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f8db6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7f415f3b0 0x7fd7f406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f8db6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f406da20 0x7fd7f40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f8db6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40d3f80 con 0x7fd7f406da20 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f8db6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40d3e00 con 0x7fd7f415f3b0 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f8db6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f40ca770 con 0x7fd7f40d5d30 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f406da20 0x7fd7f40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f406da20 0x7fd7f40ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54542/0 (socket says 172.21.15.53:54542) 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 -- 172.21.15.53:0/409451481 learned_addr learned my addr 172.21.15.53:0/409451481 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f1d9b640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7f415f3b0 0x7fd7f406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f259c640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7f40d5d30 0x7fd7f4155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 -- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7f40d5d30 msgr2=0x7fd7f4155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7f40d5d30 0x7fd7f4155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 -- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7f415f3b0 msgr2=0x7fd7f406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:04.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7f415f3b0 0x7fd7f406d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 -- 172.21.15.53:0/409451481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f4158eb0 con 0x7fd7f406da20 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f1d9b640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7f415f3b0 0x7fd7f406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f2d9d640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f406da20 0x7fd7f40ce370 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec007e40 tx=0x7fd7ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7eaffd640 1 -- 172.21.15.53:0/409451481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec014620 con 0x7fd7f406da20 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7eaffd640 1 -- 172.21.15.53:0/409451481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec019070 con 0x7fd7f406da20 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7eaffd640 1 -- 172.21.15.53:0/409451481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec014a20 con 0x7fd7f406da20 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.865+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f41f5bb0 con 0x7fd7f406da20 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.866+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f41f4df0 con 0x7fd7f406da20 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.866+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f41f56e0 con 0x7fd7f406da20 2024-05-10T10:37:04.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.867+0000 7fd7eaffd640 1 -- 172.21.15.53:0/409451481 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd7ec014bc0 con 0x7fd7f406da20 2024-05-10T10:37:04.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.868+0000 7fd7eaffd640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd7d40762d0 0x7fd7d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:04.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.868+0000 7fd7eaffd640 1 -- 172.21.15.53:0/409451481 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fd7ec098140 con 0x7fd7f406da20 2024-05-10T10:37:04.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.868+0000 7fd7f259c640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd7d40762d0 0x7fd7d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:04.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.868+0000 7fd7f259c640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd7d40762d0 0x7fd7d4078790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000f60 tx=0x7fd7dc007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:04.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:04.872+0000 7fd7eaffd640 1 -- 172.21.15.53:0/409451481 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd7f4158eb0 con 0x7fd7f406da20 2024-05-10T10:37:05.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.102+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f406a220 con 0x7fd7f406da20 2024-05-10T10:37:05.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.102+0000 7fd7eaffd640 1 -- 172.21.15.53:0/409451481 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5189 (secure 0 0 0) 0x7fd7ec061880 con 0x7fd7f406da20 2024-05-10T10:37:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:05.104 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:36:32.408868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":13,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:37:05.104 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2024-05-10T10:37:05.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd7d40762d0 msgr2=0x7fd7d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd7d40762d0 0x7fd7d4078790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000f60 tx=0x7fd7dc007cc0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f406da20 msgr2=0x7fd7f40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f406da20 0x7fd7f40ce370 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec007e40 tx=0x7fd7ec00e4b0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 shutdown_connections 2024-05-10T10:37:05.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd7d40762d0 0x7fd7d4078790 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7f406da20 0x7fd7f40ce370 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7f415f3b0 0x7fd7f406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 --2- 172.21.15.53:0/409451481 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7f40d5d30 0x7fd7f4155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 >> 172.21.15.53:0/409451481 conn(0x7fd7f40d5130 msgr2=0x7fd7f40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 shutdown_connections 2024-05-10T10:37:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.104+0000 7fd7f8db6640 1 -- 172.21.15.53:0/409451481 wait complete. 2024-05-10T10:37:05.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:05.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.218+0000 7f359f79d640 1 Processor -- start 2024-05-10T10:37:05.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.218+0000 7f359f79d640 1 -- start start 2024-05-10T10:37:05.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359f79d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f35980d1aa0 0x7f35980d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359f79d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35980d3340 0x7f35980ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359f79d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3598072330 con 0x7f359815d290 2024-05-10T10:37:05.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359f79d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f35980d37d0 con 0x7f35980d3340 2024-05-10T10:37:05.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359f79d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3598071d70 con 0x7f35980d1aa0 2024-05-10T10:37:05.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359dd3a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35980d3340 0x7f35980ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359d539640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f359815d290 0x7f3598071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43684/0 (socket says 172.21.15.53:43684) 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.219+0000 7f359d539640 1 -- 172.21.15.53:0/3516659300 learned_addr learned my addr 172.21.15.53:0/3516659300 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f359cd38640 1 --2- 172.21.15.53:0/3516659300 >> v2:172.21.15.135:3300/0 conn(0x7f35980d1aa0 0x7f35980d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f359dd3a640 1 -- 172.21.15.53:0/3516659300 >> v2:172.21.15.135:3300/0 conn(0x7f35980d1aa0 msgr2=0x7f35980d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f359dd3a640 1 --2- 172.21.15.53:0/3516659300 >> v2:172.21.15.135:3300/0 conn(0x7f35980d1aa0 0x7f35980d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f359dd3a640 1 -- 172.21.15.53:0/3516659300 >> v1:172.21.15.53:6789/0 conn(0x7f359815d290 legacy=0x7f3598071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f3597fff640 1 -- 172.21.15.53:0/3516659300 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763517037 0 0) 0x7f3598072330 con 0x7f359815d290 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f359dd3a640 1 -- 172.21.15.53:0/3516659300 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3598072050 con 0x7f35980d3340 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f359cd38640 1 --2- 172.21.15.53:0/3516659300 >> v2:172.21.15.135:3300/0 conn(0x7f35980d1aa0 0x7f35980d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.220+0000 7f359dd3a640 1 --2- 172.21.15.53:0/3516659300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d3340 0x7f35980ce270 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f359806d220 tx=0x7f358802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4591faf0cf9ce8fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f3597fff640 1 -- 172.21.15.53:0/3516659300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588007740 con 0x7f35980d3340 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f3597fff640 1 -- 172.21.15.53:0/3516659300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35880078e0 con 0x7f35980d3340 2024-05-10T10:37:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f3597fff640 1 -- 172.21.15.53:0/3516659300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588007cb0 con 0x7f35980d3340 2024-05-10T10:37:05.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 -- 172.21.15.53:0/3516659300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d3340 msgr2=0x7f35980ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 --2- 172.21.15.53:0/3516659300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d3340 0x7f35980ce270 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f359806d220 tx=0x7f358802a680 comp rx=0 tx=0).stop 2024-05-10T10:37:05.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 -- 172.21.15.53:0/3516659300 shutdown_connections 2024-05-10T10:37:05.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 --2- 172.21.15.53:0/3516659300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d3340 0x7f35980ce270 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 --2- 172.21.15.53:0/3516659300 >> v2:172.21.15.135:3300/0 conn(0x7f35980d1aa0 0x7f35980d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 -- 172.21.15.53:0/3516659300 >> 172.21.15.53:0/3516659300 conn(0x7f359809ba20 msgr2=0x7f359809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:05.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 -- 172.21.15.53:0/3516659300 shutdown_connections 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.221+0000 7f359f79d640 1 -- 172.21.15.53:0/3516659300 wait complete. 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.222+0000 7f359f79d640 1 Processor -- start 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359f79d640 1 -- start start 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359f79d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d1aa0 0x7f35980b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359f79d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35980d3340 0x7f359814a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359f79d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359815d290 0x7f359814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359f79d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3598071a50 con 0x7f35980d1aa0 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359f79d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35980cdd50 con 0x7f359815d290 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359f79d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f35980ce7b0 con 0x7f35980d3340 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359d539640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d1aa0 0x7f35980b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359d539640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d1aa0 0x7f35980b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54566/0 (socket says 172.21.15.53:54566) 2024-05-10T10:37:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.223+0000 7f359d539640 1 -- 172.21.15.53:0/246963785 learned_addr learned my addr 172.21.15.53:0/246963785 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359dd3a640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359815d290 0x7f359814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359cd38640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35980d3340 0x7f359814a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359d539640 1 -- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35980d3340 msgr2=0x7f359814a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359d539640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35980d3340 0x7f359814a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359d539640 1 -- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359815d290 msgr2=0x7f359814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359d539640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359815d290 0x7f359814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359d539640 1 -- 172.21.15.53:0/246963785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359814e1b0 con 0x7f35980d1aa0 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359cd38640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35980d3340 0x7f359814a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359dd3a640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359815d290 0x7f359814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359d539640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d1aa0 0x7f35980b4400 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f359000c430 tx=0x7f359000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f3595ffb640 1 -- 172.21.15.53:0/246963785 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900074e0 con 0x7f35980d1aa0 2024-05-10T10:37:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f3595ffb640 1 -- 172.21.15.53:0/246963785 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35900047e0 con 0x7f35980d1aa0 2024-05-10T10:37:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359814b2b0 con 0x7f35980d1aa0 2024-05-10T10:37:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f3595ffb640 1 -- 172.21.15.53:0/246963785 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590004bd0 con 0x7f35980d1aa0 2024-05-10T10:37:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.224+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359814b870 con 0x7f35980d1aa0 2024-05-10T10:37:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.225+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35980693f0 con 0x7f35980d1aa0 2024-05-10T10:37:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.226+0000 7f3595ffb640 1 -- 172.21.15.53:0/246963785 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3590011070 con 0x7f35980d1aa0 2024-05-10T10:37:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.226+0000 7f3595ffb640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f35840762d0 0x7f3584078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.227+0000 7f3595ffb640 1 -- 172.21.15.53:0/246963785 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f35900974f0 con 0x7f35980d1aa0 2024-05-10T10:37:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.227+0000 7f359cd38640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f35840762d0 0x7f3584078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.227+0000 7f359cd38640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f35840762d0 0x7f3584078790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f35980b3ba0 tx=0x7f358c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.231+0000 7f3595ffb640 1 -- 172.21.15.53:0/246963785 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3590060b90 con 0x7f35980d1aa0 2024-05-10T10:37:05.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.463+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35980d25f0 con 0x7f35980d1aa0 2024-05-10T10:37:05.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.464+0000 7f3595ffb640 1 -- 172.21.15.53:0/246963785 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5189 (secure 0 0 0) 0x7f3590060490 con 0x7f35980d1aa0 2024-05-10T10:37:05.466 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:05.466 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:35:51.741227+0000","modified":"2024-05-10T10:36:32.408868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":13,"addr":"172.21.15.53:6835/1934136965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1934136965},{"type":"v1","addr":"172.21.15.53:6835","nonce":1934136965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5622,"qdb_cluster":[5622]},"id":4}]} 2024-05-10T10:37:05.466 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2024-05-10T10:37:05.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f35840762d0 msgr2=0x7f3584078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f35840762d0 0x7f3584078790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f35980b3ba0 tx=0x7f358c008040 comp rx=0 tx=0).stop 2024-05-10T10:37:05.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d1aa0 msgr2=0x7f35980b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d1aa0 0x7f35980b4400 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f359000c430 tx=0x7f359000c8f0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 shutdown_connections 2024-05-10T10:37:05.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f35840762d0 0x7f3584078790 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359815d290 0x7f359814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35980d3340 0x7f359814a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 --2- 172.21.15.53:0/246963785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35980d1aa0 0x7f35980b4400 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 >> 172.21.15.53:0/246963785 conn(0x7f359809ba20 msgr2=0x7f35980b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:05.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 shutdown_connections 2024-05-10T10:37:05.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.466+0000 7f359f79d640 1 -- 172.21.15.53:0/246963785 wait complete. 2024-05-10T10:37:05.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:37:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.582+0000 7fdf51f04640 1 Processor -- start 2024-05-10T10:37:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.582+0000 7fdf51f04640 1 -- start start 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf51f04640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c06e4d0 0x7fdf4c0d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf51f04640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4c0cd4b0 con 0x7fdf4c0b8970 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf51f04640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4c0cd720 con 0x7fdf4c0d2260 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf51f04640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c0cda00 con 0x7fdf4c06e4d0 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf4b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdf4c0b8970 0x7fdf4c06ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43686/0 (socket says 172.21.15.53:43686) 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf4b7fe640 1 -- 172.21.15.53:0/62144912 learned_addr learned my addr 172.21.15.53:0/62144912 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf4affd640 1 --2- 172.21.15.53:0/62144912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c06e4d0 0x7fdf4c0d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf4affd640 1 -- 172.21.15.53:0/62144912 >> v1:172.21.15.135:6789/0 conn(0x7fdf4c0d2260 legacy=0x7fdf4c0ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf4affd640 1 -- 172.21.15.53:0/62144912 >> v1:172.21.15.53:6789/0 conn(0x7fdf4c0b8970 legacy=0x7fdf4c06ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.583+0000 7fdf4affd640 1 -- 172.21.15.53:0/62144912 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c0cdce0 con 0x7fdf4c06e4d0 2024-05-10T10:37:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.584+0000 7fdf4affd640 1 --2- 172.21.15.53:0/62144912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c06e4d0 0x7fdf4c0d1d20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdf400097d0 tx=0x7fdf4002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63dc0854761c5cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.586+0000 7fdf4a7fc640 1 -- 172.21.15.53:0/62144912 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40037070 con 0x7fdf4c06e4d0 2024-05-10T10:37:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.586+0000 7fdf4a7fc640 1 -- 172.21.15.53:0/62144912 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf40032440 con 0x7fdf4c06e4d0 2024-05-10T10:37:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.586+0000 7fdf4a7fc640 1 -- 172.21.15.53:0/62144912 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40032810 con 0x7fdf4c06e4d0 2024-05-10T10:37:05.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.587+0000 7fdf51f04640 1 -- 172.21.15.53:0/62144912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c06e4d0 msgr2=0x7fdf4c0d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.587+0000 7fdf51f04640 1 --2- 172.21.15.53:0/62144912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c06e4d0 0x7fdf4c0d1d20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdf400097d0 tx=0x7fdf4002a210 comp rx=0 tx=0).stop 2024-05-10T10:37:05.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.587+0000 7fdf51f04640 1 -- 172.21.15.53:0/62144912 shutdown_connections 2024-05-10T10:37:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.587+0000 7fdf51f04640 1 --2- 172.21.15.53:0/62144912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c06e4d0 0x7fdf4c0d1d20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.587+0000 7fdf51f04640 1 -- 172.21.15.53:0/62144912 >> 172.21.15.53:0/62144912 conn(0x7fdf4c0d2dc0 msgr2=0x7fdf4c0d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.587+0000 7fdf51f04640 1 -- 172.21.15.53:0/62144912 shutdown_connections 2024-05-10T10:37:05.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.587+0000 7fdf51f04640 1 -- 172.21.15.53:0/62144912 wait complete. 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.588+0000 7fdf51f04640 1 Processor -- start 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.588+0000 7fdf51f04640 1 -- start start 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf51f04640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf4c06e4d0 0x7fdf4c0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf51f04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf4c0b8970 0x7fdf4c1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf51f04640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c0d2260 0x7fdf4c1f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf51f04640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c0d2a10 con 0x7fdf4c0b8970 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf51f04640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c0b5c10 con 0x7fdf4c06e4d0 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf51f04640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c0ce910 con 0x7fdf4c0d2260 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf4c0b8970 0x7fdf4c1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf4c0b8970 0x7fdf4c1f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54576/0 (socket says 172.21.15.53:54576) 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 -- 172.21.15.53:0/3122643284 learned_addr learned my addr 172.21.15.53:0/3122643284 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4b7fe640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf4c06e4d0 0x7fdf4c0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4bfff640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c0d2260 0x7fdf4c1f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 -- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c0d2260 msgr2=0x7fdf4c1f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c0d2260 0x7fdf4c1f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 -- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf4c06e4d0 msgr2=0x7fdf4c0caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf4c06e4d0 0x7fdf4c0caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4affd640 1 -- 172.21.15.53:0/3122643284 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c1f5ff0 con 0x7fdf4c0b8970 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.589+0000 7fdf4b7fe640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf4c06e4d0 0x7fdf4c0caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.590+0000 7fdf4affd640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf4c0b8970 0x7fdf4c1f4e00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdf40035040 tx=0x7fdf400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.590+0000 7fdf33fff640 1 -- 172.21.15.53:0/3122643284 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40037040 con 0x7fdf4c0b8970 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.590+0000 7fdf33fff640 1 -- 172.21.15.53:0/3122643284 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf40030770 con 0x7fdf4c0b8970 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.590+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c1f62d0 con 0x7fdf4c0b8970 2024-05-10T10:37:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.590+0000 7fdf33fff640 1 -- 172.21.15.53:0/3122643284 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40030b80 con 0x7fdf4c0b8970 2024-05-10T10:37:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.590+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c0cb930 con 0x7fdf4c0b8970 2024-05-10T10:37:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.591+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c1f27e0 con 0x7fdf4c0b8970 2024-05-10T10:37:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.591+0000 7fdf33fff640 1 -- 172.21.15.53:0/3122643284 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdf40044020 con 0x7fdf4c0b8970 2024-05-10T10:37:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.592+0000 7fdf33fff640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdf380762d0 0x7fdf38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.592+0000 7fdf33fff640 1 -- 172.21.15.53:0/3122643284 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fdf400b7d10 con 0x7fdf4c0b8970 2024-05-10T10:37:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.592+0000 7fdf4b7fe640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdf380762d0 0x7fdf38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.593+0000 7fdf4b7fe640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdf380762d0 0x7fdf38078790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005880 tx=0x7fdf3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.597+0000 7fdf33fff640 1 -- 172.21.15.53:0/3122643284 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdf40032e20 con 0x7fdf4c0b8970 2024-05-10T10:37:05.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.788+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf4c06ea10 con 0x7fdf4c0b8970 2024-05-10T10:37:05.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.789+0000 7fdf33fff640 1 -- 172.21.15.53:0/3122643284 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17123 (secure 0 0 0) 0x7fdf400814e0 con 0x7fdf4c0b8970 2024-05-10T10:37:05.791 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:05.791 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":63,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:36:23.496667+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:35:49.396339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T10:35:50.695199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:05.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdf380762d0 msgr2=0x7fdf38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdf380762d0 0x7fdf38078790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005880 tx=0x7fdf3c009040 comp rx=0 tx=0).stop 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf4c0b8970 msgr2=0x7fdf4c1f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf4c0b8970 0x7fdf4c1f4e00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdf40035040 tx=0x7fdf400046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 shutdown_connections 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdf380762d0 0x7fdf38078790 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdf4c0d2260 0x7fdf4c1f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf4c0b8970 0x7fdf4c1f4e00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 --2- 172.21.15.53:0/3122643284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf4c06e4d0 0x7fdf4c0caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 >> 172.21.15.53:0/3122643284 conn(0x7fdf4c0d2dc0 msgr2=0x7fdf4c070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 shutdown_connections 2024-05-10T10:37:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.792+0000 7fdf51f04640 1 -- 172.21.15.53:0/3122643284 wait complete. 2024-05-10T10:37:05.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:37:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.908+0000 7fb2f6a9b640 1 Processor -- start 2024-05-10T10:37:05.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.908+0000 7fb2f6a9b640 1 -- start start 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2f6a9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f01598c0 0x7fb2f006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2f6a9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb2f006ec40 0x7fb2f00c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2f6a9b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2f006f1c0 con 0x7fb2f00c71c0 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2f6a9b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f006f4a0 con 0x7fb2f006ec40 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2f6a9b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f006f780 con 0x7fb2f01598c0 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2effff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb2f006ec40 0x7fb2f00c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2effff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb2f006ec40 0x7fb2f00c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54590/0 (socket says 172.21.15.53:54590) 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2effff640 1 -- 172.21.15.53:0/3107000531 learned_addr learned my addr 172.21.15.53:0/3107000531 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2f4837640 1 --2- 172.21.15.53:0/3107000531 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f01598c0 0x7fb2f006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2effff640 1 -- 172.21.15.53:0/3107000531 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f01598c0 msgr2=0x7fb2f006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2effff640 1 --2- 172.21.15.53:0/3107000531 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f01598c0 0x7fb2f006e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2effff640 1 -- 172.21.15.53:0/3107000531 >> v1:172.21.15.135:6789/0 conn(0x7fb2f00c71c0 legacy=0x7fb2f015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.909+0000 7fb2effff640 1 -- 172.21.15.53:0/3107000531 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f006fa60 con 0x7fb2f006ec40 2024-05-10T10:37:05.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.910+0000 7fb2effff640 1 --2- 172.21.15.53:0/3107000531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f006ec40 0x7fb2f00c6c80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009870 tx=0x7fb2dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d60eb391664d819 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.910+0000 7fb2ef7fe640 1 -- 172.21.15.53:0/3107000531 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc037070 con 0x7fb2f006ec40 2024-05-10T10:37:05.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.910+0000 7fb2ef7fe640 1 -- 172.21.15.53:0/3107000531 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc032440 con 0x7fb2f006ec40 2024-05-10T10:37:05.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.910+0000 7fb2ef7fe640 1 -- 172.21.15.53:0/3107000531 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc032810 con 0x7fb2f006ec40 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/3107000531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f006ec40 msgr2=0x7fb2f00c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/3107000531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f006ec40 0x7fb2f00c6c80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009870 tx=0x7fb2dc02a270 comp rx=0 tx=0).stop 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/3107000531 shutdown_connections 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/3107000531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f006ec40 0x7fb2f00c6c80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/3107000531 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f01598c0 0x7fb2f006e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/3107000531 >> 172.21.15.53:0/3107000531 conn(0x7fb2f00b2900 msgr2=0x7fb2f00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/3107000531 shutdown_connections 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.911+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/3107000531 wait complete. 2024-05-10T10:37:05.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.912+0000 7fb2f6a9b640 1 Processor -- start 2024-05-10T10:37:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.912+0000 7fb2f6a9b640 1 -- start start 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f6a9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f006ec40 0x7fb2f014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f6a9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f00c71c0 0x7fb2f0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f6a9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f01598c0 0x7fb2f0073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f6a9b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0159ca0 con 0x7fb2f00c71c0 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f6a9b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f015c6e0 con 0x7fb2f01598c0 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f6a9b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f0159e20 con 0x7fb2f006ec40 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f00c71c0 0x7fb2f0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f00c71c0 0x7fb2f0072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54600/0 (socket says 172.21.15.53:54600) 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 -- 172.21.15.53:0/4192563452 learned_addr learned my addr 172.21.15.53:0/4192563452 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:05.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f4837640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f006ec40 0x7fb2f014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2f5038640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f01598c0 0x7fb2f0073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 -- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f006ec40 msgr2=0x7fb2f014d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f006ec40 0x7fb2f014d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 -- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f01598c0 msgr2=0x7fb2f0073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f01598c0 0x7fb2f0073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.913+0000 7fb2effff640 1 -- 172.21.15.53:0/4192563452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f015f4a0 con 0x7fb2f00c71c0 2024-05-10T10:37:05.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.914+0000 7fb2effff640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f00c71c0 0x7fb2f0072e40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc000c00 tx=0x7fb2dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.914+0000 7fb2ed7fa640 1 -- 172.21.15.53:0/4192563452 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc041020 con 0x7fb2f00c71c0 2024-05-10T10:37:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.914+0000 7fb2ed7fa640 1 -- 172.21.15.53:0/4192563452 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc0306c0 con 0x7fb2f00c71c0 2024-05-10T10:37:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.914+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01f3160 con 0x7fb2f00c71c0 2024-05-10T10:37:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.914+0000 7fb2ed7fa640 1 -- 172.21.15.53:0/4192563452 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc037040 con 0x7fb2f00c71c0 2024-05-10T10:37:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.914+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01f3720 con 0x7fb2f00c71c0 2024-05-10T10:37:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.915+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f006a220 con 0x7fb2f00c71c0 2024-05-10T10:37:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.915+0000 7fb2ed7fa640 1 -- 172.21.15.53:0/4192563452 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb2dc044020 con 0x7fb2f00c71c0 2024-05-10T10:37:05.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.916+0000 7fb2ed7fa640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2d40762d0 0x7fb2d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:05.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.916+0000 7fb2ed7fa640 1 -- 172.21.15.53:0/4192563452 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fb2dc0b7f80 con 0x7fb2f00c71c0 2024-05-10T10:37:05.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.916+0000 7fb2f4837640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2d40762d0 0x7fb2d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:05.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.917+0000 7fb2f4837640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2d40762d0 0x7fb2d4078790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4006fd0 tx=0x7fb2e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:05.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:05.920+0000 7fb2ed7fa640 1 -- 172.21.15.53:0/4192563452 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb2dc04b030 con 0x7fb2f00c71c0 2024-05-10T10:37:06.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.149+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb2f006e4e0 con 0x7fb2f00c71c0 2024-05-10T10:37:06.151 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:37:06.150+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:37:06.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.151+0000 7fb2ed7fa640 1 -- 172.21.15.53:0/4192563452 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7fb2dc080fd0 con 0x7fb2f00c71c0 2024-05-10T10:37:06.153 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:37:06.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2d40762d0 msgr2=0x7fb2d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2d40762d0 0x7fb2d4078790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4006fd0 tx=0x7fb2e4008040 comp rx=0 tx=0).stop 2024-05-10T10:37:06.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f00c71c0 msgr2=0x7fb2f0072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f00c71c0 0x7fb2f0072e40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc000c00 tx=0x7fb2dc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 shutdown_connections 2024-05-10T10:37:06.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2d40762d0 0x7fb2d4078790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f01598c0 0x7fb2f0073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f00c71c0 0x7fb2f0072e40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 --2- 172.21.15.53:0/4192563452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f006ec40 0x7fb2f014d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 >> 172.21.15.53:0/4192563452 conn(0x7fb2f00b2900 msgr2=0x7fb2f00c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:06.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 shutdown_connections 2024-05-10T10:37:06.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.154+0000 7fb2f6a9b640 1 -- 172.21.15.53:0/4192563452 wait complete. 2024-05-10T10:37:06.159 INFO:tasks.ceph.mds.c.smithi053.stderr: -7654> 2024-05-10T10:35:54.350+0000 7f9df95b9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:37:06.161 INFO:tasks.ceph.mds.c.smithi053.stderr: -4763> 2024-05-10T10:36:00.367+0000 7f9df55b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5165=0-4194304@1} caps={5165=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5165 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55ebd1e6cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:37:06.163 INFO:tasks.ceph.mds.c.smithi053.stderr: -2944> 2024-05-10T10:36:31.736+0000 7f9df55b1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5165=0-4194304@1} caps={5165=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5165 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ebd1e6cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:37:06.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:37:06.191 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:37:06.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.273+0000 7f32f8358640 1 Processor -- start 2024-05-10T10:37:06.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.273+0000 7f32f8358640 1 -- start start 2024-05-10T10:37:06.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.274+0000 7f32f8358640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f32f015f410 0x7f32f00d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.274+0000 7f32f8358640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f00d2800 con 0x7f32f00b56a0 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.274+0000 7f32f8358640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f00d2ae0 con 0x7f32f00d0f10 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.274+0000 7f32f8358640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f32f00d01d0 con 0x7f32f015f410 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.274+0000 7f32f68f5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f32f00b56a0 0x7f32f00d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43718/0 (socket says 172.21.15.53:43718) 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.274+0000 7f32f68f5640 1 -- 172.21.15.53:0/3505418668 learned_addr learned my addr 172.21.15.53:0/3505418668 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f60f4640 1 --2- 172.21.15.53:0/3505418668 >> v2:172.21.15.135:3300/0 conn(0x7f32f015f410 0x7f32f00d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588428873 0 0) 0x7f32f00d2800 con 0x7f32f00b56a0 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32dc003520 con 0x7f32f00b56a0 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 702408296 0 0) 0x7f32dc003520 con 0x7f32f00b56a0 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 >> v2:172.21.15.135:3300/0 conn(0x7f32f015f410 msgr2=0x7f32f00d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 --2- 172.21.15.53:0/3505418668 >> v2:172.21.15.135:3300/0 conn(0x7f32f015f410 0x7f32f00d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 >> v1:172.21.15.135:6789/0 conn(0x7f32f00d0f10 legacy=0x7f32f00b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f00d04b0 con 0x7f32f00b56a0 2024-05-10T10:37:06.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f32e80019f0 con 0x7f32f00b56a0 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 >> v1:172.21.15.53:6789/0 conn(0x7f32f00b56a0 legacy=0x7f32f00d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc004800 0x7f32dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32dc0051c0 0x7f32dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00d01d0 con 0x7f32dc0039b0 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc00c160 con 0x7f32dc004800 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc00c360 con 0x7f32dc0051c0 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f60f4640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f68f5640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32dc0051c0 0x7f32dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f58f3640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc004800 0x7f32dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f60f4640 1 -- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32dc0051c0 msgr2=0x7f32dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f60f4640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32dc0051c0 0x7f32dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f60f4640 1 -- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc004800 msgr2=0x7f32dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f60f4640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc004800 0x7f32dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f60f4640 1 -- 172.21.15.53:0/3505418668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc003520 con 0x7f32dc0039b0 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.275+0000 7f32f68f5640 1 -- 172.21.15.53:0/3505418668 reap_dead start 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.276+0000 7f32f60f4640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ac0 tx=0x7f32dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac908577d665be server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.276+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec036070 con 0x7f32dc0039b0 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.276+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ec031400 con 0x7f32dc0039b0 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.276+0000 7f32f50f2640 1 -- 172.21.15.53:0/3505418668 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec009690 con 0x7f32dc0039b0 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.277+0000 7f32f8358640 1 -- 172.21.15.53:0/3505418668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32dc0039b0 msgr2=0x7f32dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.277+0000 7f32f8358640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ac0 tx=0x7f32dc005830 comp rx=0 tx=0).stop 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.277+0000 7f32f8358640 1 -- 172.21.15.53:0/3505418668 shutdown_connections 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.277+0000 7f32f8358640 1 --2- 172.21.15.53:0/3505418668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.277+0000 7f32f8358640 1 -- 172.21.15.53:0/3505418668 >> 172.21.15.53:0/3505418668 conn(0x7f32f008ed30 msgr2=0x7f32f009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.277+0000 7f32f8358640 1 -- 172.21.15.53:0/3505418668 shutdown_connections 2024-05-10T10:37:06.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.277+0000 7f32f8358640 1 -- 172.21.15.53:0/3505418668 wait complete. 2024-05-10T10:37:06.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.278+0000 7f32f8358640 1 Processor -- start 2024-05-10T10:37:06.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.278+0000 7f32f8358640 1 -- start start 2024-05-10T10:37:06.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f8358640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc0039b0 0x7f32f01607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f8358640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32f01506e0 0x7f32f006e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f8358640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32f006e710 0x7f32f0161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f8358640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00cffb0 con 0x7f32f006e710 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f8358640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00cfe30 con 0x7f32dc0039b0 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f8358640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f32f00d09d0 con 0x7f32f01506e0 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32f006e710 0x7f32f0161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32f006e710 0x7f32f0161380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54632/0 (socket says 172.21.15.53:54632) 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 -- 172.21.15.53:0/1612686496 learned_addr learned my addr 172.21.15.53:0/1612686496 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f60f4640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc0039b0 0x7f32f01607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f58f3640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32f01506e0 0x7f32f006e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 -- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32f01506e0 msgr2=0x7f32f006e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32f01506e0 0x7f32f006e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 -- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc0039b0 msgr2=0x7f32f01607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc0039b0 0x7f32f01607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:06.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.279+0000 7f32f68f5640 1 -- 172.21.15.53:0/1612686496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f0162120 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.280+0000 7f32f68f5640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32f006e710 0x7f32f0161380 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f32f008f5f0 tx=0x7f32e800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.280+0000 7f32e6ffd640 1 -- 172.21.15.53:0/1612686496 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e800ebb0 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.280+0000 7f32e6ffd640 1 -- 172.21.15.53:0/1612686496 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e800ed50 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.280+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f015fa00 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.280+0000 7f32e6ffd640 1 -- 172.21.15.53:0/1612686496 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8014670 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.280+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f015ffc0 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.281+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f006a220 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.281+0000 7f32e6ffd640 1 -- 172.21.15.53:0/1612686496 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f32e8014810 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.282+0000 7f32e6ffd640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32d40762d0 0x7f32d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.282+0000 7f32e6ffd640 1 -- 172.21.15.53:0/1612686496 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6332+0+0 (secure 0 0 0) 0x7f32e8097610 con 0x7f32f006e710 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.282+0000 7f32f60f4640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32d40762d0 0x7f32d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.283+0000 7f32f60f4640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32d40762d0 0x7f32d4078790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f32ec033000 tx=0x7f32ec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:06.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.286+0000 7f32e6ffd640 1 -- 172.21.15.53:0/1612686496 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f32e8060c60 con 0x7f32f006e710 2024-05-10T10:37:06.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:06.516+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f32f00d6090 con 0x7f32f006e710 2024-05-10T10:37:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.155+0000 7f32e6ffd640 1 -- 172.21.15.53:0/1612686496 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f32e8060560 con 0x7f32f006e710 2024-05-10T10:37:07.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32d40762d0 msgr2=0x7f32d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:07.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32d40762d0 0x7f32d4078790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f32ec033000 tx=0x7f32ec008030 comp rx=0 tx=0).stop 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32f006e710 msgr2=0x7f32f0161380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32f006e710 0x7f32f0161380 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f32f008f5f0 tx=0x7f32e800e530 comp rx=0 tx=0).stop 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 shutdown_connections 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f32d40762d0 0x7f32d4078790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32f006e710 0x7f32f0161380 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f32f01506e0 0x7f32f006e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 --2- 172.21.15.53:0/1612686496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32dc0039b0 0x7f32f01607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.158+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 >> 172.21.15.53:0/1612686496 conn(0x7f32f008ed30 msgr2=0x7f32f015dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.159+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 shutdown_connections 2024-05-10T10:37:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.159+0000 7f32f8358640 1 -- 172.21.15.53:0/1612686496 wait complete. 2024-05-10T10:37:07.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:37:07.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.328+0000 7ff6efa2a640 1 Processor -- start 2024-05-10T10:37:07.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.329+0000 7ff6efa2a640 1 -- start start 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.329+0000 7ff6efa2a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6e815f490 0x7ff6e80d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.329+0000 7ff6efa2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d5ca0 0x7ff6e80d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.329+0000 7ff6efa2a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6e80d1080 con 0x7ff6e80d3b80 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.329+0000 7ff6efa2a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff6e80b4e70 con 0x7ff6e815f490 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.329+0000 7ff6efa2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e80b4820 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ed7c6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6e815f490 0x7ff6e80d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ed7c6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6e815f490 0x7ff6e80d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54638/0 (socket says 172.21.15.53:54638) 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ed7c6640 1 -- 172.21.15.53:0/2437467920 learned_addr learned my addr 172.21.15.53:0/2437467920 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ed7c6640 1 -- 172.21.15.53:0/2437467920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d5ca0 msgr2=0x7ff6e80d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ecfc5640 1 --2- 172.21.15.53:0/2437467920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d5ca0 0x7ff6e80d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ed7c6640 1 --2- 172.21.15.53:0/2437467920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d5ca0 0x7ff6e80d3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ed7c6640 1 -- 172.21.15.53:0/2437467920 >> v1:172.21.15.135:6789/0 conn(0x7ff6e80d3b80 legacy=0x7ff6e80cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:07.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ed7c6640 1 -- 172.21.15.53:0/2437467920 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e80b4b00 con 0x7ff6e815f490 2024-05-10T10:37:07.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.330+0000 7ff6ecfc5640 1 --2- 172.21.15.53:0/2437467920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d5ca0 0x7ff6e80d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:07.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6ed7c6640 1 --2- 172.21.15.53:0/2437467920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e815f490 0x7ff6e80d2f60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009870 tx=0x7ff6e402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2166db994d6fd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:07.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6dffff640 1 -- 172.21.15.53:0/2437467920 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4037070 con 0x7ff6e815f490 2024-05-10T10:37:07.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6dffff640 1 -- 172.21.15.53:0/2437467920 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e4032440 con 0x7ff6e815f490 2024-05-10T10:37:07.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6dffff640 1 -- 172.21.15.53:0/2437467920 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4032810 con 0x7ff6e815f490 2024-05-10T10:37:07.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6efa2a640 1 -- 172.21.15.53:0/2437467920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e815f490 msgr2=0x7ff6e80d2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:07.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/2437467920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e815f490 0x7ff6e80d2f60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009870 tx=0x7ff6e402a1c0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6efa2a640 1 -- 172.21.15.53:0/2437467920 shutdown_connections 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/2437467920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d5ca0 0x7ff6e80d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/2437467920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e815f490 0x7ff6e80d2f60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.331+0000 7ff6efa2a640 1 -- 172.21.15.53:0/2437467920 >> 172.21.15.53:0/2437467920 conn(0x7ff6e80d50f0 msgr2=0x7ff6e80d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.332+0000 7ff6efa2a640 1 -- 172.21.15.53:0/2437467920 shutdown_connections 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.332+0000 7ff6efa2a640 1 -- 172.21.15.53:0/2437467920 wait complete. 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 Processor -- start 2024-05-10T10:37:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 -- start start 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d3b80 0x7ff6e815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e80d5ca0 0x7ff6e80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6e815f490 0x7ff6e80cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e80b6310 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e815fa80 con 0x7ff6e815f490 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.333+0000 7ff6efa2a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e80b6490 con 0x7ff6e80d3b80 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e80d5ca0 0x7ff6e80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e80d5ca0 0x7ff6e80cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54644/0 (socket says 172.21.15.53:54644) 2024-05-10T10:37:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 -- 172.21.15.53:0/3731107967 learned_addr learned my addr 172.21.15.53:0/3731107967 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6edfc7640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6e815f490 0x7ff6e80cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ed7c6640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d3b80 0x7ff6e815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 -- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d3b80 msgr2=0x7ff6e815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d3b80 0x7ff6e815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 -- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6e815f490 msgr2=0x7ff6e80cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6e815f490 0x7ff6e80cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 -- 172.21.15.53:0/3731107967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e81e6e50 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6edfc7640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6e815f490 0x7ff6e80cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ed7c6640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d3b80 0x7ff6e815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ecfc5640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e80d5ca0 0x7ff6e80cda60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c7f0 tx=0x7ff6d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ddffb640 1 -- 172.21.15.53:0/3731107967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8015070 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ddffb640 1 -- 172.21.15.53:0/3731107967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d8007a30 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e81e7130 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.334+0000 7ff6ddffb640 1 -- 172.21.15.53:0/3731107967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8007e00 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.335+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e81e76f0 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.335+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e81e78f0 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.336+0000 7ff6ddffb640 1 -- 172.21.15.53:0/3731107967 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff6d8018070 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.337+0000 7ff6ddffb640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6cc0762d0 0x7ff6cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:07.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.337+0000 7ff6ddffb640 1 -- 172.21.15.53:0/3731107967 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6332+0+0 (secure 0 0 0) 0x7ff6d8097870 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.337+0000 7ff6ed7c6640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6cc0762d0 0x7ff6cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:07.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.338+0000 7ff6ed7c6640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6cc0762d0 0x7ff6cc078790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4034000 tx=0x7ff6e402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:07.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.341+0000 7ff6ddffb640 1 -- 172.21.15.53:0/3731107967 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff6e81e78f0 con 0x7ff6e80d5ca0 2024-05-10T10:37:07.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:07.530+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6e806a220 con 0x7ff6e80d5ca0 2024-05-10T10:37:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.157+0000 7ff6ddffb640 1 -- 172.21.15.53:0/3731107967 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) ==== 181+0+0 (secure 0 0 0) 0x7ff6d8060f60 con 0x7ff6e80d5ca0 2024-05-10T10:37:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:37:08.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6cc0762d0 msgr2=0x7ff6cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:08.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6cc0762d0 0x7ff6cc078790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4034000 tx=0x7ff6e402e000 comp rx=0 tx=0).stop 2024-05-10T10:37:08.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e80d5ca0 msgr2=0x7ff6e80cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:08.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e80d5ca0 0x7ff6e80cda60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c7f0 tx=0x7ff6d800ccb0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 shutdown_connections 2024-05-10T10:37:08.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff6cc0762d0 0x7ff6cc078790 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6e815f490 0x7ff6e80cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6e80d5ca0 0x7ff6e80cda60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 --2- 172.21.15.53:0/3731107967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6e80d3b80 0x7ff6e815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 >> 172.21.15.53:0/3731107967 conn(0x7ff6e80d50f0 msgr2=0x7ff6e806de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:08.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 shutdown_connections 2024-05-10T10:37:08.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.161+0000 7ff6efa2a640 1 -- 172.21.15.53:0/3731107967 wait complete. 2024-05-10T10:37:08.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:37:08.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.290+0000 7f5792654640 1 Processor -- start 2024-05-10T10:37:08.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.290+0000 7f5792654640 1 -- start start 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f5792654640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f578c15d010 0x7f578c0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f5792654640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578c0add50 con 0x7f578c0d04e0 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f5792654640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578c0ae030 con 0x7f578c0d1540 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f5792654640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f578c0b5af0 con 0x7f578c15d010 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f578b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f578c0d04e0 0x7f578c0d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43722/0 (socket says 172.21.15.53:43722) 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f578b7fe640 1 -- 172.21.15.53:0/2885227981 learned_addr learned my addr 172.21.15.53:0/2885227981 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f578bfff640 1 --2- 172.21.15.53:0/2885227981 >> v2:172.21.15.135:3300/0 conn(0x7f578c15d010 0x7f578c0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f578affd640 1 -- 172.21.15.53:0/2885227981 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149680134 0 0) 0x7f578c0add50 con 0x7f578c0d04e0 2024-05-10T10:37:08.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.291+0000 7f578affd640 1 -- 172.21.15.53:0/2885227981 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5774003520 con 0x7f578c0d04e0 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.292+0000 7f578bfff640 1 -- 172.21.15.53:0/2885227981 >> v1:172.21.15.135:6789/0 conn(0x7f578c0d1540 legacy=0x7f578c0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.292+0000 7f578bfff640 1 -- 172.21.15.53:0/2885227981 >> v1:172.21.15.53:6789/0 conn(0x7f578c0d04e0 legacy=0x7f578c0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.292+0000 7f578affd640 1 -- 172.21.15.53:0/2885227981 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1271154928 0 0) 0x7f5774003520 con 0x7f578c0d04e0 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.292+0000 7f578bfff640 1 -- 172.21.15.53:0/2885227981 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c0b5dd0 con 0x7f578c15d010 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.292+0000 7f578bfff640 1 --2- 172.21.15.53:0/2885227981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c15d010 0x7f578c0cfe90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f577c0097a0 tx=0x7f577c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b59d7f5417b191c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f578affd640 1 -- 172.21.15.53:0/2885227981 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c037070 con 0x7f578c15d010 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f578affd640 1 -- 172.21.15.53:0/2885227981 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c032440 con 0x7f578c15d010 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f578affd640 1 -- 172.21.15.53:0/2885227981 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0327f0 con 0x7f578c15d010 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f5792654640 1 -- 172.21.15.53:0/2885227981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c15d010 msgr2=0x7f578c0cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f5792654640 1 --2- 172.21.15.53:0/2885227981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c15d010 0x7f578c0cfe90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f577c0097a0 tx=0x7f577c02a230 comp rx=0 tx=0).stop 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f5792654640 1 -- 172.21.15.53:0/2885227981 shutdown_connections 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f5792654640 1 --2- 172.21.15.53:0/2885227981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c15d010 0x7f578c0cfe90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f5792654640 1 -- 172.21.15.53:0/2885227981 >> 172.21.15.53:0/2885227981 conn(0x7f578c06c4a0 msgr2=0x7f578c06c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:08.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f5792654640 1 -- 172.21.15.53:0/2885227981 shutdown_connections 2024-05-10T10:37:08.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.293+0000 7f5792654640 1 -- 172.21.15.53:0/2885227981 wait complete. 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.294+0000 7f5792654640 1 Processor -- start 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f5792654640 1 -- start start 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f5792654640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c0d04e0 0x7f578c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f5792654640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f578c0d1540 0x7f578c1f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f5792654640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f578c15d010 0x7f578c1f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f5792654640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f578c0b4490 con 0x7f578c0d1540 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f5792654640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f578c0b4310 con 0x7f578c0d04e0 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f5792654640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f578c0b4610 con 0x7f578c15d010 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f578b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f578c0d1540 0x7f578c1f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f578b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f578c0d1540 0x7f578c1f8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54648/0 (socket says 172.21.15.53:54648) 2024-05-10T10:37:08.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.295+0000 7f578b7fe640 1 -- 172.21.15.53:0/4032913450 learned_addr learned my addr 172.21.15.53:0/4032913450 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f578bfff640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c0d04e0 0x7f578c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f5790bf1640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f578c15d010 0x7f578c1f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f578b7fe640 1 -- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f578c15d010 msgr2=0x7f578c1f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f578b7fe640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f578c15d010 0x7f578c1f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f578b7fe640 1 -- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c0d04e0 msgr2=0x7f578c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f578b7fe640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c0d04e0 0x7f578c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f578b7fe640 1 -- 172.21.15.53:0/4032913450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c072830 con 0x7f578c0d1540 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f578b7fe640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f578c0d1540 0x7f578c1f8410 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5780001eb0 tx=0x7f578000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f5788ff9640 1 -- 172.21.15.53:0/4032913450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57800074e0 con 0x7f578c0d1540 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f5788ff9640 1 -- 172.21.15.53:0/4032913450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5780007680 con 0x7f578c0d1540 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578c072b10 con 0x7f578c0d1540 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f5788ff9640 1 -- 172.21.15.53:0/4032913450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780004a00 con 0x7f578c0d1540 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.296+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578c0730d0 con 0x7f578c0d1540 2024-05-10T10:37:08.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.297+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578c06a220 con 0x7f578c0d1540 2024-05-10T10:37:08.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.298+0000 7f5788ff9640 1 -- 172.21.15.53:0/4032913450 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5780011070 con 0x7f578c0d1540 2024-05-10T10:37:08.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.299+0000 7f5788ff9640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57780762d0 0x7f5778078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:08.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.299+0000 7f5788ff9640 1 -- 172.21.15.53:0/4032913450 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5865+0+0 (secure 0 0 0) 0x7f57800972e0 con 0x7f578c0d1540 2024-05-10T10:37:08.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.299+0000 7f578bfff640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57780762d0 0x7f5778078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:08.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.299+0000 7f578bfff640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57780762d0 0x7f5778078790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f577c034000 tx=0x7f577c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:08.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.303+0000 7f5788ff9640 1 -- 172.21.15.53:0/4032913450 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5780060ba0 con 0x7f578c0d1540 2024-05-10T10:37:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:08.496+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f578c157960 con 0x7f578c0d1540 2024-05-10T10:37:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.162+0000 7f5788ff9640 1 -- 172.21.15.53:0/4032913450 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) ==== 169+0+0 (secure 0 0 0) 0x7f57800604a0 con 0x7f578c0d1540 2024-05-10T10:37:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:37:09.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57780762d0 msgr2=0x7f5778078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57780762d0 0x7f5778078790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f577c034000 tx=0x7f577c02e000 comp rx=0 tx=0).stop 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f578c0d1540 msgr2=0x7f578c1f8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f578c0d1540 0x7f578c1f8410 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5780001eb0 tx=0x7f578000c8e0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 shutdown_connections 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57780762d0 0x7f5778078790 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f577c034000 tx=0x7f577c02e000 comp rx=0 tx=0).stop 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f578c15d010 0x7f578c1f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f578c0d1540 0x7f578c1f8410 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 --2- 172.21.15.53:0/4032913450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f578c0d04e0 0x7f578c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 >> 172.21.15.53:0/4032913450 conn(0x7f578c06c4a0 msgr2=0x7f578c0b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.166+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 shutdown_connections 2024-05-10T10:37:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.167+0000 7f5792654640 1 -- 172.21.15.53:0/4032913450 wait complete. 2024-05-10T10:37:09.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T10:37:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.293+0000 7fe63f61c640 1 Processor -- start 2024-05-10T10:37:09.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.294+0000 7fe63f61c640 1 -- start start 2024-05-10T10:37:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63f61c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe638162a80 0x7fe6380d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63f61c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6380d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63f61c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6380d3410 con 0x7fe6380d1720 2024-05-10T10:37:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63f61c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe6380b64b0 con 0x7fe638162a80 2024-05-10T10:37:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63f61c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe6380d08d0 con 0x7fe6380cfc00 2024-05-10T10:37:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63d3b8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe638162a80 0x7fe6380d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63cbb7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6380d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63dbb9640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe6380d1720 0x7fe6380d1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:44628/0 (socket says 172.21.15.53:44628) 2024-05-10T10:37:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63dbb9640 1 -- 172.21.15.53:0/2558535205 learned_addr learned my addr 172.21.15.53:0/2558535205 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63d3b8640 1 -- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 msgr2=0x7fe6380d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63d3b8640 1 --2- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6380d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63d3b8640 1 -- 172.21.15.53:0/2558535205 >> v1:172.21.15.135:6789/0 conn(0x7fe6380d1720 legacy=0x7fe6380d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63d3b8640 1 -- 172.21.15.53:0/2558535205 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380d0bb0 con 0x7fe638162a80 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.295+0000 7fe63cbb7640 1 --2- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6380d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.296+0000 7fe63d3b8640 1 --2- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe638162a80 0x7fe6380d2ed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe6340097a0 tx=0x7fe63402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76179ca8383310a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.296+0000 7fe62bfff640 1 -- 172.21.15.53:0/2558535205 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634037070 con 0x7fe638162a80 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.296+0000 7fe62bfff640 1 -- 172.21.15.53:0/2558535205 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe634032440 con 0x7fe638162a80 2024-05-10T10:37:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.296+0000 7fe62bfff640 1 -- 172.21.15.53:0/2558535205 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634032810 con 0x7fe638162a80 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 -- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe638162a80 msgr2=0x7fe6380d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe638162a80 0x7fe6380d2ed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe6340097a0 tx=0x7fe63402a210 comp rx=0 tx=0).stop 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 -- 172.21.15.53:0/2558535205 shutdown_connections 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6380d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2558535205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe638162a80 0x7fe6380d2ed0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 -- 172.21.15.53:0/2558535205 >> 172.21.15.53:0/2558535205 conn(0x7fe6380d50f0 msgr2=0x7fe6380d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 -- 172.21.15.53:0/2558535205 shutdown_connections 2024-05-10T10:37:09.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.297+0000 7fe63f61c640 1 -- 172.21.15.53:0/2558535205 wait complete. 2024-05-10T10:37:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.298+0000 7fe63f61c640 1 Processor -- start 2024-05-10T10:37:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.298+0000 7fe63f61c640 1 -- start start 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63f61c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6381f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63f61c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6380d1720 0x7fe6380b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63f61c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe638162a80 0x7fe63815f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63f61c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6380b6140 con 0x7fe638162a80 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63f61c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6380b6790 con 0x7fe6380d1720 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63f61c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe6380d5e40 con 0x7fe6380cfc00 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63dbb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe638162a80 0x7fe63815f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63dbb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe638162a80 0x7fe63815f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54654/0 (socket says 172.21.15.53:54654) 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63dbb9640 1 -- 172.21.15.53:0/2931974828 learned_addr learned my addr 172.21.15.53:0/2931974828 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63d3b8640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6381f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.299+0000 7fe63cbb7640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6380d1720 0x7fe6380b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63dbb9640 1 -- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 msgr2=0x7fe6381f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63dbb9640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6381f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63dbb9640 1 -- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6380d1720 msgr2=0x7fe6380b9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63dbb9640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6380d1720 0x7fe6380b9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63dbb9640 1 -- 172.21.15.53:0/2931974828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6381f5e90 con 0x7fe638162a80 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63dbb9640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe638162a80 0x7fe63815f820 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe63000cbc0 tx=0x7fe63000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe629ffb640 1 -- 172.21.15.53:0/2931974828 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630017070 con 0x7fe638162a80 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe629ffb640 1 -- 172.21.15.53:0/2931974828 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63000ad40 con 0x7fe638162a80 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6380ba000 con 0x7fe638162a80 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe629ffb640 1 -- 172.21.15.53:0/2931974828 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630012600 con 0x7fe638162a80 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.300+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6381f58d0 con 0x7fe638162a80 2024-05-10T10:37:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.301+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6380693f0 con 0x7fe638162a80 2024-05-10T10:37:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.302+0000 7fe629ffb640 1 -- 172.21.15.53:0/2931974828 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe63001a070 con 0x7fe638162a80 2024-05-10T10:37:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.302+0000 7fe629ffb640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe61c0762d0 0x7fe61c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.302+0000 7fe629ffb640 1 -- 172.21.15.53:0/2931974828 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7fe630097140 con 0x7fe638162a80 2024-05-10T10:37:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.302+0000 7fe63d3b8640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe61c0762d0 0x7fe61c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.303+0000 7fe63d3b8640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe61c0762d0 0x7fe61c078790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe634034000 tx=0x7fe63402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.307+0000 7fe629ffb640 1 -- 172.21.15.53:0/2931974828 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe6380ba000 con 0x7fe638162a80 2024-05-10T10:37:09.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.504+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe638160e40 con 0x7fe638162a80 2024-05-10T10:37:09.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.507+0000 7fe629ffb640 1 -- 172.21.15.53:0/2931974828 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fe6300605c0 con 0x7fe638162a80 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe61c0762d0 msgr2=0x7fe61c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe61c0762d0 0x7fe61c078790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe634034000 tx=0x7fe63402e000 comp rx=0 tx=0).stop 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe638162a80 msgr2=0x7fe63815f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe638162a80 0x7fe63815f820 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe63000cbc0 tx=0x7fe63000a430 comp rx=0 tx=0).stop 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 shutdown_connections 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe61c0762d0 0x7fe61c078790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe638162a80 0x7fe63815f820 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6380d1720 0x7fe6380b9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 --2- 172.21.15.53:0/2931974828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6380cfc00 0x7fe6381f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 >> 172.21.15.53:0/2931974828 conn(0x7fe6380d50f0 msgr2=0x7fe63806dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 shutdown_connections 2024-05-10T10:37:09.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.509+0000 7fe63f61c640 1 -- 172.21.15.53:0/2931974828 wait complete. 2024-05-10T10:37:09.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-10T10:37:09.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.625+0000 7f0925912640 1 Processor -- start 2024-05-10T10:37:09.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.625+0000 7f0925912640 1 -- start start 2024-05-10T10:37:09.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f0925912640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200c6c80 0x7f09200c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f0925912640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f09200cee40 0x7f09200cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f0925912640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f092006f1c0 con 0x7f09201598c0 2024-05-10T10:37:09.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f0925912640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f09200c7620 con 0x7f09200cee40 2024-05-10T10:37:09.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f0925912640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f092006f4a0 con 0x7f09200c6c80 2024-05-10T10:37:09.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f091f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f09200cee40 0x7f09200cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f091effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f09201598c0 0x7f092006e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43726/0 (socket says 172.21.15.53:43726) 2024-05-10T10:37:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f091effd640 1 -- 172.21.15.53:0/754588356 learned_addr learned my addr 172.21.15.53:0/754588356 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.626+0000 7f091e7fc640 1 --2- 172.21.15.53:0/754588356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200c6c80 0x7f09200c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091f7fe640 1 -- 172.21.15.53:0/754588356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200c6c80 msgr2=0x7f09200c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091f7fe640 1 --2- 172.21.15.53:0/754588356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200c6c80 0x7f09200c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091f7fe640 1 -- 172.21.15.53:0/754588356 >> v1:172.21.15.53:6789/0 conn(0x7f09201598c0 legacy=0x7f092006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091f7fe640 1 -- 172.21.15.53:0/754588356 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092006f780 con 0x7f09200cee40 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091f7fe640 1 --2- 172.21.15.53:0/754588356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200cee40 0x7f09200cf2d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0918009d90 tx=0x7f091802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f84e425fd2abd26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091dffb640 1 -- 172.21.15.53:0/754588356 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918007740 con 0x7f09200cee40 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091dffb640 1 -- 172.21.15.53:0/754588356 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09180078e0 con 0x7f09200cee40 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.627+0000 7f091dffb640 1 -- 172.21.15.53:0/754588356 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918007cb0 con 0x7f09200cee40 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 -- 172.21.15.53:0/754588356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200cee40 msgr2=0x7f09200cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 --2- 172.21.15.53:0/754588356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200cee40 0x7f09200cf2d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0918009d90 tx=0x7f091802a660 comp rx=0 tx=0).stop 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 -- 172.21.15.53:0/754588356 shutdown_connections 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 --2- 172.21.15.53:0/754588356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200cee40 0x7f09200cf2d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 --2- 172.21.15.53:0/754588356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200c6c80 0x7f09200c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 -- 172.21.15.53:0/754588356 >> 172.21.15.53:0/754588356 conn(0x7f09200b2900 msgr2=0x7f09200b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 -- 172.21.15.53:0/754588356 shutdown_connections 2024-05-10T10:37:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.628+0000 7f0925912640 1 -- 172.21.15.53:0/754588356 wait complete. 2024-05-10T10:37:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.629+0000 7f0925912640 1 Processor -- start 2024-05-10T10:37:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.629+0000 7f0925912640 1 -- start start 2024-05-10T10:37:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f0925912640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200c6c80 0x7f092014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f0925912640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200cee40 0x7f09200cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f0925912640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09201598c0 0x7f09200c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f0925912640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f092015c6e0 con 0x7f09200c6c80 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f0925912640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09200ce920 con 0x7f09201598c0 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f0925912640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f092006ee10 con 0x7f09200cee40 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f091effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200c6c80 0x7f092014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f091effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200c6c80 0x7f092014fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54676/0 (socket says 172.21.15.53:54676) 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f091effd640 1 -- 172.21.15.53:0/1273853950 learned_addr learned my addr 172.21.15.53:0/1273853950 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f091e7fc640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200cee40 0x7f09200cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.630+0000 7f091f7fe640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09201598c0 0x7f09200c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f091effd640 1 -- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200cee40 msgr2=0x7f09200cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f091effd640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200cee40 0x7f09200cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f091effd640 1 -- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09201598c0 msgr2=0x7f09200c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f091effd640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09201598c0 0x7f09200c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f091effd640 1 -- 172.21.15.53:0/1273853950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09201502e0 con 0x7f09200c6c80 2024-05-10T10:37:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f091effd640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200c6c80 0x7f092014fda0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f091000c3c0 tx=0x7f091000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f0907fff640 1 -- 172.21.15.53:0/1273853950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910017070 con 0x7f09200c6c80 2024-05-10T10:37:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f0907fff640 1 -- 172.21.15.53:0/1273853950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0910005ce0 con 0x7f09200c6c80 2024-05-10T10:37:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f0907fff640 1 -- 172.21.15.53:0/1273853950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100072f0 con 0x7f09200c6c80 2024-05-10T10:37:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09200af2d0 con 0x7f09200c6c80 2024-05-10T10:37:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.631+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09200af890 con 0x7f09200c6c80 2024-05-10T10:37:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.632+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0920068ca0 con 0x7f09200c6c80 2024-05-10T10:37:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.633+0000 7f0907fff640 1 -- 172.21.15.53:0/1273853950 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f091001a070 con 0x7f09200c6c80 2024-05-10T10:37:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.633+0000 7f0907fff640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f090c0762d0 0x7f090c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.633+0000 7f0907fff640 1 -- 172.21.15.53:0/1273853950 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f0910096760 con 0x7f09200c6c80 2024-05-10T10:37:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.633+0000 7f091e7fc640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f090c0762d0 0x7f090c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:09.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.634+0000 7f091e7fc640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f090c0762d0 0x7f090c078790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f092015cc70 tx=0x7f0914008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:09.638 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.637+0000 7f0907fff640 1 -- 172.21.15.53:0/1273853950 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0920068ca0 con 0x7f09200c6c80 2024-05-10T10:37:09.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:09.902+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f092006a220 con 0x7f09200c6c80 2024-05-10T10:37:10.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.164+0000 7f0907fff640 1 -- 172.21.15.53:0/1273853950 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v185) ==== 169+0+0 (secure 0 0 0) 0x7f09100602c0 con 0x7f09200c6c80 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f090c0762d0 msgr2=0x7f090c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f090c0762d0 0x7f090c078790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f092015cc70 tx=0x7f0914008040 comp rx=0 tx=0).stop 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200c6c80 msgr2=0x7f092014fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200c6c80 0x7f092014fda0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f091000c3c0 tx=0x7f091000c880 comp rx=0 tx=0).stop 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 shutdown_connections 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f090c0762d0 0x7f090c078790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09201598c0 0x7f09200c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09200cee40 0x7f09200cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 --2- 172.21.15.53:0/1273853950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09200c6c80 0x7f092014fda0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 >> 172.21.15.53:0/1273853950 conn(0x7f09200b2900 msgr2=0x7f092006e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 shutdown_connections 2024-05-10T10:37:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.166+0000 7f0925912640 1 -- 172.21.15.53:0/1273853950 wait complete. 2024-05-10T10:37:10.180 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-10T10:37:10.180 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-10T10:37:10.180 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-10T10:37:10.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:37:10.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.284+0000 7fbf3086d640 1 Processor -- start 2024-05-10T10:37:10.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.284+0000 7fbf3086d640 1 -- start start 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf3086d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf2c15f8c0 0x7fbf2c06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf3086d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c0b0a90 con 0x7fbf2c06e620 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf3086d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c0b3540 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf3086d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c0b23f0 con 0x7fbf2c15f8c0 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf29d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf2c15f8c0 0x7fbf2c06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf2ad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbf2c06e620 0x7fbf2c0d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43730/0 (socket says 172.21.15.53:43730) 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf29d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf2c15f8c0 0x7fbf2c06e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54682/0 (socket says 172.21.15.53:54682) 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf2ad9d640 1 -- 172.21.15.53:0/3557858543 learned_addr learned my addr 172.21.15.53:0/3557858543 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf29d9b640 1 -- 172.21.15.53:0/3557858543 >> v1:172.21.15.135:6789/0 conn(0x7fbf2c15f4e0 legacy=0x7fbf2c06d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf29d9b640 1 -- 172.21.15.53:0/3557858543 >> v1:172.21.15.53:6789/0 conn(0x7fbf2c06e620 legacy=0x7fbf2c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf29d9b640 1 -- 172.21.15.53:0/3557858543 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c0b3820 con 0x7fbf2c15f8c0 2024-05-10T10:37:10.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.285+0000 7fbf2959a640 1 -- 172.21.15.53:0/3557858543 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243501977 0 0) 0x7fbf2c0b0a90 con 0x7fbf2c06e620 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.286+0000 7fbf29d9b640 1 --2- 172.21.15.53:0/3557858543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f8c0 0x7fbf2c06e0e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbf200097a0 tx=0x7fbf2002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184eba4c50175a33 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.286+0000 7fbf2959a640 1 -- 172.21.15.53:0/3557858543 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20037070 con 0x7fbf2c15f8c0 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.286+0000 7fbf2959a640 1 -- 172.21.15.53:0/3557858543 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2002ae50 con 0x7fbf2c15f8c0 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.286+0000 7fbf2959a640 1 -- 172.21.15.53:0/3557858543 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20032660 con 0x7fbf2c15f8c0 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.287+0000 7fbf3086d640 1 -- 172.21.15.53:0/3557858543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f8c0 msgr2=0x7fbf2c06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.287+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3557858543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f8c0 0x7fbf2c06e0e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbf200097a0 tx=0x7fbf2002a140 comp rx=0 tx=0).stop 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.287+0000 7fbf3086d640 1 -- 172.21.15.53:0/3557858543 shutdown_connections 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.287+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3557858543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f8c0 0x7fbf2c06e0e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.287+0000 7fbf3086d640 1 -- 172.21.15.53:0/3557858543 >> 172.21.15.53:0/3557858543 conn(0x7fbf2c0d2df0 msgr2=0x7fbf2c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:10.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.287+0000 7fbf3086d640 1 -- 172.21.15.53:0/3557858543 shutdown_connections 2024-05-10T10:37:10.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.287+0000 7fbf3086d640 1 -- 172.21.15.53:0/3557858543 wait complete. 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.288+0000 7fbf3086d640 1 Processor -- start 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.288+0000 7fbf3086d640 1 -- start start 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf3086d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf2c06e620 0x7fbf2c0d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf3086d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f4e0 0x7fbf2c14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf3086d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf2c15f8c0 0x7fbf2c1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf3086d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c0b3de0 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf3086d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c0b4150 con 0x7fbf2c06e620 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf3086d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c0b45e0 con 0x7fbf2c15f8c0 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf29d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f4e0 0x7fbf2c14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf29d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f4e0 0x7fbf2c14b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54696/0 (socket says 172.21.15.53:54696) 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf29d9b640 1 -- 172.21.15.53:0/3046483075 learned_addr learned my addr 172.21.15.53:0/3046483075 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf29d9b640 1 -- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf2c15f8c0 msgr2=0x7fbf2c1f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf29d9b640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf2c15f8c0 0x7fbf2c1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf29d9b640 1 -- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf2c06e620 msgr2=0x7fbf2c0d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.289+0000 7fbf2a59c640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf2c06e620 0x7fbf2c0d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf29d9b640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf2c06e620 0x7fbf2c0d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf29d9b640 1 -- 172.21.15.53:0/3046483075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c157f50 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf2a59c640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf2c06e620 0x7fbf2c0d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf29d9b640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f4e0 0x7fbf2c14b1a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbf200098c0 tx=0x7fbf200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf1affd640 1 -- 172.21.15.53:0/3046483075 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20042070 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf1affd640 1 -- 172.21.15.53:0/3046483075 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2003e670 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c158230 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf1affd640 1 -- 172.21.15.53:0/3046483075 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20037040 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.290+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c0c8600 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.291+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c06a220 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.292+0000 7fbf1affd640 1 -- 172.21.15.53:0/3046483075 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf20045070 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.292+0000 7fbf1affd640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf0c0762d0 0x7fbf0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.292+0000 7fbf1affd640 1 -- 172.21.15.53:0/3046483075 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7fbf200b7d50 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.292+0000 7fbf2a59c640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf0c0762d0 0x7fbf0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.293+0000 7fbf2a59c640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf0c0762d0 0x7fbf0c078790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c003eb0 tx=0x7fbf1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.296+0000 7fbf1affd640 1 -- 172.21.15.53:0/3046483075 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbf200bb030 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.493+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf2c06d4d0 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.494+0000 7fbf1affd640 1 -- 172.21.15.53:0/3046483075 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13294 (secure 0 0 0) 0x7fbf20081790 con 0x7fbf2c15f4e0 2024-05-10T10:37:10.495 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:10.495 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":66,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:37:09.159239+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/1934136965":"2024-05-11T10:37:06.151484+0000","172.21.15.53:6833/1934136965":"2024-05-11T10:37:06.151484+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:10.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf0c0762d0 msgr2=0x7fbf0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf0c0762d0 0x7fbf0c078790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c003eb0 tx=0x7fbf1c009040 comp rx=0 tx=0).stop 2024-05-10T10:37:10.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f4e0 msgr2=0x7fbf2c14b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f4e0 0x7fbf2c14b1a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbf200098c0 tx=0x7fbf200046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 shutdown_connections 2024-05-10T10:37:10.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf0c0762d0 0x7fbf0c078790 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf2c15f8c0 0x7fbf2c1f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf2c15f4e0 0x7fbf2c14b1a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 --2- 172.21.15.53:0/3046483075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf2c06e620 0x7fbf2c0d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 >> 172.21.15.53:0/3046483075 conn(0x7fbf2c0d2df0 msgr2=0x7fbf2c070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:10.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 shutdown_connections 2024-05-10T10:37:10.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.496+0000 7fbf3086d640 1 -- 172.21.15.53:0/3046483075 wait complete. 2024-05-10T10:37:10.509 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:33:28.171977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T10:37:10.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:37:10.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.614+0000 7f49bcca0640 1 Processor -- start 2024-05-10T10:37:10.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.615+0000 7f49bcca0640 1 -- start start 2024-05-10T10:37:10.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.615+0000 7f49bcca0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49b80cffd0 0x7f49b80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.615+0000 7f49bcca0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49b80cd500 0x7f49b80d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.615+0000 7f49bcca0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b806e720 con 0x7f49b80cf3e0 2024-05-10T10:37:10.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.615+0000 7f49bcca0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f49b806de80 con 0x7f49b80cffd0 2024-05-10T10:37:10.622 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.615+0000 7f49bcca0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f49b806e160 con 0x7f49b80cd500 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b5d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49b80cffd0 0x7f49b80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f49b80cf3e0 0x7f49b80cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43732/0 (socket says 172.21.15.53:43732) 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b659c640 1 -- 172.21.15.53:0/2419126406 learned_addr learned my addr 172.21.15.53:0/2419126406 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b6d9d640 1 --2- 172.21.15.53:0/2419126406 >> v2:172.21.15.135:3300/0 conn(0x7f49b80cd500 0x7f49b80d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b5d9b640 1 -- 172.21.15.53:0/2419126406 >> v2:172.21.15.135:3300/0 conn(0x7f49b80cd500 msgr2=0x7f49b80d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b5d9b640 1 --2- 172.21.15.53:0/2419126406 >> v2:172.21.15.135:3300/0 conn(0x7f49b80cd500 0x7f49b80d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b5d9b640 1 -- 172.21.15.53:0/2419126406 >> v1:172.21.15.53:6789/0 conn(0x7f49b80cf3e0 legacy=0x7f49b80cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.616+0000 7f49b5d9b640 1 -- 172.21.15.53:0/2419126406 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b806e440 con 0x7f49b80cffd0 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49b5d9b640 1 --2- 172.21.15.53:0/2419126406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cffd0 0x7f49b80b3de0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f49b8073e80 tx=0x7f49ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d164e94c2a90bb3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49b559a640 1 -- 172.21.15.53:0/2419126406 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac037070 con 0x7f49b80cffd0 2024-05-10T10:37:10.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49b559a640 1 -- 172.21.15.53:0/2419126406 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac032440 con 0x7f49b80cffd0 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49b559a640 1 -- 172.21.15.53:0/2419126406 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac032810 con 0x7f49b80cffd0 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 -- 172.21.15.53:0/2419126406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cffd0 msgr2=0x7f49b80b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 --2- 172.21.15.53:0/2419126406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cffd0 0x7f49b80b3de0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f49b8073e80 tx=0x7f49ac02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 -- 172.21.15.53:0/2419126406 shutdown_connections 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 --2- 172.21.15.53:0/2419126406 >> v2:172.21.15.135:3300/0 conn(0x7f49b80cd500 0x7f49b80d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 --2- 172.21.15.53:0/2419126406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cffd0 0x7f49b80b3de0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 -- 172.21.15.53:0/2419126406 >> 172.21.15.53:0/2419126406 conn(0x7f49b80d2eb0 msgr2=0x7f49b80d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 -- 172.21.15.53:0/2419126406 shutdown_connections 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.617+0000 7f49bcca0640 1 -- 172.21.15.53:0/2419126406 wait complete. 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.619+0000 7f49bcca0640 1 Processor -- start 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.619+0000 7f49bcca0640 1 -- start start 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.619+0000 7f49bcca0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b80cd500 0x7f49b8157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49bcca0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cf3e0 0x7f49b809cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49bcca0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49b80cffd0 0x7f49b80c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49bcca0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49b80d16c0 con 0x7f49b80cf3e0 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49bcca0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49b80b4510 con 0x7f49b80cd500 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49bcca0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f49b80d1840 con 0x7f49b80cffd0 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cf3e0 0x7f49b809cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cf3e0 0x7f49b809cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54706/0 (socket says 172.21.15.53:54706) 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 -- 172.21.15.53:0/609758884 learned_addr learned my addr 172.21.15.53:0/609758884 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b659c640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b80cd500 0x7f49b8157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 -- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49b80cffd0 msgr2=0x7f49b80c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49b80cffd0 0x7f49b80c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 -- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b80cd500 msgr2=0x7f49b8157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b80cd500 0x7f49b8157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 -- 172.21.15.53:0/609758884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b809d650 con 0x7f49b80cf3e0 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49b5d9b640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cf3e0 0x7f49b809cc70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f49ac000c00 tx=0x7f49ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49a6ffd640 1 -- 172.21.15.53:0/609758884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac037040 con 0x7f49b80cf3e0 2024-05-10T10:37:10.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49a6ffd640 1 -- 172.21.15.53:0/609758884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac042070 con 0x7f49b80cf3e0 2024-05-10T10:37:10.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b809d930 con 0x7f49b80cf3e0 2024-05-10T10:37:10.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49a6ffd640 1 -- 172.21.15.53:0/609758884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac030b90 con 0x7f49b80cf3e0 2024-05-10T10:37:10.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.620+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b809def0 con 0x7f49b80cf3e0 2024-05-10T10:37:10.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.621+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b809e130 con 0x7f49b80cf3e0 2024-05-10T10:37:10.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.622+0000 7f49a6ffd640 1 -- 172.21.15.53:0/609758884 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f49ac047460 con 0x7f49b80cf3e0 2024-05-10T10:37:10.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.623+0000 7f49a6ffd640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49980762d0 0x7f4998078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.623+0000 7f49a6ffd640 1 -- 172.21.15.53:0/609758884 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f49ac0b7f70 con 0x7f49b80cf3e0 2024-05-10T10:37:10.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.623+0000 7f49b659c640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49980762d0 0x7f4998078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.627 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.623+0000 7f49b659c640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49980762d0 0x7f4998078790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49b8050940 tx=0x7f49a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.627+0000 7f49a6ffd640 1 -- 172.21.15.53:0/609758884 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f49b809e130 con 0x7f49b80cf3e0 2024-05-10T10:37:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.815+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49b806a220 con 0x7f49b80cf3e0 2024-05-10T10:37:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.816+0000 7f49a6ffd640 1 -- 172.21.15.53:0/609758884 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f49ac032e20 con 0x7f49b80cf3e0 2024-05-10T10:37:10.817 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-05-10T10:37:10.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49980762d0 msgr2=0x7f4998078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49980762d0 0x7f4998078790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49b8050940 tx=0x7f49a8009040 comp rx=0 tx=0).stop 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cf3e0 msgr2=0x7f49b809cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cf3e0 0x7f49b809cc70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f49ac000c00 tx=0x7f49ac0046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 shutdown_connections 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f49980762d0 0x7f4998078790 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f49b80cffd0 0x7f49b80c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49b80cf3e0 0x7f49b809cc70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 --2- 172.21.15.53:0/609758884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b80cd500 0x7f49b8157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 >> 172.21.15.53:0/609758884 conn(0x7f49b80d2eb0 msgr2=0x7f49b806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 shutdown_connections 2024-05-10T10:37:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.818+0000 7f49bcca0640 1 -- 172.21.15.53:0/609758884 wait complete. 2024-05-10T10:37:10.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-10T10:37:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.933+0000 7f9ba301d640 1 Processor -- start 2024-05-10T10:37:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.933+0000 7f9ba301d640 1 -- start start 2024-05-10T10:37:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9ba301d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c1598c0 0x7f9b9c06e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9ba301d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9b9c0c7390 0x7f9b9c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9ba301d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b9c06f1c0 con 0x7f9b9c06ec40 2024-05-10T10:37:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9ba301d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9b9c06f4a0 con 0x7f9b9c0c7390 2024-05-10T10:37:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9ba301d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c06f780 con 0x7f9b9c1598c0 2024-05-10T10:37:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9ba0db9640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c1598c0 0x7f9b9c06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9ba15ba640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9b9c0c7390 0x7f9b9c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9b9bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9b9c06ec40 0x7f9b9c0c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:44648/0 (socket says 172.21.15.53:44648) 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.934+0000 7f9b9bfff640 1 -- 172.21.15.53:0/3012582286 learned_addr learned my addr 172.21.15.53:0/3012582286 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.935+0000 7f9ba15ba640 1 -- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c1598c0 msgr2=0x7f9b9c06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.935+0000 7f9ba15ba640 1 --2- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c1598c0 0x7f9b9c06e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.935+0000 7f9ba15ba640 1 -- 172.21.15.53:0/3012582286 >> v1:172.21.15.135:6789/0 conn(0x7f9b9c06ec40 legacy=0x7f9b9c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.935+0000 7f9ba15ba640 1 -- 172.21.15.53:0/3012582286 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c06fa60 con 0x7f9b9c0c7390 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.935+0000 7f9b9b7fe640 1 -- 172.21.15.53:0/3012582286 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3039113968 0 0) 0x7f9b9c06f1c0 con 0x7f9b9c06ec40 2024-05-10T10:37:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.935+0000 7f9ba0db9640 1 --2- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c1598c0 0x7f9b9c06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba15ba640 1 --2- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c0cee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009d90 tx=0x7f9b9402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6347521e31a4c4c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9b9b7fe640 1 -- 172.21.15.53:0/3012582286 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94007740 con 0x7f9b9c0c7390 2024-05-10T10:37:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9b9b7fe640 1 -- 172.21.15.53:0/3012582286 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b940078e0 con 0x7f9b9c0c7390 2024-05-10T10:37:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9b9b7fe640 1 -- 172.21.15.53:0/3012582286 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94007cb0 con 0x7f9b9c0c7390 2024-05-10T10:37:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba301d640 1 -- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c0c7390 msgr2=0x7f9b9c0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c0cee40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009d90 tx=0x7f9b9402a660 comp rx=0 tx=0).stop 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba301d640 1 -- 172.21.15.53:0/3012582286 shutdown_connections 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c0cee40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3012582286 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c1598c0 0x7f9b9c06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba301d640 1 -- 172.21.15.53:0/3012582286 >> 172.21.15.53:0/3012582286 conn(0x7f9b9c0b2900 msgr2=0x7f9b9c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.936+0000 7f9ba301d640 1 -- 172.21.15.53:0/3012582286 shutdown_connections 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.937+0000 7f9ba301d640 1 -- 172.21.15.53:0/3012582286 wait complete. 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.938+0000 7f9ba301d640 1 Processor -- start 2024-05-10T10:37:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.938+0000 7f9ba301d640 1 -- start start 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.938+0000 7f9ba301d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c06ec40 0x7f9b9c149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba301d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba301d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c1598c0 0x7f9b9c1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba301d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c159ca0 con 0x7f9b9c0c7390 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba301d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c15c6e0 con 0x7f9b9c1598c0 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba301d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c159e20 con 0x7f9b9c06ec40 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54712/0 (socket says 172.21.15.53:54712) 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 -- 172.21.15.53:0/3259879977 learned_addr learned my addr 172.21.15.53:0/3259879977 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba0db9640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c06ec40 0x7f9b9c149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba15ba640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c1598c0 0x7f9b9c1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 -- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c06ec40 msgr2=0x7f9b9c149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c06ec40 0x7f9b9c149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 -- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c1598c0 msgr2=0x7f9b9c1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c1598c0 0x7f9b9c1583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 -- 172.21.15.53:0/3259879977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c0af630 con 0x7f9b9c0c7390 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b9bfff640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c071df0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c4d0 tx=0x7f9b9000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b997fa640 1 -- 172.21.15.53:0/3259879977 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90018070 con 0x7f9b9c0c7390 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b997fa640 1 -- 172.21.15.53:0/3259879977 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b90007500 con 0x7f9b9c0c7390 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c0af910 con 0x7f9b9c0c7390 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.939+0000 7f9b997fa640 1 -- 172.21.15.53:0/3259879977 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90004a20 con 0x7f9b9c0c7390 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.940+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c149ab0 con 0x7f9b9c0c7390 2024-05-10T10:37:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.940+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9c06a220 con 0x7f9b9c0c7390 2024-05-10T10:37:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.941+0000 7f9b997fa640 1 -- 172.21.15.53:0/3259879977 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9b9001b070 con 0x7f9b9c0c7390 2024-05-10T10:37:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.942+0000 7f9b997fa640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b880762d0 0x7f9b88078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.942+0000 7f9b997fa640 1 -- 172.21.15.53:0/3259879977 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f9b90096770 con 0x7f9b9c0c7390 2024-05-10T10:37:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.942+0000 7f9ba0db9640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b880762d0 0x7f9b88078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.942+0000 7f9ba0db9640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b880762d0 0x7f9b88078790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c157040 tx=0x7f9b8c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:10.947 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:10.946+0000 7f9b997fa640 1 -- 172.21.15.53:0/3259879977 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9b9005fbf0 con 0x7f9b9c0c7390 2024-05-10T10:37:11.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:11.214+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f9b9c06e4e0 con 0x7f9b9c0c7390 2024-05-10T10:37:12.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.170+0000 7f9b997fa640 1 -- 172.21.15.53:0/3259879977 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v187) ==== 169+0+0 (secure 0 0 0) 0x7f9b90063ef0 con 0x7f9b9c0c7390 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.174+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b880762d0 msgr2=0x7f9b88078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.174+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b880762d0 0x7f9b88078790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c157040 tx=0x7f9b8c009190 comp rx=0 tx=0).stop 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.174+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b9c0c7390 msgr2=0x7f9b9c071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.174+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c071df0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c4d0 tx=0x7f9b9000c990 comp rx=0 tx=0).stop 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 shutdown_connections 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b880762d0 0x7f9b88078790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b9c1598c0 0x7f9b9c1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b9c0c7390 0x7f9b9c071df0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 --2- 172.21.15.53:0/3259879977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b9c06ec40 0x7f9b9c149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 >> 172.21.15.53:0/3259879977 conn(0x7f9b9c0b2900 msgr2=0x7f9b9c0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:12.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 shutdown_connections 2024-05-10T10:37:12.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.175+0000 7f9ba301d640 1 -- 172.21.15.53:0/3259879977 wait complete. 2024-05-10T10:37:12.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T10:37:12.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.302+0000 7f7064837640 1 Processor -- start 2024-05-10T10:37:12.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.302+0000 7f7064837640 1 -- start start 2024-05-10T10:37:12.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f7064837640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70600cffd0 0x7f70600b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f7064837640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f706006e720 con 0x7f70600cd500 2024-05-10T10:37:12.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f7064837640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f706006de80 con 0x7f70600cf3e0 2024-05-10T10:37:12.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f7064837640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f706006e160 con 0x7f70600cffd0 2024-05-10T10:37:12.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f705dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70600cffd0 0x7f70600b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f705dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70600cffd0 0x7f70600b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54718/0 (socket says 172.21.15.53:54718) 2024-05-10T10:37:12.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f705dd9b640 1 -- 172.21.15.53:0/1272740706 learned_addr learned my addr 172.21.15.53:0/1272740706 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:12.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.303+0000 7f705ed9d640 1 --1- 172.21.15.53:0/1272740706 >> v1:172.21.15.53:6789/0 conn(0x7f70600cd500 0x7f70600d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43740/0 (socket says 172.21.15.53:43740) 2024-05-10T10:37:12.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.304+0000 7f705dd9b640 1 -- 172.21.15.53:0/1272740706 >> v1:172.21.15.135:6789/0 conn(0x7f70600cf3e0 legacy=0x7f70600cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.304+0000 7f705dd9b640 1 -- 172.21.15.53:0/1272740706 >> v1:172.21.15.53:6789/0 conn(0x7f70600cd500 legacy=0x7f70600d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.304+0000 7f705dd9b640 1 -- 172.21.15.53:0/1272740706 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706006e440 con 0x7f70600cffd0 2024-05-10T10:37:12.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.304+0000 7f705d59a640 1 -- 172.21.15.53:0/1272740706 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933445056 0 0) 0x7f706006e720 con 0x7f70600cd500 2024-05-10T10:37:12.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.304+0000 7f705dd9b640 1 --2- 172.21.15.53:0/1272740706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f70600b3de0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7054009870 tx=0x7f705402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fb99e0847b14de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.304+0000 7f705d59a640 1 -- 172.21.15.53:0/1272740706 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054037070 con 0x7f70600cffd0 2024-05-10T10:37:12.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f705d59a640 1 -- 172.21.15.53:0/1272740706 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f705402ae50 con 0x7f70600cffd0 2024-05-10T10:37:12.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f705d59a640 1 -- 172.21.15.53:0/1272740706 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054032860 con 0x7f70600cffd0 2024-05-10T10:37:12.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f7064837640 1 -- 172.21.15.53:0/1272740706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 msgr2=0x7f70600b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f7064837640 1 --2- 172.21.15.53:0/1272740706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f70600b3de0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7054009870 tx=0x7f705402a300 comp rx=0 tx=0).stop 2024-05-10T10:37:12.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f7064837640 1 -- 172.21.15.53:0/1272740706 shutdown_connections 2024-05-10T10:37:12.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f7064837640 1 --2- 172.21.15.53:0/1272740706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f70600b3de0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f7064837640 1 -- 172.21.15.53:0/1272740706 >> 172.21.15.53:0/1272740706 conn(0x7f70600d2eb0 msgr2=0x7f70600d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:12.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f7064837640 1 -- 172.21.15.53:0/1272740706 shutdown_connections 2024-05-10T10:37:12.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.305+0000 7f7064837640 1 -- 172.21.15.53:0/1272740706 wait complete. 2024-05-10T10:37:12.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 Processor -- start 2024-05-10T10:37:12.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 -- start start 2024-05-10T10:37:12.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70600cd500 0x7f706015c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70600cf3e0 0x7f7060160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f7060161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f70600d1890 con 0x7f70600cffd0 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70600b4510 con 0x7f70600cf3e0 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f7064837640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f70600d1a10 con 0x7f70600cd500 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f705ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f7060161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f705ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f7060161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54730/0 (socket says 172.21.15.53:54730) 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.307+0000 7f705ed9d640 1 -- 172.21.15.53:0/966018574 learned_addr learned my addr 172.21.15.53:0/966018574 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705dd9b640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70600cf3e0 0x7f7060160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705e59c640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70600cd500 0x7f706015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705ed9d640 1 -- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70600cd500 msgr2=0x7f706015c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705ed9d640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70600cd500 0x7f706015c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705ed9d640 1 -- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70600cf3e0 msgr2=0x7f7060160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705ed9d640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70600cf3e0 0x7f7060160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705ed9d640 1 -- 172.21.15.53:0/966018574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706015fa50 con 0x7f70600cffd0 2024-05-10T10:37:12.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f705ed9d640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f7060161030 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f705800c780 tx=0x7f705800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f704effd640 1 -- 172.21.15.53:0/966018574 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058018070 con 0x7f70600cffd0 2024-05-10T10:37:12.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f704effd640 1 -- 172.21.15.53:0/966018574 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f705800a640 con 0x7f70600cffd0 2024-05-10T10:37:12.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706015fd30 con 0x7f70600cffd0 2024-05-10T10:37:12.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f704effd640 1 -- 172.21.15.53:0/966018574 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705800aa70 con 0x7f70600cffd0 2024-05-10T10:37:12.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.308+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70601f6010 con 0x7f70600cffd0 2024-05-10T10:37:12.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.309+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706006a220 con 0x7f70600cffd0 2024-05-10T10:37:12.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.310+0000 7f704effd640 1 -- 172.21.15.53:0/966018574 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f705801b070 con 0x7f70600cffd0 2024-05-10T10:37:12.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.310+0000 7f704effd640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70400762d0 0x7f7040078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.311+0000 7f704effd640 1 -- 172.21.15.53:0/966018574 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f70580969b0 con 0x7f70600cffd0 2024-05-10T10:37:12.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.311+0000 7f705e59c640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70400762d0 0x7f7040078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.311+0000 7f705e59c640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70400762d0 0x7f7040078790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7050000f60 tx=0x7f7050009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.315+0000 7f704effd640 1 -- 172.21.15.53:0/966018574 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f705805fe20 con 0x7f70600cffd0 2024-05-10T10:37:12.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.506+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f70601f61e0 con 0x7f70600cffd0 2024-05-10T10:37:12.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.509+0000 7f704effd640 1 -- 172.21.15.53:0/966018574 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f7058064120 con 0x7f70600cffd0 2024-05-10T10:37:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70400762d0 msgr2=0x7f7040078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70400762d0 0x7f7040078790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7050000f60 tx=0x7f7050009040 comp rx=0 tx=0).stop 2024-05-10T10:37:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 msgr2=0x7f7060161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f7060161030 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f705800c780 tx=0x7f705800cc40 comp rx=0 tx=0).stop 2024-05-10T10:37:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 shutdown_connections 2024-05-10T10:37:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f70400762d0 0x7f7040078790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70600cffd0 0x7f7060161030 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70600cf3e0 0x7f7060160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 --2- 172.21.15.53:0/966018574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70600cd500 0x7f706015c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 >> 172.21.15.53:0/966018574 conn(0x7f70600d2eb0 msgr2=0x7f7060070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 shutdown_connections 2024-05-10T10:37:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.511+0000 7f7064837640 1 -- 172.21.15.53:0/966018574 wait complete. 2024-05-10T10:37:12.524 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:37:12.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:37:12.570 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:37:12.616 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:37:12.616 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:12.616 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:12.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:37:12.633 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:37:12.633 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:12.652 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:37:12.652 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:12.652 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:12.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:37:12.690 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:37:12.690 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:12.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:37:12.710 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T10:37:12.710 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:12.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.833+0000 7f3017c6d640 1 Processor -- start 2024-05-10T10:37:12.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.834+0000 7f3017c6d640 1 -- start start 2024-05-10T10:37:12.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.834+0000 7f3017c6d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30100d5cb0 0x7f30100d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.834+0000 7f3017c6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010162b30 0x7f30100d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.834+0000 7f3017c6d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30100b45f0 con 0x7f3010162750 2024-05-10T10:37:12.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.834+0000 7f3017c6d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f30100b4f20 con 0x7f30100d5cb0 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.834+0000 7f3017c6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f30100d0fe0 con 0x7f3010162b30 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3015208640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30100d5cb0 0x7f30100d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3015a09640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3010162750 0x7f30100d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49856/0 (socket says 172.21.15.53:49856) 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3015a09640 1 -- 172.21.15.53:0/1715439708 learned_addr learned my addr 172.21.15.53:0/1715439708 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f301620a640 1 --2- 172.21.15.53:0/1715439708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010162b30 0x7f30100d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3015208640 1 -- 172.21.15.53:0/1715439708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010162b30 msgr2=0x7f30100d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3015208640 1 --2- 172.21.15.53:0/1715439708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010162b30 0x7f30100d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3015208640 1 -- 172.21.15.53:0/1715439708 >> v1:172.21.15.53:6789/0 conn(0x7f3010162750 legacy=0x7f30100d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3015208640 1 -- 172.21.15.53:0/1715439708 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30100b48d0 con 0x7f30100d5cb0 2024-05-10T10:37:12.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.835+0000 7f3014a07640 1 -- 172.21.15.53:0/1715439708 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252840313 0 0) 0x7f30100b45f0 con 0x7f3010162750 2024-05-10T10:37:12.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3015208640 1 --2- 172.21.15.53:0/1715439708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30100d5cb0 0x7f30100d3630 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f30040097a0 tx=0x7f300402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc4a588478d2181 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3014a07640 1 -- 172.21.15.53:0/1715439708 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004037070 con 0x7f30100d5cb0 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3014a07640 1 -- 172.21.15.53:0/1715439708 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300402ae50 con 0x7f30100d5cb0 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3014a07640 1 -- 172.21.15.53:0/1715439708 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004032660 con 0x7f30100d5cb0 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 -- 172.21.15.53:0/1715439708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30100d5cb0 msgr2=0x7f30100d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 --2- 172.21.15.53:0/1715439708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30100d5cb0 0x7f30100d3630 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f30040097a0 tx=0x7f300402a100 comp rx=0 tx=0).stop 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 -- 172.21.15.53:0/1715439708 shutdown_connections 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 --2- 172.21.15.53:0/1715439708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010162b30 0x7f30100d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 --2- 172.21.15.53:0/1715439708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30100d5cb0 0x7f30100d3630 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 -- 172.21.15.53:0/1715439708 >> 172.21.15.53:0/1715439708 conn(0x7f301009e3b0 msgr2=0x7f30100d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 -- 172.21.15.53:0/1715439708 shutdown_connections 2024-05-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.836+0000 7f3017c6d640 1 -- 172.21.15.53:0/1715439708 wait complete. 2024-05-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 Processor -- start 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 -- start start 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3010162750 0x7f30100ba610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3010162b30 0x7f3010156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010157860 0x7f30100b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f30100b53b0 con 0x7f3010162750 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30100d0d80 con 0x7f3010162b30 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3017c6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f30100b5530 con 0x7f3010157860 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3015a09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3010162750 0x7f30100ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.838+0000 7f3015a09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3010162750 0x7f30100ba610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37316/0 (socket says 172.21.15.53:37316) 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015a09640 1 -- 172.21.15.53:0/3631436098 learned_addr learned my addr 172.21.15.53:0/3631436098 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f301620a640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010157860 0x7f30100b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015208640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3010162b30 0x7f3010156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015a09640 1 -- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010157860 msgr2=0x7f30100b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015a09640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010157860 0x7f30100b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015a09640 1 -- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3010162b30 msgr2=0x7f3010156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015a09640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3010162b30 0x7f3010156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015a09640 1 -- 172.21.15.53:0/3631436098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30100b9fd0 con 0x7f3010162750 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f301620a640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010157860 0x7f30100b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015208640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3010162b30 0x7f3010156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3015a09640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3010162750 0x7f30100ba610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f300000c890 tx=0x7f300000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f300e7fc640 1 -- 172.21.15.53:0/3631436098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000018070 con 0x7f3010162750 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f300e7fc640 1 -- 172.21.15.53:0/3631436098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300000a640 con 0x7f3010162750 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30100b7e10 con 0x7f3010162750 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f300e7fc640 1 -- 172.21.15.53:0/3631436098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300000aa30 con 0x7f3010162750 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.839+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30100b9c80 con 0x7f3010162750 2024-05-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.840+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301006a220 con 0x7f3010162750 2024-05-10T10:37:12.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.841+0000 7f300e7fc640 1 -- 172.21.15.53:0/3631436098 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f300001b070 con 0x7f3010162750 2024-05-10T10:37:12.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.841+0000 7f300e7fc640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2ffc0762d0 0x7f2ffc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:12.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.842+0000 7f300e7fc640 1 -- 172.21.15.53:0/3631436098 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f3000096b90 con 0x7f3010162750 2024-05-10T10:37:12.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.842+0000 7f3015208640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2ffc0762d0 0x7f2ffc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:12.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.842+0000 7f3015208640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2ffc0762d0 0x7f2ffc078790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3004034000 tx=0x7f300402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:12.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:12.846+0000 7f300e7fc640 1 -- 172.21.15.53:0/3631436098 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f30000605c0 con 0x7f3010162750 2024-05-10T10:37:13.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.075+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30100d18c0 con 0x7f3010162750 2024-05-10T10:37:13.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.075+0000 7f300e7fc640 1 -- 172.21.15.53:0/3631436098 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3875 (secure 0 0 0) 0x7f300005fec0 con 0x7f3010162750 2024-05-10T10:37:13.077 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:13.077 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/4120603893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":4120603893},{"type":"v1","addr":"172.21.15.135:6837","nonce":4120603893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2770137191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2770137191},{"type":"v1","addr":"172.21.15.135:6836","nonce":2770137191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2093169790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2093169790},{"type":"v1","addr":"172.21.15.53:6834","nonce":2093169790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1895368166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1895368166},{"type":"v1","addr":"172.21.15.53:6835","nonce":1895368166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-05-10T10:37:13.077 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 38 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2ffc0762d0 msgr2=0x7f2ffc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2ffc0762d0 0x7f2ffc078790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3004034000 tx=0x7f300402e000 comp rx=0 tx=0).stop 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3010162750 msgr2=0x7f30100ba610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3010162750 0x7f30100ba610 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f300000c890 tx=0x7f300000cd50 comp rx=0 tx=0).stop 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 shutdown_connections 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2ffc0762d0 0x7f2ffc078790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3010157860 0x7f30100b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3010162b30 0x7f3010156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 --2- 172.21.15.53:0/3631436098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3010162750 0x7f30100ba610 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 >> 172.21.15.53:0/3631436098 conn(0x7f301009e3b0 msgr2=0x7f30100d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 shutdown_connections 2024-05-10T10:37:13.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.077+0000 7f3017c6d640 1 -- 172.21.15.53:0/3631436098 wait complete. 2024-05-10T10:37:13.090 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:37:13.090 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T10:37:13.090 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:37:13.090 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:37:13.091 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:37:13.091 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T10:37:13.091 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:37:13.091 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:37:13.091 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:37:13.091 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T10:37:13.091 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:37:13.091 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:37:13.092 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:37:13.092 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T10:37:13.092 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:37:13.092 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:37:13.100 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:37:13.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:37:13.101 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:37:13.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:37:13.103 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:37:13.103 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:37:13.104 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:37:13.104 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:37:13.143 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:37:13.143 DEBUG:teuthology.parallel:result is None 2024-05-10T10:37:13.144 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:37:13.144 DEBUG:teuthology.parallel:result is None 2024-05-10T10:37:13.146 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:37:13.146 DEBUG:teuthology.parallel:result is None 2024-05-10T10:37:13.147 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:37:13.147 DEBUG:teuthology.parallel:result is None 2024-05-10T10:37:13.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T10:37:13.222 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:37:13.222 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:37:13.229 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:37:13.229 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:37:13.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.256+0000 7fd5b77e7640 1 Processor -- start 2024-05-10T10:37:13.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.257+0000 7fd5b77e7640 1 -- start start 2024-05-10T10:37:13.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.257+0000 7fd5b77e7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5b015f3b0 0x7fd5b00d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.257+0000 7fd5b77e7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b00d30b0 con 0x7fd5b015f790 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b4d82640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd5b015f790 0x7fd5b00c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49892/0 (socket says 172.21.15.53:49892) 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b4d82640 1 -- 172.21.15.53:0/2540801561 learned_addr learned my addr 172.21.15.53:0/2540801561 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b77e7640 1 -- 172.21.15.53:0/2540801561 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b00d3390 con 0x7fd5b00d5d30 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b77e7640 1 -- 172.21.15.53:0/2540801561 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b00c8c80 con 0x7fd5b015f3b0 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b5583640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5b015f3b0 0x7fd5b00d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947115012 0 0) 0x7fd5b00d30b0 con 0x7fd5b015f790 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd59c003520 con 0x7fd5b015f790 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322189093 0 0) 0x7fd5b00d3390 con 0x7fd5b00d5d30 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5b00d30b0 con 0x7fd5b00d5d30 2024-05-10T10:37:13.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 <== mon.1 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3061286582 0 0) 0x7fd5b00d30b0 con 0x7fd5b00d5d30 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5b015f3b0 msgr2=0x7fd5b00d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5b015f3b0 0x7fd5b00d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 >> v1:172.21.15.53:6789/0 conn(0x7fd5b015f790 legacy=0x7fd5b00c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b00c8f60 con 0x7fd5b00d5d30 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 <== mon.1 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fd5a80019f0 con 0x7fd5b00d5d30 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 >> v1:172.21.15.135:6789/0 conn(0x7fd5b00d5d30 legacy=0x7fd5b00d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd59c004bd0 0x7fd59c004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd59c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd59c004160 0x7fd59c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00c8c80 con 0x7fd59c004160 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c00bf30 con 0x7fd59c004bd0 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c00c130 con 0x7fd59c0037a0 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b4d82640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd59c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b5d84640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd59c004160 0x7fd59c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.258+0000 7fd5b5583640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd59c004bd0 0x7fd59c004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.259+0000 7fd5b4d82640 1 -- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd59c004bd0 msgr2=0x7fd59c004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.259+0000 7fd5b4d82640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd59c004bd0 0x7fd59c004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.259+0000 7fd5b4d82640 1 -- 172.21.15.53:0/2540801561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd59c004160 msgr2=0x7fd59c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.259+0000 7fd5b4d82640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd59c004160 0x7fd59c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.259+0000 7fd5b4d82640 1 -- 172.21.15.53:0/2540801561 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b00d30b0 con 0x7fd59c0037a0 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.259+0000 7fd5b5d84640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd59c004160 0x7fd59c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.259+0000 7fd5b4d82640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd59c003c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400c160 tx=0x7fd59c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0daf42dca6c074 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4007c40 con 0x7fd59c0037a0 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a4005af0 con 0x7fd59c0037a0 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5a3fff640 1 -- 172.21.15.53:0/2540801561 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4007070 con 0x7fd59c0037a0 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5b77e7640 1 -- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 msgr2=0x7fd59c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5b77e7640 1 --2- 172.21.15.53:0/2540801561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd59c003c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400c160 tx=0x7fd59c005830 comp rx=0 tx=0).stop 2024-05-10T10:37:13.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5b5d84640 1 -- 172.21.15.53:0/2540801561 reap_dead start 2024-05-10T10:37:13.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5b77e7640 1 -- 172.21.15.53:0/2540801561 shutdown_connections 2024-05-10T10:37:13.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5b77e7640 1 -- 172.21.15.53:0/2540801561 >> 172.21.15.53:0/2540801561 conn(0x7fd5b00d5130 msgr2=0x7fd5b00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:13.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5b77e7640 1 -- 172.21.15.53:0/2540801561 shutdown_connections 2024-05-10T10:37:13.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.260+0000 7fd5b77e7640 1 -- 172.21.15.53:0/2540801561 wait complete. 2024-05-10T10:37:13.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.261+0000 7fd5b77e7640 1 Processor -- start 2024-05-10T10:37:13.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.261+0000 7fd5b77e7640 1 -- start start 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b77e7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd5b01f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b77e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5b01601c0 0x7fd5b0156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b77e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5b01574c0 0x7fd5b0155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b77e7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00d3870 con 0x7fd5b01574c0 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b77e7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00d36f0 con 0x7fd5b01601c0 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b77e7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b00d39f0 con 0x7fd59c0037a0 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b5d84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5b01574c0 0x7fd5b0155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b5d84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5b01574c0 0x7fd5b0155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37368/0 (socket says 172.21.15.53:37368) 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b5d84640 1 -- 172.21.15.53:0/1715771874 learned_addr learned my addr 172.21.15.53:0/1715771874 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b5583640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd5b01f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.262+0000 7fd5b4d82640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5b01601c0 0x7fd5b0156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5b5d84640 1 -- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 msgr2=0x7fd5b01f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5b5d84640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd5b01f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5b5d84640 1 -- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5b01601c0 msgr2=0x7fd5b0156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5b5d84640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5b01601c0 0x7fd5b0156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5b5d84640 1 -- 172.21.15.53:0/1715771874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01f5350 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5b5d84640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5b01574c0 0x7fd5b0155000 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00c8c30 tx=0x7fd5a800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5a1ffb640 1 -- 172.21.15.53:0/1715771874 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800c0e0 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5a1ffb640 1 -- 172.21.15.53:0/1715771874 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a8019070 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.263+0000 7fd5a1ffb640 1 -- 172.21.15.53:0/1715771874 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800fde0 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.264+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01579f0 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.264+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b0160c40 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.265+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b006a220 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.265+0000 7fd5a1ffb640 1 -- 172.21.15.53:0/1715771874 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd5a8003970 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.266+0000 7fd5a1ffb640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd594076300 0x7fd5940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.266+0000 7fd5a1ffb640 1 -- 172.21.15.53:0/1715771874 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7fd5a80973b0 con 0x7fd5b01574c0 2024-05-10T10:37:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.266+0000 7fd5b5583640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd594076300 0x7fd5940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:13.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.266+0000 7fd5b5583640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd594076300 0x7fd5940787c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd5b008e820 tx=0x7fd5ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:13.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.270+0000 7fd5a1ffb640 1 -- 172.21.15.53:0/1715771874 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd5a8060700 con 0x7fd5b01574c0 2024-05-10T10:37:13.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:13.472+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd5b015bf10 con 0x7fd5b01574c0 2024-05-10T10:37:14.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.189+0000 7fd5a1ffb640 1 -- 172.21.15.53:0/1715771874 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) ==== 97+0+0 (secure 0 0 0) 0x7fd5a8064a00 con 0x7fd5b01574c0 2024-05-10T10:37:14.191 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2024-05-10T10:37:14.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.192+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd594076300 msgr2=0x7fd5940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd594076300 0x7fd5940787c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd5b008e820 tx=0x7fd5ac008040 comp rx=0 tx=0).stop 2024-05-10T10:37:14.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5b01574c0 msgr2=0x7fd5b0155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5b01574c0 0x7fd5b0155000 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00c8c30 tx=0x7fd5a800f5a0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 shutdown_connections 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd594076300 0x7fd5940787c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5b01574c0 0x7fd5b0155000 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5b01601c0 0x7fd5b0156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 --2- 172.21.15.53:0/1715771874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd59c0037a0 0x7fd5b01f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 >> 172.21.15.53:0/1715771874 conn(0x7fd5b00d5130 msgr2=0x7fd5b006cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 shutdown_connections 2024-05-10T10:37:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.193+0000 7fd5b77e7640 1 -- 172.21.15.53:0/1715771874 wait complete. 2024-05-10T10:37:14.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T10:37:14.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:37:14.274 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:37:14.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.318+0000 7f8f72724640 1 Processor -- start 2024-05-10T10:37:14.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.318+0000 7f8f72724640 1 -- start start 2024-05-10T10:37:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.319+0000 7f8f72724640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f6c06e460 0x7f8f6c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.319+0000 7f8f72724640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6c0d1630 con 0x7f8f6c0cfb70 2024-05-10T10:37:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.319+0000 7f8f72724640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6c0d1910 con 0x7f8f6c0cff50 2024-05-10T10:37:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.319+0000 7f8f72724640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6c0d1bf0 con 0x7f8f6c06e460 2024-05-10T10:37:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.319+0000 7f8f6b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f6c06e460 0x7f8f6c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.319+0000 7f8f6bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8f6c0cfb70 0x7f8f6c06dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49904/0 (socket says 172.21.15.53:49904) 2024-05-10T10:37:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.319+0000 7f8f6bfff640 1 -- 172.21.15.53:0/4081770188 learned_addr learned my addr 172.21.15.53:0/4081770188 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:14.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.320+0000 7f8f6b7fe640 1 -- 172.21.15.53:0/4081770188 >> v1:172.21.15.135:6789/0 conn(0x7f8f6c0cff50 legacy=0x7f8f6c15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:14.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.320+0000 7f8f6b7fe640 1 -- 172.21.15.53:0/4081770188 >> v1:172.21.15.53:6789/0 conn(0x7f8f6c0cfb70 legacy=0x7f8f6c06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.320+0000 7f8f6b7fe640 1 -- 172.21.15.53:0/4081770188 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c0d1ed0 con 0x7f8f6c06e460 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.320+0000 7f8f6b7fe640 1 --2- 172.21.15.53:0/4081770188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c06e460 0x7f8f6c15eac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f600097a0 tx=0x7f8f6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ead62e0b7b82992 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.320+0000 7f8f6affd640 1 -- 172.21.15.53:0/4081770188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60037070 con 0x7f8f6c06e460 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f6affd640 1 -- 172.21.15.53:0/4081770188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f60032440 con 0x7f8f6c06e460 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f6affd640 1 -- 172.21.15.53:0/4081770188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60032810 con 0x7f8f6c06e460 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f72724640 1 -- 172.21.15.53:0/4081770188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c06e460 msgr2=0x7f8f6c15eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f72724640 1 --2- 172.21.15.53:0/4081770188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c06e460 0x7f8f6c15eac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f600097a0 tx=0x7f8f6002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f72724640 1 -- 172.21.15.53:0/4081770188 shutdown_connections 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f72724640 1 --2- 172.21.15.53:0/4081770188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c06e460 0x7f8f6c15eac0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f72724640 1 -- 172.21.15.53:0/4081770188 >> 172.21.15.53:0/4081770188 conn(0x7f8f6c0d2e50 msgr2=0x7f8f6c0d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f72724640 1 -- 172.21.15.53:0/4081770188 shutdown_connections 2024-05-10T10:37:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.321+0000 7f8f72724640 1 -- 172.21.15.53:0/4081770188 wait complete. 2024-05-10T10:37:14.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.322+0000 7f8f72724640 1 Processor -- start 2024-05-10T10:37:14.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.323+0000 7f8f72724640 1 -- start start 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.323+0000 7f8f72724640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f6c06e460 0x7f8f6c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.323+0000 7f8f72724640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6c0cfb70 0x7f8f6c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.323+0000 7f8f72724640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c0cff50 0x7f8f6c09faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.323+0000 7f8f72724640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c15f890 con 0x7f8f6c0cff50 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.323+0000 7f8f72724640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c15f710 con 0x7f8f6c0cfb70 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.323+0000 7f8f72724640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c0d11c0 con 0x7f8f6c06e460 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c0cff50 0x7f8f6c09faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c0cff50 0x7f8f6c09faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37396/0 (socket says 172.21.15.53:37396) 2024-05-10T10:37:14.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 -- 172.21.15.53:0/3786025231 learned_addr learned my addr 172.21.15.53:0/3786025231 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f6b7fe640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6c0cfb70 0x7f8f6c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f6bfff640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f6c06e460 0x7f8f6c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 -- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f6c06e460 msgr2=0x7f8f6c0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f6c06e460 0x7f8f6c0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 -- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6c0cfb70 msgr2=0x7f8f6c15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6c0cfb70 0x7f8f6c15d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 -- 172.21.15.53:0/3786025231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c0b2660 con 0x7f8f6c0cff50 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f6bfff640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f6c06e460 0x7f8f6c0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f70cc1640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c0cff50 0x7f8f6c09faa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b970 tx=0x7f8f6400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f68ff9640 1 -- 172.21.15.53:0/3786025231 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64019070 con 0x7f8f6c0cff50 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f68ff9640 1 -- 172.21.15.53:0/3786025231 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6400cae0 con 0x7f8f6c0cff50 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c0b2940 con 0x7f8f6c0cff50 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f68ff9640 1 -- 172.21.15.53:0/3786025231 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64014440 con 0x7f8f6c0cff50 2024-05-10T10:37:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.324+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c0b2f00 con 0x7f8f6c0cff50 2024-05-10T10:37:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.325+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c0693f0 con 0x7f8f6c0cff50 2024-05-10T10:37:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.326+0000 7f8f68ff9640 1 -- 172.21.15.53:0/3786025231 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8f6401c070 con 0x7f8f6c0cff50 2024-05-10T10:37:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.326+0000 7f8f68ff9640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f58076300 0x7f8f580787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.326+0000 7f8f68ff9640 1 -- 172.21.15.53:0/3786025231 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7f8f64096fc0 con 0x7f8f6c0cff50 2024-05-10T10:37:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.327+0000 7f8f6bfff640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f58076300 0x7f8f580787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.327+0000 7f8f6bfff640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f58076300 0x7f8f580787c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000f60 tx=0x7f8f5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.331+0000 7f8f68ff9640 1 -- 172.21.15.53:0/3786025231 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8f640603f0 con 0x7f8f6c0cff50 2024-05-10T10:37:14.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.593+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8f6c0ceec0 con 0x7f8f6c0cff50 2024-05-10T10:37:14.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.594+0000 7f8f68ff9640 1 -- 172.21.15.53:0/3786025231 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f8f640646f0 con 0x7f8f6c0cff50 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg==", 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:37:14.596 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ==", 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.597 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA==", 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew==", 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.598 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA==", 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA==", 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw==", 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.600 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA==", 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2024-05-10T10:37:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g==", 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A==", 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA==", 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg==", 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.603 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg==", 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g==", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw==", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.605 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw==", 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1mKY2tIhAAzPSpOQLOFy1GEKdl5bPBEw==", 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw==", 2024-05-10T10:37:14.607 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ==", 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:37:14.608 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:37:14.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f58076300 msgr2=0x7f8f580787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f58076300 0x7f8f580787c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000f60 tx=0x7f8f5c009040 comp rx=0 tx=0).stop 2024-05-10T10:37:14.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c0cff50 msgr2=0x7f8f6c09faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c0cff50 0x7f8f6c09faa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b970 tx=0x7f8f6400be30 comp rx=0 tx=0).stop 2024-05-10T10:37:14.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 shutdown_connections 2024-05-10T10:37:14.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f58076300 0x7f8f580787c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6c0cff50 0x7f8f6c09faa0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6c0cfb70 0x7f8f6c15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 --2- 172.21.15.53:0/3786025231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f6c06e460 0x7f8f6c0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 >> 172.21.15.53:0/3786025231 conn(0x7f8f6c0d2e50 msgr2=0x7f8f6c15fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:14.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 shutdown_connections 2024-05-10T10:37:14.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.596+0000 7f8f72724640 1 -- 172.21.15.53:0/3786025231 wait complete. 2024-05-10T10:37:14.611 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:37:14.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.752+0000 7fc577e59640 1 Processor -- start 2024-05-10T10:37:14.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.753+0000 7fc577e59640 1 -- start start 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.753+0000 7fc577e59640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700d5cb0 0x7fc5700d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.753+0000 7fc577e59640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5700b45f0 con 0x7fc570162b30 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.753+0000 7fc577e59640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5700b48d0 con 0x7fc570162750 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.753+0000 7fc577e59640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc5700b4f20 con 0x7fc5700d5cb0 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc5763f6640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc570162b30 0x7fc5700b53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49916/0 (socket says 172.21.15.53:49916) 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc5763f6640 1 -- 172.21.15.53:0/3598669240 learned_addr learned my addr 172.21.15.53:0/3598669240 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc5753f4640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700d5cb0 0x7fc5700d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791922570 0 0) 0x7fc5700b45f0 con 0x7fc570162b30 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc55c003520 con 0x7fc570162b30 2024-05-10T10:37:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546857194 0 0) 0x7fc5700b48d0 con 0x7fc570162750 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5700b45f0 con 0x7fc570162750 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 487871658 0 0) 0x7fc55c003520 con 0x7fc570162b30 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700d5cb0 msgr2=0x7fc5700d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700d5cb0 0x7fc5700d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 >> v1:172.21.15.135:6789/0 conn(0x7fc570162750 legacy=0x7fc5700d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5700b4bb0 con 0x7fc570162b30 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fc5680019f0 con 0x7fc570162b30 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 >> v1:172.21.15.53:6789/0 conn(0x7fc570162b30 legacy=0x7fc5700b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc55c003b70 0x7fc55c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc55c004800 0x7fc55c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc55c005180 0x7fc55c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc5700b4f20 con 0x7fc55c005180 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c00bf00 con 0x7fc55c003b70 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc55c00c0b0 con 0x7fc55c004800 2024-05-10T10:37:14.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.754+0000 7fc5763f6640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc55c005180 0x7fc55c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc575bf5640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc55c003b70 0x7fc55c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc5753f4640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc55c004800 0x7fc55c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc5763f6640 1 -- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc55c004800 msgr2=0x7fc55c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc5763f6640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc55c004800 0x7fc55c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc5763f6640 1 -- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc55c003b70 msgr2=0x7fc55c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc5763f6640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc55c003b70 0x7fc55c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc5763f6640 1 -- 172.21.15.53:0/3598669240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55c003520 con 0x7fc55c005180 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.755+0000 7fc5763f6640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc55c005180 0x7fc55c00b9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc56800a800 tx=0x7fc55c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10277fdee1da65af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56800a430 con 0x7fc55c005180 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56802f070 con 0x7fc55c005180 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc574bf3640 1 -- 172.21.15.53:0/3598669240 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56802bc30 con 0x7fc55c005180 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc577e59640 1 -- 172.21.15.53:0/3598669240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc55c005180 msgr2=0x7fc55c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc577e59640 1 --2- 172.21.15.53:0/3598669240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc55c005180 0x7fc55c00b9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc56800a800 tx=0x7fc55c005830 comp rx=0 tx=0).stop 2024-05-10T10:37:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc5763f6640 1 -- 172.21.15.53:0/3598669240 reap_dead start 2024-05-10T10:37:14.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc577e59640 1 -- 172.21.15.53:0/3598669240 shutdown_connections 2024-05-10T10:37:14.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc577e59640 1 -- 172.21.15.53:0/3598669240 >> 172.21.15.53:0/3598669240 conn(0x7fc57009e3b0 msgr2=0x7fc5700d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:14.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc577e59640 1 -- 172.21.15.53:0/3598669240 shutdown_connections 2024-05-10T10:37:14.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.756+0000 7fc577e59640 1 -- 172.21.15.53:0/3598669240 wait complete. 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 Processor -- start 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 -- start start 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc570155680 0x7fc57015f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5701f89a0 0x7fc5700cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700cb330 0x7fc5700cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc5700d3a40 con 0x7fc570155680 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5700d38c0 con 0x7fc5701f89a0 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc577e59640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc5700d3bc0 con 0x7fc5700cb330 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc575bf5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc570155680 0x7fc57015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc575bf5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc570155680 0x7fc57015f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37422/0 (socket says 172.21.15.53:37422) 2024-05-10T10:37:14.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.758+0000 7fc575bf5640 1 -- 172.21.15.53:0/637242986 learned_addr learned my addr 172.21.15.53:0/637242986 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc5763f6640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700cb330 0x7fc5700cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc5753f4640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5701f89a0 0x7fc5700cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc575bf5640 1 -- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700cb330 msgr2=0x7fc5700cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc575bf5640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700cb330 0x7fc5700cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc575bf5640 1 -- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5701f89a0 msgr2=0x7fc5700cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc575bf5640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5701f89a0 0x7fc5700cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc575bf5640 1 -- 172.21.15.53:0/637242986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5701f8e20 con 0x7fc570155680 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc575bf5640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc570155680 0x7fc57015f990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00f7a0 tx=0x7fc56c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc5667fc640 1 -- 172.21.15.53:0/637242986 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00fe50 con 0x7fc570155680 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc5667fc640 1 -- 172.21.15.53:0/637242986 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c018070 con 0x7fc570155680 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5700ce250 con 0x7fc570155680 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc5667fc640 1 -- 172.21.15.53:0/637242986 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00d730 con 0x7fc570155680 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.759+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5700ce810 con 0x7fc570155680 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.760+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5700cb740 con 0x7fc570155680 2024-05-10T10:37:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.761+0000 7fc5667fc640 1 -- 172.21.15.53:0/637242986 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc56c00d8d0 con 0x7fc570155680 2024-05-10T10:37:14.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.761+0000 7fc5667fc640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc554076300 0x7fc5540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:14.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.761+0000 7fc5667fc640 1 -- 172.21.15.53:0/637242986 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7fc56c0973b0 con 0x7fc570155680 2024-05-10T10:37:14.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.761+0000 7fc5753f4640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc554076300 0x7fc5540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:14.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.762+0000 7fc5753f4640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc554076300 0x7fc5540787c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc5700ceca0 tx=0x7fc560008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:14.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.766+0000 7fc5667fc640 1 -- 172.21.15.53:0/637242986 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc5700cb740 con 0x7fc570155680 2024-05-10T10:37:14.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.960+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc57006a220 con 0x7fc570155680 2024-05-10T10:37:14.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.960+0000 7fc5667fc640 1 -- 172.21.15.53:0/637242986 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fc56c095e20 con 0x7fc570155680 2024-05-10T10:37:14.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:14.962 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:37:14.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:37:14.962 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:37:14.962 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:37:14.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:14.962 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc554076300 msgr2=0x7fc5540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc554076300 0x7fc5540787c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc5700ceca0 tx=0x7fc560008040 comp rx=0 tx=0).stop 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc570155680 msgr2=0x7fc57015f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc570155680 0x7fc57015f990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00f7a0 tx=0x7fc56c00fc60 comp rx=0 tx=0).stop 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 shutdown_connections 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc554076300 0x7fc5540787c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5700cb330 0x7fc5700cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5701f89a0 0x7fc5700cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 --2- 172.21.15.53:0/637242986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc570155680 0x7fc57015f990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 >> 172.21.15.53:0/637242986 conn(0x7fc57009e3b0 msgr2=0x7fc5700b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 shutdown_connections 2024-05-10T10:37:14.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:14.962+0000 7fc577e59640 1 -- 172.21.15.53:0/637242986 wait complete. 2024-05-10T10:37:14.974 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:37:14.974 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:37:15.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.080+0000 7f4b7dc07640 1 Processor -- start 2024-05-10T10:37:15.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.080+0000 7f4b7dc07640 1 -- start start 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.081+0000 7f4b7dc07640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4b78162e00 0x7f4b78163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.081+0000 7f4b7dc07640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b780716f0 con 0x7f4b7815f370 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.081+0000 7f4b7dc07640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b780d2510 con 0x7f4b781639a0 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.081+0000 7f4b7dc07640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4b78070220 con 0x7f4b78162e00 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.081+0000 7f4b777fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4b7815f370 0x7f4b780d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49930/0 (socket says 172.21.15.53:49930) 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.081+0000 7f4b777fe640 1 -- 172.21.15.53:0/166424556 learned_addr learned my addr 172.21.15.53:0/166424556 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.081+0000 7f4b77fff640 1 --2- 172.21.15.53:0/166424556 >> v2:172.21.15.135:3300/0 conn(0x7f4b78162e00 0x7f4b78163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 449180882 0 0) 0x7f4b780716f0 con 0x7f4b7815f370 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b60003520 con 0x7f4b7815f370 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3152286990 0 0) 0x7f4b60003520 con 0x7f4b7815f370 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 >> v2:172.21.15.135:3300/0 conn(0x7f4b78162e00 msgr2=0x7f4b78163290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 --2- 172.21.15.53:0/166424556 >> v2:172.21.15.135:3300/0 conn(0x7f4b78162e00 0x7f4b78163290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 >> v1:172.21.15.135:6789/0 conn(0x7f4b781639a0 legacy=0x7f4b78070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b78070500 con 0x7f4b7815f370 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4b68001670 con 0x7f4b7815f370 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 >> v1:172.21.15.53:6789/0 conn(0x7f4b7815f370 legacy=0x7f4b780d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600039b0 0x7f4b60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b60004800 0x7f4b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b600051c0 0x7f4b6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78070220 con 0x7f4b600051c0 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6000c070 con 0x7f4b60004800 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6000c270 con 0x7f4b600039b0 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b77fff640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b600051c0 0x7f4b6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b777fe640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600039b0 0x7f4b60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b76ffd640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b60004800 0x7f4b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b77fff640 1 -- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600039b0 msgr2=0x7f4b60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:15.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b77fff640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600039b0 0x7f4b60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b77fff640 1 -- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b60004800 msgr2=0x7f4b60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:15.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b77fff640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b60004800 0x7f4b60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.082+0000 7f4b77fff640 1 -- 172.21.15.53:0/166424556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b60003520 con 0x7f4b600051c0 2024-05-10T10:37:15.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.083+0000 7f4b77fff640 1 -- 172.21.15.53:0/166424556 reap_dead start 2024-05-10T10:37:15.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.083+0000 7f4b77fff640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b600051c0 0x7f4b6000b9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009fc0 tx=0x7f4b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61e6e5477448e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:15.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.083+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70035020 con 0x7f4b600051c0 2024-05-10T10:37:15.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.083+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b70003bd0 con 0x7f4b600051c0 2024-05-10T10:37:15.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.083+0000 7f4b767fc640 1 -- 172.21.15.53:0/166424556 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70009c10 con 0x7f4b600051c0 2024-05-10T10:37:15.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.084+0000 7f4b7dc07640 1 -- 172.21.15.53:0/166424556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b600051c0 msgr2=0x7f4b6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.084+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b600051c0 0x7f4b6000b9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009fc0 tx=0x7f4b60005830 comp rx=0 tx=0).stop 2024-05-10T10:37:15.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.084+0000 7f4b7dc07640 1 -- 172.21.15.53:0/166424556 shutdown_connections 2024-05-10T10:37:15.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.084+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/166424556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b600051c0 0x7f4b6000b9c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.084+0000 7f4b7dc07640 1 -- 172.21.15.53:0/166424556 >> 172.21.15.53:0/166424556 conn(0x7f4b780d2e10 msgr2=0x7f4b780d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.084+0000 7f4b7dc07640 1 -- 172.21.15.53:0/166424556 shutdown_connections 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.084+0000 7f4b7dc07640 1 -- 172.21.15.53:0/166424556 wait complete. 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.085+0000 7f4b7dc07640 1 Processor -- start 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.085+0000 7f4b7dc07640 1 -- start start 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b7dc07640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600051c0 0x7f4b78158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b7dc07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b78161420 0x7f4b78156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b7dc07640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b78156bb0 0x7f4b780c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b7dc07640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78163290 con 0x7f4b78161420 2024-05-10T10:37:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b7dc07640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78070ac0 con 0x7f4b78156bb0 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b7dc07640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4b78163410 con 0x7f4b600051c0 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b78161420 0x7f4b78156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b78161420 0x7f4b78156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37434/0 (socket says 172.21.15.53:37434) 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 -- 172.21.15.53:0/3080943603 learned_addr learned my addr 172.21.15.53:0/3080943603 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b777fe640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600051c0 0x7f4b78158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 -- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600051c0 msgr2=0x7f4b78158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600051c0 0x7f4b78158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 -- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b78156bb0 msgr2=0x7f4b780c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b77fff640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b78156bb0 0x7f4b780c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b78156bb0 0x7f4b780c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 -- 172.21.15.53:0/3080943603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b781573c0 con 0x7f4b78161420 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.086+0000 7f4b76ffd640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b78161420 0x7f4b78156500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00f650 tx=0x7f4b6c00fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.087+0000 7f4b5ffff640 1 -- 172.21.15.53:0/3080943603 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0050d0 con 0x7f4b78161420 2024-05-10T10:37:15.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.087+0000 7f4b5ffff640 1 -- 172.21.15.53:0/3080943603 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b6c018070 con 0x7f4b78161420 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.087+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b781576a0 con 0x7f4b78161420 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.087+0000 7f4b5ffff640 1 -- 172.21.15.53:0/3080943603 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c005490 con 0x7f4b78161420 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.087+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b78157c60 con 0x7f4b78161420 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.088+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b7806a220 con 0x7f4b78161420 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.088+0000 7f4b5ffff640 1 -- 172.21.15.53:0/3080943603 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b6c00d730 con 0x7f4b78161420 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.089+0000 7f4b5ffff640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b640762d0 0x7f4b64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.089+0000 7f4b5ffff640 1 -- 172.21.15.53:0/3080943603 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7f4b6c097100 con 0x7f4b78161420 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.089+0000 7f4b777fe640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b640762d0 0x7f4b64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:15.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.089+0000 7f4b777fe640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b640762d0 0x7f4b64078790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b68001050 tx=0x7f4b68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:15.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.093+0000 7f4b5ffff640 1 -- 172.21.15.53:0/3080943603 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4b6c060cb0 con 0x7f4b78161420 2024-05-10T10:37:15.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:15.286+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b780b8d90 con 0x7f4b78161420 2024-05-10T10:37:16.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.200+0000 7f4b5ffff640 1 -- 172.21.15.53:0/3080943603 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f4b6c0605b0 con 0x7f4b78161420 2024-05-10T10:37:16.201 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b640762d0 msgr2=0x7f4b64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b640762d0 0x7f4b64078790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b68001050 tx=0x7f4b68009040 comp rx=0 tx=0).stop 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b78161420 msgr2=0x7f4b78156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b78161420 0x7f4b78156500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00f650 tx=0x7f4b6c00fb10 comp rx=0 tx=0).stop 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 shutdown_connections 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b640762d0 0x7f4b64078790 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b78156bb0 0x7f4b780c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b78161420 0x7f4b78156500 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 --2- 172.21.15.53:0/3080943603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b600051c0 0x7f4b78158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 >> 172.21.15.53:0/3080943603 conn(0x7f4b780d2e10 msgr2=0x7f4b7815e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:16.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.203+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 shutdown_connections 2024-05-10T10:37:16.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.204+0000 7f4b7dc07640 1 -- 172.21.15.53:0/3080943603 wait complete. 2024-05-10T10:37:16.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:37:16.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.337+0000 7f15e9df7640 1 Processor -- start 2024-05-10T10:37:16.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.337+0000 7f15e9df7640 1 -- start start 2024-05-10T10:37:16.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e9df7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 0x7f15e406df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e9df7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e4157a40 con 0x7f15e406c6a0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e9df7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e4160d80 con 0x7f15e415d810 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e9df7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f15e4161990 con 0x7f15e406c2c0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e2ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f15e406c6a0 0x7f15e40739e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49940/0 (socket says 172.21.15.53:49940) 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e2ffd640 1 -- 172.21.15.53:0/3679964686 learned_addr learned my addr 172.21.15.53:0/3679964686 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e37fe640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 0x7f15e406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3815694191 0 0) 0x7f15d8001110 con 0x7f15e406c6a0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.338+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15cc003520 con 0x7f15e406c6a0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2810192415 0 0) 0x7f15cc003520 con 0x7f15e406c6a0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 msgr2=0x7f15e406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 0x7f15e406df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 >> v1:172.21.15.135:6789/0 conn(0x7f15e415d810 legacy=0x7f15e415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e4161060 con 0x7f15e406c6a0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f15d8001840 con 0x7f15e406c6a0 2024-05-10T10:37:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 >> v1:172.21.15.53:6789/0 conn(0x7f15e406c6a0 legacy=0x7f15e40739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15cc0039e0 0x7f15cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 0x7f15cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15cc004d40 0x7f15cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4161990 con 0x7f15cc004d40 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc00b9c0 con 0x7f15cc0039e0 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f15cc00bbc0 con 0x7f15e406c2c0 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e3fff640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15cc004d40 0x7f15cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e37fe640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15cc0039e0 0x7f15cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e2ffd640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 0x7f15cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e3fff640 1 -- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 msgr2=0x7f15cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e3fff640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e406c2c0 0x7f15cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e3fff640 1 -- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15cc0039e0 msgr2=0x7f15cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e3fff640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15cc0039e0 0x7f15cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.339+0000 7f15e3fff640 1 -- 172.21.15.53:0/3679964686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc003520 con 0x7f15cc004d40 2024-05-10T10:37:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.340+0000 7f15e3fff640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15cc004d40 0x7f15cc0051c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f15dc001f60 tx=0x7f15cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cc6d1ed4f0cb1c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.340+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc001b30 con 0x7f15cc004d40 2024-05-10T10:37:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.340+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15dc031620 con 0x7f15cc004d40 2024-05-10T10:37:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.340+0000 7f15e27fc640 1 -- 172.21.15.53:0/3679964686 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc031a10 con 0x7f15cc004d40 2024-05-10T10:37:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.341+0000 7f15e9df7640 1 -- 172.21.15.53:0/3679964686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15cc004d40 msgr2=0x7f15cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.341+0000 7f15e9df7640 1 --2- 172.21.15.53:0/3679964686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15cc004d40 0x7f15cc0051c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f15dc001f60 tx=0x7f15cc005830 comp rx=0 tx=0).stop 2024-05-10T10:37:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.341+0000 7f15e3fff640 1 -- 172.21.15.53:0/3679964686 reap_dead start 2024-05-10T10:37:16.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.341+0000 7f15e9df7640 1 -- 172.21.15.53:0/3679964686 shutdown_connections 2024-05-10T10:37:16.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.341+0000 7f15e9df7640 1 -- 172.21.15.53:0/3679964686 >> 172.21.15.53:0/3679964686 conn(0x7f15e40d0c90 msgr2=0x7f15e40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:16.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.341+0000 7f15e9df7640 1 -- 172.21.15.53:0/3679964686 shutdown_connections 2024-05-10T10:37:16.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.341+0000 7f15e9df7640 1 -- 172.21.15.53:0/3679964686 wait complete. 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.342+0000 7f15e9df7640 1 Processor -- start 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.342+0000 7f15e9df7640 1 -- start start 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e9df7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e40b5b10 0x7f15e40c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e9df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e40b6610 0x7f15e40c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e9df7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e40b5ef0 0x7f15e414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e9df7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40cd440 con 0x7f15e40b6610 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e9df7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4161620 con 0x7f15e40b5b10 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e9df7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f15e406cb20 con 0x7f15e40b5ef0 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e40b6610 0x7f15e40c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e40b6610 0x7f15e40c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37462/0 (socket says 172.21.15.53:37462) 2024-05-10T10:37:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 -- 172.21.15.53:0/1975074515 learned_addr learned my addr 172.21.15.53:0/1975074515 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e3fff640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e40b5ef0 0x7f15e414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e37fe640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e40b5b10 0x7f15e40c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 -- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e40b5ef0 msgr2=0x7f15e414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e40b5ef0 0x7f15e414aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 -- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e40b5b10 msgr2=0x7f15e40c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e40b5b10 0x7f15e40c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 -- 172.21.15.53:0/1975074515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e40c6360 con 0x7f15e40b6610 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15e2ffd640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e40b6610 0x7f15e40c7d60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f15d80019b0 tx=0x7f15d8007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15cbfff640 1 -- 172.21.15.53:0/1975074515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8004e30 con 0x7f15e40b6610 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.343+0000 7f15cbfff640 1 -- 172.21.15.53:0/1975074515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d8018070 con 0x7f15e40b6610 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.344+0000 7f15cbfff640 1 -- 172.21.15.53:0/1975074515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d800fc00 con 0x7f15e40b6610 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.344+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e40c6620 con 0x7f15e40b6610 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.344+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e40c6be0 con 0x7f15e40b6610 2024-05-10T10:37:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.345+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e406a220 con 0x7f15e40b6610 2024-05-10T10:37:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.345+0000 7f15cbfff640 1 -- 172.21.15.53:0/1975074515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f15d801d460 con 0x7f15e40b6610 2024-05-10T10:37:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.346+0000 7f15cbfff640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15d0076300 0x7f15d00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.346+0000 7f15cbfff640 1 -- 172.21.15.53:0/1975074515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5619+0+0 (secure 0 0 0) 0x7f15d8097490 con 0x7f15e40b6610 2024-05-10T10:37:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.346+0000 7f15e37fe640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15d0076300 0x7f15d00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.347+0000 7f15e37fe640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15d0076300 0x7f15d00787c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009bc0 tx=0x7f15d4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:16.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.351+0000 7f15cbfff640 1 -- 172.21.15.53:0/1975074515 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f15e4157a40 con 0x7f15e40b6610 2024-05-10T10:37:16.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:16.542+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f15e406f510 con 0x7f15e40b6610 2024-05-10T10:37:17.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.206+0000 7f15cbfff640 1 -- 172.21.15.53:0/1975074515 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7f15d8060ed0 con 0x7f15e40b6610 2024-05-10T10:37:17.208 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:37:17.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.209+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15d0076300 msgr2=0x7f15d00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:17.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.209+0000 7f15e9df7640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15d0076300 0x7f15d00787c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009bc0 tx=0x7f15d4009190 comp rx=0 tx=0).stop 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.209+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e40b6610 msgr2=0x7f15e40c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.209+0000 7f15e9df7640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e40b6610 0x7f15e40c7d60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f15d80019b0 tx=0x7f15d8007660 comp rx=0 tx=0).stop 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 shutdown_connections 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15d0076300 0x7f15d00787c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f15e40b5ef0 0x7f15e414aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e40b6610 0x7f15e40c7d60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 --2- 172.21.15.53:0/1975074515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15e40b5b10 0x7f15e40c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 >> 172.21.15.53:0/1975074515 conn(0x7f15e40d0c90 msgr2=0x7f15e40b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 shutdown_connections 2024-05-10T10:37:17.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.210+0000 7f15e9df7640 1 -- 172.21.15.53:0/1975074515 wait complete. 2024-05-10T10:37:17.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:37:17.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.348+0000 7f6eafc6d640 1 Processor -- start 2024-05-10T10:37:17.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.348+0000 7f6eafc6d640 1 -- start start 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.349+0000 7f6eafc6d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6ea8162ce0 0x7f6ea80b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.349+0000 7f6eafc6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea80b4e20 0x7f6ea80d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.349+0000 7f6eafc6d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea80d4190 con 0x7f6ea8162900 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.349+0000 7f6eafc6d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea80cfc40 con 0x7f6ea8162ce0 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.349+0000 7f6eafc6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80d3eb0 con 0x7f6ea80b4e20 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6eae20a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea80b4e20 0x7f6ea80d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6ead208640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6ea8162ce0 0x7f6ea80b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6eae20a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea80b4e20 0x7f6ea80d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:47234/0 (socket says 172.21.15.53:47234) 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6ead208640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6ea8162ce0 0x7f6ea80b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:50092/0 (socket says 172.21.15.53:50092) 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6eada09640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f6ea8162900 0x7f6ea80d3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36382/0 (socket says 172.21.15.53:36382) 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6eae20a640 1 -- 172.21.15.53:0/242553712 learned_addr learned my addr 172.21.15.53:0/242553712 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6ead208640 1 -- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea80b4e20 msgr2=0x7f6ea80d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:17.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6ead208640 1 --2- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea80b4e20 0x7f6ea80d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6ead208640 1 -- 172.21.15.53:0/242553712 >> v1:172.21.15.135:6789/0 conn(0x7f6ea8162900 legacy=0x7f6ea80d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:17.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6ead208640 1 -- 172.21.15.53:0/242553712 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea80cfee0 con 0x7f6ea8162ce0 2024-05-10T10:37:17.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6eaca07640 1 -- 172.21.15.53:0/242553712 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3231761020 0 0) 0x7f6ea4001110 con 0x7f6ea8162900 2024-05-10T10:37:17.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.350+0000 7f6eae20a640 1 --2- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea80b4e20 0x7f6ea80d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:17.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.351+0000 7f6ead208640 1 --2- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80b48e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e980097a0 tx=0x7f6e9802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa9199aeefbeccbe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:17.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.351+0000 7f6eaca07640 1 -- 172.21.15.53:0/242553712 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98037070 con 0x7f6ea8162ce0 2024-05-10T10:37:17.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.351+0000 7f6eaca07640 1 -- 172.21.15.53:0/242553712 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9802ae50 con 0x7f6ea8162ce0 2024-05-10T10:37:17.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.351+0000 7f6eaca07640 1 -- 172.21.15.53:0/242553712 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98032660 con 0x7f6ea8162ce0 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.351+0000 7f6eafc6d640 1 -- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea8162ce0 msgr2=0x7f6ea80b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.351+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80b48e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e980097a0 tx=0x7f6e9802a100 comp rx=0 tx=0).stop 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.352+0000 7f6eafc6d640 1 -- 172.21.15.53:0/242553712 shutdown_connections 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.352+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea80b4e20 0x7f6ea80d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.352+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/242553712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80b48e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.352+0000 7f6eafc6d640 1 -- 172.21.15.53:0/242553712 >> 172.21.15.53:0/242553712 conn(0x7f6ea809e430 msgr2=0x7f6ea80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.352+0000 7f6eafc6d640 1 -- 172.21.15.53:0/242553712 shutdown_connections 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.352+0000 7f6eafc6d640 1 -- 172.21.15.53:0/242553712 wait complete. 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.353+0000 7f6eafc6d640 1 Processor -- start 2024-05-10T10:37:17.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.353+0000 7f6eafc6d640 1 -- start start 2024-05-10T10:37:17.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eafc6d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea80b4e20 0x7f6ea80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:17.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eafc6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea8162900 0x7f6ea814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eafc6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eafc6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80b4180 con 0x7f6ea8162ce0 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eafc6d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80d3b20 con 0x7f6ea80b4e20 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eafc6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80b4300 con 0x7f6ea8162900 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80ba4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37474/0 (socket says 172.21.15.53:37474) 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 -- 172.21.15.53:0/21842715 learned_addr learned my addr 172.21.15.53:0/21842715 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6ead208640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea8162900 0x7f6ea814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eada09640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea80b4e20 0x7f6ea80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 -- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea8162900 msgr2=0x7f6ea814f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea8162900 0x7f6ea814f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 -- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea80b4e20 msgr2=0x7f6ea80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea80b4e20 0x7f6ea80cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 -- 172.21.15.53:0/21842715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea814f8a0 con 0x7f6ea8162ce0 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6ead208640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea8162900 0x7f6ea814f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.354+0000 7f6eae20a640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80ba4a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009e70 tx=0x7f6ea000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.355+0000 7f6e9e7fc640 1 -- 172.21.15.53:0/21842715 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000cbe0 con 0x7f6ea8162ce0 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.355+0000 7f6e9e7fc640 1 -- 172.21.15.53:0/21842715 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea000cd80 con 0x7f6ea8162ce0 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.355+0000 7f6e9e7fc640 1 -- 172.21.15.53:0/21842715 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0003bb0 con 0x7f6ea8162ce0 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.355+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea80ce6b0 con 0x7f6ea8162ce0 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.355+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea80cec70 con 0x7f6ea8162ce0 2024-05-10T10:37:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.356+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea806a220 con 0x7f6ea8162ce0 2024-05-10T10:37:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.357+0000 7f6e9e7fc640 1 -- 172.21.15.53:0/21842715 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6ea001b030 con 0x7f6ea8162ce0 2024-05-10T10:37:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.357+0000 7f6e9e7fc640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e8c0762d0 0x7f6e8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.357+0000 7f6e9e7fc640 1 -- 172.21.15.53:0/21842715 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6001+0+0 (secure 0 0 0) 0x7f6ea0097950 con 0x7f6ea8162ce0 2024-05-10T10:37:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.357+0000 7f6eada09640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e8c0762d0 0x7f6e8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:17.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.358+0000 7f6eada09640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e8c0762d0 0x7f6e8c078790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4000f60 tx=0x7f6ea4007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:17.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.362+0000 7f6e9e7fc640 1 -- 172.21.15.53:0/21842715 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6ea0022020 con 0x7f6ea8162ce0 2024-05-10T10:37:17.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:17.591+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6ea809e840 con 0x7f6ea8162ce0 2024-05-10T10:37:18.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.208+0000 7f6e9e7fc640 1 -- 172.21.15.53:0/21842715 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v41) ==== 174+0+0 (secure 0 0 0) 0x7f6ea814f8a0 con 0x7f6ea8162ce0 2024-05-10T10:37:18.210 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 11 and data pool 12 2024-05-10T10:37:18.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.212+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e8c0762d0 msgr2=0x7f6e8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.212+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e8c0762d0 0x7f6e8c078790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4000f60 tx=0x7f6ea4007b50 comp rx=0 tx=0).stop 2024-05-10T10:37:18.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.212+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea8162ce0 msgr2=0x7f6ea80ba4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.212+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80ba4a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009e70 tx=0x7f6ea000c490 comp rx=0 tx=0).stop 2024-05-10T10:37:18.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.213+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 shutdown_connections 2024-05-10T10:37:18.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.213+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6e8c0762d0 0x7f6e8c078790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.213+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea8162ce0 0x7f6ea80ba4a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.213+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ea8162900 0x7f6ea814f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.213+0000 7f6eafc6d640 1 --2- 172.21.15.53:0/21842715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea80b4e20 0x7f6ea80cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.213+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 >> 172.21.15.53:0/21842715 conn(0x7f6ea809e430 msgr2=0x7f6ea806df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:18.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.214+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 shutdown_connections 2024-05-10T10:37:18.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.215+0000 7f6eafc6d640 1 -- 172.21.15.53:0/21842715 wait complete. 2024-05-10T10:37:18.244 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:37:18.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.351+0000 7f6723d6a640 1 Processor -- start 2024-05-10T10:37:18.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.351+0000 7f6723d6a640 1 -- start start 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6723d6a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f671c162b30 0x7f671c169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6723d6a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671c0b45f0 con 0x7f671c162750 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6723d6a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671c0b48d0 con 0x7f671c0d5cb0 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6723d6a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f671c0b4f20 con 0x7f671c162b30 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6722307640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f671c162b30 0x7f671c169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6721b06640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f671c162750 0x7f671c0d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49942/0 (socket says 172.21.15.53:49942) 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6721b06640 1 -- 172.21.15.53:0/309551217 learned_addr learned my addr 172.21.15.53:0/309551217 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6722307640 1 -- 172.21.15.53:0/309551217 >> v1:172.21.15.135:6789/0 conn(0x7f671c0d5cb0 legacy=0x7f671c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6722307640 1 -- 172.21.15.53:0/309551217 >> v1:172.21.15.53:6789/0 conn(0x7f671c162750 legacy=0x7f671c0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.352+0000 7f6722307640 1 -- 172.21.15.53:0/309551217 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c0b4bb0 con 0x7f671c162b30 2024-05-10T10:37:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.353+0000 7f6722307640 1 --2- 172.21.15.53:0/309551217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162b30 0x7f671c169fe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6714009d90 tx=0x7f671402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a881c8271efb426 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.353+0000 7f6720b04640 1 -- 172.21.15.53:0/309551217 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714031600 con 0x7f671c162b30 2024-05-10T10:37:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.353+0000 7f6720b04640 1 -- 172.21.15.53:0/309551217 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67140317a0 con 0x7f671c162b30 2024-05-10T10:37:18.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.353+0000 7f6720b04640 1 -- 172.21.15.53:0/309551217 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714031b70 con 0x7f671c162b30 2024-05-10T10:37:18.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.354+0000 7f6723d6a640 1 -- 172.21.15.53:0/309551217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162b30 msgr2=0x7f671c169fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.354+0000 7f6723d6a640 1 --2- 172.21.15.53:0/309551217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162b30 0x7f671c169fe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6714009d90 tx=0x7f671402a610 comp rx=0 tx=0).stop 2024-05-10T10:37:18.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.354+0000 7f6723d6a640 1 -- 172.21.15.53:0/309551217 shutdown_connections 2024-05-10T10:37:18.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.354+0000 7f6723d6a640 1 --2- 172.21.15.53:0/309551217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162b30 0x7f671c169fe0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.354+0000 7f6723d6a640 1 -- 172.21.15.53:0/309551217 >> 172.21.15.53:0/309551217 conn(0x7f671c09e3b0 msgr2=0x7f671c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:18.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.354+0000 7f6723d6a640 1 -- 172.21.15.53:0/309551217 shutdown_connections 2024-05-10T10:37:18.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.354+0000 7f6723d6a640 1 -- 172.21.15.53:0/309551217 wait complete. 2024-05-10T10:37:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.355+0000 7f6723d6a640 1 Processor -- start 2024-05-10T10:37:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.355+0000 7f6723d6a640 1 -- start start 2024-05-10T10:37:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6723d6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162750 0x7f671c1505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6723d6a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c162b30 0x7f671c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6723d6a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f671c0cc120 0x7f671c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6723d6a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f671c0d0fe0 con 0x7f671c162750 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6723d6a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f671c0d3f90 con 0x7f671c162b30 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6723d6a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f671c0d0d80 con 0x7f671c0cc120 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162750 0x7f671c1505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162750 0x7f671c1505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37480/0 (socket says 172.21.15.53:37480) 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 -- 172.21.15.53:0/3849774238 learned_addr learned my addr 172.21.15.53:0/3849774238 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721305640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c162b30 0x7f671c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6722307640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f671c0cc120 0x7f671c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 -- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f671c0cc120 msgr2=0x7f671c0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f671c0cc120 0x7f671c0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 -- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c162b30 msgr2=0x7f671c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c162b30 0x7f671c0ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721b06640 1 -- 172.21.15.53:0/3849774238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c150ae0 con 0x7f671c162750 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6722307640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f671c0cc120 0x7f671c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.356+0000 7f6721305640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c162b30 0x7f671c0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.357+0000 7f6721b06640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162750 0x7f671c1505a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f671800c430 tx=0x7f671800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.357+0000 7f67127fc640 1 -- 172.21.15.53:0/3849774238 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718017070 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.357+0000 7f67127fc640 1 -- 172.21.15.53:0/3849774238 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6718005ce0 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.357+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c0ca660 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.357+0000 7f67127fc640 1 -- 172.21.15.53:0/3849774238 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180072f0 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.357+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c0ca910 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.358+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c06a220 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.359+0000 7f67127fc640 1 -- 172.21.15.53:0/3849774238 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f671801a070 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.359+0000 7f67127fc640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.359+0000 7f67127fc640 1 -- 172.21.15.53:0/3849774238 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7f6718096b90 con 0x7f671c162750 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.359+0000 7f6721305640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.360+0000 7f6721305640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f670c006fd0 tx=0x7f670c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.364+0000 7f67127fc640 1 -- 172.21.15.53:0/3849774238 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6718060360 con 0x7f671c162750 2024-05-10T10:37:18.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.556+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f671c0d37f0 con 0x7f671c162750 2024-05-10T10:37:18.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.557+0000 7f67127fc640 1 -- 172.21.15.53:0/3849774238 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16958 (secure 0 0 0) 0x7f671805fc60 con 0x7f671c162750 2024-05-10T10:37:18.559 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:18.559 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":73,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:37:17.592904+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:37:15.291160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T10:37:16.547933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 msgr2=0x7f6700078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f670c006fd0 tx=0x7f670c008040 comp rx=0 tx=0).stop 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162750 msgr2=0x7f671c1505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162750 0x7f671c1505a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f671800c430 tx=0x7f671800c8f0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 shutdown_connections 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f671c0cc120 0x7f671c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f671c162b30 0x7f671c0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 --2- 172.21.15.53:0/3849774238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f671c162750 0x7f671c1505a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 >> 172.21.15.53:0/3849774238 conn(0x7f671c09e3b0 msgr2=0x7f671c0d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 shutdown_connections 2024-05-10T10:37:18.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.559+0000 7f6723d6a640 1 -- 172.21.15.53:0/3849774238 wait complete. 2024-05-10T10:37:18.572 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:37:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.673+0000 7fb6b94e8640 1 Processor -- start 2024-05-10T10:37:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.674+0000 7fb6b94e8640 1 -- start start 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.674+0000 7fb6b94e8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.674+0000 7fb6b94e8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb6b40aed60 0x7fb6b40af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.674+0000 7fb6b94e8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b40b5cc0 con 0x7fb6b4071360 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.674+0000 7fb6b94e8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b406fb70 con 0x7fb6b40aed60 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.674+0000 7fb6b94e8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b40703f0 con 0x7fb6b415be70 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb6b40aed60 0x7fb6b40af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b2ffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b27fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb6b4071360 0x7fb6b4070880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36412/0 (socket says 172.21.15.53:36412) 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b27fc640 1 -- 172.21.15.53:0/549803218 learned_addr learned my addr 172.21.15.53:0/549803218 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b1ffb640 1 -- 172.21.15.53:0/549803218 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405412439 0 0) 0x7fb6b40b5cc0 con 0x7fb6b4071360 2024-05-10T10:37:18.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b1ffb640 1 -- 172.21.15.53:0/549803218 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6a0003520 con 0x7fb6b4071360 2024-05-10T10:37:18.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b37fe640 1 -- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 msgr2=0x7fb6b40b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:18.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b37fe640 1 --2- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b37fe640 1 -- 172.21.15.53:0/549803218 >> v1:172.21.15.135:6789/0 conn(0x7fb6b4071360 legacy=0x7fb6b4070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b37fe640 1 -- 172.21.15.53:0/549803218 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b415c250 con 0x7fb6b40aed60 2024-05-10T10:37:18.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.675+0000 7fb6b2ffd640 1 --2- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:18.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.676+0000 7fb6b37fe640 1 --2- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b40aed60 0x7fb6b40af1f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40097a0 tx=0x7fb6a402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88a62584856d3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.676+0000 7fb6b1ffb640 1 -- 172.21.15.53:0/549803218 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4037070 con 0x7fb6b40aed60 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.676+0000 7fb6b1ffb640 1 -- 172.21.15.53:0/549803218 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a4032440 con 0x7fb6b40aed60 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.676+0000 7fb6b1ffb640 1 -- 172.21.15.53:0/549803218 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40327f0 con 0x7fb6b40aed60 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 -- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b40aed60 msgr2=0x7fb6b40af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b40aed60 0x7fb6b40af1f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40097a0 tx=0x7fb6a402a180 comp rx=0 tx=0).stop 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 -- 172.21.15.53:0/549803218 shutdown_connections 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b40aed60 0x7fb6b40af1f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/549803218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 -- 172.21.15.53:0/549803218 >> 172.21.15.53:0/549803218 conn(0x7fb6b40d1d70 msgr2=0x7fb6b40d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 -- 172.21.15.53:0/549803218 shutdown_connections 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.677+0000 7fb6b94e8640 1 -- 172.21.15.53:0/549803218 wait complete. 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.678+0000 7fb6b94e8640 1 Processor -- start 2024-05-10T10:37:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.678+0000 7fb6b94e8640 1 -- start start 2024-05-10T10:37:18.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b94e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b4071360 0x7fb6b4150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b94e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6b40aed60 0x7fb6b415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b94e8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b94e8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b4070080 con 0x7fb6b40aed60 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b94e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b406fda0 con 0x7fb6b4071360 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b94e8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b40b5690 con 0x7fb6b415be70 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6b40aed60 0x7fb6b415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6b40aed60 0x7fb6b415ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37494/0 (socket says 172.21.15.53:37494) 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 -- 172.21.15.53:0/1986746151 learned_addr learned my addr 172.21.15.53:0/1986746151 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b37fe640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b2ffd640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b4071360 0x7fb6b4150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 -- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 msgr2=0x7fb6b40ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 -- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b4071360 msgr2=0x7fb6b4150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b4071360 0x7fb6b4150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.679+0000 7fb6b27fc640 1 -- 172.21.15.53:0/1986746151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b4150e40 con 0x7fb6b40aed60 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.680+0000 7fb6b27fc640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6b40aed60 0x7fb6b415ec70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c400 tx=0x7fb69c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.680+0000 7fb6ab7fe640 1 -- 172.21.15.53:0/1986746151 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c017070 con 0x7fb6b40aed60 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.680+0000 7fb6ab7fe640 1 -- 172.21.15.53:0/1986746151 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69c005d00 con 0x7fb6b40aed60 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.680+0000 7fb6ab7fe640 1 -- 172.21.15.53:0/1986746151 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c007310 con 0x7fb6b40aed60 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.680+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b415da30 con 0x7fb6b40aed60 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.680+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b415e2d0 con 0x7fb6b40aed60 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.681+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b415de20 con 0x7fb6b40aed60 2024-05-10T10:37:18.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.681+0000 7fb6ab7fe640 1 -- 172.21.15.53:0/1986746151 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb69c01a070 con 0x7fb6b40aed60 2024-05-10T10:37:18.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.682+0000 7fb6ab7fe640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6940762d0 0x7fb694078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:18.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.682+0000 7fb6ab7fe640 1 -- 172.21.15.53:0/1986746151 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7fb69c096ae0 con 0x7fb6b40aed60 2024-05-10T10:37:18.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.682+0000 7fb6b2ffd640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6940762d0 0x7fb694078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:18.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.683+0000 7fb6b2ffd640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6940762d0 0x7fb694078790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb6b415b770 tx=0x7fb6ac00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:18.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.686+0000 7fb6ab7fe640 1 -- 172.21.15.53:0/1986746151 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb69c05fcf0 con 0x7fb6b40aed60 2024-05-10T10:37:18.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.882+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6b40b62c0 con 0x7fb6b40aed60 2024-05-10T10:37:18.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.883+0000 7fb6ab7fe640 1 -- 172.21.15.53:0/1986746151 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16958 (secure 0 0 0) 0x7fb69c063ff0 con 0x7fb6b40aed60 2024-05-10T10:37:18.885 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:18.885 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":73,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:37:17.592904+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:37:15.291160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T10:37:16.547933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6940762d0 msgr2=0x7fb694078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6940762d0 0x7fb694078790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb6b415b770 tx=0x7fb6ac00b040 comp rx=0 tx=0).stop 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6b40aed60 msgr2=0x7fb6b415ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6b40aed60 0x7fb6b415ec70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c400 tx=0x7fb69c00c8c0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 shutdown_connections 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6940762d0 0x7fb694078790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6b415be70 0x7fb6b40ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6b40aed60 0x7fb6b415ec70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 --2- 172.21.15.53:0/1986746151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b4071360 0x7fb6b4150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:18.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 >> 172.21.15.53:0/1986746151 conn(0x7fb6b40d1d70 msgr2=0x7fb6b40b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:18.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 shutdown_connections 2024-05-10T10:37:18.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:18.885+0000 7fb6b94e8640 1 -- 172.21.15.53:0/1986746151 wait complete. 2024-05-10T10:37:18.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:37:19.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.003+0000 7f6982f24640 1 Processor -- start 2024-05-10T10:37:19.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.003+0000 7f6982f24640 1 -- start start 2024-05-10T10:37:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.003+0000 7f6982f24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f697c15f410 0x7f697c0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.003+0000 7f6982f24640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f697c0d0f10 0x7f697c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.003+0000 7f6982f24640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697c0d2800 con 0x7f697c0b54d0 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.003+0000 7f6982f24640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f697c0d2ae0 con 0x7f697c0d0f10 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.003+0000 7f6982f24640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f697c0d01d0 con 0x7f697c15f410 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f697bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f697c0d0f10 0x7f697c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f69814c1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f697c0b54d0 0x7f697c0d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49946/0 (socket says 172.21.15.53:49946) 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f69814c1640 1 -- 172.21.15.53:0/1911896286 learned_addr learned my addr 172.21.15.53:0/1911896286 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f6980cc0640 1 --2- 172.21.15.53:0/1911896286 >> v2:172.21.15.135:3300/0 conn(0x7f697c15f410 0x7f697c0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f697bfff640 1 -- 172.21.15.53:0/1911896286 >> v2:172.21.15.135:3300/0 conn(0x7f697c15f410 msgr2=0x7f697c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f697bfff640 1 --2- 172.21.15.53:0/1911896286 >> v2:172.21.15.135:3300/0 conn(0x7f697c15f410 0x7f697c0d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f697bfff640 1 -- 172.21.15.53:0/1911896286 >> v1:172.21.15.53:6789/0 conn(0x7f697c0b54d0 legacy=0x7f697c0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.004+0000 7f697bfff640 1 -- 172.21.15.53:0/1911896286 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c0d04b0 con 0x7f697c0d0f10 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f697bfff640 1 --2- 172.21.15.53:0/1911896286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0d0f10 0x7f697c0b4f90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f697c08f160 tx=0x7f696c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f1491898296617 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f697b7fe640 1 -- 172.21.15.53:0/1911896286 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c007740 con 0x7f697c0d0f10 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f697b7fe640 1 -- 172.21.15.53:0/1911896286 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c0078e0 con 0x7f697c0d0f10 2024-05-10T10:37:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f697b7fe640 1 -- 172.21.15.53:0/1911896286 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c007cb0 con 0x7f697c0d0f10 2024-05-10T10:37:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 -- 172.21.15.53:0/1911896286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0d0f10 msgr2=0x7f697c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 --2- 172.21.15.53:0/1911896286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0d0f10 0x7f697c0b4f90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f697c08f160 tx=0x7f696c02a5f0 comp rx=0 tx=0).stop 2024-05-10T10:37:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 -- 172.21.15.53:0/1911896286 shutdown_connections 2024-05-10T10:37:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 --2- 172.21.15.53:0/1911896286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0d0f10 0x7f697c0b4f90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 --2- 172.21.15.53:0/1911896286 >> v2:172.21.15.135:3300/0 conn(0x7f697c15f410 0x7f697c0d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 -- 172.21.15.53:0/1911896286 >> 172.21.15.53:0/1911896286 conn(0x7f697c08ed30 msgr2=0x7f697c09e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 -- 172.21.15.53:0/1911896286 shutdown_connections 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.005+0000 7f6982f24640 1 -- 172.21.15.53:0/1911896286 wait complete. 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 Processor -- start 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 -- start start 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0b54d0 0x7f697c14be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f697c0d0f10 0x7f697c0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f697c15f410 0x7f697c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f697c0cffb0 con 0x7f697c0b54d0 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f697c0cfe30 con 0x7f697c0d0f10 2024-05-10T10:37:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.007+0000 7f6982f24640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f697c0d3de0 con 0x7f697c15f410 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0b54d0 0x7f697c14be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0b54d0 0x7f697c14be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37514/0 (socket says 172.21.15.53:37514) 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 -- 172.21.15.53:0/3501395151 learned_addr learned my addr 172.21.15.53:0/3501395151 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 -- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f697c15f410 msgr2=0x7f697c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f697c15f410 0x7f697c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 -- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f697c0d0f10 msgr2=0x7f697c0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f697c0d0f10 0x7f697c0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 -- 172.21.15.53:0/3501395151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c14f8a0 con 0x7f697c0b54d0 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6980cc0640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0b54d0 0x7f697c14be40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f697400d5f0 tx=0x7f697400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f69797fa640 1 -- 172.21.15.53:0/3501395151 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740125f0 con 0x7f697c0b54d0 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f69797fa640 1 -- 172.21.15.53:0/3501395151 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6974019070 con 0x7f697c0b54d0 2024-05-10T10:37:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f69797fa640 1 -- 172.21.15.53:0/3501395151 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740129d0 con 0x7f697c0b54d0 2024-05-10T10:37:19.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c0ce780 con 0x7f697c0b54d0 2024-05-10T10:37:19.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.008+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c0ced40 con 0x7f697c0b54d0 2024-05-10T10:37:19.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.009+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c06a220 con 0x7f697c0b54d0 2024-05-10T10:37:19.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.010+0000 7f69797fa640 1 -- 172.21.15.53:0/3501395151 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6974012b70 con 0x7f697c0b54d0 2024-05-10T10:37:19.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.010+0000 7f69797fa640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f695c076300 0x7f695c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:19.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.010+0000 7f69797fa640 1 -- 172.21.15.53:0/3501395151 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7f6974097cd0 con 0x7f697c0b54d0 2024-05-10T10:37:19.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.011+0000 7f697bfff640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f695c076300 0x7f695c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:19.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.011+0000 7f697bfff640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f695c076300 0x7f695c0787c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f696c034000 tx=0x7f696c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:19.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.015+0000 7f69797fa640 1 -- 172.21.15.53:0/3501395151 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6974061400 con 0x7f697c0b54d0 2024-05-10T10:37:19.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:19.246+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f697c0b4930 con 0x7f697c0b54d0 2024-05-10T10:37:20.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.232+0000 7f69797fa640 1 -- 172.21.15.53:0/3501395151 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f6974060d00 con 0x7f697c0b54d0 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f695c076300 msgr2=0x7f695c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f695c076300 0x7f695c0787c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f696c034000 tx=0x7f696c02e000 comp rx=0 tx=0).stop 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0b54d0 msgr2=0x7f697c14be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0b54d0 0x7f697c14be40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f697400d5f0 tx=0x7f697400dab0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 shutdown_connections 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f695c076300 0x7f695c0787c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f697c15f410 0x7f697c0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f697c0d0f10 0x7f697c0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 --2- 172.21.15.53:0/3501395151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0b54d0 0x7f697c14be40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 >> 172.21.15.53:0/3501395151 conn(0x7f697c08ed30 msgr2=0x7f697c1638b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 shutdown_connections 2024-05-10T10:37:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.235+0000 7f6982f24640 1 -- 172.21.15.53:0/3501395151 wait complete. 2024-05-10T10:37:20.250 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T10:37:20.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:37:20.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.364+0000 7fc16b546640 1 Processor -- start 2024-05-10T10:37:20.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.365+0000 7fc16b546640 1 -- start start 2024-05-10T10:37:20.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.365+0000 7fc16b546640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc164159340 0x7fc1640ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:20.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.365+0000 7fc16b546640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc1640ceda0 0x7fc1640cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.365+0000 7fc16b546640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1640cbed0 con 0x7fc1640cbac0 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.365+0000 7fc16b546640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc164159720 con 0x7fc164159340 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.365+0000 7fc16b546640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc16406df20 con 0x7fc1640ceda0 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.366+0000 7fc1692e2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc164159340 0x7fc1640ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.366+0000 7fc1692e2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc164159340 0x7fc1640ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37518/0 (socket says 172.21.15.53:37518) 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.366+0000 7fc1692e2640 1 -- 172.21.15.53:0/3639831758 learned_addr learned my addr 172.21.15.53:0/3639831758 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.366+0000 7fc1692e2640 1 -- 172.21.15.53:0/3639831758 >> v2:172.21.15.135:3300/0 conn(0x7fc1640ceda0 msgr2=0x7fc1640cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.366+0000 7fc1692e2640 1 --2- 172.21.15.53:0/3639831758 >> v2:172.21.15.135:3300/0 conn(0x7fc1640ceda0 0x7fc1640cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.366+0000 7fc1692e2640 1 -- 172.21.15.53:0/3639831758 >> v1:172.21.15.135:6789/0 conn(0x7fc1640cbac0 legacy=0x7fc16415f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.366+0000 7fc1692e2640 1 -- 172.21.15.53:0/3639831758 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1640cf3d0 con 0x7fc164159340 2024-05-10T10:37:20.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc1692e2640 1 --2- 172.21.15.53:0/3639831758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc1640ce860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc16406d370 tx=0x7fc16002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2152711fd1b87715 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc15bfff640 1 -- 172.21.15.53:0/3639831758 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160037070 con 0x7fc164159340 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc15bfff640 1 -- 172.21.15.53:0/3639831758 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc160032440 con 0x7fc164159340 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc15bfff640 1 -- 172.21.15.53:0/3639831758 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160032810 con 0x7fc164159340 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 -- 172.21.15.53:0/3639831758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 msgr2=0x7fc1640ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 --2- 172.21.15.53:0/3639831758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc1640ce860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc16406d370 tx=0x7fc16002a230 comp rx=0 tx=0).stop 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 -- 172.21.15.53:0/3639831758 shutdown_connections 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 --2- 172.21.15.53:0/3639831758 >> v2:172.21.15.135:3300/0 conn(0x7fc1640ceda0 0x7fc1640cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 --2- 172.21.15.53:0/3639831758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc1640ce860 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 -- 172.21.15.53:0/3639831758 >> 172.21.15.53:0/3639831758 conn(0x7fc1640d0c90 msgr2=0x7fc1640d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 -- 172.21.15.53:0/3639831758 shutdown_connections 2024-05-10T10:37:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.367+0000 7fc16b546640 1 -- 172.21.15.53:0/3639831758 wait complete. 2024-05-10T10:37:20.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 Processor -- start 2024-05-10T10:37:20.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 -- start start 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1640cbac0 0x7fc164158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640ceda0 0x7fc1641483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc16414f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640cfc70 con 0x7fc164159340 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640cf990 con 0x7fc1640ceda0 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc16b546640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc16415bfd0 con 0x7fc1640cbac0 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.369+0000 7fc169ae3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc16414f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc16414f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37534/0 (socket says 172.21.15.53:37534) 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 -- 172.21.15.53:0/321304476 learned_addr learned my addr 172.21.15.53:0/321304476 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 -- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1640cbac0 msgr2=0x7fc164158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1640cbac0 0x7fc164158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 -- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640ceda0 msgr2=0x7fc1641483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc168ae1640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640ceda0 0x7fc1641483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640ceda0 0x7fc1641483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 -- 172.21.15.53:0/321304476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16415cee0 con 0x7fc164159340 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc168ae1640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640ceda0 0x7fc1641483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc169ae3640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc16414f960 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00c780 tx=0x7fc15c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc159ffb640 1 -- 172.21.15.53:0/321304476 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c017070 con 0x7fc164159340 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc159ffb640 1 -- 172.21.15.53:0/321304476 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c003da0 con 0x7fc164159340 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc159ffb640 1 -- 172.21.15.53:0/321304476 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0052b0 con 0x7fc164159340 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.370+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc164073d60 con 0x7fc164159340 2024-05-10T10:37:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.371+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc164074040 con 0x7fc164159340 2024-05-10T10:37:20.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.371+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16406a220 con 0x7fc164159340 2024-05-10T10:37:20.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.372+0000 7fc159ffb640 1 -- 172.21.15.53:0/321304476 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc15c01a070 con 0x7fc164159340 2024-05-10T10:37:20.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.372+0000 7fc159ffb640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1480762d0 0x7fc148078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:20.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.373+0000 7fc159ffb640 1 -- 172.21.15.53:0/321304476 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc15c096dd0 con 0x7fc164159340 2024-05-10T10:37:20.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.373+0000 7fc1692e2640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1480762d0 0x7fc148078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:20.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.373+0000 7fc1692e2640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1480762d0 0x7fc148078790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc160034000 tx=0x7fc16002c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:20.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.377+0000 7fc159ffb640 1 -- 172.21.15.53:0/321304476 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc15c060500 con 0x7fc164159340 2024-05-10T10:37:20.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:20.607+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc1640ce920 con 0x7fc164159340 2024-05-10T10:37:21.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.237+0000 7fc159ffb640 1 -- 172.21.15.53:0/321304476 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fc15c05fe00 con 0x7fc164159340 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1480762d0 msgr2=0x7fc148078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1480762d0 0x7fc148078790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc160034000 tx=0x7fc16002c000 comp rx=0 tx=0).stop 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 msgr2=0x7fc16414f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc16414f960 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00c780 tx=0x7fc15c00cc40 comp rx=0 tx=0).stop 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 shutdown_connections 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc1480762d0 0x7fc148078790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc164159340 0x7fc16414f960 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640ceda0 0x7fc1641483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 --2- 172.21.15.53:0/321304476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1640cbac0 0x7fc164158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 >> 172.21.15.53:0/321304476 conn(0x7fc1640d0c90 msgr2=0x7fc16414ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:21.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 shutdown_connections 2024-05-10T10:37:21.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.241+0000 7fc16b546640 1 -- 172.21.15.53:0/321304476 wait complete. 2024-05-10T10:37:21.255 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:37:21.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:21.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.364+0000 7f0f1fa23640 1 Processor -- start 2024-05-10T10:37:21.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.365+0000 7f0f1fa23640 1 -- start start 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.365+0000 7f0f1fa23640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f1815f290 0x7f0f180d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.365+0000 7f0f1fa23640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0f180702e0 0x7f0f180d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.365+0000 7f0f1fa23640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f1806f3c0 con 0x7f0f180d3280 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.365+0000 7f0f1fa23640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0f180d14c0 con 0x7f0f180702e0 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.365+0000 7f0f1fa23640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0f180d2300 con 0x7f0f1815f290 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1cfbe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0f180702e0 0x7f0f180d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1cfbe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0f180702e0 0x7f0f180d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37536/0 (socket says 172.21.15.53:37536) 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1cfbe640 1 -- 172.21.15.53:0/2969880365 learned_addr learned my addr 172.21.15.53:0/2969880365 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1d7bf640 1 --2- 172.21.15.53:0/2969880365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f1815f290 0x7f0f180d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1cfbe640 1 -- 172.21.15.53:0/2969880365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f1815f290 msgr2=0x7f0f180d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:21.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1cfbe640 1 --2- 172.21.15.53:0/2969880365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f1815f290 0x7f0f180d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1cfbe640 1 -- 172.21.15.53:0/2969880365 >> v1:172.21.15.135:6789/0 conn(0x7f0f180d3280 legacy=0x7f0f180cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:21.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1cfbe640 1 -- 172.21.15.53:0/2969880365 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f180d16f0 con 0x7f0f180702e0 2024-05-10T10:37:21.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.366+0000 7f0f1d7bf640 1 --2- 172.21.15.53:0/2969880365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f1815f290 0x7f0f180d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:21.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f1cfbe640 1 --2- 172.21.15.53:0/2969880365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f180702e0 0x7f0f180d2d40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009870 tx=0x7f0f0c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d52b478ac432a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f17fff640 1 -- 172.21.15.53:0/2969880365 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c037070 con 0x7f0f180702e0 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f17fff640 1 -- 172.21.15.53:0/2969880365 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f0c032440 con 0x7f0f180702e0 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f17fff640 1 -- 172.21.15.53:0/2969880365 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c032810 con 0x7f0f180702e0 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f1fa23640 1 -- 172.21.15.53:0/2969880365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f180702e0 msgr2=0x7f0f180d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/2969880365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f180702e0 0x7f0f180d2d40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009870 tx=0x7f0f0c02a2a0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f1fa23640 1 -- 172.21.15.53:0/2969880365 shutdown_connections 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/2969880365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f180702e0 0x7f0f180d2d40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/2969880365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f1815f290 0x7f0f180d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.367+0000 7f0f1fa23640 1 -- 172.21.15.53:0/2969880365 >> 172.21.15.53:0/2969880365 conn(0x7f0f1809ba20 msgr2=0x7f0f1809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.368+0000 7f0f1fa23640 1 -- 172.21.15.53:0/2969880365 shutdown_connections 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.368+0000 7f0f1fa23640 1 -- 172.21.15.53:0/2969880365 wait complete. 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.369+0000 7f0f1fa23640 1 Processor -- start 2024-05-10T10:37:21.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.369+0000 7f0f1fa23640 1 -- start start 2024-05-10T10:37:21.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1fa23640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f180702e0 0x7f0f180c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1fa23640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f180d3280 0x7f0f1815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1fa23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1815f290 0x7f0f181e9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1fa23640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f180d1f90 con 0x7f0f1815f290 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1fa23640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f180d1cb0 con 0x7f0f180702e0 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1fa23640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1815f670 con 0x7f0f180d3280 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1815f290 0x7f0f181e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1815f290 0x7f0f181e9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37540/0 (socket says 172.21.15.53:37540) 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 -- 172.21.15.53:0/1486263300 learned_addr learned my addr 172.21.15.53:0/1486263300 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1d7bf640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f180702e0 0x7f0f180c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1cfbe640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f180d3280 0x7f0f1815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 -- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f180d3280 msgr2=0x7f0f1815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f180d3280 0x7f0f1815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 -- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f180702e0 msgr2=0x7f0f180c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f180702e0 0x7f0f180c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 -- 172.21.15.53:0/1486263300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181ea8c0 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1dfc0640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1815f290 0x7f0f181e9890 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c410 tx=0x7f0f1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f15ffb640 1 -- 172.21.15.53:0/1486263300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10011790 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f15ffb640 1 -- 172.21.15.53:0/1486263300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f10017070 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.370+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f180c76f0 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.371+0000 7f0f15ffb640 1 -- 172.21.15.53:0/1486263300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10011b70 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.371+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f180c73e0 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.372+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f180c78c0 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.372+0000 7f0f15ffb640 1 -- 172.21.15.53:0/1486263300 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0f10011d10 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.373+0000 7f0f15ffb640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f040762d0 0x7f0f04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.373+0000 7f0f15ffb640 1 -- 172.21.15.53:0/1486263300 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f0f100973e0 con 0x7f0f1815f290 2024-05-10T10:37:21.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.373+0000 7f0f1d7bf640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f040762d0 0x7f0f04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.373+0000 7f0f1d7bf640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f040762d0 0x7f0f04078790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f080075b0 tx=0x7f0f08007540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.378+0000 7f0f15ffb640 1 -- 172.21.15.53:0/1486263300 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0f10021020 con 0x7f0f1815f290 2024-05-10T10:37:21.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.609+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f180693f0 con 0x7f0f1815f290 2024-05-10T10:37:21.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.610+0000 7f0f15ffb640 1 -- 172.21.15.53:0/1486263300 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5186 (secure 0 0 0) 0x7f0f10060b10 con 0x7f0f1815f290 2024-05-10T10:37:21.612 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:21.612 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:37:20.609173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6834/1984524370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1984524370},{"type":"v1","addr":"172.21.15.53:6834","nonce":1984524370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6102,"qdb_cluster":[6102]},"id":5}]} 2024-05-10T10:37:21.612 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 45 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f040762d0 msgr2=0x7f0f04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f040762d0 0x7f0f04078790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f080075b0 tx=0x7f0f08007540 comp rx=0 tx=0).stop 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1815f290 msgr2=0x7f0f181e9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1815f290 0x7f0f181e9890 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c410 tx=0x7f0f1000c8d0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 shutdown_connections 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f040762d0 0x7f0f04078790 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1815f290 0x7f0f181e9890 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f180d3280 0x7f0f1815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 --2- 172.21.15.53:0/1486263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f180702e0 0x7f0f180c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 >> 172.21.15.53:0/1486263300 conn(0x7f0f1809ba20 msgr2=0x7f0f180b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:21.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 shutdown_connections 2024-05-10T10:37:21.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.612+0000 7f0f1fa23640 1 -- 172.21.15.53:0/1486263300 wait complete. 2024-05-10T10:37:21.625 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:37:21.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.728+0000 7f4c65672640 1 Processor -- start 2024-05-10T10:37:21.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.728+0000 7f4c65672640 1 -- start start 2024-05-10T10:37:21.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c65672640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c60163730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c65672640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c600caf50 0x7f4c6015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c65672640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c60160240 con 0x7f4c6015f730 2024-05-10T10:37:21.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c65672640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4c60163c70 con 0x7f4c600caf50 2024-05-10T10:37:21.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c65672640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4c60160520 con 0x7f4c6006fa60 2024-05-10T10:37:21.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c600caf50 0x7f4c6015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c600caf50 0x7f4c6015fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37544/0 (socket says 172.21.15.53:37544) 2024-05-10T10:37:21.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5f7fe640 1 -- 172.21.15.53:0/2805290146 learned_addr learned my addr 172.21.15.53:0/2805290146 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:21.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5effd640 1 --2- 172.21.15.53:0/2805290146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c60163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5f7fe640 1 -- 172.21.15.53:0/2805290146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 msgr2=0x7f4c60163730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:21.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5f7fe640 1 --2- 172.21.15.53:0/2805290146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c60163730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5f7fe640 1 -- 172.21.15.53:0/2805290146 >> v1:172.21.15.135:6789/0 conn(0x7f4c6015f730 legacy=0x7f4c600ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:21.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.729+0000 7f4c5f7fe640 1 -- 172.21.15.53:0/2805290146 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c60162f30 con 0x7f4c600caf50 2024-05-10T10:37:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.730+0000 7f4c5f7fe640 1 --2- 172.21.15.53:0/2805290146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015fb90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009870 tx=0x7f4c5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b027d244cc1ed8d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.730+0000 7f4c5dffb640 1 -- 172.21.15.53:0/2805290146 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58037070 con 0x7f4c600caf50 2024-05-10T10:37:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.730+0000 7f4c5dffb640 1 -- 172.21.15.53:0/2805290146 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c58032440 con 0x7f4c600caf50 2024-05-10T10:37:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.730+0000 7f4c5dffb640 1 -- 172.21.15.53:0/2805290146 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58032810 con 0x7f4c600caf50 2024-05-10T10:37:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 -- 172.21.15.53:0/2805290146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 msgr2=0x7f4c6015fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 --2- 172.21.15.53:0/2805290146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015fb90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009870 tx=0x7f4c5802a180 comp rx=0 tx=0).stop 2024-05-10T10:37:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 -- 172.21.15.53:0/2805290146 shutdown_connections 2024-05-10T10:37:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 --2- 172.21.15.53:0/2805290146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015fb90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 --2- 172.21.15.53:0/2805290146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c60163730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 -- 172.21.15.53:0/2805290146 >> 172.21.15.53:0/2805290146 conn(0x7f4c600d1510 msgr2=0x7f4c600d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 -- 172.21.15.53:0/2805290146 shutdown_connections 2024-05-10T10:37:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.731+0000 7f4c65672640 1 -- 172.21.15.53:0/2805290146 wait complete. 2024-05-10T10:37:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.732+0000 7f4c65672640 1 Processor -- start 2024-05-10T10:37:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.732+0000 7f4c65672640 1 -- start start 2024-05-10T10:37:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c65672640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c600b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c65672640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c65672640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6015f730 0x7f4c60145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c65672640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6006c560 con 0x7f4c600caf50 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c65672640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6006eae0 con 0x7f4c6015f730 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c65672640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6006c6e0 con 0x7f4c6006fa60 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37548/0 (socket says 172.21.15.53:37548) 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/3500881807 learned_addr learned my addr 172.21.15.53:0/3500881807 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5effd640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c600b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 msgr2=0x7f4c600b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c600b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5f7fe640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6015f730 0x7f4c60145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6015f730 msgr2=0x7f4c60145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6015f730 0x7f4c60145770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.733+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/3500881807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c60145cb0 con 0x7f4c600caf50 2024-05-10T10:37:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.734+0000 7f4c5e7fc640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015cae0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b3e0 tx=0x7f4c4800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.734+0000 7f4c577fe640 1 -- 172.21.15.53:0/3500881807 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48015020 con 0x7f4c600caf50 2024-05-10T10:37:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.734+0000 7f4c577fe640 1 -- 172.21.15.53:0/3500881807 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c48011850 con 0x7f4c600caf50 2024-05-10T10:37:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.734+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6015ba40 con 0x7f4c600caf50 2024-05-10T10:37:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.734+0000 7f4c577fe640 1 -- 172.21.15.53:0/3500881807 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48011c40 con 0x7f4c600caf50 2024-05-10T10:37:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.734+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6015c000 con 0x7f4c600caf50 2024-05-10T10:37:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.735+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6006a220 con 0x7f4c600caf50 2024-05-10T10:37:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.736+0000 7f4c577fe640 1 -- 172.21.15.53:0/3500881807 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c48018020 con 0x7f4c600caf50 2024-05-10T10:37:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.736+0000 7f4c577fe640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c400762d0 0x7f4c40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.736+0000 7f4c577fe640 1 -- 172.21.15.53:0/3500881807 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f4c48097610 con 0x7f4c600caf50 2024-05-10T10:37:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.736+0000 7f4c5effd640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c400762d0 0x7f4c40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.737+0000 7f4c5effd640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c400762d0 0x7f4c40078790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4c50008ef0 tx=0x7f4c50008440 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:21.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.740+0000 7f4c577fe640 1 -- 172.21.15.53:0/3500881807 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4c4801f020 con 0x7f4c600caf50 2024-05-10T10:37:21.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.932+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c60068610 con 0x7f4c600caf50 2024-05-10T10:37:21.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.933+0000 7f4c577fe640 1 -- 172.21.15.53:0/3500881807 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16945 (secure 0 0 0) 0x7f4c48060e70 con 0x7f4c600caf50 2024-05-10T10:37:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:21.934 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":74,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:37:19.223104+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:37:15.291160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T10:37:16.547933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c400762d0 msgr2=0x7f4c40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c400762d0 0x7f4c40078790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4c50008ef0 tx=0x7f4c50008440 comp rx=0 tx=0).stop 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 msgr2=0x7f4c6015cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015cae0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b3e0 tx=0x7f4c4800b8a0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 shutdown_connections 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c400762d0 0x7f4c40078790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c6015f730 0x7f4c60145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600caf50 0x7f4c6015cae0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 --2- 172.21.15.53:0/3500881807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c6006fa60 0x7f4c600b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 >> 172.21.15.53:0/3500881807 conn(0x7f4c600d1510 msgr2=0x7f4c600d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 shutdown_connections 2024-05-10T10:37:21.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:21.935+0000 7f4c65672640 1 -- 172.21.15.53:0/3500881807 wait complete. 2024-05-10T10:37:21.947 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:37:21.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:37:22.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.049+0000 7f3771d90640 1 Processor -- start 2024-05-10T10:37:22.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.050+0000 7f3771d90640 1 -- start start 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.050+0000 7f3771d90640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f376c0d3450 0x7f376c0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.050+0000 7f3771d90640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c0702e0 0x7f376c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.050+0000 7f3771d90640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376c06f3c0 con 0x7f376c15f290 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.050+0000 7f3771d90640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f376c0d14c0 con 0x7f376c0d3450 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.050+0000 7f3771d90640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f376c0d2300 con 0x7f376c0702e0 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376bfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c0702e0 0x7f376c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376bfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c0702e0 0x7f376c06d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:47346/0 (socket says 172.21.15.53:47346) 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376affd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f376c0d3450 0x7f376c0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376bfff640 1 -- 172.21.15.53:0/691732376 learned_addr learned my addr 172.21.15.53:0/691732376 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376bfff640 1 -- 172.21.15.53:0/691732376 >> v2:172.21.15.135:3300/0 conn(0x7f376c0d3450 msgr2=0x7f376c0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376bfff640 1 --2- 172.21.15.53:0/691732376 >> v2:172.21.15.135:3300/0 conn(0x7f376c0d3450 0x7f376c0d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376bfff640 1 -- 172.21.15.53:0/691732376 >> v1:172.21.15.135:6789/0 conn(0x7f376c15f290 legacy=0x7f376c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.051+0000 7f376bfff640 1 -- 172.21.15.53:0/691732376 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c0d16f0 con 0x7f376c0702e0 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.052+0000 7f376bfff640 1 --2- 172.21.15.53:0/691732376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c0702e0 0x7f376c06d9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3764009e50 tx=0x7f376402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2385c30fb87600 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.052+0000 7f376a7fc640 1 -- 172.21.15.53:0/691732376 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764036070 con 0x7f376c0702e0 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.052+0000 7f376a7fc640 1 -- 172.21.15.53:0/691732376 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3764031600 con 0x7f376c0702e0 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.052+0000 7f376a7fc640 1 -- 172.21.15.53:0/691732376 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640319d0 con 0x7f376c0702e0 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 -- 172.21.15.53:0/691732376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c0702e0 msgr2=0x7f376c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 --2- 172.21.15.53:0/691732376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c0702e0 0x7f376c06d9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3764009e50 tx=0x7f376402a7a0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 -- 172.21.15.53:0/691732376 shutdown_connections 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 --2- 172.21.15.53:0/691732376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c0702e0 0x7f376c06d9f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 --2- 172.21.15.53:0/691732376 >> v2:172.21.15.135:3300/0 conn(0x7f376c0d3450 0x7f376c0d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 -- 172.21.15.53:0/691732376 >> 172.21.15.53:0/691732376 conn(0x7f376c09ba20 msgr2=0x7f376c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:22.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 -- 172.21.15.53:0/691732376 shutdown_connections 2024-05-10T10:37:22.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.053+0000 7f3771d90640 1 -- 172.21.15.53:0/691732376 wait complete. 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.054+0000 7f3771d90640 1 Processor -- start 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.054+0000 7f3771d90640 1 -- start start 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f3771d90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f376c0702e0 0x7f376c14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f3771d90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f376c0d3450 0x7f376c09d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f3771d90640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c15f290 0x7f376c09d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f3771d90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f376c0d1f90 con 0x7f376c0d3450 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f3771d90640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f376c0d1cb0 con 0x7f376c0702e0 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f3771d90640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f376c15f670 con 0x7f376c15f290 2024-05-10T10:37:22.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f376c0d3450 0x7f376c09d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f376c0d3450 0x7f376c09d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37562/0 (socket says 172.21.15.53:37562) 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 -- 172.21.15.53:0/2329183417 learned_addr learned my addr 172.21.15.53:0/2329183417 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376b7fe640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f376c0702e0 0x7f376c14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376bfff640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c15f290 0x7f376c09d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 -- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c15f290 msgr2=0x7f376c09d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c15f290 0x7f376c09d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 -- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f376c0702e0 msgr2=0x7f376c14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f376c0702e0 0x7f376c14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 -- 172.21.15.53:0/2329183417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c09fe40 con 0x7f376c0d3450 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.055+0000 7f376affd640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f376c0d3450 0x7f376c09d0c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f376000e930 tx=0x7f376000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.056+0000 7f3753fff640 1 -- 172.21.15.53:0/2329183417 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760017070 con 0x7f376c0d3450 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.056+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c0a0120 con 0x7f376c0d3450 2024-05-10T10:37:22.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.056+0000 7f3753fff640 1 -- 172.21.15.53:0/2329183417 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376000ca00 con 0x7f376c0d3450 2024-05-10T10:37:22.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.056+0000 7f3753fff640 1 -- 172.21.15.53:0/2329183417 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000cdf0 con 0x7f376c0d3450 2024-05-10T10:37:22.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.056+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c0a06e0 con 0x7f376c0d3450 2024-05-10T10:37:22.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.057+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c0a0a10 con 0x7f376c0d3450 2024-05-10T10:37:22.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.057+0000 7f3753fff640 1 -- 172.21.15.53:0/2329183417 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f376001a070 con 0x7f376c0d3450 2024-05-10T10:37:22.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.058+0000 7f3753fff640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f37580762d0 0x7f3758078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.058+0000 7f3753fff640 1 -- 172.21.15.53:0/2329183417 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f3760097b30 con 0x7f376c0d3450 2024-05-10T10:37:22.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.058+0000 7f376b7fe640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f37580762d0 0x7f3758078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.058+0000 7f376b7fe640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f37580762d0 0x7f3758078790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f375c001780 tx=0x7f375c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.062+0000 7f3753fff640 1 -- 172.21.15.53:0/2329183417 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3760060d80 con 0x7f376c0d3450 2024-05-10T10:37:22.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.251+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f376c06d3d0 con 0x7f37580762d0 2024-05-10T10:37:22.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.258+0000 7f3753fff640 1 -- 172.21.15.53:0/2329183417 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246740 (secure 0 0 0) 0x7f376c06d3d0 con 0x7f37580762d0 2024-05-10T10:37:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:22.265 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:37:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f37580762d0 msgr2=0x7f3758078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f37580762d0 0x7f3758078790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f375c001780 tx=0x7f375c009040 comp rx=0 tx=0).stop 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f376c0d3450 msgr2=0x7f376c09d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f376c0d3450 0x7f376c09d0c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f376000e930 tx=0x7f376000edf0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 shutdown_connections 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f37580762d0 0x7f3758078790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f376c15f290 0x7f376c09d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f376c0d3450 0x7f376c09d0c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 --2- 172.21.15.53:0/2329183417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f376c0702e0 0x7f376c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 >> 172.21.15.53:0/2329183417 conn(0x7f376c09ba20 msgr2=0x7f376c09be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 shutdown_connections 2024-05-10T10:37:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.261+0000 7f3771d90640 1 -- 172.21.15.53:0/2329183417 wait complete. 2024-05-10T10:37:22.273 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-10T10:37:22.211124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240884,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733836,"statfs":{"total":773094113280,"available":772847448064,"internally_reserved":0,"allocated":8232960,"data_stored":4349152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001535"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214356+0000","last_change":"2024-05-10T10:37:17.214356+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214356+0000","last_undegraded":"2024-05-10T10:37:17.214356+0000","last_fullsized":"2024-05-10T10:37:17.214356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218881+0000","last_change":"2024-05-10T10:37:17.218881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218881+0000","last_undegraded":"2024-05-10T10:37:17.218881+0000","last_fullsized":"2024-05-10T10:37:17.218881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219561+0000","last_change":"2024-05-10T10:37:17.219561+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219561+0000","last_undegraded":"2024-05-10T10:37:17.219561+0000","last_fullsized":"2024-05-10T10:37:17.219561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220528+0000","last_change":"2024-05-10T10:37:17.220528+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220528+0000","last_undegraded":"2024-05-10T10:37:17.220528+0000","last_fullsized":"2024-05-10T10:37:17.220528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217963+0000","last_change":"2024-05-10T10:37:17.217963+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217963+0000","last_undegraded":"2024-05-10T10:37:17.217963+0000","last_fullsized":"2024-05-10T10:37:17.217963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220201+0000","last_change":"2024-05-10T10:37:17.220201+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220201+0000","last_undegraded":"2024-05-10T10:37:17.220201+0000","last_fullsized":"2024-05-10T10:37:17.220201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222149+0000","last_change":"2024-05-10T10:37:17.222149+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222149+0000","last_undegraded":"2024-05-10T10:37:17.222149+0000","last_fullsized":"2024-05-10T10:37:17.222149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215093+0000","last_change":"2024-05-10T10:37:17.215093+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215093+0000","last_undegraded":"2024-05-10T10:37:17.215093+0000","last_fullsized":"2024-05-10T10:37:17.215093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216792+0000","last_change":"2024-05-10T10:37:17.216792+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216792+0000","last_undegraded":"2024-05-10T10:37:17.216792+0000","last_fullsized":"2024-05-10T10:37:17.216792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218788+0000","last_change":"2024-05-10T10:37:17.218788+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218788+0000","last_undegraded":"2024-05-10T10:37:17.218788+0000","last_fullsized":"2024-05-10T10:37:17.218788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220858+0000","last_change":"2024-05-10T10:37:17.220858+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220858+0000","last_undegraded":"2024-05-10T10:37:17.220858+0000","last_fullsized":"2024-05-10T10:37:17.220858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218915+0000","last_change":"2024-05-10T10:37:17.218915+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218915+0000","last_undegraded":"2024-05-10T10:37:17.218915+0000","last_fullsized":"2024-05-10T10:37:17.218915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216194+0000","last_change":"2024-05-10T10:37:17.216194+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216194+0000","last_undegraded":"2024-05-10T10:37:17.216194+0000","last_fullsized":"2024-05-10T10:37:17.216194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219010+0000","last_change":"2024-05-10T10:37:17.219010+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219010+0000","last_undegraded":"2024-05-10T10:37:17.219010+0000","last_fullsized":"2024-05-10T10:37:17.219010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217797+0000","last_change":"2024-05-10T10:37:17.217797+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217797+0000","last_undegraded":"2024-05-10T10:37:17.217797+0000","last_fullsized":"2024-05-10T10:37:17.217797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215236+0000","last_change":"2024-05-10T10:37:17.215236+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215236+0000","last_undegraded":"2024-05-10T10:37:17.215236+0000","last_fullsized":"2024-05-10T10:37:17.215236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217685+0000","last_change":"2024-05-10T10:37:17.217685+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217685+0000","last_undegraded":"2024-05-10T10:37:17.217685+0000","last_fullsized":"2024-05-10T10:37:17.217685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220781+0000","last_change":"2024-05-10T10:37:17.220781+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220781+0000","last_undegraded":"2024-05-10T10:37:17.220781+0000","last_fullsized":"2024-05-10T10:37:17.220781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215713+0000","last_change":"2024-05-10T10:37:17.215713+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215713+0000","last_undegraded":"2024-05-10T10:37:17.215713+0000","last_fullsized":"2024-05-10T10:37:17.215713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217141+0000","last_change":"2024-05-10T10:37:17.217141+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217141+0000","last_undegraded":"2024-05-10T10:37:17.217141+0000","last_fullsized":"2024-05-10T10:37:17.217141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217274+0000","last_change":"2024-05-10T10:37:17.217274+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217274+0000","last_undegraded":"2024-05-10T10:37:17.217274+0000","last_fullsized":"2024-05-10T10:37:17.217274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215881+0000","last_change":"2024-05-10T10:37:17.215881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215881+0000","last_undegraded":"2024-05-10T10:37:17.215881+0000","last_fullsized":"2024-05-10T10:37:17.215881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218122+0000","last_change":"2024-05-10T10:37:17.218122+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218122+0000","last_undegraded":"2024-05-10T10:37:17.218122+0000","last_fullsized":"2024-05-10T10:37:17.218122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217328+0000","last_change":"2024-05-10T10:37:17.217328+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217328+0000","last_undegraded":"2024-05-10T10:37:17.217328+0000","last_fullsized":"2024-05-10T10:37:17.217328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216827+0000","last_change":"2024-05-10T10:37:17.216827+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216827+0000","last_undegraded":"2024-05-10T10:37:17.216827+0000","last_fullsized":"2024-05-10T10:37:17.216827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216132+0000","last_change":"2024-05-10T10:37:17.216132+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216132+0000","last_undegraded":"2024-05-10T10:37:17.216132+0000","last_fullsized":"2024-05-10T10:37:17.216132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214571+0000","last_change":"2024-05-10T10:37:17.214571+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214571+0000","last_undegraded":"2024-05-10T10:37:17.214571+0000","last_fullsized":"2024-05-10T10:37:17.214571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215835+0000","last_change":"2024-05-10T10:37:17.215835+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215835+0000","last_undegraded":"2024-05-10T10:37:17.215835+0000","last_fullsized":"2024-05-10T10:37:17.215835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216848+0000","last_change":"2024-05-10T10:37:17.216848+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216848+0000","last_undegraded":"2024-05-10T10:37:17.216848+0000","last_fullsized":"2024-05-10T10:37:17.216848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213937+0000","last_change":"2024-05-10T10:37:17.213937+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213937+0000","last_undegraded":"2024-05-10T10:37:17.213937+0000","last_fullsized":"2024-05-10T10:37:17.213937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217597+0000","last_change":"2024-05-10T10:37:17.217597+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217597+0000","last_undegraded":"2024-05-10T10:37:17.217597+0000","last_fullsized":"2024-05-10T10:37:17.217597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214590+0000","last_change":"2024-05-10T10:37:17.214590+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214590+0000","last_undegraded":"2024-05-10T10:37:17.214590+0000","last_fullsized":"2024-05-10T10:37:17.214590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218472+0000","last_change":"2024-05-10T10:37:17.218472+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218472+0000","last_undegraded":"2024-05-10T10:37:17.218472+0000","last_fullsized":"2024-05-10T10:37:17.218472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234893+0000","last_change":"2024-05-10T10:37:17.234091+0000","last_active":"2024-05-10T10:37:17.234893+0000","last_peered":"2024-05-10T10:37:17.234893+0000","last_clean":"2024-05-10T10:37:17.234893+0000","last_became_active":"2024-05-10T10:37:17.233792+0000","last_became_peered":"2024-05-10T10:37:17.233792+0000","last_unstale":"2024-05-10T10:37:17.234893+0000","last_undegraded":"2024-05-10T10:37:17.234893+0000","last_fullsized":"2024-05-10T10:37:17.234893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:15:16.625576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214546+0000","last_change":"2024-05-10T10:37:17.214546+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214546+0000","last_undegraded":"2024-05-10T10:37:17.214546+0000","last_fullsized":"2024-05-10T10:37:17.214546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233815+0000","last_change":"2024-05-10T10:37:17.229429+0000","last_active":"2024-05-10T10:37:17.233815+0000","last_peered":"2024-05-10T10:37:17.233815+0000","last_clean":"2024-05-10T10:37:17.233815+0000","last_became_active":"2024-05-10T10:37:17.229121+0000","last_became_peered":"2024-05-10T10:37:17.229121+0000","last_unstale":"2024-05-10T10:37:17.233815+0000","last_undegraded":"2024-05-10T10:37:17.233815+0000","last_fullsized":"2024-05-10T10:37:17.233815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:19.839294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218554+0000","last_change":"2024-05-10T10:37:17.218554+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218554+0000","last_undegraded":"2024-05-10T10:37:17.218554+0000","last_fullsized":"2024-05-10T10:37:17.218554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229641+0000","last_change":"2024-05-10T10:37:17.225715+0000","last_active":"2024-05-10T10:37:17.229641+0000","last_peered":"2024-05-10T10:37:17.229641+0000","last_clean":"2024-05-10T10:37:17.229641+0000","last_became_active":"2024-05-10T10:37:17.225250+0000","last_became_peered":"2024-05-10T10:37:17.225250+0000","last_unstale":"2024-05-10T10:37:17.229641+0000","last_undegraded":"2024-05-10T10:37:17.229641+0000","last_fullsized":"2024-05-10T10:37:17.229641+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:39.904069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216317+0000","last_change":"2024-05-10T10:37:17.216317+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216317+0000","last_undegraded":"2024-05-10T10:37:17.216317+0000","last_fullsized":"2024-05-10T10:37:17.216317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231454+0000","last_change":"2024-05-10T10:37:17.228203+0000","last_active":"2024-05-10T10:37:17.231454+0000","last_peered":"2024-05-10T10:37:17.231454+0000","last_clean":"2024-05-10T10:37:17.231454+0000","last_became_active":"2024-05-10T10:37:17.227939+0000","last_became_peered":"2024-05-10T10:37:17.227939+0000","last_unstale":"2024-05-10T10:37:17.231454+0000","last_undegraded":"2024-05-10T10:37:17.231454+0000","last_fullsized":"2024-05-10T10:37:17.231454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:19.500154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215764+0000","last_change":"2024-05-10T10:37:17.215764+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215764+0000","last_undegraded":"2024-05-10T10:37:17.215764+0000","last_fullsized":"2024-05-10T10:37:17.215764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.225465+0000","last_change":"2024-05-10T10:37:17.222699+0000","last_active":"2024-05-10T10:37:17.225465+0000","last_peered":"2024-05-10T10:37:17.225465+0000","last_clean":"2024-05-10T10:37:17.225465+0000","last_became_active":"2024-05-10T10:37:17.222389+0000","last_became_peered":"2024-05-10T10:37:17.222389+0000","last_unstale":"2024-05-10T10:37:17.225465+0000","last_undegraded":"2024-05-10T10:37:17.225465+0000","last_fullsized":"2024-05-10T10:37:17.225465+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:59:22.414675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215567+0000","last_change":"2024-05-10T10:37:17.215567+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215567+0000","last_undegraded":"2024-05-10T10:37:17.215567+0000","last_fullsized":"2024-05-10T10:37:17.215567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231160+0000","last_change":"2024-05-10T10:37:17.227524+0000","last_active":"2024-05-10T10:37:17.231160+0000","last_peered":"2024-05-10T10:37:17.231160+0000","last_clean":"2024-05-10T10:37:17.231160+0000","last_became_active":"2024-05-10T10:37:17.227256+0000","last_became_peered":"2024-05-10T10:37:17.227256+0000","last_unstale":"2024-05-10T10:37:17.231160+0000","last_undegraded":"2024-05-10T10:37:17.231160+0000","last_fullsized":"2024-05-10T10:37:17.231160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217227+0000","last_change":"2024-05-10T10:37:17.217227+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217227+0000","last_undegraded":"2024-05-10T10:37:17.217227+0000","last_fullsized":"2024-05-10T10:37:17.217227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.222393+0000","last_change":"2024-05-10T10:37:17.218206+0000","last_active":"2024-05-10T10:37:17.222393+0000","last_peered":"2024-05-10T10:37:17.222393+0000","last_clean":"2024-05-10T10:37:17.222393+0000","last_became_active":"2024-05-10T10:37:17.217591+0000","last_became_peered":"2024-05-10T10:37:17.217591+0000","last_unstale":"2024-05-10T10:37:17.222393+0000","last_undegraded":"2024-05-10T10:37:17.222393+0000","last_fullsized":"2024-05-10T10:37:17.222393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:15:45.683363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223820+0000","last_change":"2024-05-10T10:37:17.223820+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223820+0000","last_undegraded":"2024-05-10T10:37:17.223820+0000","last_fullsized":"2024-05-10T10:37:17.223820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231519+0000","last_change":"2024-05-10T10:37:17.230114+0000","last_active":"2024-05-10T10:37:17.231519+0000","last_peered":"2024-05-10T10:37:17.231519+0000","last_clean":"2024-05-10T10:37:17.231519+0000","last_became_active":"2024-05-10T10:37:17.229847+0000","last_became_peered":"2024-05-10T10:37:17.229847+0000","last_unstale":"2024-05-10T10:37:17.231519+0000","last_undegraded":"2024-05-10T10:37:17.231519+0000","last_fullsized":"2024-05-10T10:37:17.231519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:47:31.740584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214560+0000","last_change":"2024-05-10T10:37:17.214560+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214560+0000","last_undegraded":"2024-05-10T10:37:17.214560+0000","last_fullsized":"2024-05-10T10:37:17.214560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227281+0000","last_change":"2024-05-10T10:37:17.224805+0000","last_active":"2024-05-10T10:37:17.227281+0000","last_peered":"2024-05-10T10:37:17.227281+0000","last_clean":"2024-05-10T10:37:17.227281+0000","last_became_active":"2024-05-10T10:37:17.224508+0000","last_became_peered":"2024-05-10T10:37:17.224508+0000","last_unstale":"2024-05-10T10:37:17.227281+0000","last_undegraded":"2024-05-10T10:37:17.227281+0000","last_fullsized":"2024-05-10T10:37:17.227281+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215515+0000","last_change":"2024-05-10T10:37:17.215515+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215515+0000","last_undegraded":"2024-05-10T10:37:17.215515+0000","last_fullsized":"2024-05-10T10:37:17.215515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228363+0000","last_change":"2024-05-10T10:37:17.225567+0000","last_active":"2024-05-10T10:37:17.228363+0000","last_peered":"2024-05-10T10:37:17.228363+0000","last_clean":"2024-05-10T10:37:17.228363+0000","last_became_active":"2024-05-10T10:37:17.225301+0000","last_became_peered":"2024-05-10T10:37:17.225301+0000","last_unstale":"2024-05-10T10:37:17.228363+0000","last_undegraded":"2024-05-10T10:37:17.228363+0000","last_fullsized":"2024-05-10T10:37:17.228363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216485+0000","last_change":"2024-05-10T10:37:17.216485+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216485+0000","last_undegraded":"2024-05-10T10:37:17.216485+0000","last_fullsized":"2024-05-10T10:37:17.216485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.232470+0000","last_change":"2024-05-10T10:37:17.231914+0000","last_active":"2024-05-10T10:37:17.232470+0000","last_peered":"2024-05-10T10:37:17.232470+0000","last_clean":"2024-05-10T10:37:17.232470+0000","last_became_active":"2024-05-10T10:37:17.231631+0000","last_became_peered":"2024-05-10T10:37:17.231631+0000","last_unstale":"2024-05-10T10:37:17.232470+0000","last_undegraded":"2024-05-10T10:37:17.232470+0000","last_fullsized":"2024-05-10T10:37:17.232470+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:01:34.692557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220258+0000","last_change":"2024-05-10T10:37:17.220258+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220258+0000","last_undegraded":"2024-05-10T10:37:17.220258+0000","last_fullsized":"2024-05-10T10:37:17.220258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233970+0000","last_change":"2024-05-10T10:37:17.229898+0000","last_active":"2024-05-10T10:37:17.233970+0000","last_peered":"2024-05-10T10:37:17.233970+0000","last_clean":"2024-05-10T10:37:17.233970+0000","last_became_active":"2024-05-10T10:37:17.229599+0000","last_became_peered":"2024-05-10T10:37:17.229599+0000","last_unstale":"2024-05-10T10:37:17.233970+0000","last_undegraded":"2024-05-10T10:37:17.233970+0000","last_fullsized":"2024-05-10T10:37:17.233970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:30.149365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215708+0000","last_change":"2024-05-10T10:37:17.215708+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215708+0000","last_undegraded":"2024-05-10T10:37:17.215708+0000","last_fullsized":"2024-05-10T10:37:17.215708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228415+0000","last_change":"2024-05-10T10:37:17.223996+0000","last_active":"2024-05-10T10:37:17.228415+0000","last_peered":"2024-05-10T10:37:17.228415+0000","last_clean":"2024-05-10T10:37:17.228415+0000","last_became_active":"2024-05-10T10:37:17.223309+0000","last_became_peered":"2024-05-10T10:37:17.223309+0000","last_unstale":"2024-05-10T10:37:17.228415+0000","last_undegraded":"2024-05-10T10:37:17.228415+0000","last_fullsized":"2024-05-10T10:37:17.228415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:11.520143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223632+0000","last_change":"2024-05-10T10:37:17.223632+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223632+0000","last_undegraded":"2024-05-10T10:37:17.223632+0000","last_fullsized":"2024-05-10T10:37:17.223632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229437+0000","last_change":"2024-05-10T10:37:17.226617+0000","last_active":"2024-05-10T10:37:17.229437+0000","last_peered":"2024-05-10T10:37:17.229437+0000","last_clean":"2024-05-10T10:37:17.229437+0000","last_became_active":"2024-05-10T10:37:17.226339+0000","last_became_peered":"2024-05-10T10:37:17.226339+0000","last_unstale":"2024-05-10T10:37:17.229437+0000","last_undegraded":"2024-05-10T10:37:17.229437+0000","last_fullsized":"2024-05-10T10:37:17.229437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:36:44.271136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223403+0000","last_change":"2024-05-10T10:37:17.223403+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223403+0000","last_undegraded":"2024-05-10T10:37:17.223403+0000","last_fullsized":"2024-05-10T10:37:17.223403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234908+0000","last_change":"2024-05-10T10:37:17.234183+0000","last_active":"2024-05-10T10:37:17.234908+0000","last_peered":"2024-05-10T10:37:17.234908+0000","last_clean":"2024-05-10T10:37:17.234908+0000","last_became_active":"2024-05-10T10:37:17.233852+0000","last_became_peered":"2024-05-10T10:37:17.233852+0000","last_unstale":"2024-05-10T10:37:17.234908+0000","last_undegraded":"2024-05-10T10:37:17.234908+0000","last_fullsized":"2024-05-10T10:37:17.234908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:57:04.326406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222318+0000","last_change":"2024-05-10T10:37:17.222318+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222318+0000","last_undegraded":"2024-05-10T10:37:17.222318+0000","last_fullsized":"2024-05-10T10:37:17.222318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227850+0000","last_change":"2024-05-10T10:37:17.224923+0000","last_active":"2024-05-10T10:37:17.227850+0000","last_peered":"2024-05-10T10:37:17.227850+0000","last_clean":"2024-05-10T10:37:17.227850+0000","last_became_active":"2024-05-10T10:37:17.224594+0000","last_became_peered":"2024-05-10T10:37:17.224594+0000","last_unstale":"2024-05-10T10:37:17.227850+0000","last_undegraded":"2024-05-10T10:37:17.227850+0000","last_fullsized":"2024-05-10T10:37:17.227850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:58:38.099023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214436+0000","last_change":"2024-05-10T10:37:17.214436+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214436+0000","last_undegraded":"2024-05-10T10:37:17.214436+0000","last_fullsized":"2024-05-10T10:37:17.214436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229429+0000","last_change":"2024-05-10T10:37:17.225719+0000","last_active":"2024-05-10T10:37:17.229429+0000","last_peered":"2024-05-10T10:37:17.229429+0000","last_clean":"2024-05-10T10:37:17.229429+0000","last_became_active":"2024-05-10T10:37:17.225430+0000","last_became_peered":"2024-05-10T10:37:17.225430+0000","last_unstale":"2024-05-10T10:37:17.229429+0000","last_undegraded":"2024-05-10T10:37:17.229429+0000","last_fullsized":"2024-05-10T10:37:17.229429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:09.884646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217568+0000","last_change":"2024-05-10T10:37:17.217568+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217568+0000","last_undegraded":"2024-05-10T10:37:17.217568+0000","last_fullsized":"2024-05-10T10:37:17.217568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228794+0000","last_change":"2024-05-10T10:37:17.226246+0000","last_active":"2024-05-10T10:37:17.228794+0000","last_peered":"2024-05-10T10:37:17.228794+0000","last_clean":"2024-05-10T10:37:17.228794+0000","last_became_active":"2024-05-10T10:37:17.225996+0000","last_became_peered":"2024-05-10T10:37:17.225996+0000","last_unstale":"2024-05-10T10:37:17.228794+0000","last_undegraded":"2024-05-10T10:37:17.228794+0000","last_fullsized":"2024-05-10T10:37:17.228794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:55:53.951642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216780+0000","last_change":"2024-05-10T10:37:17.216780+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216780+0000","last_undegraded":"2024-05-10T10:37:17.216780+0000","last_fullsized":"2024-05-10T10:37:17.216780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229357+0000","last_change":"2024-05-10T10:37:17.223948+0000","last_active":"2024-05-10T10:37:17.229357+0000","last_peered":"2024-05-10T10:37:17.229357+0000","last_clean":"2024-05-10T10:37:17.229357+0000","last_became_active":"2024-05-10T10:37:17.223293+0000","last_became_peered":"2024-05-10T10:37:17.223293+0000","last_unstale":"2024-05-10T10:37:17.229357+0000","last_undegraded":"2024-05-10T10:37:17.229357+0000","last_fullsized":"2024-05-10T10:37:17.229357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:08.802431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215262+0000","last_change":"2024-05-10T10:37:17.215262+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215262+0000","last_undegraded":"2024-05-10T10:37:17.215262+0000","last_fullsized":"2024-05-10T10:37:17.215262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234053+0000","last_change":"2024-05-10T10:37:17.232761+0000","last_active":"2024-05-10T10:37:17.234053+0000","last_peered":"2024-05-10T10:37:17.234053+0000","last_clean":"2024-05-10T10:37:17.234053+0000","last_became_active":"2024-05-10T10:37:17.232401+0000","last_became_peered":"2024-05-10T10:37:17.232401+0000","last_unstale":"2024-05-10T10:37:17.234053+0000","last_undegraded":"2024-05-10T10:37:17.234053+0000","last_fullsized":"2024-05-10T10:37:17.234053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:39:14.930472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214673+0000","last_change":"2024-05-10T10:37:17.214673+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214673+0000","last_undegraded":"2024-05-10T10:37:17.214673+0000","last_fullsized":"2024-05-10T10:37:17.214673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233192+0000","last_change":"2024-05-10T10:37:17.227272+0000","last_active":"2024-05-10T10:37:17.233192+0000","last_peered":"2024-05-10T10:37:17.233192+0000","last_clean":"2024-05-10T10:37:17.233192+0000","last_became_active":"2024-05-10T10:37:17.224267+0000","last_became_peered":"2024-05-10T10:37:17.224267+0000","last_unstale":"2024-05-10T10:37:17.233192+0000","last_undegraded":"2024-05-10T10:37:17.233192+0000","last_fullsized":"2024-05-10T10:37:17.233192+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:23:59.926354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215169+0000","last_change":"2024-05-10T10:37:17.215169+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215169+0000","last_undegraded":"2024-05-10T10:37:17.215169+0000","last_fullsized":"2024-05-10T10:37:17.215169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233428+0000","last_change":"2024-05-10T10:37:17.230132+0000","last_active":"2024-05-10T10:37:17.233428+0000","last_peered":"2024-05-10T10:37:17.233428+0000","last_clean":"2024-05-10T10:37:17.233428+0000","last_became_active":"2024-05-10T10:37:17.229845+0000","last_became_peered":"2024-05-10T10:37:17.229845+0000","last_unstale":"2024-05-10T10:37:17.233428+0000","last_undegraded":"2024-05-10T10:37:17.233428+0000","last_fullsized":"2024-05-10T10:37:17.233428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:16:39.529753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221900+0000","last_change":"2024-05-10T10:37:17.221900+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221900+0000","last_undegraded":"2024-05-10T10:37:17.221900+0000","last_fullsized":"2024-05-10T10:37:17.221900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233394+0000","last_change":"2024-05-10T10:37:17.229451+0000","last_active":"2024-05-10T10:37:17.233394+0000","last_peered":"2024-05-10T10:37:17.233394+0000","last_clean":"2024-05-10T10:37:17.233394+0000","last_became_active":"2024-05-10T10:37:17.228407+0000","last_became_peered":"2024-05-10T10:37:17.228407+0000","last_unstale":"2024-05-10T10:37:17.233394+0000","last_undegraded":"2024-05-10T10:37:17.233394+0000","last_fullsized":"2024-05-10T10:37:17.233394+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:43:25.143088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216340+0000","last_change":"2024-05-10T10:37:17.216340+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216340+0000","last_undegraded":"2024-05-10T10:37:17.216340+0000","last_fullsized":"2024-05-10T10:37:17.216340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228381+0000","last_change":"2024-05-10T10:37:17.224792+0000","last_active":"2024-05-10T10:37:17.228381+0000","last_peered":"2024-05-10T10:37:17.228381+0000","last_clean":"2024-05-10T10:37:17.228381+0000","last_became_active":"2024-05-10T10:37:17.224443+0000","last_became_peered":"2024-05-10T10:37:17.224443+0000","last_unstale":"2024-05-10T10:37:17.228381+0000","last_undegraded":"2024-05-10T10:37:17.228381+0000","last_fullsized":"2024-05-10T10:37:17.228381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:13.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221621+0000","last_change":"2024-05-10T10:37:17.221621+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221621+0000","last_undegraded":"2024-05-10T10:37:17.221621+0000","last_fullsized":"2024-05-10T10:37:17.221621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.230785+0000","last_change":"2024-05-10T10:37:17.230099+0000","last_active":"2024-05-10T10:37:17.230785+0000","last_peered":"2024-05-10T10:37:17.230785+0000","last_clean":"2024-05-10T10:37:17.230785+0000","last_became_active":"2024-05-10T10:37:17.229704+0000","last_became_peered":"2024-05-10T10:37:17.229704+0000","last_unstale":"2024-05-10T10:37:17.230785+0000","last_undegraded":"2024-05-10T10:37:17.230785+0000","last_fullsized":"2024-05-10T10:37:17.230785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:43:08.567175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218157+0000","last_change":"2024-05-10T10:37:17.218157+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218157+0000","last_undegraded":"2024-05-10T10:37:17.218157+0000","last_fullsized":"2024-05-10T10:37:17.218157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229670+0000","last_change":"2024-05-10T10:37:17.226288+0000","last_active":"2024-05-10T10:37:17.229670+0000","last_peered":"2024-05-10T10:37:17.229670+0000","last_clean":"2024-05-10T10:37:17.229670+0000","last_became_active":"2024-05-10T10:37:17.226017+0000","last_became_peered":"2024-05-10T10:37:17.226017+0000","last_unstale":"2024-05-10T10:37:17.229670+0000","last_undegraded":"2024-05-10T10:37:17.229670+0000","last_fullsized":"2024-05-10T10:37:17.229670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:50.410699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214140+0000","last_change":"2024-05-10T10:37:17.214140+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214140+0000","last_undegraded":"2024-05-10T10:37:17.214140+0000","last_fullsized":"2024-05-10T10:37:17.214140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229395+0000","last_change":"2024-05-10T10:37:17.224891+0000","last_active":"2024-05-10T10:37:17.229395+0000","last_peered":"2024-05-10T10:37:17.229395+0000","last_clean":"2024-05-10T10:37:17.229395+0000","last_became_active":"2024-05-10T10:37:17.224639+0000","last_became_peered":"2024-05-10T10:37:17.224639+0000","last_unstale":"2024-05-10T10:37:17.229395+0000","last_undegraded":"2024-05-10T10:37:17.229395+0000","last_fullsized":"2024-05-10T10:37:17.229395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222142+0000","last_change":"2024-05-10T10:37:17.222142+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222142+0000","last_undegraded":"2024-05-10T10:37:17.222142+0000","last_fullsized":"2024-05-10T10:37:17.222142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231471+0000","last_change":"2024-05-10T10:37:17.228371+0000","last_active":"2024-05-10T10:37:17.231471+0000","last_peered":"2024-05-10T10:37:17.231471+0000","last_clean":"2024-05-10T10:37:17.231471+0000","last_became_active":"2024-05-10T10:37:17.227960+0000","last_became_peered":"2024-05-10T10:37:17.227960+0000","last_unstale":"2024-05-10T10:37:17.231471+0000","last_undegraded":"2024-05-10T10:37:17.231471+0000","last_fullsized":"2024-05-10T10:37:17.231471+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:19.319484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222569+0000","last_change":"2024-05-10T10:37:17.222569+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222569+0000","last_undegraded":"2024-05-10T10:37:17.222569+0000","last_fullsized":"2024-05-10T10:37:17.222569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229319+0000","last_change":"2024-05-10T10:37:17.223541+0000","last_active":"2024-05-10T10:37:17.229319+0000","last_peered":"2024-05-10T10:37:17.229319+0000","last_clean":"2024-05-10T10:37:17.229319+0000","last_became_active":"2024-05-10T10:37:17.223228+0000","last_became_peered":"2024-05-10T10:37:17.223228+0000","last_unstale":"2024-05-10T10:37:17.229319+0000","last_undegraded":"2024-05-10T10:37:17.229319+0000","last_fullsized":"2024-05-10T10:37:17.229319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:14.320723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220334+0000","last_change":"2024-05-10T10:37:17.220334+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220334+0000","last_undegraded":"2024-05-10T10:37:17.220334+0000","last_fullsized":"2024-05-10T10:37:17.220334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.236807+0000","last_change":"2024-05-10T10:37:17.232666+0000","last_active":"2024-05-10T10:37:17.236807+0000","last_peered":"2024-05-10T10:37:17.236807+0000","last_clean":"2024-05-10T10:37:17.236807+0000","last_became_active":"2024-05-10T10:37:17.232429+0000","last_became_peered":"2024-05-10T10:37:17.232429+0000","last_unstale":"2024-05-10T10:37:17.236807+0000","last_undegraded":"2024-05-10T10:37:17.236807+0000","last_fullsized":"2024-05-10T10:37:17.236807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:31.641499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220751+0000","last_change":"2024-05-10T10:37:17.220751+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220751+0000","last_undegraded":"2024-05-10T10:37:17.220751+0000","last_fullsized":"2024-05-10T10:37:17.220751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228619+0000","last_change":"2024-05-10T10:37:17.227889+0000","last_active":"2024-05-10T10:37:17.228619+0000","last_peered":"2024-05-10T10:37:17.228619+0000","last_clean":"2024-05-10T10:37:17.228619+0000","last_became_active":"2024-05-10T10:37:17.227312+0000","last_became_peered":"2024-05-10T10:37:17.227312+0000","last_unstale":"2024-05-10T10:37:17.228619+0000","last_undegraded":"2024-05-10T10:37:17.228619+0000","last_fullsized":"2024-05-10T10:37:17.228619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:37.508242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213972+0000","last_change":"2024-05-10T10:37:17.213972+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213972+0000","last_undegraded":"2024-05-10T10:37:17.213972+0000","last_fullsized":"2024-05-10T10:37:17.213972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233461+0000","last_change":"2024-05-10T10:37:17.230789+0000","last_active":"2024-05-10T10:37:17.233461+0000","last_peered":"2024-05-10T10:37:17.233461+0000","last_clean":"2024-05-10T10:37:17.233461+0000","last_became_active":"2024-05-10T10:37:17.230513+0000","last_became_peered":"2024-05-10T10:37:17.230513+0000","last_unstale":"2024-05-10T10:37:17.233461+0000","last_undegraded":"2024-05-10T10:37:17.233461+0000","last_fullsized":"2024-05-10T10:37:17.233461+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:15.368266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":215,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.219452+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:37:17.219452+0000","last_peered":"2024-05-10T10:37:17.219452+0000","last_clean":"2024-05-10T10:37:17.219452+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:37:17.219452+0000","last_undegraded":"2024-05-10T10:37:17.219452+0000","last_fullsized":"2024-05-10T10:37:17.219452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28464,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343376,"statfs":{"total":96636764160,"available":96607617024,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:37:22.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:37:22.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.416+0000 7f8f4032b640 1 Processor -- start 2024-05-10T10:37:22.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.416+0000 7f8f4032b640 1 -- start start 2024-05-10T10:37:22.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.417+0000 7f8f4032b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 0x7f8f380d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.417+0000 7f8f4032b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f38072330 con 0x7f8f3815d290 2024-05-10T10:37:22.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.417+0000 7f8f4032b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f38071d70 con 0x7f8f380d3340 2024-05-10T10:37:22.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.417+0000 7f8f4032b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8f380d37d0 con 0x7f8f380d1aa0 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3e0c7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8f3815d290 0x7f8f38071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49952/0 (socket says 172.21.15.53:49952) 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3e0c7640 1 -- 172.21.15.53:0/3288617839 learned_addr learned my addr 172.21.15.53:0/3288617839 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559620230 0 0) 0x7f8f38072330 con 0x7f8f3815d290 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d8c6640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 0x7f8f380d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f24003520 con 0x7f8f3815d290 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2918709131 0 0) 0x7f8f24003520 con 0x7f8f3815d290 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 msgr2=0x7f8f380d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 0x7f8f380d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 >> v1:172.21.15.135:6789/0 conn(0x7f8f380d3340 legacy=0x7f8f380ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f38072050 con 0x7f8f3815d290 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f8f34001670 con 0x7f8f3815d290 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 >> v1:172.21.15.53:6789/0 conn(0x7f8f3815d290 legacy=0x7f8f38071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f240039e0 0x7f8f24003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f24004800 0x7f8f24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 0x7f8f240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f380d37d0 con 0x7f8f24004800 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2400ba70 con 0x7f8f240039e0 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2400bc70 con 0x7f8f380d1aa0 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3d8c6640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f24004800 0x7f8f24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3e0c7640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f240039e0 0x7f8f24003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.418+0000 7f8f3e8c8640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 0x7f8f240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d8c6640 1 -- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 msgr2=0x7f8f240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d8c6640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f380d1aa0 0x7f8f240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d8c6640 1 -- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f240039e0 msgr2=0x7f8f24003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d8c6640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f240039e0 0x7f8f24003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d8c6640 1 -- 172.21.15.53:0/3288617839 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f24003520 con 0x7f8f24004800 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3e0c7640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f240039e0 0x7f8f24003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d8c6640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f24004800 0x7f8f24004c80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8f380d1f00 tx=0x7f8f24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e260ad336896e415 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28036070 con 0x7f8f24004800 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f28031400 con 0x7f8f24004800 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.419+0000 7f8f3d0c5640 1 -- 172.21.15.53:0/3288617839 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28009690 con 0x7f8f24004800 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.420+0000 7f8f4032b640 1 -- 172.21.15.53:0/3288617839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f24004800 msgr2=0x7f8f24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.420+0000 7f8f4032b640 1 --2- 172.21.15.53:0/3288617839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f24004800 0x7f8f24004c80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8f380d1f00 tx=0x7f8f24005830 comp rx=0 tx=0).stop 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.420+0000 7f8f3e8c8640 1 -- 172.21.15.53:0/3288617839 reap_dead start 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.420+0000 7f8f4032b640 1 -- 172.21.15.53:0/3288617839 shutdown_connections 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.420+0000 7f8f4032b640 1 -- 172.21.15.53:0/3288617839 >> 172.21.15.53:0/3288617839 conn(0x7f8f3809ba20 msgr2=0x7f8f3809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.420+0000 7f8f4032b640 1 -- 172.21.15.53:0/3288617839 shutdown_connections 2024-05-10T10:37:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.420+0000 7f8f4032b640 1 -- 172.21.15.53:0/3288617839 wait complete. 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.421+0000 7f8f4032b640 1 Processor -- start 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.421+0000 7f8f4032b640 1 -- start start 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f4032b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f24004800 0x7f8f381f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f4032b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3815c500 0x7f8f380ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f4032b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f3815f840 0x7f8f3815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f4032b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f38071a50 con 0x7f8f3815f840 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f4032b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f380cdd50 con 0x7f8f3815c500 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f4032b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8f380d2e00 con 0x7f8f24004800 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f3815f840 0x7f8f3815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f3815f840 0x7f8f3815fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37582/0 (socket says 172.21.15.53:37582) 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 -- 172.21.15.53:0/1603450969 learned_addr learned my addr 172.21.15.53:0/1603450969 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 -- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f24004800 msgr2=0x7f8f381f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f24004800 0x7f8f381f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 -- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3815c500 msgr2=0x7f8f380ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3815c500 0x7f8f380ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.422+0000 7f8f3e8c8640 1 -- 172.21.15.53:0/1603450969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f380c8150 con 0x7f8f3815f840 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.423+0000 7f8f3e8c8640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f3815f840 0x7f8f3815fcc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8f380d3f40 tx=0x7f8f3000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.423+0000 7f8f2effd640 1 -- 172.21.15.53:0/1603450969 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30003970 con 0x7f8f3815f840 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.423+0000 7f8f2effd640 1 -- 172.21.15.53:0/1603450969 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f30018070 con 0x7f8f3815f840 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.423+0000 7f8f2effd640 1 -- 172.21.15.53:0/1603450969 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30003d50 con 0x7f8f3815f840 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.423+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f380cf500 con 0x7f8f3815f840 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.423+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f380cf200 con 0x7f8f3815f840 2024-05-10T10:37:22.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.424+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3815c940 con 0x7f8f3815f840 2024-05-10T10:37:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.424+0000 7f8f2effd640 1 -- 172.21.15.53:0/1603450969 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8f300050d0 con 0x7f8f3815f840 2024-05-10T10:37:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.425+0000 7f8f2effd640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f1c076300 0x7f8f1c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.425+0000 7f8f2effd640 1 -- 172.21.15.53:0/1603450969 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f8f30097400 con 0x7f8f3815f840 2024-05-10T10:37:22.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.425+0000 7f8f3e0c7640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f1c076300 0x7f8f1c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.426+0000 7f8f3e0c7640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f1c076300 0x7f8f1c0787c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8f34001cd0 tx=0x7f8f34001b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.429+0000 7f8f2effd640 1 -- 172.21.15.53:0/1603450969 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8f30060bd0 con 0x7f8f3815f840 2024-05-10T10:37:22.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.621+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f380cd440 con 0x7f8f1c076300 2024-05-10T10:37:22.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.627+0000 7f8f2effd640 1 -- 172.21.15.53:0/1603450969 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246740 (secure 0 0 0) 0x7f8f3400e920 con 0x7f8f1c076300 2024-05-10T10:37:22.631 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:22.632 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f1c076300 msgr2=0x7f8f1c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f1c076300 0x7f8f1c0787c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8f34001cd0 tx=0x7f8f34001b60 comp rx=0 tx=0).stop 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f3815f840 msgr2=0x7f8f3815fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f3815f840 0x7f8f3815fcc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8f380d3f40 tx=0x7f8f3000eb90 comp rx=0 tx=0).stop 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 shutdown_connections 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8f1c076300 0x7f8f1c0787c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f3815f840 0x7f8f3815fcc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3815c500 0x7f8f380ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 --2- 172.21.15.53:0/1603450969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f24004800 0x7f8f381f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 >> 172.21.15.53:0/1603450969 conn(0x7f8f3809ba20 msgr2=0x7f8f380b7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 shutdown_connections 2024-05-10T10:37:22.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.631+0000 7f8f4032b640 1 -- 172.21.15.53:0/1603450969 wait complete. 2024-05-10T10:37:22.642 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-10T10:37:22.211124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240884,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733836,"statfs":{"total":773094113280,"available":772847448064,"internally_reserved":0,"allocated":8232960,"data_stored":4349152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001535"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214356+0000","last_change":"2024-05-10T10:37:17.214356+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214356+0000","last_undegraded":"2024-05-10T10:37:17.214356+0000","last_fullsized":"2024-05-10T10:37:17.214356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218881+0000","last_change":"2024-05-10T10:37:17.218881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218881+0000","last_undegraded":"2024-05-10T10:37:17.218881+0000","last_fullsized":"2024-05-10T10:37:17.218881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219561+0000","last_change":"2024-05-10T10:37:17.219561+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219561+0000","last_undegraded":"2024-05-10T10:37:17.219561+0000","last_fullsized":"2024-05-10T10:37:17.219561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220528+0000","last_change":"2024-05-10T10:37:17.220528+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220528+0000","last_undegraded":"2024-05-10T10:37:17.220528+0000","last_fullsized":"2024-05-10T10:37:17.220528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217963+0000","last_change":"2024-05-10T10:37:17.217963+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217963+0000","last_undegraded":"2024-05-10T10:37:17.217963+0000","last_fullsized":"2024-05-10T10:37:17.217963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220201+0000","last_change":"2024-05-10T10:37:17.220201+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220201+0000","last_undegraded":"2024-05-10T10:37:17.220201+0000","last_fullsized":"2024-05-10T10:37:17.220201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222149+0000","last_change":"2024-05-10T10:37:17.222149+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222149+0000","last_undegraded":"2024-05-10T10:37:17.222149+0000","last_fullsized":"2024-05-10T10:37:17.222149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215093+0000","last_change":"2024-05-10T10:37:17.215093+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215093+0000","last_undegraded":"2024-05-10T10:37:17.215093+0000","last_fullsized":"2024-05-10T10:37:17.215093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216792+0000","last_change":"2024-05-10T10:37:17.216792+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216792+0000","last_undegraded":"2024-05-10T10:37:17.216792+0000","last_fullsized":"2024-05-10T10:37:17.216792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218788+0000","last_change":"2024-05-10T10:37:17.218788+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218788+0000","last_undegraded":"2024-05-10T10:37:17.218788+0000","last_fullsized":"2024-05-10T10:37:17.218788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220858+0000","last_change":"2024-05-10T10:37:17.220858+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220858+0000","last_undegraded":"2024-05-10T10:37:17.220858+0000","last_fullsized":"2024-05-10T10:37:17.220858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218915+0000","last_change":"2024-05-10T10:37:17.218915+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218915+0000","last_undegraded":"2024-05-10T10:37:17.218915+0000","last_fullsized":"2024-05-10T10:37:17.218915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216194+0000","last_change":"2024-05-10T10:37:17.216194+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216194+0000","last_undegraded":"2024-05-10T10:37:17.216194+0000","last_fullsized":"2024-05-10T10:37:17.216194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219010+0000","last_change":"2024-05-10T10:37:17.219010+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219010+0000","last_undegraded":"2024-05-10T10:37:17.219010+0000","last_fullsized":"2024-05-10T10:37:17.219010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217797+0000","last_change":"2024-05-10T10:37:17.217797+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217797+0000","last_undegraded":"2024-05-10T10:37:17.217797+0000","last_fullsized":"2024-05-10T10:37:17.217797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215236+0000","last_change":"2024-05-10T10:37:17.215236+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215236+0000","last_undegraded":"2024-05-10T10:37:17.215236+0000","last_fullsized":"2024-05-10T10:37:17.215236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217685+0000","last_change":"2024-05-10T10:37:17.217685+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217685+0000","last_undegraded":"2024-05-10T10:37:17.217685+0000","last_fullsized":"2024-05-10T10:37:17.217685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220781+0000","last_change":"2024-05-10T10:37:17.220781+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220781+0000","last_undegraded":"2024-05-10T10:37:17.220781+0000","last_fullsized":"2024-05-10T10:37:17.220781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215713+0000","last_change":"2024-05-10T10:37:17.215713+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215713+0000","last_undegraded":"2024-05-10T10:37:17.215713+0000","last_fullsized":"2024-05-10T10:37:17.215713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217141+0000","last_change":"2024-05-10T10:37:17.217141+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217141+0000","last_undegraded":"2024-05-10T10:37:17.217141+0000","last_fullsized":"2024-05-10T10:37:17.217141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217274+0000","last_change":"2024-05-10T10:37:17.217274+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217274+0000","last_undegraded":"2024-05-10T10:37:17.217274+0000","last_fullsized":"2024-05-10T10:37:17.217274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215881+0000","last_change":"2024-05-10T10:37:17.215881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215881+0000","last_undegraded":"2024-05-10T10:37:17.215881+0000","last_fullsized":"2024-05-10T10:37:17.215881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218122+0000","last_change":"2024-05-10T10:37:17.218122+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218122+0000","last_undegraded":"2024-05-10T10:37:17.218122+0000","last_fullsized":"2024-05-10T10:37:17.218122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217328+0000","last_change":"2024-05-10T10:37:17.217328+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217328+0000","last_undegraded":"2024-05-10T10:37:17.217328+0000","last_fullsized":"2024-05-10T10:37:17.217328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216827+0000","last_change":"2024-05-10T10:37:17.216827+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216827+0000","last_undegraded":"2024-05-10T10:37:17.216827+0000","last_fullsized":"2024-05-10T10:37:17.216827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216132+0000","last_change":"2024-05-10T10:37:17.216132+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216132+0000","last_undegraded":"2024-05-10T10:37:17.216132+0000","last_fullsized":"2024-05-10T10:37:17.216132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214571+0000","last_change":"2024-05-10T10:37:17.214571+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214571+0000","last_undegraded":"2024-05-10T10:37:17.214571+0000","last_fullsized":"2024-05-10T10:37:17.214571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215835+0000","last_change":"2024-05-10T10:37:17.215835+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215835+0000","last_undegraded":"2024-05-10T10:37:17.215835+0000","last_fullsized":"2024-05-10T10:37:17.215835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216848+0000","last_change":"2024-05-10T10:37:17.216848+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216848+0000","last_undegraded":"2024-05-10T10:37:17.216848+0000","last_fullsized":"2024-05-10T10:37:17.216848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213937+0000","last_change":"2024-05-10T10:37:17.213937+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213937+0000","last_undegraded":"2024-05-10T10:37:17.213937+0000","last_fullsized":"2024-05-10T10:37:17.213937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217597+0000","last_change":"2024-05-10T10:37:17.217597+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217597+0000","last_undegraded":"2024-05-10T10:37:17.217597+0000","last_fullsized":"2024-05-10T10:37:17.217597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214590+0000","last_change":"2024-05-10T10:37:17.214590+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214590+0000","last_undegraded":"2024-05-10T10:37:17.214590+0000","last_fullsized":"2024-05-10T10:37:17.214590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218472+0000","last_change":"2024-05-10T10:37:17.218472+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218472+0000","last_undegraded":"2024-05-10T10:37:17.218472+0000","last_fullsized":"2024-05-10T10:37:17.218472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234893+0000","last_change":"2024-05-10T10:37:17.234091+0000","last_active":"2024-05-10T10:37:17.234893+0000","last_peered":"2024-05-10T10:37:17.234893+0000","last_clean":"2024-05-10T10:37:17.234893+0000","last_became_active":"2024-05-10T10:37:17.233792+0000","last_became_peered":"2024-05-10T10:37:17.233792+0000","last_unstale":"2024-05-10T10:37:17.234893+0000","last_undegraded":"2024-05-10T10:37:17.234893+0000","last_fullsized":"2024-05-10T10:37:17.234893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:15:16.625576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214546+0000","last_change":"2024-05-10T10:37:17.214546+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214546+0000","last_undegraded":"2024-05-10T10:37:17.214546+0000","last_fullsized":"2024-05-10T10:37:17.214546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233815+0000","last_change":"2024-05-10T10:37:17.229429+0000","last_active":"2024-05-10T10:37:17.233815+0000","last_peered":"2024-05-10T10:37:17.233815+0000","last_clean":"2024-05-10T10:37:17.233815+0000","last_became_active":"2024-05-10T10:37:17.229121+0000","last_became_peered":"2024-05-10T10:37:17.229121+0000","last_unstale":"2024-05-10T10:37:17.233815+0000","last_undegraded":"2024-05-10T10:37:17.233815+0000","last_fullsized":"2024-05-10T10:37:17.233815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:19.839294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218554+0000","last_change":"2024-05-10T10:37:17.218554+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218554+0000","last_undegraded":"2024-05-10T10:37:17.218554+0000","last_fullsized":"2024-05-10T10:37:17.218554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229641+0000","last_change":"2024-05-10T10:37:17.225715+0000","last_active":"2024-05-10T10:37:17.229641+0000","last_peered":"2024-05-10T10:37:17.229641+0000","last_clean":"2024-05-10T10:37:17.229641+0000","last_became_active":"2024-05-10T10:37:17.225250+0000","last_became_peered":"2024-05-10T10:37:17.225250+0000","last_unstale":"2024-05-10T10:37:17.229641+0000","last_undegraded":"2024-05-10T10:37:17.229641+0000","last_fullsized":"2024-05-10T10:37:17.229641+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:39.904069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216317+0000","last_change":"2024-05-10T10:37:17.216317+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216317+0000","last_undegraded":"2024-05-10T10:37:17.216317+0000","last_fullsized":"2024-05-10T10:37:17.216317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231454+0000","last_change":"2024-05-10T10:37:17.228203+0000","last_active":"2024-05-10T10:37:17.231454+0000","last_peered":"2024-05-10T10:37:17.231454+0000","last_clean":"2024-05-10T10:37:17.231454+0000","last_became_active":"2024-05-10T10:37:17.227939+0000","last_became_peered":"2024-05-10T10:37:17.227939+0000","last_unstale":"2024-05-10T10:37:17.231454+0000","last_undegraded":"2024-05-10T10:37:17.231454+0000","last_fullsized":"2024-05-10T10:37:17.231454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:19.500154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215764+0000","last_change":"2024-05-10T10:37:17.215764+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215764+0000","last_undegraded":"2024-05-10T10:37:17.215764+0000","last_fullsized":"2024-05-10T10:37:17.215764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.225465+0000","last_change":"2024-05-10T10:37:17.222699+0000","last_active":"2024-05-10T10:37:17.225465+0000","last_peered":"2024-05-10T10:37:17.225465+0000","last_clean":"2024-05-10T10:37:17.225465+0000","last_became_active":"2024-05-10T10:37:17.222389+0000","last_became_peered":"2024-05-10T10:37:17.222389+0000","last_unstale":"2024-05-10T10:37:17.225465+0000","last_undegraded":"2024-05-10T10:37:17.225465+0000","last_fullsized":"2024-05-10T10:37:17.225465+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:59:22.414675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215567+0000","last_change":"2024-05-10T10:37:17.215567+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215567+0000","last_undegraded":"2024-05-10T10:37:17.215567+0000","last_fullsized":"2024-05-10T10:37:17.215567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231160+0000","last_change":"2024-05-10T10:37:17.227524+0000","last_active":"2024-05-10T10:37:17.231160+0000","last_peered":"2024-05-10T10:37:17.231160+0000","last_clean":"2024-05-10T10:37:17.231160+0000","last_became_active":"2024-05-10T10:37:17.227256+0000","last_became_peered":"2024-05-10T10:37:17.227256+0000","last_unstale":"2024-05-10T10:37:17.231160+0000","last_undegraded":"2024-05-10T10:37:17.231160+0000","last_fullsized":"2024-05-10T10:37:17.231160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217227+0000","last_change":"2024-05-10T10:37:17.217227+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217227+0000","last_undegraded":"2024-05-10T10:37:17.217227+0000","last_fullsized":"2024-05-10T10:37:17.217227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.222393+0000","last_change":"2024-05-10T10:37:17.218206+0000","last_active":"2024-05-10T10:37:17.222393+0000","last_peered":"2024-05-10T10:37:17.222393+0000","last_clean":"2024-05-10T10:37:17.222393+0000","last_became_active":"2024-05-10T10:37:17.217591+0000","last_became_peered":"2024-05-10T10:37:17.217591+0000","last_unstale":"2024-05-10T10:37:17.222393+0000","last_undegraded":"2024-05-10T10:37:17.222393+0000","last_fullsized":"2024-05-10T10:37:17.222393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:15:45.683363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223820+0000","last_change":"2024-05-10T10:37:17.223820+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223820+0000","last_undegraded":"2024-05-10T10:37:17.223820+0000","last_fullsized":"2024-05-10T10:37:17.223820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231519+0000","last_change":"2024-05-10T10:37:17.230114+0000","last_active":"2024-05-10T10:37:17.231519+0000","last_peered":"2024-05-10T10:37:17.231519+0000","last_clean":"2024-05-10T10:37:17.231519+0000","last_became_active":"2024-05-10T10:37:17.229847+0000","last_became_peered":"2024-05-10T10:37:17.229847+0000","last_unstale":"2024-05-10T10:37:17.231519+0000","last_undegraded":"2024-05-10T10:37:17.231519+0000","last_fullsized":"2024-05-10T10:37:17.231519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:47:31.740584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214560+0000","last_change":"2024-05-10T10:37:17.214560+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214560+0000","last_undegraded":"2024-05-10T10:37:17.214560+0000","last_fullsized":"2024-05-10T10:37:17.214560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227281+0000","last_change":"2024-05-10T10:37:17.224805+0000","last_active":"2024-05-10T10:37:17.227281+0000","last_peered":"2024-05-10T10:37:17.227281+0000","last_clean":"2024-05-10T10:37:17.227281+0000","last_became_active":"2024-05-10T10:37:17.224508+0000","last_became_peered":"2024-05-10T10:37:17.224508+0000","last_unstale":"2024-05-10T10:37:17.227281+0000","last_undegraded":"2024-05-10T10:37:17.227281+0000","last_fullsized":"2024-05-10T10:37:17.227281+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215515+0000","last_change":"2024-05-10T10:37:17.215515+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215515+0000","last_undegraded":"2024-05-10T10:37:17.215515+0000","last_fullsized":"2024-05-10T10:37:17.215515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228363+0000","last_change":"2024-05-10T10:37:17.225567+0000","last_active":"2024-05-10T10:37:17.228363+0000","last_peered":"2024-05-10T10:37:17.228363+0000","last_clean":"2024-05-10T10:37:17.228363+0000","last_became_active":"2024-05-10T10:37:17.225301+0000","last_became_peered":"2024-05-10T10:37:17.225301+0000","last_unstale":"2024-05-10T10:37:17.228363+0000","last_undegraded":"2024-05-10T10:37:17.228363+0000","last_fullsized":"2024-05-10T10:37:17.228363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216485+0000","last_change":"2024-05-10T10:37:17.216485+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216485+0000","last_undegraded":"2024-05-10T10:37:17.216485+0000","last_fullsized":"2024-05-10T10:37:17.216485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.232470+0000","last_change":"2024-05-10T10:37:17.231914+0000","last_active":"2024-05-10T10:37:17.232470+0000","last_peered":"2024-05-10T10:37:17.232470+0000","last_clean":"2024-05-10T10:37:17.232470+0000","last_became_active":"2024-05-10T10:37:17.231631+0000","last_became_peered":"2024-05-10T10:37:17.231631+0000","last_unstale":"2024-05-10T10:37:17.232470+0000","last_undegraded":"2024-05-10T10:37:17.232470+0000","last_fullsized":"2024-05-10T10:37:17.232470+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:01:34.692557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220258+0000","last_change":"2024-05-10T10:37:17.220258+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220258+0000","last_undegraded":"2024-05-10T10:37:17.220258+0000","last_fullsized":"2024-05-10T10:37:17.220258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233970+0000","last_change":"2024-05-10T10:37:17.229898+0000","last_active":"2024-05-10T10:37:17.233970+0000","last_peered":"2024-05-10T10:37:17.233970+0000","last_clean":"2024-05-10T10:37:17.233970+0000","last_became_active":"2024-05-10T10:37:17.229599+0000","last_became_peered":"2024-05-10T10:37:17.229599+0000","last_unstale":"2024-05-10T10:37:17.233970+0000","last_undegraded":"2024-05-10T10:37:17.233970+0000","last_fullsized":"2024-05-10T10:37:17.233970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:30.149365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215708+0000","last_change":"2024-05-10T10:37:17.215708+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215708+0000","last_undegraded":"2024-05-10T10:37:17.215708+0000","last_fullsized":"2024-05-10T10:37:17.215708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228415+0000","last_change":"2024-05-10T10:37:17.223996+0000","last_active":"2024-05-10T10:37:17.228415+0000","last_peered":"2024-05-10T10:37:17.228415+0000","last_clean":"2024-05-10T10:37:17.228415+0000","last_became_active":"2024-05-10T10:37:17.223309+0000","last_became_peered":"2024-05-10T10:37:17.223309+0000","last_unstale":"2024-05-10T10:37:17.228415+0000","last_undegraded":"2024-05-10T10:37:17.228415+0000","last_fullsized":"2024-05-10T10:37:17.228415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:11.520143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223632+0000","last_change":"2024-05-10T10:37:17.223632+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223632+0000","last_undegraded":"2024-05-10T10:37:17.223632+0000","last_fullsized":"2024-05-10T10:37:17.223632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229437+0000","last_change":"2024-05-10T10:37:17.226617+0000","last_active":"2024-05-10T10:37:17.229437+0000","last_peered":"2024-05-10T10:37:17.229437+0000","last_clean":"2024-05-10T10:37:17.229437+0000","last_became_active":"2024-05-10T10:37:17.226339+0000","last_became_peered":"2024-05-10T10:37:17.226339+0000","last_unstale":"2024-05-10T10:37:17.229437+0000","last_undegraded":"2024-05-10T10:37:17.229437+0000","last_fullsized":"2024-05-10T10:37:17.229437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:36:44.271136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223403+0000","last_change":"2024-05-10T10:37:17.223403+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223403+0000","last_undegraded":"2024-05-10T10:37:17.223403+0000","last_fullsized":"2024-05-10T10:37:17.223403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234908+0000","last_change":"2024-05-10T10:37:17.234183+0000","last_active":"2024-05-10T10:37:17.234908+0000","last_peered":"2024-05-10T10:37:17.234908+0000","last_clean":"2024-05-10T10:37:17.234908+0000","last_became_active":"2024-05-10T10:37:17.233852+0000","last_became_peered":"2024-05-10T10:37:17.233852+0000","last_unstale":"2024-05-10T10:37:17.234908+0000","last_undegraded":"2024-05-10T10:37:17.234908+0000","last_fullsized":"2024-05-10T10:37:17.234908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:57:04.326406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222318+0000","last_change":"2024-05-10T10:37:17.222318+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222318+0000","last_undegraded":"2024-05-10T10:37:17.222318+0000","last_fullsized":"2024-05-10T10:37:17.222318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227850+0000","last_change":"2024-05-10T10:37:17.224923+0000","last_active":"2024-05-10T10:37:17.227850+0000","last_peered":"2024-05-10T10:37:17.227850+0000","last_clean":"2024-05-10T10:37:17.227850+0000","last_became_active":"2024-05-10T10:37:17.224594+0000","last_became_peered":"2024-05-10T10:37:17.224594+0000","last_unstale":"2024-05-10T10:37:17.227850+0000","last_undegraded":"2024-05-10T10:37:17.227850+0000","last_fullsized":"2024-05-10T10:37:17.227850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:58:38.099023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214436+0000","last_change":"2024-05-10T10:37:17.214436+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214436+0000","last_undegraded":"2024-05-10T10:37:17.214436+0000","last_fullsized":"2024-05-10T10:37:17.214436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229429+0000","last_change":"2024-05-10T10:37:17.225719+0000","last_active":"2024-05-10T10:37:17.229429+0000","last_peered":"2024-05-10T10:37:17.229429+0000","last_clean":"2024-05-10T10:37:17.229429+0000","last_became_active":"2024-05-10T10:37:17.225430+0000","last_became_peered":"2024-05-10T10:37:17.225430+0000","last_unstale":"2024-05-10T10:37:17.229429+0000","last_undegraded":"2024-05-10T10:37:17.229429+0000","last_fullsized":"2024-05-10T10:37:17.229429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:09.884646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217568+0000","last_change":"2024-05-10T10:37:17.217568+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217568+0000","last_undegraded":"2024-05-10T10:37:17.217568+0000","last_fullsized":"2024-05-10T10:37:17.217568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228794+0000","last_change":"2024-05-10T10:37:17.226246+0000","last_active":"2024-05-10T10:37:17.228794+0000","last_peered":"2024-05-10T10:37:17.228794+0000","last_clean":"2024-05-10T10:37:17.228794+0000","last_became_active":"2024-05-10T10:37:17.225996+0000","last_became_peered":"2024-05-10T10:37:17.225996+0000","last_unstale":"2024-05-10T10:37:17.228794+0000","last_undegraded":"2024-05-10T10:37:17.228794+0000","last_fullsized":"2024-05-10T10:37:17.228794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:55:53.951642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216780+0000","last_change":"2024-05-10T10:37:17.216780+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216780+0000","last_undegraded":"2024-05-10T10:37:17.216780+0000","last_fullsized":"2024-05-10T10:37:17.216780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229357+0000","last_change":"2024-05-10T10:37:17.223948+0000","last_active":"2024-05-10T10:37:17.229357+0000","last_peered":"2024-05-10T10:37:17.229357+0000","last_clean":"2024-05-10T10:37:17.229357+0000","last_became_active":"2024-05-10T10:37:17.223293+0000","last_became_peered":"2024-05-10T10:37:17.223293+0000","last_unstale":"2024-05-10T10:37:17.229357+0000","last_undegraded":"2024-05-10T10:37:17.229357+0000","last_fullsized":"2024-05-10T10:37:17.229357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:08.802431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215262+0000","last_change":"2024-05-10T10:37:17.215262+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215262+0000","last_undegraded":"2024-05-10T10:37:17.215262+0000","last_fullsized":"2024-05-10T10:37:17.215262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234053+0000","last_change":"2024-05-10T10:37:17.232761+0000","last_active":"2024-05-10T10:37:17.234053+0000","last_peered":"2024-05-10T10:37:17.234053+0000","last_clean":"2024-05-10T10:37:17.234053+0000","last_became_active":"2024-05-10T10:37:17.232401+0000","last_became_peered":"2024-05-10T10:37:17.232401+0000","last_unstale":"2024-05-10T10:37:17.234053+0000","last_undegraded":"2024-05-10T10:37:17.234053+0000","last_fullsized":"2024-05-10T10:37:17.234053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:39:14.930472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214673+0000","last_change":"2024-05-10T10:37:17.214673+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214673+0000","last_undegraded":"2024-05-10T10:37:17.214673+0000","last_fullsized":"2024-05-10T10:37:17.214673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233192+0000","last_change":"2024-05-10T10:37:17.227272+0000","last_active":"2024-05-10T10:37:17.233192+0000","last_peered":"2024-05-10T10:37:17.233192+0000","last_clean":"2024-05-10T10:37:17.233192+0000","last_became_active":"2024-05-10T10:37:17.224267+0000","last_became_peered":"2024-05-10T10:37:17.224267+0000","last_unstale":"2024-05-10T10:37:17.233192+0000","last_undegraded":"2024-05-10T10:37:17.233192+0000","last_fullsized":"2024-05-10T10:37:17.233192+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:23:59.926354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215169+0000","last_change":"2024-05-10T10:37:17.215169+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215169+0000","last_undegraded":"2024-05-10T10:37:17.215169+0000","last_fullsized":"2024-05-10T10:37:17.215169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233428+0000","last_change":"2024-05-10T10:37:17.230132+0000","last_active":"2024-05-10T10:37:17.233428+0000","last_peered":"2024-05-10T10:37:17.233428+0000","last_clean":"2024-05-10T10:37:17.233428+0000","last_became_active":"2024-05-10T10:37:17.229845+0000","last_became_peered":"2024-05-10T10:37:17.229845+0000","last_unstale":"2024-05-10T10:37:17.233428+0000","last_undegraded":"2024-05-10T10:37:17.233428+0000","last_fullsized":"2024-05-10T10:37:17.233428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:16:39.529753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221900+0000","last_change":"2024-05-10T10:37:17.221900+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221900+0000","last_undegraded":"2024-05-10T10:37:17.221900+0000","last_fullsized":"2024-05-10T10:37:17.221900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233394+0000","last_change":"2024-05-10T10:37:17.229451+0000","last_active":"2024-05-10T10:37:17.233394+0000","last_peered":"2024-05-10T10:37:17.233394+0000","last_clean":"2024-05-10T10:37:17.233394+0000","last_became_active":"2024-05-10T10:37:17.228407+0000","last_became_peered":"2024-05-10T10:37:17.228407+0000","last_unstale":"2024-05-10T10:37:17.233394+0000","last_undegraded":"2024-05-10T10:37:17.233394+0000","last_fullsized":"2024-05-10T10:37:17.233394+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:43:25.143088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216340+0000","last_change":"2024-05-10T10:37:17.216340+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216340+0000","last_undegraded":"2024-05-10T10:37:17.216340+0000","last_fullsized":"2024-05-10T10:37:17.216340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228381+0000","last_change":"2024-05-10T10:37:17.224792+0000","last_active":"2024-05-10T10:37:17.228381+0000","last_peered":"2024-05-10T10:37:17.228381+0000","last_clean":"2024-05-10T10:37:17.228381+0000","last_became_active":"2024-05-10T10:37:17.224443+0000","last_became_peered":"2024-05-10T10:37:17.224443+0000","last_unstale":"2024-05-10T10:37:17.228381+0000","last_undegraded":"2024-05-10T10:37:17.228381+0000","last_fullsized":"2024-05-10T10:37:17.228381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:13.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221621+0000","last_change":"2024-05-10T10:37:17.221621+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221621+0000","last_undegraded":"2024-05-10T10:37:17.221621+0000","last_fullsized":"2024-05-10T10:37:17.221621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.230785+0000","last_change":"2024-05-10T10:37:17.230099+0000","last_active":"2024-05-10T10:37:17.230785+0000","last_peered":"2024-05-10T10:37:17.230785+0000","last_clean":"2024-05-10T10:37:17.230785+0000","last_became_active":"2024-05-10T10:37:17.229704+0000","last_became_peered":"2024-05-10T10:37:17.229704+0000","last_unstale":"2024-05-10T10:37:17.230785+0000","last_undegraded":"2024-05-10T10:37:17.230785+0000","last_fullsized":"2024-05-10T10:37:17.230785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:43:08.567175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218157+0000","last_change":"2024-05-10T10:37:17.218157+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218157+0000","last_undegraded":"2024-05-10T10:37:17.218157+0000","last_fullsized":"2024-05-10T10:37:17.218157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229670+0000","last_change":"2024-05-10T10:37:17.226288+0000","last_active":"2024-05-10T10:37:17.229670+0000","last_peered":"2024-05-10T10:37:17.229670+0000","last_clean":"2024-05-10T10:37:17.229670+0000","last_became_active":"2024-05-10T10:37:17.226017+0000","last_became_peered":"2024-05-10T10:37:17.226017+0000","last_unstale":"2024-05-10T10:37:17.229670+0000","last_undegraded":"2024-05-10T10:37:17.229670+0000","last_fullsized":"2024-05-10T10:37:17.229670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:50.410699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214140+0000","last_change":"2024-05-10T10:37:17.214140+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214140+0000","last_undegraded":"2024-05-10T10:37:17.214140+0000","last_fullsized":"2024-05-10T10:37:17.214140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229395+0000","last_change":"2024-05-10T10:37:17.224891+0000","last_active":"2024-05-10T10:37:17.229395+0000","last_peered":"2024-05-10T10:37:17.229395+0000","last_clean":"2024-05-10T10:37:17.229395+0000","last_became_active":"2024-05-10T10:37:17.224639+0000","last_became_peered":"2024-05-10T10:37:17.224639+0000","last_unstale":"2024-05-10T10:37:17.229395+0000","last_undegraded":"2024-05-10T10:37:17.229395+0000","last_fullsized":"2024-05-10T10:37:17.229395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222142+0000","last_change":"2024-05-10T10:37:17.222142+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222142+0000","last_undegraded":"2024-05-10T10:37:17.222142+0000","last_fullsized":"2024-05-10T10:37:17.222142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231471+0000","last_change":"2024-05-10T10:37:17.228371+0000","last_active":"2024-05-10T10:37:17.231471+0000","last_peered":"2024-05-10T10:37:17.231471+0000","last_clean":"2024-05-10T10:37:17.231471+0000","last_became_active":"2024-05-10T10:37:17.227960+0000","last_became_peered":"2024-05-10T10:37:17.227960+0000","last_unstale":"2024-05-10T10:37:17.231471+0000","last_undegraded":"2024-05-10T10:37:17.231471+0000","last_fullsized":"2024-05-10T10:37:17.231471+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:19.319484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222569+0000","last_change":"2024-05-10T10:37:17.222569+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222569+0000","last_undegraded":"2024-05-10T10:37:17.222569+0000","last_fullsized":"2024-05-10T10:37:17.222569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229319+0000","last_change":"2024-05-10T10:37:17.223541+0000","last_active":"2024-05-10T10:37:17.229319+0000","last_peered":"2024-05-10T10:37:17.229319+0000","last_clean":"2024-05-10T10:37:17.229319+0000","last_became_active":"2024-05-10T10:37:17.223228+0000","last_became_peered":"2024-05-10T10:37:17.223228+0000","last_unstale":"2024-05-10T10:37:17.229319+0000","last_undegraded":"2024-05-10T10:37:17.229319+0000","last_fullsized":"2024-05-10T10:37:17.229319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:14.320723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220334+0000","last_change":"2024-05-10T10:37:17.220334+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220334+0000","last_undegraded":"2024-05-10T10:37:17.220334+0000","last_fullsized":"2024-05-10T10:37:17.220334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.236807+0000","last_change":"2024-05-10T10:37:17.232666+0000","last_active":"2024-05-10T10:37:17.236807+0000","last_peered":"2024-05-10T10:37:17.236807+0000","last_clean":"2024-05-10T10:37:17.236807+0000","last_became_active":"2024-05-10T10:37:17.232429+0000","last_became_peered":"2024-05-10T10:37:17.232429+0000","last_unstale":"2024-05-10T10:37:17.236807+0000","last_undegraded":"2024-05-10T10:37:17.236807+0000","last_fullsized":"2024-05-10T10:37:17.236807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:31.641499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220751+0000","last_change":"2024-05-10T10:37:17.220751+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220751+0000","last_undegraded":"2024-05-10T10:37:17.220751+0000","last_fullsized":"2024-05-10T10:37:17.220751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228619+0000","last_change":"2024-05-10T10:37:17.227889+0000","last_active":"2024-05-10T10:37:17.228619+0000","last_peered":"2024-05-10T10:37:17.228619+0000","last_clean":"2024-05-10T10:37:17.228619+0000","last_became_active":"2024-05-10T10:37:17.227312+0000","last_became_peered":"2024-05-10T10:37:17.227312+0000","last_unstale":"2024-05-10T10:37:17.228619+0000","last_undegraded":"2024-05-10T10:37:17.228619+0000","last_fullsized":"2024-05-10T10:37:17.228619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:37.508242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213972+0000","last_change":"2024-05-10T10:37:17.213972+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213972+0000","last_undegraded":"2024-05-10T10:37:17.213972+0000","last_fullsized":"2024-05-10T10:37:17.213972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233461+0000","last_change":"2024-05-10T10:37:17.230789+0000","last_active":"2024-05-10T10:37:17.233461+0000","last_peered":"2024-05-10T10:37:17.233461+0000","last_clean":"2024-05-10T10:37:17.233461+0000","last_became_active":"2024-05-10T10:37:17.230513+0000","last_became_peered":"2024-05-10T10:37:17.230513+0000","last_unstale":"2024-05-10T10:37:17.233461+0000","last_undegraded":"2024-05-10T10:37:17.233461+0000","last_fullsized":"2024-05-10T10:37:17.233461+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:15.368266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":215,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.219452+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:37:17.219452+0000","last_peered":"2024-05-10T10:37:17.219452+0000","last_clean":"2024-05-10T10:37:17.219452+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:37:17.219452+0000","last_undegraded":"2024-05-10T10:37:17.219452+0000","last_fullsized":"2024-05-10T10:37:17.219452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28464,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343376,"statfs":{"total":96636764160,"available":96607617024,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:37:22.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:37:22.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.787+0000 7f93b5922640 1 Processor -- start 2024-05-10T10:37:22.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.787+0000 7f93b5922640 1 -- start start 2024-05-10T10:37:22.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93b5922640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b00c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93b5922640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93b00cee40 0x7f93b00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93b5922640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b006f1c0 con 0x7f93b01598c0 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93b5922640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f93b00c7620 con 0x7f93b00cee40 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93b5922640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f93b006f4a0 con 0x7f93b00c6c80 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93af7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93b00cee40 0x7f93b00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93aeffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f93b01598c0 0x7f93b006e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42086/0 (socket says 172.21.15.53:42086) 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93af7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93b00cee40 0x7f93b00cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54120/0 (socket says 172.21.15.53:54120) 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93aeffd640 1 -- 172.21.15.53:0/637416342 learned_addr learned my addr 172.21.15.53:0/637416342 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93ae7fc640 1 --2- 172.21.15.53:0/637416342 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b00c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93af7fe640 1 -- 172.21.15.53:0/637416342 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 msgr2=0x7f93b00c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93af7fe640 1 --2- 172.21.15.53:0/637416342 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b00c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93af7fe640 1 -- 172.21.15.53:0/637416342 >> v1:172.21.15.53:6789/0 conn(0x7f93b01598c0 legacy=0x7f93b006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93af7fe640 1 -- 172.21.15.53:0/637416342 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b006f780 con 0x7f93b00cee40 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.788+0000 7f93adffb640 1 -- 172.21.15.53:0/637416342 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805203653 0 0) 0x7f93b006f1c0 con 0x7f93b01598c0 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.789+0000 7f93af7fe640 1 --2- 172.21.15.53:0/637416342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b00cf2d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009d90 tx=0x7f93a802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ccf32721ae9bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.789+0000 7f93adffb640 1 -- 172.21.15.53:0/637416342 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8007740 con 0x7f93b00cee40 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.789+0000 7f93adffb640 1 -- 172.21.15.53:0/637416342 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a80078e0 con 0x7f93b00cee40 2024-05-10T10:37:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.789+0000 7f93adffb640 1 -- 172.21.15.53:0/637416342 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8007cb0 con 0x7f93b00cee40 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.789+0000 7f93b5922640 1 -- 172.21.15.53:0/637416342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 msgr2=0x7f93b00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.789+0000 7f93b5922640 1 --2- 172.21.15.53:0/637416342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b00cf2d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009d90 tx=0x7f93a802a660 comp rx=0 tx=0).stop 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.790+0000 7f93b5922640 1 -- 172.21.15.53:0/637416342 shutdown_connections 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.790+0000 7f93b5922640 1 --2- 172.21.15.53:0/637416342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b00cf2d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.790+0000 7f93b5922640 1 --2- 172.21.15.53:0/637416342 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b00c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.790+0000 7f93b5922640 1 -- 172.21.15.53:0/637416342 >> 172.21.15.53:0/637416342 conn(0x7f93b00b2900 msgr2=0x7f93b00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.790+0000 7f93b5922640 1 -- 172.21.15.53:0/637416342 shutdown_connections 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.790+0000 7f93b5922640 1 -- 172.21.15.53:0/637416342 wait complete. 2024-05-10T10:37:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.791+0000 7f93b5922640 1 Processor -- start 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.791+0000 7f93b5922640 1 -- start start 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93b5922640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b01567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93b5922640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93b5922640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93b01598c0 0x7f93b00c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93b5922640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93b015c6e0 con 0x7f93b00cee40 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93b5922640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93b00ce920 con 0x7f93b01598c0 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93b5922640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f93b006ee10 con 0x7f93b00c6c80 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b015eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54124/0 (socket says 172.21.15.53:54124) 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 -- 172.21.15.53:0/338509880 learned_addr learned my addr 172.21.15.53:0/338509880 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93aeffd640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b01567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93af7fe640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93b01598c0 0x7f93b00c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 -- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 msgr2=0x7f93b01567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b01567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 -- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93b01598c0 msgr2=0x7f93b00c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93b01598c0 0x7f93b00c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 -- 172.21.15.53:0/338509880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b0156ce0 con 0x7f93b00cee40 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93ae7fc640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b015eeb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f93a400d830 tx=0x7f93a400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f9397fff640 1 -- 172.21.15.53:0/338509880 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a400b6c0 con 0x7f93b00cee40 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f9397fff640 1 -- 172.21.15.53:0/338509880 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a4019070 con 0x7f93b00cee40 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f9397fff640 1 -- 172.21.15.53:0/338509880 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a400ba80 con 0x7f93b00cee40 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.792+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b01f4bc0 con 0x7f93b00cee40 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.793+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01f5180 con 0x7f93b00cee40 2024-05-10T10:37:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.793+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b006a220 con 0x7f93b00cee40 2024-05-10T10:37:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.794+0000 7f9397fff640 1 -- 172.21.15.53:0/338509880 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f93a401e460 con 0x7f93b00cee40 2024-05-10T10:37:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.795+0000 7f9397fff640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f939c0762d0 0x7f939c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.795+0000 7f9397fff640 1 -- 172.21.15.53:0/338509880 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f93a4097e70 con 0x7f93b00cee40 2024-05-10T10:37:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.795+0000 7f93aeffd640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f939c0762d0 0x7f939c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.795+0000 7f93aeffd640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f939c0762d0 0x7f939c078790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000f60 tx=0x7f93a0007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:22.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.799+0000 7f9397fff640 1 -- 172.21.15.53:0/338509880 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f93a4061640 con 0x7f93b00cee40 2024-05-10T10:37:22.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.985+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93b00b23a0 con 0x7f939c0762d0 2024-05-10T10:37:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.991+0000 7f9397fff640 1 -- 172.21.15.53:0/338509880 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246740 (secure 0 0 0) 0x7f93b00b23a0 con 0x7f939c0762d0 2024-05-10T10:37:22.994 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:22.996 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:37:22.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f939c0762d0 msgr2=0x7f939c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f939c0762d0 0x7f939c078790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000f60 tx=0x7f93a0007c00 comp rx=0 tx=0).stop 2024-05-10T10:37:22.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 msgr2=0x7f93b015eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b015eeb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f93a400d830 tx=0x7f93a400dcf0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 shutdown_connections 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f939c0762d0 0x7f939c078790 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93b01598c0 0x7f93b00c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93b00cee40 0x7f93b015eeb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 --2- 172.21.15.53:0/338509880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93b00c6c80 0x7f93b01567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 >> 172.21.15.53:0/338509880 conn(0x7f93b00b2900 msgr2=0x7f93b00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 shutdown_connections 2024-05-10T10:37:22.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:22.994+0000 7f93b5922640 1 -- 172.21.15.53:0/338509880 wait complete. 2024-05-10T10:37:23.006 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-10T10:37:22.211124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240884,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733836,"statfs":{"total":773094113280,"available":772847448064,"internally_reserved":0,"allocated":8232960,"data_stored":4349152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001535"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214356+0000","last_change":"2024-05-10T10:37:17.214356+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214356+0000","last_undegraded":"2024-05-10T10:37:17.214356+0000","last_fullsized":"2024-05-10T10:37:17.214356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218881+0000","last_change":"2024-05-10T10:37:17.218881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218881+0000","last_undegraded":"2024-05-10T10:37:17.218881+0000","last_fullsized":"2024-05-10T10:37:17.218881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219561+0000","last_change":"2024-05-10T10:37:17.219561+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219561+0000","last_undegraded":"2024-05-10T10:37:17.219561+0000","last_fullsized":"2024-05-10T10:37:17.219561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220528+0000","last_change":"2024-05-10T10:37:17.220528+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220528+0000","last_undegraded":"2024-05-10T10:37:17.220528+0000","last_fullsized":"2024-05-10T10:37:17.220528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217963+0000","last_change":"2024-05-10T10:37:17.217963+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217963+0000","last_undegraded":"2024-05-10T10:37:17.217963+0000","last_fullsized":"2024-05-10T10:37:17.217963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220201+0000","last_change":"2024-05-10T10:37:17.220201+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220201+0000","last_undegraded":"2024-05-10T10:37:17.220201+0000","last_fullsized":"2024-05-10T10:37:17.220201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222149+0000","last_change":"2024-05-10T10:37:17.222149+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222149+0000","last_undegraded":"2024-05-10T10:37:17.222149+0000","last_fullsized":"2024-05-10T10:37:17.222149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215093+0000","last_change":"2024-05-10T10:37:17.215093+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215093+0000","last_undegraded":"2024-05-10T10:37:17.215093+0000","last_fullsized":"2024-05-10T10:37:17.215093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216792+0000","last_change":"2024-05-10T10:37:17.216792+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216792+0000","last_undegraded":"2024-05-10T10:37:17.216792+0000","last_fullsized":"2024-05-10T10:37:17.216792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218788+0000","last_change":"2024-05-10T10:37:17.218788+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218788+0000","last_undegraded":"2024-05-10T10:37:17.218788+0000","last_fullsized":"2024-05-10T10:37:17.218788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220858+0000","last_change":"2024-05-10T10:37:17.220858+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220858+0000","last_undegraded":"2024-05-10T10:37:17.220858+0000","last_fullsized":"2024-05-10T10:37:17.220858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218915+0000","last_change":"2024-05-10T10:37:17.218915+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218915+0000","last_undegraded":"2024-05-10T10:37:17.218915+0000","last_fullsized":"2024-05-10T10:37:17.218915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216194+0000","last_change":"2024-05-10T10:37:17.216194+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216194+0000","last_undegraded":"2024-05-10T10:37:17.216194+0000","last_fullsized":"2024-05-10T10:37:17.216194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219010+0000","last_change":"2024-05-10T10:37:17.219010+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219010+0000","last_undegraded":"2024-05-10T10:37:17.219010+0000","last_fullsized":"2024-05-10T10:37:17.219010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217797+0000","last_change":"2024-05-10T10:37:17.217797+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217797+0000","last_undegraded":"2024-05-10T10:37:17.217797+0000","last_fullsized":"2024-05-10T10:37:17.217797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215236+0000","last_change":"2024-05-10T10:37:17.215236+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215236+0000","last_undegraded":"2024-05-10T10:37:17.215236+0000","last_fullsized":"2024-05-10T10:37:17.215236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217685+0000","last_change":"2024-05-10T10:37:17.217685+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217685+0000","last_undegraded":"2024-05-10T10:37:17.217685+0000","last_fullsized":"2024-05-10T10:37:17.217685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220781+0000","last_change":"2024-05-10T10:37:17.220781+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220781+0000","last_undegraded":"2024-05-10T10:37:17.220781+0000","last_fullsized":"2024-05-10T10:37:17.220781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215713+0000","last_change":"2024-05-10T10:37:17.215713+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215713+0000","last_undegraded":"2024-05-10T10:37:17.215713+0000","last_fullsized":"2024-05-10T10:37:17.215713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217141+0000","last_change":"2024-05-10T10:37:17.217141+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217141+0000","last_undegraded":"2024-05-10T10:37:17.217141+0000","last_fullsized":"2024-05-10T10:37:17.217141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217274+0000","last_change":"2024-05-10T10:37:17.217274+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217274+0000","last_undegraded":"2024-05-10T10:37:17.217274+0000","last_fullsized":"2024-05-10T10:37:17.217274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215881+0000","last_change":"2024-05-10T10:37:17.215881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215881+0000","last_undegraded":"2024-05-10T10:37:17.215881+0000","last_fullsized":"2024-05-10T10:37:17.215881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218122+0000","last_change":"2024-05-10T10:37:17.218122+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218122+0000","last_undegraded":"2024-05-10T10:37:17.218122+0000","last_fullsized":"2024-05-10T10:37:17.218122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217328+0000","last_change":"2024-05-10T10:37:17.217328+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217328+0000","last_undegraded":"2024-05-10T10:37:17.217328+0000","last_fullsized":"2024-05-10T10:37:17.217328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216827+0000","last_change":"2024-05-10T10:37:17.216827+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216827+0000","last_undegraded":"2024-05-10T10:37:17.216827+0000","last_fullsized":"2024-05-10T10:37:17.216827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216132+0000","last_change":"2024-05-10T10:37:17.216132+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216132+0000","last_undegraded":"2024-05-10T10:37:17.216132+0000","last_fullsized":"2024-05-10T10:37:17.216132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214571+0000","last_change":"2024-05-10T10:37:17.214571+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214571+0000","last_undegraded":"2024-05-10T10:37:17.214571+0000","last_fullsized":"2024-05-10T10:37:17.214571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215835+0000","last_change":"2024-05-10T10:37:17.215835+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215835+0000","last_undegraded":"2024-05-10T10:37:17.215835+0000","last_fullsized":"2024-05-10T10:37:17.215835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216848+0000","last_change":"2024-05-10T10:37:17.216848+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216848+0000","last_undegraded":"2024-05-10T10:37:17.216848+0000","last_fullsized":"2024-05-10T10:37:17.216848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213937+0000","last_change":"2024-05-10T10:37:17.213937+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213937+0000","last_undegraded":"2024-05-10T10:37:17.213937+0000","last_fullsized":"2024-05-10T10:37:17.213937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217597+0000","last_change":"2024-05-10T10:37:17.217597+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217597+0000","last_undegraded":"2024-05-10T10:37:17.217597+0000","last_fullsized":"2024-05-10T10:37:17.217597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214590+0000","last_change":"2024-05-10T10:37:17.214590+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214590+0000","last_undegraded":"2024-05-10T10:37:17.214590+0000","last_fullsized":"2024-05-10T10:37:17.214590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218472+0000","last_change":"2024-05-10T10:37:17.218472+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218472+0000","last_undegraded":"2024-05-10T10:37:17.218472+0000","last_fullsized":"2024-05-10T10:37:17.218472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234893+0000","last_change":"2024-05-10T10:37:17.234091+0000","last_active":"2024-05-10T10:37:17.234893+0000","last_peered":"2024-05-10T10:37:17.234893+0000","last_clean":"2024-05-10T10:37:17.234893+0000","last_became_active":"2024-05-10T10:37:17.233792+0000","last_became_peered":"2024-05-10T10:37:17.233792+0000","last_unstale":"2024-05-10T10:37:17.234893+0000","last_undegraded":"2024-05-10T10:37:17.234893+0000","last_fullsized":"2024-05-10T10:37:17.234893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:15:16.625576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214546+0000","last_change":"2024-05-10T10:37:17.214546+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214546+0000","last_undegraded":"2024-05-10T10:37:17.214546+0000","last_fullsized":"2024-05-10T10:37:17.214546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233815+0000","last_change":"2024-05-10T10:37:17.229429+0000","last_active":"2024-05-10T10:37:17.233815+0000","last_peered":"2024-05-10T10:37:17.233815+0000","last_clean":"2024-05-10T10:37:17.233815+0000","last_became_active":"2024-05-10T10:37:17.229121+0000","last_became_peered":"2024-05-10T10:37:17.229121+0000","last_unstale":"2024-05-10T10:37:17.233815+0000","last_undegraded":"2024-05-10T10:37:17.233815+0000","last_fullsized":"2024-05-10T10:37:17.233815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:19.839294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218554+0000","last_change":"2024-05-10T10:37:17.218554+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218554+0000","last_undegraded":"2024-05-10T10:37:17.218554+0000","last_fullsized":"2024-05-10T10:37:17.218554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229641+0000","last_change":"2024-05-10T10:37:17.225715+0000","last_active":"2024-05-10T10:37:17.229641+0000","last_peered":"2024-05-10T10:37:17.229641+0000","last_clean":"2024-05-10T10:37:17.229641+0000","last_became_active":"2024-05-10T10:37:17.225250+0000","last_became_peered":"2024-05-10T10:37:17.225250+0000","last_unstale":"2024-05-10T10:37:17.229641+0000","last_undegraded":"2024-05-10T10:37:17.229641+0000","last_fullsized":"2024-05-10T10:37:17.229641+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:39.904069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216317+0000","last_change":"2024-05-10T10:37:17.216317+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216317+0000","last_undegraded":"2024-05-10T10:37:17.216317+0000","last_fullsized":"2024-05-10T10:37:17.216317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231454+0000","last_change":"2024-05-10T10:37:17.228203+0000","last_active":"2024-05-10T10:37:17.231454+0000","last_peered":"2024-05-10T10:37:17.231454+0000","last_clean":"2024-05-10T10:37:17.231454+0000","last_became_active":"2024-05-10T10:37:17.227939+0000","last_became_peered":"2024-05-10T10:37:17.227939+0000","last_unstale":"2024-05-10T10:37:17.231454+0000","last_undegraded":"2024-05-10T10:37:17.231454+0000","last_fullsized":"2024-05-10T10:37:17.231454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:19.500154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215764+0000","last_change":"2024-05-10T10:37:17.215764+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215764+0000","last_undegraded":"2024-05-10T10:37:17.215764+0000","last_fullsized":"2024-05-10T10:37:17.215764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.225465+0000","last_change":"2024-05-10T10:37:17.222699+0000","last_active":"2024-05-10T10:37:17.225465+0000","last_peered":"2024-05-10T10:37:17.225465+0000","last_clean":"2024-05-10T10:37:17.225465+0000","last_became_active":"2024-05-10T10:37:17.222389+0000","last_became_peered":"2024-05-10T10:37:17.222389+0000","last_unstale":"2024-05-10T10:37:17.225465+0000","last_undegraded":"2024-05-10T10:37:17.225465+0000","last_fullsized":"2024-05-10T10:37:17.225465+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:59:22.414675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215567+0000","last_change":"2024-05-10T10:37:17.215567+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215567+0000","last_undegraded":"2024-05-10T10:37:17.215567+0000","last_fullsized":"2024-05-10T10:37:17.215567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231160+0000","last_change":"2024-05-10T10:37:17.227524+0000","last_active":"2024-05-10T10:37:17.231160+0000","last_peered":"2024-05-10T10:37:17.231160+0000","last_clean":"2024-05-10T10:37:17.231160+0000","last_became_active":"2024-05-10T10:37:17.227256+0000","last_became_peered":"2024-05-10T10:37:17.227256+0000","last_unstale":"2024-05-10T10:37:17.231160+0000","last_undegraded":"2024-05-10T10:37:17.231160+0000","last_fullsized":"2024-05-10T10:37:17.231160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217227+0000","last_change":"2024-05-10T10:37:17.217227+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217227+0000","last_undegraded":"2024-05-10T10:37:17.217227+0000","last_fullsized":"2024-05-10T10:37:17.217227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.222393+0000","last_change":"2024-05-10T10:37:17.218206+0000","last_active":"2024-05-10T10:37:17.222393+0000","last_peered":"2024-05-10T10:37:17.222393+0000","last_clean":"2024-05-10T10:37:17.222393+0000","last_became_active":"2024-05-10T10:37:17.217591+0000","last_became_peered":"2024-05-10T10:37:17.217591+0000","last_unstale":"2024-05-10T10:37:17.222393+0000","last_undegraded":"2024-05-10T10:37:17.222393+0000","last_fullsized":"2024-05-10T10:37:17.222393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:15:45.683363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223820+0000","last_change":"2024-05-10T10:37:17.223820+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223820+0000","last_undegraded":"2024-05-10T10:37:17.223820+0000","last_fullsized":"2024-05-10T10:37:17.223820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231519+0000","last_change":"2024-05-10T10:37:17.230114+0000","last_active":"2024-05-10T10:37:17.231519+0000","last_peered":"2024-05-10T10:37:17.231519+0000","last_clean":"2024-05-10T10:37:17.231519+0000","last_became_active":"2024-05-10T10:37:17.229847+0000","last_became_peered":"2024-05-10T10:37:17.229847+0000","last_unstale":"2024-05-10T10:37:17.231519+0000","last_undegraded":"2024-05-10T10:37:17.231519+0000","last_fullsized":"2024-05-10T10:37:17.231519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:47:31.740584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214560+0000","last_change":"2024-05-10T10:37:17.214560+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214560+0000","last_undegraded":"2024-05-10T10:37:17.214560+0000","last_fullsized":"2024-05-10T10:37:17.214560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227281+0000","last_change":"2024-05-10T10:37:17.224805+0000","last_active":"2024-05-10T10:37:17.227281+0000","last_peered":"2024-05-10T10:37:17.227281+0000","last_clean":"2024-05-10T10:37:17.227281+0000","last_became_active":"2024-05-10T10:37:17.224508+0000","last_became_peered":"2024-05-10T10:37:17.224508+0000","last_unstale":"2024-05-10T10:37:17.227281+0000","last_undegraded":"2024-05-10T10:37:17.227281+0000","last_fullsized":"2024-05-10T10:37:17.227281+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215515+0000","last_change":"2024-05-10T10:37:17.215515+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215515+0000","last_undegraded":"2024-05-10T10:37:17.215515+0000","last_fullsized":"2024-05-10T10:37:17.215515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228363+0000","last_change":"2024-05-10T10:37:17.225567+0000","last_active":"2024-05-10T10:37:17.228363+0000","last_peered":"2024-05-10T10:37:17.228363+0000","last_clean":"2024-05-10T10:37:17.228363+0000","last_became_active":"2024-05-10T10:37:17.225301+0000","last_became_peered":"2024-05-10T10:37:17.225301+0000","last_unstale":"2024-05-10T10:37:17.228363+0000","last_undegraded":"2024-05-10T10:37:17.228363+0000","last_fullsized":"2024-05-10T10:37:17.228363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216485+0000","last_change":"2024-05-10T10:37:17.216485+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216485+0000","last_undegraded":"2024-05-10T10:37:17.216485+0000","last_fullsized":"2024-05-10T10:37:17.216485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.232470+0000","last_change":"2024-05-10T10:37:17.231914+0000","last_active":"2024-05-10T10:37:17.232470+0000","last_peered":"2024-05-10T10:37:17.232470+0000","last_clean":"2024-05-10T10:37:17.232470+0000","last_became_active":"2024-05-10T10:37:17.231631+0000","last_became_peered":"2024-05-10T10:37:17.231631+0000","last_unstale":"2024-05-10T10:37:17.232470+0000","last_undegraded":"2024-05-10T10:37:17.232470+0000","last_fullsized":"2024-05-10T10:37:17.232470+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:01:34.692557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220258+0000","last_change":"2024-05-10T10:37:17.220258+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220258+0000","last_undegraded":"2024-05-10T10:37:17.220258+0000","last_fullsized":"2024-05-10T10:37:17.220258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233970+0000","last_change":"2024-05-10T10:37:17.229898+0000","last_active":"2024-05-10T10:37:17.233970+0000","last_peered":"2024-05-10T10:37:17.233970+0000","last_clean":"2024-05-10T10:37:17.233970+0000","last_became_active":"2024-05-10T10:37:17.229599+0000","last_became_peered":"2024-05-10T10:37:17.229599+0000","last_unstale":"2024-05-10T10:37:17.233970+0000","last_undegraded":"2024-05-10T10:37:17.233970+0000","last_fullsized":"2024-05-10T10:37:17.233970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:30.149365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215708+0000","last_change":"2024-05-10T10:37:17.215708+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215708+0000","last_undegraded":"2024-05-10T10:37:17.215708+0000","last_fullsized":"2024-05-10T10:37:17.215708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228415+0000","last_change":"2024-05-10T10:37:17.223996+0000","last_active":"2024-05-10T10:37:17.228415+0000","last_peered":"2024-05-10T10:37:17.228415+0000","last_clean":"2024-05-10T10:37:17.228415+0000","last_became_active":"2024-05-10T10:37:17.223309+0000","last_became_peered":"2024-05-10T10:37:17.223309+0000","last_unstale":"2024-05-10T10:37:17.228415+0000","last_undegraded":"2024-05-10T10:37:17.228415+0000","last_fullsized":"2024-05-10T10:37:17.228415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:11.520143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223632+0000","last_change":"2024-05-10T10:37:17.223632+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223632+0000","last_undegraded":"2024-05-10T10:37:17.223632+0000","last_fullsized":"2024-05-10T10:37:17.223632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229437+0000","last_change":"2024-05-10T10:37:17.226617+0000","last_active":"2024-05-10T10:37:17.229437+0000","last_peered":"2024-05-10T10:37:17.229437+0000","last_clean":"2024-05-10T10:37:17.229437+0000","last_became_active":"2024-05-10T10:37:17.226339+0000","last_became_peered":"2024-05-10T10:37:17.226339+0000","last_unstale":"2024-05-10T10:37:17.229437+0000","last_undegraded":"2024-05-10T10:37:17.229437+0000","last_fullsized":"2024-05-10T10:37:17.229437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:36:44.271136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223403+0000","last_change":"2024-05-10T10:37:17.223403+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223403+0000","last_undegraded":"2024-05-10T10:37:17.223403+0000","last_fullsized":"2024-05-10T10:37:17.223403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234908+0000","last_change":"2024-05-10T10:37:17.234183+0000","last_active":"2024-05-10T10:37:17.234908+0000","last_peered":"2024-05-10T10:37:17.234908+0000","last_clean":"2024-05-10T10:37:17.234908+0000","last_became_active":"2024-05-10T10:37:17.233852+0000","last_became_peered":"2024-05-10T10:37:17.233852+0000","last_unstale":"2024-05-10T10:37:17.234908+0000","last_undegraded":"2024-05-10T10:37:17.234908+0000","last_fullsized":"2024-05-10T10:37:17.234908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:57:04.326406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222318+0000","last_change":"2024-05-10T10:37:17.222318+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222318+0000","last_undegraded":"2024-05-10T10:37:17.222318+0000","last_fullsized":"2024-05-10T10:37:17.222318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227850+0000","last_change":"2024-05-10T10:37:17.224923+0000","last_active":"2024-05-10T10:37:17.227850+0000","last_peered":"2024-05-10T10:37:17.227850+0000","last_clean":"2024-05-10T10:37:17.227850+0000","last_became_active":"2024-05-10T10:37:17.224594+0000","last_became_peered":"2024-05-10T10:37:17.224594+0000","last_unstale":"2024-05-10T10:37:17.227850+0000","last_undegraded":"2024-05-10T10:37:17.227850+0000","last_fullsized":"2024-05-10T10:37:17.227850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:58:38.099023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214436+0000","last_change":"2024-05-10T10:37:17.214436+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214436+0000","last_undegraded":"2024-05-10T10:37:17.214436+0000","last_fullsized":"2024-05-10T10:37:17.214436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229429+0000","last_change":"2024-05-10T10:37:17.225719+0000","last_active":"2024-05-10T10:37:17.229429+0000","last_peered":"2024-05-10T10:37:17.229429+0000","last_clean":"2024-05-10T10:37:17.229429+0000","last_became_active":"2024-05-10T10:37:17.225430+0000","last_became_peered":"2024-05-10T10:37:17.225430+0000","last_unstale":"2024-05-10T10:37:17.229429+0000","last_undegraded":"2024-05-10T10:37:17.229429+0000","last_fullsized":"2024-05-10T10:37:17.229429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:09.884646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217568+0000","last_change":"2024-05-10T10:37:17.217568+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217568+0000","last_undegraded":"2024-05-10T10:37:17.217568+0000","last_fullsized":"2024-05-10T10:37:17.217568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228794+0000","last_change":"2024-05-10T10:37:17.226246+0000","last_active":"2024-05-10T10:37:17.228794+0000","last_peered":"2024-05-10T10:37:17.228794+0000","last_clean":"2024-05-10T10:37:17.228794+0000","last_became_active":"2024-05-10T10:37:17.225996+0000","last_became_peered":"2024-05-10T10:37:17.225996+0000","last_unstale":"2024-05-10T10:37:17.228794+0000","last_undegraded":"2024-05-10T10:37:17.228794+0000","last_fullsized":"2024-05-10T10:37:17.228794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:55:53.951642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216780+0000","last_change":"2024-05-10T10:37:17.216780+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216780+0000","last_undegraded":"2024-05-10T10:37:17.216780+0000","last_fullsized":"2024-05-10T10:37:17.216780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229357+0000","last_change":"2024-05-10T10:37:17.223948+0000","last_active":"2024-05-10T10:37:17.229357+0000","last_peered":"2024-05-10T10:37:17.229357+0000","last_clean":"2024-05-10T10:37:17.229357+0000","last_became_active":"2024-05-10T10:37:17.223293+0000","last_became_peered":"2024-05-10T10:37:17.223293+0000","last_unstale":"2024-05-10T10:37:17.229357+0000","last_undegraded":"2024-05-10T10:37:17.229357+0000","last_fullsized":"2024-05-10T10:37:17.229357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:08.802431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215262+0000","last_change":"2024-05-10T10:37:17.215262+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215262+0000","last_undegraded":"2024-05-10T10:37:17.215262+0000","last_fullsized":"2024-05-10T10:37:17.215262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234053+0000","last_change":"2024-05-10T10:37:17.232761+0000","last_active":"2024-05-10T10:37:17.234053+0000","last_peered":"2024-05-10T10:37:17.234053+0000","last_clean":"2024-05-10T10:37:17.234053+0000","last_became_active":"2024-05-10T10:37:17.232401+0000","last_became_peered":"2024-05-10T10:37:17.232401+0000","last_unstale":"2024-05-10T10:37:17.234053+0000","last_undegraded":"2024-05-10T10:37:17.234053+0000","last_fullsized":"2024-05-10T10:37:17.234053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:39:14.930472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214673+0000","last_change":"2024-05-10T10:37:17.214673+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214673+0000","last_undegraded":"2024-05-10T10:37:17.214673+0000","last_fullsized":"2024-05-10T10:37:17.214673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233192+0000","last_change":"2024-05-10T10:37:17.227272+0000","last_active":"2024-05-10T10:37:17.233192+0000","last_peered":"2024-05-10T10:37:17.233192+0000","last_clean":"2024-05-10T10:37:17.233192+0000","last_became_active":"2024-05-10T10:37:17.224267+0000","last_became_peered":"2024-05-10T10:37:17.224267+0000","last_unstale":"2024-05-10T10:37:17.233192+0000","last_undegraded":"2024-05-10T10:37:17.233192+0000","last_fullsized":"2024-05-10T10:37:17.233192+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:23:59.926354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215169+0000","last_change":"2024-05-10T10:37:17.215169+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215169+0000","last_undegraded":"2024-05-10T10:37:17.215169+0000","last_fullsized":"2024-05-10T10:37:17.215169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233428+0000","last_change":"2024-05-10T10:37:17.230132+0000","last_active":"2024-05-10T10:37:17.233428+0000","last_peered":"2024-05-10T10:37:17.233428+0000","last_clean":"2024-05-10T10:37:17.233428+0000","last_became_active":"2024-05-10T10:37:17.229845+0000","last_became_peered":"2024-05-10T10:37:17.229845+0000","last_unstale":"2024-05-10T10:37:17.233428+0000","last_undegraded":"2024-05-10T10:37:17.233428+0000","last_fullsized":"2024-05-10T10:37:17.233428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:16:39.529753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221900+0000","last_change":"2024-05-10T10:37:17.221900+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221900+0000","last_undegraded":"2024-05-10T10:37:17.221900+0000","last_fullsized":"2024-05-10T10:37:17.221900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233394+0000","last_change":"2024-05-10T10:37:17.229451+0000","last_active":"2024-05-10T10:37:17.233394+0000","last_peered":"2024-05-10T10:37:17.233394+0000","last_clean":"2024-05-10T10:37:17.233394+0000","last_became_active":"2024-05-10T10:37:17.228407+0000","last_became_peered":"2024-05-10T10:37:17.228407+0000","last_unstale":"2024-05-10T10:37:17.233394+0000","last_undegraded":"2024-05-10T10:37:17.233394+0000","last_fullsized":"2024-05-10T10:37:17.233394+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:43:25.143088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216340+0000","last_change":"2024-05-10T10:37:17.216340+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216340+0000","last_undegraded":"2024-05-10T10:37:17.216340+0000","last_fullsized":"2024-05-10T10:37:17.216340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228381+0000","last_change":"2024-05-10T10:37:17.224792+0000","last_active":"2024-05-10T10:37:17.228381+0000","last_peered":"2024-05-10T10:37:17.228381+0000","last_clean":"2024-05-10T10:37:17.228381+0000","last_became_active":"2024-05-10T10:37:17.224443+0000","last_became_peered":"2024-05-10T10:37:17.224443+0000","last_unstale":"2024-05-10T10:37:17.228381+0000","last_undegraded":"2024-05-10T10:37:17.228381+0000","last_fullsized":"2024-05-10T10:37:17.228381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:13.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221621+0000","last_change":"2024-05-10T10:37:17.221621+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221621+0000","last_undegraded":"2024-05-10T10:37:17.221621+0000","last_fullsized":"2024-05-10T10:37:17.221621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.230785+0000","last_change":"2024-05-10T10:37:17.230099+0000","last_active":"2024-05-10T10:37:17.230785+0000","last_peered":"2024-05-10T10:37:17.230785+0000","last_clean":"2024-05-10T10:37:17.230785+0000","last_became_active":"2024-05-10T10:37:17.229704+0000","last_became_peered":"2024-05-10T10:37:17.229704+0000","last_unstale":"2024-05-10T10:37:17.230785+0000","last_undegraded":"2024-05-10T10:37:17.230785+0000","last_fullsized":"2024-05-10T10:37:17.230785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:43:08.567175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218157+0000","last_change":"2024-05-10T10:37:17.218157+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218157+0000","last_undegraded":"2024-05-10T10:37:17.218157+0000","last_fullsized":"2024-05-10T10:37:17.218157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229670+0000","last_change":"2024-05-10T10:37:17.226288+0000","last_active":"2024-05-10T10:37:17.229670+0000","last_peered":"2024-05-10T10:37:17.229670+0000","last_clean":"2024-05-10T10:37:17.229670+0000","last_became_active":"2024-05-10T10:37:17.226017+0000","last_became_peered":"2024-05-10T10:37:17.226017+0000","last_unstale":"2024-05-10T10:37:17.229670+0000","last_undegraded":"2024-05-10T10:37:17.229670+0000","last_fullsized":"2024-05-10T10:37:17.229670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:50.410699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214140+0000","last_change":"2024-05-10T10:37:17.214140+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214140+0000","last_undegraded":"2024-05-10T10:37:17.214140+0000","last_fullsized":"2024-05-10T10:37:17.214140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229395+0000","last_change":"2024-05-10T10:37:17.224891+0000","last_active":"2024-05-10T10:37:17.229395+0000","last_peered":"2024-05-10T10:37:17.229395+0000","last_clean":"2024-05-10T10:37:17.229395+0000","last_became_active":"2024-05-10T10:37:17.224639+0000","last_became_peered":"2024-05-10T10:37:17.224639+0000","last_unstale":"2024-05-10T10:37:17.229395+0000","last_undegraded":"2024-05-10T10:37:17.229395+0000","last_fullsized":"2024-05-10T10:37:17.229395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222142+0000","last_change":"2024-05-10T10:37:17.222142+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222142+0000","last_undegraded":"2024-05-10T10:37:17.222142+0000","last_fullsized":"2024-05-10T10:37:17.222142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231471+0000","last_change":"2024-05-10T10:37:17.228371+0000","last_active":"2024-05-10T10:37:17.231471+0000","last_peered":"2024-05-10T10:37:17.231471+0000","last_clean":"2024-05-10T10:37:17.231471+0000","last_became_active":"2024-05-10T10:37:17.227960+0000","last_became_peered":"2024-05-10T10:37:17.227960+0000","last_unstale":"2024-05-10T10:37:17.231471+0000","last_undegraded":"2024-05-10T10:37:17.231471+0000","last_fullsized":"2024-05-10T10:37:17.231471+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:19.319484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222569+0000","last_change":"2024-05-10T10:37:17.222569+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222569+0000","last_undegraded":"2024-05-10T10:37:17.222569+0000","last_fullsized":"2024-05-10T10:37:17.222569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229319+0000","last_change":"2024-05-10T10:37:17.223541+0000","last_active":"2024-05-10T10:37:17.229319+0000","last_peered":"2024-05-10T10:37:17.229319+0000","last_clean":"2024-05-10T10:37:17.229319+0000","last_became_active":"2024-05-10T10:37:17.223228+0000","last_became_peered":"2024-05-10T10:37:17.223228+0000","last_unstale":"2024-05-10T10:37:17.229319+0000","last_undegraded":"2024-05-10T10:37:17.229319+0000","last_fullsized":"2024-05-10T10:37:17.229319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:14.320723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220334+0000","last_change":"2024-05-10T10:37:17.220334+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220334+0000","last_undegraded":"2024-05-10T10:37:17.220334+0000","last_fullsized":"2024-05-10T10:37:17.220334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.236807+0000","last_change":"2024-05-10T10:37:17.232666+0000","last_active":"2024-05-10T10:37:17.236807+0000","last_peered":"2024-05-10T10:37:17.236807+0000","last_clean":"2024-05-10T10:37:17.236807+0000","last_became_active":"2024-05-10T10:37:17.232429+0000","last_became_peered":"2024-05-10T10:37:17.232429+0000","last_unstale":"2024-05-10T10:37:17.236807+0000","last_undegraded":"2024-05-10T10:37:17.236807+0000","last_fullsized":"2024-05-10T10:37:17.236807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:31.641499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220751+0000","last_change":"2024-05-10T10:37:17.220751+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220751+0000","last_undegraded":"2024-05-10T10:37:17.220751+0000","last_fullsized":"2024-05-10T10:37:17.220751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228619+0000","last_change":"2024-05-10T10:37:17.227889+0000","last_active":"2024-05-10T10:37:17.228619+0000","last_peered":"2024-05-10T10:37:17.228619+0000","last_clean":"2024-05-10T10:37:17.228619+0000","last_became_active":"2024-05-10T10:37:17.227312+0000","last_became_peered":"2024-05-10T10:37:17.227312+0000","last_unstale":"2024-05-10T10:37:17.228619+0000","last_undegraded":"2024-05-10T10:37:17.228619+0000","last_fullsized":"2024-05-10T10:37:17.228619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:37.508242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213972+0000","last_change":"2024-05-10T10:37:17.213972+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213972+0000","last_undegraded":"2024-05-10T10:37:17.213972+0000","last_fullsized":"2024-05-10T10:37:17.213972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233461+0000","last_change":"2024-05-10T10:37:17.230789+0000","last_active":"2024-05-10T10:37:17.233461+0000","last_peered":"2024-05-10T10:37:17.233461+0000","last_clean":"2024-05-10T10:37:17.233461+0000","last_became_active":"2024-05-10T10:37:17.230513+0000","last_became_peered":"2024-05-10T10:37:17.230513+0000","last_unstale":"2024-05-10T10:37:17.233461+0000","last_undegraded":"2024-05-10T10:37:17.233461+0000","last_fullsized":"2024-05-10T10:37:17.233461+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:15.368266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":215,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.219452+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:37:17.219452+0000","last_peered":"2024-05-10T10:37:17.219452+0000","last_clean":"2024-05-10T10:37:17.219452+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:37:17.219452+0000","last_undegraded":"2024-05-10T10:37:17.219452+0000","last_fullsized":"2024-05-10T10:37:17.219452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28464,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343376,"statfs":{"total":96636764160,"available":96607617024,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:37:23.010 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3f is not active+clean 2024-05-10T10:37:23.010 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214356+0000', 'last_change': '2024-05-10T10:37:17.214356+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214356+0000', 'last_undegraded': '2024-05-10T10:37:17.214356+0000', 'last_fullsized': '2024-05-10T10:37:17.214356+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:37:23.010 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3e is not active+clean 2024-05-10T10:37:23.010 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.218881+0000', 'last_change': '2024-05-10T10:37:17.218881+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.218881+0000', 'last_undegraded': '2024-05-10T10:37:17.218881+0000', 'last_fullsized': '2024-05-10T10:37:17.218881+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3d is not active+clean 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.219561+0000', 'last_change': '2024-05-10T10:37:17.219561+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.219561+0000', 'last_undegraded': '2024-05-10T10:37:17.219561+0000', 'last_fullsized': '2024-05-10T10:37:17.219561+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3c is not active+clean 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.220528+0000', 'last_change': '2024-05-10T10:37:17.220528+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.220528+0000', 'last_undegraded': '2024-05-10T10:37:17.220528+0000', 'last_fullsized': '2024-05-10T10:37:17.220528+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3b is not active+clean 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217963+0000', 'last_change': '2024-05-10T10:37:17.217963+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217963+0000', 'last_undegraded': '2024-05-10T10:37:17.217963+0000', 'last_fullsized': '2024-05-10T10:37:17.217963+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3a is not active+clean 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.220201+0000', 'last_change': '2024-05-10T10:37:17.220201+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.220201+0000', 'last_undegraded': '2024-05-10T10:37:17.220201+0000', 'last_fullsized': '2024-05-10T10:37:17.220201+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:PG 12.39 is not active+clean 2024-05-10T10:37:23.011 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.222149+0000', 'last_change': '2024-05-10T10:37:17.222149+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.222149+0000', 'last_undegraded': '2024-05-10T10:37:17.222149+0000', 'last_fullsized': '2024-05-10T10:37:17.222149+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:PG 12.38 is not active+clean 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215093+0000', 'last_change': '2024-05-10T10:37:17.215093+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215093+0000', 'last_undegraded': '2024-05-10T10:37:17.215093+0000', 'last_fullsized': '2024-05-10T10:37:17.215093+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:PG 12.37 is not active+clean 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216792+0000', 'last_change': '2024-05-10T10:37:17.216792+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216792+0000', 'last_undegraded': '2024-05-10T10:37:17.216792+0000', 'last_fullsized': '2024-05-10T10:37:17.216792+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:PG 12.36 is not active+clean 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.218788+0000', 'last_change': '2024-05-10T10:37:17.218788+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.218788+0000', 'last_undegraded': '2024-05-10T10:37:17.218788+0000', 'last_fullsized': '2024-05-10T10:37:17.218788+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:PG 12.35 is not active+clean 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.220858+0000', 'last_change': '2024-05-10T10:37:17.220858+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.220858+0000', 'last_undegraded': '2024-05-10T10:37:17.220858+0000', 'last_fullsized': '2024-05-10T10:37:17.220858+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:PG 12.34 is not active+clean 2024-05-10T10:37:23.012 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.218915+0000', 'last_change': '2024-05-10T10:37:17.218915+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.218915+0000', 'last_undegraded': '2024-05-10T10:37:17.218915+0000', 'last_fullsized': '2024-05-10T10:37:17.218915+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:PG 12.33 is not active+clean 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216194+0000', 'last_change': '2024-05-10T10:37:17.216194+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216194+0000', 'last_undegraded': '2024-05-10T10:37:17.216194+0000', 'last_fullsized': '2024-05-10T10:37:17.216194+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:PG 12.32 is not active+clean 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.219010+0000', 'last_change': '2024-05-10T10:37:17.219010+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.219010+0000', 'last_undegraded': '2024-05-10T10:37:17.219010+0000', 'last_fullsized': '2024-05-10T10:37:17.219010+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:PG 12.31 is not active+clean 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217797+0000', 'last_change': '2024-05-10T10:37:17.217797+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217797+0000', 'last_undegraded': '2024-05-10T10:37:17.217797+0000', 'last_fullsized': '2024-05-10T10:37:17.217797+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:PG 12.30 is not active+clean 2024-05-10T10:37:23.013 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215236+0000', 'last_change': '2024-05-10T10:37:17.215236+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215236+0000', 'last_undegraded': '2024-05-10T10:37:17.215236+0000', 'last_fullsized': '2024-05-10T10:37:17.215236+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:37:23.014 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2f is not active+clean 2024-05-10T10:37:23.014 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217685+0000', 'last_change': '2024-05-10T10:37:17.217685+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217685+0000', 'last_undegraded': '2024-05-10T10:37:17.217685+0000', 'last_fullsized': '2024-05-10T10:37:17.217685+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.014 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2e is not active+clean 2024-05-10T10:37:23.014 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.220781+0000', 'last_change': '2024-05-10T10:37:17.220781+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.220781+0000', 'last_undegraded': '2024-05-10T10:37:17.220781+0000', 'last_fullsized': '2024-05-10T10:37:17.220781+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:37:23.014 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2d is not active+clean 2024-05-10T10:37:23.014 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215713+0000', 'last_change': '2024-05-10T10:37:17.215713+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215713+0000', 'last_undegraded': '2024-05-10T10:37:17.215713+0000', 'last_fullsized': '2024-05-10T10:37:17.215713+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2c is not active+clean 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217141+0000', 'last_change': '2024-05-10T10:37:17.217141+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217141+0000', 'last_undegraded': '2024-05-10T10:37:17.217141+0000', 'last_fullsized': '2024-05-10T10:37:17.217141+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2b is not active+clean 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217274+0000', 'last_change': '2024-05-10T10:37:17.217274+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217274+0000', 'last_undegraded': '2024-05-10T10:37:17.217274+0000', 'last_fullsized': '2024-05-10T10:37:17.217274+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2a is not active+clean 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215881+0000', 'last_change': '2024-05-10T10:37:17.215881+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215881+0000', 'last_undegraded': '2024-05-10T10:37:17.215881+0000', 'last_fullsized': '2024-05-10T10:37:17.215881+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:PG 12.29 is not active+clean 2024-05-10T10:37:23.015 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.218122+0000', 'last_change': '2024-05-10T10:37:17.218122+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.218122+0000', 'last_undegraded': '2024-05-10T10:37:17.218122+0000', 'last_fullsized': '2024-05-10T10:37:17.218122+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.017 INFO:tasks.ceph.ceph_manager.ceph:PG 12.28 is not active+clean 2024-05-10T10:37:23.017 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217328+0000', 'last_change': '2024-05-10T10:37:17.217328+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217328+0000', 'last_undegraded': '2024-05-10T10:37:17.217328+0000', 'last_fullsized': '2024-05-10T10:37:17.217328+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.017 INFO:tasks.ceph.ceph_manager.ceph:PG 12.27 is not active+clean 2024-05-10T10:37:23.017 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216827+0000', 'last_change': '2024-05-10T10:37:17.216827+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216827+0000', 'last_undegraded': '2024-05-10T10:37:17.216827+0000', 'last_fullsized': '2024-05-10T10:37:17.216827+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:PG 12.26 is not active+clean 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216132+0000', 'last_change': '2024-05-10T10:37:17.216132+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216132+0000', 'last_undegraded': '2024-05-10T10:37:17.216132+0000', 'last_fullsized': '2024-05-10T10:37:17.216132+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:PG 12.25 is not active+clean 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214571+0000', 'last_change': '2024-05-10T10:37:17.214571+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214571+0000', 'last_undegraded': '2024-05-10T10:37:17.214571+0000', 'last_fullsized': '2024-05-10T10:37:17.214571+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:PG 12.24 is not active+clean 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215835+0000', 'last_change': '2024-05-10T10:37:17.215835+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215835+0000', 'last_undegraded': '2024-05-10T10:37:17.215835+0000', 'last_fullsized': '2024-05-10T10:37:17.215835+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:PG 12.23 is not active+clean 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216848+0000', 'last_change': '2024-05-10T10:37:17.216848+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216848+0000', 'last_undegraded': '2024-05-10T10:37:17.216848+0000', 'last_fullsized': '2024-05-10T10:37:17.216848+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:PG 12.22 is not active+clean 2024-05-10T10:37:23.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.213937+0000', 'last_change': '2024-05-10T10:37:17.213937+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.213937+0000', 'last_undegraded': '2024-05-10T10:37:17.213937+0000', 'last_fullsized': '2024-05-10T10:37:17.213937+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:PG 12.21 is not active+clean 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217597+0000', 'last_change': '2024-05-10T10:37:17.217597+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217597+0000', 'last_undegraded': '2024-05-10T10:37:17.217597+0000', 'last_fullsized': '2024-05-10T10:37:17.217597+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:PG 12.20 is not active+clean 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214590+0000', 'last_change': '2024-05-10T10:37:17.214590+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214590+0000', 'last_undegraded': '2024-05-10T10:37:17.214590+0000', 'last_fullsized': '2024-05-10T10:37:17.214590+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:PG 12.18 is not active+clean 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.218472+0000', 'last_change': '2024-05-10T10:37:17.218472+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.218472+0000', 'last_undegraded': '2024-05-10T10:37:17.218472+0000', 'last_fullsized': '2024-05-10T10:37:17.218472+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:PG 12.19 is not active+clean 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214546+0000', 'last_change': '2024-05-10T10:37:17.214546+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214546+0000', 'last_undegraded': '2024-05-10T10:37:17.214546+0000', 'last_fullsized': '2024-05-10T10:37:17.214546+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1a is not active+clean 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.218554+0000', 'last_change': '2024-05-10T10:37:17.218554+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.218554+0000', 'last_undegraded': '2024-05-10T10:37:17.218554+0000', 'last_fullsized': '2024-05-10T10:37:17.218554+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1b is not active+clean 2024-05-10T10:37:23.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216317+0000', 'last_change': '2024-05-10T10:37:17.216317+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216317+0000', 'last_undegraded': '2024-05-10T10:37:17.216317+0000', 'last_fullsized': '2024-05-10T10:37:17.216317+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1c is not active+clean 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215764+0000', 'last_change': '2024-05-10T10:37:17.215764+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215764+0000', 'last_undegraded': '2024-05-10T10:37:17.215764+0000', 'last_fullsized': '2024-05-10T10:37:17.215764+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1d is not active+clean 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215567+0000', 'last_change': '2024-05-10T10:37:17.215567+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215567+0000', 'last_undegraded': '2024-05-10T10:37:17.215567+0000', 'last_fullsized': '2024-05-10T10:37:17.215567+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1e is not active+clean 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217227+0000', 'last_change': '2024-05-10T10:37:17.217227+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217227+0000', 'last_undegraded': '2024-05-10T10:37:17.217227+0000', 'last_fullsized': '2024-05-10T10:37:17.217227+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1f is not active+clean 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.223820+0000', 'last_change': '2024-05-10T10:37:17.223820+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.223820+0000', 'last_undegraded': '2024-05-10T10:37:17.223820+0000', 'last_fullsized': '2024-05-10T10:37:17.223820+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:PG 12.10 is not active+clean 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214560+0000', 'last_change': '2024-05-10T10:37:17.214560+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214560+0000', 'last_undegraded': '2024-05-10T10:37:17.214560+0000', 'last_fullsized': '2024-05-10T10:37:17.214560+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:PG 12.11 is not active+clean 2024-05-10T10:37:23.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215515+0000', 'last_change': '2024-05-10T10:37:17.215515+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215515+0000', 'last_undegraded': '2024-05-10T10:37:17.215515+0000', 'last_fullsized': '2024-05-10T10:37:17.215515+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:PG 12.12 is not active+clean 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216485+0000', 'last_change': '2024-05-10T10:37:17.216485+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216485+0000', 'last_undegraded': '2024-05-10T10:37:17.216485+0000', 'last_fullsized': '2024-05-10T10:37:17.216485+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:PG 12.13 is not active+clean 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.220258+0000', 'last_change': '2024-05-10T10:37:17.220258+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.220258+0000', 'last_undegraded': '2024-05-10T10:37:17.220258+0000', 'last_fullsized': '2024-05-10T10:37:17.220258+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:PG 12.14 is not active+clean 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215708+0000', 'last_change': '2024-05-10T10:37:17.215708+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215708+0000', 'last_undegraded': '2024-05-10T10:37:17.215708+0000', 'last_fullsized': '2024-05-10T10:37:17.215708+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:PG 12.15 is not active+clean 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.223632+0000', 'last_change': '2024-05-10T10:37:17.223632+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.223632+0000', 'last_undegraded': '2024-05-10T10:37:17.223632+0000', 'last_fullsized': '2024-05-10T10:37:17.223632+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:PG 12.16 is not active+clean 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.223403+0000', 'last_change': '2024-05-10T10:37:17.223403+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.223403+0000', 'last_undegraded': '2024-05-10T10:37:17.223403+0000', 'last_fullsized': '2024-05-10T10:37:17.223403+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:PG 12.17 is not active+clean 2024-05-10T10:37:23.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.222318+0000', 'last_change': '2024-05-10T10:37:17.222318+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.222318+0000', 'last_undegraded': '2024-05-10T10:37:17.222318+0000', 'last_fullsized': '2024-05-10T10:37:17.222318+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.022 INFO:tasks.ceph.ceph_manager.ceph:PG 12.8 is not active+clean 2024-05-10T10:37:23.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214436+0000', 'last_change': '2024-05-10T10:37:17.214436+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214436+0000', 'last_undegraded': '2024-05-10T10:37:17.214436+0000', 'last_fullsized': '2024-05-10T10:37:17.214436+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:37:23.022 INFO:tasks.ceph.ceph_manager.ceph:PG 12.9 is not active+clean 2024-05-10T10:37:23.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.217568+0000', 'last_change': '2024-05-10T10:37:17.217568+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.217568+0000', 'last_undegraded': '2024-05-10T10:37:17.217568+0000', 'last_fullsized': '2024-05-10T10:37:17.217568+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.022 INFO:tasks.ceph.ceph_manager.ceph:PG 12.a is not active+clean 2024-05-10T10:37:23.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216780+0000', 'last_change': '2024-05-10T10:37:17.216780+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216780+0000', 'last_undegraded': '2024-05-10T10:37:17.216780+0000', 'last_fullsized': '2024-05-10T10:37:17.216780+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:37:23.022 INFO:tasks.ceph.ceph_manager.ceph:PG 12.b is not active+clean 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215262+0000', 'last_change': '2024-05-10T10:37:17.215262+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215262+0000', 'last_undegraded': '2024-05-10T10:37:17.215262+0000', 'last_fullsized': '2024-05-10T10:37:17.215262+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:PG 12.c is not active+clean 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214673+0000', 'last_change': '2024-05-10T10:37:17.214673+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214673+0000', 'last_undegraded': '2024-05-10T10:37:17.214673+0000', 'last_fullsized': '2024-05-10T10:37:17.214673+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:PG 12.e is not active+clean 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.215169+0000', 'last_change': '2024-05-10T10:37:17.215169+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.215169+0000', 'last_undegraded': '2024-05-10T10:37:17.215169+0000', 'last_fullsized': '2024-05-10T10:37:17.215169+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:PG 12.f is not active+clean 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.221900+0000', 'last_change': '2024-05-10T10:37:17.221900+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.221900+0000', 'last_undegraded': '2024-05-10T10:37:17.221900+0000', 'last_fullsized': '2024-05-10T10:37:17.221900+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:PG 12.0 is not active+clean 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.216340+0000', 'last_change': '2024-05-10T10:37:17.216340+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.216340+0000', 'last_undegraded': '2024-05-10T10:37:17.216340+0000', 'last_fullsized': '2024-05-10T10:37:17.216340+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1 is not active+clean 2024-05-10T10:37:23.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.221621+0000', 'last_change': '2024-05-10T10:37:17.221621+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.221621+0000', 'last_undegraded': '2024-05-10T10:37:17.221621+0000', 'last_fullsized': '2024-05-10T10:37:17.221621+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2 is not active+clean 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.218157+0000', 'last_change': '2024-05-10T10:37:17.218157+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.218157+0000', 'last_undegraded': '2024-05-10T10:37:17.218157+0000', 'last_fullsized': '2024-05-10T10:37:17.218157+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3 is not active+clean 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.214140+0000', 'last_change': '2024-05-10T10:37:17.214140+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.214140+0000', 'last_undegraded': '2024-05-10T10:37:17.214140+0000', 'last_fullsized': '2024-05-10T10:37:17.214140+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:PG 12.4 is not active+clean 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.222142+0000', 'last_change': '2024-05-10T10:37:17.222142+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.222142+0000', 'last_undegraded': '2024-05-10T10:37:17.222142+0000', 'last_fullsized': '2024-05-10T10:37:17.222142+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:PG 12.5 is not active+clean 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.222569+0000', 'last_change': '2024-05-10T10:37:17.222569+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.222569+0000', 'last_undegraded': '2024-05-10T10:37:17.222569+0000', 'last_fullsized': '2024-05-10T10:37:17.222569+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:PG 12.6 is not active+clean 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.220334+0000', 'last_change': '2024-05-10T10:37:17.220334+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.220334+0000', 'last_undegraded': '2024-05-10T10:37:17.220334+0000', 'last_fullsized': '2024-05-10T10:37:17.220334+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:37:23.024 INFO:tasks.ceph.ceph_manager.ceph:PG 12.7 is not active+clean 2024-05-10T10:37:23.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.220751+0000', 'last_change': '2024-05-10T10:37:17.220751+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.220751+0000', 'last_undegraded': '2024-05-10T10:37:17.220751+0000', 'last_fullsized': '2024-05-10T10:37:17.220751+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:37:23.025 INFO:tasks.ceph.ceph_manager.ceph:PG 12.d is not active+clean 2024-05-10T10:37:23.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:37:17.213972+0000', 'last_change': '2024-05-10T10:37:17.213972+0000', 'last_active': '2024-05-10T10:37:17.201953+0000', 'last_peered': '2024-05-10T10:37:17.201953+0000', 'last_clean': '2024-05-10T10:37:17.201953+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:37:17.213972+0000', 'last_undegraded': '2024-05-10T10:37:17.213972+0000', 'last_fullsized': '2024-05-10T10:37:17.213972+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'last_clean_scrub_stamp': '2024-05-10T10:37:17.201953+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:37:23.025 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-10T10:37:23.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.149+0000 7facca9dd640 1 Processor -- start 2024-05-10T10:37:23.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.149+0000 7facca9dd640 1 -- start start 2024-05-10T10:37:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.149+0000 7facca9dd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7facc415f790 0x7facc40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.149+0000 7facca9dd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7facc40d5d30 0x7facc40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.149+0000 7facca9dd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc40d30b0 con 0x7facc415f3b0 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.149+0000 7facca9dd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7facc40d3390 con 0x7facc415f790 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.149+0000 7facca9dd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7facc40c8c80 con 0x7facc40d5d30 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7facc415f790 0x7facc40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7facc415f790 0x7facc40cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54136/0 (socket says 172.21.15.53:54136) 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc37fe640 1 -- 172.21.15.53:0/1972561754 learned_addr learned my addr 172.21.15.53:0/1972561754 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc8f7a640 1 --2- 172.21.15.53:0/1972561754 >> v2:172.21.15.135:3300/0 conn(0x7facc40d5d30 0x7facc40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc37fe640 1 -- 172.21.15.53:0/1972561754 >> v2:172.21.15.135:3300/0 conn(0x7facc40d5d30 msgr2=0x7facc40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc37fe640 1 --2- 172.21.15.53:0/1972561754 >> v2:172.21.15.135:3300/0 conn(0x7facc40d5d30 0x7facc40d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc37fe640 1 -- 172.21.15.53:0/1972561754 >> v1:172.21.15.135:6789/0 conn(0x7facc415f3b0 legacy=0x7facc40d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.150+0000 7facc37fe640 1 -- 172.21.15.53:0/1972561754 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40c8f60 con 0x7facc415f790 2024-05-10T10:37:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facc37fe640 1 --2- 172.21.15.53:0/1972561754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc415f790 0x7facc40cfcd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7facc408d220 tx=0x7facb802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42ee3b8f4193aac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facc2ffd640 1 -- 172.21.15.53:0/1972561754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8037070 con 0x7facc415f790 2024-05-10T10:37:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facc2ffd640 1 -- 172.21.15.53:0/1972561754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb8032440 con 0x7facc415f790 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facc2ffd640 1 -- 172.21.15.53:0/1972561754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8032810 con 0x7facc415f790 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 -- 172.21.15.53:0/1972561754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc415f790 msgr2=0x7facc40cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 --2- 172.21.15.53:0/1972561754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc415f790 0x7facc40cfcd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7facc408d220 tx=0x7facb802a230 comp rx=0 tx=0).stop 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 -- 172.21.15.53:0/1972561754 shutdown_connections 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 --2- 172.21.15.53:0/1972561754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc415f790 0x7facc40cfcd0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 --2- 172.21.15.53:0/1972561754 >> v2:172.21.15.135:3300/0 conn(0x7facc40d5d30 0x7facc40d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 -- 172.21.15.53:0/1972561754 >> 172.21.15.53:0/1972561754 conn(0x7facc40d5130 msgr2=0x7facc40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 -- 172.21.15.53:0/1972561754 shutdown_connections 2024-05-10T10:37:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.151+0000 7facca9dd640 1 -- 172.21.15.53:0/1972561754 wait complete. 2024-05-10T10:37:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.153+0000 7facca9dd640 1 Processor -- start 2024-05-10T10:37:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.153+0000 7facca9dd640 1 -- start start 2024-05-10T10:37:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40d5d30 0x7facc41f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facc415f3b0 0x7facc41504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facc41f58c0 0x7facc40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7facc40d3f80 con 0x7facc40d5d30 2024-05-10T10:37:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7facc40d3e00 con 0x7facc41f58c0 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7facc40ca770 con 0x7facc415f3b0 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40d5d30 0x7facc41f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40d5d30 0x7facc41f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54142/0 (socket says 172.21.15.53:54142) 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 -- 172.21.15.53:0/888959258 learned_addr learned my addr 172.21.15.53:0/888959258 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc8f7a640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facc41f58c0 0x7facc40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc37fe640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facc415f3b0 0x7facc41504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 -- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facc415f3b0 msgr2=0x7facc41504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facc415f3b0 0x7facc41504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 -- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facc41f58c0 msgr2=0x7facc40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facc41f58c0 0x7facc40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 -- 172.21.15.53:0/888959258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc41f6290 con 0x7facc40d5d30 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc3fff640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40d5d30 0x7facc41f5d50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7facb400c430 tx=0x7facb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc0ff9640 1 -- 172.21.15.53:0/888959258 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4011790 con 0x7facc40d5d30 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc0ff9640 1 -- 172.21.15.53:0/888959258 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb4017070 con 0x7facc40d5d30 2024-05-10T10:37:23.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc40cc640 con 0x7facc40d5d30 2024-05-10T10:37:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facc0ff9640 1 -- 172.21.15.53:0/888959258 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4011b70 con 0x7facc40d5d30 2024-05-10T10:37:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.154+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc40ccc00 con 0x7facc40d5d30 2024-05-10T10:37:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.155+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc406a220 con 0x7facc40d5d30 2024-05-10T10:37:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.156+0000 7facc0ff9640 1 -- 172.21.15.53:0/888959258 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7facb4011d10 con 0x7facc40d5d30 2024-05-10T10:37:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.156+0000 7facc0ff9640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7facb00762d0 0x7facb0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.157+0000 7facc0ff9640 1 -- 172.21.15.53:0/888959258 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7facb4097570 con 0x7facc40d5d30 2024-05-10T10:37:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.157+0000 7facc37fe640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7facb00762d0 0x7facb0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.157+0000 7facc37fe640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7facb00762d0 0x7facb0078790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7facb8034000 tx=0x7facb802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.161+0000 7facc0ff9640 1 -- 172.21.15.53:0/888959258 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7facb4060dd0 con 0x7facc40d5d30 2024-05-10T10:37:23.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.428+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7facc4150a30 con 0x7facc40d5d30 2024-05-10T10:37:23.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.428+0000 7facc0ff9640 1 -- 172.21.15.53:0/888959258 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1034 (secure 0 0 0) 0x7facb40606d0 con 0x7facc40d5d30 2024-05-10T10:37:23.429 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:23.429 INFO:teuthology.orchestra.run.smithi053.stdout:{"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":245,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1715337207,"num_in_osds":8,"osd_in_since":1715337199,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":246665216,"bytes_avail":772847448064,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":45,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6102}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-05-10T10:37:06.207012+0000","services":{}},"progress_events":{}} 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7facb00762d0 msgr2=0x7facb0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7facb00762d0 0x7facb0078790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7facb8034000 tx=0x7facb802e000 comp rx=0 tx=0).stop 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40d5d30 msgr2=0x7facc41f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40d5d30 0x7facc41f5d50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7facb400c430 tx=0x7facb400c8f0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 shutdown_connections 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7facb00762d0 0x7facb0078790 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facc41f58c0 0x7facc40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facc415f3b0 0x7facc41504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 --2- 172.21.15.53:0/888959258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40d5d30 0x7facc41f5d50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 >> 172.21.15.53:0/888959258 conn(0x7facc40d5130 msgr2=0x7facc40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 shutdown_connections 2024-05-10T10:37:23.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.430+0000 7facca9dd640 1 -- 172.21.15.53:0/888959258 wait complete. 2024-05-10T10:37:23.442 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-10T10:37:23.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:37:23.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.545+0000 7f934c1dd640 1 Processor -- start 2024-05-10T10:37:23.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.545+0000 7f934c1dd640 1 -- start start 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f934c1dd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93440d0230 0x7f93440cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f934c1dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f934c1dd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f934415fc90 con 0x7f93440b8860 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f934c1dd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f934406ec30 con 0x7f93440d0230 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f934c1dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f934406ee50 con 0x7f93440cfbb0 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f9349778640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93440d0230 0x7f93440cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f9349778640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93440d0230 0x7f93440cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54144/0 (socket says 172.21.15.53:54144) 2024-05-10T10:37:23.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f9349778640 1 -- 172.21.15.53:0/818690074 learned_addr learned my addr 172.21.15.53:0/818690074 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:23.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f934a77a640 1 --2- 172.21.15.53:0/818690074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f9349778640 1 -- 172.21.15.53:0/818690074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 msgr2=0x7f93440ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:23.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f9349778640 1 --2- 172.21.15.53:0/818690074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f9349778640 1 -- 172.21.15.53:0/818690074 >> v1:172.21.15.135:6789/0 conn(0x7f93440b8860 legacy=0x7f934406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.546+0000 7f9349778640 1 -- 172.21.15.53:0/818690074 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93440b8c40 con 0x7f93440d0230 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.547+0000 7f9349778640 1 --2- 172.21.15.53:0/818690074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440d0230 0x7f93440cf670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f93340098a0 tx=0x7f933402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4f83d17372bcfd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.547+0000 7f9348f77640 1 -- 172.21.15.53:0/818690074 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334037070 con 0x7f93440d0230 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.547+0000 7f9348f77640 1 -- 172.21.15.53:0/818690074 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9334032440 con 0x7f93440d0230 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.547+0000 7f9348f77640 1 -- 172.21.15.53:0/818690074 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334032810 con 0x7f93440d0230 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 -- 172.21.15.53:0/818690074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440d0230 msgr2=0x7f93440cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 --2- 172.21.15.53:0/818690074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440d0230 0x7f93440cf670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f93340098a0 tx=0x7f933402a200 comp rx=0 tx=0).stop 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 -- 172.21.15.53:0/818690074 shutdown_connections 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 --2- 172.21.15.53:0/818690074 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 --2- 172.21.15.53:0/818690074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440d0230 0x7f93440cf670 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 -- 172.21.15.53:0/818690074 >> 172.21.15.53:0/818690074 conn(0x7f93440d2df0 msgr2=0x7f93440d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 -- 172.21.15.53:0/818690074 shutdown_connections 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.548+0000 7f934c1dd640 1 -- 172.21.15.53:0/818690074 wait complete. 2024-05-10T10:37:23.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.549+0000 7f934c1dd640 1 Processor -- start 2024-05-10T10:37:23.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.549+0000 7f934c1dd640 1 -- start start 2024-05-10T10:37:23.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440b8860 0x7f9344161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93440d0230 0x7f9344156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93440cdb30 con 0x7f93440b8860 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93440cd850 con 0x7f93440d0230 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f93440cde10 con 0x7f93440cfbb0 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440b8860 0x7f9344161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440b8860 0x7f9344161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54158/0 (socket says 172.21.15.53:54158) 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 -- 172.21.15.53:0/3503479888 learned_addr learned my addr 172.21.15.53:0/3503479888 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934a77a640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93440d0230 0x7f9344156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349778640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 -- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 msgr2=0x7f93440cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 -- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93440d0230 msgr2=0x7f9344156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93440d0230 0x7f9344156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 -- 172.21.15.53:0/3503479888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9344161e90 con 0x7f93440b8860 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f9349f79640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440b8860 0x7f9344161810 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f934000c400 tx=0x7f934000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f933a7fc640 1 -- 172.21.15.53:0/3503479888 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400052d0 con 0x7f93440b8860 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f933a7fc640 1 -- 172.21.15.53:0/3503479888 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9340005490 con 0x7f93440b8860 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9344162170 con 0x7f93440b8860 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f933a7fc640 1 -- 172.21.15.53:0/3503479888 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340007260 con 0x7f93440b8860 2024-05-10T10:37:23.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.550+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9344162730 con 0x7f93440b8860 2024-05-10T10:37:23.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.551+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934406a220 con 0x7f93440b8860 2024-05-10T10:37:23.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.552+0000 7f933a7fc640 1 -- 172.21.15.53:0/3503479888 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9340007400 con 0x7f93440b8860 2024-05-10T10:37:23.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.553+0000 7f933a7fc640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93280762d0 0x7f9328078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:23.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.553+0000 7f933a7fc640 1 -- 172.21.15.53:0/3503479888 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f9340096ce0 con 0x7f93440b8860 2024-05-10T10:37:23.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.553+0000 7f9349778640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93280762d0 0x7f9328078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:23.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.554+0000 7f9349778640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93280762d0 0x7f9328078790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9334034000 tx=0x7f933402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:23.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.557+0000 7f933a7fc640 1 -- 172.21.15.53:0/3503479888 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f93400604b0 con 0x7f93440b8860 2024-05-10T10:37:23.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.747+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93440d1e20 con 0x7f93280762d0 2024-05-10T10:37:23.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.753+0000 7f933a7fc640 1 -- 172.21.15.53:0/3503479888 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246740 (secure 0 0 0) 0x7f93440d1e20 con 0x7f93280762d0 2024-05-10T10:37:23.756 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:23.757 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93280762d0 msgr2=0x7f9328078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93280762d0 0x7f9328078790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9334034000 tx=0x7f933402e000 comp rx=0 tx=0).stop 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440b8860 msgr2=0x7f9344161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440b8860 0x7f9344161810 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f934000c400 tx=0x7f934000c8c0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 shutdown_connections 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f93280762d0 0x7f9328078790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93440d0230 0x7f9344156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93440cfbb0 0x7f93440cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 --2- 172.21.15.53:0/3503479888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93440b8860 0x7f9344161810 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:23.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 >> 172.21.15.53:0/3503479888 conn(0x7f93440d2df0 msgr2=0x7f93440d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:23.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 shutdown_connections 2024-05-10T10:37:23.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:23.756+0000 7f934c1dd640 1 -- 172.21.15.53:0/3503479888 wait complete. 2024-05-10T10:37:23.768 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-05-10T10:37:22.211124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240884,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733836,"statfs":{"total":773094113280,"available":772847448064,"internally_reserved":0,"allocated":8232960,"data_stored":4349152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001535"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214356+0000","last_change":"2024-05-10T10:37:17.214356+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214356+0000","last_undegraded":"2024-05-10T10:37:17.214356+0000","last_fullsized":"2024-05-10T10:37:17.214356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218881+0000","last_change":"2024-05-10T10:37:17.218881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218881+0000","last_undegraded":"2024-05-10T10:37:17.218881+0000","last_fullsized":"2024-05-10T10:37:17.218881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219561+0000","last_change":"2024-05-10T10:37:17.219561+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219561+0000","last_undegraded":"2024-05-10T10:37:17.219561+0000","last_fullsized":"2024-05-10T10:37:17.219561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220528+0000","last_change":"2024-05-10T10:37:17.220528+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220528+0000","last_undegraded":"2024-05-10T10:37:17.220528+0000","last_fullsized":"2024-05-10T10:37:17.220528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217963+0000","last_change":"2024-05-10T10:37:17.217963+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217963+0000","last_undegraded":"2024-05-10T10:37:17.217963+0000","last_fullsized":"2024-05-10T10:37:17.217963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220201+0000","last_change":"2024-05-10T10:37:17.220201+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220201+0000","last_undegraded":"2024-05-10T10:37:17.220201+0000","last_fullsized":"2024-05-10T10:37:17.220201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222149+0000","last_change":"2024-05-10T10:37:17.222149+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222149+0000","last_undegraded":"2024-05-10T10:37:17.222149+0000","last_fullsized":"2024-05-10T10:37:17.222149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215093+0000","last_change":"2024-05-10T10:37:17.215093+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215093+0000","last_undegraded":"2024-05-10T10:37:17.215093+0000","last_fullsized":"2024-05-10T10:37:17.215093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216792+0000","last_change":"2024-05-10T10:37:17.216792+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216792+0000","last_undegraded":"2024-05-10T10:37:17.216792+0000","last_fullsized":"2024-05-10T10:37:17.216792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218788+0000","last_change":"2024-05-10T10:37:17.218788+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218788+0000","last_undegraded":"2024-05-10T10:37:17.218788+0000","last_fullsized":"2024-05-10T10:37:17.218788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220858+0000","last_change":"2024-05-10T10:37:17.220858+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220858+0000","last_undegraded":"2024-05-10T10:37:17.220858+0000","last_fullsized":"2024-05-10T10:37:17.220858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218915+0000","last_change":"2024-05-10T10:37:17.218915+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218915+0000","last_undegraded":"2024-05-10T10:37:17.218915+0000","last_fullsized":"2024-05-10T10:37:17.218915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216194+0000","last_change":"2024-05-10T10:37:17.216194+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216194+0000","last_undegraded":"2024-05-10T10:37:17.216194+0000","last_fullsized":"2024-05-10T10:37:17.216194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.219010+0000","last_change":"2024-05-10T10:37:17.219010+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.219010+0000","last_undegraded":"2024-05-10T10:37:17.219010+0000","last_fullsized":"2024-05-10T10:37:17.219010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217797+0000","last_change":"2024-05-10T10:37:17.217797+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217797+0000","last_undegraded":"2024-05-10T10:37:17.217797+0000","last_fullsized":"2024-05-10T10:37:17.217797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215236+0000","last_change":"2024-05-10T10:37:17.215236+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215236+0000","last_undegraded":"2024-05-10T10:37:17.215236+0000","last_fullsized":"2024-05-10T10:37:17.215236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217685+0000","last_change":"2024-05-10T10:37:17.217685+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217685+0000","last_undegraded":"2024-05-10T10:37:17.217685+0000","last_fullsized":"2024-05-10T10:37:17.217685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220781+0000","last_change":"2024-05-10T10:37:17.220781+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220781+0000","last_undegraded":"2024-05-10T10:37:17.220781+0000","last_fullsized":"2024-05-10T10:37:17.220781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215713+0000","last_change":"2024-05-10T10:37:17.215713+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215713+0000","last_undegraded":"2024-05-10T10:37:17.215713+0000","last_fullsized":"2024-05-10T10:37:17.215713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217141+0000","last_change":"2024-05-10T10:37:17.217141+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217141+0000","last_undegraded":"2024-05-10T10:37:17.217141+0000","last_fullsized":"2024-05-10T10:37:17.217141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217274+0000","last_change":"2024-05-10T10:37:17.217274+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217274+0000","last_undegraded":"2024-05-10T10:37:17.217274+0000","last_fullsized":"2024-05-10T10:37:17.217274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215881+0000","last_change":"2024-05-10T10:37:17.215881+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215881+0000","last_undegraded":"2024-05-10T10:37:17.215881+0000","last_fullsized":"2024-05-10T10:37:17.215881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218122+0000","last_change":"2024-05-10T10:37:17.218122+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218122+0000","last_undegraded":"2024-05-10T10:37:17.218122+0000","last_fullsized":"2024-05-10T10:37:17.218122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217328+0000","last_change":"2024-05-10T10:37:17.217328+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217328+0000","last_undegraded":"2024-05-10T10:37:17.217328+0000","last_fullsized":"2024-05-10T10:37:17.217328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216827+0000","last_change":"2024-05-10T10:37:17.216827+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216827+0000","last_undegraded":"2024-05-10T10:37:17.216827+0000","last_fullsized":"2024-05-10T10:37:17.216827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216132+0000","last_change":"2024-05-10T10:37:17.216132+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216132+0000","last_undegraded":"2024-05-10T10:37:17.216132+0000","last_fullsized":"2024-05-10T10:37:17.216132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214571+0000","last_change":"2024-05-10T10:37:17.214571+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214571+0000","last_undegraded":"2024-05-10T10:37:17.214571+0000","last_fullsized":"2024-05-10T10:37:17.214571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215835+0000","last_change":"2024-05-10T10:37:17.215835+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215835+0000","last_undegraded":"2024-05-10T10:37:17.215835+0000","last_fullsized":"2024-05-10T10:37:17.215835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216848+0000","last_change":"2024-05-10T10:37:17.216848+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216848+0000","last_undegraded":"2024-05-10T10:37:17.216848+0000","last_fullsized":"2024-05-10T10:37:17.216848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213937+0000","last_change":"2024-05-10T10:37:17.213937+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213937+0000","last_undegraded":"2024-05-10T10:37:17.213937+0000","last_fullsized":"2024-05-10T10:37:17.213937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217597+0000","last_change":"2024-05-10T10:37:17.217597+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217597+0000","last_undegraded":"2024-05-10T10:37:17.217597+0000","last_fullsized":"2024-05-10T10:37:17.217597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214590+0000","last_change":"2024-05-10T10:37:17.214590+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214590+0000","last_undegraded":"2024-05-10T10:37:17.214590+0000","last_fullsized":"2024-05-10T10:37:17.214590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218472+0000","last_change":"2024-05-10T10:37:17.218472+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218472+0000","last_undegraded":"2024-05-10T10:37:17.218472+0000","last_fullsized":"2024-05-10T10:37:17.218472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234893+0000","last_change":"2024-05-10T10:37:17.234091+0000","last_active":"2024-05-10T10:37:17.234893+0000","last_peered":"2024-05-10T10:37:17.234893+0000","last_clean":"2024-05-10T10:37:17.234893+0000","last_became_active":"2024-05-10T10:37:17.233792+0000","last_became_peered":"2024-05-10T10:37:17.233792+0000","last_unstale":"2024-05-10T10:37:17.234893+0000","last_undegraded":"2024-05-10T10:37:17.234893+0000","last_fullsized":"2024-05-10T10:37:17.234893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:15:16.625576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214546+0000","last_change":"2024-05-10T10:37:17.214546+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214546+0000","last_undegraded":"2024-05-10T10:37:17.214546+0000","last_fullsized":"2024-05-10T10:37:17.214546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233815+0000","last_change":"2024-05-10T10:37:17.229429+0000","last_active":"2024-05-10T10:37:17.233815+0000","last_peered":"2024-05-10T10:37:17.233815+0000","last_clean":"2024-05-10T10:37:17.233815+0000","last_became_active":"2024-05-10T10:37:17.229121+0000","last_became_peered":"2024-05-10T10:37:17.229121+0000","last_unstale":"2024-05-10T10:37:17.233815+0000","last_undegraded":"2024-05-10T10:37:17.233815+0000","last_fullsized":"2024-05-10T10:37:17.233815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:12:19.839294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218554+0000","last_change":"2024-05-10T10:37:17.218554+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218554+0000","last_undegraded":"2024-05-10T10:37:17.218554+0000","last_fullsized":"2024-05-10T10:37:17.218554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229641+0000","last_change":"2024-05-10T10:37:17.225715+0000","last_active":"2024-05-10T10:37:17.229641+0000","last_peered":"2024-05-10T10:37:17.229641+0000","last_clean":"2024-05-10T10:37:17.229641+0000","last_became_active":"2024-05-10T10:37:17.225250+0000","last_became_peered":"2024-05-10T10:37:17.225250+0000","last_unstale":"2024-05-10T10:37:17.229641+0000","last_undegraded":"2024-05-10T10:37:17.229641+0000","last_fullsized":"2024-05-10T10:37:17.229641+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:39.904069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216317+0000","last_change":"2024-05-10T10:37:17.216317+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216317+0000","last_undegraded":"2024-05-10T10:37:17.216317+0000","last_fullsized":"2024-05-10T10:37:17.216317+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231454+0000","last_change":"2024-05-10T10:37:17.228203+0000","last_active":"2024-05-10T10:37:17.231454+0000","last_peered":"2024-05-10T10:37:17.231454+0000","last_clean":"2024-05-10T10:37:17.231454+0000","last_became_active":"2024-05-10T10:37:17.227939+0000","last_became_peered":"2024-05-10T10:37:17.227939+0000","last_unstale":"2024-05-10T10:37:17.231454+0000","last_undegraded":"2024-05-10T10:37:17.231454+0000","last_fullsized":"2024-05-10T10:37:17.231454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:19.500154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215764+0000","last_change":"2024-05-10T10:37:17.215764+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215764+0000","last_undegraded":"2024-05-10T10:37:17.215764+0000","last_fullsized":"2024-05-10T10:37:17.215764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.225465+0000","last_change":"2024-05-10T10:37:17.222699+0000","last_active":"2024-05-10T10:37:17.225465+0000","last_peered":"2024-05-10T10:37:17.225465+0000","last_clean":"2024-05-10T10:37:17.225465+0000","last_became_active":"2024-05-10T10:37:17.222389+0000","last_became_peered":"2024-05-10T10:37:17.222389+0000","last_unstale":"2024-05-10T10:37:17.225465+0000","last_undegraded":"2024-05-10T10:37:17.225465+0000","last_fullsized":"2024-05-10T10:37:17.225465+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:59:22.414675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215567+0000","last_change":"2024-05-10T10:37:17.215567+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215567+0000","last_undegraded":"2024-05-10T10:37:17.215567+0000","last_fullsized":"2024-05-10T10:37:17.215567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231160+0000","last_change":"2024-05-10T10:37:17.227524+0000","last_active":"2024-05-10T10:37:17.231160+0000","last_peered":"2024-05-10T10:37:17.231160+0000","last_clean":"2024-05-10T10:37:17.231160+0000","last_became_active":"2024-05-10T10:37:17.227256+0000","last_became_peered":"2024-05-10T10:37:17.227256+0000","last_unstale":"2024-05-10T10:37:17.231160+0000","last_undegraded":"2024-05-10T10:37:17.231160+0000","last_fullsized":"2024-05-10T10:37:17.231160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217227+0000","last_change":"2024-05-10T10:37:17.217227+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217227+0000","last_undegraded":"2024-05-10T10:37:17.217227+0000","last_fullsized":"2024-05-10T10:37:17.217227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.222393+0000","last_change":"2024-05-10T10:37:17.218206+0000","last_active":"2024-05-10T10:37:17.222393+0000","last_peered":"2024-05-10T10:37:17.222393+0000","last_clean":"2024-05-10T10:37:17.222393+0000","last_became_active":"2024-05-10T10:37:17.217591+0000","last_became_peered":"2024-05-10T10:37:17.217591+0000","last_unstale":"2024-05-10T10:37:17.222393+0000","last_undegraded":"2024-05-10T10:37:17.222393+0000","last_fullsized":"2024-05-10T10:37:17.222393+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:15:45.683363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223820+0000","last_change":"2024-05-10T10:37:17.223820+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223820+0000","last_undegraded":"2024-05-10T10:37:17.223820+0000","last_fullsized":"2024-05-10T10:37:17.223820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231519+0000","last_change":"2024-05-10T10:37:17.230114+0000","last_active":"2024-05-10T10:37:17.231519+0000","last_peered":"2024-05-10T10:37:17.231519+0000","last_clean":"2024-05-10T10:37:17.231519+0000","last_became_active":"2024-05-10T10:37:17.229847+0000","last_became_peered":"2024-05-10T10:37:17.229847+0000","last_unstale":"2024-05-10T10:37:17.231519+0000","last_undegraded":"2024-05-10T10:37:17.231519+0000","last_fullsized":"2024-05-10T10:37:17.231519+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:47:31.740584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214560+0000","last_change":"2024-05-10T10:37:17.214560+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214560+0000","last_undegraded":"2024-05-10T10:37:17.214560+0000","last_fullsized":"2024-05-10T10:37:17.214560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227281+0000","last_change":"2024-05-10T10:37:17.224805+0000","last_active":"2024-05-10T10:37:17.227281+0000","last_peered":"2024-05-10T10:37:17.227281+0000","last_clean":"2024-05-10T10:37:17.227281+0000","last_became_active":"2024-05-10T10:37:17.224508+0000","last_became_peered":"2024-05-10T10:37:17.224508+0000","last_unstale":"2024-05-10T10:37:17.227281+0000","last_undegraded":"2024-05-10T10:37:17.227281+0000","last_fullsized":"2024-05-10T10:37:17.227281+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215515+0000","last_change":"2024-05-10T10:37:17.215515+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215515+0000","last_undegraded":"2024-05-10T10:37:17.215515+0000","last_fullsized":"2024-05-10T10:37:17.215515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228363+0000","last_change":"2024-05-10T10:37:17.225567+0000","last_active":"2024-05-10T10:37:17.228363+0000","last_peered":"2024-05-10T10:37:17.228363+0000","last_clean":"2024-05-10T10:37:17.228363+0000","last_became_active":"2024-05-10T10:37:17.225301+0000","last_became_peered":"2024-05-10T10:37:17.225301+0000","last_unstale":"2024-05-10T10:37:17.228363+0000","last_undegraded":"2024-05-10T10:37:17.228363+0000","last_fullsized":"2024-05-10T10:37:17.228363+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216485+0000","last_change":"2024-05-10T10:37:17.216485+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216485+0000","last_undegraded":"2024-05-10T10:37:17.216485+0000","last_fullsized":"2024-05-10T10:37:17.216485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.232470+0000","last_change":"2024-05-10T10:37:17.231914+0000","last_active":"2024-05-10T10:37:17.232470+0000","last_peered":"2024-05-10T10:37:17.232470+0000","last_clean":"2024-05-10T10:37:17.232470+0000","last_became_active":"2024-05-10T10:37:17.231631+0000","last_became_peered":"2024-05-10T10:37:17.231631+0000","last_unstale":"2024-05-10T10:37:17.232470+0000","last_undegraded":"2024-05-10T10:37:17.232470+0000","last_fullsized":"2024-05-10T10:37:17.232470+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:01:34.692557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220258+0000","last_change":"2024-05-10T10:37:17.220258+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220258+0000","last_undegraded":"2024-05-10T10:37:17.220258+0000","last_fullsized":"2024-05-10T10:37:17.220258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233970+0000","last_change":"2024-05-10T10:37:17.229898+0000","last_active":"2024-05-10T10:37:17.233970+0000","last_peered":"2024-05-10T10:37:17.233970+0000","last_clean":"2024-05-10T10:37:17.233970+0000","last_became_active":"2024-05-10T10:37:17.229599+0000","last_became_peered":"2024-05-10T10:37:17.229599+0000","last_unstale":"2024-05-10T10:37:17.233970+0000","last_undegraded":"2024-05-10T10:37:17.233970+0000","last_fullsized":"2024-05-10T10:37:17.233970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:30.149365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215708+0000","last_change":"2024-05-10T10:37:17.215708+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215708+0000","last_undegraded":"2024-05-10T10:37:17.215708+0000","last_fullsized":"2024-05-10T10:37:17.215708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228415+0000","last_change":"2024-05-10T10:37:17.223996+0000","last_active":"2024-05-10T10:37:17.228415+0000","last_peered":"2024-05-10T10:37:17.228415+0000","last_clean":"2024-05-10T10:37:17.228415+0000","last_became_active":"2024-05-10T10:37:17.223309+0000","last_became_peered":"2024-05-10T10:37:17.223309+0000","last_unstale":"2024-05-10T10:37:17.228415+0000","last_undegraded":"2024-05-10T10:37:17.228415+0000","last_fullsized":"2024-05-10T10:37:17.228415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:11.520143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223632+0000","last_change":"2024-05-10T10:37:17.223632+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223632+0000","last_undegraded":"2024-05-10T10:37:17.223632+0000","last_fullsized":"2024-05-10T10:37:17.223632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229437+0000","last_change":"2024-05-10T10:37:17.226617+0000","last_active":"2024-05-10T10:37:17.229437+0000","last_peered":"2024-05-10T10:37:17.229437+0000","last_clean":"2024-05-10T10:37:17.229437+0000","last_became_active":"2024-05-10T10:37:17.226339+0000","last_became_peered":"2024-05-10T10:37:17.226339+0000","last_unstale":"2024-05-10T10:37:17.229437+0000","last_undegraded":"2024-05-10T10:37:17.229437+0000","last_fullsized":"2024-05-10T10:37:17.229437+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:36:44.271136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.223403+0000","last_change":"2024-05-10T10:37:17.223403+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.223403+0000","last_undegraded":"2024-05-10T10:37:17.223403+0000","last_fullsized":"2024-05-10T10:37:17.223403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234908+0000","last_change":"2024-05-10T10:37:17.234183+0000","last_active":"2024-05-10T10:37:17.234908+0000","last_peered":"2024-05-10T10:37:17.234908+0000","last_clean":"2024-05-10T10:37:17.234908+0000","last_became_active":"2024-05-10T10:37:17.233852+0000","last_became_peered":"2024-05-10T10:37:17.233852+0000","last_unstale":"2024-05-10T10:37:17.234908+0000","last_undegraded":"2024-05-10T10:37:17.234908+0000","last_fullsized":"2024-05-10T10:37:17.234908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:57:04.326406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222318+0000","last_change":"2024-05-10T10:37:17.222318+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222318+0000","last_undegraded":"2024-05-10T10:37:17.222318+0000","last_fullsized":"2024-05-10T10:37:17.222318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.227850+0000","last_change":"2024-05-10T10:37:17.224923+0000","last_active":"2024-05-10T10:37:17.227850+0000","last_peered":"2024-05-10T10:37:17.227850+0000","last_clean":"2024-05-10T10:37:17.227850+0000","last_became_active":"2024-05-10T10:37:17.224594+0000","last_became_peered":"2024-05-10T10:37:17.224594+0000","last_unstale":"2024-05-10T10:37:17.227850+0000","last_undegraded":"2024-05-10T10:37:17.227850+0000","last_fullsized":"2024-05-10T10:37:17.227850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:58:38.099023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214436+0000","last_change":"2024-05-10T10:37:17.214436+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214436+0000","last_undegraded":"2024-05-10T10:37:17.214436+0000","last_fullsized":"2024-05-10T10:37:17.214436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229429+0000","last_change":"2024-05-10T10:37:17.225719+0000","last_active":"2024-05-10T10:37:17.229429+0000","last_peered":"2024-05-10T10:37:17.229429+0000","last_clean":"2024-05-10T10:37:17.229429+0000","last_became_active":"2024-05-10T10:37:17.225430+0000","last_became_peered":"2024-05-10T10:37:17.225430+0000","last_unstale":"2024-05-10T10:37:17.229429+0000","last_undegraded":"2024-05-10T10:37:17.229429+0000","last_fullsized":"2024-05-10T10:37:17.229429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:09.884646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.217568+0000","last_change":"2024-05-10T10:37:17.217568+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.217568+0000","last_undegraded":"2024-05-10T10:37:17.217568+0000","last_fullsized":"2024-05-10T10:37:17.217568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228794+0000","last_change":"2024-05-10T10:37:17.226246+0000","last_active":"2024-05-10T10:37:17.228794+0000","last_peered":"2024-05-10T10:37:17.228794+0000","last_clean":"2024-05-10T10:37:17.228794+0000","last_became_active":"2024-05-10T10:37:17.225996+0000","last_became_peered":"2024-05-10T10:37:17.225996+0000","last_unstale":"2024-05-10T10:37:17.228794+0000","last_undegraded":"2024-05-10T10:37:17.228794+0000","last_fullsized":"2024-05-10T10:37:17.228794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:55:53.951642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216780+0000","last_change":"2024-05-10T10:37:17.216780+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216780+0000","last_undegraded":"2024-05-10T10:37:17.216780+0000","last_fullsized":"2024-05-10T10:37:17.216780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229357+0000","last_change":"2024-05-10T10:37:17.223948+0000","last_active":"2024-05-10T10:37:17.229357+0000","last_peered":"2024-05-10T10:37:17.229357+0000","last_clean":"2024-05-10T10:37:17.229357+0000","last_became_active":"2024-05-10T10:37:17.223293+0000","last_became_peered":"2024-05-10T10:37:17.223293+0000","last_unstale":"2024-05-10T10:37:17.229357+0000","last_undegraded":"2024-05-10T10:37:17.229357+0000","last_fullsized":"2024-05-10T10:37:17.229357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:31:08.802431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215262+0000","last_change":"2024-05-10T10:37:17.215262+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215262+0000","last_undegraded":"2024-05-10T10:37:17.215262+0000","last_fullsized":"2024-05-10T10:37:17.215262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.234053+0000","last_change":"2024-05-10T10:37:17.232761+0000","last_active":"2024-05-10T10:37:17.234053+0000","last_peered":"2024-05-10T10:37:17.234053+0000","last_clean":"2024-05-10T10:37:17.234053+0000","last_became_active":"2024-05-10T10:37:17.232401+0000","last_became_peered":"2024-05-10T10:37:17.232401+0000","last_unstale":"2024-05-10T10:37:17.234053+0000","last_undegraded":"2024-05-10T10:37:17.234053+0000","last_fullsized":"2024-05-10T10:37:17.234053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:39:14.930472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214673+0000","last_change":"2024-05-10T10:37:17.214673+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214673+0000","last_undegraded":"2024-05-10T10:37:17.214673+0000","last_fullsized":"2024-05-10T10:37:17.214673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233192+0000","last_change":"2024-05-10T10:37:17.227272+0000","last_active":"2024-05-10T10:37:17.233192+0000","last_peered":"2024-05-10T10:37:17.233192+0000","last_clean":"2024-05-10T10:37:17.233192+0000","last_became_active":"2024-05-10T10:37:17.224267+0000","last_became_peered":"2024-05-10T10:37:17.224267+0000","last_unstale":"2024-05-10T10:37:17.233192+0000","last_undegraded":"2024-05-10T10:37:17.233192+0000","last_fullsized":"2024-05-10T10:37:17.233192+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:23:59.926354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.215169+0000","last_change":"2024-05-10T10:37:17.215169+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.215169+0000","last_undegraded":"2024-05-10T10:37:17.215169+0000","last_fullsized":"2024-05-10T10:37:17.215169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233428+0000","last_change":"2024-05-10T10:37:17.230132+0000","last_active":"2024-05-10T10:37:17.233428+0000","last_peered":"2024-05-10T10:37:17.233428+0000","last_clean":"2024-05-10T10:37:17.233428+0000","last_became_active":"2024-05-10T10:37:17.229845+0000","last_became_peered":"2024-05-10T10:37:17.229845+0000","last_unstale":"2024-05-10T10:37:17.233428+0000","last_undegraded":"2024-05-10T10:37:17.233428+0000","last_fullsized":"2024-05-10T10:37:17.233428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:16:39.529753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221900+0000","last_change":"2024-05-10T10:37:17.221900+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221900+0000","last_undegraded":"2024-05-10T10:37:17.221900+0000","last_fullsized":"2024-05-10T10:37:17.221900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233394+0000","last_change":"2024-05-10T10:37:17.229451+0000","last_active":"2024-05-10T10:37:17.233394+0000","last_peered":"2024-05-10T10:37:17.233394+0000","last_clean":"2024-05-10T10:37:17.233394+0000","last_became_active":"2024-05-10T10:37:17.228407+0000","last_became_peered":"2024-05-10T10:37:17.228407+0000","last_unstale":"2024-05-10T10:37:17.233394+0000","last_undegraded":"2024-05-10T10:37:17.233394+0000","last_fullsized":"2024-05-10T10:37:17.233394+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:43:25.143088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.216340+0000","last_change":"2024-05-10T10:37:17.216340+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.216340+0000","last_undegraded":"2024-05-10T10:37:17.216340+0000","last_fullsized":"2024-05-10T10:37:17.216340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228381+0000","last_change":"2024-05-10T10:37:17.224792+0000","last_active":"2024-05-10T10:37:17.228381+0000","last_peered":"2024-05-10T10:37:17.228381+0000","last_clean":"2024-05-10T10:37:17.228381+0000","last_became_active":"2024-05-10T10:37:17.224443+0000","last_became_peered":"2024-05-10T10:37:17.224443+0000","last_unstale":"2024-05-10T10:37:17.228381+0000","last_undegraded":"2024-05-10T10:37:17.228381+0000","last_fullsized":"2024-05-10T10:37:17.228381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:13.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.221621+0000","last_change":"2024-05-10T10:37:17.221621+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.221621+0000","last_undegraded":"2024-05-10T10:37:17.221621+0000","last_fullsized":"2024-05-10T10:37:17.221621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.230785+0000","last_change":"2024-05-10T10:37:17.230099+0000","last_active":"2024-05-10T10:37:17.230785+0000","last_peered":"2024-05-10T10:37:17.230785+0000","last_clean":"2024-05-10T10:37:17.230785+0000","last_became_active":"2024-05-10T10:37:17.229704+0000","last_became_peered":"2024-05-10T10:37:17.229704+0000","last_unstale":"2024-05-10T10:37:17.230785+0000","last_undegraded":"2024-05-10T10:37:17.230785+0000","last_fullsized":"2024-05-10T10:37:17.230785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:43:08.567175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.218157+0000","last_change":"2024-05-10T10:37:17.218157+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.218157+0000","last_undegraded":"2024-05-10T10:37:17.218157+0000","last_fullsized":"2024-05-10T10:37:17.218157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229670+0000","last_change":"2024-05-10T10:37:17.226288+0000","last_active":"2024-05-10T10:37:17.229670+0000","last_peered":"2024-05-10T10:37:17.229670+0000","last_clean":"2024-05-10T10:37:17.229670+0000","last_became_active":"2024-05-10T10:37:17.226017+0000","last_became_peered":"2024-05-10T10:37:17.226017+0000","last_unstale":"2024-05-10T10:37:17.229670+0000","last_undegraded":"2024-05-10T10:37:17.229670+0000","last_fullsized":"2024-05-10T10:37:17.229670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:50.410699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.214140+0000","last_change":"2024-05-10T10:37:17.214140+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.214140+0000","last_undegraded":"2024-05-10T10:37:17.214140+0000","last_fullsized":"2024-05-10T10:37:17.214140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229395+0000","last_change":"2024-05-10T10:37:17.224891+0000","last_active":"2024-05-10T10:37:17.229395+0000","last_peered":"2024-05-10T10:37:17.229395+0000","last_clean":"2024-05-10T10:37:17.229395+0000","last_became_active":"2024-05-10T10:37:17.224639+0000","last_became_peered":"2024-05-10T10:37:17.224639+0000","last_unstale":"2024-05-10T10:37:17.229395+0000","last_undegraded":"2024-05-10T10:37:17.229395+0000","last_fullsized":"2024-05-10T10:37:17.229395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:19.628224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222142+0000","last_change":"2024-05-10T10:37:17.222142+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222142+0000","last_undegraded":"2024-05-10T10:37:17.222142+0000","last_fullsized":"2024-05-10T10:37:17.222142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.231471+0000","last_change":"2024-05-10T10:37:17.228371+0000","last_active":"2024-05-10T10:37:17.231471+0000","last_peered":"2024-05-10T10:37:17.231471+0000","last_clean":"2024-05-10T10:37:17.231471+0000","last_became_active":"2024-05-10T10:37:17.227960+0000","last_became_peered":"2024-05-10T10:37:17.227960+0000","last_unstale":"2024-05-10T10:37:17.231471+0000","last_undegraded":"2024-05-10T10:37:17.231471+0000","last_fullsized":"2024-05-10T10:37:17.231471+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:19.319484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.222569+0000","last_change":"2024-05-10T10:37:17.222569+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.222569+0000","last_undegraded":"2024-05-10T10:37:17.222569+0000","last_fullsized":"2024-05-10T10:37:17.222569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.229319+0000","last_change":"2024-05-10T10:37:17.223541+0000","last_active":"2024-05-10T10:37:17.229319+0000","last_peered":"2024-05-10T10:37:17.229319+0000","last_clean":"2024-05-10T10:37:17.229319+0000","last_became_active":"2024-05-10T10:37:17.223228+0000","last_became_peered":"2024-05-10T10:37:17.223228+0000","last_unstale":"2024-05-10T10:37:17.229319+0000","last_undegraded":"2024-05-10T10:37:17.229319+0000","last_fullsized":"2024-05-10T10:37:17.229319+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:14.320723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220334+0000","last_change":"2024-05-10T10:37:17.220334+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220334+0000","last_undegraded":"2024-05-10T10:37:17.220334+0000","last_fullsized":"2024-05-10T10:37:17.220334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.236807+0000","last_change":"2024-05-10T10:37:17.232666+0000","last_active":"2024-05-10T10:37:17.236807+0000","last_peered":"2024-05-10T10:37:17.236807+0000","last_clean":"2024-05-10T10:37:17.236807+0000","last_became_active":"2024-05-10T10:37:17.232429+0000","last_became_peered":"2024-05-10T10:37:17.232429+0000","last_unstale":"2024-05-10T10:37:17.236807+0000","last_undegraded":"2024-05-10T10:37:17.236807+0000","last_fullsized":"2024-05-10T10:37:17.236807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:31.641499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.220751+0000","last_change":"2024-05-10T10:37:17.220751+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.220751+0000","last_undegraded":"2024-05-10T10:37:17.220751+0000","last_fullsized":"2024-05-10T10:37:17.220751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.228619+0000","last_change":"2024-05-10T10:37:17.227889+0000","last_active":"2024-05-10T10:37:17.228619+0000","last_peered":"2024-05-10T10:37:17.228619+0000","last_clean":"2024-05-10T10:37:17.228619+0000","last_became_active":"2024-05-10T10:37:17.227312+0000","last_became_peered":"2024-05-10T10:37:17.227312+0000","last_unstale":"2024-05-10T10:37:17.228619+0000","last_undegraded":"2024-05-10T10:37:17.228619+0000","last_fullsized":"2024-05-10T10:37:17.228619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:37.508242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-05-10T10:37:17.213972+0000","last_change":"2024-05-10T10:37:17.213972+0000","last_active":"2024-05-10T10:37:17.201953+0000","last_peered":"2024-05-10T10:37:17.201953+0000","last_clean":"2024-05-10T10:37:17.201953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:37:17.213972+0000","last_undegraded":"2024-05-10T10:37:17.213972+0000","last_fullsized":"2024-05-10T10:37:17.213972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.233461+0000","last_change":"2024-05-10T10:37:17.230789+0000","last_active":"2024-05-10T10:37:17.233461+0000","last_peered":"2024-05-10T10:37:17.233461+0000","last_clean":"2024-05-10T10:37:17.233461+0000","last_became_active":"2024-05-10T10:37:17.230513+0000","last_became_peered":"2024-05-10T10:37:17.230513+0000","last_unstale":"2024-05-10T10:37:17.233461+0000","last_undegraded":"2024-05-10T10:37:17.233461+0000","last_fullsized":"2024-05-10T10:37:17.233461+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:15.368266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":215,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-10T10:37:17.219452+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:37:17.219452+0000","last_peered":"2024-05-10T10:37:17.219452+0000","last_clean":"2024-05-10T10:37:17.219452+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:37:17.219452+0000","last_undegraded":"2024-05-10T10:37:17.219452+0000","last_fullsized":"2024-05-10T10:37:17.219452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28464,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343376,"statfs":{"total":96636764160,"available":96607617024,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":396052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":986420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:37:24.195 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:37:24.195+0000 7f5d8fdfe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:37:26.772 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:37:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.889+0000 7ff36d628640 1 Processor -- start 2024-05-10T10:37:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.889+0000 7ff36d628640 1 -- start start 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff36d628640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 0x7ff3680d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff36d628640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3680b45f0 con 0x7ff3680d5cb0 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff36d628640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3680b48d0 con 0x7ff368162b30 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff36d628640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff3680b4f20 con 0x7ff368162750 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff3667fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff3680d5cb0 0x7ff3680d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42098/0 (socket says 172.21.15.53:42098) 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff3667fc640 1 -- 172.21.15.53:0/3766823076 learned_addr learned my addr 172.21.15.53:0/3766823076 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 0x7ff3680d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173230541 0 0) 0x7ff3680b45f0 con 0x7ff3680d5cb0 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.890+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff350003520 con 0x7ff3680d5cb0 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372360783 0 0) 0x7ff3680b48d0 con 0x7ff368162b30 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3680b45f0 con 0x7ff368162b30 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3327816650 0 0) 0x7ff350003520 con 0x7ff3680d5cb0 2024-05-10T10:37:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 msgr2=0x7ff3680d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 0x7ff3680d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 >> v1:172.21.15.135:6789/0 conn(0x7ff368162b30 legacy=0x7ff3680b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3680b4bb0 con 0x7ff3680d5cb0 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7ff35c001670 con 0x7ff3680d5cb0 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 >> v1:172.21.15.53:6789/0 conn(0x7ff3680d5cb0 legacy=0x7ff3680d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350003b70 0x7ff350003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 0x7ff350004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350004d40 0x7ff3500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680b4f20 con 0x7ff350003b70 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff35000bb50 con 0x7ff350004d40 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff35000bd20 con 0x7ff368162750 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350003b70 0x7ff350003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff3667fc640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 0x7ff350004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff3677fe640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350004d40 0x7ff3500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff366ffd640 1 -- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 msgr2=0x7ff350004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368162750 0x7ff350004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff366ffd640 1 -- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350004d40 msgr2=0x7ff3500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350004d40 0x7ff3500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.891+0000 7ff366ffd640 1 -- 172.21.15.53:0/3766823076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff350003520 con 0x7ff350003b70 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.892+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350003b70 0x7ff350003f70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff358009af0 tx=0x7ff350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbf808c64256901 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.892+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358036070 con 0x7ff350003b70 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.892+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff358031400 con 0x7ff350003b70 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.892+0000 7ff365ffb640 1 -- 172.21.15.53:0/3766823076 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358009690 con 0x7ff350003b70 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.893+0000 7ff36d628640 1 -- 172.21.15.53:0/3766823076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350003b70 msgr2=0x7ff350003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.893+0000 7ff36d628640 1 --2- 172.21.15.53:0/3766823076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350003b70 0x7ff350003f70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff358009af0 tx=0x7ff350005830 comp rx=0 tx=0).stop 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.893+0000 7ff3677fe640 1 -- 172.21.15.53:0/3766823076 reap_dead start 2024-05-10T10:37:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.893+0000 7ff36d628640 1 -- 172.21.15.53:0/3766823076 shutdown_connections 2024-05-10T10:37:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.893+0000 7ff36d628640 1 -- 172.21.15.53:0/3766823076 >> 172.21.15.53:0/3766823076 conn(0x7ff36809e3b0 msgr2=0x7ff3680d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.893+0000 7ff36d628640 1 -- 172.21.15.53:0/3766823076 shutdown_connections 2024-05-10T10:37:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.893+0000 7ff36d628640 1 -- 172.21.15.53:0/3766823076 wait complete. 2024-05-10T10:37:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.894+0000 7ff36d628640 1 Processor -- start 2024-05-10T10:37:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.894+0000 7ff36d628640 1 -- start start 2024-05-10T10:37:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff36d628640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350003b70 0x7ff3680ba750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff36d628640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff368157380 0x7ff36814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff36d628640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3681606c0 0x7ff368160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff36d628640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680d0fe0 con 0x7ff368157380 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff36d628640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680d0670 con 0x7ff350003b70 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff36d628640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff3680d07f0 con 0x7ff3681606c0 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff3667fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff368157380 0x7ff36814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff3667fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff368157380 0x7ff36814e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54174/0 (socket says 172.21.15.53:54174) 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff3667fc640 1 -- 172.21.15.53:0/3658085670 learned_addr learned my addr 172.21.15.53:0/3658085670 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350003b70 0x7ff3680ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.895+0000 7ff3677fe640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3681606c0 0x7ff368160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff3667fc640 1 -- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3681606c0 msgr2=0x7ff368160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff3667fc640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3681606c0 0x7ff368160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff3667fc640 1 -- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350003b70 msgr2=0x7ff3680ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff3667fc640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350003b70 0x7ff3680ba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff3667fc640 1 -- 172.21.15.53:0/3658085670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff368158880 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff3667fc640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff368157380 0x7ff36814e480 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007fd0 tx=0x7ff35c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff34f7fe640 1 -- 172.21.15.53:0/3658085670 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c00f730 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff34f7fe640 1 -- 172.21.15.53:0/3658085670 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff35c018070 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff368158b60 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff34f7fe640 1 -- 172.21.15.53:0/3658085670 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c00fb30 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.896+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff368159120 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.897+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36806a220 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.898+0000 7ff34f7fe640 1 -- 172.21.15.53:0/3658085670 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff35c00fcd0 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.898+0000 7ff34f7fe640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3540762d0 0x7ff354078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.899+0000 7ff34f7fe640 1 -- 172.21.15.53:0/3658085670 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7ff35c097590 con 0x7ff368157380 2024-05-10T10:37:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.899+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3540762d0 0x7ff354078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:26.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.899+0000 7ff366ffd640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3540762d0 0x7ff354078790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff358033000 tx=0x7ff358008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:26.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:26.903+0000 7ff34f7fe640 1 -- 172.21.15.53:0/3658085670 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff35c0607a0 con 0x7ff368157380 2024-05-10T10:37:27.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.091+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3680b5cf0 con 0x7ff3540762d0 2024-05-10T10:37:27.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.098+0000 7ff34f7fe640 1 -- 172.21.15.53:0/3658085670 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7ff3680b5cf0 con 0x7ff3540762d0 2024-05-10T10:37:27.101 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:27.102 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.100+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3540762d0 msgr2=0x7ff354078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.100+0000 7ff36d628640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3540762d0 0x7ff354078790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff358033000 tx=0x7ff358008030 comp rx=0 tx=0).stop 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.100+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff368157380 msgr2=0x7ff36814e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.100+0000 7ff36d628640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff368157380 0x7ff36814e480 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007fd0 tx=0x7ff35c007430 comp rx=0 tx=0).stop 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 shutdown_connections 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff3540762d0 0x7ff354078790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3681606c0 0x7ff368160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff368157380 0x7ff36814e480 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 --2- 172.21.15.53:0/3658085670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff350003b70 0x7ff3680ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 >> 172.21.15.53:0/3658085670 conn(0x7ff36809e3b0 msgr2=0x7ff3680d4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 shutdown_connections 2024-05-10T10:37:27.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.101+0000 7ff36d628640 1 -- 172.21.15.53:0/3658085670 wait complete. 2024-05-10T10:37:27.114 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-05-10T10:37:26.212277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245280,"kb_used_data":8296,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729440,"statfs":{"total":773094113280,"available":772842946560,"internally_reserved":0,"allocated":8495104,"data_stored":4464844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002247"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227341+0000","last_change":"2024-05-10T10:37:18.230518+0000","last_active":"2024-05-10T10:37:19.227341+0000","last_peered":"2024-05-10T10:37:19.227341+0000","last_clean":"2024-05-10T10:37:19.227341+0000","last_became_active":"2024-05-10T10:37:18.230266+0000","last_became_peered":"2024-05-10T10:37:18.230266+0000","last_unstale":"2024-05-10T10:37:19.227341+0000","last_undegraded":"2024-05-10T10:37:19.227341+0000","last_fullsized":"2024-05-10T10:37:19.227341+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:32.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217632+0000","last_change":"2024-05-10T10:37:18.239347+0000","last_active":"2024-05-10T10:37:20.217632+0000","last_peered":"2024-05-10T10:37:20.217632+0000","last_clean":"2024-05-10T10:37:20.217632+0000","last_became_active":"2024-05-10T10:37:18.239055+0000","last_became_peered":"2024-05-10T10:37:18.239055+0000","last_unstale":"2024-05-10T10:37:20.217632+0000","last_undegraded":"2024-05-10T10:37:20.217632+0000","last_fullsized":"2024-05-10T10:37:20.217632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:34:43.280014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333255+0000","last_change":"2024-05-10T10:37:18.244602+0000","last_active":"2024-05-10T10:37:20.333255+0000","last_peered":"2024-05-10T10:37:20.333255+0000","last_clean":"2024-05-10T10:37:20.333255+0000","last_became_active":"2024-05-10T10:37:18.244322+0000","last_became_peered":"2024-05-10T10:37:18.244322+0000","last_unstale":"2024-05-10T10:37:20.333255+0000","last_undegraded":"2024-05-10T10:37:20.333255+0000","last_fullsized":"2024-05-10T10:37:20.333255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:25.318575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227966+0000","last_change":"2024-05-10T10:37:18.234412+0000","last_active":"2024-05-10T10:37:19.227966+0000","last_peered":"2024-05-10T10:37:19.227966+0000","last_clean":"2024-05-10T10:37:19.227966+0000","last_became_active":"2024-05-10T10:37:18.234136+0000","last_became_peered":"2024-05-10T10:37:18.234136+0000","last_unstale":"2024-05-10T10:37:19.227966+0000","last_undegraded":"2024-05-10T10:37:19.227966+0000","last_fullsized":"2024-05-10T10:37:19.227966+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:18.508727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706225+0000","last_change":"2024-05-10T10:37:18.240265+0000","last_active":"2024-05-10T10:37:19.706225+0000","last_peered":"2024-05-10T10:37:19.706225+0000","last_clean":"2024-05-10T10:37:19.706225+0000","last_became_active":"2024-05-10T10:37:18.239984+0000","last_became_peered":"2024-05-10T10:37:18.239984+0000","last_unstale":"2024-05-10T10:37:19.706225+0000","last_undegraded":"2024-05-10T10:37:19.706225+0000","last_fullsized":"2024-05-10T10:37:19.706225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:23:36.247871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333590+0000","last_change":"2024-05-10T10:37:18.225496+0000","last_active":"2024-05-10T10:37:20.333590+0000","last_peered":"2024-05-10T10:37:20.333590+0000","last_clean":"2024-05-10T10:37:20.333590+0000","last_became_active":"2024-05-10T10:37:18.225214+0000","last_became_peered":"2024-05-10T10:37:18.225214+0000","last_unstale":"2024-05-10T10:37:20.333590+0000","last_undegraded":"2024-05-10T10:37:20.333590+0000","last_fullsized":"2024-05-10T10:37:20.333590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:48:53.954080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334321+0000","last_change":"2024-05-10T10:37:18.239247+0000","last_active":"2024-05-10T10:37:20.334321+0000","last_peered":"2024-05-10T10:37:20.334321+0000","last_clean":"2024-05-10T10:37:20.334321+0000","last_became_active":"2024-05-10T10:37:18.238971+0000","last_became_peered":"2024-05-10T10:37:18.238971+0000","last_unstale":"2024-05-10T10:37:20.334321+0000","last_undegraded":"2024-05-10T10:37:20.334321+0000","last_fullsized":"2024-05-10T10:37:20.334321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:20.630591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.705977+0000","last_change":"2024-05-10T10:37:18.236206+0000","last_active":"2024-05-10T10:37:19.705977+0000","last_peered":"2024-05-10T10:37:19.705977+0000","last_clean":"2024-05-10T10:37:19.705977+0000","last_became_active":"2024-05-10T10:37:18.235891+0000","last_became_peered":"2024-05-10T10:37:18.235891+0000","last_unstale":"2024-05-10T10:37:19.705977+0000","last_undegraded":"2024-05-10T10:37:19.705977+0000","last_fullsized":"2024-05-10T10:37:19.705977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:00.534001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226673+0000","last_change":"2024-05-10T10:37:18.226609+0000","last_active":"2024-05-10T10:37:19.226673+0000","last_peered":"2024-05-10T10:37:19.226673+0000","last_clean":"2024-05-10T10:37:19.226673+0000","last_became_active":"2024-05-10T10:37:18.226110+0000","last_became_peered":"2024-05-10T10:37:18.226110+0000","last_unstale":"2024-05-10T10:37:19.226673+0000","last_undegraded":"2024-05-10T10:37:19.226673+0000","last_fullsized":"2024-05-10T10:37:19.226673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:45:22.261297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227372+0000","last_change":"2024-05-10T10:37:18.225103+0000","last_active":"2024-05-10T10:37:19.227372+0000","last_peered":"2024-05-10T10:37:19.227372+0000","last_clean":"2024-05-10T10:37:19.227372+0000","last_became_active":"2024-05-10T10:37:18.224815+0000","last_became_peered":"2024-05-10T10:37:18.224815+0000","last_unstale":"2024-05-10T10:37:19.227372+0000","last_undegraded":"2024-05-10T10:37:19.227372+0000","last_fullsized":"2024-05-10T10:37:19.227372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:16.370633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217300+0000","last_change":"2024-05-10T10:37:18.240588+0000","last_active":"2024-05-10T10:37:20.217300+0000","last_peered":"2024-05-10T10:37:20.217300+0000","last_clean":"2024-05-10T10:37:20.217300+0000","last_became_active":"2024-05-10T10:37:18.240173+0000","last_became_peered":"2024-05-10T10:37:18.240173+0000","last_unstale":"2024-05-10T10:37:20.217300+0000","last_undegraded":"2024-05-10T10:37:20.217300+0000","last_fullsized":"2024-05-10T10:37:20.217300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:44.876706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.228126+0000","last_change":"2024-05-10T10:37:18.239410+0000","last_active":"2024-05-10T10:37:19.228126+0000","last_peered":"2024-05-10T10:37:19.228126+0000","last_clean":"2024-05-10T10:37:19.228126+0000","last_became_active":"2024-05-10T10:37:18.239101+0000","last_became_peered":"2024-05-10T10:37:18.239101+0000","last_unstale":"2024-05-10T10:37:19.228126+0000","last_undegraded":"2024-05-10T10:37:19.228126+0000","last_fullsized":"2024-05-10T10:37:19.228126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:07.139557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706185+0000","last_change":"2024-05-10T10:37:18.242928+0000","last_active":"2024-05-10T10:37:19.706185+0000","last_peered":"2024-05-10T10:37:19.706185+0000","last_clean":"2024-05-10T10:37:19.706185+0000","last_became_active":"2024-05-10T10:37:18.242573+0000","last_became_peered":"2024-05-10T10:37:18.242573+0000","last_unstale":"2024-05-10T10:37:19.706185+0000","last_undegraded":"2024-05-10T10:37:19.706185+0000","last_fullsized":"2024-05-10T10:37:19.706185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:31:26.424688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333727+0000","last_change":"2024-05-10T10:37:18.234463+0000","last_active":"2024-05-10T10:37:20.333727+0000","last_peered":"2024-05-10T10:37:20.333727+0000","last_clean":"2024-05-10T10:37:20.333727+0000","last_became_active":"2024-05-10T10:37:18.234161+0000","last_became_peered":"2024-05-10T10:37:18.234161+0000","last_unstale":"2024-05-10T10:37:20.333727+0000","last_undegraded":"2024-05-10T10:37:20.333727+0000","last_fullsized":"2024-05-10T10:37:20.333727+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:35.647603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217082+0000","last_change":"2024-05-10T10:37:18.227193+0000","last_active":"2024-05-10T10:37:20.217082+0000","last_peered":"2024-05-10T10:37:20.217082+0000","last_clean":"2024-05-10T10:37:20.217082+0000","last_became_active":"2024-05-10T10:37:18.226922+0000","last_became_peered":"2024-05-10T10:37:18.226922+0000","last_unstale":"2024-05-10T10:37:20.217082+0000","last_undegraded":"2024-05-10T10:37:20.217082+0000","last_fullsized":"2024-05-10T10:37:20.217082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:18.508727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227347+0000","last_change":"2024-05-10T10:37:18.224697+0000","last_active":"2024-05-10T10:37:19.227347+0000","last_peered":"2024-05-10T10:37:19.227347+0000","last_clean":"2024-05-10T10:37:19.227347+0000","last_became_active":"2024-05-10T10:37:18.224407+0000","last_became_peered":"2024-05-10T10:37:18.224407+0000","last_unstale":"2024-05-10T10:37:19.227347+0000","last_undegraded":"2024-05-10T10:37:19.227347+0000","last_fullsized":"2024-05-10T10:37:19.227347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:34:43.280014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706313+0000","last_change":"2024-05-10T10:37:18.242071+0000","last_active":"2024-05-10T10:37:19.706313+0000","last_peered":"2024-05-10T10:37:19.706313+0000","last_clean":"2024-05-10T10:37:19.706313+0000","last_became_active":"2024-05-10T10:37:18.241804+0000","last_became_peered":"2024-05-10T10:37:18.241804+0000","last_unstale":"2024-05-10T10:37:19.706313+0000","last_undegraded":"2024-05-10T10:37:19.706313+0000","last_fullsized":"2024-05-10T10:37:19.706313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:18:31.135265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333520+0000","last_change":"2024-05-10T10:37:18.236074+0000","last_active":"2024-05-10T10:37:20.333520+0000","last_peered":"2024-05-10T10:37:20.333520+0000","last_clean":"2024-05-10T10:37:20.333520+0000","last_became_active":"2024-05-10T10:37:18.235809+0000","last_became_peered":"2024-05-10T10:37:18.235809+0000","last_unstale":"2024-05-10T10:37:20.333520+0000","last_undegraded":"2024-05-10T10:37:20.333520+0000","last_fullsized":"2024-05-10T10:37:20.333520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:40.906436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333374+0000","last_change":"2024-05-10T10:37:18.243404+0000","last_active":"2024-05-10T10:37:20.333374+0000","last_peered":"2024-05-10T10:37:20.333374+0000","last_clean":"2024-05-10T10:37:20.333374+0000","last_became_active":"2024-05-10T10:37:18.243102+0000","last_became_peered":"2024-05-10T10:37:18.243102+0000","last_unstale":"2024-05-10T10:37:20.333374+0000","last_undegraded":"2024-05-10T10:37:20.333374+0000","last_fullsized":"2024-05-10T10:37:20.333374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:37.891942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706322+0000","last_change":"2024-05-10T10:37:18.244994+0000","last_active":"2024-05-10T10:37:19.706322+0000","last_peered":"2024-05-10T10:37:19.706322+0000","last_clean":"2024-05-10T10:37:19.706322+0000","last_became_active":"2024-05-10T10:37:18.244718+0000","last_became_peered":"2024-05-10T10:37:18.244718+0000","last_unstale":"2024-05-10T10:37:19.706322+0000","last_undegraded":"2024-05-10T10:37:19.706322+0000","last_fullsized":"2024-05-10T10:37:19.706322+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:59.320587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227794+0000","last_change":"2024-05-10T10:37:18.243046+0000","last_active":"2024-05-10T10:37:19.227794+0000","last_peered":"2024-05-10T10:37:19.227794+0000","last_clean":"2024-05-10T10:37:19.227794+0000","last_became_active":"2024-05-10T10:37:18.242746+0000","last_became_peered":"2024-05-10T10:37:18.242746+0000","last_unstale":"2024-05-10T10:37:19.227794+0000","last_undegraded":"2024-05-10T10:37:19.227794+0000","last_fullsized":"2024-05-10T10:37:19.227794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:55:04.379911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706176+0000","last_change":"2024-05-10T10:37:18.247023+0000","last_active":"2024-05-10T10:37:19.706176+0000","last_peered":"2024-05-10T10:37:19.706176+0000","last_clean":"2024-05-10T10:37:19.706176+0000","last_became_active":"2024-05-10T10:37:18.246753+0000","last_became_peered":"2024-05-10T10:37:18.246753+0000","last_unstale":"2024-05-10T10:37:19.706176+0000","last_undegraded":"2024-05-10T10:37:19.706176+0000","last_fullsized":"2024-05-10T10:37:19.706176+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:46:00.952641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226928+0000","last_change":"2024-05-10T10:37:18.230938+0000","last_active":"2024-05-10T10:37:19.226928+0000","last_peered":"2024-05-10T10:37:19.226928+0000","last_clean":"2024-05-10T10:37:19.226928+0000","last_became_active":"2024-05-10T10:37:18.230631+0000","last_became_peered":"2024-05-10T10:37:18.230631+0000","last_unstale":"2024-05-10T10:37:19.226928+0000","last_undegraded":"2024-05-10T10:37:19.226928+0000","last_fullsized":"2024-05-10T10:37:19.226928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:20:29.866130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217154+0000","last_change":"2024-05-10T10:37:18.228898+0000","last_active":"2024-05-10T10:37:20.217154+0000","last_peered":"2024-05-10T10:37:20.217154+0000","last_clean":"2024-05-10T10:37:20.217154+0000","last_became_active":"2024-05-10T10:37:18.228574+0000","last_became_peered":"2024-05-10T10:37:18.228574+0000","last_unstale":"2024-05-10T10:37:20.217154+0000","last_undegraded":"2024-05-10T10:37:20.217154+0000","last_fullsized":"2024-05-10T10:37:20.217154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:07.139557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217587+0000","last_change":"2024-05-10T10:37:18.228133+0000","last_active":"2024-05-10T10:37:20.217587+0000","last_peered":"2024-05-10T10:37:20.217587+0000","last_clean":"2024-05-10T10:37:20.217587+0000","last_became_active":"2024-05-10T10:37:18.227858+0000","last_became_peered":"2024-05-10T10:37:18.227858+0000","last_unstale":"2024-05-10T10:37:20.217587+0000","last_undegraded":"2024-05-10T10:37:20.217587+0000","last_fullsized":"2024-05-10T10:37:20.217587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:32:08.408186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227519+0000","last_change":"2024-05-10T10:37:18.236539+0000","last_active":"2024-05-10T10:37:19.227519+0000","last_peered":"2024-05-10T10:37:19.227519+0000","last_clean":"2024-05-10T10:37:19.227519+0000","last_became_active":"2024-05-10T10:37:18.236214+0000","last_became_peered":"2024-05-10T10:37:18.236214+0000","last_unstale":"2024-05-10T10:37:19.227519+0000","last_undegraded":"2024-05-10T10:37:19.227519+0000","last_fullsized":"2024-05-10T10:37:19.227519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:02:39.479798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226868+0000","last_change":"2024-05-10T10:37:18.227340+0000","last_active":"2024-05-10T10:37:19.226868+0000","last_peered":"2024-05-10T10:37:19.226868+0000","last_clean":"2024-05-10T10:37:19.226868+0000","last_became_active":"2024-05-10T10:37:18.227065+0000","last_became_peered":"2024-05-10T10:37:18.227065+0000","last_unstale":"2024-05-10T10:37:19.226868+0000","last_undegraded":"2024-05-10T10:37:19.226868+0000","last_fullsized":"2024-05-10T10:37:19.226868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:18.534620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706460+0000","last_change":"2024-05-10T10:37:18.238978+0000","last_active":"2024-05-10T10:37:19.706460+0000","last_peered":"2024-05-10T10:37:19.706460+0000","last_clean":"2024-05-10T10:37:19.706460+0000","last_became_active":"2024-05-10T10:37:18.238702+0000","last_became_peered":"2024-05-10T10:37:18.238702+0000","last_unstale":"2024-05-10T10:37:19.706460+0000","last_undegraded":"2024-05-10T10:37:19.706460+0000","last_fullsized":"2024-05-10T10:37:19.706460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:13:29.283170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217167+0000","last_change":"2024-05-10T10:37:18.222928+0000","last_active":"2024-05-10T10:37:20.217167+0000","last_peered":"2024-05-10T10:37:20.217167+0000","last_clean":"2024-05-10T10:37:20.217167+0000","last_became_active":"2024-05-10T10:37:18.222369+0000","last_became_peered":"2024-05-10T10:37:18.222369+0000","last_unstale":"2024-05-10T10:37:20.217167+0000","last_undegraded":"2024-05-10T10:37:20.217167+0000","last_fullsized":"2024-05-10T10:37:20.217167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:32:05.870422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226696+0000","last_change":"2024-05-10T10:37:18.235701+0000","last_active":"2024-05-10T10:37:19.226696+0000","last_peered":"2024-05-10T10:37:19.226696+0000","last_clean":"2024-05-10T10:37:19.226696+0000","last_became_active":"2024-05-10T10:37:18.235390+0000","last_became_peered":"2024-05-10T10:37:18.235390+0000","last_unstale":"2024-05-10T10:37:19.226696+0000","last_undegraded":"2024-05-10T10:37:19.226696+0000","last_fullsized":"2024-05-10T10:37:19.226696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:09:44.876706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333090+0000","last_change":"2024-05-10T10:37:18.227004+0000","last_active":"2024-05-10T10:37:20.333090+0000","last_peered":"2024-05-10T10:37:20.333090+0000","last_clean":"2024-05-10T10:37:20.333090+0000","last_became_active":"2024-05-10T10:37:18.226706+0000","last_became_peered":"2024-05-10T10:37:18.226706+0000","last_unstale":"2024-05-10T10:37:20.333090+0000","last_undegraded":"2024-05-10T10:37:20.333090+0000","last_fullsized":"2024-05-10T10:37:20.333090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:14.248837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227512+0000","last_change":"2024-05-10T10:37:18.229231+0000","last_active":"2024-05-10T10:37:19.227512+0000","last_peered":"2024-05-10T10:37:19.227512+0000","last_clean":"2024-05-10T10:37:19.227512+0000","last_became_active":"2024-05-10T10:37:18.228964+0000","last_became_peered":"2024-05-10T10:37:18.228964+0000","last_unstale":"2024-05-10T10:37:19.227512+0000","last_undegraded":"2024-05-10T10:37:19.227512+0000","last_fullsized":"2024-05-10T10:37:19.227512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:37.891942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333145+0000","last_change":"2024-05-10T10:37:18.235478+0000","last_active":"2024-05-10T10:37:20.333145+0000","last_peered":"2024-05-10T10:37:20.333145+0000","last_clean":"2024-05-10T10:37:20.333145+0000","last_became_active":"2024-05-10T10:37:18.235209+0000","last_became_peered":"2024-05-10T10:37:18.235209+0000","last_unstale":"2024-05-10T10:37:20.333145+0000","last_undegraded":"2024-05-10T10:37:20.333145+0000","last_fullsized":"2024-05-10T10:37:20.333145+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:12.522510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706424+0000","last_change":"2024-05-10T10:37:17.234091+0000","last_active":"2024-05-10T10:37:19.706424+0000","last_peered":"2024-05-10T10:37:19.706424+0000","last_clean":"2024-05-10T10:37:19.706424+0000","last_became_active":"2024-05-10T10:37:17.233792+0000","last_became_peered":"2024-05-10T10:37:17.233792+0000","last_unstale":"2024-05-10T10:37:19.706424+0000","last_undegraded":"2024-05-10T10:37:19.706424+0000","last_fullsized":"2024-05-10T10:37:19.706424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:38.874001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217221+0000","last_change":"2024-05-10T10:37:18.229660+0000","last_active":"2024-05-10T10:37:20.217221+0000","last_peered":"2024-05-10T10:37:20.217221+0000","last_clean":"2024-05-10T10:37:20.217221+0000","last_became_active":"2024-05-10T10:37:18.229385+0000","last_became_peered":"2024-05-10T10:37:18.229385+0000","last_unstale":"2024-05-10T10:37:20.217221+0000","last_undegraded":"2024-05-10T10:37:20.217221+0000","last_fullsized":"2024-05-10T10:37:20.217221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:33.351070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226876+0000","last_change":"2024-05-10T10:37:17.229429+0000","last_active":"2024-05-10T10:37:19.226876+0000","last_peered":"2024-05-10T10:37:19.226876+0000","last_clean":"2024-05-10T10:37:19.226876+0000","last_became_active":"2024-05-10T10:37:17.229121+0000","last_became_peered":"2024-05-10T10:37:17.229121+0000","last_unstale":"2024-05-10T10:37:19.226876+0000","last_undegraded":"2024-05-10T10:37:19.226876+0000","last_fullsized":"2024-05-10T10:37:19.226876+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:25.579975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.228198+0000","last_change":"2024-05-10T10:37:18.226829+0000","last_active":"2024-05-10T10:37:19.228198+0000","last_peered":"2024-05-10T10:37:19.228198+0000","last_clean":"2024-05-10T10:37:19.228198+0000","last_became_active":"2024-05-10T10:37:18.226537+0000","last_became_peered":"2024-05-10T10:37:18.226537+0000","last_unstale":"2024-05-10T10:37:19.228198+0000","last_undegraded":"2024-05-10T10:37:19.228198+0000","last_fullsized":"2024-05-10T10:37:19.228198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:15:46.685729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226963+0000","last_change":"2024-05-10T10:37:17.225715+0000","last_active":"2024-05-10T10:37:19.226963+0000","last_peered":"2024-05-10T10:37:19.226963+0000","last_clean":"2024-05-10T10:37:19.226963+0000","last_became_active":"2024-05-10T10:37:17.225250+0000","last_became_peered":"2024-05-10T10:37:17.225250+0000","last_unstale":"2024-05-10T10:37:19.226963+0000","last_undegraded":"2024-05-10T10:37:19.226963+0000","last_fullsized":"2024-05-10T10:37:19.226963+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:43:25.143088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227964+0000","last_change":"2024-05-10T10:37:18.228933+0000","last_active":"2024-05-10T10:37:19.227964+0000","last_peered":"2024-05-10T10:37:19.227964+0000","last_clean":"2024-05-10T10:37:19.227964+0000","last_became_active":"2024-05-10T10:37:18.228622+0000","last_became_peered":"2024-05-10T10:37:18.228622+0000","last_unstale":"2024-05-10T10:37:19.227964+0000","last_undegraded":"2024-05-10T10:37:19.227964+0000","last_fullsized":"2024-05-10T10:37:19.227964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:41.947368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334216+0000","last_change":"2024-05-10T10:37:17.228203+0000","last_active":"2024-05-10T10:37:20.334216+0000","last_peered":"2024-05-10T10:37:20.334216+0000","last_clean":"2024-05-10T10:37:20.334216+0000","last_became_active":"2024-05-10T10:37:17.227939+0000","last_became_peered":"2024-05-10T10:37:17.227939+0000","last_unstale":"2024-05-10T10:37:20.334216+0000","last_undegraded":"2024-05-10T10:37:20.334216+0000","last_fullsized":"2024-05-10T10:37:20.334216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:37.508242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217230+0000","last_change":"2024-05-10T10:37:18.241930+0000","last_active":"2024-05-10T10:37:20.217230+0000","last_peered":"2024-05-10T10:37:20.217230+0000","last_clean":"2024-05-10T10:37:20.217230+0000","last_became_active":"2024-05-10T10:37:18.241033+0000","last_became_peered":"2024-05-10T10:37:18.241033+0000","last_unstale":"2024-05-10T10:37:20.217230+0000","last_undegraded":"2024-05-10T10:37:20.217230+0000","last_fullsized":"2024-05-10T10:37:20.217230+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:39.817383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.228060+0000","last_change":"2024-05-10T10:37:17.222699+0000","last_active":"2024-05-10T10:37:19.228060+0000","last_peered":"2024-05-10T10:37:19.228060+0000","last_clean":"2024-05-10T10:37:19.228060+0000","last_became_active":"2024-05-10T10:37:17.222389+0000","last_became_peered":"2024-05-10T10:37:17.222389+0000","last_unstale":"2024-05-10T10:37:19.228060+0000","last_undegraded":"2024-05-10T10:37:19.228060+0000","last_fullsized":"2024-05-10T10:37:19.228060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:21.940248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217572+0000","last_change":"2024-05-10T10:37:18.242188+0000","last_active":"2024-05-10T10:37:20.217572+0000","last_peered":"2024-05-10T10:37:20.217572+0000","last_clean":"2024-05-10T10:37:20.217572+0000","last_became_active":"2024-05-10T10:37:18.239733+0000","last_became_peered":"2024-05-10T10:37:18.239733+0000","last_unstale":"2024-05-10T10:37:20.217572+0000","last_undegraded":"2024-05-10T10:37:20.217572+0000","last_fullsized":"2024-05-10T10:37:20.217572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:27:32.054664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226828+0000","last_change":"2024-05-10T10:37:17.227524+0000","last_active":"2024-05-10T10:37:19.226828+0000","last_peered":"2024-05-10T10:37:19.226828+0000","last_clean":"2024-05-10T10:37:19.226828+0000","last_became_active":"2024-05-10T10:37:17.227256+0000","last_became_peered":"2024-05-10T10:37:17.227256+0000","last_unstale":"2024-05-10T10:37:19.226828+0000","last_undegraded":"2024-05-10T10:37:19.226828+0000","last_fullsized":"2024-05-10T10:37:19.226828+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:39.904069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227442+0000","last_change":"2024-05-10T10:37:18.233808+0000","last_active":"2024-05-10T10:37:19.227442+0000","last_peered":"2024-05-10T10:37:19.227442+0000","last_clean":"2024-05-10T10:37:19.227442+0000","last_became_active":"2024-05-10T10:37:18.233337+0000","last_became_peered":"2024-05-10T10:37:18.233337+0000","last_unstale":"2024-05-10T10:37:19.227442+0000","last_undegraded":"2024-05-10T10:37:19.227442+0000","last_fullsized":"2024-05-10T10:37:19.227442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:39.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217228+0000","last_change":"2024-05-10T10:37:17.218206+0000","last_active":"2024-05-10T10:37:20.217228+0000","last_peered":"2024-05-10T10:37:20.217228+0000","last_clean":"2024-05-10T10:37:20.217228+0000","last_became_active":"2024-05-10T10:37:17.217591+0000","last_became_peered":"2024-05-10T10:37:17.217591+0000","last_unstale":"2024-05-10T10:37:20.217228+0000","last_undegraded":"2024-05-10T10:37:20.217228+0000","last_fullsized":"2024-05-10T10:37:20.217228+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:11.520143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334372+0000","last_change":"2024-05-10T10:37:18.239737+0000","last_active":"2024-05-10T10:37:20.334372+0000","last_peered":"2024-05-10T10:37:20.334372+0000","last_clean":"2024-05-10T10:37:20.334372+0000","last_became_active":"2024-05-10T10:37:18.239401+0000","last_became_peered":"2024-05-10T10:37:18.239401+0000","last_unstale":"2024-05-10T10:37:20.334372+0000","last_undegraded":"2024-05-10T10:37:20.334372+0000","last_fullsized":"2024-05-10T10:37:20.334372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:32:40.906436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706014+0000","last_change":"2024-05-10T10:37:17.230114+0000","last_active":"2024-05-10T10:37:19.706014+0000","last_peered":"2024-05-10T10:37:19.706014+0000","last_clean":"2024-05-10T10:37:19.706014+0000","last_became_active":"2024-05-10T10:37:17.229847+0000","last_became_peered":"2024-05-10T10:37:17.229847+0000","last_unstale":"2024-05-10T10:37:19.706014+0000","last_undegraded":"2024-05-10T10:37:19.706014+0000","last_fullsized":"2024-05-10T10:37:19.706014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:58.470867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217052+0000","last_change":"2024-05-10T10:37:18.221594+0000","last_active":"2024-05-10T10:37:20.217052+0000","last_peered":"2024-05-10T10:37:20.217052+0000","last_clean":"2024-05-10T10:37:20.217052+0000","last_became_active":"2024-05-10T10:37:18.221310+0000","last_became_peered":"2024-05-10T10:37:18.221310+0000","last_unstale":"2024-05-10T10:37:20.217052+0000","last_undegraded":"2024-05-10T10:37:20.217052+0000","last_fullsized":"2024-05-10T10:37:20.217052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:02:39.479798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227447+0000","last_change":"2024-05-10T10:37:17.224805+0000","last_active":"2024-05-10T10:37:19.227447+0000","last_peered":"2024-05-10T10:37:19.227447+0000","last_clean":"2024-05-10T10:37:19.227447+0000","last_became_active":"2024-05-10T10:37:17.224508+0000","last_became_peered":"2024-05-10T10:37:17.224508+0000","last_unstale":"2024-05-10T10:37:19.227447+0000","last_undegraded":"2024-05-10T10:37:19.227447+0000","last_fullsized":"2024-05-10T10:37:19.227447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:23:59.926354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.228195+0000","last_change":"2024-05-10T10:37:18.233952+0000","last_active":"2024-05-10T10:37:19.228195+0000","last_peered":"2024-05-10T10:37:19.228195+0000","last_clean":"2024-05-10T10:37:19.228195+0000","last_became_active":"2024-05-10T10:37:18.233671+0000","last_became_peered":"2024-05-10T10:37:18.233671+0000","last_unstale":"2024-05-10T10:37:19.228195+0000","last_undegraded":"2024-05-10T10:37:19.228195+0000","last_fullsized":"2024-05-10T10:37:19.228195+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:33.351070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227605+0000","last_change":"2024-05-10T10:37:17.225567+0000","last_active":"2024-05-10T10:37:19.227605+0000","last_peered":"2024-05-10T10:37:19.227605+0000","last_clean":"2024-05-10T10:37:19.227605+0000","last_became_active":"2024-05-10T10:37:17.225301+0000","last_became_peered":"2024-05-10T10:37:17.225301+0000","last_unstale":"2024-05-10T10:37:19.227605+0000","last_undegraded":"2024-05-10T10:37:19.227605+0000","last_fullsized":"2024-05-10T10:37:19.227605+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:36:44.271136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333992+0000","last_change":"2024-05-10T10:37:18.227737+0000","last_active":"2024-05-10T10:37:20.333992+0000","last_peered":"2024-05-10T10:37:20.333992+0000","last_clean":"2024-05-10T10:37:20.333992+0000","last_became_active":"2024-05-10T10:37:18.227445+0000","last_became_peered":"2024-05-10T10:37:18.227445+0000","last_unstale":"2024-05-10T10:37:20.333992+0000","last_undegraded":"2024-05-10T10:37:20.333992+0000","last_fullsized":"2024-05-10T10:37:20.333992+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:51:51.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706375+0000","last_change":"2024-05-10T10:37:17.231914+0000","last_active":"2024-05-10T10:37:19.706375+0000","last_peered":"2024-05-10T10:37:19.706375+0000","last_clean":"2024-05-10T10:37:19.706375+0000","last_became_active":"2024-05-10T10:37:17.231631+0000","last_became_peered":"2024-05-10T10:37:17.231631+0000","last_unstale":"2024-05-10T10:37:19.706375+0000","last_undegraded":"2024-05-10T10:37:19.706375+0000","last_fullsized":"2024-05-10T10:37:19.706375+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:25:53.947881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333256+0000","last_change":"2024-05-10T10:37:18.241254+0000","last_active":"2024-05-10T10:37:20.333256+0000","last_peered":"2024-05-10T10:37:20.333256+0000","last_clean":"2024-05-10T10:37:20.333256+0000","last_became_active":"2024-05-10T10:37:18.240943+0000","last_became_peered":"2024-05-10T10:37:18.240943+0000","last_unstale":"2024-05-10T10:37:20.333256+0000","last_undegraded":"2024-05-10T10:37:20.333256+0000","last_fullsized":"2024-05-10T10:37:20.333256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:20.630591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334388+0000","last_change":"2024-05-10T10:37:17.229898+0000","last_active":"2024-05-10T10:37:20.334388+0000","last_peered":"2024-05-10T10:37:20.334388+0000","last_clean":"2024-05-10T10:37:20.334388+0000","last_became_active":"2024-05-10T10:37:17.229599+0000","last_became_peered":"2024-05-10T10:37:17.229599+0000","last_unstale":"2024-05-10T10:37:20.334388+0000","last_undegraded":"2024-05-10T10:37:20.334388+0000","last_fullsized":"2024-05-10T10:37:20.334388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:10.070859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.228283+0000","last_change":"2024-05-10T10:37:18.225001+0000","last_active":"2024-05-10T10:37:19.228283+0000","last_peered":"2024-05-10T10:37:19.228283+0000","last_clean":"2024-05-10T10:37:19.228283+0000","last_became_active":"2024-05-10T10:37:18.224691+0000","last_became_peered":"2024-05-10T10:37:18.224691+0000","last_unstale":"2024-05-10T10:37:19.228283+0000","last_undegraded":"2024-05-10T10:37:19.228283+0000","last_fullsized":"2024-05-10T10:37:19.228283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:39.817383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227966+0000","last_change":"2024-05-10T10:37:17.223996+0000","last_active":"2024-05-10T10:37:19.227966+0000","last_peered":"2024-05-10T10:37:19.227966+0000","last_clean":"2024-05-10T10:37:19.227966+0000","last_became_active":"2024-05-10T10:37:17.223309+0000","last_became_peered":"2024-05-10T10:37:17.223309+0000","last_unstale":"2024-05-10T10:37:19.227966+0000","last_undegraded":"2024-05-10T10:37:19.227966+0000","last_fullsized":"2024-05-10T10:37:19.227966+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:50.410699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334247+0000","last_change":"2024-05-10T10:37:18.225884+0000","last_active":"2024-05-10T10:37:20.334247+0000","last_peered":"2024-05-10T10:37:20.334247+0000","last_clean":"2024-05-10T10:37:20.334247+0000","last_became_active":"2024-05-10T10:37:18.225567+0000","last_became_peered":"2024-05-10T10:37:18.225567+0000","last_unstale":"2024-05-10T10:37:20.334247+0000","last_undegraded":"2024-05-10T10:37:20.334247+0000","last_fullsized":"2024-05-10T10:37:20.334247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:14.248837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226770+0000","last_change":"2024-05-10T10:37:17.226617+0000","last_active":"2024-05-10T10:37:19.226770+0000","last_peered":"2024-05-10T10:37:19.226770+0000","last_clean":"2024-05-10T10:37:19.226770+0000","last_became_active":"2024-05-10T10:37:17.226339+0000","last_became_peered":"2024-05-10T10:37:17.226339+0000","last_unstale":"2024-05-10T10:37:19.226770+0000","last_undegraded":"2024-05-10T10:37:19.226770+0000","last_fullsized":"2024-05-10T10:37:19.226770+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:56.625767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334391+0000","last_change":"2024-05-10T10:37:18.230436+0000","last_active":"2024-05-10T10:37:20.334391+0000","last_peered":"2024-05-10T10:37:20.334391+0000","last_clean":"2024-05-10T10:37:20.334391+0000","last_became_active":"2024-05-10T10:37:18.230041+0000","last_became_peered":"2024-05-10T10:37:18.230041+0000","last_unstale":"2024-05-10T10:37:20.334391+0000","last_undegraded":"2024-05-10T10:37:20.334391+0000","last_fullsized":"2024-05-10T10:37:20.334391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:10.887013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.705988+0000","last_change":"2024-05-10T10:37:17.234183+0000","last_active":"2024-05-10T10:37:19.705988+0000","last_peered":"2024-05-10T10:37:19.705988+0000","last_clean":"2024-05-10T10:37:19.705988+0000","last_became_active":"2024-05-10T10:37:17.233852+0000","last_became_peered":"2024-05-10T10:37:17.233852+0000","last_unstale":"2024-05-10T10:37:19.705988+0000","last_undegraded":"2024-05-10T10:37:19.705988+0000","last_fullsized":"2024-05-10T10:37:19.705988+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:59:37.690335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334473+0000","last_change":"2024-05-10T10:37:18.224472+0000","last_active":"2024-05-10T10:37:20.334473+0000","last_peered":"2024-05-10T10:37:20.334473+0000","last_clean":"2024-05-10T10:37:20.334473+0000","last_became_active":"2024-05-10T10:37:18.224211+0000","last_became_peered":"2024-05-10T10:37:18.224211+0000","last_unstale":"2024-05-10T10:37:20.334473+0000","last_undegraded":"2024-05-10T10:37:20.334473+0000","last_fullsized":"2024-05-10T10:37:20.334473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:51.413066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217459+0000","last_change":"2024-05-10T10:37:17.224923+0000","last_active":"2024-05-10T10:37:20.217459+0000","last_peered":"2024-05-10T10:37:20.217459+0000","last_clean":"2024-05-10T10:37:20.217459+0000","last_became_active":"2024-05-10T10:37:17.224594+0000","last_became_peered":"2024-05-10T10:37:17.224594+0000","last_unstale":"2024-05-10T10:37:20.217459+0000","last_undegraded":"2024-05-10T10:37:20.217459+0000","last_fullsized":"2024-05-10T10:37:20.217459+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:24.316209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333260+0000","last_change":"2024-05-10T10:37:18.244053+0000","last_active":"2024-05-10T10:37:20.333260+0000","last_peered":"2024-05-10T10:37:20.333260+0000","last_clean":"2024-05-10T10:37:20.333260+0000","last_became_active":"2024-05-10T10:37:18.243798+0000","last_became_peered":"2024-05-10T10:37:18.243798+0000","last_unstale":"2024-05-10T10:37:20.333260+0000","last_undegraded":"2024-05-10T10:37:20.333260+0000","last_fullsized":"2024-05-10T10:37:20.333260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:34:43.280014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227550+0000","last_change":"2024-05-10T10:37:17.225719+0000","last_active":"2024-05-10T10:37:19.227550+0000","last_peered":"2024-05-10T10:37:19.227550+0000","last_clean":"2024-05-10T10:37:19.227550+0000","last_became_active":"2024-05-10T10:37:17.225430+0000","last_became_peered":"2024-05-10T10:37:17.225430+0000","last_unstale":"2024-05-10T10:37:19.227550+0000","last_undegraded":"2024-05-10T10:37:19.227550+0000","last_fullsized":"2024-05-10T10:37:19.227550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:16:39.529753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706084+0000","last_change":"2024-05-10T10:37:18.239440+0000","last_active":"2024-05-10T10:37:19.706084+0000","last_peered":"2024-05-10T10:37:19.706084+0000","last_clean":"2024-05-10T10:37:19.706084+0000","last_became_active":"2024-05-10T10:37:18.239144+0000","last_became_peered":"2024-05-10T10:37:18.239144+0000","last_unstale":"2024-05-10T10:37:19.706084+0000","last_undegraded":"2024-05-10T10:37:19.706084+0000","last_fullsized":"2024-05-10T10:37:19.706084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:30:25.163894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227536+0000","last_change":"2024-05-10T10:37:17.226246+0000","last_active":"2024-05-10T10:37:19.227536+0000","last_peered":"2024-05-10T10:37:19.227536+0000","last_clean":"2024-05-10T10:37:19.227536+0000","last_became_active":"2024-05-10T10:37:17.225996+0000","last_became_peered":"2024-05-10T10:37:17.225996+0000","last_unstale":"2024-05-10T10:37:19.227536+0000","last_undegraded":"2024-05-10T10:37:19.227536+0000","last_fullsized":"2024-05-10T10:37:19.227536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:19.500154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.705979+0000","last_change":"2024-05-10T10:37:18.239628+0000","last_active":"2024-05-10T10:37:19.705979+0000","last_peered":"2024-05-10T10:37:19.705979+0000","last_clean":"2024-05-10T10:37:19.705979+0000","last_became_active":"2024-05-10T10:37:18.239342+0000","last_became_peered":"2024-05-10T10:37:18.239342+0000","last_unstale":"2024-05-10T10:37:19.705979+0000","last_undegraded":"2024-05-10T10:37:19.705979+0000","last_fullsized":"2024-05-10T10:37:19.705979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:57.738882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706448+0000","last_change":"2024-05-10T10:37:17.223948+0000","last_active":"2024-05-10T10:37:19.706448+0000","last_peered":"2024-05-10T10:37:19.706448+0000","last_clean":"2024-05-10T10:37:19.706448+0000","last_became_active":"2024-05-10T10:37:17.223293+0000","last_became_peered":"2024-05-10T10:37:17.223293+0000","last_unstale":"2024-05-10T10:37:19.706448+0000","last_undegraded":"2024-05-10T10:37:19.706448+0000","last_fullsized":"2024-05-10T10:37:19.706448+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:33:40.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333949+0000","last_change":"2024-05-10T10:37:18.243905+0000","last_active":"2024-05-10T10:37:20.333949+0000","last_peered":"2024-05-10T10:37:20.333949+0000","last_clean":"2024-05-10T10:37:20.333949+0000","last_became_active":"2024-05-10T10:37:18.243651+0000","last_became_peered":"2024-05-10T10:37:18.243651+0000","last_unstale":"2024-05-10T10:37:20.333949+0000","last_undegraded":"2024-05-10T10:37:20.333949+0000","last_fullsized":"2024-05-10T10:37:20.333949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:33.212499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333606+0000","last_change":"2024-05-10T10:37:17.232761+0000","last_active":"2024-05-10T10:37:20.333606+0000","last_peered":"2024-05-10T10:37:20.333606+0000","last_clean":"2024-05-10T10:37:20.333606+0000","last_became_active":"2024-05-10T10:37:17.232401+0000","last_became_peered":"2024-05-10T10:37:17.232401+0000","last_unstale":"2024-05-10T10:37:20.333606+0000","last_undegraded":"2024-05-10T10:37:20.333606+0000","last_fullsized":"2024-05-10T10:37:20.333606+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:45:21.258930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217460+0000","last_change":"2024-05-10T10:37:18.232110+0000","last_active":"2024-05-10T10:37:20.217460+0000","last_peered":"2024-05-10T10:37:20.217460+0000","last_clean":"2024-05-10T10:37:20.217460+0000","last_became_active":"2024-05-10T10:37:18.231836+0000","last_became_peered":"2024-05-10T10:37:18.231836+0000","last_unstale":"2024-05-10T10:37:20.217460+0000","last_undegraded":"2024-05-10T10:37:20.217460+0000","last_fullsized":"2024-05-10T10:37:20.217460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:18.534620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334045+0000","last_change":"2024-05-10T10:37:17.227272+0000","last_active":"2024-05-10T10:37:20.334045+0000","last_peered":"2024-05-10T10:37:20.334045+0000","last_clean":"2024-05-10T10:37:20.334045+0000","last_became_active":"2024-05-10T10:37:17.224267+0000","last_became_peered":"2024-05-10T10:37:17.224267+0000","last_unstale":"2024-05-10T10:37:20.334045+0000","last_undegraded":"2024-05-10T10:37:20.334045+0000","last_fullsized":"2024-05-10T10:37:20.334045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:30:06.137190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217645+0000","last_change":"2024-05-10T10:37:18.228735+0000","last_active":"2024-05-10T10:37:20.217645+0000","last_peered":"2024-05-10T10:37:20.217645+0000","last_clean":"2024-05-10T10:37:20.217645+0000","last_became_active":"2024-05-10T10:37:18.228430+0000","last_became_peered":"2024-05-10T10:37:18.228430+0000","last_unstale":"2024-05-10T10:37:20.217645+0000","last_undegraded":"2024-05-10T10:37:20.217645+0000","last_fullsized":"2024-05-10T10:37:20.217645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:10:37.891942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333809+0000","last_change":"2024-05-10T10:37:17.230132+0000","last_active":"2024-05-10T10:37:20.333809+0000","last_peered":"2024-05-10T10:37:20.333809+0000","last_clean":"2024-05-10T10:37:20.333809+0000","last_became_active":"2024-05-10T10:37:17.229845+0000","last_became_peered":"2024-05-10T10:37:17.229845+0000","last_unstale":"2024-05-10T10:37:20.333809+0000","last_undegraded":"2024-05-10T10:37:20.333809+0000","last_fullsized":"2024-05-10T10:37:20.333809+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:02:17.506361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334546+0000","last_change":"2024-05-10T10:37:18.231620+0000","last_active":"2024-05-10T10:37:20.334546+0000","last_peered":"2024-05-10T10:37:20.334546+0000","last_clean":"2024-05-10T10:37:20.334546+0000","last_became_active":"2024-05-10T10:37:18.231338+0000","last_became_peered":"2024-05-10T10:37:18.231338+0000","last_unstale":"2024-05-10T10:37:20.334546+0000","last_undegraded":"2024-05-10T10:37:20.334546+0000","last_fullsized":"2024-05-10T10:37:20.334546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:36:45.273502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333353+0000","last_change":"2024-05-10T10:37:17.229451+0000","last_active":"2024-05-10T10:37:20.333353+0000","last_peered":"2024-05-10T10:37:20.333353+0000","last_clean":"2024-05-10T10:37:20.333353+0000","last_became_active":"2024-05-10T10:37:17.228407+0000","last_became_peered":"2024-05-10T10:37:17.228407+0000","last_unstale":"2024-05-10T10:37:20.333353+0000","last_undegraded":"2024-05-10T10:37:20.333353+0000","last_fullsized":"2024-05-10T10:37:20.333353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:32.348704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.226588+0000","last_change":"2024-05-10T10:37:18.228162+0000","last_active":"2024-05-10T10:37:19.226588+0000","last_peered":"2024-05-10T10:37:19.226588+0000","last_clean":"2024-05-10T10:37:19.226588+0000","last_became_active":"2024-05-10T10:37:18.227904+0000","last_became_peered":"2024-05-10T10:37:18.227904+0000","last_unstale":"2024-05-10T10:37:19.226588+0000","last_undegraded":"2024-05-10T10:37:19.226588+0000","last_fullsized":"2024-05-10T10:37:19.226588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:30.119144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227180+0000","last_change":"2024-05-10T10:37:17.224792+0000","last_active":"2024-05-10T10:37:19.227180+0000","last_peered":"2024-05-10T10:37:19.227180+0000","last_clean":"2024-05-10T10:37:19.227180+0000","last_became_active":"2024-05-10T10:37:17.224443+0000","last_became_peered":"2024-05-10T10:37:17.224443+0000","last_unstale":"2024-05-10T10:37:19.227180+0000","last_undegraded":"2024-05-10T10:37:19.227180+0000","last_fullsized":"2024-05-10T10:37:19.227180+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:09.884646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334280+0000","last_change":"2024-05-10T10:37:18.223582+0000","last_active":"2024-05-10T10:37:20.334280+0000","last_peered":"2024-05-10T10:37:20.334280+0000","last_clean":"2024-05-10T10:37:20.334280+0000","last_became_active":"2024-05-10T10:37:18.223309+0000","last_became_peered":"2024-05-10T10:37:18.223309+0000","last_unstale":"2024-05-10T10:37:20.334280+0000","last_undegraded":"2024-05-10T10:37:20.334280+0000","last_fullsized":"2024-05-10T10:37:20.334280+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:49:25.318575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217765+0000","last_change":"2024-05-10T10:37:17.230099+0000","last_active":"2024-05-10T10:37:20.217765+0000","last_peered":"2024-05-10T10:37:20.217765+0000","last_clean":"2024-05-10T10:37:20.217765+0000","last_became_active":"2024-05-10T10:37:17.229704+0000","last_became_peered":"2024-05-10T10:37:17.229704+0000","last_unstale":"2024-05-10T10:37:20.217765+0000","last_undegraded":"2024-05-10T10:37:20.217765+0000","last_fullsized":"2024-05-10T10:37:20.217765+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:40:13.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.228217+0000","last_change":"2024-05-10T10:37:18.226729+0000","last_active":"2024-05-10T10:37:19.228217+0000","last_peered":"2024-05-10T10:37:19.228217+0000","last_clean":"2024-05-10T10:37:19.228217+0000","last_became_active":"2024-05-10T10:37:18.226121+0000","last_became_peered":"2024-05-10T10:37:18.226121+0000","last_unstale":"2024-05-10T10:37:19.228217+0000","last_undegraded":"2024-05-10T10:37:19.228217+0000","last_fullsized":"2024-05-10T10:37:19.228217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:58:39.101389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227219+0000","last_change":"2024-05-10T10:37:17.226288+0000","last_active":"2024-05-10T10:37:19.227219+0000","last_peered":"2024-05-10T10:37:19.227219+0000","last_clean":"2024-05-10T10:37:19.227219+0000","last_became_active":"2024-05-10T10:37:17.226017+0000","last_became_peered":"2024-05-10T10:37:17.226017+0000","last_unstale":"2024-05-10T10:37:19.227219+0000","last_undegraded":"2024-05-10T10:37:19.227219+0000","last_fullsized":"2024-05-10T10:37:19.227219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:21:15.368266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.217237+0000","last_change":"2024-05-10T10:37:18.220014+0000","last_active":"2024-05-10T10:37:20.217237+0000","last_peered":"2024-05-10T10:37:20.217237+0000","last_clean":"2024-05-10T10:37:20.217237+0000","last_became_active":"2024-05-10T10:37:18.219708+0000","last_became_peered":"2024-05-10T10:37:18.219708+0000","last_unstale":"2024-05-10T10:37:20.217237+0000","last_undegraded":"2024-05-10T10:37:20.217237+0000","last_fullsized":"2024-05-10T10:37:20.217237+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:30:46.268281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227077+0000","last_change":"2024-05-10T10:37:17.224891+0000","last_active":"2024-05-10T10:37:19.227077+0000","last_peered":"2024-05-10T10:37:19.227077+0000","last_clean":"2024-05-10T10:37:19.227077+0000","last_became_active":"2024-05-10T10:37:17.224639+0000","last_became_peered":"2024-05-10T10:37:17.224639+0000","last_unstale":"2024-05-10T10:37:19.227077+0000","last_undegraded":"2024-05-10T10:37:19.227077+0000","last_fullsized":"2024-05-10T10:37:19.227077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:22.023190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334163+0000","last_change":"2024-05-10T10:37:18.224853+0000","last_active":"2024-05-10T10:37:20.334163+0000","last_peered":"2024-05-10T10:37:20.334163+0000","last_clean":"2024-05-10T10:37:20.334163+0000","last_became_active":"2024-05-10T10:37:18.224561+0000","last_became_peered":"2024-05-10T10:37:18.224561+0000","last_unstale":"2024-05-10T10:37:20.334163+0000","last_undegraded":"2024-05-10T10:37:20.334163+0000","last_fullsized":"2024-05-10T10:37:20.334163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:43:09.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334027+0000","last_change":"2024-05-10T10:37:17.228371+0000","last_active":"2024-05-10T10:37:20.334027+0000","last_peered":"2024-05-10T10:37:20.334027+0000","last_clean":"2024-05-10T10:37:20.334027+0000","last_became_active":"2024-05-10T10:37:17.227960+0000","last_became_peered":"2024-05-10T10:37:17.227960+0000","last_unstale":"2024-05-10T10:37:20.334027+0000","last_undegraded":"2024-05-10T10:37:20.334027+0000","last_fullsized":"2024-05-10T10:37:20.334027+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:22:54.176214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334237+0000","last_change":"2024-05-10T10:37:18.239787+0000","last_active":"2024-05-10T10:37:20.334237+0000","last_peered":"2024-05-10T10:37:20.334237+0000","last_clean":"2024-05-10T10:37:20.334237+0000","last_became_active":"2024-05-10T10:37:18.239419+0000","last_became_peered":"2024-05-10T10:37:18.239419+0000","last_unstale":"2024-05-10T10:37:20.334237+0000","last_undegraded":"2024-05-10T10:37:20.334237+0000","last_fullsized":"2024-05-10T10:37:20.334237+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:12.522510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.706258+0000","last_change":"2024-05-10T10:37:17.223541+0000","last_active":"2024-05-10T10:37:19.706258+0000","last_peered":"2024-05-10T10:37:19.706258+0000","last_clean":"2024-05-10T10:37:19.706258+0000","last_became_active":"2024-05-10T10:37:17.223228+0000","last_became_peered":"2024-05-10T10:37:17.223228+0000","last_unstale":"2024-05-10T10:37:19.706258+0000","last_undegraded":"2024-05-10T10:37:19.706258+0000","last_fullsized":"2024-05-10T10:37:19.706258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:14:16.038474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333555+0000","last_change":"2024-05-10T10:37:18.241242+0000","last_active":"2024-05-10T10:37:20.333555+0000","last_peered":"2024-05-10T10:37:20.333555+0000","last_clean":"2024-05-10T10:37:20.333555+0000","last_became_active":"2024-05-10T10:37:18.240951+0000","last_became_peered":"2024-05-10T10:37:18.240951+0000","last_unstale":"2024-05-10T10:37:20.333555+0000","last_undegraded":"2024-05-10T10:37:20.333555+0000","last_fullsized":"2024-05-10T10:37:20.333555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:51.413066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334446+0000","last_change":"2024-05-10T10:37:17.232666+0000","last_active":"2024-05-10T10:37:20.334446+0000","last_peered":"2024-05-10T10:37:20.334446+0000","last_clean":"2024-05-10T10:37:20.334446+0000","last_became_active":"2024-05-10T10:37:17.232429+0000","last_became_peered":"2024-05-10T10:37:17.232429+0000","last_unstale":"2024-05-10T10:37:20.334446+0000","last_undegraded":"2024-05-10T10:37:20.334446+0000","last_fullsized":"2024-05-10T10:37:20.334446+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:46:06.968531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.333442+0000","last_change":"2024-05-10T10:37:18.237866+0000","last_active":"2024-05-10T10:37:20.333442+0000","last_peered":"2024-05-10T10:37:20.333442+0000","last_clean":"2024-05-10T10:37:20.333442+0000","last_became_active":"2024-05-10T10:37:18.236491+0000","last_became_peered":"2024-05-10T10:37:18.236491+0000","last_unstale":"2024-05-10T10:37:20.333442+0000","last_undegraded":"2024-05-10T10:37:20.333442+0000","last_fullsized":"2024-05-10T10:37:20.333442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:44:10.887013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.332957+0000","last_change":"2024-05-10T10:37:17.227889+0000","last_active":"2024-05-10T10:37:20.332957+0000","last_peered":"2024-05-10T10:37:20.332957+0000","last_clean":"2024-05-10T10:37:20.332957+0000","last_became_active":"2024-05-10T10:37:17.227312+0000","last_became_peered":"2024-05-10T10:37:17.227312+0000","last_unstale":"2024-05-10T10:37:20.332957+0000","last_undegraded":"2024-05-10T10:37:20.332957+0000","last_fullsized":"2024-05-10T10:37:20.332957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:29.116777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227127+0000","last_change":"2024-05-10T10:37:18.231494+0000","last_active":"2024-05-10T10:37:19.227127+0000","last_peered":"2024-05-10T10:37:19.227127+0000","last_clean":"2024-05-10T10:37:19.227127+0000","last_became_active":"2024-05-10T10:37:18.231219+0000","last_became_peered":"2024-05-10T10:37:18.231219+0000","last_unstale":"2024-05-10T10:37:19.227127+0000","last_undegraded":"2024-05-10T10:37:19.227127+0000","last_fullsized":"2024-05-10T10:37:19.227127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:17.201953+0000","last_clean_scrub_stamp":"2024-05-10T10:37:17.201953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:32:08.408186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:20.334084+0000","last_change":"2024-05-10T10:37:17.230789+0000","last_active":"2024-05-10T10:37:20.334084+0000","last_peered":"2024-05-10T10:37:20.334084+0000","last_clean":"2024-05-10T10:37:20.334084+0000","last_became_active":"2024-05-10T10:37:17.230513+0000","last_became_peered":"2024-05-10T10:37:17.230513+0000","last_unstale":"2024-05-10T10:37:20.334084+0000","last_undegraded":"2024-05-10T10:37:20.334084+0000","last_fullsized":"2024-05-10T10:37:20.334084+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:37:16.199587+0000","last_clean_scrub_stamp":"2024-05-10T10:37:16.199587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:01:38.815016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":219,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-10T10:37:19.227606+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:37:19.227606+0000","last_peered":"2024-05-10T10:37:19.227606+0000","last_clean":"2024-05-10T10:37:19.227606+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:37:19.227606+0000","last_undegraded":"2024-05-10T10:37:19.227606+0000","last_fullsized":"2024-05-10T10:37:19.227606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":1000301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":411523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":909312,"data_stored":409945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":925696,"data_stored":411627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":909312,"data_stored":410493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":410653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":913408,"data_stored":410023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1499136,"data_stored":1000279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:37:27.118 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:37:27.118 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T10:37:27.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.265+0000 7fd2a29c8640 1 Processor -- start 2024-05-10T10:37:27.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.265+0000 7fd2a29c8640 1 -- start start 2024-05-10T10:37:27.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd2a29c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd29c0d2d10 0x7fd29c0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd2a29c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0d1f50 0x7fd29c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd2a29c8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd29c0d31a0 con 0x7fd29c0d3df0 2024-05-10T10:37:27.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd2a29c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd29c0d23c0 con 0x7fd29c0d2d10 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd2a29c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd29c0cfd50 con 0x7fd29c0d1f50 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd29c0d3df0 0x7fd29c0b4aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42106/0 (socket says 172.21.15.53:42106) 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29bfff640 1 -- 172.21.15.53:0/3910858715 learned_addr learned my addr 172.21.15.53:0/3910858715 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768820751 0 0) 0x7fd29c0d31a0 con 0x7fd29c0d3df0 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd284003520 con 0x7fd29c0d3df0 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd2a0f65640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0d1f50 0x7fd29c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29b7fe640 1 --2- 172.21.15.53:0/3910858715 >> v2:172.21.15.135:3300/0 conn(0x7fd29c0d2d10 0x7fd29c0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 337955913 0 0) 0x7fd284003520 con 0x7fd29c0d3df0 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0d1f50 msgr2=0x7fd29c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0d1f50 0x7fd29c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 >> v2:172.21.15.135:3300/0 conn(0x7fd29c0d2d10 msgr2=0x7fd29c0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 --2- 172.21.15.53:0/3910858715 >> v2:172.21.15.135:3300/0 conn(0x7fd29c0d2d10 0x7fd29c0d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c0d41d0 con 0x7fd29c0d3df0 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fd28c001670 con 0x7fd29c0d3df0 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29b7fe640 1 --2- 172.21.15.53:0/3910858715 >> v2:172.21.15.135:3300/0 conn(0x7fd29c0d2d10 0x7fd29c0d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 >> v1:172.21.15.53:6789/0 conn(0x7fd29c0d3df0 legacy=0x7fd29c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2840039b0 0x7fd284003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd284004800 0x7fd284004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840051c0 0x7fd28400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c0d23c0 con 0x7fd2840039b0 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd28400bfb0 con 0x7fd2840051c0 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd28400c180 con 0x7fd284004800 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd2a0f65640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0d1f50 0x7fd29c0d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.266+0000 7fd29bfff640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2840039b0 0x7fd284003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.267+0000 7fd29b7fe640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd284004800 0x7fd284004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.267+0000 7fd2a0f65640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840051c0 0x7fd28400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.267+0000 7fd29bfff640 1 -- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd284004800 msgr2=0x7fd284004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.267+0000 7fd29bfff640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd284004800 0x7fd284004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.267+0000 7fd29bfff640 1 -- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840051c0 msgr2=0x7fd28400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.267+0000 7fd29bfff640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840051c0 0x7fd28400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.267+0000 7fd29bfff640 1 -- 172.21.15.53:0/3910858715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd284003520 con 0x7fd2840039b0 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd29bfff640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2840039b0 0x7fd284003dd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c240 tx=0x7fd284005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4a11b5927bbff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c007d50 con 0x7fd2840039b0 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c02e070 con 0x7fd2840039b0 2024-05-10T10:37:27.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd29affd640 1 -- 172.21.15.53:0/3910858715 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c005d30 con 0x7fd2840039b0 2024-05-10T10:37:27.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3910858715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2840039b0 msgr2=0x7fd284003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd2a29c8640 1 --2- 172.21.15.53:0/3910858715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2840039b0 0x7fd284003dd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c240 tx=0x7fd284005830 comp rx=0 tx=0).stop 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd2a0f65640 1 -- 172.21.15.53:0/3910858715 reap_dead start 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3910858715 shutdown_connections 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3910858715 >> 172.21.15.53:0/3910858715 conn(0x7fd29c09e450 msgr2=0x7fd29c0d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3910858715 shutdown_connections 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.268+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3910858715 wait complete. 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.269+0000 7fd2a29c8640 1 Processor -- start 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.270+0000 7fd2a29c8640 1 -- start start 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.270+0000 7fd2a29c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840039b0 0x7fd29c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.270+0000 7fd2a29c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0ba560 0x7fd29c154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.270+0000 7fd2a29c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd29c0ce2d0 0x7fd29c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.270+0000 7fd2a29c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c0b57d0 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.270+0000 7fd2a29c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c0b5d90 con 0x7fd2840039b0 2024-05-10T10:37:27.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.270+0000 7fd2a29c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd29c0b5ab0 con 0x7fd29c0ba560 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd29c0ce2d0 0x7fd29c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd29c0ce2d0 0x7fd29c0ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54192/0 (socket says 172.21.15.53:54192) 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 -- 172.21.15.53:0/3239151559 learned_addr learned my addr 172.21.15.53:0/3239151559 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 -- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0ba560 msgr2=0x7fd29c154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0ba560 0x7fd29c154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 -- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840039b0 msgr2=0x7fd29c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840039b0 0x7fd29c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 -- 172.21.15.53:0/3239151559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c0cdae0 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a0f65640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd29c0ce2d0 0x7fd29c0ce750 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0d2330 tx=0x7fd294007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd298ff9640 1 -- 172.21.15.53:0/3239151559 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294015480 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd298ff9640 1 -- 172.21.15.53:0/3239151559 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd29401a070 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29c0cd520 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd298ff9640 1 -- 172.21.15.53:0/3239151559 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294015860 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.271+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29c0b7de0 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.272+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29c06a220 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.273+0000 7fd298ff9640 1 -- 172.21.15.53:0/3239151559 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd294015a00 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.274+0000 7fd298ff9640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2880762d0 0x7fd288078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.274+0000 7fd298ff9640 1 -- 172.21.15.53:0/3239151559 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fd2940983d0 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.274+0000 7fd29bfff640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2880762d0 0x7fd288078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.274+0000 7fd29bfff640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2880762d0 0x7fd288078790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd28c033000 tx=0x7fd28c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.278+0000 7fd298ff9640 1 -- 172.21.15.53:0/3239151559 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd294010050 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.540+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd29c0d27c0 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.542+0000 7fd298ff9640 1 -- 172.21.15.53:0/3239151559 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fd294061b00 con 0x7fd29c0ce2d0 2024-05-10T10:37:27.543 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:37:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:37:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2024-05-10T10:37:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2024-05-10T10:37:27.543 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-10T10:37:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2024-05-10T10:37:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2880762d0 msgr2=0x7fd288078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2880762d0 0x7fd288078790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd28c033000 tx=0x7fd28c007000 comp rx=0 tx=0).stop 2024-05-10T10:37:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd29c0ce2d0 msgr2=0x7fd29c0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd29c0ce2d0 0x7fd29c0ce750 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0d2330 tx=0x7fd294007700 comp rx=0 tx=0).stop 2024-05-10T10:37:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 shutdown_connections 2024-05-10T10:37:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2880762d0 0x7fd288078790 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd29c0ce2d0 0x7fd29c0ce750 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd29c0ba560 0x7fd29c154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 --2- 172.21.15.53:0/3239151559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2840039b0 0x7fd29c0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 >> 172.21.15.53:0/3239151559 conn(0x7fd29c09e450 msgr2=0x7fd29c06e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 shutdown_connections 2024-05-10T10:37:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.545+0000 7fd2a29c8640 1 -- 172.21.15.53:0/3239151559 wait complete. 2024-05-10T10:37:27.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:27.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.660+0000 7f3dd15af640 1 Processor -- start 2024-05-10T10:37:27.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.660+0000 7f3dd15af640 1 -- start start 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dd15af640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3dcc15f8c0 0x7f3dcc06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dd15af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3dcc06e620 0x7f3dcc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dd15af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc0b0a90 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dd15af640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc0b23f0 con 0x7f3dcc15f8c0 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dd15af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc0b4150 con 0x7f3dcc06e620 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dca7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3dcc15f8c0 0x7f3dcc06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dca7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3dcc15f8c0 0x7f3dcc06e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54196/0 (socket says 172.21.15.53:54196) 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dca7fc640 1 -- 172.21.15.53:0/963728416 learned_addr learned my addr 172.21.15.53:0/963728416 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:27.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dcb7fe640 1 --2- 172.21.15.53:0/963728416 >> v2:172.21.15.135:3300/0 conn(0x7f3dcc06e620 0x7f3dcc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dca7fc640 1 -- 172.21.15.53:0/963728416 >> v2:172.21.15.135:3300/0 conn(0x7f3dcc06e620 msgr2=0x7f3dcc0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dca7fc640 1 --2- 172.21.15.53:0/963728416 >> v2:172.21.15.135:3300/0 conn(0x7f3dcc06e620 0x7f3dcc0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dca7fc640 1 -- 172.21.15.53:0/963728416 >> v1:172.21.15.135:6789/0 conn(0x7f3dcc15f4e0 legacy=0x7f3dcc06d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dca7fc640 1 -- 172.21.15.53:0/963728416 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0b3540 con 0x7f3dcc15f8c0 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.661+0000 7f3dcb7fe640 1 --2- 172.21.15.53:0/963728416 >> v2:172.21.15.135:3300/0 conn(0x7f3dcc06e620 0x7f3dcc0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.662+0000 7f3dca7fc640 1 --2- 172.21.15.53:0/963728416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f8c0 0x7f3dcc06e0e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0508e0 tx=0x7f3dc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bb0a70466a0912 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.662+0000 7f3dc9ffb640 1 -- 172.21.15.53:0/963728416 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0037070 con 0x7f3dcc15f8c0 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.662+0000 7f3dc9ffb640 1 -- 172.21.15.53:0/963728416 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc0032440 con 0x7f3dcc15f8c0 2024-05-10T10:37:27.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.662+0000 7f3dc9ffb640 1 -- 172.21.15.53:0/963728416 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0032810 con 0x7f3dcc15f8c0 2024-05-10T10:37:27.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 -- 172.21.15.53:0/963728416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f8c0 msgr2=0x7f3dcc06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 --2- 172.21.15.53:0/963728416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f8c0 0x7f3dcc06e0e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0508e0 tx=0x7f3dc002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 -- 172.21.15.53:0/963728416 shutdown_connections 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 --2- 172.21.15.53:0/963728416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f8c0 0x7f3dcc06e0e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 --2- 172.21.15.53:0/963728416 >> v2:172.21.15.135:3300/0 conn(0x7f3dcc06e620 0x7f3dcc0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 -- 172.21.15.53:0/963728416 >> 172.21.15.53:0/963728416 conn(0x7f3dcc0d2df0 msgr2=0x7f3dcc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 -- 172.21.15.53:0/963728416 shutdown_connections 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.663+0000 7f3dd15af640 1 -- 172.21.15.53:0/963728416 wait complete. 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.664+0000 7f3dd15af640 1 Processor -- start 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.664+0000 7f3dd15af640 1 -- start start 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dd15af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc06e620 0x7f3dcc1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dd15af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f4e0 0x7f3dcc1552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dd15af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc15f8c0 0x7f3dcc1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dd15af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0b3de0 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dd15af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0b3b00 con 0x7f3dcc15f8c0 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dd15af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0b45e0 con 0x7f3dcc06e620 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f4e0 0x7f3dcc1552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f4e0 0x7f3dcc1552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54204/0 (socket says 172.21.15.53:54204) 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 -- 172.21.15.53:0/1312804775 learned_addr learned my addr 172.21.15.53:0/1312804775 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dcaffd640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc06e620 0x7f3dcc1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 -- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc06e620 msgr2=0x7f3dcc1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc06e620 0x7f3dcc1f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 -- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc15f8c0 msgr2=0x7f3dcc1579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dcb7fe640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc15f8c0 0x7f3dcc1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc15f8c0 0x7f3dcc1579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.665+0000 7f3dca7fc640 1 -- 172.21.15.53:0/1312804775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc157ef0 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.666+0000 7f3dca7fc640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f4e0 0x7f3dcc1552a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3dc002a220 tx=0x7f3dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.666+0000 7f3db37fe640 1 -- 172.21.15.53:0/1312804775 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0041020 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.666+0000 7f3db37fe640 1 -- 172.21.15.53:0/1312804775 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc00306c0 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.666+0000 7f3db37fe640 1 -- 172.21.15.53:0/1312804775 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0037040 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.666+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1581b0 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.666+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc1f5050 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.667+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc06a220 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.667+0000 7f3db37fe640 1 -- 172.21.15.53:0/1312804775 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3dc0044020 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.668+0000 7f3db37fe640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db80762d0 0x7f3db8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.668+0000 7f3db37fe640 1 -- 172.21.15.53:0/1312804775 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f3dc00b7f90 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.668+0000 7f3dcaffd640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db80762d0 0x7f3db8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.669+0000 7f3dcaffd640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db80762d0 0x7f3db8078790 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0015a0 tx=0x7f3dbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:27.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.672+0000 7f3db37fe640 1 -- 172.21.15.53:0/1312804775 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3dc004b030 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.907+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dcc15ed00 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.908+0000 7f3db37fe640 1 -- 172.21.15.53:0/1312804775 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5186 (secure 0 0 0) 0x7f3dc0081760 con 0x7f3dcc15f4e0 2024-05-10T10:37:27.909 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:27.909 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:37:20.609173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6834/1984524370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1984524370},{"type":"v1","addr":"172.21.15.53:6834","nonce":1984524370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6102,"qdb_cluster":[6102]},"id":5}]} 2024-05-10T10:37:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 45 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db80762d0 msgr2=0x7f3db8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db80762d0 0x7f3db8078790 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0015a0 tx=0x7f3dbc009040 comp rx=0 tx=0).stop 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f4e0 msgr2=0x7f3dcc1552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f4e0 0x7f3dcc1552a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3dc002a220 tx=0x7f3dc00046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 shutdown_connections 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db80762d0 0x7f3db8078790 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc15f8c0 0x7f3dcc1579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc15f4e0 0x7f3dcc1552a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 --2- 172.21.15.53:0/1312804775 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc06e620 0x7f3dcc1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 >> 172.21.15.53:0/1312804775 conn(0x7f3dcc0d2df0 msgr2=0x7f3dcc06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 shutdown_connections 2024-05-10T10:37:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:27.910+0000 7f3dd15af640 1 -- 172.21.15.53:0/1312804775 wait complete. 2024-05-10T10:37:28.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:37:17.592875+0000', 'modified': '2024-05-10T10:37:20.609173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6102}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6102': {'gid': 6102, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6834/1984524370', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1984524370}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 1984524370}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6102, 'qdb_cluster': [6102]} 2024-05-10T10:37:28.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:37:28.325 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T10:37:28.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.435+0000 7f101db19640 1 Processor -- start 2024-05-10T10:37:28.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.435+0000 7f101db19640 1 -- start start 2024-05-10T10:37:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f101db19640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 0x7f10180d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f101db19640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1018160c90 con 0x7f10180d1230 2024-05-10T10:37:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f101db19640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10180ce5f0 con 0x7f10180d03c0 2024-05-10T10:37:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f101db19640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f10180ce8d0 con 0x7f10181608b0 2024-05-10T10:37:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f1016ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f10180d1230 0x7f10180b2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42118/0 (socket says 172.21.15.53:42118) 2024-05-10T10:37:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f1016ffd640 1 -- 172.21.15.53:0/3407977638 learned_addr learned my addr 172.21.15.53:0/3407977638 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:28.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2510663504 0 0) 0x7f100c001110 con 0x7f10180d1230 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1000003520 con 0x7f10180d1230 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10177fe640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 0x7f10180d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1794346736 0 0) 0x7f1000003520 con 0x7f10180d1230 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 msgr2=0x7f10180d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 0x7f10180d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 >> v1:172.21.15.135:6789/0 conn(0x7f10180d03c0 legacy=0x7f10180b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10180cebb0 con 0x7f10180d1230 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f100c001840 con 0x7f10180d1230 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 >> v1:172.21.15.53:6789/0 conn(0x7f10180d1230 legacy=0x7f10180b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10000039e0 0x7f1000003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1000004800 0x7f1000004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 0x7f1000005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10180ce8d0 con 0x7f10000039e0 2024-05-10T10:37:28.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f100000baf0 con 0x7f1000004800 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.436+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f100000bcf0 con 0x7f10181608b0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f10177fe640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10000039e0 0x7f1000003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f1016ffd640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1000004800 0x7f1000004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f1017fff640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 0x7f1000005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f10177fe640 1 -- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 msgr2=0x7f1000005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f10177fe640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 0x7f1000005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f10177fe640 1 -- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1000004800 msgr2=0x7f1000004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f10177fe640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1000004800 0x7f1000004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f10177fe640 1 -- 172.21.15.53:0/3407977638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1000003520 con 0x7f10000039e0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.437+0000 7f1017fff640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10181608b0 0x7f1000005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f10177fe640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10000039e0 0x7f1000003e00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f100800ba30 tx=0x7f1000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2529a52d0dbd919 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008030070 con 0x7f10000039e0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100802ade0 con 0x7f10000039e0 2024-05-10T10:37:28.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f10167fc640 1 -- 172.21.15.53:0/3407977638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100800b600 con 0x7f10000039e0 2024-05-10T10:37:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f101db19640 1 -- 172.21.15.53:0/3407977638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10000039e0 msgr2=0x7f1000003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f101db19640 1 --2- 172.21.15.53:0/3407977638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10000039e0 0x7f1000003e00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f100800ba30 tx=0x7f1000005830 comp rx=0 tx=0).stop 2024-05-10T10:37:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f1017fff640 1 -- 172.21.15.53:0/3407977638 reap_dead start 2024-05-10T10:37:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f101db19640 1 -- 172.21.15.53:0/3407977638 shutdown_connections 2024-05-10T10:37:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f101db19640 1 -- 172.21.15.53:0/3407977638 >> 172.21.15.53:0/3407977638 conn(0x7f10180d2df0 msgr2=0x7f10180d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f101db19640 1 -- 172.21.15.53:0/3407977638 shutdown_connections 2024-05-10T10:37:28.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.438+0000 7f101db19640 1 -- 172.21.15.53:0/3407977638 wait complete. 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.439+0000 7f101db19640 1 Processor -- start 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f101db19640 1 -- start start 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f101db19640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10000039e0 0x7f10181f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f101db19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10180b8770 0x7f10181f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f101db19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10180c8860 0x7f10180cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f101db19640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10180d1f10 con 0x7f10180c8860 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f101db19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10180d1d90 con 0x7f10180b8770 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f101db19640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f10180d0cf0 con 0x7f10000039e0 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.440+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10180c8860 0x7f10180cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10180c8860 0x7f10180cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54214/0 (socket says 172.21.15.53:54214) 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 -- 172.21.15.53:0/4043281722 learned_addr learned my addr 172.21.15.53:0/4043281722 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 -- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10000039e0 msgr2=0x7f10181f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f10177fe640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10000039e0 0x7f10181f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1016ffd640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10180b8770 0x7f10181f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10000039e0 0x7f10181f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 -- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10180b8770 msgr2=0x7f10181f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10180b8770 0x7f10181f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 -- 172.21.15.53:0/4043281722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10180c9260 con 0x7f10180c8860 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f10177fe640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10000039e0 0x7f10181f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1016ffd640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10180b8770 0x7f10181f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f1017fff640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10180c8860 0x7f10180cb9b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1010000f80 tx=0x7f101000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f0ffffff640 1 -- 172.21.15.53:0/4043281722 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010006b50 con 0x7f10180c8860 2024-05-10T10:37:28.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f0ffffff640 1 -- 172.21.15.53:0/4043281722 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1010006d10 con 0x7f10180c8860 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f0ffffff640 1 -- 172.21.15.53:0/4043281722 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010003970 con 0x7f10180c8860 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10180c9540 con 0x7f10180c8860 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.441+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10180c9b00 con 0x7f10180c8860 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.443+0000 7f0ffffff640 1 -- 172.21.15.53:0/4043281722 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f101001a020 con 0x7f10180c8860 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.443+0000 7f0ffffff640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f10040762d0 0x7f1004078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.444+0000 7f0ffffff640 1 -- 172.21.15.53:0/4043281722 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f1010096ef0 con 0x7f10180c8860 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.444+0000 7f10177fe640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f10040762d0 0x7f1004078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.444+0000 7f1020836740 1 Processor -- start 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.444+0000 7f1020836740 1 -- start start 2024-05-10T10:37:28.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.444+0000 7f10177fe640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f10040762d0 0x7f1004078790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f100800ba00 tx=0x7f1008030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a3f50 0x55f36e3a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a4330 0x55f36e3a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1020836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e376bb0 0x55f36e37a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1020836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f36e2e08d0 con 0x55f36e376bb0 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f36e299c10 con 0x55f36e3a4330 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f36e2dc890 con 0x55f36e3a3f50 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e376bb0 0x55f36e37a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e376bb0 0x55f36e37a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54220/0 (socket says 172.21.15.53:54220) 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 -- 172.21.15.53:0/1937255884 learned_addr learned my addr 172.21.15.53:0/1937255884 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f10177fe640 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a3f50 0x55f36e3a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1016ffd640 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a4330 0x55f36e3a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 -- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a3f50 msgr2=0x55f36e3a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a3f50 0x55f36e3a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 -- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a4330 msgr2=0x55f36e3a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a4330 0x55f36e3a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.445+0000 7f1017fff640 1 -- 172.21.15.53:0/1937255884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36e2a15d0 con 0x55f36e376bb0 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.446+0000 7f1017fff640 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e376bb0 0x55f36e37a3c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f10100013d0 tx=0x7f101005f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156ac934d5d49ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.446+0000 7f0ffd7fa640 1 -- 172.21.15.53:0/1937255884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101005ff20 con 0x55f36e376bb0 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.446+0000 7f0ffd7fa640 1 -- 172.21.15.53:0/1937255884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10100642d0 con 0x55f36e376bb0 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.446+0000 7f0ffd7fa640 1 -- 172.21.15.53:0/1937255884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101005f380 con 0x55f36e376bb0 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.446+0000 7f1020836740 1 -- 172.21.15.53:0/1937255884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e376bb0 msgr2=0x55f36e37a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.446+0000 7f1020836740 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e376bb0 0x55f36e37a3c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f10100013d0 tx=0x7f101005f750 comp rx=0 tx=0).stop 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 -- 172.21.15.53:0/1937255884 shutdown_connections 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e376bb0 0x55f36e37a3c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a4330 0x55f36e3a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 --2- 172.21.15.53:0/1937255884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a3f50 0x55f36e3a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 -- 172.21.15.53:0/1937255884 >> 172.21.15.53:0/1937255884 conn(0x55f36e378e20 msgr2=0x55f36e3c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 -- 172.21.15.53:0/1937255884 shutdown_connections 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 -- 172.21.15.53:0/1937255884 wait complete. 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 -- start start 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.447+0000 7f1020836740 1 Processor -- start 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3aa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e39dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1020836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e39e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1020836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f36e37bfd0 con 0x55f36e3c38e0 2024-05-10T10:37:28.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f36e29d760 con 0x55f36e3a3f50 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f36e37c2b0 con 0x55f36e3a4330 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1020836740 10 client.0 _ll_register_callbacks cb 0x55f36e2d6cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e39e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e39e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54228/0 (socket says 172.21.15.53:54228) 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 -- 172.21.15.53:0/294694780 learned_addr learned my addr 172.21.15.53:0/294694780 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f10177fe640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1016ffd640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e39dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 -- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 msgr2=0x55f36e39dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e39dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 -- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 msgr2=0x55f36e3aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3aa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f1017fff640 1 -- 172.21.15.53:0/294694780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36dff22d0 con 0x55f36e3c38e0 2024-05-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.448+0000 7f10177fe640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1017fff640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e39e250 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f101005d190 tx=0x7f1010001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa73206d4f1d2fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f0ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f0ff67fc640 1 -- 172.21.15.53:0/294694780 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101005ff20 con 0x55f36e3c38e0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f0ff67fc640 1 -- 172.21.15.53:0/294694780 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f1010003b10 con 0x55f36e3c38e0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10100642d0 con 0x55f36e3c38e0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f0ff67fc640 1 -- 172.21.15.53:0/294694780 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1010003d10 con 0x55f36e3c38e0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 10 client.6255 fetch_fsmap learned FSMap version 45 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55f36e0bc6b0 con 0x55f36e3c38e0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f0ff67fc640 1 -- 172.21.15.53:0/294694780 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10100642d0 con 0x55f36e3c38e0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f0ff67fc640 1 -- 172.21.15.53:0/294694780 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f1010081380 con 0x55f36e3c38e0 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 10 client.6255 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 10 client.6255 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 20 client.6255 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3d5220 0x55f36e3d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 4 client.6255 mds_command: new command op to 6102 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 --> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f36e264760 con 0x55f36e3d5220 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.449+0000 7f10177fe640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3d5220 0x55f36e3d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.450+0000 7f10177fe640 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3d5220 0x55f36e3d5600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2a61e620593a25b server_cookie=2848ed2a7671fa68 in_seq=0 out_seq=0 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.450+0000 7f0ff67fc640 10 client.6255 ms_handle_connect on v2:172.21.15.53:6832/1984524370 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.450+0000 7f0ff67fc640 1 -- 172.21.15.53:0/294694780 <== mds.0 v2:172.21.15.53:6832/1984524370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f36e264760 con 0x55f36e3d5220 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.450+0000 7f0ff67fc640 10 client.6255 handle_command_reply: tid=0 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.450+0000 7f1020836740 1 client.6255 shutdown 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 msgr2=0x55f36e39e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e39e250 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f101005d190 tx=0x7f1010001360 comp rx=0 tx=0).stop 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 shutdown_connections 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3d5220 0x55f36e3d5600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e39e250 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e39dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 --2- 172.21.15.53:0/294694780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 >> 172.21.15.53:0/294694780 conn(0x55f36e376bb0 msgr2=0x55f36e3a6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f0ff67fc640 0 client.6255 ms_handle_reset on v2:172.21.15.53:6832/1984524370 2024-05-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 shutdown_connections 2024-05-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 1 -- 172.21.15.53:0/294694780 wait complete. 2024-05-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.451+0000 7f1020836740 20 client.6255 trim_cache size 0 max 16384 2024-05-10T10:37:28.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.469+0000 7f1020836740 1 Processor -- start 2024-05-10T10:37:28.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1020836740 1 -- start start 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1020836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3a4330 0x55f36e3a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3c38e0 0x55f36e3a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1020836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1018201860 con 0x55f36e3a4330 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10180b7760 con 0x55f36e3a3f50 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f10182011f0 con 0x55f36e3c38e0 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.470+0000 7f1016ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3a4330 0x55f36e3a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3a4330 0x55f36e3a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54244/0 (socket says 172.21.15.53:54244) 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 -- 172.21.15.53:0/887056401 learned_addr learned my addr 172.21.15.53:0/887056401 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1017fff640 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3c38e0 0x55f36e3a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f10177fe640 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 -- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3c38e0 msgr2=0x55f36e3a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3c38e0 0x55f36e3a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 -- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 msgr2=0x55f36e3be780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3be780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 -- 172.21.15.53:0/887056401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36e2a00f0 con 0x55f36e3a4330 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f10177fe640 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f1016ffd640 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3a4330 0x55f36e3a3050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f100c001eb0 tx=0x7f100c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a36dc2bc68695ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f0ffcff9640 1 -- 172.21.15.53:0/887056401 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c01c070 con 0x55f36e3a4330 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f0ffcff9640 1 -- 172.21.15.53:0/887056401 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c0075a0 con 0x55f36e3a4330 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.471+0000 7f0ffcff9640 1 -- 172.21.15.53:0/887056401 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c007750 con 0x55f36e3a4330 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 -- 172.21.15.53:0/887056401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3a4330 msgr2=0x55f36e3a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3a4330 0x55f36e3a3050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f100c001eb0 tx=0x7f100c004bb0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 -- 172.21.15.53:0/887056401 shutdown_connections 2024-05-10T10:37:28.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3c38e0 0x55f36e3a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3a4330 0x55f36e3a3050 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 --2- 172.21.15.53:0/887056401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 -- 172.21.15.53:0/887056401 >> 172.21.15.53:0/887056401 conn(0x55f36e376bb0 msgr2=0x55f36e3a6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 -- 172.21.15.53:0/887056401 shutdown_connections 2024-05-10T10:37:28.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 -- 172.21.15.53:0/887056401 wait complete. 2024-05-10T10:37:28.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 -- start start 2024-05-10T10:37:28.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.472+0000 7f1020836740 1 Processor -- start 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3bea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1020836740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e3a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1020836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e3b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1020836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f36e37bfd0 con 0x55f36e3c38e0 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f36e29d760 con 0x55f36e3a3f50 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1020836740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f36e3c7c60 con 0x55f36e3a4330 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1020836740 10 client.0 _ll_register_callbacks cb 0x7f101005f870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e3b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1017fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e3b78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54250/0 (socket says 172.21.15.53:54250) 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1017fff640 1 -- 172.21.15.53:0/2467758638 learned_addr learned my addr 172.21.15.53:0/2467758638 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f1016ffd640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e3a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.473+0000 7f10177fe640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1017fff640 1 -- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 msgr2=0x55f36e3a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1017fff640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e3a7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1017fff640 1 -- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 msgr2=0x55f36e3bea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1017fff640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3bea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1017fff640 1 -- 172.21.15.53:0/2467758638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36e3a7f40 con 0x55f36e3c38e0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1016ffd640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e3a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1017fff640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e3b78a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f101005feb0 tx=0x7f101005c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385c25cf7200b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f0ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f0ff67fc640 1 -- 172.21.15.53:0/2467758638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10100642d0 con 0x55f36e3c38e0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f0ff67fc640 1 -- 172.21.15.53:0/2467758638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f1010003b10 con 0x55f36e3c38e0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100c0075a0 con 0x55f36e3c38e0 2024-05-10T10:37:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f0ff67fc640 1 -- 172.21.15.53:0/2467758638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f101005ff20 con 0x55f36e3c38e0 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 10 client.6261 fetch_fsmap learned FSMap version 45 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55f36e3a7f40 con 0x55f36e3c38e0 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f0ff67fc640 1 -- 172.21.15.53:0/2467758638 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c0075a0 con 0x55f36e3c38e0 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f0ff67fc640 1 -- 172.21.15.53:0/2467758638 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f101005b890 con 0x55f36e3c38e0 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 10 client.6261 fetch_fsmap finished waiting for FSMap version 45 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 10 client.6261 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 20 client.6261 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3e46a0 0x55f36e3e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 4 client.6261 mds_command: new command op to 6102 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 --> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] -- command(tid 0: {"prefix": "status"}) -- 0x55f36e37c620 con 0x55f36e3e46a0 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.474+0000 7f10177fe640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3e46a0 0x55f36e3e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f0ff67fc640 10 client.6261 ms_handle_connect on v2:172.21.15.53:6832/1984524370 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f10177fe640 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3e46a0 0x55f36e3e4a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10c486076991031 server_cookie=ddb34462d1d3e184 in_seq=0 out_seq=0 2024-05-10T10:37:28.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f0ff67fc640 1 -- 172.21.15.53:0/2467758638 <== mds.0 v2:172.21.15.53:6832/1984524370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f36e37c620 con 0x55f36e3e46a0 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f0ff67fc640 10 client.6261 handle_command_reply: tid=0 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 client.6261 shutdown 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 msgr2=0x55f36e3b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e3b78a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f101005feb0 tx=0x7f101005c6a0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 shutdown_connections 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:6832/1984524370,v1:172.21.15.53:6834/1984524370] conn(0x55f36e3e46a0 0x55f36e3e4a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f36e3c38e0 0x55f36e3b78a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f0ff67fc640 0 client.6261 ms_handle_reset on v2:172.21.15.53:6832/1984524370 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f36e3a4330 0x55f36e3a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 --2- 172.21.15.53:0/2467758638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f36e3a3f50 0x55f36e3bea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 >> 172.21.15.53:0/2467758638 conn(0x55f36e376bb0 msgr2=0x55f36e3d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 shutdown_connections 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 1 -- 172.21.15.53:0/2467758638 wait complete. 2024-05-10T10:37:28.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.475+0000 7f1020836740 20 client.6261 trim_cache size 0 max 16384 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 6102, 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 10.265229629, 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 45, 2024-05-10T10:37:28.483 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 74, 2024-05-10T10:37:28.484 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 74, 2024-05-10T10:37:28.484 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 15.251930872999999 2024-05-10T10:37:28.484 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:37:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f10040762d0 msgr2=0x7f1004078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f10040762d0 0x7f1004078790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f100800ba00 tx=0x7f1008030000 comp rx=0 tx=0).stop 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10180c8860 msgr2=0x7f10180cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10180c8860 0x7f10180cb9b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1010000f80 tx=0x7f101000eb90 comp rx=0 tx=0).stop 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 shutdown_connections 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f10040762d0 0x7f1004078790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10180c8860 0x7f10180cb9b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10180b8770 0x7f10181f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 --2- 172.21.15.53:0/4043281722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10000039e0 0x7f10181f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 >> 172.21.15.53:0/4043281722 conn(0x7f10180d2df0 msgr2=0x7f101815dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 shutdown_connections 2024-05-10T10:37:28.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.476+0000 7f101db19640 1 -- 172.21.15.53:0/4043281722 wait complete. 2024-05-10T10:37:28.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:37:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.591+0000 7fc80fdc1640 1 Processor -- start 2024-05-10T10:37:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.591+0000 7fc80fdc1640 1 -- start start 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80fdc1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc808162940 0x7fc8080b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80fdc1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc8080d3d80 0x7fc8080d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80fdc1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8080d2d20 con 0x7fc8080d59d0 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80fdc1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc808162d20 con 0x7fc808162940 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80fdc1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc8080d3000 con 0x7fc8080d3d80 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80db5d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc808162940 0x7fc8080b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80db5d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc808162940 0x7fc8080b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54254/0 (socket says 172.21.15.53:54254) 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.592+0000 7fc80db5d640 1 -- 172.21.15.53:0/534288858 learned_addr learned my addr 172.21.15.53:0/534288858 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.593+0000 7fc80e35e640 1 --2- 172.21.15.53:0/534288858 >> v2:172.21.15.135:3300/0 conn(0x7fc8080d3d80 0x7fc8080d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.593+0000 7fc80db5d640 1 -- 172.21.15.53:0/534288858 >> v2:172.21.15.135:3300/0 conn(0x7fc8080d3d80 msgr2=0x7fc8080d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.593+0000 7fc80db5d640 1 --2- 172.21.15.53:0/534288858 >> v2:172.21.15.135:3300/0 conn(0x7fc8080d3d80 0x7fc8080d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.593+0000 7fc80db5d640 1 -- 172.21.15.53:0/534288858 >> v1:172.21.15.135:6789/0 conn(0x7fc8080d59d0 legacy=0x7fc8080d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.593+0000 7fc80db5d640 1 -- 172.21.15.53:0/534288858 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8080d32e0 con 0x7fc808162940 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.593+0000 7fc80db5d640 1 --2- 172.21.15.53:0/534288858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc808162940 0x7fc8080b5e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc8080d5e50 tx=0x7fc7f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4df71273fa3e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80cb5b640 1 -- 172.21.15.53:0/534288858 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8036070 con 0x7fc808162940 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80cb5b640 1 -- 172.21.15.53:0/534288858 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8031600 con 0x7fc808162940 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80cb5b640 1 -- 172.21.15.53:0/534288858 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80319d0 con 0x7fc808162940 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 -- 172.21.15.53:0/534288858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc808162940 msgr2=0x7fc8080b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/534288858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc808162940 0x7fc8080b5e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc8080d5e50 tx=0x7fc7f802a6a0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 -- 172.21.15.53:0/534288858 shutdown_connections 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/534288858 >> v2:172.21.15.135:3300/0 conn(0x7fc8080d3d80 0x7fc8080d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/534288858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc808162940 0x7fc8080b5e20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 -- 172.21.15.53:0/534288858 >> 172.21.15.53:0/534288858 conn(0x7fc80809e380 msgr2=0x7fc8080d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 -- 172.21.15.53:0/534288858 shutdown_connections 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.594+0000 7fc80fdc1640 1 -- 172.21.15.53:0/534288858 wait complete. 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.595+0000 7fc80fdc1640 1 Processor -- start 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.595+0000 7fc80fdc1640 1 -- start start 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80fdc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8080d3d80 0x7fc808159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80fdc1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8080d59d0 0x7fc8080cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80fdc1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc808162940 0x7fc808158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80fdc1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8080d2390 con 0x7fc8080d3d80 2024-05-10T10:37:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80fdc1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8080cfc50 con 0x7fc808162940 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80fdc1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc8080d2510 con 0x7fc8080d59d0 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80db5d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8080d3d80 0x7fc808159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80db5d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8080d3d80 0x7fc808159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54268/0 (socket says 172.21.15.53:54268) 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.596+0000 7fc80db5d640 1 -- 172.21.15.53:0/1871397069 learned_addr learned my addr 172.21.15.53:0/1871397069 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80e35e640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc808162940 0x7fc808158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80d35c640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8080d59d0 0x7fc8080cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80db5d640 1 -- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8080d59d0 msgr2=0x7fc8080cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80db5d640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8080d59d0 0x7fc8080cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80db5d640 1 -- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc808162940 msgr2=0x7fc808158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80db5d640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc808162940 0x7fc808158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80db5d640 1 -- 172.21.15.53:0/1871397069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8080b91d0 con 0x7fc8080d3d80 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80db5d640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8080d3d80 0x7fc808159940 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8000f80 tx=0x7fc7f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc8067fc640 1 -- 172.21.15.53:0/1871397069 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8036050 con 0x7fc8080d3d80 2024-05-10T10:37:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc8067fc640 1 -- 172.21.15.53:0/1871397069 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8042070 con 0x7fc8080d3d80 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8080b94b0 con 0x7fc8080d3d80 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc8067fc640 1 -- 172.21.15.53:0/1871397069 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8009510 con 0x7fc8080d3d80 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.597+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8080b9a70 con 0x7fc8080d3d80 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.598+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80806a220 con 0x7fc8080d3d80 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.599+0000 7fc8067fc640 1 -- 172.21.15.53:0/1871397069 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc7f8007c70 con 0x7fc8080d3d80 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.599+0000 7fc8067fc640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7f40762d0 0x7fc7f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.599+0000 7fc8067fc640 1 -- 172.21.15.53:0/1871397069 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc7f80b8560 con 0x7fc8080d3d80 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.599+0000 7fc80d35c640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7f40762d0 0x7fc7f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.600+0000 7fc80d35c640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7f40762d0 0x7fc7f4078790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc80815e8f0 tx=0x7fc7fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.604+0000 7fc8067fc640 1 -- 172.21.15.53:0/1871397069 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc7f8081dc0 con 0x7fc8080d3d80 2024-05-10T10:37:28.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.834+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8080d2730 con 0x7fc8080d3d80 2024-05-10T10:37:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.834+0000 7fc8067fc640 1 -- 172.21.15.53:0/1871397069 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fc7f80816c0 con 0x7fc8080d3d80 2024-05-10T10:37:28.835 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7f40762d0 msgr2=0x7fc7f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7f40762d0 0x7fc7f4078790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc80815e8f0 tx=0x7fc7fc009040 comp rx=0 tx=0).stop 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8080d3d80 msgr2=0x7fc808159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8080d3d80 0x7fc808159940 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8000f80 tx=0x7fc7f8004a40 comp rx=0 tx=0).stop 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 shutdown_connections 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7f40762d0 0x7fc7f4078790 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc808162940 0x7fc808158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8080d59d0 0x7fc8080cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 --2- 172.21.15.53:0/1871397069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8080d3d80 0x7fc808159940 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 >> 172.21.15.53:0/1871397069 conn(0x7fc80809e380 msgr2=0x7fc8080b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 shutdown_connections 2024-05-10T10:37:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:28.836+0000 7fc80fdc1640 1 -- 172.21.15.53:0/1871397069 wait complete. 2024-05-10T10:37:28.847 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:37:28.848 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:37:28.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:28.848 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:37:28.865 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:37:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:37:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:37:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:37:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42520sec preferred_lft 42520sec 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:28.867 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:37:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:37:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:37:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:28.868 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:37:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:37:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:37:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:28.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:28.869 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:37:28.929 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:37:28.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:28.931 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:37:28.988 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:37:28.988 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:37:28.988 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:37:28.988 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:37:28.989 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:37:28.989 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:37:28.989 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:37:28.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:37:28.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:37:28.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:37:28.989 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:37:29.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:37:29.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:37:29.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:29.118 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:37:29.134 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:37:29.134 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:37:29.134 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:37:29.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:29.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:37:29.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:37:29.200 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:29.256 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:37:29.256 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:37:29.256 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:29.312 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:37:29.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:29.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:29.397 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:29.398 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:37:29.398 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:37:29.406 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:29.406 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:37:29.406 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:37:29.407 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:37:29.407 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:37:29.416 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:29.416 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:37:29.416 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:37:29.416 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:37:29.416 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:37:29.417 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:37:29.419 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:37:29.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:29.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:29.445 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:37:29.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:37:29.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:37:29.548 INFO:teuthology.orchestra.run.smithi053.stdout:client5890 2024-05-10T10:37:29.548 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:29.548 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:29.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:29.565 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-05-10T10:37:29.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:29.627 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-05-10T10:37:29.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:29.684 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-05-10T10:37:29.742 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:37:29.743 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:37:29.743 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:37:29.743 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:37:29.743 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-05-10T10:37:29.743 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:37:29.849 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2024-05-10T10:37:29.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:37:29.858 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:29.858 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:29.915 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:29.915 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:29.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:37:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.116+0000 7fc0855af640 1 Processor -- start 2024-05-10T10:37:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.116+0000 7fc0855af640 1 -- start start 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.117+0000 7fc0855af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc0801608b0 0x7fc0800d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.117+0000 7fc0855af640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc080160c90 con 0x7fc0800d03c0 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.117+0000 7fc0855af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0800ce5f0 con 0x7fc0800d1230 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.117+0000 7fc0855af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc0800ce8d0 con 0x7fc0801608b0 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.117+0000 7fc07f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc0800d03c0 0x7fc0800b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42132/0 (socket says 172.21.15.53:42132) 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.117+0000 7fc07f7fe640 1 -- 172.21.15.53:0/1940734236 learned_addr learned my addr 172.21.15.53:0/1940734236 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.117+0000 7fc07effd640 1 --2- 172.21.15.53:0/1940734236 >> v2:172.21.15.135:3300/0 conn(0x7fc0801608b0 0x7fc0800d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184968958 0 0) 0x7fc070001110 con 0x7fc0800d03c0 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc06c003520 con 0x7fc0800d03c0 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3741445131 0 0) 0x7fc06c003520 con 0x7fc0800d03c0 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 >> v2:172.21.15.135:3300/0 conn(0x7fc0801608b0 msgr2=0x7fc0800d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 --2- 172.21.15.53:0/1940734236 >> v2:172.21.15.135:3300/0 conn(0x7fc0801608b0 0x7fc0800d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 >> v1:172.21.15.135:6789/0 conn(0x7fc0800d1230 legacy=0x7fc0800b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0800cebb0 con 0x7fc0800d03c0 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fc070001840 con 0x7fc0800d03c0 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 >> v1:172.21.15.53:6789/0 conn(0x7fc0800d03c0 legacy=0x7fc0800b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0039b0 0x7fc06c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc06c0043b0 0x7fc06c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc06c005080 0x7fc06c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800ce8d0 con 0x7fc06c0043b0 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c00bfe0 con 0x7fc06c0039b0 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc06c00c1e0 con 0x7fc06c005080 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07e7fc640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc06c0043b0 0x7fc06c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07effd640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0039b0 0x7fc06c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07f7fe640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc06c005080 0x7fc06c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07e7fc640 1 -- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc06c005080 msgr2=0x7fc06c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07e7fc640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc06c005080 0x7fc06c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07e7fc640 1 -- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0039b0 msgr2=0x7fc06c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07e7fc640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0039b0 0x7fc06c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07e7fc640 1 -- 172.21.15.53:0/1940734236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c003520 con 0x7fc06c0043b0 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07f7fe640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc06c005080 0x7fc06c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.118+0000 7fc07f7fe640 1 -- 172.21.15.53:0/1940734236 reap_dead start 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.119+0000 7fc07e7fc640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc06c0043b0 0x7fc06c004880 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc068001860 tx=0x7fc06c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd1c53b36934599 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.119+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068037070 con 0x7fc06c0043b0 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.119+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06802abe0 con 0x7fc06c0043b0 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.119+0000 7fc07dffb640 1 -- 172.21.15.53:0/1940734236 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068001430 con 0x7fc06c0043b0 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.120+0000 7fc0855af640 1 -- 172.21.15.53:0/1940734236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc06c0043b0 msgr2=0x7fc06c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.120+0000 7fc0855af640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc06c0043b0 0x7fc06c004880 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc068001860 tx=0x7fc06c005830 comp rx=0 tx=0).stop 2024-05-10T10:37:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.120+0000 7fc0855af640 1 -- 172.21.15.53:0/1940734236 shutdown_connections 2024-05-10T10:37:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.120+0000 7fc0855af640 1 --2- 172.21.15.53:0/1940734236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc06c0043b0 0x7fc06c004880 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.120+0000 7fc0855af640 1 -- 172.21.15.53:0/1940734236 >> 172.21.15.53:0/1940734236 conn(0x7fc0800d2df0 msgr2=0x7fc0800d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.120+0000 7fc0855af640 1 -- 172.21.15.53:0/1940734236 shutdown_connections 2024-05-10T10:37:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.120+0000 7fc0855af640 1 -- 172.21.15.53:0/1940734236 wait complete. 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.121+0000 7fc0855af640 1 Processor -- start 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.121+0000 7fc0855af640 1 -- start start 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc0855af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0043b0 0x7fc08014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc0855af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800c9010 0x7fc08014ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc0855af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0800c9490 0x7fc08014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc0855af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800d1f10 con 0x7fc0800c9490 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc0855af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800d1d90 con 0x7fc06c0043b0 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc0855af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc0800d0cf0 con 0x7fc0800c9010 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0800c9490 0x7fc08014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0800c9490 0x7fc08014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54286/0 (socket says 172.21.15.53:54286) 2024-05-10T10:37:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 -- 172.21.15.53:0/3003200914 learned_addr learned my addr 172.21.15.53:0/3003200914 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07effd640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0043b0 0x7fc08014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07e7fc640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800c9010 0x7fc08014ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 -- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800c9010 msgr2=0x7fc08014ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800c9010 0x7fc08014ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 -- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0043b0 msgr2=0x7fc08014fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0043b0 0x7fc08014fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.122+0000 7fc07f7fe640 1 -- 172.21.15.53:0/3003200914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0800c69c0 con 0x7fc0800c9490 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.123+0000 7fc07f7fe640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0800c9490 0x7fc08014f360 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc070001eb0 tx=0x7fc070007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.123+0000 7fc0777fe640 1 -- 172.21.15.53:0/3003200914 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07000e610 con 0x7fc0800c9490 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.123+0000 7fc0777fe640 1 -- 172.21.15.53:0/3003200914 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc070018070 con 0x7fc0800c9490 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.123+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0800c6ca0 con 0x7fc0800c9490 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.123+0000 7fc0777fe640 1 -- 172.21.15.53:0/3003200914 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07000e9f0 con 0x7fc0800c9490 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.123+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0800c7260 con 0x7fc0800c9490 2024-05-10T10:37:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.124+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08006a220 con 0x7fc0800c9490 2024-05-10T10:37:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.124+0000 7fc0777fe640 1 -- 172.21.15.53:0/3003200914 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc07000eb90 con 0x7fc0800c9490 2024-05-10T10:37:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.125+0000 7fc0777fe640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0600762d0 0x7fc060078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.125+0000 7fc0777fe640 1 -- 172.21.15.53:0/3003200914 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc070097250 con 0x7fc0800c9490 2024-05-10T10:37:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.125+0000 7fc07effd640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0600762d0 0x7fc060078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.126+0000 7fc07effd640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0600762d0 0x7fc060078790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc078014510 tx=0x7fc07800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.130+0000 7fc0777fe640 1 -- 172.21.15.53:0/3003200914 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc080160c90 con 0x7fc0800c9490 2024-05-10T10:37:30.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.338+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc0800c6290 con 0x7fc0800c9490 2024-05-10T10:37:30.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.338+0000 7fc0777fe640 1 -- 172.21.15.53:0/3003200914 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) ==== 80+0+0 (secure 0 0 0) 0x7fc0700603d0 con 0x7fc0800c9490 2024-05-10T10:37:30.339 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0600762d0 msgr2=0x7fc060078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0600762d0 0x7fc060078790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc078014510 tx=0x7fc07800b040 comp rx=0 tx=0).stop 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0800c9490 msgr2=0x7fc08014f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0800c9490 0x7fc08014f360 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc070001eb0 tx=0x7fc070007420 comp rx=0 tx=0).stop 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 shutdown_connections 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0600762d0 0x7fc060078790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0800c9490 0x7fc08014f360 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800c9010 0x7fc08014ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 --2- 172.21.15.53:0/3003200914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc06c0043b0 0x7fc08014fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 >> 172.21.15.53:0/3003200914 conn(0x7fc0800d2df0 msgr2=0x7fc0800cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:30.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 shutdown_connections 2024-05-10T10:37:30.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.340+0000 7fc0855af640 1 -- 172.21.15.53:0/3003200914 wait complete. 2024-05-10T10:37:30.351 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:37:30.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:30.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:30.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:37:30.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:30.405 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:37:30.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.579+0000 7ff0d18a7640 1 Processor -- start 2024-05-10T10:37:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.579+0000 7ff0d18a7640 1 -- start start 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0d18a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff0cc15f4e0 0x7ff0cc06d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0d18a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0d18a7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff0cc06e450 0x7ff0cc0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0d18a7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc0b23f0 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0d18a7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc0b4150 con 0x7ff0cc06e450 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0d18a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc06e8c0 con 0x7ff0cc15f8c0 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff0cc15f4e0 0x7ff0cc06d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff0cc15f4e0 0x7ff0cc06d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54302/0 (socket says 172.21.15.53:54302) 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 -- 172.21.15.53:0/131658581 learned_addr learned my addr 172.21.15.53:0/131658581 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0cb7fe640 1 --2- 172.21.15.53:0/131658581 >> v2:172.21.15.135:3300/0 conn(0x7ff0cc06e450 0x7ff0cc0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0ca7fc640 1 --2- 172.21.15.53:0/131658581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 -- 172.21.15.53:0/131658581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 msgr2=0x7ff0cc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 --2- 172.21.15.53:0/131658581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 -- 172.21.15.53:0/131658581 >> v2:172.21.15.135:3300/0 conn(0x7ff0cc06e450 msgr2=0x7ff0cc0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 --2- 172.21.15.53:0/131658581 >> v2:172.21.15.135:3300/0 conn(0x7ff0cc06e450 0x7ff0cc0afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0caffd640 1 -- 172.21.15.53:0/131658581 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc0b3540 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.580+0000 7ff0cb7fe640 1 --2- 172.21.15.53:0/131658581 >> v2:172.21.15.135:3300/0 conn(0x7ff0cc06e450 0x7ff0cc0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.581+0000 7ff0caffd640 1 --2- 172.21.15.53:0/131658581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc06d9d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc073990 tx=0x7ff0bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1a908455a35e6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.581+0000 7ff0c9ffb640 1 -- 172.21.15.53:0/131658581 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc037070 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.581+0000 7ff0c9ffb640 1 -- 172.21.15.53:0/131658581 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0bc032440 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.581+0000 7ff0c9ffb640 1 -- 172.21.15.53:0/131658581 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc032810 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 -- 172.21.15.53:0/131658581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 msgr2=0x7ff0cc06d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/131658581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc06d9d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc073990 tx=0x7ff0bc02a140 comp rx=0 tx=0).stop 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 -- 172.21.15.53:0/131658581 shutdown_connections 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/131658581 >> v2:172.21.15.135:3300/0 conn(0x7ff0cc06e450 0x7ff0cc0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/131658581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/131658581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc06d9d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 -- 172.21.15.53:0/131658581 >> 172.21.15.53:0/131658581 conn(0x7ff0cc0d2df0 msgr2=0x7ff0cc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 -- 172.21.15.53:0/131658581 shutdown_connections 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.582+0000 7ff0d18a7640 1 -- 172.21.15.53:0/131658581 wait complete. 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.583+0000 7ff0d18a7640 1 Processor -- start 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.583+0000 7ff0d18a7640 1 -- start start 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0d18a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0cc06e450 0x7ff0cc154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0d18a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0d18a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0d18a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0b3de0 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0d18a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0b3b00 con 0x7ff0cc06e450 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0d18a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc0b45e0 con 0x7ff0cc15f8c0 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0ca7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0ca7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54306/0 (socket says 172.21.15.53:54306) 2024-05-10T10:37:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0ca7fc640 1 -- 172.21.15.53:0/1537164471 learned_addr learned my addr 172.21.15.53:0/1537164471 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.584+0000 7ff0caffd640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0cc06e450 0x7ff0cc154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0cb7fe640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0ca7fc640 1 -- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 msgr2=0x7ff0cc0c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0ca7fc640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc0c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0ca7fc640 1 -- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0cc06e450 msgr2=0x7ff0cc154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0ca7fc640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0cc06e450 0x7ff0cc154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0ca7fc640 1 -- 172.21.15.53:0/1537164471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc154b70 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0ca7fc640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc156440 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000d800 tx=0x7ff0c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0b3fff640 1 -- 172.21.15.53:0/1537164471 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000b6c0 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0b3fff640 1 -- 172.21.15.53:0/1537164471 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c0019070 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc155120 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0b3fff640 1 -- 172.21.15.53:0/1537164471 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000ba80 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.585+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc155960 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.586+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc0d23d0 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.587+0000 7ff0b3fff640 1 -- 172.21.15.53:0/1537164471 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff0c000bc20 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.587+0000 7ff0b3fff640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0b8076300 0x7ff0b80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.587+0000 7ff0b3fff640 1 -- 172.21.15.53:0/1537164471 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7ff0c0097ba0 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.587+0000 7ff0caffd640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0b8076300 0x7ff0b80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.588+0000 7ff0caffd640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0b8076300 0x7ff0b80787c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc034000 tx=0x7ff0bc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.591+0000 7ff0b3fff640 1 -- 172.21.15.53:0/1537164471 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff0cc0d23d0 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.820+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0cc06a220 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.821+0000 7ff0b3fff640 1 -- 172.21.15.53:0/1537164471 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5186 (secure 0 0 0) 0x7ff0c0067070 con 0x7ff0cc15f4e0 2024-05-10T10:37:30.822 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:30.822 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:37:20.609173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6834/1984524370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1984524370},{"type":"v1","addr":"172.21.15.53:6834","nonce":1984524370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6102,"qdb_cluster":[6102]},"id":5}]} 2024-05-10T10:37:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 45 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0b8076300 msgr2=0x7ff0b80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0b8076300 0x7ff0b80787c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc034000 tx=0x7ff0bc02e000 comp rx=0 tx=0).stop 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 msgr2=0x7ff0cc156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc156440 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000d800 tx=0x7ff0c000dcc0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 shutdown_connections 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff0b8076300 0x7ff0b80787c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0cc15f8c0 0x7ff0cc0c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0cc15f4e0 0x7ff0cc156440 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 --2- 172.21.15.53:0/1537164471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0cc06e450 0x7ff0cc154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 >> 172.21.15.53:0/1537164471 conn(0x7ff0cc0d2df0 msgr2=0x7ff0cc071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 shutdown_connections 2024-05-10T10:37:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:30.823+0000 7ff0d18a7640 1 -- 172.21.15.53:0/1537164471 wait complete. 2024-05-10T10:37:30.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-10T10:37:30.965 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:37:30.965 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:37:30.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:37:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.069+0000 7f1510e3f640 1 Processor -- start 2024-05-10T10:37:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.070+0000 7f1510e3f640 1 -- start start 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.070+0000 7f1510e3f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f150c15f3b0 0x7f150c0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f1510e3f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150c0d30b0 con 0x7f150c15f790 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f1510e3f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150c0d3390 con 0x7f150c0d5d30 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f1510e3f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f150c0c8c80 con 0x7f150c15f3b0 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f1509d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f150c15f790 0x7f150c0c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42140/0 (socket says 172.21.15.53:42140) 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f1509d9b640 1 -- 172.21.15.53:0/1458461287 learned_addr learned my addr 172.21.15.53:0/1458461287 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984952431 0 0) 0x7f150c0d30b0 con 0x7f150c15f790 2024-05-10T10:37:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14f4003520 con 0x7f150c15f790 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150a59c640 1 --2- 172.21.15.53:0/1458461287 >> v2:172.21.15.135:3300/0 conn(0x7f150c15f3b0 0x7f150c0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4250973420 0 0) 0x7f14f4003520 con 0x7f150c15f790 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 >> v2:172.21.15.135:3300/0 conn(0x7f150c15f3b0 msgr2=0x7f150c0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 --2- 172.21.15.53:0/1458461287 >> v2:172.21.15.135:3300/0 conn(0x7f150c15f3b0 0x7f150c0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 >> v1:172.21.15.135:6789/0 conn(0x7f150c0d5d30 legacy=0x7f150c0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c0c8f60 con 0x7f150c15f790 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f1500001670 con 0x7f150c15f790 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 >> v1:172.21.15.53:6789/0 conn(0x7f150c15f790 legacy=0x7f150c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14f40039b0 0x7f14f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f4004800 0x7f14f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f40051c0 0x7f14f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0c8c80 con 0x7f14f4004800 2024-05-10T10:37:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14f400c060 con 0x7f14f40039b0 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f14f400c260 con 0x7f14f40051c0 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.071+0000 7f1509d9b640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f4004800 0x7f14f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f150a59c640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14f40039b0 0x7f14f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f150ad9d640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f40051c0 0x7f14f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f1509d9b640 1 -- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f40051c0 msgr2=0x7f14f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f1509d9b640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f40051c0 0x7f14f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f1509d9b640 1 -- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14f40039b0 msgr2=0x7f14f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f1509d9b640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14f40039b0 0x7f14f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f1509d9b640 1 -- 172.21.15.53:0/1458461287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f4003520 con 0x7f14f4004800 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f150ad9d640 1 -- 172.21.15.53:0/1458461287 reap_dead start 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.072+0000 7f1509d9b640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f4004800 0x7f14f4004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f150000c210 tx=0x7f14f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d487e166f7a4548 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500007cf0 con 0x7f14f4004800 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500005af0 con 0x7f14f4004800 2024-05-10T10:37:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f150959a640 1 -- 172.21.15.53:0/1458461287 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500007070 con 0x7f14f4004800 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f1510e3f640 1 -- 172.21.15.53:0/1458461287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f4004800 msgr2=0x7f14f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f4004800 0x7f14f4004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f150000c210 tx=0x7f14f4005830 comp rx=0 tx=0).stop 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f1510e3f640 1 -- 172.21.15.53:0/1458461287 shutdown_connections 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1458461287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f4004800 0x7f14f4004c80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f1510e3f640 1 -- 172.21.15.53:0/1458461287 >> 172.21.15.53:0/1458461287 conn(0x7f150c0d5130 msgr2=0x7f150c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f1510e3f640 1 -- 172.21.15.53:0/1458461287 shutdown_connections 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.073+0000 7f1510e3f640 1 -- 172.21.15.53:0/1458461287 wait complete. 2024-05-10T10:37:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 Processor -- start 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 -- start start 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f4004800 0x7f150c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f150c0b5510 0x7f150c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0b7d20 0x7f150c0b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0d3870 con 0x7f150c0b7d20 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0d36f0 con 0x7f150c0b5510 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f1510e3f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f150c0d39f0 con 0x7f14f4004800 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f150ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0b7d20 0x7f150c0b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f150ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0b7d20 0x7f150c0b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54324/0 (socket says 172.21.15.53:54324) 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.075+0000 7f150ad9d640 1 -- 172.21.15.53:0/1969269361 learned_addr learned my addr 172.21.15.53:0/1969269361 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f150a59c640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f4004800 0x7f150c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f1509d9b640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f150c0b5510 0x7f150c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f150ad9d640 1 -- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f4004800 msgr2=0x7f150c0b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f150ad9d640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f4004800 0x7f150c0b89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f150ad9d640 1 -- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f150c0b5510 msgr2=0x7f150c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f150ad9d640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f150c0b5510 0x7f150c15b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f150ad9d640 1 -- 172.21.15.53:0/1969269361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c0b8f20 con 0x7f150c0b7d20 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f150ad9d640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0b7d20 0x7f150c0b81a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f150c06cca0 tx=0x7f150400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f14faffd640 1 -- 172.21.15.53:0/1969269361 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504003970 con 0x7f150c0b7d20 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f14faffd640 1 -- 172.21.15.53:0/1969269361 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504018070 con 0x7f150c0b7d20 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c0b91a0 con 0x7f150c0b7d20 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f14faffd640 1 -- 172.21.15.53:0/1969269361 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504003d70 con 0x7f150c0b7d20 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.076+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c0cda30 con 0x7f150c0b7d20 2024-05-10T10:37:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.077+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c06a220 con 0x7f150c0b7d20 2024-05-10T10:37:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.078+0000 7f14faffd640 1 -- 172.21.15.53:0/1969269361 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f15040050d0 con 0x7f150c0b7d20 2024-05-10T10:37:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.079+0000 7f14faffd640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f14ec076300 0x7f14ec0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.079+0000 7f14faffd640 1 -- 172.21.15.53:0/1969269361 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f1504097420 con 0x7f150c0b7d20 2024-05-10T10:37:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.079+0000 7f150a59c640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f14ec076300 0x7f14ec0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.079+0000 7f150a59c640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f14ec076300 0x7f14ec0787c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009b80 tx=0x7f14fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.083+0000 7f14faffd640 1 -- 172.21.15.53:0/1969269361 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f150c0b8f20 con 0x7f150c0b7d20 2024-05-10T10:37:31.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.311+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f150c06e470 con 0x7f150c0b7d20 2024-05-10T10:37:31.312 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:37:31.312+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:37:31.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.314+0000 7f14faffd640 1 -- 172.21.15.53:0/1969269361 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7f1504060ac0 con 0x7f150c0b7d20 2024-05-10T10:37:31.314 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f14ec076300 msgr2=0x7f14ec0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f14ec076300 0x7f14ec0787c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009b80 tx=0x7f14fc008040 comp rx=0 tx=0).stop 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0b7d20 msgr2=0x7f150c0b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0b7d20 0x7f150c0b81a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f150c06cca0 tx=0x7f150400eb90 comp rx=0 tx=0).stop 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 shutdown_connections 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f14ec076300 0x7f14ec0787c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0b7d20 0x7f150c0b81a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f150c0b5510 0x7f150c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 --2- 172.21.15.53:0/1969269361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14f4004800 0x7f150c0b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 >> 172.21.15.53:0/1969269361 conn(0x7f150c0d5130 msgr2=0x7f150c0d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 shutdown_connections 2024-05-10T10:37:31.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.317+0000 7f1510e3f640 1 -- 172.21.15.53:0/1969269361 wait complete. 2024-05-10T10:37:31.319 INFO:tasks.ceph.mds.a.smithi053.stderr: -4596> 2024-05-10T10:37:24.195+0000 7f5d8fdfe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:37:31.329 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-10T10:37:31.350 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:37:31.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.374+0000 7ff45c55bf40 1 Processor -- start 2024-05-10T10:37:31.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.374+0000 7ff45c55bf40 1 -- start start 2024-05-10T10:37:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45c55bf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884d4df00 0x55e884e0f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45c55bf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d99450 0x55e884d7ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45c55bf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884e10450 0x55e884e0a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45c55bf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e884d7d020 con 0x55e884e10450 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45c55bf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e884cf6f50 con 0x55e884d4df00 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45c55bf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e884d7d300 con 0x55e884d99450 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884e10450 0x55e884e0a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884e10450 0x55e884e0a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54352/0 (socket says 172.21.15.53:54352) 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 -- 172.21.15.53:0/374030881 learned_addr learned my addr 172.21.15.53:0/374030881 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff459854640 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d99450 0x55e884d7ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a055640 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884d4df00 0x55e884e0f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 -- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d99450 msgr2=0x55e884d7ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d99450 0x55e884d7ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 -- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884d4df00 msgr2=0x55e884e0f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884d4df00 0x55e884e0f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.375+0000 7ff45a856640 1 -- 172.21.15.53:0/374030881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e884e0ebc0 con 0x55e884e10450 2024-05-10T10:37:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45a856640 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884e10450 0x55e884e0a490 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff454025b10 tx=0x7ff454050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8645d49d0bfe21a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff458852640 1 -- 172.21.15.53:0/374030881 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454054040 con 0x55e884e10450 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff458852640 1 -- 172.21.15.53:0/374030881 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4540509a0 con 0x55e884e10450 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff458852640 1 -- 172.21.15.53:0/374030881 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454050d50 con 0x55e884e10450 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 -- 172.21.15.53:0/374030881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884e10450 msgr2=0x55e884e0a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884e10450 0x55e884e0a490 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff454025b10 tx=0x7ff454050000 comp rx=0 tx=0).stop 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 -- 172.21.15.53:0/374030881 shutdown_connections 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884e10450 0x55e884e0a490 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d99450 0x55e884d7ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/374030881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884d4df00 0x55e884e0f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 -- 172.21.15.53:0/374030881 >> 172.21.15.53:0/374030881 conn(0x55e884d79810 msgr2=0x55e884d7a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 -- 172.21.15.53:0/374030881 shutdown_connections 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.376+0000 7ff45c55bf40 1 -- 172.21.15.53:0/374030881 wait complete. 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.377+0000 7ff45c55bf40 1 Processor -- start 2024-05-10T10:37:31.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.377+0000 7ff45c55bf40 1 -- start start 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d4df00 0x55e884e0f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884d99450 0x55e884d7e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884e10450 0x55e884ec07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e884d95ef0 con 0x55e884d99450 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e884e0fc90 con 0x55e884e10450 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e884d95c50 con 0x55e884d4df00 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884d99450 0x55e884d7e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884d99450 0x55e884d7e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54354/0 (socket says 172.21.15.53:54354) 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 -- 172.21.15.53:0/3277120060 learned_addr learned my addr 172.21.15.53:0/3277120060 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45a055640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d4df00 0x55e884e0f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45a856640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884e10450 0x55e884ec07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 -- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d4df00 msgr2=0x55e884e0f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e884d4df00 0x55e884e0f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 -- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884e10450 msgr2=0x55e884ec07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884e10450 0x55e884ec07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 -- 172.21.15.53:0/3277120060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e884d7b0e0 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45a856640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e884e10450 0x55e884ec07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff459854640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884d99450 0x55e884d7e7b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff45000dfa0 tx=0x7ff450003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff44b7fe640 1 -- 172.21.15.53:0/3277120060 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450014030 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff44b7fe640 1 -- 172.21.15.53:0/3277120060 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff45000be50 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e884eb7e30 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff44b7fe640 1 -- 172.21.15.53:0/3277120060 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450010860 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.378+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e884e262c0 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.380+0000 7ff44b7fe640 1 -- 172.21.15.53:0/3277120060 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff450017030 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.380+0000 7ff44b7fe640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff43c074cb0 0x7ff43c077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.381+0000 7ff44b7fe640 1 -- 172.21.15.53:0/3277120060 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff450096700 con 0x55e884d99450 2024-05-10T10:37:31.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.381+0000 7ff45a055640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff43c074cb0 0x7ff43c077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.381+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e884d7ecf0 0x55e884ec1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.381+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75) -- 0x55e884e0a260 con 0x55e884d7ecf0 2024-05-10T10:37:31.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.381+0000 7ff45a856640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e884d7ecf0 0x55e884ec1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.381+0000 7ff45a856640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e884d7ecf0 0x55e884ec1220 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.381+0000 7ff45a055640 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff43c074cb0 0x7ff43c077170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55e884e28330 tx=0x7ff44c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45a856640 1 -- 172.21.15.53:0/3277120060 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v75'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff454056030 con 0x55e884d7ecf0 2024-05-10T10:37:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e884d7ecf0 msgr2=0x55e884ec1220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e884d7ecf0 0x55e884ec1220 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff43c074cb0 msgr2=0x7ff43c077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff43c074cb0 0x7ff43c077170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55e884e28330 tx=0x7ff44c00b040 comp rx=0 tx=0).stop 2024-05-10T10:37:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884d99450 msgr2=0x55e884d7e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 --2- 172.21.15.53:0/3277120060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e884d99450 0x55e884d7e7b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff45000dfa0 tx=0x7ff450003f20 comp rx=0 tx=0).stop 2024-05-10T10:37:31.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45a856640 1 -- 172.21.15.53:0/3277120060 reap_dead start 2024-05-10T10:37:31.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 shutdown_connections 2024-05-10T10:37:31.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 >> 172.21.15.53:0/3277120060 conn(0x55e884d79810 msgr2=0x55e884d94f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:31.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 shutdown_connections 2024-05-10T10:37:31.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.384+0000 7ff45c55bf40 1 -- 172.21.15.53:0/3277120060 wait complete. 2024-05-10T10:37:31.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-10T10:37:31.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.535+0000 7faaabd78640 1 Processor -- start 2024-05-10T10:37:31.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.535+0000 7faaabd78640 1 -- start start 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.536+0000 7faaabd78640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faaa40d3e80 0x7faaa40b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.536+0000 7faaabd78640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaa4162fa0 con 0x7faaa40d2230 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.536+0000 7faaabd78640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaa40d2f70 con 0x7faaa4162bc0 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.536+0000 7faaabd78640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7faaa40d1170 con 0x7faaa40d3e80 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.536+0000 7faaa9313640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faaa40d3e80 0x7faaa40b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.536+0000 7faaaa315640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faaa40d2230 0x7faaa40b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42162/0 (socket says 172.21.15.53:42162) 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.536+0000 7faaaa315640 1 -- 172.21.15.53:0/1376622560 learned_addr learned my addr 172.21.15.53:0/1376622560 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.537+0000 7faaa9313640 1 -- 172.21.15.53:0/1376622560 >> v1:172.21.15.135:6789/0 conn(0x7faaa4162bc0 legacy=0x7faaa40d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.537+0000 7faaa9313640 1 -- 172.21.15.53:0/1376622560 >> v1:172.21.15.53:6789/0 conn(0x7faaa40d2230 legacy=0x7faaa40b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.537+0000 7faaa9313640 1 -- 172.21.15.53:0/1376622560 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa40d3250 con 0x7faaa40d3e80 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.537+0000 7faaa9313640 1 --2- 172.21.15.53:0/1376622560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d3e80 0x7faaa40b24f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faa980097a0 tx=0x7faa9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de797ac878f6d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.537+0000 7faaa8b12640 1 -- 172.21.15.53:0/1376622560 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98037070 con 0x7faaa40d3e80 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.537+0000 7faaa8b12640 1 -- 172.21.15.53:0/1376622560 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa98032440 con 0x7faaa40d3e80 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaa8b12640 1 -- 172.21.15.53:0/1376622560 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98032810 con 0x7faaa40d3e80 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaabd78640 1 -- 172.21.15.53:0/1376622560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d3e80 msgr2=0x7faaa40b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaabd78640 1 --2- 172.21.15.53:0/1376622560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d3e80 0x7faaa40b24f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faa980097a0 tx=0x7faa9802a1a0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaabd78640 1 -- 172.21.15.53:0/1376622560 shutdown_connections 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaabd78640 1 --2- 172.21.15.53:0/1376622560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d3e80 0x7faaa40b24f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaabd78640 1 -- 172.21.15.53:0/1376622560 >> 172.21.15.53:0/1376622560 conn(0x7faaa40d5190 msgr2=0x7faaa40d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:31.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaabd78640 1 -- 172.21.15.53:0/1376622560 shutdown_connections 2024-05-10T10:37:31.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.538+0000 7faaabd78640 1 -- 172.21.15.53:0/1376622560 wait complete. 2024-05-10T10:37:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.539+0000 7faaabd78640 1 Processor -- start 2024-05-10T10:37:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.539+0000 7faaabd78640 1 -- start start 2024-05-10T10:37:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaabd78640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d2230 0x7faaa415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaabd78640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faaa40d3e80 0x7faaa41613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaabd78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaa4162bc0 0x7faaa415d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaabd78640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faaa40b1cf0 con 0x7faaa40d2230 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaabd78640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faaa40d5f90 con 0x7faaa4162bc0 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaabd78640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7faaa40b4030 con 0x7faaa40d3e80 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaa9b14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d2230 0x7faaa415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaa9b14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d2230 0x7faaa415ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54374/0 (socket says 172.21.15.53:54374) 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaa9b14640 1 -- 172.21.15.53:0/1408044101 learned_addr learned my addr 172.21.15.53:0/1408044101 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaaa315640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaa4162bc0 0x7faaa415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.540+0000 7faaa9313640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faaa40d3e80 0x7faaa41613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa9b14640 1 -- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faaa40d3e80 msgr2=0x7faaa41613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa9b14640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faaa40d3e80 0x7faaa41613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa9b14640 1 -- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaa4162bc0 msgr2=0x7faaa415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa9b14640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaa4162bc0 0x7faaa415d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa9b14640 1 -- 172.21.15.53:0/1408044101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa4161a50 con 0x7faaa40d2230 2024-05-10T10:37:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa9b14640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d2230 0x7faaa415ea40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7faa9400c730 tx=0x7faa9400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa27fc640 1 -- 172.21.15.53:0/1408044101 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94016020 con 0x7faaa40d2230 2024-05-10T10:37:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa27fc640 1 -- 172.21.15.53:0/1408044101 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa94006d50 con 0x7faaa40d2230 2024-05-10T10:37:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa415c2c0 con 0x7faaa40d2230 2024-05-10T10:37:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaa27fc640 1 -- 172.21.15.53:0/1408044101 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94003bb0 con 0x7faaa40d2230 2024-05-10T10:37:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.541+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa415c880 con 0x7faaa40d2230 2024-05-10T10:37:31.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.542+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa406a220 con 0x7faaa40d2230 2024-05-10T10:37:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.543+0000 7faaa27fc640 1 -- 172.21.15.53:0/1408044101 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faa9400a850 con 0x7faaa40d2230 2024-05-10T10:37:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.543+0000 7faaa27fc640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa900762d0 0x7faa90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.543+0000 7faaa27fc640 1 -- 172.21.15.53:0/1408044101 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7faa940972c0 con 0x7faaa40d2230 2024-05-10T10:37:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.543+0000 7faaa9313640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa900762d0 0x7faa90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.544+0000 7faaa9313640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa900762d0 0x7faa90078790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faa98034000 tx=0x7faa9802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.547+0000 7faaa27fc640 1 -- 172.21.15.53:0/1408044101 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7faa94020020 con 0x7faaa40d2230 2024-05-10T10:37:31.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:31.777+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7faaa40b3df0 con 0x7faaa40d2230 2024-05-10T10:37:32.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.315+0000 7faaa27fc640 1 -- 172.21.15.53:0/1408044101 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) ==== 162+0+0 (secure 0 0 0) 0x7faa94060480 con 0x7faaa40d2230 2024-05-10T10:37:32.316 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa900762d0 msgr2=0x7faa90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa900762d0 0x7faa90078790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faa98034000 tx=0x7faa9802e000 comp rx=0 tx=0).stop 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d2230 msgr2=0x7faaa415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d2230 0x7faaa415ea40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7faa9400c730 tx=0x7faa9400cbf0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 shutdown_connections 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7faa900762d0 0x7faa90078790 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaa4162bc0 0x7faaa415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faaa40d3e80 0x7faaa41613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 --2- 172.21.15.53:0/1408044101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa40d2230 0x7faaa415ea40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 >> 172.21.15.53:0/1408044101 conn(0x7faaa40d5190 msgr2=0x7faaa40c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 shutdown_connections 2024-05-10T10:37:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.318+0000 7faaabd78640 1 -- 172.21.15.53:0/1408044101 wait complete. 2024-05-10T10:37:32.334 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:32.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.443+0000 7f9ba9b0a640 1 Processor -- start 2024-05-10T10:37:32.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.443+0000 7f9ba9b0a640 1 -- start start 2024-05-10T10:37:32.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.444+0000 7f9ba9b0a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ba40d5cb0 0x7f9ba40d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:32.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.444+0000 7f9ba9b0a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba40d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.444+0000 7f9ba9b0a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba40b45f0 con 0x7f9ba4162750 2024-05-10T10:37:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.444+0000 7f9ba9b0a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba40b4f20 con 0x7f9ba40d5cb0 2024-05-10T10:37:32.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.444+0000 7f9ba9b0a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba40d0fe0 con 0x7f9ba4162b30 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.444+0000 7f9ba3fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba40d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.444+0000 7f9ba2ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ba40d5cb0 0x7f9ba40d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba37fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9ba4162750 0x7f9ba40d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:39170/0 (socket says 172.21.15.53:39170) 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba37fe640 1 -- 172.21.15.53:0/2045160959 learned_addr learned my addr 172.21.15.53:0/2045160959 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba3fff640 1 -- 172.21.15.53:0/2045160959 >> v2:172.21.15.135:3300/0 conn(0x7f9ba40d5cb0 msgr2=0x7f9ba40d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba3fff640 1 --2- 172.21.15.53:0/2045160959 >> v2:172.21.15.135:3300/0 conn(0x7f9ba40d5cb0 0x7f9ba40d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba3fff640 1 -- 172.21.15.53:0/2045160959 >> v1:172.21.15.135:6789/0 conn(0x7f9ba4162750 legacy=0x7f9ba40d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba27fc640 1 -- 172.21.15.53:0/2045160959 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756241433 0 0) 0x7f9ba40b45f0 con 0x7f9ba4162750 2024-05-10T10:37:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba3fff640 1 -- 172.21.15.53:0/2045160959 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba40b48d0 con 0x7f9ba4162b30 2024-05-10T10:37:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.445+0000 7f9ba2ffd640 1 --2- 172.21.15.53:0/2045160959 >> v2:172.21.15.135:3300/0 conn(0x7f9ba40d5cb0 0x7f9ba40d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.446+0000 7f9ba3fff640 1 --2- 172.21.15.53:0/2045160959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba40d3b70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009d90 tx=0x7f9b9c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67edf0c2ccfcd292 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.446+0000 7f9ba27fc640 1 -- 172.21.15.53:0/2045160959 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c031600 con 0x7f9ba4162b30 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.446+0000 7f9ba27fc640 1 -- 172.21.15.53:0/2045160959 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0317a0 con 0x7f9ba4162b30 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.446+0000 7f9ba27fc640 1 -- 172.21.15.53:0/2045160959 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c031b70 con 0x7f9ba4162b30 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.446+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/2045160959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 msgr2=0x7f9ba40d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.446+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/2045160959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba40d3b70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009d90 tx=0x7f9b9c02a5d0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.447+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/2045160959 shutdown_connections 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.447+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/2045160959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba40d3b70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.447+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/2045160959 >> v2:172.21.15.135:3300/0 conn(0x7f9ba40d5cb0 0x7f9ba40d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.447+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/2045160959 >> 172.21.15.53:0/2045160959 conn(0x7f9ba409e3b0 msgr2=0x7f9ba40d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.447+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/2045160959 shutdown_connections 2024-05-10T10:37:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.447+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/2045160959 wait complete. 2024-05-10T10:37:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.448+0000 7f9ba9b0a640 1 Processor -- start 2024-05-10T10:37:32.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.448+0000 7f9ba9b0a640 1 -- start start 2024-05-10T10:37:32.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba9b0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba4162750 0x7f9ba41514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:32.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba9b0a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba415fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:32.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba9b0a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ba4160340 0x7f9ba40ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba9b0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba40b53b0 con 0x7f9ba4162750 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba9b0a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba40d0d80 con 0x7f9ba4160340 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba9b0a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba40b5530 con 0x7f9ba4162b30 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba4162750 0x7f9ba41514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba4162750 0x7f9ba41514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54380/0 (socket says 172.21.15.53:54380) 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 -- 172.21.15.53:0/164628006 learned_addr learned my addr 172.21.15.53:0/164628006 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba2ffd640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba415fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba3fff640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ba4160340 0x7f9ba40ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 -- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 msgr2=0x7f9ba415fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba415fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 -- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ba4160340 msgr2=0x7f9ba40ba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ba4160340 0x7f9ba40ba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 -- 172.21.15.53:0/164628006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba41f9060 con 0x7f9ba4162750 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.449+0000 7f9ba37fe640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba4162750 0x7f9ba41514b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c4d0 tx=0x7f9b9400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.450+0000 7f9b8bfff640 1 -- 172.21.15.53:0/164628006 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94018070 con 0x7f9ba4162750 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.450+0000 7f9b8bfff640 1 -- 172.21.15.53:0/164628006 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b94007500 con 0x7f9ba4162750 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.450+0000 7f9b8bfff640 1 -- 172.21.15.53:0/164628006 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94004a20 con 0x7f9ba4162750 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.450+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba41f82a0 con 0x7f9ba4162750 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.450+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba41f8860 con 0x7f9ba4162750 2024-05-10T10:37:32.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.451+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba406a220 con 0x7f9ba4162750 2024-05-10T10:37:32.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.451+0000 7f9b8bfff640 1 -- 172.21.15.53:0/164628006 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9b9401b070 con 0x7f9ba4162750 2024-05-10T10:37:32.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.452+0000 7f9b8bfff640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b90076300 0x7f9b900787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:32.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.452+0000 7f9b8bfff640 1 -- 172.21.15.53:0/164628006 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f9b94096a70 con 0x7f9ba4162750 2024-05-10T10:37:32.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.452+0000 7f9ba2ffd640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b90076300 0x7f9b900787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:32.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.453+0000 7f9ba2ffd640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b90076300 0x7f9b900787c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40d1440 tx=0x7f9b98009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:32.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.456+0000 7f9b8bfff640 1 -- 172.21.15.53:0/164628006 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9b9405fba0 con 0x7f9ba4162750 2024-05-10T10:37:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.685+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ba4160750 con 0x7f9ba4162750 2024-05-10T10:37:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.685+0000 7f9b8bfff640 1 -- 172.21.15.53:0/164628006 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5180 (secure 0 0 0) 0x7f9b94060e20 con 0x7f9ba4162750 2024-05-10T10:37:32.686 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:32.687 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:37:32.318466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-10T10:37:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 48 2024-05-10T10:37:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b90076300 msgr2=0x7f9b900787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b90076300 0x7f9b900787c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40d1440 tx=0x7f9b98009190 comp rx=0 tx=0).stop 2024-05-10T10:37:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba4162750 msgr2=0x7f9ba41514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba4162750 0x7f9ba41514b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c4d0 tx=0x7f9b9400c990 comp rx=0 tx=0).stop 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 shutdown_connections 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9b90076300 0x7f9b900787c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ba4160340 0x7f9ba40ba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ba4162b30 0x7f9ba415fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 --2- 172.21.15.53:0/164628006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba4162750 0x7f9ba41514b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.687+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 >> 172.21.15.53:0/164628006 conn(0x7f9ba409e3b0 msgr2=0x7f9ba406dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.688+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 shutdown_connections 2024-05-10T10:37:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:32.688+0000 7f9ba9b0a640 1 -- 172.21.15.53:0/164628006 wait complete. 2024-05-10T10:37:33.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:37:17.592875+0000', 'modified': '2024-05-10T10:37:32.318466+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 75, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6291': {'gid': 6291, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.53:6834/3202950630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 3202950630}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 3202950630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:37:33.102 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6291:up:replay 2024-05-10T10:37:34.103 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:34.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.219+0000 7f04b04fc640 1 Processor -- start 2024-05-10T10:37:34.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.220+0000 7f04b04fc640 1 -- start start 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04b04fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f04a815f790 0x7f04a80cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04b04fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a80d5d30 0x7f04a80d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04b04fc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a80d30b0 con 0x7f04a815f3b0 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04b04fc640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f04a80d3390 con 0x7f04a815f790 2024-05-10T10:37:34.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04b04fc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f04a80c8c80 con 0x7f04a80d5d30 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04aea99640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a80d5d30 0x7f04a80d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04ae298640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f04a815f3b0 0x7f04a80d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:53882/0 (socket says 172.21.15.53:53882) 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04ae298640 1 -- 172.21.15.53:0/127921347 learned_addr learned my addr 172.21.15.53:0/127921347 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04ada97640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f04a815f790 0x7f04a80cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04ad296640 1 -- 172.21.15.53:0/127921347 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241536803 0 0) 0x7f04a80d30b0 con 0x7f04a815f3b0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04ad296640 1 -- 172.21.15.53:0/127921347 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0494003520 con 0x7f04a815f3b0 2024-05-10T10:37:34.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04aea99640 1 -- 172.21.15.53:0/127921347 >> v2:172.21.15.135:3300/0 conn(0x7f04a815f790 msgr2=0x7f04a80cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04aea99640 1 --2- 172.21.15.53:0/127921347 >> v2:172.21.15.135:3300/0 conn(0x7f04a815f790 0x7f04a80cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04aea99640 1 -- 172.21.15.53:0/127921347 >> v1:172.21.15.135:6789/0 conn(0x7f04a815f3b0 legacy=0x7f04a80d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04aea99640 1 -- 172.21.15.53:0/127921347 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a80c8f60 con 0x7f04a80d5d30 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.221+0000 7f04ada97640 1 --2- 172.21.15.53:0/127921347 >> v2:172.21.15.135:3300/0 conn(0x7f04a815f790 0x7f04a80cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.222+0000 7f04aea99640 1 --2- 172.21.15.53:0/127921347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a80d5d30 0x7f04a80d0210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009d90 tx=0x7f04a002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70540db37b51928 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.222+0000 7f04ad296640 1 -- 172.21.15.53:0/127921347 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0007740 con 0x7f04a80d5d30 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.222+0000 7f04ad296640 1 -- 172.21.15.53:0/127921347 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a00078e0 con 0x7f04a80d5d30 2024-05-10T10:37:34.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.222+0000 7f04ad296640 1 -- 172.21.15.53:0/127921347 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0007c90 con 0x7f04a80d5d30 2024-05-10T10:37:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 -- 172.21.15.53:0/127921347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a80d5d30 msgr2=0x7f04a80d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127921347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a80d5d30 0x7f04a80d0210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009d90 tx=0x7f04a002a660 comp rx=0 tx=0).stop 2024-05-10T10:37:34.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 -- 172.21.15.53:0/127921347 shutdown_connections 2024-05-10T10:37:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127921347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a80d5d30 0x7f04a80d0210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127921347 >> v2:172.21.15.135:3300/0 conn(0x7f04a815f790 0x7f04a80cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 -- 172.21.15.53:0/127921347 >> 172.21.15.53:0/127921347 conn(0x7f04a80d5130 msgr2=0x7f04a80d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 -- 172.21.15.53:0/127921347 shutdown_connections 2024-05-10T10:37:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.223+0000 7f04b04fc640 1 -- 172.21.15.53:0/127921347 wait complete. 2024-05-10T10:37:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 Processor -- start 2024-05-10T10:37:34.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 -- start start 2024-05-10T10:37:34.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a80d5d30 0x7f04a814d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a815f3b0 0x7f04a814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04a80ce880 0x7f04a814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04a80d3f80 con 0x7f04a80d5d30 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04a80d3e00 con 0x7f04a80ce880 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.225+0000 7f04b04fc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f04a80ca770 con 0x7f04a815f3b0 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a80d5d30 0x7f04a814d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a80d5d30 0x7f04a814d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60476/0 (socket says 172.21.15.53:60476) 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 -- 172.21.15.53:0/127784785 learned_addr learned my addr 172.21.15.53:0/127784785 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04aea99640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04a80ce880 0x7f04a814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ada97640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a815f3b0 0x7f04a814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 -- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a815f3b0 msgr2=0x7f04a814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a815f3b0 0x7f04a814e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 -- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04a80ce880 msgr2=0x7f04a814f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04a80ce880 0x7f04a814f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 -- 172.21.15.53:0/127784785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a814eb50 con 0x7f04a80d5d30 2024-05-10T10:37:34.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f04ae298640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a80d5d30 0x7f04a814d070 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f04a400c430 tx=0x7f04a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:34.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f049effd640 1 -- 172.21.15.53:0/127784785 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4017070 con 0x7f04a80d5d30 2024-05-10T10:37:34.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.226+0000 7f049effd640 1 -- 172.21.15.53:0/127784785 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a4005ce0 con 0x7f04a80d5d30 2024-05-10T10:37:34.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.227+0000 7f049effd640 1 -- 172.21.15.53:0/127784785 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a40072f0 con 0x7f04a80d5d30 2024-05-10T10:37:34.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.227+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a814ee30 con 0x7f04a80d5d30 2024-05-10T10:37:34.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.227+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a814f3f0 con 0x7f04a80d5d30 2024-05-10T10:37:34.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.228+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a806a220 con 0x7f04a80d5d30 2024-05-10T10:37:34.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.229+0000 7f049effd640 1 -- 172.21.15.53:0/127784785 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f04a401a070 con 0x7f04a80d5d30 2024-05-10T10:37:34.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.229+0000 7f049effd640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f048c0762d0 0x7f048c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:34.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.229+0000 7f049effd640 1 -- 172.21.15.53:0/127784785 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f04a4096ba0 con 0x7f04a80d5d30 2024-05-10T10:37:34.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.229+0000 7f04ada97640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f048c0762d0 0x7f048c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:34.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.230+0000 7f04ada97640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f048c0762d0 0x7f048c078790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0498009560 tx=0x7f0498009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:34.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.234+0000 7f049effd640 1 -- 172.21.15.53:0/127784785 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f04a405fcd0 con 0x7f04a80d5d30 2024-05-10T10:37:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.467+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04a814f5f0 con 0x7f04a80d5d30 2024-05-10T10:37:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.467+0000 7f049effd640 1 -- 172.21.15.53:0/127784785 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5180 (secure 0 0 0) 0x7f04a4063fd0 con 0x7f04a80d5d30 2024-05-10T10:37:34.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:34.468 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:37:33.325962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-10T10:37:34.469 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 50 2024-05-10T10:37:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f048c0762d0 msgr2=0x7f048c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f048c0762d0 0x7f048c078790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0498009560 tx=0x7f0498009190 comp rx=0 tx=0).stop 2024-05-10T10:37:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a80d5d30 msgr2=0x7f04a814d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a80d5d30 0x7f04a814d070 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f04a400c430 tx=0x7f04a400c8f0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 shutdown_connections 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f048c0762d0 0x7f048c078790 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04a80ce880 0x7f04a814f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f04a815f3b0 0x7f04a814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 --2- 172.21.15.53:0/127784785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a80d5d30 0x7f04a814d070 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 >> 172.21.15.53:0/127784785 conn(0x7f04a80d5130 msgr2=0x7f04a8161b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.469+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 shutdown_connections 2024-05-10T10:37:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:34.470+0000 7f04b04fc640 1 -- 172.21.15.53:0/127784785 wait complete. 2024-05-10T10:37:34.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:37:17.592875+0000', 'modified': '2024-05-10T10:37:33.325962+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 75, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6291': {'gid': 6291, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.53:6834/3202950630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 3202950630}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 3202950630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:37:34.884 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6291:up:rejoin 2024-05-10T10:37:35.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:37:36.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.000+0000 7f413650a640 1 Processor -- start 2024-05-10T10:37:36.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.000+0000 7f413650a640 1 -- start start 2024-05-10T10:37:36.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f413650a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4130162750 0x7f41300d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f413650a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f41300d5cb0 0x7f41300d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f413650a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4130162b30 0x7f41300d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f413650a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f41300b4f20 con 0x7f4130162b30 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f413650a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f41300d0fe0 con 0x7f41300d5cb0 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f413650a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41300b45f0 con 0x7f4130162750 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f4134aa7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4130162b30 0x7f41300d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f4134aa7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4130162b30 0x7f41300d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60488/0 (socket says 172.21.15.53:60488) 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f4134aa7640 1 -- 172.21.15.53:0/792976183 learned_addr learned my addr 172.21.15.53:0/792976183 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f412ffff640 1 --2- 172.21.15.53:0/792976183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4130162750 0x7f41300d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.001+0000 7f412f7fe640 1 --2- 172.21.15.53:0/792976183 >> v2:172.21.15.135:3300/0 conn(0x7f41300d5cb0 0x7f41300d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.002+0000 7f4134aa7640 1 -- 172.21.15.53:0/792976183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4130162750 msgr2=0x7f41300d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.002+0000 7f4134aa7640 1 --2- 172.21.15.53:0/792976183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4130162750 0x7f41300d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.002+0000 7f4134aa7640 1 -- 172.21.15.53:0/792976183 >> v2:172.21.15.135:3300/0 conn(0x7f41300d5cb0 msgr2=0x7f41300d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.002+0000 7f4134aa7640 1 --2- 172.21.15.53:0/792976183 >> v2:172.21.15.135:3300/0 conn(0x7f41300d5cb0 0x7f41300d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.002+0000 7f4134aa7640 1 -- 172.21.15.53:0/792976183 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41300b48d0 con 0x7f4130162b30 2024-05-10T10:37:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.002+0000 7f412f7fe640 1 --2- 172.21.15.53:0/792976183 >> v2:172.21.15.135:3300/0 conn(0x7f41300d5cb0 0x7f41300d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.002+0000 7f4134aa7640 1 --2- 172.21.15.53:0/792976183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300d3b70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f413009eae0 tx=0x7f412802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0e7e1926b8f54f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f412effd640 1 -- 172.21.15.53:0/792976183 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128036070 con 0x7f4130162b30 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f412effd640 1 -- 172.21.15.53:0/792976183 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4128031600 con 0x7f4130162b30 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f412effd640 1 -- 172.21.15.53:0/792976183 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280319d0 con 0x7f4130162b30 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 -- 172.21.15.53:0/792976183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 msgr2=0x7f41300d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 --2- 172.21.15.53:0/792976183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300d3b70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f413009eae0 tx=0x7f412802a610 comp rx=0 tx=0).stop 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 -- 172.21.15.53:0/792976183 shutdown_connections 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 --2- 172.21.15.53:0/792976183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300d3b70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 --2- 172.21.15.53:0/792976183 >> v2:172.21.15.135:3300/0 conn(0x7f41300d5cb0 0x7f41300d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 --2- 172.21.15.53:0/792976183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4130162750 0x7f41300d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 -- 172.21.15.53:0/792976183 >> 172.21.15.53:0/792976183 conn(0x7f413009e3b0 msgr2=0x7f41300d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 -- 172.21.15.53:0/792976183 shutdown_connections 2024-05-10T10:37:36.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.003+0000 7f413650a640 1 -- 172.21.15.53:0/792976183 wait complete. 2024-05-10T10:37:36.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.004+0000 7f413650a640 1 Processor -- start 2024-05-10T10:37:36.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.005+0000 7f413650a640 1 -- start start 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.005+0000 7f413650a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4130162750 0x7f41301535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.005+0000 7f413650a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.005+0000 7f413650a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41300b9a60 0x7f41300b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.005+0000 7f413650a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41300d0d30 con 0x7f4130162b30 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.005+0000 7f413650a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41300d0bb0 con 0x7f4130162750 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.005+0000 7f413650a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41300d1660 con 0x7f41300b9a60 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60500/0 (socket says 172.21.15.53:60500) 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 -- 172.21.15.53:0/3059267973 learned_addr learned my addr 172.21.15.53:0/3059267973 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f4134aa7640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41300b9a60 0x7f41300b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412ffff640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4130162750 0x7f41301535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 -- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41300b9a60 msgr2=0x7f41300b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41300b9a60 0x7f41300b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 -- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4130162750 msgr2=0x7f41301535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4130162750 0x7f41301535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 -- 172.21.15.53:0/3059267973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41300b9f70 con 0x7f4130162b30 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f4134aa7640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41300b9a60 0x7f41300b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412ffff640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4130162750 0x7f41301535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412f7fe640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300b9520 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f412400c7c0 tx=0x7f412400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412cff9640 1 -- 172.21.15.53:0/3059267973 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124015070 con 0x7f4130162b30 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412cff9640 1 -- 172.21.15.53:0/3059267973 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4124007a30 con 0x7f4130162b30 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f412cff9640 1 -- 172.21.15.53:0/3059267973 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124007e00 con 0x7f4130162b30 2024-05-10T10:37:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41300ba250 con 0x7f4130162b30 2024-05-10T10:37:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.006+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41300c8940 con 0x7f4130162b30 2024-05-10T10:37:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.007+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413006a220 con 0x7f4130162b30 2024-05-10T10:37:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.008+0000 7f412cff9640 1 -- 172.21.15.53:0/3059267973 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4124018070 con 0x7f4130162b30 2024-05-10T10:37:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.009+0000 7f412cff9640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f411c0762d0 0x7f411c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.009+0000 7f412cff9640 1 -- 172.21.15.53:0/3059267973 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f4124097770 con 0x7f4130162b30 2024-05-10T10:37:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.009+0000 7f412ffff640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f411c0762d0 0x7f411c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.009+0000 7f412ffff640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f411c0762d0 0x7f411c078790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4120006fd0 tx=0x7f4120008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.013+0000 7f412cff9640 1 -- 172.21.15.53:0/3059267973 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4124060ef0 con 0x7f4130162b30 2024-05-10T10:37:36.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.243+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41300b42a0 con 0x7f4130162b30 2024-05-10T10:37:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.243+0000 7f412cff9640 1 -- 172.21.15.53:0/3059267973 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5187 (secure 0 0 0) 0x7f41240607f0 con 0x7f4130162b30 2024-05-10T10:37:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:37:36.244 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:37:35.325274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":5}]} 2024-05-10T10:37:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 51 2024-05-10T10:37:36.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.245+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f411c0762d0 msgr2=0x7f411c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.245+0000 7f413650a640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f411c0762d0 0x7f411c078790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4120006fd0 tx=0x7f4120008040 comp rx=0 tx=0).stop 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.245+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 msgr2=0x7f41300b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.245+0000 7f413650a640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300b9520 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f412400c7c0 tx=0x7f412400cc80 comp rx=0 tx=0).stop 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 shutdown_connections 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f411c0762d0 0x7f411c078790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41300b9a60 0x7f41300b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4130162b30 0x7f41300b9520 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 --2- 172.21.15.53:0/3059267973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4130162750 0x7f41301535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 >> 172.21.15.53:0/3059267973 conn(0x7f413009e3b0 msgr2=0x7f41300d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 shutdown_connections 2024-05-10T10:37:36.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.246+0000 7f413650a640 1 -- 172.21.15.53:0/3059267973 wait complete. 2024-05-10T10:37:36.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:37:17.592875+0000', 'modified': '2024-05-10T10:37:35.325274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 75, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6291': {'gid': 6291, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6834/3202950630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 3202950630}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 3202950630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6291, 'qdb_cluster': [6291]} 2024-05-10T10:37:36.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:37:36.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T10:37:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.768+0000 7f6cc8209640 1 Processor -- start 2024-05-10T10:37:36.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.769+0000 7f6cc8209640 1 -- start start 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.769+0000 7f6cc8209640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6cc015f3b0 0x7f6cc00d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.769+0000 7f6cc8209640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6cc015f790 0x7f6cc00d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.769+0000 7f6cc8209640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc00d5d30 0x7f6cc00c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.769+0000 7f6cc8209640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc00d30b0 con 0x7f6cc015f790 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.769+0000 7f6cc8209640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc00d3390 con 0x7f6cc015f3b0 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.769+0000 7f6cc8209640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc00c8c80 con 0x7f6cc00d5d30 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6cc015f790 0x7f6cc00d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6cc015f790 0x7f6cc00d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60508/0 (socket says 172.21.15.53:60508) 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 -- 172.21.15.53:0/272259807 learned_addr learned my addr 172.21.15.53:0/272259807 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 -- 172.21.15.53:0/272259807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc00d5d30 msgr2=0x7f6cc00c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/272259807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc00d5d30 0x7f6cc00c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/272259807 >> v2:172.21.15.135:3300/0 conn(0x7f6cc015f3b0 0x7f6cc00d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/272259807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc00d5d30 0x7f6cc00c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 -- 172.21.15.53:0/272259807 >> v2:172.21.15.135:3300/0 conn(0x7f6cc015f3b0 msgr2=0x7f6cc00d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/272259807 >> v2:172.21.15.135:3300/0 conn(0x7f6cc015f3b0 0x7f6cc00d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.770+0000 7f6cc57a4640 1 -- 172.21.15.53:0/272259807 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc00c8f60 con 0x7f6cc015f790 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/272259807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f790 0x7f6cc00d3c30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6cc008e820 tx=0x7f6cb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4307ee356f3145ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc4fa3640 1 -- 172.21.15.53:0/272259807 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4037070 con 0x7f6cc015f790 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc4fa3640 1 -- 172.21.15.53:0/272259807 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4032440 con 0x7f6cc015f790 2024-05-10T10:37:36.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc4fa3640 1 -- 172.21.15.53:0/272259807 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4032810 con 0x7f6cc015f790 2024-05-10T10:37:36.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 -- 172.21.15.53:0/272259807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f790 msgr2=0x7f6cc00d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 --2- 172.21.15.53:0/272259807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f790 0x7f6cc00d3c30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6cc008e820 tx=0x7f6cb402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 -- 172.21.15.53:0/272259807 shutdown_connections 2024-05-10T10:37:36.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 --2- 172.21.15.53:0/272259807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc00d5d30 0x7f6cc00c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 --2- 172.21.15.53:0/272259807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f790 0x7f6cc00d3c30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 --2- 172.21.15.53:0/272259807 >> v2:172.21.15.135:3300/0 conn(0x7f6cc015f3b0 0x7f6cc00d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 -- 172.21.15.53:0/272259807 >> 172.21.15.53:0/272259807 conn(0x7f6cc00d5130 msgr2=0x7f6cc00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 -- 172.21.15.53:0/272259807 shutdown_connections 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.771+0000 7f6cc8209640 1 -- 172.21.15.53:0/272259807 wait complete. 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 Processor -- start 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 -- start start 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cc00d5d30 0x7f6cc0154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f3b0 0x7f6cc0157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc0157ff0 0x7f6cc0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc00c9e20 con 0x7f6cc015f3b0 2024-05-10T10:37:36.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc00c9ca0 con 0x7f6cc00d5d30 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc8209640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc00c9fa0 con 0x7f6cc0157ff0 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.773+0000 7f6cc57a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f3b0 0x7f6cc0157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f3b0 0x7f6cc0157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60522/0 (socket says 172.21.15.53:60522) 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 -- 172.21.15.53:0/3030026310 learned_addr learned my addr 172.21.15.53:0/3030026310 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 -- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc0157ff0 msgr2=0x7f6cc0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc0157ff0 0x7f6cc0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 -- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cc00d5d30 msgr2=0x7f6cc0154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cc00d5d30 0x7f6cc0154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 -- 172.21.15.53:0/3030026310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc0154b70 con 0x7f6cc015f3b0 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f3b0 0x7f6cc0157ab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6cb402a220 tx=0x7f6cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cbe7fc640 1 -- 172.21.15.53:0/3030026310 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4037040 con 0x7f6cc015f3b0 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cbe7fc640 1 -- 172.21.15.53:0/3030026310 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4042070 con 0x7f6cc015f3b0 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc01584b0 con 0x7f6cc015f3b0 2024-05-10T10:37:36.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cbe7fc640 1 -- 172.21.15.53:0/3030026310 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4030bb0 con 0x7f6cc015f3b0 2024-05-10T10:37:36.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.774+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc01f5f60 con 0x7f6cc015f3b0 2024-05-10T10:37:36.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.776+0000 7f6cbe7fc640 1 -- 172.21.15.53:0/3030026310 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6cb4047460 con 0x7f6cc015f3b0 2024-05-10T10:37:36.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.776+0000 7f6cbe7fc640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cac0762d0 0x7f6cac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.776+0000 7f6cbe7fc640 1 -- 172.21.15.53:0/3030026310 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6cb40b84a0 con 0x7f6cc015f3b0 2024-05-10T10:37:36.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.777+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cac0762d0 0x7f6cac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.777+0000 7f6ccaffe740 1 Processor -- start 2024-05-10T10:37:36.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.777+0000 7f6ccaffe740 1 -- start start 2024-05-10T10:37:36.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.777+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cac0762d0 0x7f6cac078790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007e60 tx=0x7f6cb0007390 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394075760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394075ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394045bb0 0x5603940493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560393faf8d0 con 0x560394072f50 2024-05-10T10:37:36.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560393f68c10 con 0x560394045bb0 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560393fab890 con 0x560394073330 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394075760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60528/0 (socket says 172.21.15.53:60528) 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/1999782150 learned_addr learned my addr 172.21.15.53:0/1999782150 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394045bb0 0x5603940493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394075ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 msgr2=0x560394075ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394075ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394045bb0 msgr2=0x5603940493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394045bb0 0x5603940493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/1999782150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560393f705d0 con 0x560394072f50 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394075ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.778+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394045bb0 0x5603940493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394075760 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000e580 tx=0x7f6cb000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5eea086f3669c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ca77fe640 1 -- 172.21.15.53:0/1999782150 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb001f070 con 0x560394072f50 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ca77fe640 1 -- 172.21.15.53:0/1999782150 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb001a880 con 0x560394072f50 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ca77fe640 1 -- 172.21.15.53:0/1999782150 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb001ac30 con 0x560394072f50 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1999782150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 msgr2=0x560394075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394075760 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000e580 tx=0x7f6cb000f000 comp rx=0 tx=0).stop 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1999782150 shutdown_connections 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394045bb0 0x5603940493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394075ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1999782150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394075760 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1999782150 >> 172.21.15.53:0/1999782150 conn(0x560394047e20 msgr2=0x560394090760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1999782150 shutdown_connections 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.779+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1999782150 wait complete. 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.780+0000 7f6ccaffe740 1 -- start start 2024-05-10T10:37:36.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.780+0000 7f6ccaffe740 1 Processor -- start 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394076690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394068860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394068da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56039404afd0 con 0x560394072f50 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560393f6c760 con 0x560394069530 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56039404b2b0 con 0x560394073330 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6ccaffe740 10 client.0 _ll_register_callbacks cb 0x560393fa5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394076690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394076690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60540/0 (socket says 172.21.15.53:60540) 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/2759655606 learned_addr learned my addr 172.21.15.53:0/2759655606 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394068da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394068860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 msgr2=0x560394068860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394068860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 msgr2=0x560394068da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394068da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc5fa5640 1 -- 172.21.15.53:0/2759655606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560393cc12d0 con 0x560394072f50 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.781+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394076690 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x56039404d4b0 tx=0x7f6cb0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0ccd17f4a8ea16 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/2759655606 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0032060 con 0x560394072f50 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/2759655606 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6cb001a680 con 0x560394072f50 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb001a880 con 0x560394072f50 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/2759655606 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6cb0038050 con 0x560394072f50 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 10 client.6333 fetch_fsmap learned FSMap version 51 2024-05-10T10:37:36.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x560393d8b6b0 con 0x560394072f50 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/2759655606 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb001a880 con 0x560394072f50 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/2759655606 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f6cb003b020 con 0x560394072f50 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 10 client.6333 fetch_fsmap finished waiting for FSMap version 51 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 10 client.6333 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 20 client.6333 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940780c0 0x5603940784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 4 client.6333 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560393f33760 con 0x5603940780c0 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.782+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940780c0 0x5603940784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ca4ff9640 10 client.6333 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940780c0 0x5603940784a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47044403360a814f server_cookie=ddf600c20b8333e4 in_seq=0 out_seq=0 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/2759655606 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560393f33760 con 0x5603940780c0 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ca4ff9640 10 client.6333 handle_command_reply: tid=0 2024-05-10T10:37:36.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ccaffe740 1 client.6333 shutdown 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 msgr2=0x560394076690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394076690 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x56039404d4b0 tx=0x7f6cb0013000 comp rx=0 tx=0).stop 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 shutdown_connections 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.783+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940780c0 0x5603940784a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x560394068860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ca4ff9640 0 client.6333 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2759655606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x560394076690 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 >> 172.21.15.53:0/2759655606 conn(0x560394045bb0 msgr2=0x5603940498a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 shutdown_connections 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2759655606 wait complete. 2024-05-10T10:37:36.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.784+0000 7f6ccaffe740 20 client.6333 trim_cache size 0 max 16384 2024-05-10T10:37:36.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 Processor -- start 2024-05-10T10:37:36.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 -- start start 2024-05-10T10:37:36.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x5603940450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x5603940493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0200fa0 con 0x560394072f50 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0200300 con 0x560394069530 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc0200950 con 0x560394073330 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6cc57a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x5603940450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6cc57a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x5603940450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60554/0 (socket says 172.21.15.53:60554) 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.803+0000 7f6cc57a4640 1 -- 172.21.15.53:0/2115013352 learned_addr learned my addr 172.21.15.53:0/2115013352 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6cc57a4640 1 -- 172.21.15.53:0/2115013352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 msgr2=0x5603940493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/2115013352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x5603940493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6cc57a4640 1 -- 172.21.15.53:0/2115013352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 msgr2=0x560394066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/2115013352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6cc57a4640 1 -- 172.21.15.53:0/2115013352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560394066ea0 con 0x560394072f50 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/2115013352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x5603940450f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6cb407e6f0 tx=0x7f6cb407d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ec225a39d85c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6ca6ffd640 1 -- 172.21.15.53:0/2115013352 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4081480 con 0x560394072f50 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6ca6ffd640 1 -- 172.21.15.53:0/2115013352 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4085830 con 0x560394072f50 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6ca6ffd640 1 -- 172.21.15.53:0/2115013352 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb408a250 con 0x560394072f50 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2115013352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 msgr2=0x5603940450f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.804+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2115013352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x5603940450f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6cb407e6f0 tx=0x7f6cb407d340 comp rx=0 tx=0).stop 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2115013352 shutdown_connections 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2115013352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394073330 0x5603940493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2115013352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394072f50 0x5603940450f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/2115013352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394066830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2115013352 >> 172.21.15.53:0/2115013352 conn(0x560394045bb0 msgr2=0x5603940764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2115013352 shutdown_connections 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 -- 172.21.15.53:0/2115013352 wait complete. 2024-05-10T10:37:36.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 -- start start 2024-05-10T10:37:36.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.805+0000 7f6ccaffe740 1 Processor -- start 2024-05-10T10:37:36.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394066200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394072f50 0x560394066740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6ccaffe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394073330 0x560394090760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560394093560 con 0x560394073330 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560394078e80 con 0x560394069530 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6ccaffe740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560394092b50 con 0x560394072f50 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6ccaffe740 10 client.0 _ll_register_callbacks cb 0x7f6cb000e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc67a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394073330 0x560394090760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc67a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394073330 0x560394090760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60568/0 (socket says 172.21.15.53:60568) 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc67a6640 1 -- 172.21.15.53:0/1613029285 learned_addr learned my addr 172.21.15.53:0/1613029285 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc57a4640 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394072f50 0x560394066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394066200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc67a6640 1 -- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394072f50 msgr2=0x560394066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394072f50 0x560394066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc67a6640 1 -- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 msgr2=0x560394066200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.806+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394066200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6cc67a6640 1 -- 172.21.15.53:0/1613029285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560394094090 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6cc67a6640 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394073330 0x560394090760 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800ef30 tx=0x7f6cb80106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3b045e104d167c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/1613029285 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8010e50 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/1613029285 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6cb8004030 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb4085830 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/1613029285 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6cb8023050 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 10 client.6339 fetch_fsmap learned FSMap version 51 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x560394089400 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/1613029285 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4085830 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/1613029285 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f6cb8026020 con 0x560394073330 2024-05-10T10:37:36.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 10 client.6339 fetch_fsmap finished waiting for FSMap version 51 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 10 client.6339 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 20 client.6339 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940b3610 0x5603940b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 4 client.6339 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "status"}) -- 0x560393fbd4e0 con 0x5603940b3610 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.807+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940b3610 0x5603940b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6cc5fa5640 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940b3610 0x5603940b39f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af88d9e3c5116927 server_cookie=f460351adefff79c in_seq=0 out_seq=0 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ca4ff9640 10 client.6339 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ca4ff9640 1 -- 172.21.15.53:0/1613029285 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560393fbd4e0 con 0x5603940b3610 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ca4ff9640 10 client.6339 handle_command_reply: tid=0 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 client.6339 shutdown 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394073330 msgr2=0x560394090760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394073330 0x560394090760 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800ef30 tx=0x7f6cb80106e0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 shutdown_connections 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5603940b3610 0x5603940b39f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560394073330 0x560394090760 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560394072f50 0x560394066740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 --2- 172.21.15.53:0/1613029285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560394069530 0x560394066200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 >> 172.21.15.53:0/1613029285 conn(0x560394045bb0 msgr2=0x560394093730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ca4ff9640 0 client.6339 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 shutdown_connections 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 1 -- 172.21.15.53:0/1613029285 wait complete. 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.808+0000 7f6ccaffe740 20 client.6339 trim_cache size 0 max 16384 2024-05-10T10:37:36.816 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 6291, 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 4.4860367549999998, 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 51, 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 75, 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 75, 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 5.4565951469999998 2024-05-10T10:37:36.817 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:37:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cac0762d0 msgr2=0x7f6cac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cac0762d0 0x7f6cac078790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007e60 tx=0x7f6cb0007390 comp rx=0 tx=0).stop 2024-05-10T10:37:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f3b0 msgr2=0x7f6cc0157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f3b0 0x7f6cc0157ab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6cb402a220 tx=0x7f6cb40046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 shutdown_connections 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6cac0762d0 0x7f6cac078790 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6cc0157ff0 0x7f6cc0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc015f3b0 0x7f6cc0157ab0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 --2- 172.21.15.53:0/3030026310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cc00d5d30 0x7f6cc0154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 >> 172.21.15.53:0/3030026310 conn(0x7f6cc00d5130 msgr2=0x7f6cc00cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 shutdown_connections 2024-05-10T10:37:36.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.809+0000 7f6cc8209640 1 -- 172.21.15.53:0/3030026310 wait complete. 2024-05-10T10:37:36.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:37:36.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.923+0000 7fc9a0c99640 1 Processor -- start 2024-05-10T10:37:36.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.923+0000 7fc9a0c99640 1 -- start start 2024-05-10T10:37:36.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc9a0c99640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc99c15d290 0x7fc99c071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc9a0c99640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc99c0d3510 0x7fc99c0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc9a0c99640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc99c072330 con 0x7fc99c071880 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc9a0c99640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc99c071d70 con 0x7fc99c0d3510 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc9a0c99640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc99c072050 con 0x7fc99c15d290 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc99ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc99c0d3510 0x7fc99c0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc999d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc99c071880 0x7fc99c0d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55414/0 (socket says 172.21.15.53:55414) 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.924+0000 7fc999d9b640 1 -- 172.21.15.53:0/3743039586 learned_addr learned my addr 172.21.15.53:0/3743039586 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.925+0000 7fc99a59c640 1 --2- 172.21.15.53:0/3743039586 >> v2:172.21.15.135:3300/0 conn(0x7fc99c15d290 0x7fc99c071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.925+0000 7fc99ad9d640 1 -- 172.21.15.53:0/3743039586 >> v2:172.21.15.135:3300/0 conn(0x7fc99c15d290 msgr2=0x7fc99c071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.925+0000 7fc99ad9d640 1 --2- 172.21.15.53:0/3743039586 >> v2:172.21.15.135:3300/0 conn(0x7fc99c15d290 0x7fc99c071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.925+0000 7fc99ad9d640 1 -- 172.21.15.53:0/3743039586 >> v1:172.21.15.53:6789/0 conn(0x7fc99c071880 legacy=0x7fc99c0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.925+0000 7fc99959a640 1 -- 172.21.15.53:0/3743039586 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219420263 0 0) 0x7fc99c072330 con 0x7fc99c071880 2024-05-10T10:37:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.925+0000 7fc99ad9d640 1 -- 172.21.15.53:0/3743039586 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c0cda70 con 0x7fc99c0d3510 2024-05-10T10:37:36.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc99ad9d640 1 --2- 172.21.15.53:0/3743039586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c0ce270 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0701d0 tx=0x7fc99402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5752d92ba315aae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc99959a640 1 -- 172.21.15.53:0/3743039586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994007740 con 0x7fc99c0d3510 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc99959a640 1 -- 172.21.15.53:0/3743039586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9940078e0 con 0x7fc99c0d3510 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc99959a640 1 -- 172.21.15.53:0/3743039586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994007cb0 con 0x7fc99c0d3510 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 -- 172.21.15.53:0/3743039586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 msgr2=0x7fc99c0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/3743039586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c0ce270 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0701d0 tx=0x7fc99402a680 comp rx=0 tx=0).stop 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 -- 172.21.15.53:0/3743039586 shutdown_connections 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/3743039586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c0ce270 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/3743039586 >> v2:172.21.15.135:3300/0 conn(0x7fc99c15d290 0x7fc99c071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 -- 172.21.15.53:0/3743039586 >> 172.21.15.53:0/3743039586 conn(0x7fc99c09ba20 msgr2=0x7fc99c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 -- 172.21.15.53:0/3743039586 shutdown_connections 2024-05-10T10:37:36.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.926+0000 7fc9a0c99640 1 -- 172.21.15.53:0/3743039586 wait complete. 2024-05-10T10:37:36.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 Processor -- start 2024-05-10T10:37:36.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 -- start start 2024-05-10T10:37:36.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc99c071880 0x7fc99c0b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc99c15d290 0x7fc99c1f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c0d1c20 con 0x7fc99c0d3510 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c0d1aa0 con 0x7fc99c071880 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.928+0000 7fc9a0c99640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc99c159860 con 0x7fc99c15d290 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60580/0 (socket says 172.21.15.53:60580) 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 -- 172.21.15.53:0/73473910 learned_addr learned my addr 172.21.15.53:0/73473910 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc99ad9d640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc99c15d290 0x7fc99c1f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc99a59c640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc99c071880 0x7fc99c0b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 -- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc99c15d290 msgr2=0x7fc99c1f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc99c15d290 0x7fc99c1f30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 -- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc99c071880 msgr2=0x7fc99c0b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc99c071880 0x7fc99c0b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:36.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 -- 172.21.15.53:0/73473910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c0cb580 con 0x7fc99c0d3510 2024-05-10T10:37:36.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc999d9b640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c159320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c490 tx=0x7fc99000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc98affd640 1 -- 172.21.15.53:0/73473910 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990018070 con 0x7fc99c0d3510 2024-05-10T10:37:36.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc98affd640 1 -- 172.21.15.53:0/73473910 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9900074e0 con 0x7fc99c0d3510 2024-05-10T10:37:36.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c0cb860 con 0x7fc99c0d3510 2024-05-10T10:37:36.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc98affd640 1 -- 172.21.15.53:0/73473910 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990004a20 con 0x7fc99c0d3510 2024-05-10T10:37:36.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.929+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c0cbe20 con 0x7fc99c0d3510 2024-05-10T10:37:36.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.930+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c0cc020 con 0x7fc99c0d3510 2024-05-10T10:37:36.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.931+0000 7fc98affd640 1 -- 172.21.15.53:0/73473910 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc99001b070 con 0x7fc99c0d3510 2024-05-10T10:37:36.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.932+0000 7fc98affd640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc97c0762d0 0x7fc97c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:36.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.932+0000 7fc98affd640 1 -- 172.21.15.53:0/73473910 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fc990096b30 con 0x7fc99c0d3510 2024-05-10T10:37:36.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.932+0000 7fc99a59c640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc97c0762d0 0x7fc97c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:36.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.932+0000 7fc99a59c640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc97c0762d0 0x7fc97c078790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009970 tx=0x7fc98c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:36.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:36.936+0000 7fc98affd640 1 -- 172.21.15.53:0/73473910 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc99c0cc020 con 0x7fc99c0d3510 2024-05-10T10:37:37.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.165+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc99c0693f0 con 0x7fc99c0d3510 2024-05-10T10:37:37.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.166+0000 7fc98affd640 1 -- 172.21.15.53:0/73473910 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7fc990068060 con 0x7fc99c0d3510 2024-05-10T10:37:37.166 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:37:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.167+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc97c0762d0 msgr2=0x7fc97c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.167+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc97c0762d0 0x7fc97c078790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009970 tx=0x7fc98c008040 comp rx=0 tx=0).stop 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 msgr2=0x7fc99c159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c159320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c490 tx=0x7fc99000c950 comp rx=0 tx=0).stop 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 shutdown_connections 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc97c0762d0 0x7fc97c078790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc99c15d290 0x7fc99c1f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c0d3510 0x7fc99c159320 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 --2- 172.21.15.53:0/73473910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc99c071880 0x7fc99c0b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 >> 172.21.15.53:0/73473910 conn(0x7fc99c09ba20 msgr2=0x7fc99c0b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 shutdown_connections 2024-05-10T10:37:37.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37.168+0000 7fc9a0c99640 1 -- 172.21.15.53:0/73473910 wait complete. 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:37:37.179 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:37:37.180 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:37:37.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.180 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:37:37.196 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:37:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:37:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:37:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42512sec preferred_lft 42512sec 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:37:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:37:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:37.199 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:37:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:37:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:37:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:37:37.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.199 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:37:37.260 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:37:37.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.262 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:37:37.318 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:37:37.319 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:37:37.319 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:37:37.319 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:37:37.319 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:37:37.319 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:37:37.319 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:37:37.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:37:37.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:37:37.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:37:37.320 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:37:37.324 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:37:37.323+0000 7f3e71eb3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:37:37.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:37:37.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:37:37.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.412 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:37:37.428 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:37:37.428 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:37:37.429 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:37:37.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:37:37.494 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:37:37.494 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:37.550 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:37:37.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:37:37.551 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:37.606 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:37:37.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:37.691 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:37.691 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:37:37.692 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:37:37.700 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:37.700 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:37:37.700 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:37:37.700 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:37:37.701 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:37:37.705 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:37:37.705 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:37:37.705 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:37:37.705 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:37:37.705 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:37:37.705 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:37:37.706 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:37:37.706 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:37:37.706 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:37:37.706 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:37:37.706 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:37:37.706 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:37:37.706 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:37:37.707 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:37:37.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:37.733 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:37:37.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:37:37.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:37:37.832 INFO:teuthology.orchestra.run.smithi053.stdout:client6351 2024-05-10T10:37:37.832 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:37.832 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:37.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:37.848 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-05-10T10:37:37.906 INFO:teuthology.orchestra.run.smithi053.stdout:file_undamaged 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> import json 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> import sys 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> try: 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> except OSError as e: 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> sys.exit(e.errno) 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> print(json.dumps( 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> dict([(a, getattr(s, a)) for a in attrs]), 2024-05-10T10:37:37.907 DEBUG:teuthology.orchestra.run.smithi053:> indent=2)) 2024-05-10T10:37:37.908 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:37:38.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-05-10T10:37:38.025 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-10T10:37:38.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:38.025 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-10T10:37:38.082 INFO:teuthology.orchestra.run.smithi053.stdout:2 2024-05-10T10:37:38.082 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:38.082 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:38.139 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:37:38.139 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:37:38.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:37:38.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.341+0000 7f6f15fb6640 1 Processor -- start 2024-05-10T10:37:38.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.341+0000 7f6f15fb6640 1 -- start start 2024-05-10T10:37:38.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f15fb6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f10168220 0x7f6f1016a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f15fb6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f100b0a90 con 0x7f6f1015f8c0 2024-05-10T10:37:38.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f15fb6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f100b3540 con 0x7f6f1015f4e0 2024-05-10T10:37:38.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f15fb6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6f100b23f0 con 0x7f6f10168220 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6f1015f8c0 0x7f6f1006e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55430/0 (socket says 172.21.15.53:55430) 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0effd640 1 -- 172.21.15.53:0/3521805135 learned_addr learned my addr 172.21.15.53:0/3521805135 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0ffff640 1 --2- 172.21.15.53:0/3521805135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f10168220 0x7f6f1016a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337363808 0 0) 0x7f6f100b0a90 con 0x7f6f1015f8c0 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ef8003520 con 0x7f6f1015f8c0 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461967794 0 0) 0x7f6f100b3540 con 0x7f6f1015f4e0 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f100b0a90 con 0x7f6f1015f4e0 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3157450964 0 0) 0x7f6ef8003520 con 0x7f6f1015f8c0 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0ffff640 1 -- 172.21.15.53:0/3521805135 >> v1:172.21.15.135:6789/0 conn(0x7f6f1015f4e0 legacy=0x7f6f1006d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0ffff640 1 -- 172.21.15.53:0/3521805135 >> v1:172.21.15.53:6789/0 conn(0x7f6f1015f8c0 legacy=0x7f6f1006e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.342+0000 7f6f0ffff640 1 -- 172.21.15.53:0/3521805135 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f100b3820 con 0x7f6f10168220 2024-05-10T10:37:38.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.343+0000 7f6f0ffff640 1 --2- 172.21.15.53:0/3521805135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f10168220 0x7f6f1016a610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009dc0 tx=0x7f6f0802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f73026011ab990 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.343+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04001670 con 0x7f6f10168220 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.343+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f08007540 con 0x7f6f10168220 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.343+0000 7f6f0e7fc640 1 -- 172.21.15.53:0/3521805135 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080078f0 con 0x7f6f10168220 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.344+0000 7f6f15fb6640 1 -- 172.21.15.53:0/3521805135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f10168220 msgr2=0x7f6f1016a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.344+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/3521805135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f10168220 0x7f6f1016a610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009dc0 tx=0x7f6f0802a5d0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.344+0000 7f6f15fb6640 1 -- 172.21.15.53:0/3521805135 shutdown_connections 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.344+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/3521805135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f10168220 0x7f6f1016a610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.344+0000 7f6f15fb6640 1 -- 172.21.15.53:0/3521805135 >> 172.21.15.53:0/3521805135 conn(0x7f6f100d2df0 msgr2=0x7f6f100d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.344+0000 7f6f15fb6640 1 -- 172.21.15.53:0/3521805135 shutdown_connections 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.344+0000 7f6f15fb6640 1 -- 172.21.15.53:0/3521805135 wait complete. 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.345+0000 7f6f15fb6640 1 Processor -- start 2024-05-10T10:37:38.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.345+0000 7f6f15fb6640 1 -- start start 2024-05-10T10:37:38.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f15fb6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f1015f4e0 0x7f6f100c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f15fb6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f1015f8c0 0x7f6f1015ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f15fb6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f10168220 0x7f6f100c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f15fb6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f100b3de0 con 0x7f6f1015f8c0 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f15fb6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6f100b4150 con 0x7f6f10168220 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f15fb6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1006e7f0 con 0x7f6f1015f4e0 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f1015f8c0 0x7f6f1015ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f1015f8c0 0x7f6f1015ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60586/0 (socket says 172.21.15.53:60586) 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f0effd640 1 -- 172.21.15.53:0/919421402 learned_addr learned my addr 172.21.15.53:0/919421402 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f0f7fe640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f1015f4e0 0x7f6f100c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.346+0000 7f6f0ffff640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f10168220 0x7f6f100c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f0effd640 1 -- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f1015f4e0 msgr2=0x7f6f100c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f0effd640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f1015f4e0 0x7f6f100c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f0effd640 1 -- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f10168220 msgr2=0x7f6f100c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f0effd640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f10168220 0x7f6f100c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f0effd640 1 -- 172.21.15.53:0/919421402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f101583b0 con 0x7f6f1015f8c0 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f0effd640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f1015f8c0 0x7f6f1015ae10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c700 tx=0x7f6f0400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6ef7fff640 1 -- 172.21.15.53:0/919421402 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04017070 con 0x7f6f1015f8c0 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6ef7fff640 1 -- 172.21.15.53:0/919421402 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f04005ad0 con 0x7f6f1015f8c0 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6ef7fff640 1 -- 172.21.15.53:0/919421402 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04007070 con 0x7f6f1015f8c0 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f10158690 con 0x7f6f1015f8c0 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.347+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f10158f80 con 0x7f6f1015f8c0 2024-05-10T10:37:38.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.348+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1006a220 con 0x7f6f1015f8c0 2024-05-10T10:37:38.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.349+0000 7f6ef7fff640 1 -- 172.21.15.53:0/919421402 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6f0401a070 con 0x7f6f1015f8c0 2024-05-10T10:37:38.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.349+0000 7f6ef7fff640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6efc0762d0 0x7f6efc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.349+0000 7f6ef7fff640 1 -- 172.21.15.53:0/919421402 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6f04096cb0 con 0x7f6f1015f8c0 2024-05-10T10:37:38.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.350+0000 7f6f0f7fe640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6efc0762d0 0x7f6efc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.350+0000 7f6f0f7fe640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6efc0762d0 0x7f6efc078790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f00007fd0 tx=0x7f6f00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.354+0000 7f6ef7fff640 1 -- 172.21.15.53:0/919421402 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6f0405fd40 con 0x7f6f1015f8c0 2024-05-10T10:37:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.559+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6f10158860 con 0x7f6f1015f8c0 2024-05-10T10:37:38.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.559+0000 7f6ef7fff640 1 -- 172.21.15.53:0/919421402 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v75) ==== 80+0+122 (secure 0 0 0) 0x7f6f04064040 con 0x7f6f1015f8c0 2024-05-10T10:37:38.560 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6834/1984524370 2024-05-11T10:37:31.312633+0000 2024-05-10T10:37:38.560 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6832/1984524370 2024-05-11T10:37:31.312633+0000 2024-05-10T10:37:38.561 INFO:teuthology.orchestra.run.smithi053.stderr:listed 2 entries 2024-05-10T10:37:38.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6efc0762d0 msgr2=0x7f6efc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6efc0762d0 0x7f6efc078790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f00007fd0 tx=0x7f6f00009040 comp rx=0 tx=0).stop 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f1015f8c0 msgr2=0x7f6f1015ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f1015f8c0 0x7f6f1015ae10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c700 tx=0x7f6f0400cbc0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 shutdown_connections 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6efc0762d0 0x7f6efc078790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f10168220 0x7f6f100c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f1015f8c0 0x7f6f1015ae10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 --2- 172.21.15.53:0/919421402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f1015f4e0 0x7f6f100c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 >> 172.21.15.53:0/919421402 conn(0x7f6f100d2df0 msgr2=0x7f6f10162340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 shutdown_connections 2024-05-10T10:37:38.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.562+0000 7f6f15fb6640 1 -- 172.21.15.53:0/919421402 wait complete. 2024-05-10T10:37:38.573 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:37:38.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:38.574 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:37:38.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:37:38.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:38.603 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:37:38.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:38.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-05-10T10:37:38.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.805+0000 7f0b0e978640 1 Processor -- start 2024-05-10T10:37:38.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.806+0000 7f0b0e978640 1 -- start start 2024-05-10T10:37:38.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.806+0000 7f0b0e978640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b0815f490 0x7f0b080d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.806+0000 7f0b0e978640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.806+0000 7f0b0e978640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b080d1080 con 0x7f0b080d3b80 2024-05-10T10:37:38.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.806+0000 7f0b0e978640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0b080b4e70 con 0x7f0b0815f490 2024-05-10T10:37:38.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.806+0000 7f0b0e978640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0b080b4820 con 0x7f0b080d5ca0 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.806+0000 7f0b07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b0815f490 0x7f0b080d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b0cf15640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0b080d3b80 0x7f0b080cfc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55446/0 (socket says 172.21.15.53:55446) 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b0cf15640 1 -- 172.21.15.53:0/3728467954 learned_addr learned my addr 172.21.15.53:0/3728467954 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b0815f490 0x7f0b080d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60598/0 (socket says 172.21.15.53:60598) 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b077fe640 1 --2- 172.21.15.53:0/3728467954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b07fff640 1 -- 172.21.15.53:0/3728467954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 msgr2=0x7f0b080d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3728467954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080d3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b07fff640 1 -- 172.21.15.53:0/3728467954 >> v1:172.21.15.53:6789/0 conn(0x7f0b080d3b80 legacy=0x7f0b080cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b07fff640 1 -- 172.21.15.53:0/3728467954 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b080b4b00 con 0x7f0b0815f490 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b06ffd640 1 -- 172.21.15.53:0/3728467954 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3772782013 0 0) 0x7f0b080d1080 con 0x7f0b080d3b80 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.807+0000 7f0b077fe640 1 --2- 172.21.15.53:0/3728467954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3728467954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b0815f490 0x7f0b080d2f60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097a0 tx=0x7f0af802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d577bd47fa7bfa44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b06ffd640 1 -- 172.21.15.53:0/3728467954 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8037070 con 0x7f0b0815f490 2024-05-10T10:37:38.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b06ffd640 1 -- 172.21.15.53:0/3728467954 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af802ae50 con 0x7f0b0815f490 2024-05-10T10:37:38.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b06ffd640 1 -- 172.21.15.53:0/3728467954 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8032660 con 0x7f0b0815f490 2024-05-10T10:37:38.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 -- 172.21.15.53:0/3728467954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b0815f490 msgr2=0x7f0b080d2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 --2- 172.21.15.53:0/3728467954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b0815f490 0x7f0b080d2f60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097a0 tx=0x7f0af802a100 comp rx=0 tx=0).stop 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 -- 172.21.15.53:0/3728467954 shutdown_connections 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 --2- 172.21.15.53:0/3728467954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 --2- 172.21.15.53:0/3728467954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b0815f490 0x7f0b080d2f60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 -- 172.21.15.53:0/3728467954 >> 172.21.15.53:0/3728467954 conn(0x7f0b080d50f0 msgr2=0x7f0b080d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 -- 172.21.15.53:0/3728467954 shutdown_connections 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.808+0000 7f0b0e978640 1 -- 172.21.15.53:0/3728467954 wait complete. 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 Processor -- start 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 -- start start 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b080d3b80 0x7f0b0815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b0815f490 0x7f0b080ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b080b6310 con 0x7f0b080d3b80 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0815fa80 con 0x7f0b0815f490 2024-05-10T10:37:38.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.810+0000 7f0b0e978640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0b080b6490 con 0x7f0b080d5ca0 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b080d3b80 0x7f0b0815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b080d3b80 0x7f0b0815a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60612/0 (socket says 172.21.15.53:60612) 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 -- 172.21.15.53:0/878772569 learned_addr learned my addr 172.21.15.53:0/878772569 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b077fe640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b0815f490 0x7f0b080ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 -- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 msgr2=0x7f0b080cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 -- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b0815f490 msgr2=0x7f0b080ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b0815f490 0x7f0b080ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 -- 172.21.15.53:0/878772569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b080b39d0 con 0x7f0b080d3b80 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b07fff640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b080d3b80 0x7f0b0815a890 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097a0 tx=0x7f0af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b04ff9640 1 -- 172.21.15.53:0/878772569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8037040 con 0x7f0b080d3b80 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b04ff9640 1 -- 172.21.15.53:0/878772569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af8004030 con 0x7f0b080d3b80 2024-05-10T10:37:38.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b080cc730 con 0x7f0b080d3b80 2024-05-10T10:37:38.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b04ff9640 1 -- 172.21.15.53:0/878772569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8006e10 con 0x7f0b080d3b80 2024-05-10T10:37:38.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.811+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b080b3f50 con 0x7f0b080d3b80 2024-05-10T10:37:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.813+0000 7f0b04ff9640 1 -- 172.21.15.53:0/878772569 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0af803d650 con 0x7f0b080d3b80 2024-05-10T10:37:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.813+0000 7f0b04ff9640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0af4076300 0x7f0af40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.814+0000 7f0b04ff9640 1 -- 172.21.15.53:0/878772569 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f0af80b7ff0 con 0x7f0b080d3b80 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.814+0000 7f0b077fe640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0af4076300 0x7f0af40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.814+0000 7f0b1176c740 1 Processor -- start 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.814+0000 7f0b1176c740 1 -- start start 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.814+0000 7f0b077fe640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0af4076300 0x7f0af40787c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0b081f62c0 tx=0x7f0afc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4341ea0 0x564ba42f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4324fc0 0x564ba4324340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324880 0x564ba42f9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b1176c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564ba425e1c0 con 0x564ba4341ea0 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ba4217ca0 con 0x564ba4324880 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x564ba425a180 con 0x564ba4324fc0 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4341ea0 0x564ba42f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4341ea0 0x564ba42f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60614/0 (socket says 172.21.15.53:60614) 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 -- 172.21.15.53:0/2934541812 learned_addr learned my addr 172.21.15.53:0/2934541812 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b077fe640 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4324fc0 0x564ba4324340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324880 0x564ba42f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 -- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4324fc0 msgr2=0x564ba4324340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4324fc0 0x564ba4324340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 -- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324880 msgr2=0x564ba42f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324880 0x564ba42f9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 -- 172.21.15.53:0/2934541812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba41fbdb0 con 0x564ba4341ea0 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324880 0x564ba42f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0b07fff640 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4341ea0 0x564ba42f4830 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0af80b6230 tx=0x7f0af8080de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3930b17972e53795 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0aedffb640 1 -- 172.21.15.53:0/2934541812 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af804b030 con 0x564ba4341ea0 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0aedffb640 1 -- 172.21.15.53:0/2934541812 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af802ae50 con 0x564ba4341ea0 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.815+0000 7f0aedffb640 1 -- 172.21.15.53:0/2934541812 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80815e0 con 0x564ba4341ea0 2024-05-10T10:37:38.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 -- 172.21.15.53:0/2934541812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4341ea0 msgr2=0x564ba42f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4341ea0 0x564ba42f4830 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0af80b6230 tx=0x7f0af8080de0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 -- 172.21.15.53:0/2934541812 shutdown_connections 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324880 0x564ba42f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4324fc0 0x564ba4324340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2934541812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4341ea0 0x564ba42f4830 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 -- 172.21.15.53:0/2934541812 >> 172.21.15.53:0/2934541812 conn(0x564ba42f6aa0 msgr2=0x564ba433f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 -- 172.21.15.53:0/2934541812 shutdown_connections 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.816+0000 7f0b1176c740 1 -- 172.21.15.53:0/2934541812 wait complete. 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.817+0000 7f0b1176c740 1 -- start start 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.817+0000 7f0b1176c740 1 Processor -- start 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4324fc0 0x564ba4314a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba431fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba434f310 0x564ba4327bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564ba425e1c0 con 0x564ba4324fc0 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ba41e4be0 con 0x564ba434f310 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x564ba42f8c50 con 0x564ba4341ea0 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 10 client.0 _ll_register_callbacks cb 0x564ba4212810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4324fc0 0x564ba4314a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4324fc0 0x564ba4314a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60624/0 (socket says 172.21.15.53:60624) 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 -- 172.21.15.53:0/3902889811 learned_addr learned my addr 172.21.15.53:0/3902889811 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 -- 172.21.15.53:0/3902889811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 msgr2=0x564ba431fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba431fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 -- 172.21.15.53:0/3902889811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba434f310 msgr2=0x564ba4327bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba434f310 0x564ba4327bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 -- 172.21.15.53:0/3902889811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba403ba20 con 0x564ba4324fc0 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4324fc0 0x564ba4314a50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0af807d4d0 tx=0x7f0af8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f4f2d9c715d34b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0ae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:38.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3902889811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80b4060 con 0x564ba4324fc0 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3902889811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f0af8080140 con 0x564ba4324fc0 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af802ae50 con 0x564ba4324fc0 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3902889811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0af8080ee0 con 0x564ba4324fc0 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 10 client.6372 fetch_fsmap learned FSMap version 51 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.818+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x564ba3f2f310 con 0x564ba4324fc0 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3902889811 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af802ae50 con 0x564ba4324fc0 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3902889811 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f0af804e060 con 0x564ba4324fc0 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b1176c740 10 client.6372 fetch_fsmap finished waiting for FSMap version 51 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b1176c740 10 client.6372 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b1176c740 20 client.6372 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba4328100 0x564ba4336f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b1176c740 4 client.6372 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ba42bb570 con 0x564ba4328100 2024-05-10T10:37:38.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b077fe640 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba4328100 0x564ba4336f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0b077fe640 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba4328100 0x564ba4336f00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c6cd5abe8efc52 server_cookie=ec2131eabe2731a1 in_seq=0 out_seq=0 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.819+0000 7f0ae6ffd640 10 client.6372 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3902889811 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ba42bb570 con 0x564ba4328100 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0ae6ffd640 10 client.6372 handle_command_reply: tid=0 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 client.6372 shutdown 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4324fc0 msgr2=0x564ba4314a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4324fc0 0x564ba4314a50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0af807d4d0 tx=0x7f0af8009190 comp rx=0 tx=0).stop 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 shutdown_connections 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba4328100 0x564ba4336f00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba434f310 0x564ba4327bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba431fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3902889811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba4324fc0 0x564ba4314a50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 >> 172.21.15.53:0/3902889811 conn(0x564ba4324880 msgr2=0x564ba42f3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0ae6ffd640 0 client.6372 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:38.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 shutdown_connections 2024-05-10T10:37:38.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 1 -- 172.21.15.53:0/3902889811 wait complete. 2024-05-10T10:37:38.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.820+0000 7f0b1176c740 20 client.6372 trim_cache size 0 max 16384 2024-05-10T10:37:38.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.838+0000 7f0b1176c740 1 Processor -- start 2024-05-10T10:37:38.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b1176c740 1 -- start start 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba42f5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba42f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba433dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b1176c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08201c50 con 0x564ba434f310 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08200c90 con 0x564ba4324fc0 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0b081f7160 con 0x564ba4341ea0 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b0cf15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba433dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b0cf15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba433dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60638/0 (socket says 172.21.15.53:60638) 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b0cf15640 1 -- 172.21.15.53:0/2335441644 learned_addr learned my addr 172.21.15.53:0/2335441644 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.839+0000 7f0b077fe640 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba42f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b07fff640 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba42f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b0cf15640 1 -- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 msgr2=0x564ba42f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba42f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b0cf15640 1 -- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 msgr2=0x564ba42f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba42f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b0cf15640 1 -- 172.21.15.53:0/2335441644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba409c1b0 con 0x564ba434f310 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b07fff640 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba42f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba433dc70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0b00007290 tx=0x7f0b00007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aa76245bbb2f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0aed7fa640 1 -- 172.21.15.53:0/2335441644 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000079a0 con 0x564ba434f310 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0aed7fa640 1 -- 172.21.15.53:0/2335441644 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b00019070 con 0x564ba434f310 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0aed7fa640 1 -- 172.21.15.53:0/2335441644 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00001be0 con 0x564ba434f310 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b1176c740 1 -- 172.21.15.53:0/2335441644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 msgr2=0x564ba433dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba433dc70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0b00007290 tx=0x7f0b00007750 comp rx=0 tx=0).stop 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b1176c740 1 -- 172.21.15.53:0/2335441644 shutdown_connections 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.840+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba433dc70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.841+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba42f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.841+0000 7f0b1176c740 1 --2- 172.21.15.53:0/2335441644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba42f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.841+0000 7f0b1176c740 1 -- 172.21.15.53:0/2335441644 >> 172.21.15.53:0/2335441644 conn(0x564ba4324880 msgr2=0x564ba4338170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.841+0000 7f0b1176c740 1 -- 172.21.15.53:0/2335441644 shutdown_connections 2024-05-10T10:37:38.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.841+0000 7f0b1176c740 1 -- 172.21.15.53:0/2335441644 wait complete. 2024-05-10T10:37:38.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.841+0000 7f0b1176c740 1 -- start start 2024-05-10T10:37:38.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.841+0000 7f0b1176c740 1 Processor -- start 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba433f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba433f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b1176c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba4320780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b1176c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564ba43636a0 con 0x564ba434f310 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08200740 con 0x564ba4324fc0 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b1176c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x564ba42f9e30 con 0x564ba4341ea0 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b1176c740 10 client.0 _ll_register_callbacks cb 0x7f0af80808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba4320780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba4320780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60650/0 (socket says 172.21.15.53:60650) 2024-05-10T10:37:38.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 -- 172.21.15.53:0/3229807294 learned_addr learned my addr 172.21.15.53:0/3229807294 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba433f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b077fe640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba433f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 -- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 msgr2=0x564ba433f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba433f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 -- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 msgr2=0x564ba433f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba433f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b0cf15640 1 -- 172.21.15.53:0/3229807294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba3f08490 con 0x564ba434f310 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b077fe640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba433f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.842+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba433f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b0cf15640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba4320780 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0b00001830 tx=0x7f0b00013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840c6ae2064ade52 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0ae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:38.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3229807294 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00007a60 con 0x564ba434f310 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3229807294 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f0b00004d70 con 0x564ba434f310 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b00019070 con 0x564ba434f310 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3229807294 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0b0001d450 con 0x564ba434f310 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 10 client.6378 fetch_fsmap learned FSMap version 51 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x564ba42f46f0 con 0x564ba434f310 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3229807294 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b00019070 con 0x564ba434f310 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3229807294 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f0b0001d5f0 con 0x564ba434f310 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 10 client.6378 fetch_fsmap finished waiting for FSMap version 51 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 10 client.6378 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 20 client.6378 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba434e6b0 0x564ba4320cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 4 client.6378 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x564ba41fc910 con 0x564ba434e6b0 2024-05-10T10:37:38.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.843+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba434e6b0 0x564ba4320cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.844+0000 7f0ae6ffd640 10 client.6378 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.844+0000 7f0b07fff640 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba434e6b0 0x564ba4320cc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673fd936049d1dc3 server_cookie=daac5030e2dbb1f5 in_seq=0 out_seq=0 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.844+0000 7f0ae6ffd640 1 -- 172.21.15.53:0/3229807294 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ba41fc910 con 0x564ba434e6b0 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.844+0000 7f0ae6ffd640 10 client.6378 handle_command_reply: tid=0 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.844+0000 7f0b1176c740 1 client.6378 shutdown 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 msgr2=0x564ba4320780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba4320780 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0b00001830 tx=0x7f0b00013000 comp rx=0 tx=0).stop 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 shutdown_connections 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x564ba434e6b0 0x564ba4320cc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ba434f310 0x564ba4320780 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ba4341ea0 0x564ba433f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0ae6ffd640 0 client.6378 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 --2- 172.21.15.53:0/3229807294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ba4324fc0 0x564ba433f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 >> 172.21.15.53:0/3229807294 conn(0x564ba4324880 msgr2=0x564ba43278e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 shutdown_connections 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 1 -- 172.21.15.53:0/3229807294 wait complete. 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b1176c740 20 client.6378 trim_cache size 0 max 16384 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "4cc7ac88-34b3-4866-8c4d-2b17ae9251ef", 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:37:38.852 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:37:38.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0af4076300 msgr2=0x7f0af40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0af4076300 0x7f0af40787c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0b081f62c0 tx=0x7f0afc008040 comp rx=0 tx=0).stop 2024-05-10T10:37:38.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b080d3b80 msgr2=0x7f0b0815a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b080d3b80 0x7f0b0815a890 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097a0 tx=0x7f0af80046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 shutdown_connections 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0af4076300 0x7f0af40787c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b0815f490 0x7f0b080ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b080d5ca0 0x7f0b080cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 --2- 172.21.15.53:0/878772569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b080d3b80 0x7f0b0815a890 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.845+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 >> 172.21.15.53:0/878772569 conn(0x7f0b080d50f0 msgr2=0x7f0b080b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.846+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 shutdown_connections 2024-05-10T10:37:38.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.846+0000 7f0b0e978640 1 -- 172.21.15.53:0/878772569 wait complete. 2024-05-10T10:37:38.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:37:38.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-10T10:37:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.962+0000 7f9ae6958640 1 Processor -- start 2024-05-10T10:37:38.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.962+0000 7f9ae6958640 1 -- start start 2024-05-10T10:37:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9ae6958640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162900 0x7f9ae00d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9ae6958640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ae0162ce0 0x7f9ae00b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9ae6958640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae00d4190 con 0x7f9ae00d3950 2024-05-10T10:37:38.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9ae6958640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae00cfc40 con 0x7f9ae0162ce0 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9ae6958640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae00d3eb0 con 0x7f9ae0162900 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9ae4ef5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9ae00d3950 0x7f9ae00d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55452/0 (socket says 172.21.15.53:55452) 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9ae4ef5640 1 -- 172.21.15.53:0/4262352126 learned_addr learned my addr 172.21.15.53:0/4262352126 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4262352126 >> v2:172.21.15.135:3300/0 conn(0x7f9ae0162ce0 0x7f9ae00b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9adffff640 1 --2- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162900 0x7f9ae00d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9adeffd640 1 -- 172.21.15.53:0/4262352126 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892307756 0 0) 0x7f9ad8001490 con 0x7f9ae00d3950 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.963+0000 7f9adeffd640 1 -- 172.21.15.53:0/4262352126 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ac8003520 con 0x7f9ae00d3950 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.964+0000 7f9adeffd640 1 -- 172.21.15.53:0/4262352126 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2748805222 0 0) 0x7f9ac8003520 con 0x7f9ae00d3950 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.964+0000 7f9adf7fe640 1 -- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162900 msgr2=0x7f9ae00d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.964+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162900 0x7f9ae00d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.964+0000 7f9adf7fe640 1 -- 172.21.15.53:0/4262352126 >> v1:172.21.15.53:6789/0 conn(0x7f9ae00d3950 legacy=0x7f9ae00d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.964+0000 7f9adf7fe640 1 -- 172.21.15.53:0/4262352126 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae00cfee0 con 0x7f9ae0162ce0 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.964+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae0162ce0 0x7f9ae00b48e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097a0 tx=0x7f9ad402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c0a0b0c7ea62ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9adeffd640 1 -- 172.21.15.53:0/4262352126 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4037070 con 0x7f9ae0162ce0 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9adeffd640 1 -- 172.21.15.53:0/4262352126 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad402ab10 con 0x7f9ae0162ce0 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9adeffd640 1 -- 172.21.15.53:0/4262352126 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8001bc0 con 0x7f9ae0162ce0 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 -- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae0162ce0 msgr2=0x7f9ae00b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 --2- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae0162ce0 0x7f9ae00b48e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097a0 tx=0x7f9ad402a080 comp rx=0 tx=0).stop 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 -- 172.21.15.53:0/4262352126 shutdown_connections 2024-05-10T10:37:38.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 --2- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae0162ce0 0x7f9ae00b48e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 --2- 172.21.15.53:0/4262352126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162900 0x7f9ae00d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 -- 172.21.15.53:0/4262352126 >> 172.21.15.53:0/4262352126 conn(0x7f9ae009e430 msgr2=0x7f9ae00d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 -- 172.21.15.53:0/4262352126 shutdown_connections 2024-05-10T10:37:38.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.965+0000 7f9ae6958640 1 -- 172.21.15.53:0/4262352126 wait complete. 2024-05-10T10:37:38.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.966+0000 7f9ae6958640 1 Processor -- start 2024-05-10T10:37:38.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9ae6958640 1 -- start start 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9ae6958640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae00d3950 0x7f9ae00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9ae6958640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ae0162900 0x7f9ae015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9ae6958640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162ce0 0x7f9ae00b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9ae6958640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae00b4300 con 0x7f9ae0162900 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9ae6958640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae00b4180 con 0x7f9ae00d3950 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9ae6958640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae00d10d0 con 0x7f9ae0162ce0 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9adf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ae0162900 0x7f9ae015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9adf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ae0162900 0x7f9ae015b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60656/0 (socket says 172.21.15.53:60656) 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.967+0000 7f9adf7fe640 1 -- 172.21.15.53:0/446888989 learned_addr learned my addr 172.21.15.53:0/446888989 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adffff640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae00d3950 0x7f9ae00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9ae4ef5640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162ce0 0x7f9ae00b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adf7fe640 1 -- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162ce0 msgr2=0x7f9ae00b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162ce0 0x7f9ae00b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adf7fe640 1 -- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae00d3950 msgr2=0x7f9ae00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae00d3950 0x7f9ae00ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adf7fe640 1 -- 172.21.15.53:0/446888989 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae015c470 con 0x7f9ae0162900 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ae0162900 0x7f9ae015b670 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4035040 tx=0x7f9ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adcff9640 1 -- 172.21.15.53:0/446888989 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4041070 con 0x7f9ae0162900 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adcff9640 1 -- 172.21.15.53:0/446888989 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad403c660 con 0x7f9ae0162900 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae015c750 con 0x7f9ae0162900 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9adcff9640 1 -- 172.21.15.53:0/446888989 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4037040 con 0x7f9ae0162900 2024-05-10T10:37:38.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.968+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae015cd10 con 0x7f9ae0162900 2024-05-10T10:37:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.970+0000 7f9adcff9640 1 -- 172.21.15.53:0/446888989 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9ad4044070 con 0x7f9ae0162900 2024-05-10T10:37:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.970+0000 7f9adcff9640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9acc0762d0 0x7f9acc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.970+0000 7f9adcff9640 1 -- 172.21.15.53:0/446888989 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f9ad40b7e70 con 0x7f9ae0162900 2024-05-10T10:37:38.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.971+0000 7f9adffff640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9acc0762d0 0x7f9acc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.971+0000 7f9ae9673740 1 Processor -- start 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.971+0000 7f9adffff640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9acc0762d0 0x7f9acc078790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009b40 tx=0x7f9ad0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.971+0000 7f9ae9673740 1 -- start start 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d15af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 0x55dbe0d45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d45b40 0x55dbe0d1a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9ae9673740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0ce2690 con 0x55dbe0d63160 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0c7ff30 con 0x55dbe0d45b40 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55dbe0c38b20 con 0x55dbe0d46280 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d15af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d15af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60666/0 (socket says 172.21.15.53:60666) 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 -- 172.21.15.53:0/108420979 learned_addr learned my addr 172.21.15.53:0/108420979 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 -- 172.21.15.53:0/108420979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 msgr2=0x55dbe0d45600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 --2- 172.21.15.53:0/108420979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 0x55dbe0d45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 -- 172.21.15.53:0/108420979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d45b40 msgr2=0x55dbe0d1a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 --2- 172.21.15.53:0/108420979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d45b40 0x55dbe0d1a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 -- 172.21.15.53:0/108420979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe0c30e60 con 0x55dbe0d63160 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.972+0000 7f9adffff640 1 --2- 172.21.15.53:0/108420979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d15af0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0011f10 tx=0x7f9ad0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c1c42ddf67886 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ac5ffb640 1 -- 172.21.15.53:0/108420979 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad001b070 con 0x55dbe0d63160 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ac5ffb640 1 -- 172.21.15.53:0/108420979 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad0027070 con 0x55dbe0d63160 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ac5ffb640 1 -- 172.21.15.53:0/108420979 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0003a40 con 0x55dbe0d63160 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 -- 172.21.15.53:0/108420979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 msgr2=0x55dbe0d15af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 --2- 172.21.15.53:0/108420979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d15af0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0011f10 tx=0x7f9ad0014000 comp rx=0 tx=0).stop 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 -- 172.21.15.53:0/108420979 shutdown_connections 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 --2- 172.21.15.53:0/108420979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d45b40 0x55dbe0d1a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 --2- 172.21.15.53:0/108420979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 0x55dbe0d45600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 --2- 172.21.15.53:0/108420979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d15af0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 -- 172.21.15.53:0/108420979 >> 172.21.15.53:0/108420979 conn(0x55dbe0d17d60 msgr2=0x55dbe0d60600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 -- 172.21.15.53:0/108420979 shutdown_connections 2024-05-10T10:37:38.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.973+0000 7f9ae9673740 1 -- 172.21.15.53:0/108420979 wait complete. 2024-05-10T10:37:38.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.974+0000 7f9ae9673740 1 -- start start 2024-05-10T10:37:38.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.974+0000 7f9ae9673740 1 Processor -- start 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d46280 0x55dbe0d32f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae9673740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0c3c5e0 con 0x55dbe0d46280 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0c7bef0 con 0x55dbe0d63160 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55dbe0d19f10 con 0x55dbe0d61d20 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae9673740 10 client.0 _ll_register_callbacks cb 0x55dbe0c37320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d46280 0x55dbe0d32f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d46280 0x55dbe0d32f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60678/0 (socket says 172.21.15.53:60678) 2024-05-10T10:37:38.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 -- 172.21.15.53:0/4277721544 learned_addr learned my addr 172.21.15.53:0/4277721544 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9ae4ef5640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 -- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 msgr2=0x55dbe0d422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 -- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 msgr2=0x55dbe0d617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 -- 172.21.15.53:0/4277721544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe0a5b7d0 con 0x55dbe0d46280 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9adffff640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d46280 0x55dbe0d32f30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0024040 tx=0x7f9ad0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fbfffb22d981b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.975+0000 7f9abeffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9abeffd640 1 -- 172.21.15.53:0/4277721544 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad001b040 con 0x55dbe0d46280 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9abeffd640 1 -- 172.21.15.53:0/4277721544 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f9ad0036cc0 con 0x55dbe0d46280 2024-05-10T10:37:38.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad0027070 con 0x55dbe0d46280 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9abeffd640 1 -- 172.21.15.53:0/4277721544 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9ad0022730 con 0x55dbe0d46280 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 10 client.6390 fetch_fsmap learned FSMap version 51 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55dbe09913f0 con 0x55dbe0d46280 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9abeffd640 1 -- 172.21.15.53:0/4277721544 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad0027070 con 0x55dbe0d46280 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9abeffd640 1 -- 172.21.15.53:0/4277721544 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f9ad00228d0 con 0x55dbe0d46280 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 10 client.6390 fetch_fsmap finished waiting for FSMap version 51 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 10 client.6390 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 20 client.6390 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d477c0 0x55dbe0d47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 4 client.6390 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe0c8db40 con 0x55dbe0d477c0 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.976+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d477c0 0x55dbe0d47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9abeffd640 10 client.6390 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:38.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d477c0 0x55dbe0d47bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c005b67470d5fa server_cookie=ddc94b8d9ddcf509 in_seq=0 out_seq=0 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9abeffd640 1 -- 172.21.15.53:0/4277721544 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbe0c8db40 con 0x55dbe0d477c0 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9abeffd640 10 client.6390 handle_command_reply: tid=0 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 client.6390 shutdown 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d46280 msgr2=0x55dbe0d32f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d46280 0x55dbe0d32f30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0024040 tx=0x7f9ad0004730 comp rx=0 tx=0).stop 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 shutdown_connections 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d477c0 0x55dbe0d47bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4277721544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d46280 0x55dbe0d32f30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 >> 172.21.15.53:0/4277721544 conn(0x55dbe0d45b40 msgr2=0x55dbe0d627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9abeffd640 0 client.6390 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 shutdown_connections 2024-05-10T10:37:38.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.977+0000 7f9ae9673740 1 -- 172.21.15.53:0/4277721544 wait complete. 2024-05-10T10:37:38.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.978+0000 7f9ae9673740 20 client.6390 trim_cache size 0 max 16384 2024-05-10T10:37:38.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 Processor -- start 2024-05-10T10:37:38.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 -- start start 2024-05-10T10:37:38.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 0x55dbe0d14b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d61d20 0x55dbe0d31450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d31990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae0201ff0 con 0x55dbe0d61d20 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae0201090 con 0x55dbe0d63160 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae00b8b50 con 0x55dbe0d46280 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9adf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d61d20 0x55dbe0d31450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.996+0000 7f9adf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d61d20 0x55dbe0d31450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60694/0 (socket says 172.21.15.53:60694) 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9adf7fe640 1 -- 172.21.15.53:0/2679928693 learned_addr learned my addr 172.21.15.53:0/2679928693 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9adf7fe640 1 -- 172.21.15.53:0/2679928693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 msgr2=0x55dbe0d14b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/2679928693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 0x55dbe0d14b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9adf7fe640 1 -- 172.21.15.53:0/2679928693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 msgr2=0x55dbe0d31990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/2679928693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d31990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9adf7fe640 1 -- 172.21.15.53:0/2679928693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe09510f0 con 0x55dbe0d61d20 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/2679928693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d61d20 0x55dbe0d31450 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40b6160 tx=0x7f9ad4006a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f03e6db366b917 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9ac57fa640 1 -- 172.21.15.53:0/2679928693 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40b5070 con 0x55dbe0d61d20 2024-05-10T10:37:38.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9ac57fa640 1 -- 172.21.15.53:0/2679928693 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad402ab10 con 0x55dbe0d61d20 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.997+0000 7f9ac57fa640 1 -- 172.21.15.53:0/2679928693 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad404b040 con 0x55dbe0d61d20 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 -- 172.21.15.53:0/2679928693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d61d20 msgr2=0x55dbe0d31450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 --2- 172.21.15.53:0/2679928693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d61d20 0x55dbe0d31450 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40b6160 tx=0x7f9ad4006a50 comp rx=0 tx=0).stop 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 -- 172.21.15.53:0/2679928693 shutdown_connections 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 --2- 172.21.15.53:0/2679928693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d63160 0x55dbe0d31990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 --2- 172.21.15.53:0/2679928693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d61d20 0x55dbe0d31450 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 --2- 172.21.15.53:0/2679928693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d46280 0x55dbe0d14b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 -- 172.21.15.53:0/2679928693 >> 172.21.15.53:0/2679928693 conn(0x55dbe0d45b40 msgr2=0x55dbe0d16830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 -- 172.21.15.53:0/2679928693 shutdown_connections 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 -- 172.21.15.53:0/2679928693 wait complete. 2024-05-10T10:37:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 -- start start 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.998+0000 7f9ae9673740 1 Processor -- start 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d46280 0x55dbe0d60710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d60c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae9673740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae9673740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae00b91a0 con 0x55dbe0d63160 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0ddeb30 con 0x55dbe0d46280 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae9673740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55dbe0d47fc0 con 0x55dbe0d61d20 2024-05-10T10:37:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae9673740 10 client.0 _ll_register_callbacks cb 0x7f9ad0011de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae4ef5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae4ef5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d83040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60700/0 (socket says 172.21.15.53:60700) 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9ae4ef5640 1 -- 172.21.15.53:0/4199414545 learned_addr learned my addr 172.21.15.53:0/4199414545 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9adffff640 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d46280 0x55dbe0d60710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:38.999+0000 7f9adf7fe640 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d60c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae4ef5640 1 -- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 msgr2=0x55dbe0d60c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae4ef5640 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d60c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae4ef5640 1 -- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d46280 msgr2=0x55dbe0d60710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae4ef5640 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d46280 0x55dbe0d60710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae4ef5640 1 -- 172.21.15.53:0/4199414545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe0d48210 con 0x55dbe0d63160 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae4ef5640 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d83040 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8002230 tx=0x7f9ad8007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84861d5cf009aac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9abeffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9abeffd640 1 -- 172.21.15.53:0/4199414545 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8019020 con 0x55dbe0d63160 2024-05-10T10:37:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9abeffd640 1 -- 172.21.15.53:0/4199414545 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f9ad8011b10 con 0x55dbe0d63160 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad402ab10 con 0x55dbe0d63160 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9abeffd640 1 -- 172.21.15.53:0/4199414545 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9ad8023050 con 0x55dbe0d63160 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 10 client.6396 fetch_fsmap learned FSMap version 51 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55dbe0d49a30 con 0x55dbe0d63160 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9abeffd640 1 -- 172.21.15.53:0/4199414545 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad402ab10 con 0x55dbe0d63160 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9abeffd640 1 -- 172.21.15.53:0/4199414545 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f9ad8026020 con 0x55dbe0d63160 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 10 client.6396 fetch_fsmap finished waiting for FSMap version 51 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 10 client.6396 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 20 client.6396 populate_metadata read hostname 'smithi053' 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d83580 0x55dbe0d83990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 4 client.6396 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbe0c879b0 con 0x55dbe0d83580 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.000+0000 7f9adffff640 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d83580 0x55dbe0d83990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:37:39.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9adffff640 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d83580 0x55dbe0d83990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da47e97ae546def2 server_cookie=8c120d8eba2a31e9 in_seq=0 out_seq=0 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9abeffd640 10 client.6396 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9abeffd640 1 -- 172.21.15.53:0/4199414545 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dbe0c879b0 con 0x55dbe0d83580 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9abeffd640 10 client.6396 handle_command_reply: tid=0 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 client.6396 shutdown 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 msgr2=0x55dbe0d83040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d83040 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8002230 tx=0x7f9ad8007bd0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 shutdown_connections 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55dbe0d83580 0x55dbe0d83990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbe0d63160 0x55dbe0d83040 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55dbe0d61d20 0x55dbe0d60c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 --2- 172.21.15.53:0/4199414545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbe0d46280 0x55dbe0d60710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9abeffd640 0 client.6396 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 >> 172.21.15.53:0/4199414545 conn(0x55dbe0d45b40 msgr2=0x55dbe0d84250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 shutdown_connections 2024-05-10T10:37:39.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 1 -- 172.21.15.53:0/4199414545 wait complete. 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.001+0000 7f9ae9673740 20 client.6396 trim_cache size 0 max 16384 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: "4cc7ac88-34b3-4866-8c4d-2b17ae9251ef": { 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/subdir", 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "4cc7ac88-34b3-4866-8c4d-2b17ae9251ef", 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive" 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:39.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:37:39.009 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:37:39.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9acc0762d0 msgr2=0x7f9acc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9acc0762d0 0x7f9acc078790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009b40 tx=0x7f9ad0009190 comp rx=0 tx=0).stop 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ae0162900 msgr2=0x7f9ae015b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ae0162900 0x7f9ae015b670 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4035040 tx=0x7f9ad40046c0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 shutdown_connections 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9acc0762d0 0x7f9acc078790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ae0162ce0 0x7f9ae00b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ae0162900 0x7f9ae015b670 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 --2- 172.21.15.53:0/446888989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ae00d3950 0x7f9ae00ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 >> 172.21.15.53:0/446888989 conn(0x7f9ae009e430 msgr2=0x7f9ae01602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 shutdown_connections 2024-05-10T10:37:39.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:37:39.002+0000 7f9ae6958640 1 -- 172.21.15.53:0/446888989 wait complete. 2024-05-10T10:37:39.014 INFO:tasks.cephfs.filesystem:scrub status for tag:4cc7ac88-34b3-4866-8c4d-2b17ae9251ef - {'path': '/subdir', 'tag': '4cc7ac88-34b3-4866-8c4d-2b17ae9251ef', 'options': 'recursive'} 2024-05-10T10:37:42.343 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:37:42.342+0000 7f39f976c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-10T10:37:44.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:37:44.530 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:38:09.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:09.016 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-10T10:38:09.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.133+0000 7f2d30787640 1 Processor -- start 2024-05-10T10:38:09.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.133+0000 7f2d30787640 1 -- start start 2024-05-10T10:38:09.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d30787640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d28159340 0x7f2d280ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d30787640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2d280ceda0 0x7f2d280cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d30787640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d280cf3d0 con 0x7f2d280cbac0 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d30787640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2d280cbf30 con 0x7f2d280ceda0 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d30787640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2d28159720 con 0x7f2d28159340 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d2dd22640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2d280ceda0 0x7f2d280cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d2dd22640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2d280ceda0 0x7f2d280cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43070/0 (socket says 172.21.15.53:43070) 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d2dd22640 1 -- 172.21.15.53:0/1038750763 learned_addr learned my addr 172.21.15.53:0/1038750763 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.134+0000 7f2d2e523640 1 --2- 172.21.15.53:0/1038750763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d28159340 0x7f2d280ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.135+0000 7f2d2dd22640 1 -- 172.21.15.53:0/1038750763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d28159340 msgr2=0x7f2d280ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.135+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/1038750763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d28159340 0x7f2d280ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.135+0000 7f2d2dd22640 1 -- 172.21.15.53:0/1038750763 >> v1:172.21.15.135:6789/0 conn(0x7f2d280cbac0 legacy=0x7f2d2815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.135+0000 7f2d2dd22640 1 -- 172.21.15.53:0/1038750763 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d280cf6b0 con 0x7f2d280ceda0 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.135+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/1038750763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d280ceda0 0x7f2d280cb580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800aa10 tx=0x7f2d1802b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948b0b8e971da69a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d2d521640 1 -- 172.21.15.53:0/1038750763 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800b040 con 0x7f2d280ceda0 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d2d521640 1 -- 172.21.15.53:0/1038750763 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d18030710 con 0x7f2d280ceda0 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d2d521640 1 -- 172.21.15.53:0/1038750763 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18030ae0 con 0x7f2d280ceda0 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 -- 172.21.15.53:0/1038750763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d280ceda0 msgr2=0x7f2d280cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 --2- 172.21.15.53:0/1038750763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d280ceda0 0x7f2d280cb580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800aa10 tx=0x7f2d1802b610 comp rx=0 tx=0).stop 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 -- 172.21.15.53:0/1038750763 shutdown_connections 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 --2- 172.21.15.53:0/1038750763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d280ceda0 0x7f2d280cb580 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 --2- 172.21.15.53:0/1038750763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d28159340 0x7f2d280ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 -- 172.21.15.53:0/1038750763 >> 172.21.15.53:0/1038750763 conn(0x7f2d280d0c90 msgr2=0x7f2d280d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 -- 172.21.15.53:0/1038750763 shutdown_connections 2024-05-10T10:38:09.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.136+0000 7f2d30787640 1 -- 172.21.15.53:0/1038750763 wait complete. 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.137+0000 7f2d30787640 1 Processor -- start 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d30787640 1 -- start start 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d30787640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d280cbac0 0x7f2d280ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d30787640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d280ceda0 0x7f2d280c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d30787640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d28159340 0x7f2d280c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d30787640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280cfc70 con 0x7f2d28159340 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d30787640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2806df20 con 0x7f2d280cbac0 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d30787640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2815bfd0 con 0x7f2d280ceda0 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.138+0000 7f2d2ed24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d28159340 0x7f2d280c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d28159340 0x7f2d280c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43082/0 (socket says 172.21.15.53:43082) 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 -- 172.21.15.53:0/909072067 learned_addr learned my addr 172.21.15.53:0/909072067 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d280ceda0 0x7f2d280c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2e523640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d280cbac0 0x7f2d280ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 -- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d280ceda0 msgr2=0x7f2d280c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d280ceda0 0x7f2d280c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 -- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d280cbac0 msgr2=0x7f2d280ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d280cbac0 0x7f2d280ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 -- 172.21.15.53:0/909072067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d280c5890 con 0x7f2d28159340 2024-05-10T10:38:09.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2e523640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d280cbac0 0x7f2d280ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d28159340 0x7f2d280c6e60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000c760 tx=0x7f2d2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d1effd640 1 -- 172.21.15.53:0/909072067 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20003b70 con 0x7f2d28159340 2024-05-10T10:38:09.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d1effd640 1 -- 172.21.15.53:0/909072067 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d20003d30 con 0x7f2d28159340 2024-05-10T10:38:09.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d1effd640 1 -- 172.21.15.53:0/909072067 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20005310 con 0x7f2d28159340 2024-05-10T10:38:09.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d280c5b70 con 0x7f2d28159340 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.139+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d280c6130 con 0x7f2d28159340 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.141+0000 7f2d1effd640 1 -- 172.21.15.53:0/909072067 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2d200054b0 con 0x7f2d28159340 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.141+0000 7f2d1effd640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2d0c0762d0 0x7f2d0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.142+0000 7f2d1effd640 1 -- 172.21.15.53:0/909072067 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f2d200975c0 con 0x7f2d28159340 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.142+0000 7f2d2e523640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2d0c0762d0 0x7f2d0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.142+0000 7f2d2e523640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2d0c0762d0 0x7f2d0c078790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d24006fd0 tx=0x7f2d24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.142+0000 7f2d33577740 1 Processor -- start 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d33577740 1 -- start start 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d33577740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719812af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x557719842600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719842b40 0x557719817950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d33577740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5577197df690 con 0x557719860160 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55771977cf30 con 0x557719843280 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557719735b20 con 0x557719842b40 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d2e523640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719812af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d2e523640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719812af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43092/0 (socket says 172.21.15.53:43092) 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.143+0000 7f2d2e523640 1 -- 172.21.15.53:0/2089744582 learned_addr learned my addr 172.21.15.53:0/2089744582 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719842b40 0x557719817950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x557719842600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2e523640 1 -- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719842b40 msgr2=0x557719817950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2e523640 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719842b40 0x557719817950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2e523640 1 -- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 msgr2=0x557719842600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2e523640 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x557719842600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2e523640 1 -- 172.21.15.53:0/2089744582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771972de60 con 0x557719860160 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x557719842600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d2e523640 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719812af0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400fb30 tx=0x7f2d24012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02a75e9c69fe799 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d0bfff640 1 -- 172.21.15.53:0/2089744582 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2401b050 con 0x557719860160 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d0bfff640 1 -- 172.21.15.53:0/2089744582 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d24025070 con 0x557719860160 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.144+0000 7f2d0bfff640 1 -- 172.21.15.53:0/2089744582 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24020440 con 0x557719860160 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 -- 172.21.15.53:0/2089744582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 msgr2=0x557719812af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719812af0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400fb30 tx=0x7f2d24012000 comp rx=0 tx=0).stop 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 -- 172.21.15.53:0/2089744582 shutdown_connections 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719842b40 0x557719817950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x557719842600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 --2- 172.21.15.53:0/2089744582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719812af0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 -- 172.21.15.53:0/2089744582 >> 172.21.15.53:0/2089744582 conn(0x557719814d60 msgr2=0x55771985d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 -- 172.21.15.53:0/2089744582 shutdown_connections 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.145+0000 7f2d33577740 1 -- 172.21.15.53:0/2089744582 wait complete. 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.146+0000 7f2d33577740 1 -- start start 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.146+0000 7f2d33577740 1 Processor -- start 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719843280 0x55771985e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719860160 0x5577198456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55771982f950 0x55771982fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5577197395e0 con 0x55771982f950 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557719778ef0 con 0x557719860160 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557719816f10 con 0x557719843280 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 10 client.0 _ll_register_callbacks cb 0x557719734320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55771982f950 0x55771982fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55771982f950 0x55771982fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43102/0 (socket says 172.21.15.53:43102) 2024-05-10T10:38:09.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4124498042 learned_addr learned my addr 172.21.15.53:0/4124498042 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719860160 0x5577198456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2e523640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719843280 0x55771985e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719843280 msgr2=0x55771985e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719843280 0x55771985e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719860160 msgr2=0x5577198456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719860160 0x5577198456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4124498042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577195587d0 con 0x55771982f950 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2e523640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719843280 0x55771985e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55771982f950 0x55771982fdd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000c730 tx=0x7f2d20060150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfa373840996260 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d097fa640 1 -- 172.21.15.53:0/4124498042 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200649f0 con 0x55771982f950 2024-05-10T10:38:09.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d097fa640 1 -- 172.21.15.53:0/4124498042 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f2d20065b80 con 0x55771982f950 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.147+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d24025070 con 0x55771982f950 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d097fa640 1 -- 172.21.15.53:0/4124498042 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2d20069410 con 0x55771982f950 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 10 client.6408 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55771948e3f0 con 0x55771982f950 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d097fa640 1 -- 172.21.15.53:0/4124498042 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d24025070 con 0x55771982f950 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d097fa640 1 -- 172.21.15.53:0/4124498042 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f2d2005f8a0 con 0x55771982f950 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 10 client.6408 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 10 client.6408 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 20 client.6408 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719843c40 0x557719844050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 4 client.6408 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:38:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55771978ab40 con 0x557719843c40 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d2e523640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719843c40 0x557719844050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d2e523640 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719843c40 0x557719844050 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9eb4dcc26d6ee83 server_cookie=3470411219bfbe4a in_seq=0 out_seq=0 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.148+0000 7f2d097fa640 10 client.6408 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d097fa640 1 -- 172.21.15.53:0/4124498042 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55771978ab40 con 0x557719843c40 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d097fa640 10 client.6408 handle_command_reply: tid=0 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 client.6408 shutdown 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55771982f950 msgr2=0x55771982fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55771982f950 0x55771982fdd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000c730 tx=0x7f2d20060150 comp rx=0 tx=0).stop 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 shutdown_connections 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719843c40 0x557719844050 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55771982f950 0x55771982fdd0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719860160 0x5577198456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 --2- 172.21.15.53:0/4124498042 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719843280 0x55771985e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d097fa640 0 client.6408 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 >> 172.21.15.53:0/4124498042 conn(0x557719842b40 msgr2=0x55771985f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 shutdown_connections 2024-05-10T10:38:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.149+0000 7f2d33577740 1 -- 172.21.15.53:0/4124498042 wait complete. 2024-05-10T10:38:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.150+0000 7f2d33577740 20 client.6408 trim_cache size 0 max 16384 2024-05-10T10:38:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 Processor -- start 2024-05-10T10:38:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 -- start start 2024-05-10T10:38:09.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55771982f950 0x557719870a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x5577198165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719816b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2d282020a0 con 0x557719860160 2024-05-10T10:38:09.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28201080 con 0x557719843280 2024-05-10T10:38:09.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.168+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2d280c9b50 con 0x55771982f950 2024-05-10T10:38:09.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719816b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719816b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43116/0 (socket says 172.21.15.53:43116) 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4187462954 learned_addr learned my addr 172.21.15.53:0/4187462954 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x5577198165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55771982f950 msgr2=0x557719870a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2e523640 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55771982f950 0x557719870a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55771982f950 0x557719870a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 msgr2=0x5577198165e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x5577198165e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 -- 172.21.15.53:0/4187462954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771944e0f0 con 0x557719860160 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x5577198165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2e523640 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55771982f950 0x557719870a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719816b20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2d20060340 tx=0x7f2d2001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f996935932f13686 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d0b7fe640 1 -- 172.21.15.53:0/4187462954 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200649f0 con 0x557719860160 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d0b7fe640 1 -- 172.21.15.53:0/4187462954 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2005c8d0 con 0x557719860160 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.169+0000 7f2d0b7fe640 1 -- 172.21.15.53:0/4187462954 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20065d80 con 0x557719860160 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 -- 172.21.15.53:0/4187462954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 msgr2=0x557719816b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719816b20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2d20060340 tx=0x7f2d2001d000 comp rx=0 tx=0).stop 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 -- 172.21.15.53:0/4187462954 shutdown_connections 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719860160 0x557719816b20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557719843280 0x5577198165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 --2- 172.21.15.53:0/4187462954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55771982f950 0x557719870a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 -- 172.21.15.53:0/4187462954 >> 172.21.15.53:0/4187462954 conn(0x557719842b40 msgr2=0x5577198455b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 -- 172.21.15.53:0/4187462954 shutdown_connections 2024-05-10T10:38:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 -- 172.21.15.53:0/4187462954 wait complete. 2024-05-10T10:38:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.170+0000 7f2d33577740 1 -- start start 2024-05-10T10:38:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 1 Processor -- start 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55771982f950 0x5577198d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719843280 0x55771986f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719860160 0x55771986fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557719819b10 con 0x557719843280 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28200a70 con 0x55771982f950 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557719844ff0 con 0x557719860160 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d33577740 10 client.0 _ll_register_callbacks cb 0x7f2d2005cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.171+0000 7f2d2dd22640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719843280 0x55771986f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719843280 0x55771986f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43124/0 (socket says 172.21.15.53:43124) 2024-05-10T10:38:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 -- 172.21.15.53:0/3156096034 learned_addr learned my addr 172.21.15.53:0/3156096034 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2e523640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55771982f950 0x5577198d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2ed24640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719860160 0x55771986fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 -- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719860160 msgr2=0x55771986fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719860160 0x55771986fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 -- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55771982f950 msgr2=0x5577198d8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55771982f950 0x5577198d8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 -- 172.21.15.53:0/3156096034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557719812d00 con 0x557719843280 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2e523640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55771982f950 0x5577198d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d2dd22640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719843280 0x55771986f7f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2d18006da0 tx=0x7f2d18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8460c9ba4b36d118 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d097fa640 1 -- 172.21.15.53:0/3156096034 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18045680 con 0x557719843280 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d097fa640 1 -- 172.21.15.53:0/3156096034 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f2d18045820 con 0x557719843280 2024-05-10T10:38:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2005c8d0 con 0x557719843280 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d097fa640 1 -- 172.21.15.53:0/3156096034 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2d1804d050 con 0x557719843280 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d33577740 10 client.6414 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.172+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x557719845b20 con 0x557719843280 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d097fa640 1 -- 172.21.15.53:0/3156096034 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2005c8d0 con 0x557719843280 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d097fa640 1 -- 172.21.15.53:0/3156096034 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f2d18050020 con 0x557719843280 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 10 client.6414 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 10 client.6414 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 20 client.6414 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719870330 0x55771985ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 4 client.6414 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577197849b0 con 0x557719870330 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d2e523640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719870330 0x55771985ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d097fa640 10 client.6414 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d2e523640 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719870330 0x55771985ba10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43cf9208d20576 server_cookie=eb145d2edf95bfdb in_seq=0 out_seq=0 2024-05-10T10:38:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d097fa640 1 -- 172.21.15.53:0/3156096034 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5577197849b0 con 0x557719870330 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d097fa640 10 client.6414 handle_command_reply: tid=0 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 1 client.6414 shutdown 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719843280 msgr2=0x55771986f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719843280 0x55771986f7f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2d18006da0 tx=0x7f2d18004730 comp rx=0 tx=0).stop 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 shutdown_connections 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.173+0000 7f2d33577740 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x557719870330 0x55771985ba10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d33577740 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557719860160 0x55771986fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d33577740 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557719843280 0x55771986f7f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d33577740 1 --2- 172.21.15.53:0/3156096034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55771982f950 0x5577198d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d097fa640 0 client.6414 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 >> 172.21.15.53:0/3156096034 conn(0x557719842b40 msgr2=0x5577198165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 shutdown_connections 2024-05-10T10:38:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d33577740 1 -- 172.21.15.53:0/3156096034 wait complete. 2024-05-10T10:38:09.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d33577740 20 client.6414 trim_cache size 0 max 16384 2024-05-10T10:38:09.183 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:38:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:38:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:38:09.183 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:38:09.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2d0c0762d0 msgr2=0x7f2d0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2d0c0762d0 0x7f2d0c078790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d24006fd0 tx=0x7f2d24008040 comp rx=0 tx=0).stop 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d28159340 msgr2=0x7f2d280c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d28159340 0x7f2d280c6e60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000c760 tx=0x7f2d2000cc20 comp rx=0 tx=0).stop 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 shutdown_connections 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2d0c0762d0 0x7f2d0c078790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d28159340 0x7f2d280c6e60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2d280ceda0 0x7f2d280c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 --2- 172.21.15.53:0/909072067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d280cbac0 0x7f2d280ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 >> 172.21.15.53:0/909072067 conn(0x7f2d280d0c90 msgr2=0x7f2d280b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 shutdown_connections 2024-05-10T10:38:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.174+0000 7f2d30787640 1 -- 172.21.15.53:0/909072067 wait complete. 2024-05-10T10:38:09.187 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:38:09.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:38:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.290+0000 7f9fc8d78640 1 Processor -- start 2024-05-10T10:38:09.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.290+0000 7f9fc8d78640 1 -- start start 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fc8d78640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9fc406fa60 0x7f9fc4163730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fc8d78640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc4160240 con 0x7f9fc415f730 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fc8d78640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc4160520 con 0x7f9fc40caf50 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fc8d78640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc4163c70 con 0x7f9fc406fa60 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fbbfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9fc415f730 0x7f9fc40ca840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57900/0 (socket says 172.21.15.53:57900) 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fbbfff640 1 -- 172.21.15.53:0/2445287339 learned_addr learned my addr 172.21.15.53:0/2445287339 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6430888 0 0) 0x7f9fc4160240 con 0x7f9fc415f730 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fc259c640 1 --2- 172.21.15.53:0/2445287339 >> v2:172.21.15.135:3300/0 conn(0x7f9fc406fa60 0x7f9fc4163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.291+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fb0003520 con 0x7f9fc415f730 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2658768166 0 0) 0x7f9fb0003520 con 0x7f9fc415f730 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 >> v2:172.21.15.135:3300/0 conn(0x7f9fc406fa60 msgr2=0x7f9fc4163730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 --2- 172.21.15.53:0/2445287339 >> v2:172.21.15.135:3300/0 conn(0x7f9fc406fa60 0x7f9fc4163730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 >> v1:172.21.15.135:6789/0 conn(0x7f9fc40caf50 legacy=0x7f9fc406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc4162f30 con 0x7f9fc415f730 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f9fac001670 con 0x7f9fc415f730 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 >> v1:172.21.15.53:6789/0 conn(0x7f9fc415f730 legacy=0x7f9fc40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb00039b0 0x7f9fb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb00051c0 0x7f9fb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4163c70 con 0x7f9fb0004800 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb000c160 con 0x7f9fb00039b0 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb000c360 con 0x7f9fb00051c0 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fbbfff640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc259c640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb00039b0 0x7f9fb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc2d9d640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb00051c0 0x7f9fb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fbbfff640 1 -- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb00051c0 msgr2=0x7f9fb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fbbfff640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fb00051c0 0x7f9fb000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fbbfff640 1 -- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb00039b0 msgr2=0x7f9fb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fbbfff640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb00039b0 0x7f9fb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fbbfff640 1 -- 172.21.15.53:0/2445287339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb0003520 con 0x7f9fb0004800 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.292+0000 7f9fc2d9d640 1 -- 172.21.15.53:0/2445287339 reap_dead start 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fbbfff640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c240 tx=0x7f9fb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c516222d4f2b48 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac007d10 con 0x7f9fb0004800 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fac005ad0 con 0x7f9fb0004800 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc1d9b640 1 -- 172.21.15.53:0/2445287339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac007070 con 0x7f9fb0004800 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2445287339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb0004800 msgr2=0x7f9fb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c240 tx=0x7f9fb0005830 comp rx=0 tx=0).stop 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2445287339 shutdown_connections 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2445287339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2445287339 >> 172.21.15.53:0/2445287339 conn(0x7f9fc40d1510 msgr2=0x7f9fc40d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2445287339 shutdown_connections 2024-05-10T10:38:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.293+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2445287339 wait complete. 2024-05-10T10:38:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.295+0000 7f9fc8d78640 1 Processor -- start 2024-05-10T10:38:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.295+0000 7f9fc8d78640 1 -- start start 2024-05-10T10:38:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc8d78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb0004800 0x7f9fc4145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc8d78640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fc415ea10 0x7f9fc415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc8d78640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc415d360 0x7f9fc4145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc8d78640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4163730 con 0x7f9fc415d360 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc8d78640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc406eae0 con 0x7f9fb0004800 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc8d78640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc41638b0 con 0x7f9fc415ea10 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc415d360 0x7f9fc4145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc415d360 0x7f9fc4145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43142/0 (socket says 172.21.15.53:43142) 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 -- 172.21.15.53:0/2289337680 learned_addr learned my addr 172.21.15.53:0/2289337680 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc259c640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb0004800 0x7f9fc4145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fbbfff640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fc415ea10 0x7f9fc415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 -- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fc415ea10 msgr2=0x7f9fc415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fc415ea10 0x7f9fc415ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 -- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb0004800 msgr2=0x7f9fc4145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb0004800 0x7f9fc4145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 -- 172.21.15.53:0/2289337680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc415d970 con 0x7f9fc415d360 2024-05-10T10:38:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc259c640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb0004800 0x7f9fc4145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fbbfff640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fc415ea10 0x7f9fc415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc2d9d640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc415d360 0x7f9fc4145d70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40b7760 tx=0x7f9fbc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fbaffd640 1 -- 172.21.15.53:0/2289337680 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc003970 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fbaffd640 1 -- 172.21.15.53:0/2289337680 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc018070 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fbaffd640 1 -- 172.21.15.53:0/2289337680 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc003d70 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.296+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc415dc50 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.297+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc415e210 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.297+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc406a220 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.298+0000 7f9fbaffd640 1 -- 172.21.15.53:0/2289337680 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9fbc0050d0 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.298+0000 7f9fbaffd640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9fa4076300 0x7f9fa40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.299+0000 7f9fbaffd640 1 -- 172.21.15.53:0/2289337680 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f9fbc097450 con 0x7f9fc415d360 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.299+0000 7f9fc259c640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9fa4076300 0x7f9fa40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.299+0000 7f9fc259c640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9fa4076300 0x7f9fa40787c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009c40 tx=0x7f9fb4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.303+0000 7f9fbaffd640 1 -- 172.21.15.53:0/2289337680 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9fbc060b40 con 0x7f9fc415d360 2024-05-10T10:38:09.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.533+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fc4068610 con 0x7f9fc415d360 2024-05-10T10:38:09.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.534+0000 7f9fbaffd640 1 -- 172.21.15.53:0/2289337680 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5187 (secure 0 0 0) 0x7f9fbc060440 con 0x7f9fc415d360 2024-05-10T10:38:09.536 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:09.536 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:37:42.324997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":5}]} 2024-05-10T10:38:09.536 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 52 2024-05-10T10:38:09.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9fa4076300 msgr2=0x7f9fa40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9fa4076300 0x7f9fa40787c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009c40 tx=0x7f9fb4008040 comp rx=0 tx=0).stop 2024-05-10T10:38:09.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc415d360 msgr2=0x7f9fc4145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc415d360 0x7f9fc4145d70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40b7760 tx=0x7f9fbc00eb90 comp rx=0 tx=0).stop 2024-05-10T10:38:09.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 shutdown_connections 2024-05-10T10:38:09.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9fa4076300 0x7f9fa40787c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc415d360 0x7f9fc4145d70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fc415ea10 0x7f9fc415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 --2- 172.21.15.53:0/2289337680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb0004800 0x7f9fc4145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 >> 172.21.15.53:0/2289337680 conn(0x7f9fc40d1510 msgr2=0x7f9fc41611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 shutdown_connections 2024-05-10T10:38:09.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.536+0000 7f9fc8d78640 1 -- 172.21.15.53:0/2289337680 wait complete. 2024-05-10T10:38:09.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-10T10:38:09.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.652+0000 7f1b7efda640 1 Processor -- start 2024-05-10T10:38:09.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.653+0000 7f1b7efda640 1 -- start start 2024-05-10T10:38:09.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.653+0000 7f1b7efda640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1b7815f410 0x7f1b780d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.653+0000 7f1b7efda640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1b780d0f10 0x7f1b780b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.653+0000 7f1b7efda640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b780d2800 con 0x7f1b780b54d0 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.653+0000 7f1b7efda640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1b780d2ae0 con 0x7f1b7815f410 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.653+0000 7f1b7efda640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1b780d01d0 con 0x7f1b780d0f10 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.653+0000 7f1b7cd76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1b7815f410 0x7f1b780d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.654+0000 7f1b7cd76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1b7815f410 0x7f1b780d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43144/0 (socket says 172.21.15.53:43144) 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.654+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1586494652 learned_addr learned my addr 172.21.15.53:0/1586494652 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.654+0000 7f1b77fff640 1 --2- 172.21.15.53:0/1586494652 >> v2:172.21.15.135:3300/0 conn(0x7f1b780d0f10 0x7f1b780b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.654+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1586494652 >> v2:172.21.15.135:3300/0 conn(0x7f1b780d0f10 msgr2=0x7f1b780b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.654+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1586494652 >> v2:172.21.15.135:3300/0 conn(0x7f1b780d0f10 0x7f1b780b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.654+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1586494652 >> v1:172.21.15.135:6789/0 conn(0x7f1b780b54d0 legacy=0x7f1b780d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.654+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1586494652 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780d04b0 con 0x7f1b7815f410 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1586494652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7815f410 0x7f1b780d09d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1b780b7ee0 tx=0x7f1b6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8128a6d1a31fbd16 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b777fe640 1 -- 172.21.15.53:0/1586494652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c037070 con 0x7f1b7815f410 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b777fe640 1 -- 172.21.15.53:0/1586494652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c032440 con 0x7f1b7815f410 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b777fe640 1 -- 172.21.15.53:0/1586494652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c032810 con 0x7f1b7815f410 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 -- 172.21.15.53:0/1586494652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7815f410 msgr2=0x7f1b780d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1586494652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7815f410 0x7f1b780d09d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1b780b7ee0 tx=0x7f1b6c02a1e0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 -- 172.21.15.53:0/1586494652 shutdown_connections 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1586494652 >> v2:172.21.15.135:3300/0 conn(0x7f1b780d0f10 0x7f1b780b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1586494652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b7815f410 0x7f1b780d09d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 -- 172.21.15.53:0/1586494652 >> 172.21.15.53:0/1586494652 conn(0x7f1b7808ed30 msgr2=0x7f1b7809e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 -- 172.21.15.53:0/1586494652 shutdown_connections 2024-05-10T10:38:09.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.655+0000 7f1b7efda640 1 -- 172.21.15.53:0/1586494652 wait complete. 2024-05-10T10:38:09.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.657+0000 7f1b7efda640 1 Processor -- start 2024-05-10T10:38:09.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.657+0000 7f1b7efda640 1 -- start start 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.657+0000 7f1b7efda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b780b54d0 0x7f1b781f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.657+0000 7f1b7efda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b780d0f10 0x7f1b781f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.657+0000 7f1b7efda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b7815f410 0x7f1b781f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7efda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780cffb0 con 0x7f1b780b54d0 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7efda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780cfe30 con 0x7f1b7815f410 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7efda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780d3de0 con 0x7f1b780d0f10 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b780b54d0 0x7f1b781f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b780b54d0 0x7f1b781f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43148/0 (socket says 172.21.15.53:43148) 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1261750625 learned_addr learned my addr 172.21.15.53:0/1261750625 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b780d0f10 msgr2=0x7f1b781f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b780d0f10 0x7f1b781f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b7815f410 msgr2=0x7f1b781f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b7815f410 0x7f1b781f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 -- 172.21.15.53:0/1261750625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b781f2ba0 con 0x7f1b780b54d0 2024-05-10T10:38:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b780b54d0 0x7f1b781f2660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c02a220 tx=0x7f1b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b757fa640 1 -- 172.21.15.53:0/1261750625 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c041020 con 0x7f1b780b54d0 2024-05-10T10:38:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b757fa640 1 -- 172.21.15.53:0/1261750625 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c0306c0 con 0x7f1b780b54d0 2024-05-10T10:38:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b781f59d0 con 0x7f1b780b54d0 2024-05-10T10:38:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b757fa640 1 -- 172.21.15.53:0/1261750625 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c037040 con 0x7f1b780b54d0 2024-05-10T10:38:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.658+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b781f5f90 con 0x7f1b780b54d0 2024-05-10T10:38:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.660+0000 7f1b757fa640 1 -- 172.21.15.53:0/1261750625 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1b6c044020 con 0x7f1b780b54d0 2024-05-10T10:38:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.661+0000 7f1b757fa640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1b5c0762d0 0x7f1b5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.661+0000 7f1b757fa640 1 -- 172.21.15.53:0/1261750625 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1b6c0b7ff0 con 0x7f1b780b54d0 2024-05-10T10:38:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.661+0000 7f1b77fff640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1b5c0762d0 0x7f1b5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.661+0000 7f1b81dc9740 1 Processor -- start 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.661+0000 7f1b81dc9740 1 -- start start 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.661+0000 7f1b77fff640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1b5c0762d0 0x7f1b5c078790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b781f86f0 tx=0x7f1b64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d08f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0925b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09260b0 0x55b5d08faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b5d07e6920 con 0x55b5d09436d0 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5d085bdd0 con 0x55b5d09267f0 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b5d080c3e0 con 0x55b5d09260b0 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d08f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d08f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43162/0 (socket says 172.21.15.53:43162) 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 -- 172.21.15.53:0/177015006 learned_addr learned my addr 172.21.15.53:0/177015006 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b77fff640 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0925b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7d577640 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09260b0 0x55b5d08faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 -- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09260b0 msgr2=0x55b5d08faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09260b0 0x55b5d08faec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 -- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 msgr2=0x55b5d0925b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0925b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 -- 172.21.15.53:0/177015006 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d063d7c0 con 0x55b5d09436d0 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7d577640 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09260b0 0x55b5d08faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.662+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d08f62f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b61e0 tx=0x7f1b6c07d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6007e3494b97b8d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b627fc640 1 -- 172.21.15.53:0/177015006 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0816d0 con 0x55b5d09436d0 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b627fc640 1 -- 172.21.15.53:0/177015006 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c080fd0 con 0x55b5d09436d0 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b627fc640 1 -- 172.21.15.53:0/177015006 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c089b30 con 0x55b5d09436d0 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 -- 172.21.15.53:0/177015006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 msgr2=0x55b5d08f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d08f62f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b61e0 tx=0x7f1b6c07d750 comp rx=0 tx=0).stop 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 -- 172.21.15.53:0/177015006 shutdown_connections 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09260b0 0x55b5d08faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0925b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/177015006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d08f62f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 -- 172.21.15.53:0/177015006 >> 172.21.15.53:0/177015006 conn(0x55b5d08f8560 msgr2=0x55b5d0940b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 -- 172.21.15.53:0/177015006 shutdown_connections 2024-05-10T10:38:09.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.663+0000 7f1b81dc9740 1 -- 172.21.15.53:0/177015006 wait complete. 2024-05-10T10:38:09.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.664+0000 7f1b81dc9740 1 -- start start 2024-05-10T10:38:09.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.664+0000 7f1b81dc9740 1 Processor -- start 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0921640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d0927f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 0x55b5d0941ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b5d0864440 con 0x55b5d09436d0 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5d085f120 con 0x55b5d09267f0 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b5d086d210 con 0x55b5d0928480 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b81dc9740 10 client.0 _ll_register_callbacks cb 0x55b5d0804690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d0927f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d0927f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43164/0 (socket says 172.21.15.53:43164) 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 -- 172.21.15.53:0/4050971115 learned_addr learned my addr 172.21.15.53:0/4050971115 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b7d577640 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 0x55b5d0941ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0921640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 -- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 msgr2=0x55b5d0941ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 0x55b5d0941ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 -- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 msgr2=0x55b5d0921640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0921640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 -- 172.21.15.53:0/4050971115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d069df50 con 0x55b5d09436d0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b77fff640 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d0927f40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400deb0 tx=0x7f1b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac64a5f6d6081c6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.665+0000 7f1b577fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b577fe640 1 -- 172.21.15.53:0/4050971115 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64024020 con 0x55b5d09436d0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b577fe640 1 -- 172.21.15.53:0/4050971115 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1b6400f040 con 0x55b5d09436d0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c080fd0 con 0x55b5d09436d0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b577fe640 1 -- 172.21.15.53:0/4050971115 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1b6402e050 con 0x55b5d09436d0 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 10 client.6432 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55b5d05310b0 con 0x55b5d09436d0 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b577fe640 1 -- 172.21.15.53:0/4050971115 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c080fd0 con 0x55b5d09436d0 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b577fe640 1 -- 172.21.15.53:0/4050971115 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f1b64031020 con 0x55b5d09436d0 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 10 client.6432 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 10 client.6432 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 20 client.6432 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d0926fa0 0x55b5d095df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 4 client.6432 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5d086da20 con 0x55b5d0926fa0 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.666+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d0926fa0 0x55b5d095df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b577fe640 10 client.6432 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d0926fa0 0x55b5d095df90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa19624428a1ef53 server_cookie=bcf76cc7bc3298d6 in_seq=0 out_seq=0 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b577fe640 1 -- 172.21.15.53:0/4050971115 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5d086da20 con 0x55b5d0926fa0 2024-05-10T10:38:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b577fe640 10 client.6432 handle_command_reply: tid=0 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 client.6432 shutdown 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 msgr2=0x55b5d0927f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d0927f40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400deb0 tx=0x7f1b640046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 shutdown_connections 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d0926fa0 0x55b5d095df90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 0x55b5d0941ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09436d0 0x55b5d0927f40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b577fe640 0 client.6432 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/4050971115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d0921640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 >> 172.21.15.53:0/4050971115 conn(0x55b5d09260b0 msgr2=0x55b5d08f5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 shutdown_connections 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.667+0000 7f1b81dc9740 1 -- 172.21.15.53:0/4050971115 wait complete. 2024-05-10T10:38:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.668+0000 7f1b81dc9740 20 client.6432 trim_cache size 0 max 16384 2024-05-10T10:38:09.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.681+0000 7f1b81dc9740 1 Processor -- start 2024-05-10T10:38:09.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.681+0000 7f1b81dc9740 1 -- start start 2024-05-10T10:38:09.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09267f0 0x55b5d093f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 0x55b5d0917a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09436d0 0x55b5d0917f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78201030 con 0x55b5d09267f0 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780ca820 con 0x55b5d09436d0 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1b782009e0 con 0x55b5d0928480 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09267f0 0x55b5d093f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09267f0 0x55b5d093f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43178/0 (socket says 172.21.15.53:43178) 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 -- 172.21.15.53:0/3141044547 learned_addr learned my addr 172.21.15.53:0/3141044547 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 -- 172.21.15.53:0/3141044547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 msgr2=0x55b5d0917a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/3141044547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 0x55b5d0917a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 -- 172.21.15.53:0/3141044547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09436d0 msgr2=0x55b5d0917f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/3141044547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09436d0 0x55b5d0917f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.682+0000 7f1b7cd76640 1 -- 172.21.15.53:0/3141044547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d08f0d40 con 0x55b5d09267f0 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/3141044547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09267f0 0x55b5d093f390 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c07d5c0 tx=0x7f1b6c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5474c1ab28dbd9a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b61ffb640 1 -- 172.21.15.53:0/3141044547 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c089da0 con 0x55b5d09267f0 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b61ffb640 1 -- 172.21.15.53:0/3141044547 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c089910 con 0x55b5d09267f0 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b61ffb640 1 -- 172.21.15.53:0/3141044547 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c080230 con 0x55b5d09267f0 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 -- 172.21.15.53:0/3141044547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09267f0 msgr2=0x55b5d093f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/3141044547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09267f0 0x55b5d093f390 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c07d5c0 tx=0x7f1b6c048000 comp rx=0 tx=0).stop 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 -- 172.21.15.53:0/3141044547 shutdown_connections 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/3141044547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09436d0 0x55b5d0917f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/3141044547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d0928480 0x55b5d0917a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/3141044547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d09267f0 0x55b5d093f390 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 -- 172.21.15.53:0/3141044547 >> 172.21.15.53:0/3141044547 conn(0x55b5d09260b0 msgr2=0x55b5d08fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 -- 172.21.15.53:0/3141044547 shutdown_connections 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.683+0000 7f1b81dc9740 1 -- 172.21.15.53:0/3141044547 wait complete. 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.684+0000 7f1b81dc9740 1 -- start start 2024-05-10T10:38:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.684+0000 7f1b81dc9740 1 Processor -- start 2024-05-10T10:38:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d09bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d0928480 0x55b5d091f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b81dc9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09436d0 0x55b5d093f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b5d0926e20 con 0x55b5d0928480 2024-05-10T10:38:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5d0815a20 con 0x55b5d09267f0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b81dc9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b5d0927410 con 0x55b5d09436d0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b81dc9740 10 client.0 _ll_register_callbacks cb 0x7f1b640091a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d0928480 0x55b5d091f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d0928480 0x55b5d091f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43182/0 (socket says 172.21.15.53:43182) 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 -- 172.21.15.53:0/1450947190 learned_addr learned my addr 172.21.15.53:0/1450947190 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d09bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b7d577640 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09436d0 0x55b5d093f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 -- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09436d0 msgr2=0x55b5d093f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09436d0 0x55b5d093f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 -- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 msgr2=0x55b5d09bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d09bc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 -- 172.21.15.53:0/1450947190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d08fd720 con 0x55b5d0928480 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b77fff640 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d0928480 0x55b5d091f050 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1b64029620 tx=0x7f1b64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219aa9c6db964ca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.685+0000 7f1b577fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b577fe640 1 -- 172.21.15.53:0/1450947190 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64024040 con 0x55b5d0928480 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b577fe640 1 -- 172.21.15.53:0/1450947190 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1b64020600 con 0x55b5d0928480 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c089910 con 0x55b5d0928480 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b577fe640 1 -- 172.21.15.53:0/1450947190 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1b6403c050 con 0x55b5d0928480 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 10 client.6438 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55b5d091b470 con 0x55b5d0928480 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b577fe640 1 -- 172.21.15.53:0/1450947190 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c089910 con 0x55b5d0928480 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b577fe640 1 -- 172.21.15.53:0/1450947190 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f1b6403f020 con 0x55b5d0928480 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 10 client.6438 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 10 client.6438 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 20 client.6438 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d08fb060 0x55b5d091fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 4 client.6438 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b5d0867890 con 0x55b5d08fb060 2024-05-10T10:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d08fb060 0x55b5d091fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b7cd76640 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d08fb060 0x55b5d091fde0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbdd03b7a6f7384 server_cookie=7c23eb02c3a63f73 in_seq=0 out_seq=0 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.686+0000 7f1b577fe640 10 client.6438 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b577fe640 1 -- 172.21.15.53:0/1450947190 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+236 (crc 0 0 0) 0x55b5d0867890 con 0x55b5d08fb060 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b577fe640 10 client.6438 handle_command_reply: tid=0 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 client.6438 shutdown 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d0928480 msgr2=0x55b5d091f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d0928480 0x55b5d091f050 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1b64029620 tx=0x7f1b64012000 comp rx=0 tx=0).stop 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 shutdown_connections 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55b5d08fb060 0x55b5d091fde0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b5d09436d0 0x55b5d093f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5d0928480 0x55b5d091f050 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b577fe640 0 client.6438 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 --2- 172.21.15.53:0/1450947190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5d09267f0 0x55b5d09bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 >> 172.21.15.53:0/1450947190 conn(0x55b5d09260b0 msgr2=0x55b5d08fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 shutdown_connections 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 1 -- 172.21.15.53:0/1450947190 wait complete. 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.687+0000 7f1b81dc9740 20 client.6438 trim_cache size 0 max 16384 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stdout: "damage_type": "dentry", 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 460212277, 2024-05-10T10:38:09.695 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627776, 2024-05-10T10:38:09.696 INFO:teuthology.orchestra.run.smithi053.stdout: "frag": "*", 2024-05-10T10:38:09.696 INFO:teuthology.orchestra.run.smithi053.stdout: "dname": "file_to_be_damaged", 2024-05-10T10:38:09.696 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_id": "head", 2024-05-10T10:38:09.696 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/subdir/file_to_be_damaged" 2024-05-10T10:38:09.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:09.696 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1b5c0762d0 msgr2=0x7f1b5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1b5c0762d0 0x7f1b5c078790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b781f86f0 tx=0x7f1b64008040 comp rx=0 tx=0).stop 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b780b54d0 msgr2=0x7f1b781f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b780b54d0 0x7f1b781f2660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c02a220 tx=0x7f1b6c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 shutdown_connections 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1b5c0762d0 0x7f1b5c078790 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b7815f410 0x7f1b781f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b780d0f10 0x7f1b781f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 --2- 172.21.15.53:0/1261750625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b780b54d0 0x7f1b781f2660 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 >> 172.21.15.53:0/1261750625 conn(0x7f1b7808ed30 msgr2=0x7f1b780d4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 shutdown_connections 2024-05-10T10:38:09.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.688+0000 7f1b7efda640 1 -- 172.21.15.53:0/1261750625 wait complete. 2024-05-10T10:38:09.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T10:38:09.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.801+0000 7fddd7d58640 1 Processor -- start 2024-05-10T10:38:09.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.802+0000 7fddd7d58640 1 -- start start 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.802+0000 7fddd7d58640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fddd015f790 0x7fddd00cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.802+0000 7fddd7d58640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd00d30b0 con 0x7fddd00d5d30 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.802+0000 7fddd7d58640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd00d3390 con 0x7fddd015f3b0 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.802+0000 7fddd7d58640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fddd00c8c80 con 0x7fddd015f790 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.802+0000 7fddd62f5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fddd00d5d30 0x7fddd00d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57914/0 (socket says 172.21.15.53:57914) 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.802+0000 7fddd62f5640 1 -- 172.21.15.53:0/241264817 learned_addr learned my addr 172.21.15.53:0/241264817 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd52f3640 1 --2- 172.21.15.53:0/241264817 >> v2:172.21.15.135:3300/0 conn(0x7fddd015f790 0x7fddd00cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499339622 0 0) 0x7fddd00d30b0 con 0x7fddd00d5d30 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddb8003520 con 0x7fddd00d5d30 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118176761 0 0) 0x7fddd00d3390 con 0x7fddd015f3b0 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddd00d30b0 con 0x7fddd015f3b0 2024-05-10T10:38:09.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd52f3640 1 -- 172.21.15.53:0/241264817 >> v1:172.21.15.135:6789/0 conn(0x7fddd015f3b0 legacy=0x7fddd00d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd52f3640 1 -- 172.21.15.53:0/241264817 >> v1:172.21.15.53:6789/0 conn(0x7fddd00d5d30 legacy=0x7fddd00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1678371358 0 0) 0x7fddb8003520 con 0x7fddd00d5d30 2024-05-10T10:38:09.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.803+0000 7fddd52f3640 1 -- 172.21.15.53:0/241264817 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd00c8f60 con 0x7fddd015f790 2024-05-10T10:38:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.804+0000 7fddd52f3640 1 --2- 172.21.15.53:0/241264817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd015f790 0x7fddd00cfcd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fddc40097a0 tx=0x7fddc402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bd2eb561966bfb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.804+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4037070 con 0x7fddd015f790 2024-05-10T10:38:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.804+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc402ac50 con 0x7fddd015f790 2024-05-10T10:38:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.804+0000 7fddd4af2640 1 -- 172.21.15.53:0/241264817 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00019f0 con 0x7fddd015f790 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.805+0000 7fddd7d58640 1 -- 172.21.15.53:0/241264817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd015f790 msgr2=0x7fddd00cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.805+0000 7fddd7d58640 1 --2- 172.21.15.53:0/241264817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd015f790 0x7fddd00cfcd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fddc40097a0 tx=0x7fddc402a1c0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.805+0000 7fddd7d58640 1 -- 172.21.15.53:0/241264817 shutdown_connections 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.805+0000 7fddd7d58640 1 --2- 172.21.15.53:0/241264817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd015f790 0x7fddd00cfcd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.805+0000 7fddd7d58640 1 -- 172.21.15.53:0/241264817 >> 172.21.15.53:0/241264817 conn(0x7fddd00d5130 msgr2=0x7fddd00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.805+0000 7fddd7d58640 1 -- 172.21.15.53:0/241264817 shutdown_connections 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.805+0000 7fddd7d58640 1 -- 172.21.15.53:0/241264817 wait complete. 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.806+0000 7fddd7d58640 1 Processor -- start 2024-05-10T10:38:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.806+0000 7fddd7d58640 1 -- start start 2024-05-10T10:38:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd7d58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd00d5d30 0x7fddd01f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd7d58640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd015f3b0 0x7fddd00b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd7d58640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddd00b55c0 0x7fddd015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd7d58640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddd00d3f80 con 0x7fddd015f3b0 2024-05-10T10:38:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd7d58640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fddd00d3e00 con 0x7fddd00d5d30 2024-05-10T10:38:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd7d58640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fddd00ca770 con 0x7fddd00b55c0 2024-05-10T10:38:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd015f3b0 0x7fddd00b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd015f3b0 0x7fddd00b5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43190/0 (socket says 172.21.15.53:43190) 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 -- 172.21.15.53:0/2636159676 learned_addr learned my addr 172.21.15.53:0/2636159676 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd62f5640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddd00b55c0 0x7fddd015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd5af4640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd00d5d30 0x7fddd01f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 -- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddd00b55c0 msgr2=0x7fddd015c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddd00b55c0 0x7fddd015c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 -- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd00d5d30 msgr2=0x7fddd01f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd00d5d30 0x7fddd01f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 -- 172.21.15.53:0/2636159676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd00cd950 con 0x7fddd015f3b0 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd5af4640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd00d5d30 0x7fddd01f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddd52f3640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd015f3b0 0x7fddd00b5080 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddc4035040 tx=0x7fddc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.807+0000 7fddce7fc640 1 -- 172.21.15.53:0/2636159676 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4041020 con 0x7fddd015f3b0 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.808+0000 7fddce7fc640 1 -- 172.21.15.53:0/2636159676 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc403d420 con 0x7fddd015f3b0 2024-05-10T10:38:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.808+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd01f4d80 con 0x7fddd015f3b0 2024-05-10T10:38:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.808+0000 7fddce7fc640 1 -- 172.21.15.53:0/2636159676 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4037040 con 0x7fddd015f3b0 2024-05-10T10:38:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.808+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd01f5340 con 0x7fddd015f3b0 2024-05-10T10:38:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.809+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd006a220 con 0x7fddd015f3b0 2024-05-10T10:38:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.809+0000 7fddce7fc640 1 -- 172.21.15.53:0/2636159676 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fddc4044020 con 0x7fddd015f3b0 2024-05-10T10:38:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.810+0000 7fddce7fc640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddbc0762d0 0x7fddbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.810+0000 7fddce7fc640 1 -- 172.21.15.53:0/2636159676 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fddc40b77b0 con 0x7fddd015f3b0 2024-05-10T10:38:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.810+0000 7fddd5af4640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddbc0762d0 0x7fddbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.810+0000 7fddd5af4640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddbc0762d0 0x7fddbc078790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fddc80011e0 tx=0x7fddc8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:09.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:09.814+0000 7fddce7fc640 1 -- 172.21.15.53:0/2636159676 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fddc404b070 con 0x7fddd015f3b0 2024-05-10T10:38:10.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.044+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fddd006cf70 con 0x7fddd015f3b0 2024-05-10T10:38:10.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.044+0000 7fddce7fc640 1 -- 172.21.15.53:0/2636159676 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fddc4080fc0 con 0x7fddd015f3b0 2024-05-10T10:38:10.046 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:10.046 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-10T10:38:10.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddbc0762d0 msgr2=0x7fddbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddbc0762d0 0x7fddbc078790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fddc80011e0 tx=0x7fddc8009040 comp rx=0 tx=0).stop 2024-05-10T10:38:10.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd015f3b0 msgr2=0x7fddd00b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd015f3b0 0x7fddd00b5080 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddc4035040 tx=0x7fddc40046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 shutdown_connections 2024-05-10T10:38:10.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddbc0762d0 0x7fddbc078790 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddd00b55c0 0x7fddd015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd015f3b0 0x7fddd00b5080 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 --2- 172.21.15.53:0/2636159676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddd00d5d30 0x7fddd01f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.046+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 >> 172.21.15.53:0/2636159676 conn(0x7fddd00d5130 msgr2=0x7fddd00cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.047+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 shutdown_connections 2024-05-10T10:38:10.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.047+0000 7fddd7d58640 1 -- 172.21.15.53:0/2636159676 wait complete. 2024-05-10T10:38:10.059 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T10:38:10.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:10.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-05-10T10:38:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.165+0000 7febf3472640 1 Processor -- start 2024-05-10T10:38:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.165+0000 7febf3472640 1 -- start start 2024-05-10T10:38:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf3472640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec15d010 0x7febec0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf3472640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7febec0d04e0 0x7febec0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf3472640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febec0add50 con 0x7febec0d1370 2024-05-10T10:38:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf3472640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7febec0d17e0 con 0x7febec0d04e0 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf3472640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7febec0ae030 con 0x7febec15d010 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf0a0d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7febec0d04e0 0x7febec0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf1a0f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7febec0d1370 0x7febec06dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57920/0 (socket says 172.21.15.53:57920) 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf1a0f640 1 -- 172.21.15.53:0/2926744521 learned_addr learned my addr 172.21.15.53:0/2926744521 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.166+0000 7febf120e640 1 --2- 172.21.15.53:0/2926744521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec15d010 0x7febec0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febf0a0d640 1 -- 172.21.15.53:0/2926744521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec15d010 msgr2=0x7febec0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febf0a0d640 1 --2- 172.21.15.53:0/2926744521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec15d010 0x7febec0cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febf0a0d640 1 -- 172.21.15.53:0/2926744521 >> v1:172.21.15.53:6789/0 conn(0x7febec0d1370 legacy=0x7febec06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febf0a0d640 1 -- 172.21.15.53:0/2926744521 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec0b5af0 con 0x7febec0d04e0 2024-05-10T10:38:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febe3fff640 1 -- 172.21.15.53:0/2926744521 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014561672 0 0) 0x7febec0add50 con 0x7febec0d1370 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febf0a0d640 1 --2- 172.21.15.53:0/2926744521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec0d04e0 0x7febec0d0e30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7febdc0097a0 tx=0x7febdc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1398e68ae8566af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febe3fff640 1 -- 172.21.15.53:0/2926744521 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc037070 con 0x7febec0d04e0 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.167+0000 7febe3fff640 1 -- 172.21.15.53:0/2926744521 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febdc032440 con 0x7febec0d04e0 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febe3fff640 1 -- 172.21.15.53:0/2926744521 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc032810 con 0x7febec0d04e0 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 -- 172.21.15.53:0/2926744521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec0d04e0 msgr2=0x7febec0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 --2- 172.21.15.53:0/2926744521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec0d04e0 0x7febec0d0e30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7febdc0097a0 tx=0x7febdc02a180 comp rx=0 tx=0).stop 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 -- 172.21.15.53:0/2926744521 shutdown_connections 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 --2- 172.21.15.53:0/2926744521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec0d04e0 0x7febec0d0e30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 --2- 172.21.15.53:0/2926744521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec15d010 0x7febec0cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 -- 172.21.15.53:0/2926744521 >> 172.21.15.53:0/2926744521 conn(0x7febec06c4a0 msgr2=0x7febec06c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 -- 172.21.15.53:0/2926744521 shutdown_connections 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.168+0000 7febf3472640 1 -- 172.21.15.53:0/2926744521 wait complete. 2024-05-10T10:38:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.169+0000 7febf3472640 1 Processor -- start 2024-05-10T10:38:10.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.169+0000 7febf3472640 1 -- start start 2024-05-10T10:38:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf3472640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec0d04e0 0x7febec1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf3472640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec0d1370 0x7febec1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf3472640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec15d010 0x7febec14b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf3472640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7febec0b0580 con 0x7febec15d010 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf3472640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febec0b0400 con 0x7febec0d04e0 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf3472640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7febec0b4c10 con 0x7febec0d1370 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec15d010 0x7febec14b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec15d010 0x7febec14b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43210/0 (socket says 172.21.15.53:43210) 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 -- 172.21.15.53:0/2564693938 learned_addr learned my addr 172.21.15.53:0/2564693938 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 -- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec0d1370 msgr2=0x7febec1f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec0d1370 0x7febec1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 -- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec0d04e0 msgr2=0x7febec1498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec0d04e0 0x7febec1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.170+0000 7febf1a0f640 1 -- 172.21.15.53:0/2564693938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec09d0a0 con 0x7febec15d010 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.171+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec15d010 0x7febec14b250 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7febe400caf0 tx=0x7febe400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.171+0000 7febe1ffb640 1 -- 172.21.15.53:0/2564693938 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe400aca0 con 0x7febec15d010 2024-05-10T10:38:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.171+0000 7febe1ffb640 1 -- 172.21.15.53:0/2564693938 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe400ae40 con 0x7febec15d010 2024-05-10T10:38:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.171+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec1f8df0 con 0x7febec15d010 2024-05-10T10:38:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.171+0000 7febe1ffb640 1 -- 172.21.15.53:0/2564693938 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4012620 con 0x7febec15d010 2024-05-10T10:38:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.171+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec149df0 con 0x7febec15d010 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.173+0000 7febe1ffb640 1 -- 172.21.15.53:0/2564693938 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7febe4012880 con 0x7febec15d010 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.173+0000 7febe1ffb640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7febd00762d0 0x7febd0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.173+0000 7febe1ffb640 1 -- 172.21.15.53:0/2564693938 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7febe4097e60 con 0x7febec15d010 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.173+0000 7febf120e640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7febd00762d0 0x7febd0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.174+0000 7febf618d740 1 Processor -- start 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.174+0000 7febf618d740 1 -- start start 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.174+0000 7febf120e640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7febd00762d0 0x7febd0078790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7febe8006fd0 tx=0x7febe8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b8589ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858d6fa0 0x560b8589acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf618d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858c5390 0x560b858c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf618d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560b857feef0 con 0x560b858c5390 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560b857b8010 con 0x560b8589a8d0 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560b857faeb0 con 0x560b858d6fa0 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858c5390 0x560b858c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858c5390 0x560b858c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43212/0 (socket says 172.21.15.53:43212) 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 -- 172.21.15.53:0/3121609064 learned_addr learned my addr 172.21.15.53:0/3121609064 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf0a0d640 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858d6fa0 0x560b8589acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf120e640 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b8589ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 -- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858d6fa0 msgr2=0x560b8589acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858d6fa0 0x560b8589acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 -- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 msgr2=0x560b8589ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b8589ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 -- 172.21.15.53:0/3121609064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b855d8a50 con 0x560b858c5390 2024-05-10T10:38:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febf1a0f640 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858c5390 0x560b858c5810 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7febe405d4d0 tx=0x7febe4060650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af16f76f5d5d357 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febd6ffd640 1 -- 172.21.15.53:0/3121609064 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4060ee0 con 0x560b858c5390 2024-05-10T10:38:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.175+0000 7febd6ffd640 1 -- 172.21.15.53:0/3121609064 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4065290 con 0x560b858c5390 2024-05-10T10:38:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febd6ffd640 1 -- 172.21.15.53:0/3121609064 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4061340 con 0x560b858c5390 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 -- 172.21.15.53:0/3121609064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858c5390 msgr2=0x560b858c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858c5390 0x560b858c5810 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7febe405d4d0 tx=0x7febe4060650 comp rx=0 tx=0).stop 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 -- 172.21.15.53:0/3121609064 shutdown_connections 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858c5390 0x560b858c5810 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858d6fa0 0x560b8589acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 --2- 172.21.15.53:0/3121609064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b8589ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 -- 172.21.15.53:0/3121609064 >> 172.21.15.53:0/3121609064 conn(0x560b858623e0 msgr2=0x560b858e0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 -- 172.21.15.53:0/3121609064 shutdown_connections 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.176+0000 7febf618d740 1 -- 172.21.15.53:0/3121609064 wait complete. 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.177+0000 7febf618d740 1 -- start start 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.177+0000 7febf618d740 1 Processor -- start 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf618d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf618d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560b857bbb40 con 0x560b858d6fa0 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560b858c2ee0 con 0x560b8589a8d0 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560b858f1300 con 0x560b858c5390 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf618d740 10 client.0 _ll_register_callbacks cb 0x560b857a8b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858dd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43224/0 (socket says 172.21.15.53:43224) 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 -- 172.21.15.53:0/2275141792 learned_addr learned my addr 172.21.15.53:0/2275141792 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf0a0d640 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf120e640 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 -- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 msgr2=0x560b858dcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858dcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 -- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 msgr2=0x560b858c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 -- 172.21.15.53:0/2275141792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b854d03e0 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858dd2d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7febe4096180 tx=0x7febe4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c0f336ce3ef329 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febcbfff640 1 -- 172.21.15.53:0/2275141792 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe407e020 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.178+0000 7febcbfff640 1 -- 172.21.15.53:0/2275141792 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7febe40823a0 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe4065290 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febcbfff640 1 -- 172.21.15.53:0/2275141792 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7febe4060ee0 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 10 client.6459 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x560b856391e0 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febcbfff640 1 -- 172.21.15.53:0/2275141792 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4065290 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febcbfff640 1 -- 172.21.15.53:0/2275141792 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7febe4069820 con 0x560b858d6fa0 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 10 client.6459 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 10 client.6459 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 20 client.6459 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b858f0200 0x560b858f0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 4 client.6459 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b8580cb00 con 0x560b858f0200 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf120e640 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b858f0200 0x560b858f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febcbfff640 10 client.6459 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.179+0000 7febf120e640 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b858f0200 0x560b858f0610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac7c32ab69e2b06 server_cookie=8b157ce6b3226856 in_seq=0 out_seq=0 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febcbfff640 1 -- 172.21.15.53:0/2275141792 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560b8580cb00 con 0x560b858f0200 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febcbfff640 10 client.6459 handle_command_reply: tid=0 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 client.6459 shutdown 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 msgr2=0x560b858dd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858dd2d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7febe4096180 tx=0x7febe4001250 comp rx=0 tx=0).stop 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 shutdown_connections 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b858f0200 0x560b858f0610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858dd2d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febcbfff640 0 client.6459 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 --2- 172.21.15.53:0/2275141792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 >> 172.21.15.53:0/2275141792 conn(0x560b858623e0 msgr2=0x560b85894650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 shutdown_connections 2024-05-10T10:38:10.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 1 -- 172.21.15.53:0/2275141792 wait complete. 2024-05-10T10:38:10.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.180+0000 7febf618d740 20 client.6459 trim_cache size 0 max 16384 2024-05-10T10:38:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 Processor -- start 2024-05-10T10:38:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 -- start start 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858c0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7febec201d80 con 0x560b858d6fa0 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febec200d80 con 0x560b8589a8d0 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7febec09e3d0 con 0x560b858c5390 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.199+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858d5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43236/0 (socket says 172.21.15.53:43236) 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 -- 172.21.15.53:0/2684168466 learned_addr learned my addr 172.21.15.53:0/2684168466 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 -- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 msgr2=0x560b858c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf0a0d640 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf120e640 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858c0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 -- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 msgr2=0x560b858bfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858bfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 -- 172.21.15.53:0/2684168466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b854a94a0 con 0x560b858d6fa0 2024-05-10T10:38:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febf1a0f640 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858d5d20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7febe405cf60 tx=0x7febe4095000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c30240a0865020 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febd67fc640 1 -- 172.21.15.53:0/2684168466 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe407e040 con 0x560b858d6fa0 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febd67fc640 1 -- 172.21.15.53:0/2684168466 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4065290 con 0x560b858d6fa0 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.200+0000 7febd67fc640 1 -- 172.21.15.53:0/2684168466 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4069820 con 0x560b858d6fa0 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 -- 172.21.15.53:0/2684168466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 msgr2=0x560b858d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858d5d20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7febe405cf60 tx=0x7febe4095000 comp rx=0 tx=0).stop 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 -- 172.21.15.53:0/2684168466 shutdown_connections 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858d5d20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b858c5390 0x560b858c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 --2- 172.21.15.53:0/2684168466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b8589a8d0 0x560b858bfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 -- 172.21.15.53:0/2684168466 >> 172.21.15.53:0/2684168466 conn(0x560b858623e0 msgr2=0x560b85890b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 -- 172.21.15.53:0/2684168466 shutdown_connections 2024-05-10T10:38:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 -- 172.21.15.53:0/2684168466 wait complete. 2024-05-10T10:38:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 -- start start 2024-05-10T10:38:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.201+0000 7febf618d740 1 Processor -- start 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b8589a8d0 0x560b8595a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf618d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b858c5390 0x560b858be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf618d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf618d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560b858e3640 con 0x560b858d6fa0 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febec200730 con 0x560b858c5390 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf618d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560b858dbbf0 con 0x560b8589a8d0 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf618d740 10 client.0 _ll_register_callbacks cb 0x7febe800b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf1a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858bef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43244/0 (socket says 172.21.15.53:43244) 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.202+0000 7febf1a0f640 1 -- 172.21.15.53:0/833591969 learned_addr learned my addr 172.21.15.53:0/833591969 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf0a0d640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b858c5390 0x560b858be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf120e640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b8589a8d0 0x560b8595a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf1a0f640 1 -- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b8589a8d0 msgr2=0x560b8595a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf1a0f640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b8589a8d0 0x560b8595a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf1a0f640 1 -- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b858c5390 msgr2=0x560b858be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf1a0f640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b858c5390 0x560b858be9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf1a0f640 1 -- 172.21.15.53:0/833591969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b858d9430 con 0x560b858d6fa0 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf0a0d640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b858c5390 0x560b858be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf1a0f640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858bef00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7febe400a560 tx=0x7febe405d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46f0da67c19f27e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febcbfff640 1 -- 172.21.15.53:0/833591969 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4060340 con 0x560b858d6fa0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febcbfff640 1 -- 172.21.15.53:0/833591969 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7febe401cc90 con 0x560b858d6fa0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe4065290 con 0x560b858d6fa0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febcbfff640 1 -- 172.21.15.53:0/833591969 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7febe4076020 con 0x560b858d6fa0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf618d740 10 client.6465 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x560b858d9430 con 0x560b858d6fa0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febcbfff640 1 -- 172.21.15.53:0/833591969 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4065290 con 0x560b858d6fa0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.203+0000 7febcbfff640 1 -- 172.21.15.53:0/833591969 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7febe405c850 con 0x560b858d6fa0 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf618d740 10 client.6465 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf618d740 10 client.6465 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf618d740 20 client.6465 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf618d740 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b8595d580 0x560b8595d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf618d740 4 client.6465 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-05-10T10:38:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x560b85806970 con 0x560b8595d580 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf120e640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b8595d580 0x560b8595d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febf120e640 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b8595d580 0x560b8595d990 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782acae32232d3c5 server_cookie=c981cc8d2b34a6f1 in_seq=0 out_seq=0 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.204+0000 7febcbfff640 10 client.6465 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febcbfff640 1 -- 172.21.15.53:0/833591969 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b85806970 con 0x560b8595d580 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febcbfff640 10 client.6465 handle_command_reply: tid=0 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 client.6465 shutdown 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 msgr2=0x560b858bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858bef00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7febe400a560 tx=0x7febe405d5b0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 shutdown_connections 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x560b8595d580 0x560b8595d990 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b858d6fa0 0x560b858bef00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560b858c5390 0x560b858be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febcbfff640 0 client.6465 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 --2- 172.21.15.53:0/833591969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560b8589a8d0 0x560b8595a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 >> 172.21.15.53:0/833591969 conn(0x560b858623e0 msgr2=0x560b858e2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 shutdown_connections 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 1 -- 172.21.15.53:0/833591969 wait complete. 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.205+0000 7febf618d740 20 client.6465 trim_cache size 0 max 16384 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "74c33c05-9c26-4bcf-8acc-11bf8ec32159", 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:38:10.214 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:38:10.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7febd00762d0 msgr2=0x7febd0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7febd00762d0 0x7febd0078790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7febe8006fd0 tx=0x7febe8008040 comp rx=0 tx=0).stop 2024-05-10T10:38:10.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec15d010 msgr2=0x7febec14b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec15d010 0x7febec14b250 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7febe400caf0 tx=0x7febe400a060 comp rx=0 tx=0).stop 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 shutdown_connections 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7febd00762d0 0x7febd0078790 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febec15d010 0x7febec14b250 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7febec0d1370 0x7febec1f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 --2- 172.21.15.53:0/2564693938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec0d04e0 0x7febec1498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 >> 172.21.15.53:0/2564693938 conn(0x7febec06c4a0 msgr2=0x7febec06c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 shutdown_connections 2024-05-10T10:38:10.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.206+0000 7febf3472640 1 -- 172.21.15.53:0/2564693938 wait complete. 2024-05-10T10:38:10.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:10.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-10T10:38:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.322+0000 7efd68bc1640 1 Processor -- start 2024-05-10T10:38:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.322+0000 7efd68bc1640 1 -- start start 2024-05-10T10:38:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.323+0000 7efd68bc1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efd640cffd0 0x7efd640b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.323+0000 7efd68bc1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6406e720 con 0x7efd640cd500 2024-05-10T10:38:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.323+0000 7efd68bc1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6406de80 con 0x7efd640cf3e0 2024-05-10T10:38:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.323+0000 7efd68bc1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efd6406e160 con 0x7efd640cffd0 2024-05-10T10:38:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.323+0000 7efd62d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efd640cd500 0x7efd640d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57928/0 (socket says 172.21.15.53:57928) 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.323+0000 7efd62d9d640 1 -- 172.21.15.53:0/3006567796 learned_addr learned my addr 172.21.15.53:0/3006567796 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.324+0000 7efd61d9b640 1 --2- 172.21.15.53:0/3006567796 >> v2:172.21.15.135:3300/0 conn(0x7efd640cffd0 0x7efd640b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.324+0000 7efd6159a640 1 -- 172.21.15.53:0/3006567796 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648355038 0 0) 0x7efd6406e720 con 0x7efd640cd500 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.324+0000 7efd6159a640 1 -- 172.21.15.53:0/3006567796 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd4c003520 con 0x7efd640cd500 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.324+0000 7efd6159a640 1 -- 172.21.15.53:0/3006567796 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1104459531 0 0) 0x7efd4c003520 con 0x7efd640cd500 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.324+0000 7efd61d9b640 1 -- 172.21.15.53:0/3006567796 >> v1:172.21.15.135:6789/0 conn(0x7efd640cf3e0 legacy=0x7efd640cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.324+0000 7efd61d9b640 1 -- 172.21.15.53:0/3006567796 >> v1:172.21.15.53:6789/0 conn(0x7efd640cd500 legacy=0x7efd640d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.324+0000 7efd61d9b640 1 -- 172.21.15.53:0/3006567796 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6406e440 con 0x7efd640cffd0 2024-05-10T10:38:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.325+0000 7efd61d9b640 1 --2- 172.21.15.53:0/3006567796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cffd0 0x7efd640b3de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd580097a0 tx=0x7efd5802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663bf7863d00cfd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.325+0000 7efd6159a640 1 -- 172.21.15.53:0/3006567796 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58037070 con 0x7efd640cffd0 2024-05-10T10:38:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.325+0000 7efd6159a640 1 -- 172.21.15.53:0/3006567796 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5802ac50 con 0x7efd640cffd0 2024-05-10T10:38:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.325+0000 7efd6159a640 1 -- 172.21.15.53:0/3006567796 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0019f0 con 0x7efd640cffd0 2024-05-10T10:38:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.325+0000 7efd68bc1640 1 -- 172.21.15.53:0/3006567796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cffd0 msgr2=0x7efd640b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.325+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3006567796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cffd0 0x7efd640b3de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd580097a0 tx=0x7efd5802a1c0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.326+0000 7efd68bc1640 1 -- 172.21.15.53:0/3006567796 shutdown_connections 2024-05-10T10:38:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.326+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3006567796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cffd0 0x7efd640b3de0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.326+0000 7efd68bc1640 1 -- 172.21.15.53:0/3006567796 >> 172.21.15.53:0/3006567796 conn(0x7efd640d2eb0 msgr2=0x7efd640d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.326+0000 7efd68bc1640 1 -- 172.21.15.53:0/3006567796 shutdown_connections 2024-05-10T10:38:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.326+0000 7efd68bc1640 1 -- 172.21.15.53:0/3006567796 wait complete. 2024-05-10T10:38:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.327+0000 7efd68bc1640 1 Processor -- start 2024-05-10T10:38:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.327+0000 7efd68bc1640 1 -- start start 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd68bc1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cd500 0x7efd64161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd68bc1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd640cf3e0 0x7efd640b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd68bc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd640cffd0 0x7efd640b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd68bc1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efd640d1890 con 0x7efd640cffd0 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd68bc1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd640b4510 con 0x7efd640cd500 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd68bc1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efd640d1a10 con 0x7efd640cf3e0 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd640cffd0 0x7efd640b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd640cffd0 0x7efd640b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43254/0 (socket says 172.21.15.53:43254) 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 -- 172.21.15.53:0/3373076058 learned_addr learned my addr 172.21.15.53:0/3373076058 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd61d9b640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd640cf3e0 0x7efd640b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd6259c640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cd500 0x7efd64161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 -- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd640cf3e0 msgr2=0x7efd640b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd640cf3e0 0x7efd640b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 -- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cd500 msgr2=0x7efd64161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cd500 0x7efd64161850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 -- 172.21.15.53:0/3373076058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd640b7fd0 con 0x7efd640cffd0 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd6259c640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cd500 0x7efd64161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.328+0000 7efd62d9d640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd640cffd0 0x7efd640b54a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00cc40 tx=0x7efd5c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.329+0000 7efd52ffd640 1 -- 172.21.15.53:0/3373076058 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c017070 con 0x7efd640cffd0 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.329+0000 7efd52ffd640 1 -- 172.21.15.53:0/3373076058 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c00ce50 con 0x7efd640cffd0 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.329+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd640b82b0 con 0x7efd640cffd0 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.329+0000 7efd52ffd640 1 -- 172.21.15.53:0/3373076058 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c012620 con 0x7efd640cffd0 2024-05-10T10:38:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.329+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd640b72a0 con 0x7efd640cffd0 2024-05-10T10:38:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.330+0000 7efd52ffd640 1 -- 172.21.15.53:0/3373076058 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efd5c01a070 con 0x7efd640cffd0 2024-05-10T10:38:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.331+0000 7efd52ffd640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd440762d0 0x7efd44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.331+0000 7efd52ffd640 1 -- 172.21.15.53:0/3373076058 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7efd5c097730 con 0x7efd640cffd0 2024-05-10T10:38:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.331+0000 7efd6259c640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd440762d0 0x7efd44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.331+0000 7efd6b9bb740 1 Processor -- start 2024-05-10T10:38:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6b9bb740 1 -- start start 2024-05-10T10:38:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6259c640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd440762d0 0x7efd44078790 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efd641f2660 tx=0x7efd54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba21af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 0x55a09ba51600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba51b40 0x55a09ba26950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a09b9ee690 con 0x55a09ba51b40 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a09b98bf30 con 0x55a09ba6f160 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a09b944b20 con 0x55a09ba52280 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.332+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba51b40 0x55a09ba26950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba51b40 0x55a09ba26950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43260/0 (socket says 172.21.15.53:43260) 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 -- 172.21.15.53:0/2074373150 learned_addr learned my addr 172.21.15.53:0/2074373150 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd6259c640 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba21af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd61d9b640 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 0x55a09ba51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 -- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 msgr2=0x55a09ba51600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 0x55a09ba51600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 -- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 msgr2=0x55a09ba21af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba21af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 -- 172.21.15.53:0/2074373150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09b93ce60 con 0x55a09ba51b40 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd62d9d640 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba51b40 0x55a09ba26950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efd5c095ab0 tx=0x7efd5c0602d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddac12f0a1724089 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd43fff640 1 -- 172.21.15.53:0/2074373150 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c067e60 con 0x55a09ba51b40 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd43fff640 1 -- 172.21.15.53:0/2074373150 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c0607c0 con 0x55a09ba51b40 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.333+0000 7efd43fff640 1 -- 172.21.15.53:0/2074373150 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0670d0 con 0x55a09ba51b40 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 -- 172.21.15.53:0/2074373150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba51b40 msgr2=0x55a09ba26950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba51b40 0x55a09ba26950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efd5c095ab0 tx=0x7efd5c0602d0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 -- 172.21.15.53:0/2074373150 shutdown_connections 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba51b40 0x55a09ba26950 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 0x55a09ba51600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/2074373150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba21af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 -- 172.21.15.53:0/2074373150 >> 172.21.15.53:0/2074373150 conn(0x55a09ba23d60 msgr2=0x55a09ba6c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 -- 172.21.15.53:0/2074373150 shutdown_connections 2024-05-10T10:38:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 -- 172.21.15.53:0/2074373150 wait complete. 2024-05-10T10:38:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.334+0000 7efd6b9bb740 1 -- start start 2024-05-10T10:38:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 1 Processor -- start 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba52280 0x55a09ba54340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba53660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba53ba0 0x55a09ba3f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a09b9485e0 con 0x55a09ba52280 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a09b987ef0 con 0x55a09ba6f160 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a09ba25f10 con 0x55a09ba53ba0 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.335+0000 7efd6b9bb740 10 client.0 _ll_register_callbacks cb 0x55a09b943320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba52280 0x55a09ba54340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba52280 0x55a09ba54340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43274/0 (socket says 172.21.15.53:43274) 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 -- 172.21.15.53:0/1819488480 learned_addr learned my addr 172.21.15.53:0/1819488480 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd61d9b640 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba53660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd62d9d640 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba53ba0 0x55a09ba3f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 -- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba53ba0 msgr2=0x55a09ba3f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba53ba0 0x55a09ba3f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 -- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 msgr2=0x55a09ba53660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba53660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 -- 172.21.15.53:0/1819488480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09b7677d0 con 0x55a09ba52280 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6259c640 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba52280 0x55a09ba54340 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efd5400da50 tx=0x7efd54001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740ac3328d01035f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd417fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd417fa640 1 -- 172.21.15.53:0/1819488480 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54023450 con 0x55a09ba52280 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd417fa640 1 -- 172.21.15.53:0/1819488480 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7efd54020b80 con 0x55a09ba52280 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5c0607c0 con 0x55a09ba52280 2024-05-10T10:38:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd417fa640 1 -- 172.21.15.53:0/1819488480 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efd54029070 con 0x55a09ba52280 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6b9bb740 10 client.6477 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.336+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55a09b69d3f0 con 0x55a09ba52280 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd417fa640 1 -- 172.21.15.53:0/1819488480 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c0607c0 con 0x55a09ba52280 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd417fa640 1 -- 172.21.15.53:0/1819488480 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7efd540235f0 con 0x55a09ba52280 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd6b9bb740 10 client.6477 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd6b9bb740 10 client.6477 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd6b9bb740 20 client.6477 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba3fc80 0x55a09ba6e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd6b9bb740 4 client.6477 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a09b999b40 con 0x55a09ba3fc80 2024-05-10T10:38:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd61d9b640 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba3fc80 0x55a09ba6e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd61d9b640 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba3fc80 0x55a09ba6e460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8139375321dbc716 server_cookie=591900dc8cac773a in_seq=0 out_seq=0 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.337+0000 7efd417fa640 10 client.6477 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd417fa640 1 -- 172.21.15.53:0/1819488480 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a09b999b40 con 0x55a09ba3fc80 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd417fa640 10 client.6477 handle_command_reply: tid=0 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 client.6477 shutdown 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba52280 msgr2=0x55a09ba54340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba52280 0x55a09ba54340 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efd5400da50 tx=0x7efd54001120 comp rx=0 tx=0).stop 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 shutdown_connections 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba3fc80 0x55a09ba6e460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba53ba0 0x55a09ba3f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba53660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd417fa640 0 client.6477 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1819488480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba52280 0x55a09ba54340 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 >> 172.21.15.53:0/1819488480 conn(0x55a09ba51b40 msgr2=0x55a09ba255e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 shutdown_connections 2024-05-10T10:38:10.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1819488480 wait complete. 2024-05-10T10:38:10.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.338+0000 7efd6b9bb740 20 client.6477 trim_cache size 0 max 16384 2024-05-10T10:38:10.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.356+0000 7efd6b9bb740 1 Processor -- start 2024-05-10T10:38:10.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.356+0000 7efd6b9bb740 1 -- start start 2024-05-10T10:38:10.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba52280 0x55a09ba275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba6f160 0x55a09ba904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba7bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efd64201cb0 con 0x55a09ba27cd0 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd64200d30 con 0x55a09ba52280 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efd6409e550 con 0x55a09ba6f160 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba7bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba7bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43280/0 (socket says 172.21.15.53:43280) 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 -- 172.21.15.53:0/3493782677 learned_addr learned my addr 172.21.15.53:0/3493782677 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd61d9b640 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba6f160 0x55a09ba904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd6259c640 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba52280 0x55a09ba275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 -- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba6f160 msgr2=0x55a09ba904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba6f160 0x55a09ba904a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 -- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba52280 msgr2=0x55a09ba275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba52280 0x55a09ba275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.357+0000 7efd62d9d640 1 -- 172.21.15.53:0/3493782677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09b65d0f0 con 0x55a09ba27cd0 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd62d9d640 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba7bc70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efd5c095b60 tx=0x7efd5c066000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59ae39f0143948e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd437fe640 1 -- 172.21.15.53:0/3493782677 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c067e60 con 0x55a09ba27cd0 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd437fe640 1 -- 172.21.15.53:0/3493782677 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c0607c0 con 0x55a09ba27cd0 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd437fe640 1 -- 172.21.15.53:0/3493782677 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c069320 con 0x55a09ba27cd0 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 -- 172.21.15.53:0/3493782677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 msgr2=0x55a09ba7bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba7bc70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efd5c095b60 tx=0x7efd5c066000 comp rx=0 tx=0).stop 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 -- 172.21.15.53:0/3493782677 shutdown_connections 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba7bc70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba6f160 0x55a09ba904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/3493782677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba52280 0x55a09ba275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 -- 172.21.15.53:0/3493782677 >> 172.21.15.53:0/3493782677 conn(0x55a09ba51b40 msgr2=0x55a09ba6e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 -- 172.21.15.53:0/3493782677 shutdown_connections 2024-05-10T10:38:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.358+0000 7efd6b9bb740 1 -- 172.21.15.53:0/3493782677 wait complete. 2024-05-10T10:38:10.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.359+0000 7efd6b9bb740 1 -- start start 2024-05-10T10:38:10.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.359+0000 7efd6b9bb740 1 Processor -- start 2024-05-10T10:38:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba21420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 0x55a09bae9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6b9bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba7f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efd642006e0 con 0x55a09ba27cd0 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a09baea330 con 0x55a09ba6f160 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6b9bb740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a09ba4dd70 con 0x55a09ba52280 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6b9bb740 10 client.0 _ll_register_callbacks cb 0x7efd54001cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba21420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba21420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43286/0 (socket says 172.21.15.53:43286) 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 -- 172.21.15.53:0/1718800329 learned_addr learned my addr 172.21.15.53:0/1718800329 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 -- 172.21.15.53:0/1718800329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 msgr2=0x55a09bae9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 0x55a09bae9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 -- 172.21.15.53:0/1718800329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 msgr2=0x55a09ba7f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba7f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 -- 172.21.15.53:0/1718800329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09ba4c9d0 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd6259c640 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba21420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efd54007be0 tx=0x7efd54011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb43e1c05e2fc53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.360+0000 7efd417fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd417fa640 1 -- 172.21.15.53:0/1718800329 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54023d40 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd417fa640 1 -- 172.21.15.53:0/1718800329 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7efd54023450 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5c0607c0 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd417fa640 1 -- 172.21.15.53:0/1718800329 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efd5403d050 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 10 client.6483 fetch_fsmap learned FSMap version 52 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55a09baea130 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd417fa640 1 -- 172.21.15.53:0/1718800329 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c0607c0 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd417fa640 1 -- 172.21.15.53:0/1718800329 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7efd54020040 con 0x55a09ba27cd0 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 10 client.6483 fetch_fsmap finished waiting for FSMap version 52 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 10 client.6483 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 20 client.6483 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba61f90 0x55a09ba62410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 4 client.6483 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a09ba6dde0 con 0x55a09ba61f90 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.361+0000 7efd61d9b640 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba61f90 0x55a09ba62410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd417fa640 10 client.6483 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd61d9b640 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba61f90 0x55a09ba62410 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b27de1e4031346b server_cookie=4b1db12be8400e5e in_seq=0 out_seq=0 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd417fa640 1 -- 172.21.15.53:0/1718800329 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55a09ba6dde0 con 0x55a09ba61f90 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd417fa640 10 client.6483 handle_command_reply: tid=0 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 client.6483 shutdown 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 msgr2=0x55a09ba21420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba21420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efd54007be0 tx=0x7efd54011000 comp rx=0 tx=0).stop 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 shutdown_connections 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x55a09ba61f90 0x55a09ba62410 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a09ba6f160 0x55a09ba7f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a09ba52280 0x55a09bae9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 --2- 172.21.15.53:0/1718800329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a09ba27cd0 0x55a09ba21420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd417fa640 0 client.6483 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 >> 172.21.15.53:0/1718800329 conn(0x55a09ba51b40 msgr2=0x55a09ba90380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 shutdown_connections 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 1 -- 172.21.15.53:0/1718800329 wait complete. 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.362+0000 7efd6b9bb740 20 client.6483 trim_cache size 0 max 16384 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:38:10.370 INFO:teuthology.orchestra.run.smithi053.stdout: "74c33c05-9c26-4bcf-8acc-11bf8ec32159": { 2024-05-10T10:38:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/subdir", 2024-05-10T10:38:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "74c33c05-9c26-4bcf-8acc-11bf8ec32159", 2024-05-10T10:38:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair" 2024-05-10T10:38:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:10.371 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd440762d0 msgr2=0x7efd44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd440762d0 0x7efd44078790 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efd641f2660 tx=0x7efd54009040 comp rx=0 tx=0).stop 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd640cffd0 msgr2=0x7efd640b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd640cffd0 0x7efd640b54a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00cc40 tx=0x7efd5c00a430 comp rx=0 tx=0).stop 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 shutdown_connections 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd440762d0 0x7efd44078790 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd640cffd0 0x7efd640b54a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd640cf3e0 0x7efd640b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 --2- 172.21.15.53:0/3373076058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd640cd500 0x7efd64161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 >> 172.21.15.53:0/3373076058 conn(0x7efd640d2eb0 msgr2=0x7efd640cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 shutdown_connections 2024-05-10T10:38:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:10.363+0000 7efd68bc1640 1 -- 172.21.15.53:0/3373076058 wait complete. 2024-05-10T10:38:10.375 INFO:tasks.cephfs.filesystem:scrub status for tag:74c33c05-9c26-4bcf-8acc-11bf8ec32159 - {'path': '/subdir', 'tag': '74c33c05-9c26-4bcf-8acc-11bf8ec32159', 'options': 'recursive,repair'} 2024-05-10T10:38:14.591 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:38:14.594 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:38:40.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:40.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-10T10:38:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.495+0000 7f1fa0009640 1 Processor -- start 2024-05-10T10:38:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.495+0000 7f1fa0009640 1 -- start start 2024-05-10T10:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1fa0009640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f9806c2c0 0x7f1f980d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1fa0009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f9806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1fa0009640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f98164b10 con 0x7f1f980c4190 2024-05-10T10:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1fa0009640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1f980ccec0 con 0x7f1f9806c2c0 2024-05-10T10:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1fa0009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1f981645e0 con 0x7f1f980cb770 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1f9dda5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f9806c2c0 0x7f1f980d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1f9dda5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f9806c2c0 0x7f1f980d06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56490/0 (socket says 172.21.15.53:56490) 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.496+0000 7f1f9dda5640 1 -- 172.21.15.53:0/701192235 learned_addr learned my addr 172.21.15.53:0/701192235 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.497+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/701192235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f9806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.497+0000 7f1f9dda5640 1 -- 172.21.15.53:0/701192235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 msgr2=0x7f1f9806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.497+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/701192235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f9806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.497+0000 7f1f9dda5640 1 -- 172.21.15.53:0/701192235 >> v1:172.21.15.135:6789/0 conn(0x7f1f980c4190 legacy=0x7f1f9806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.497+0000 7f1f9dda5640 1 -- 172.21.15.53:0/701192235 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f980cd0f0 con 0x7f1f9806c2c0 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.497+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/701192235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f9806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/701192235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9806c2c0 0x7f1f980d06d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009870 tx=0x7f1f9402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2124f3e4fd7d547e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1f9cda3640 1 -- 172.21.15.53:0/701192235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94037070 con 0x7f1f9806c2c0 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1f9cda3640 1 -- 172.21.15.53:0/701192235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94032440 con 0x7f1f9806c2c0 2024-05-10T10:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1f9cda3640 1 -- 172.21.15.53:0/701192235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94032810 con 0x7f1f9806c2c0 2024-05-10T10:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1fa0009640 1 -- 172.21.15.53:0/701192235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9806c2c0 msgr2=0x7f1f980d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1fa0009640 1 --2- 172.21.15.53:0/701192235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9806c2c0 0x7f1f980d06d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009870 tx=0x7f1f9402a250 comp rx=0 tx=0).stop 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1fa0009640 1 -- 172.21.15.53:0/701192235 shutdown_connections 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1fa0009640 1 --2- 172.21.15.53:0/701192235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f9806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1fa0009640 1 --2- 172.21.15.53:0/701192235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9806c2c0 0x7f1f980d06d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1fa0009640 1 -- 172.21.15.53:0/701192235 >> 172.21.15.53:0/701192235 conn(0x7f1f980d0c90 msgr2=0x7f1f980d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.498+0000 7f1fa0009640 1 -- 172.21.15.53:0/701192235 shutdown_connections 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.499+0000 7f1fa0009640 1 -- 172.21.15.53:0/701192235 wait complete. 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.500+0000 7f1fa0009640 1 Processor -- start 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.500+0000 7f1fa0009640 1 -- start start 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.500+0000 7f1fa0009640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f9806c2c0 0x7f1f980b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1fa0009640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f980c4190 0x7f1f981f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1fa0009640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f98165580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1fa0009640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98164830 con 0x7f1f980c4190 2024-05-10T10:38:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1fa0009640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f980cb2e0 con 0x7f1f9806c2c0 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1fa0009640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1f980cd390 con 0x7f1f980cb770 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f980c4190 0x7f1f981f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f980c4190 0x7f1f981f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56496/0 (socket says 172.21.15.53:56496) 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1819740190 learned_addr learned my addr 172.21.15.53:0/1819740190 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f98165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f9806c2c0 0x7f1f980b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 msgr2=0x7f1f98165580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f98165580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f9806c2c0 msgr2=0x7f1f980b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f9806c2c0 0x7f1f980b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1819740190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f980b4c80 con 0x7f1f980c4190 2024-05-10T10:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f98165580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f9806c2c0 0x7f1f980b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f980c4190 0x7f1f981f9120 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c7f0 tx=0x7f1f8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f8e7fc640 1 -- 172.21.15.53:0/1819740190 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88004230 con 0x7f1f980c4190 2024-05-10T10:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f8e7fc640 1 -- 172.21.15.53:0/1819740190 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f880043d0 con 0x7f1f980c4190 2024-05-10T10:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f980b4f60 con 0x7f1f980c4190 2024-05-10T10:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.501+0000 7f1f8e7fc640 1 -- 172.21.15.53:0/1819740190 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88007a70 con 0x7f1f980c4190 2024-05-10T10:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.502+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f980b5520 con 0x7f1f980c4190 2024-05-10T10:38:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.503+0000 7f1f8e7fc640 1 -- 172.21.15.53:0/1819740190 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f88007c10 con 0x7f1f980c4190 2024-05-10T10:38:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.504+0000 7f1f8e7fc640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f7c0762d0 0x7f1f7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.504+0000 7f1f8e7fc640 1 -- 172.21.15.53:0/1819740190 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1f88097870 con 0x7f1f980c4190 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.504+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f7c0762d0 0x7f1f7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.505+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f7c0762d0 0x7f1f7c078790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f94034000 tx=0x7f1f9402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.505+0000 7f1fa2e03740 1 Processor -- start 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.505+0000 7f1fa2e03740 1 -- start start 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fda7aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaaa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdaaab40 0x5632fda7f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5632fda47690 con 0x5632fdaaab40 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5632fd9e4f30 con 0x5632fdaab280 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5632fd99db20 con 0x5632fdac8160 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdaaab40 0x5632fda7f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdaaab40 0x5632fda7f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56508/0 (socket says 172.21.15.53:56508) 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1708994441 learned_addr learned my addr 172.21.15.53:0/1708994441 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fda7aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaaa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 msgr2=0x5632fda7aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fda7aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 msgr2=0x5632fdaaa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaaa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1708994441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fd995e60 con 0x5632fdaaab40 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdaaab40 0x5632fda7f950 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1f90001d70 tx=0x7f1f9000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790344f7273e5a6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f7b7fe640 1 -- 172.21.15.53:0/1708994441 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90019070 con 0x5632fdaaab40 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f7b7fe640 1 -- 172.21.15.53:0/1708994441 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f90003b80 con 0x5632fdaaab40 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.506+0000 7f1f7b7fe640 1 -- 172.21.15.53:0/1708994441 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000e960 con 0x5632fdaaab40 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1708994441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdaaab40 msgr2=0x5632fda7f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdaaab40 0x5632fda7f950 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1f90001d70 tx=0x7f1f9000e160 comp rx=0 tx=0).stop 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1708994441 shutdown_connections 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdaaab40 0x5632fda7f950 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaaa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1708994441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fda7aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1708994441 >> 172.21.15.53:0/1708994441 conn(0x5632fda7cd60 msgr2=0x5632fdac5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1708994441 shutdown_connections 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.507+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1708994441 wait complete. 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.508+0000 7f1fa2e03740 1 -- start start 2024-05-10T10:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.508+0000 7f1fa2e03740 1 Processor -- start 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdac6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdad56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5632fd9a15e0 con 0x5632fdac6b90 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5632fd9e0ef0 con 0x5632fdaab280 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5632fda7ef10 con 0x5632fdac8160 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1fa2e03740 10 client.0 _ll_register_callbacks cb 0x5632fd99c320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdad56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdad56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56522/0 (socket says 172.21.15.53:56522) 2024-05-10T10:38:40.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1476811139 learned_addr learned my addr 172.21.15.53:0/1476811139 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdac6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 msgr2=0x5632fdac6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdac6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 msgr2=0x5632fdaac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 -- 172.21.15.53:0/1476811139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632fd7c07d0 con 0x5632fdac6b90 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdad56f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1f900012e0 tx=0x7f1f90001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a5f95fbc98a928 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.509+0000 7f1f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1f78ff9640 1 -- 172.21.15.53:0/1476811139 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90019050 con 0x5632fdac6b90 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1f78ff9640 1 -- 172.21.15.53:0/1476811139 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1f900050d0 con 0x5632fdac6b90 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f90003b80 con 0x5632fdac6b90 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1f78ff9640 1 -- 172.21.15.53:0/1476811139 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1f9002d050 con 0x5632fdac6b90 2024-05-10T10:38:40.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 10 client.6495 fetch_fsmap learned FSMap version 53 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5632fd6f63f0 con 0x5632fdac6b90 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1f78ff9640 1 -- 172.21.15.53:0/1476811139 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f90003b80 con 0x5632fdac6b90 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1f78ff9640 1 -- 172.21.15.53:0/1476811139 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f1f90030020 con 0x5632fdac6b90 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 10 client.6495 fetch_fsmap finished waiting for FSMap version 53 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 10 client.6495 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 20 client.6495 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdad6100 0x5632fdaab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 4 client.6495 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632fd9f2b40 con 0x5632fdad6100 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.510+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdad6100 0x5632fdaab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.511+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdad6100 0x5632fdaab770 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4336436a3230062e server_cookie=288ece8a4cf8feca in_seq=0 out_seq=0 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.511+0000 7f1f78ff9640 10 client.6495 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.511+0000 7f1f78ff9640 1 -- 172.21.15.53:0/1476811139 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5632fd9f2b40 con 0x5632fdad6100 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.511+0000 7f1f78ff9640 10 client.6495 handle_command_reply: tid=0 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.511+0000 7f1fa2e03740 1 client.6495 shutdown 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.511+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 msgr2=0x5632fdad56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.511+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdad56f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1f900012e0 tx=0x7f1f90001250 comp rx=0 tx=0).stop 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 shutdown_connections 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdad6100 0x5632fdaab770 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdad56f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdac6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1476811139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdaac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1f78ff9640 0 client.6495 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 >> 172.21.15.53:0/1476811139 conn(0x5632fdaaab40 msgr2=0x5632fda7e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 shutdown_connections 2024-05-10T10:38:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1476811139 wait complete. 2024-05-10T10:38:40.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.512+0000 7f1fa2e03740 20 client.6495 trim_cache size 0 max 16384 2024-05-10T10:38:40.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.530+0000 7f1fa2e03740 1 Processor -- start 2024-05-10T10:38:40.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.530+0000 7f1fa2e03740 1 -- start start 2024-05-10T10:38:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdaab280 0x5632fda9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdac4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdac8160 0x5632fdac8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98206260 con 0x5632fdac6b90 2024-05-10T10:38:40.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98205320 con 0x5632fdac8160 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1f981fd320 con 0x5632fdaab280 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdac4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdac4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56532/0 (socket says 172.21.15.53:56532) 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1990361912 learned_addr learned my addr 172.21.15.53:0/1990361912 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdaab280 0x5632fda9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdac8160 0x5632fdac8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdaab280 msgr2=0x5632fda9a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdaab280 0x5632fda9a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdac8160 msgr2=0x5632fdac8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdac8160 0x5632fdac8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/1990361912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fd6b60f0 con 0x5632fdac6b90 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdac8160 0x5632fdac8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:40.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.531+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdaab280 0x5632fda9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdac4280 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1f88000c00 tx=0x7f1f88069040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b18f150b8bc60f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1f7affd640 1 -- 172.21.15.53:0/1990361912 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8801e070 con 0x5632fdac6b90 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1f7affd640 1 -- 172.21.15.53:0/1990361912 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88060ff0 con 0x5632fdac6b90 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1f7affd640 1 -- 172.21.15.53:0/1990361912 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f880608f0 con 0x5632fdac6b90 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1990361912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 msgr2=0x5632fdac4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdac4280 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1f88000c00 tx=0x7f1f88069040 comp rx=0 tx=0).stop 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1990361912 shutdown_connections 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdac8160 0x5632fdac8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdac4280 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/1990361912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdaab280 0x5632fda9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1990361912 >> 172.21.15.53:0/1990361912 conn(0x5632fdaaab40 msgr2=0x5632fda76150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1990361912 shutdown_connections 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.532+0000 7f1fa2e03740 1 -- 172.21.15.53:0/1990361912 wait complete. 2024-05-10T10:38:40.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.533+0000 7f1fa2e03740 1 -- start start 2024-05-10T10:38:40.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.533+0000 7f1fa2e03740 1 Processor -- start 2024-05-10T10:38:40.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdac5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdaad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1fa2e03740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdad5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981fd970 con 0x5632fdac6b90 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5632fd9996d0 con 0x5632fdaab280 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1fa2e03740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5632fdad4070 con 0x5632fdac8160 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1fa2e03740 10 client.0 _ll_register_callbacks cb 0x7f1f90017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdaad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdaad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56538/0 (socket says 172.21.15.53:56538) 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/623353800 learned_addr learned my addr 172.21.15.53:0/623353800 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdac5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdad5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 msgr2=0x5632fdad5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdad5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 msgr2=0x5632fdac5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdac5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 -- 172.21.15.53:0/623353800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632fdaae0c0 con 0x5632fdac6b90 2024-05-10T10:38:40.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9e5a6640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdad5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f9d5a4640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdaad7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1f88060880 tx=0x7f1f88097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98d6eed02a82307 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.534+0000 7f1f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f78ff9640 1 -- 172.21.15.53:0/623353800 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8801e050 con 0x5632fdac6b90 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f78ff9640 1 -- 172.21.15.53:0/623353800 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1f8805c260 con 0x5632fdac6b90 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f88060ff0 con 0x5632fdac6b90 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f78ff9640 1 -- 172.21.15.53:0/623353800 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1f880696c0 con 0x5632fdac6b90 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 10 client.6501 fetch_fsmap learned FSMap version 53 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5632fdaadd30 con 0x5632fdac6b90 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f78ff9640 1 -- 172.21.15.53:0/623353800 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88060ff0 con 0x5632fdac6b90 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f78ff9640 1 -- 172.21.15.53:0/623353800 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f1f8805f5c0 con 0x5632fdac6b90 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 10 client.6501 fetch_fsmap finished waiting for FSMap version 53 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 10 client.6501 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 20 client.6501 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:40.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdac5440 0x5632fdad56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 4 client.6501 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632fdabeb90 con 0x5632fdac5440 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdac5440 0x5632fdad56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f78ff9640 10 client.6501 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.535+0000 7f1f9dda5640 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdac5440 0x5632fdad56c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd726ac18fb795b8 server_cookie=f7d4c5a404a7ddc1 in_seq=0 out_seq=0 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1f78ff9640 1 -- 172.21.15.53:0/623353800 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5632fdabeb90 con 0x5632fdac5440 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1f78ff9640 10 client.6501 handle_command_reply: tid=0 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 client.6501 shutdown 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 msgr2=0x5632fdaad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdaad7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1f88060880 tx=0x7f1f88097040 comp rx=0 tx=0).stop 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 shutdown_connections 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5632fdac5440 0x5632fdad56c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5632fdac8160 0x5632fdad5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632fdac6b90 0x5632fdaad7f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 --2- 172.21.15.53:0/623353800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632fdaab280 0x5632fdac5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1f78ff9640 0 client.6501 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 >> 172.21.15.53:0/623353800 conn(0x5632fdaaab40 msgr2=0x5632fda76150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 shutdown_connections 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 1 -- 172.21.15.53:0/623353800 wait complete. 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa2e03740 20 client.6501 trim_cache size 0 max 16384 2024-05-10T10:38:40.544 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:38:40.545 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:38:40.545 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:38:40.545 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f7c0762d0 msgr2=0x7f1f7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa0009640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f7c0762d0 0x7f1f7c078790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f94034000 tx=0x7f1f9402e000 comp rx=0 tx=0).stop 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.536+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f980c4190 msgr2=0x7f1f981f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f980c4190 0x7f1f981f9120 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c7f0 tx=0x7f1f8800ccb0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 shutdown_connections 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f7c0762d0 0x7f1f7c078790 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f980cb770 0x7f1f98165580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f980c4190 0x7f1f981f9120 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 --2- 172.21.15.53:0/1819740190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f9806c2c0 0x7f1f980b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 >> 172.21.15.53:0/1819740190 conn(0x7f1f980d0c90 msgr2=0x7f1f98071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 shutdown_connections 2024-05-10T10:38:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.537+0000 7f1fa0009640 1 -- 172.21.15.53:0/1819740190 wait complete. 2024-05-10T10:38:40.549 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:38:40.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:38:40.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.651+0000 7fdc157f0640 1 Processor -- start 2024-05-10T10:38:40.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.651+0000 7fdc157f0640 1 -- start start 2024-05-10T10:38:40.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.652+0000 7fdc157f0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc100d1aa0 0x7fdc100d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.652+0000 7fdc157f0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc10072330 con 0x7fdc100d3340 2024-05-10T10:38:40.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.652+0000 7fdc157f0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc10071d70 con 0x7fdc1015d290 2024-05-10T10:38:40.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.652+0000 7fdc157f0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fdc100d37d0 con 0x7fdc100d1aa0 2024-05-10T10:38:40.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.652+0000 7fdc0e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc100d1aa0 0x7fdc100d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.652+0000 7fdc0f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdc100d3340 0x7fdc100ce270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50138/0 (socket says 172.21.15.53:50138) 2024-05-10T10:38:40.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.653+0000 7fdc0f7fe640 1 -- 172.21.15.53:0/3260553471 learned_addr learned my addr 172.21.15.53:0/3260553471 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:40.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.653+0000 7fdc0e7fc640 1 -- 172.21.15.53:0/3260553471 >> v1:172.21.15.135:6789/0 conn(0x7fdc1015d290 legacy=0x7fdc10071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:40.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.653+0000 7fdc0e7fc640 1 -- 172.21.15.53:0/3260553471 >> v1:172.21.15.53:6789/0 conn(0x7fdc100d3340 legacy=0x7fdc100ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.653+0000 7fdc0e7fc640 1 -- 172.21.15.53:0/3260553471 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc10072050 con 0x7fdc100d1aa0 2024-05-10T10:38:40.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.653+0000 7fdc0dffb640 1 -- 172.21.15.53:0/3260553471 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738776942 0 0) 0x7fdc10072330 con 0x7fdc100d3340 2024-05-10T10:38:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc0e7fc640 1 --2- 172.21.15.53:0/3260553471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d1aa0 0x7fdc100d2e00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80097a0 tx=0x7fdbf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5883ed1319bd64 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc0dffb640 1 -- 172.21.15.53:0/3260553471 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8037070 con 0x7fdc100d1aa0 2024-05-10T10:38:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc0dffb640 1 -- 172.21.15.53:0/3260553471 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf8032440 con 0x7fdc100d1aa0 2024-05-10T10:38:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc0dffb640 1 -- 172.21.15.53:0/3260553471 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8032810 con 0x7fdc100d1aa0 2024-05-10T10:38:40.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc157f0640 1 -- 172.21.15.53:0/3260553471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d1aa0 msgr2=0x7fdc100d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc157f0640 1 --2- 172.21.15.53:0/3260553471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d1aa0 0x7fdc100d2e00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80097a0 tx=0x7fdbf802a1e0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc157f0640 1 -- 172.21.15.53:0/3260553471 shutdown_connections 2024-05-10T10:38:40.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc157f0640 1 --2- 172.21.15.53:0/3260553471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d1aa0 0x7fdc100d2e00 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc157f0640 1 -- 172.21.15.53:0/3260553471 >> 172.21.15.53:0/3260553471 conn(0x7fdc1009ba20 msgr2=0x7fdc1009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc157f0640 1 -- 172.21.15.53:0/3260553471 shutdown_connections 2024-05-10T10:38:40.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.654+0000 7fdc157f0640 1 -- 172.21.15.53:0/3260553471 wait complete. 2024-05-10T10:38:40.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 Processor -- start 2024-05-10T10:38:40.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 -- start start 2024-05-10T10:38:40.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d1aa0 0x7fdc100c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d3340 0x7fdc100b8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc1015d290 0x7fdc100cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc10071a50 con 0x7fdc100d3340 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc100cdd50 con 0x7fdc100d1aa0 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.656+0000 7fdc157f0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdc100ce980 con 0x7fdc1015d290 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d3340 0x7fdc100b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d3340 0x7fdc100b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56554/0 (socket says 172.21.15.53:56554) 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 -- 172.21.15.53:0/978086457 learned_addr learned my addr 172.21.15.53:0/978086457 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 -- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc1015d290 msgr2=0x7fdc100cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0f7fe640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc1015d290 0x7fdc100cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0effd640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d1aa0 0x7fdc100c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc1015d290 0x7fdc100cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 -- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d1aa0 msgr2=0x7fdc100c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d1aa0 0x7fdc100c68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 -- 172.21.15.53:0/978086457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc100ca140 con 0x7fdc100d3340 2024-05-10T10:38:40.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc0e7fc640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d3340 0x7fdc100b8430 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80098c0 tx=0x7fdbf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc077fe640 1 -- 172.21.15.53:0/978086457 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8041070 con 0x7fdc100d3340 2024-05-10T10:38:40.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc077fe640 1 -- 172.21.15.53:0/978086457 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf803c930 con 0x7fdc100d3340 2024-05-10T10:38:40.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc100ca420 con 0x7fdc100d3340 2024-05-10T10:38:40.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc077fe640 1 -- 172.21.15.53:0/978086457 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8037040 con 0x7fdc100d3340 2024-05-10T10:38:40.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.657+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc100ca9e0 con 0x7fdc100d3340 2024-05-10T10:38:40.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.658+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc100b0c30 con 0x7fdc100d3340 2024-05-10T10:38:40.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.659+0000 7fdc077fe640 1 -- 172.21.15.53:0/978086457 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdbf8044070 con 0x7fdc100d3340 2024-05-10T10:38:40.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.660+0000 7fdc077fe640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbf00762d0 0x7fdbf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:40.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.660+0000 7fdc077fe640 1 -- 172.21.15.53:0/978086457 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fdbf80b8310 con 0x7fdc100d3340 2024-05-10T10:38:40.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.660+0000 7fdc0effd640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbf00762d0 0x7fdbf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:40.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.660+0000 7fdc0effd640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbf00762d0 0x7fdbf0078790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdc100a0e20 tx=0x7fdc080072f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:40.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.664+0000 7fdc077fe640 1 -- 172.21.15.53:0/978086457 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdc100b0c30 con 0x7fdc100d3340 2024-05-10T10:38:40.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.893+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc100693f0 con 0x7fdc100d3340 2024-05-10T10:38:40.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.894+0000 7fdc077fe640 1 -- 172.21.15.53:0/978086457 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7fdbf8081b20 con 0x7fdc100d3340 2024-05-10T10:38:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:40.895 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:38:14.328418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":5}]} 2024-05-10T10:38:40.896 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2024-05-10T10:38:40.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbf00762d0 msgr2=0x7fdbf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbf00762d0 0x7fdbf0078790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdc100a0e20 tx=0x7fdc080072f0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d3340 msgr2=0x7fdc100b8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d3340 0x7fdc100b8430 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80098c0 tx=0x7fdbf80046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 shutdown_connections 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdbf00762d0 0x7fdbf0078790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc1015d290 0x7fdc100cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc100d3340 0x7fdc100b8430 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 --2- 172.21.15.53:0/978086457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d1aa0 0x7fdc100c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 >> 172.21.15.53:0/978086457 conn(0x7fdc1009ba20 msgr2=0x7fdc100b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 shutdown_connections 2024-05-10T10:38:40.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:40.896+0000 7fdc157f0640 1 -- 172.21.15.53:0/978086457 wait complete. 2024-05-10T10:38:40.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-10T10:38:41.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.011+0000 7fe1d6c93640 1 Processor -- start 2024-05-10T10:38:41.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.012+0000 7fe1d6c93640 1 -- start start 2024-05-10T10:38:41.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.012+0000 7fe1d6c93640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe1d015f410 0x7fe1d00d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.012+0000 7fe1d6c93640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d00d0f10 0x7fe1d00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.012+0000 7fe1d6c93640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d00d2800 con 0x7fe1d00b54d0 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.012+0000 7fe1d6c93640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d00d2ae0 con 0x7fe1d015f410 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.012+0000 7fe1d6c93640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00d01d0 con 0x7fe1d00d0f10 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d4a2f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe1d015f410 0x7fe1d00d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d00d0f10 0x7fe1d00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d5230640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe1d00b54d0 0x7fe1d00d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:44862/0 (socket says 172.21.15.53:44862) 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d4a2f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe1d015f410 0x7fe1d00d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:51294/0 (socket says 172.21.15.53:51294) 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d5230640 1 -- 172.21.15.53:0/3873761055 learned_addr learned my addr 172.21.15.53:0/3873761055 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d4a2f640 1 -- 172.21.15.53:0/3873761055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d00d0f10 msgr2=0x7fe1d00b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/3873761055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d00d0f10 0x7fe1d00b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d4a2f640 1 -- 172.21.15.53:0/3873761055 >> v1:172.21.15.135:6789/0 conn(0x7fe1d00b54d0 legacy=0x7fe1d00d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1cf7fe640 1 -- 172.21.15.53:0/3873761055 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829208437 0 0) 0x7fe1d00d2800 con 0x7fe1d00b54d0 2024-05-10T10:38:41.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.013+0000 7fe1d4a2f640 1 -- 172.21.15.53:0/3873761055 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00d04b0 con 0x7fe1d015f410 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.014+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/3873761055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f410 0x7fe1d00d09d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40097a0 tx=0x7fe1c402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=124b966bf7d313a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.014+0000 7fe1cf7fe640 1 -- 172.21.15.53:0/3873761055 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4037070 con 0x7fe1d015f410 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.014+0000 7fe1cf7fe640 1 -- 172.21.15.53:0/3873761055 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c402ae50 con 0x7fe1d015f410 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.014+0000 7fe1cf7fe640 1 -- 172.21.15.53:0/3873761055 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4032660 con 0x7fe1d015f410 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 -- 172.21.15.53:0/3873761055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f410 msgr2=0x7fe1d00d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/3873761055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f410 0x7fe1d00d09d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40097a0 tx=0x7fe1c402a100 comp rx=0 tx=0).stop 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 -- 172.21.15.53:0/3873761055 shutdown_connections 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/3873761055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d00d0f10 0x7fe1d00b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/3873761055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d015f410 0x7fe1d00d09d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 -- 172.21.15.53:0/3873761055 >> 172.21.15.53:0/3873761055 conn(0x7fe1d008ed30 msgr2=0x7fe1d009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 -- 172.21.15.53:0/3873761055 shutdown_connections 2024-05-10T10:38:41.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.015+0000 7fe1d6c93640 1 -- 172.21.15.53:0/3873761055 wait complete. 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.016+0000 7fe1d6c93640 1 Processor -- start 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.016+0000 7fe1d6c93640 1 -- start start 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d6c93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d00b54d0 0x7fe1d00cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d6c93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d0f10 0x7fe1d00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d6c93640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f410 0x7fe1d00d5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d6c93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00cffb0 con 0x7fe1d00d0f10 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d6c93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00cfe30 con 0x7fe1d00b54d0 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d6c93640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00d3de0 con 0x7fe1d015f410 2024-05-10T10:38:41.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d0f10 0x7fe1d00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d0f10 0x7fe1d00cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56564/0 (socket says 172.21.15.53:56564) 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 -- 172.21.15.53:0/4252364826 learned_addr learned my addr 172.21.15.53:0/4252364826 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d00b54d0 0x7fe1d00cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1d5230640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f410 0x7fe1d00d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 -- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f410 msgr2=0x7fe1d00d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f410 0x7fe1d00d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 -- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d00b54d0 msgr2=0x7fe1d00cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d00b54d0 0x7fe1d00cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.017+0000 7fe1cffff640 1 -- 172.21.15.53:0/4252364826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00cdd00 con 0x7fe1d00d0f10 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.018+0000 7fe1cffff640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d0f10 0x7fe1d00cd7c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9e0 tx=0x7fe1bc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.018+0000 7fe1cd7fa640 1 -- 172.21.15.53:0/4252364826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc017070 con 0x7fe1d00d0f10 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.018+0000 7fe1cd7fa640 1 -- 172.21.15.53:0/4252364826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00c9d0 con 0x7fe1d00d0f10 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.018+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d00cdfa0 con 0x7fe1d00d0f10 2024-05-10T10:38:41.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.018+0000 7fe1cd7fa640 1 -- 172.21.15.53:0/4252364826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00cdc0 con 0x7fe1d00d0f10 2024-05-10T10:38:41.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.018+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d00ce560 con 0x7fe1d00d0f10 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.020+0000 7fe1cd7fa640 1 -- 172.21.15.53:0/4252364826 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe1bc01a070 con 0x7fe1d00d0f10 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.020+0000 7fe1cd7fa640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1b40762d0 0x7fe1b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.020+0000 7fe1cd7fa640 1 -- 172.21.15.53:0/4252364826 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fe1bc097ae0 con 0x7fe1d00d0f10 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.020+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1b40762d0 0x7fe1b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 Processor -- start 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 -- start start 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1b40762d0 0x7fe1b4078790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4034000 tx=0x7fe1c402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ca6d0 0x56207367d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x5620736acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ad0b0 0x562073681ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56207356d920 con 0x5620736ad7f0 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5620735e2dd0 con 0x5620736ca6d0 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.021+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5620735933e0 con 0x5620736ad0b0 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x5620736acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x5620736acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56568/0 (socket says 172.21.15.53:56568) 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 -- 172.21.15.53:0/4126179784 learned_addr learned my addr 172.21.15.53:0/4126179784 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ca6d0 0x56207367d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1d5230640 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ad0b0 0x562073681ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 -- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ad0b0 msgr2=0x562073681ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ad0b0 0x562073681ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 -- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ca6d0 msgr2=0x56207367d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ca6d0 0x56207367d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 -- 172.21.15.53:0/4126179784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620733c47c0 con 0x5620736ad7f0 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1cffff640 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x5620736acb70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc095da0 tx=0x7fe1bc060440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8a5d690dcf2e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1ba7fc640 1 -- 172.21.15.53:0/4126179784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc021040 con 0x5620736ad7f0 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1ba7fc640 1 -- 172.21.15.53:0/4126179784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc064f10 con 0x5620736ad7f0 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.022+0000 7fe1ba7fc640 1 -- 172.21.15.53:0/4126179784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc060070 con 0x5620736ad7f0 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 -- 172.21.15.53:0/4126179784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 msgr2=0x5620736acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x5620736acb70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc095da0 tx=0x7fe1bc060440 comp rx=0 tx=0).stop 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 -- 172.21.15.53:0/4126179784 shutdown_connections 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ad0b0 0x562073681ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x5620736acb70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/4126179784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ca6d0 0x56207367d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 -- 172.21.15.53:0/4126179784 >> 172.21.15.53:0/4126179784 conn(0x56207367f560 msgr2=0x5620736c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 -- 172.21.15.53:0/4126179784 shutdown_connections 2024-05-10T10:38:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 -- 172.21.15.53:0/4126179784 wait complete. 2024-05-10T10:38:41.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.023+0000 7fe1d99a9740 1 -- start start 2024-05-10T10:38:41.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.024+0000 7fe1d99a9740 1 Processor -- start 2024-05-10T10:38:41.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.024+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x56207369d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.024+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.024+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736aec30 0x5620736c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.024+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5620735eb440 con 0x5620736ad7f0 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.024+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5620735e6120 con 0x5620736aec30 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.024+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5620735f4210 con 0x5620736ca6d0 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d99a9740 10 client.0 _ll_register_callbacks cb 0x56207358b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x56207369d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x56207369d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56570/0 (socket says 172.21.15.53:56570) 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 -- 172.21.15.53:0/169887509 learned_addr learned my addr 172.21.15.53:0/169887509 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1cffff640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d5230640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736aec30 0x5620736c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 -- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 msgr2=0x5620736ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 -- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736aec30 msgr2=0x5620736c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736aec30 0x5620736c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 -- 172.21.15.53:0/169887509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562073424f50 con 0x5620736ad7f0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d5230640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736aec30 0x5620736c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x56207369d540 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe1c402a6f0 tx=0x7fe1c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7708e6cb76ae9bab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1af7fe640 1 -- 172.21.15.53:0/169887509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c404f020 con 0x5620736ad7f0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1af7fe640 1 -- 172.21.15.53:0/169887509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fe1c4003ce0 con 0x5620736ad7f0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1bc064f10 con 0x5620736ad7f0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1af7fe640 1 -- 172.21.15.53:0/169887509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe1c4037040 con 0x5620736ad7f0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d99a9740 10 client.6519 fetch_fsmap learned FSMap version 53 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.025+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5620732b80b0 con 0x5620736ad7f0 2024-05-10T10:38:41.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1af7fe640 1 -- 172.21.15.53:0/169887509 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc064f10 con 0x5620736ad7f0 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1af7fe640 1 -- 172.21.15.53:0/169887509 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7fe1c4054400 con 0x5620736ad7f0 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1d99a9740 10 client.6519 fetch_fsmap finished waiting for FSMap version 53 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1d99a9740 10 client.6519 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1d99a9740 20 client.6519 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736d79d0 0x5620736be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1d99a9740 4 client.6519 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620735f4a20 con 0x5620736d79d0 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1cffff640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736d79d0 0x5620736be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1cffff640 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736d79d0 0x5620736be900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf28ad027296a5 server_cookie=c22308238b4ddc63 in_seq=0 out_seq=0 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.026+0000 7fe1af7fe640 10 client.6519 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1af7fe640 1 -- 172.21.15.53:0/169887509 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620735f4a20 con 0x5620736d79d0 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1af7fe640 10 client.6519 handle_command_reply: tid=0 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 client.6519 shutdown 2024-05-10T10:38:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 msgr2=0x56207369d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x56207369d540 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe1c402a6f0 tx=0x7fe1c4004730 comp rx=0 tx=0).stop 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 shutdown_connections 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736d79d0 0x5620736be900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736aec30 0x5620736c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/169887509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ad7f0 0x56207369d540 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1af7fe640 0 client.6519 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 >> 172.21.15.53:0/169887509 conn(0x5620736ad0b0 msgr2=0x56207367ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 shutdown_connections 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 1 -- 172.21.15.53:0/169887509 wait complete. 2024-05-10T10:38:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.027+0000 7fe1d99a9740 20 client.6519 trim_cache size 0 max 16384 2024-05-10T10:38:41.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.041+0000 7fe1d99a9740 1 Processor -- start 2024-05-10T10:38:41.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.041+0000 7fe1d99a9740 1 -- start start 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x5620736a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736aec30 0x5620736a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0200df0 con 0x5620736aec30 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00c9b40 con 0x5620736ad7f0 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00b8b80 con 0x5620736ca6d0 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736aec30 0x5620736a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736aec30 0x5620736a7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56584/0 (socket says 172.21.15.53:56584) 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 -- 172.21.15.53:0/2967873335 learned_addr learned my addr 172.21.15.53:0/2967873335 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 -- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 msgr2=0x5620736c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x5620736a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d5230640 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 -- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 msgr2=0x5620736a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x5620736a85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 -- 172.21.15.53:0/2967873335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562073677d40 con 0x5620736aec30 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x5620736a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1d5230640 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1cffff640 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736aec30 0x5620736a7db0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc095e80 tx=0x7fe1bc082040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1b779eda27afdd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1b9ffb640 1 -- 172.21.15.53:0/2967873335 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc021040 con 0x5620736aec30 2024-05-10T10:38:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1b9ffb640 1 -- 172.21.15.53:0/2967873335 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc0820b0 con 0x5620736aec30 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.042+0000 7fe1b9ffb640 1 -- 172.21.15.53:0/2967873335 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc060070 con 0x5620736aec30 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 -- 172.21.15.53:0/2967873335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736aec30 msgr2=0x5620736a7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736aec30 0x5620736a7db0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc095e80 tx=0x7fe1bc082040 comp rx=0 tx=0).stop 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 -- 172.21.15.53:0/2967873335 shutdown_connections 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736ca6d0 0x5620736c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736aec30 0x5620736a7db0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/2967873335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x5620736a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 -- 172.21.15.53:0/2967873335 >> 172.21.15.53:0/2967873335 conn(0x5620736ad0b0 msgr2=0x56207369c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 -- 172.21.15.53:0/2967873335 shutdown_connections 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 -- 172.21.15.53:0/2967873335 wait complete. 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.043+0000 7fe1d99a9740 1 -- start start 2024-05-10T10:38:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.044+0000 7fe1d99a9740 1 Processor -- start 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.044+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x56207373af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.044+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736aec30 0x56207367c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.044+0000 7fe1d99a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ca6d0 0x56207367c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.044+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5620736d8a80 con 0x5620736ca6d0 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.044+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56207359ca20 con 0x5620736ad7f0 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d99a9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5620736d7e20 con 0x5620736aec30 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d99a9740 10 client.0 _ll_register_callbacks cb 0x7fe1bc009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ca6d0 0x56207367c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ca6d0 0x56207367c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56586/0 (socket says 172.21.15.53:56586) 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 -- 172.21.15.53:0/413697601 learned_addr learned my addr 172.21.15.53:0/413697601 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x56207373af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1cffff640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736aec30 0x56207367c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 -- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736aec30 msgr2=0x56207367c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736aec30 0x56207367c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 -- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 msgr2=0x56207373af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x56207373af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 -- 172.21.15.53:0/413697601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620736c11d0 con 0x5620736ca6d0 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1cffff640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736aec30 0x56207367c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d5230640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ca6d0 0x56207367c6e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8001f00 tx=0x7fe1c8007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633c9d0f33e5c15f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1af7fe640 1 -- 172.21.15.53:0/413697601 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8017460 con 0x5620736ca6d0 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1af7fe640 1 -- 172.21.15.53:0/413697601 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fe1c801a960 con 0x5620736ca6d0 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1bc0820b0 con 0x5620736ca6d0 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1af7fe640 1 -- 172.21.15.53:0/413697601 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fe1c801ab60 con 0x5620736ca6d0 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d99a9740 10 client.6525 fetch_fsmap learned FSMap version 53 2024-05-10T10:38:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.045+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x56207367d110 con 0x5620736ca6d0 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1af7fe640 1 -- 172.21.15.53:0/413697601 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc0820b0 con 0x5620736ca6d0 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1af7fe640 1 -- 172.21.15.53:0/413697601 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7fe1c801a120 con 0x5620736ca6d0 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 10 client.6525 fetch_fsmap finished waiting for FSMap version 53 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 10 client.6525 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 20 client.6525 populate_metadata read hostname 'smithi053' 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736ead80 0x56207373c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 4 client.6525 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 --> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5620735ee890 con 0x5620736ead80 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736ead80 0x56207373c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d4a2f640 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736ead80 0x56207373c310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8371ec554dc479 server_cookie=b0d17aac4e567ded in_seq=0 out_seq=0 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1af7fe640 10 client.6525 ms_handle_connect on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1af7fe640 1 -- 172.21.15.53:0/413697601 <== mds.0 v2:172.21.15.53:6832/3202950630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620735ee890 con 0x5620736ead80 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1af7fe640 10 client.6525 handle_command_reply: tid=0 2024-05-10T10:38:41.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 1 client.6525 shutdown 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ca6d0 msgr2=0x56207367c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.046+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ca6d0 0x56207367c6e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8001f00 tx=0x7fe1c8007490 comp rx=0 tx=0).stop 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 shutdown_connections 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:6832/3202950630,v1:172.21.15.53:6834/3202950630] conn(0x5620736ead80 0x56207373c310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620736ca6d0 0x56207367c6e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5620736aec30 0x56207367c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1af7fe640 0 client.6525 ms_handle_reset on v2:172.21.15.53:6832/3202950630 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 --2- 172.21.15.53:0/413697601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620736ad7f0 0x56207373af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 >> 172.21.15.53:0/413697601 conn(0x5620736ad0b0 msgr2=0x5620736c0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 shutdown_connections 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 1 -- 172.21.15.53:0/413697601 wait complete. 2024-05-10T10:38:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d99a9740 20 client.6525 trim_cache size 0 max 16384 2024-05-10T10:38:41.056 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:41.056 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1b40762d0 msgr2=0x7fe1b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1b40762d0 0x7fe1b4078790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4034000 tx=0x7fe1c402e000 comp rx=0 tx=0).stop 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d0f10 msgr2=0x7fe1d00cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d0f10 0x7fe1d00cd7c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9e0 tx=0x7fe1bc00eea0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 shutdown_connections 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe1b40762d0 0x7fe1b4078790 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1d015f410 0x7fe1d00d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00d0f10 0x7fe1d00cd7c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 --2- 172.21.15.53:0/4252364826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1d00b54d0 0x7fe1d00cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 >> 172.21.15.53:0/4252364826 conn(0x7fe1d008ed30 msgr2=0x7fe1d015e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 shutdown_connections 2024-05-10T10:38:41.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.047+0000 7fe1d6c93640 1 -- 172.21.15.53:0/4252364826 wait complete. 2024-05-10T10:38:41.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T10:38:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.160+0000 7fc25df46640 1 Processor -- start 2024-05-10T10:38:41.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.160+0000 7fc25df46640 1 -- start start 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc25df46640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2581608b0 0x7fc2580d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc25df46640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2580d1230 0x7fc2580d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc25df46640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580b2900 0x7fc2580b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc25df46640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc258160c90 con 0x7fc2580d1230 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc25df46640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc2580ce5f0 con 0x7fc2581608b0 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc25df46640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2580ce8d0 con 0x7fc2580b2900 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2580d1230 0x7fc2580d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2580d1230 0x7fc2580d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56596/0 (socket says 172.21.15.53:56596) 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 -- 172.21.15.53:0/753932461 learned_addr learned my addr 172.21.15.53:0/753932461 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc2577fe640 1 --2- 172.21.15.53:0/753932461 >> v2:172.21.15.135:3300/0 conn(0x7fc2581608b0 0x7fc2580d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc257fff640 1 --2- 172.21.15.53:0/753932461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580b2900 0x7fc2580b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 -- 172.21.15.53:0/753932461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580b2900 msgr2=0x7fc2580b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 --2- 172.21.15.53:0/753932461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580b2900 0x7fc2580b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 -- 172.21.15.53:0/753932461 >> v2:172.21.15.135:3300/0 conn(0x7fc2581608b0 msgr2=0x7fc2580d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 --2- 172.21.15.53:0/753932461 >> v2:172.21.15.135:3300/0 conn(0x7fc2581608b0 0x7fc2580d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc256ffd640 1 -- 172.21.15.53:0/753932461 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2580cebb0 con 0x7fc2580d1230 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc2577fe640 1 --2- 172.21.15.53:0/753932461 >> v2:172.21.15.135:3300/0 conn(0x7fc2581608b0 0x7fc2580d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.161+0000 7fc257fff640 1 --2- 172.21.15.53:0/753932461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580b2900 0x7fc2580b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.162+0000 7fc256ffd640 1 --2- 172.21.15.53:0/753932461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580d1230 0x7fc2580d16e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc258068c10 tx=0x7fc24c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7d7af410327332 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.162+0000 7fc2567fc640 1 -- 172.21.15.53:0/753932461 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c037070 con 0x7fc2580d1230 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.162+0000 7fc2567fc640 1 -- 172.21.15.53:0/753932461 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24c032440 con 0x7fc2580d1230 2024-05-10T10:38:41.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.162+0000 7fc2567fc640 1 -- 172.21.15.53:0/753932461 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c032810 con 0x7fc2580d1230 2024-05-10T10:38:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 -- 172.21.15.53:0/753932461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580d1230 msgr2=0x7fc2580d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 --2- 172.21.15.53:0/753932461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580d1230 0x7fc2580d16e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc258068c10 tx=0x7fc24c02a1e0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 -- 172.21.15.53:0/753932461 shutdown_connections 2024-05-10T10:38:41.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 --2- 172.21.15.53:0/753932461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580b2900 0x7fc2580b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 --2- 172.21.15.53:0/753932461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580d1230 0x7fc2580d16e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 --2- 172.21.15.53:0/753932461 >> v2:172.21.15.135:3300/0 conn(0x7fc2581608b0 0x7fc2580d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 -- 172.21.15.53:0/753932461 >> 172.21.15.53:0/753932461 conn(0x7fc2580d2df0 msgr2=0x7fc2580d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 -- 172.21.15.53:0/753932461 shutdown_connections 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.163+0000 7fc25df46640 1 -- 172.21.15.53:0/753932461 wait complete. 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.164+0000 7fc25df46640 1 Processor -- start 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.164+0000 7fc25df46640 1 -- start start 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc25df46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580b2900 0x7fc258157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc25df46640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580d1230 0x7fc2580b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc25df46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2581608b0 0x7fc2580b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc25df46640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc2580d1f10 con 0x7fc2580b2900 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc25df46640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2580d1d90 con 0x7fc2581608b0 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc25df46640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2580b32d0 con 0x7fc2580d1230 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580b2900 0x7fc258157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580b2900 0x7fc258157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56610/0 (socket says 172.21.15.53:56610) 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 -- 172.21.15.53:0/3379210248 learned_addr learned my addr 172.21.15.53:0/3379210248 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 -- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580d1230 msgr2=0x7fc2580b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc256ffd640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580d1230 0x7fc2580b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc257fff640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2581608b0 0x7fc2580b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580d1230 0x7fc2580b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 -- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2581608b0 msgr2=0x7fc2580b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2581608b0 0x7fc2580b7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.165+0000 7fc2577fe640 1 -- 172.21.15.53:0/3379210248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2580cbac0 con 0x7fc2580b2900 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.166+0000 7fc2577fe640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580b2900 0x7fc258157440 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c7f0 tx=0x7fc24800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.166+0000 7fc23ffff640 1 -- 172.21.15.53:0/3379210248 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248015070 con 0x7fc2580b2900 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.166+0000 7fc23ffff640 1 -- 172.21.15.53:0/3379210248 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc248007820 con 0x7fc2580b2900 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.166+0000 7fc23ffff640 1 -- 172.21.15.53:0/3379210248 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248007bd0 con 0x7fc2580b2900 2024-05-10T10:38:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.166+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2580cbd60 con 0x7fc2580b2900 2024-05-10T10:38:41.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.166+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2580cc320 con 0x7fc2580b2900 2024-05-10T10:38:41.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.167+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2580c7df0 con 0x7fc2580b2900 2024-05-10T10:38:41.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.168+0000 7fc23ffff640 1 -- 172.21.15.53:0/3379210248 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc248018070 con 0x7fc2580b2900 2024-05-10T10:38:41.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.168+0000 7fc23ffff640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2440762d0 0x7fc244078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.168+0000 7fc23ffff640 1 -- 172.21.15.53:0/3379210248 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fc2480978e0 con 0x7fc2580b2900 2024-05-10T10:38:41.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.169+0000 7fc256ffd640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2440762d0 0x7fc244078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.169+0000 7fc256ffd640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2440762d0 0x7fc244078790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc24c034000 tx=0x7fc24c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.173+0000 7fc23ffff640 1 -- 172.21.15.53:0/3379210248 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc248061060 con 0x7fc2580b2900 2024-05-10T10:38:41.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.402+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc25806a220 con 0x7fc2580b2900 2024-05-10T10:38:41.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.402+0000 7fc23ffff640 1 -- 172.21.15.53:0/3379210248 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc248060960 con 0x7fc2580b2900 2024-05-10T10:38:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:41.404 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T10:38:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2440762d0 msgr2=0x7fc244078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2440762d0 0x7fc244078790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc24c034000 tx=0x7fc24c02e000 comp rx=0 tx=0).stop 2024-05-10T10:38:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580b2900 msgr2=0x7fc258157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580b2900 0x7fc258157440 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c7f0 tx=0x7fc24800ccb0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 shutdown_connections 2024-05-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2440762d0 0x7fc244078790 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2581608b0 0x7fc2580b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2580d1230 0x7fc2580b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 --2- 172.21.15.53:0/3379210248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2580b2900 0x7fc258157440 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 >> 172.21.15.53:0/3379210248 conn(0x7fc2580d2df0 msgr2=0x7fc258071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 shutdown_connections 2024-05-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.404+0000 7fc25df46640 1 -- 172.21.15.53:0/3379210248 wait complete. 2024-05-10T10:38:41.416 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T10:38:41.417 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:38:41.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.519+0000 7ff13aebd640 1 Processor -- start 2024-05-10T10:38:41.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.519+0000 7ff13aebd640 1 -- start start 2024-05-10T10:38:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13aebd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1340d2270 0x7ff1340d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13aebd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1340b5e80 con 0x7ff134162dd0 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13aebd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1340d5c50 con 0x7ff1340d3980 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13aebd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff1340b5c30 con 0x7ff1340d2270 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13945a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1340d2270 0x7ff1340d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff133fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff134162dd0 0x7ff1340cfeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50154/0 (socket says 172.21.15.53:50154) 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff133fff640 1 -- 172.21.15.53:0/4243382855 learned_addr learned my addr 172.21.15.53:0/4243382855 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13945a640 1 -- 172.21.15.53:0/4243382855 >> v1:172.21.15.135:6789/0 conn(0x7ff1340d3980 legacy=0x7ff1341626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13945a640 1 -- 172.21.15.53:0/4243382855 >> v1:172.21.15.53:6789/0 conn(0x7ff134162dd0 legacy=0x7ff1340cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff13945a640 1 -- 172.21.15.53:0/4243382855 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1340d5f30 con 0x7ff1340d2270 2024-05-10T10:38:41.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.520+0000 7ff1337fe640 1 -- 172.21.15.53:0/4243382855 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076124581 0 0) 0x7ff1340b5e80 con 0x7ff134162dd0 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff13945a640 1 --2- 172.21.15.53:0/4243382855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1340d05c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009d90 tx=0x7ff12c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c097864d376062 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff1337fe640 1 -- 172.21.15.53:0/4243382855 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c007740 con 0x7ff1340d2270 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff1337fe640 1 -- 172.21.15.53:0/4243382855 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c0078e0 con 0x7ff1340d2270 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff1337fe640 1 -- 172.21.15.53:0/4243382855 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c007cb0 con 0x7ff1340d2270 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff13aebd640 1 -- 172.21.15.53:0/4243382855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 msgr2=0x7ff1340d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff13aebd640 1 --2- 172.21.15.53:0/4243382855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1340d05c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009d90 tx=0x7ff12c02a6a0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff13aebd640 1 -- 172.21.15.53:0/4243382855 shutdown_connections 2024-05-10T10:38:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff13aebd640 1 --2- 172.21.15.53:0/4243382855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1340d05c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.521+0000 7ff13aebd640 1 -- 172.21.15.53:0/4243382855 >> 172.21.15.53:0/4243382855 conn(0x7ff1340d5130 msgr2=0x7ff1340d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.522+0000 7ff13aebd640 1 -- 172.21.15.53:0/4243382855 shutdown_connections 2024-05-10T10:38:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.522+0000 7ff13aebd640 1 -- 172.21.15.53:0/4243382855 wait complete. 2024-05-10T10:38:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.523+0000 7ff13aebd640 1 Processor -- start 2024-05-10T10:38:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.523+0000 7ff13aebd640 1 -- start start 2024-05-10T10:38:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1341596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1340d3980 0x7ff13415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff134162dd0 0x7ff13415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b6890 con 0x7ff1340d2270 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340d31f0 con 0x7ff1340d3980 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff1340b6160 con 0x7ff134162dd0 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1341596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1341596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56638/0 (socket says 172.21.15.53:56638) 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 -- 172.21.15.53:0/2324305308 learned_addr learned my addr 172.21.15.53:0/2324305308 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 -- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff134162dd0 msgr2=0x7ff13415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff134162dd0 0x7ff13415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 -- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1340d3980 msgr2=0x7ff13415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1340d3980 0x7ff13415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 -- 172.21.15.53:0/2324305308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1340ccf80 con 0x7ff1340d2270 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff138c59640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1341596f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c460 tx=0x7ff12800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff1317fa640 1 -- 172.21.15.53:0/2324305308 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128016020 con 0x7ff1340d2270 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff1317fa640 1 -- 172.21.15.53:0/2324305308 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff128012870 con 0x7ff1340d2270 2024-05-10T10:38:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1340cd260 con 0x7ff1340d2270 2024-05-10T10:38:41.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff1317fa640 1 -- 172.21.15.53:0/2324305308 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128012c60 con 0x7ff1340d2270 2024-05-10T10:38:41.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.524+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1340cd820 con 0x7ff1340d2270 2024-05-10T10:38:41.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.525+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13406a220 con 0x7ff1340d2270 2024-05-10T10:38:41.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.526+0000 7ff1317fa640 1 -- 172.21.15.53:0/2324305308 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff128019020 con 0x7ff1340d2270 2024-05-10T10:38:41.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.526+0000 7ff1317fa640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1180762d0 0x7ff118078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.527+0000 7ff1317fa640 1 -- 172.21.15.53:0/2324305308 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff1280978a0 con 0x7ff1340d2270 2024-05-10T10:38:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.527+0000 7ff133fff640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1180762d0 0x7ff118078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.528+0000 7ff133fff640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1180762d0 0x7ff118078790 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff120005880 tx=0x7ff120007a00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:41.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.531+0000 7ff1317fa640 1 -- 172.21.15.53:0/2324305308 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff128020020 con 0x7ff1340d2270 2024-05-10T10:38:41.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.764+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff1340d1c60 con 0x7ff1340d2270 2024-05-10T10:38:41.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.764+0000 7ff1317fa640 1 -- 172.21.15.53:0/2324305308 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7ff128060930 con 0x7ff1340d2270 2024-05-10T10:38:41.766 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1180762d0 msgr2=0x7ff118078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1180762d0 0x7ff118078790 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff120005880 tx=0x7ff120007a00 comp rx=0 tx=0).stop 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 msgr2=0x7ff1341596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1341596f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c460 tx=0x7ff12800c920 comp rx=0 tx=0).stop 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 shutdown_connections 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1180762d0 0x7ff118078790 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff134162dd0 0x7ff13415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1340d3980 0x7ff13415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 --2- 172.21.15.53:0/2324305308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1340d2270 0x7ff1341596f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 >> 172.21.15.53:0/2324305308 conn(0x7ff1340d5130 msgr2=0x7ff1340d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 shutdown_connections 2024-05-10T10:38:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:41.766+0000 7ff13aebd640 1 -- 172.21.15.53:0/2324305308 wait complete. 2024-05-10T10:38:41.778 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:38:41.779 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:38:41.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:41.779 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:38:41.796 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:38:41.796 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:38:41.796 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:38:41.797 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42448sec preferred_lft 42448sec 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:38:41.798 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:38:41.799 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:38:41.799 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:38:41.799 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:38:41.799 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:38:41.799 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:38:41.799 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:38:41.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:41.799 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:38:41.860 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:38:41.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:41.862 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:38:41.919 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:38:41.919 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:38:41.919 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:38:41.919 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:38:41.919 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:38:41.919 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:38:41.920 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:38:41.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:38:41.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:38:41.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:38:41.920 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:38:42.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:38:42.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:38:42.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:42.050 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:38:42.066 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:38:42.067 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:38:42.067 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:38:42.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:42.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:38:42.133 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:38:42.133 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:38:42.189 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:38:42.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:38:42.190 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:38:42.245 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:38:42.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:42.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:38:42.330 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:38:42.330 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:38:42.330 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:38:42.339 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:38:42.339 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:38:42.339 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:38:42.339 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:38:42.339 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:38:42.344 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:38:42.344 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:38:42.344 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:38:42.344 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:38:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:38:42.347 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:38:42.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:42.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:38:42.413 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:38:42.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:38:42.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:38:42.511 INFO:teuthology.orchestra.run.smithi053.stdout:client6543 2024-05-10T10:38:42.512 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:38:42.512 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:42.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:42.528 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-10T10:38:42.584 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:38:42.585 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-10T10:38:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.670+0000 7f1cdd034f40 1 Processor -- start 2024-05-10T10:38:42.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.670+0000 7f1cdd034f40 1 -- start start 2024-05-10T10:38:42.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdd034f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cc80029c0 0x55a527565c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdd034f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5275497f0 0x55a527543d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdd034f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 0x55a527563390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdd034f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a527433f50 con 0x55a5275497f0 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdd034f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a527437f90 con 0x7f1cc80029c0 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdd034f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a527546e80 con 0x55a52754d0d0 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5275497f0 0x55a527543d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5275497f0 0x55a527543d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43562/0 (socket says 172.21.15.53:43562) 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 -- 172.21.15.53:0/4461737 learned_addr learned my addr 172.21.15.53:0/4461737 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdb331640 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 0x55a527563390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cdab30640 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cc80029c0 0x55a527565c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 -- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 msgr2=0x55a527563390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 0x55a527563390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 -- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cc80029c0 msgr2=0x55a527565c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cc80029c0 0x55a527565c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 -- 172.21.15.53:0/4461737 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a527560d70 con 0x55a5275497f0 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.671+0000 7f1cda32f640 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5275497f0 0x55a527543d70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40263a0 tx=0x7f1cc404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458425ca378a984b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cd932d640 1 -- 172.21.15.53:0/4461737 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4061450 con 0x55a5275497f0 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cd932d640 1 -- 172.21.15.53:0/4461737 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc4066070 con 0x55a5275497f0 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cd932d640 1 -- 172.21.15.53:0/4461737 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40617f0 con 0x55a5275497f0 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 -- 172.21.15.53:0/4461737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5275497f0 msgr2=0x55a527543d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5275497f0 0x55a527543d70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40263a0 tx=0x7f1cc404f000 comp rx=0 tx=0).stop 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 -- 172.21.15.53:0/4461737 shutdown_connections 2024-05-10T10:38:42.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 0x55a527563390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5275497f0 0x55a527543d70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/4461737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cc80029c0 0x55a527565c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 -- 172.21.15.53:0/4461737 >> 172.21.15.53:0/4461737 conn(0x55a52748b7d0 msgr2=0x55a527489e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 -- 172.21.15.53:0/4461737 shutdown_connections 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.672+0000 7f1cdd034f40 1 -- 172.21.15.53:0/4461737 wait complete. 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 Processor -- start 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 -- start start 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5275497f0 0x55a5275fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 0x55a5275fd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc80029c0 0x55a52754c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a5274b5690 con 0x7f1cc80029c0 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a527566430 con 0x55a5275497f0 2024-05-10T10:38:42.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdd034f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a5275442b0 con 0x55a52754d0d0 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdb331640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc80029c0 0x55a52754c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdb331640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc80029c0 0x55a52754c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43566/0 (socket says 172.21.15.53:43566) 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cdb331640 1 -- 172.21.15.53:0/956506162 learned_addr learned my addr 172.21.15.53:0/956506162 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.673+0000 7f1cda32f640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 0x55a5275fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdab30640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5275497f0 0x55a5275fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdb331640 1 -- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 msgr2=0x55a5275fd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdb331640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a52754d0d0 0x55a5275fd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdb331640 1 -- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5275497f0 msgr2=0x55a5275fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdb331640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5275497f0 0x55a5275fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdb331640 1 -- 172.21.15.53:0/956506162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5275faee0 con 0x7f1cc80029c0 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdb331640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc80029c0 0x55a52754c290 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x55a527561990 tx=0x7f1cd400d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cd3fff640 1 -- 172.21.15.53:0/956506162 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400de10 con 0x7f1cc80029c0 2024-05-10T10:38:42.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cd3fff640 1 -- 172.21.15.53:0/956506162 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd4017070 con 0x7f1cc80029c0 2024-05-10T10:38:42.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a5274b9470 con 0x7f1cc80029c0 2024-05-10T10:38:42.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cd3fff640 1 -- 172.21.15.53:0/956506162 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40126a0 con 0x7f1cc80029c0 2024-05-10T10:38:42.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.674+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5274b9a30 con 0x7f1cc80029c0 2024-05-10T10:38:42.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.676+0000 7f1cd3fff640 1 -- 172.21.15.53:0/956506162 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1cd4012840 con 0x7f1cc80029c0 2024-05-10T10:38:42.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.676+0000 7f1cd3fff640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1cbc074cb0 0x7f1cbc077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.676+0000 7f1cd3fff640 1 -- 172.21.15.53:0/956506162 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1cd40963e0 con 0x7f1cc80029c0 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.676+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55a5275431a0 0x55a5274ba510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.676+0000 7f1cdab30640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1cbc074cb0 0x7f1cbc077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.676+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75) -- 0x55a5274baa50 con 0x55a5275431a0 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.677+0000 7f1cda32f640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55a5275431a0 0x55a5274ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.677+0000 7f1cda32f640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55a5275431a0 0x55a5274ba510 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.677+0000 7f1cdab30640 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1cbc074cb0 0x7f1cbc077170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc005fd0 tx=0x7f1ccc007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cda32f640 1 -- 172.21.15.53:0/956506162 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v75'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1cc4061450 con 0x55a5275431a0 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55a5275431a0 msgr2=0x55a5274ba510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55a5275431a0 0x55a5274ba510 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1cbc074cb0 msgr2=0x7f1cbc077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1cbc074cb0 0x7f1cbc077170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc005fd0 tx=0x7f1ccc007040 comp rx=0 tx=0).stop 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc80029c0 msgr2=0x55a52754c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 --2- 172.21.15.53:0/956506162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc80029c0 0x55a52754c290 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x55a527561990 tx=0x7f1cd400d630 comp rx=0 tx=0).stop 2024-05-10T10:38:42.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdb331640 1 -- 172.21.15.53:0/956506162 reap_dead start 2024-05-10T10:38:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 shutdown_connections 2024-05-10T10:38:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 >> 172.21.15.53:0/956506162 conn(0x55a52748b7d0 msgr2=0x55a5274d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 shutdown_connections 2024-05-10T10:38:42.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.679+0000 7f1cdd034f40 1 -- 172.21.15.53:0/956506162 wait complete. 2024-05-10T10:38:42.685 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:38:42.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:38:42.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:38:42.777 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:38:42.777 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:38:42.778 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:38:42.778 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:42.794 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:38:42.794 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:42.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:38:42.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.993+0000 7fa313254640 1 Processor -- start 2024-05-10T10:38:42.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.993+0000 7fa313254640 1 -- start start 2024-05-10T10:38:42.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa313254640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa30c06c990 0x7fa30c163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa313254640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa30c15f620 0x7fa30c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:42.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa313254640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa30c163ba0 con 0x7fa30c0704c0 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa313254640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa30c06cd70 con 0x7fa30c15f620 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa313254640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa30c160190 con 0x7fa30c06c990 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa303fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa30c15f620 0x7fa30c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa3117f1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa30c0704c0 0x7fa30c070920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58306/0 (socket says 172.21.15.53:58306) 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa3117f1640 1 -- 172.21.15.53:0/987338868 learned_addr learned my addr 172.21.15.53:0/987338868 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa303fff640 1 -- 172.21.15.53:0/987338868 >> v2:172.21.15.135:3300/0 conn(0x7fa30c06c990 msgr2=0x7fa30c163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa310ff0640 1 --2- 172.21.15.53:0/987338868 >> v2:172.21.15.135:3300/0 conn(0x7fa30c06c990 0x7fa30c163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa303fff640 1 --2- 172.21.15.53:0/987338868 >> v2:172.21.15.135:3300/0 conn(0x7fa30c06c990 0x7fa30c163660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:42.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa3037fe640 1 -- 172.21.15.53:0/987338868 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226058937 0 0) 0x7fa30c163ba0 con 0x7fa30c0704c0 2024-05-10T10:38:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa303fff640 1 -- 172.21.15.53:0/987338868 >> v1:172.21.15.53:6789/0 conn(0x7fa30c0704c0 legacy=0x7fa30c070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.994+0000 7fa303fff640 1 -- 172.21.15.53:0/987338868 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c160470 con 0x7fa30c15f620 2024-05-10T10:38:42.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.995+0000 7fa303fff640 1 --2- 172.21.15.53:0/987338868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c15fab0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa30c06cf10 tx=0x7fa2fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d55e405ce9a1dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.995+0000 7fa3037fe640 1 -- 172.21.15.53:0/987338868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc037070 con 0x7fa30c15f620 2024-05-10T10:38:43.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.995+0000 7fa3037fe640 1 -- 172.21.15.53:0/987338868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2fc032440 con 0x7fa30c15f620 2024-05-10T10:38:43.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.995+0000 7fa3037fe640 1 -- 172.21.15.53:0/987338868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc032810 con 0x7fa30c15f620 2024-05-10T10:38:43.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 -- 172.21.15.53:0/987338868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 msgr2=0x7fa30c15fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 --2- 172.21.15.53:0/987338868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c15fab0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa30c06cf10 tx=0x7fa2fc02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 -- 172.21.15.53:0/987338868 shutdown_connections 2024-05-10T10:38:43.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 --2- 172.21.15.53:0/987338868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c15fab0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 --2- 172.21.15.53:0/987338868 >> v2:172.21.15.135:3300/0 conn(0x7fa30c06c990 0x7fa30c163660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 -- 172.21.15.53:0/987338868 >> 172.21.15.53:0/987338868 conn(0x7fa30c0b3980 msgr2=0x7fa30c0b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 -- 172.21.15.53:0/987338868 shutdown_connections 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.996+0000 7fa313254640 1 -- 172.21.15.53:0/987338868 wait complete. 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.997+0000 7fa313254640 1 Processor -- start 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.997+0000 7fa313254640 1 -- start start 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa313254640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa30c06c990 0x7fa30c1f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa313254640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa30c0704c0 0x7fa30c15e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa313254640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c14c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa313254640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c073e70 con 0x7fa30c15f620 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa313254640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c163140 con 0x7fa30c06c990 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa313254640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c0cf920 con 0x7fa30c0704c0 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c14c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c14c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43584/0 (socket says 172.21.15.53:43584) 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 -- 172.21.15.53:0/1535910503 learned_addr learned my addr 172.21.15.53:0/1535910503 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa310ff0640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa30c06c990 0x7fa30c1f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa303fff640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa30c0704c0 0x7fa30c15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 -- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa30c0704c0 msgr2=0x7fa30c15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa30c0704c0 0x7fa30c15e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 -- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa30c06c990 msgr2=0x7fa30c1f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa30c06c990 0x7fa30c1f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 -- 172.21.15.53:0/1535910503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c0b4790 con 0x7fa30c15f620 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa303fff640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa30c0704c0 0x7fa30c15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa310ff0640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa30c06c990 0x7fa30c1f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.998+0000 7fa3117f1640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c14c400 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa30400caf0 tx=0x7fa30400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.999+0000 7fa3017fa640 1 -- 172.21.15.53:0/1535910503 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30400aca0 con 0x7fa30c15f620 2024-05-10T10:38:43.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.999+0000 7fa3017fa640 1 -- 172.21.15.53:0/1535910503 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30400ae40 con 0x7fa30c15f620 2024-05-10T10:38:43.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.999+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c15c4c0 con 0x7fa30c15f620 2024-05-10T10:38:43.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.999+0000 7fa3017fa640 1 -- 172.21.15.53:0/1535910503 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304012620 con 0x7fa30c15f620 2024-05-10T10:38:43.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:42.999+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c0b4a30 con 0x7fa30c15f620 2024-05-10T10:38:43.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.000+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30c06a220 con 0x7fa30c15f620 2024-05-10T10:38:43.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.000+0000 7fa3017fa640 1 -- 172.21.15.53:0/1535910503 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa304012880 con 0x7fa30c15f620 2024-05-10T10:38:43.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.001+0000 7fa3017fa640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa2f80762d0 0x7fa2f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.001+0000 7fa3017fa640 1 -- 172.21.15.53:0/1535910503 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fa304097e40 con 0x7fa30c15f620 2024-05-10T10:38:43.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.001+0000 7fa310ff0640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa2f80762d0 0x7fa2f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.002+0000 7fa310ff0640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa2f80762d0 0x7fa2f8078790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa30800a9d0 tx=0x7fa308008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.005+0000 7fa3017fa640 1 -- 172.21.15.53:0/1535910503 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa3040615b0 con 0x7fa30c15f620 2024-05-10T10:38:43.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.208+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa30c0c7db0 con 0x7fa30c15f620 2024-05-10T10:38:43.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.208+0000 7fa3017fa640 1 -- 172.21.15.53:0/1535910503 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v75) ==== 80+0+122 (secure 0 0 0) 0x7fa304060eb0 con 0x7fa30c15f620 2024-05-10T10:38:43.210 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6834/1984524370 2024-05-11T10:37:31.312633+0000 2024-05-10T10:38:43.210 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6832/1984524370 2024-05-11T10:37:31.312633+0000 2024-05-10T10:38:43.211 INFO:teuthology.orchestra.run.smithi053.stderr:listed 2 entries 2024-05-10T10:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa2f80762d0 msgr2=0x7fa2f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa2f80762d0 0x7fa2f8078790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa30800a9d0 tx=0x7fa308008040 comp rx=0 tx=0).stop 2024-05-10T10:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 msgr2=0x7fa30c14c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c14c400 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa30400caf0 tx=0x7fa30400a060 comp rx=0 tx=0).stop 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 shutdown_connections 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa2f80762d0 0x7fa2f8078790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa30c15f620 0x7fa30c14c400 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa30c0704c0 0x7fa30c15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 --2- 172.21.15.53:0/1535910503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa30c06c990 0x7fa30c1f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 >> 172.21.15.53:0/1535910503 conn(0x7fa30c0b3980 msgr2=0x7fa30c0c71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 shutdown_connections 2024-05-10T10:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.211+0000 7fa313254640 1 -- 172.21.15.53:0/1535910503 wait complete. 2024-05-10T10:38:43.223 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:38:43.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:43.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:38:43.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:38:43.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:38:43.252 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:38:43.308 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:38:43.308 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:38:43.309 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:43.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:38:43.329 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:38:43.329 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:38:43.329 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:43.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:38:43.365 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:38:43.365 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:38:43.365 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:43.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:38:43.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:38:43.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.506+0000 7fed8ab89640 1 Processor -- start 2024-05-10T10:38:43.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.507+0000 7fed8ab89640 1 -- start start 2024-05-10T10:38:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.507+0000 7fed8ab89640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 0x7fed84071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.507+0000 7fed8ab89640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed84072330 con 0x7fed84071880 2024-05-10T10:38:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.507+0000 7fed8ab89640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed84071d70 con 0x7fed840d3510 2024-05-10T10:38:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.507+0000 7fed8ab89640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fed84072050 con 0x7fed8415d290 2024-05-10T10:38:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.507+0000 7fed83fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fed84071880 0x7fed840d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58312/0 (socket says 172.21.15.53:58312) 2024-05-10T10:38:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.507+0000 7fed83fff640 1 -- 172.21.15.53:0/3417236418 learned_addr learned my addr 172.21.15.53:0/3417236418 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532189464 0 0) 0x7fed84072330 con 0x7fed84071880 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed70003520 con 0x7fed84071880 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 0x7fed84071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4276952998 0 0) 0x7fed70003520 con 0x7fed84071880 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 msgr2=0x7fed84071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 0x7fed84071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 >> v1:172.21.15.135:6789/0 conn(0x7fed840d3510 legacy=0x7fed840ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed840cda70 con 0x7fed84071880 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fed740019f0 con 0x7fed84071880 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 0x7fed84071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 >> v1:172.21.15.53:6789/0 conn(0x7fed84071880 legacy=0x7fed840d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed700039e0 0x7fed70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed70004800 0x7fed70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 0x7fed700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed84072050 con 0x7fed700039e0 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed7000bac0 con 0x7fed70004800 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fed7000bcc0 con 0x7fed8415d290 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed700039e0 0x7fed70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed83fff640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed70004800 0x7fed70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed89126640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 0x7fed700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 -- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 msgr2=0x7fed700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8415d290 0x7fed700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 -- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed70004800 msgr2=0x7fed70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed70004800 0x7fed70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed88925640 1 -- 172.21.15.53:0/3417236418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed70003520 con 0x7fed700039e0 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.508+0000 7fed83fff640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed70004800 0x7fed70004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.509+0000 7fed88925640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed700039e0 0x7fed70003e00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098a0 tx=0x7fed70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15114aa77033a22e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.509+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00f040 con 0x7fed700039e0 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.509+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c02ace0 con 0x7fed700039e0 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.509+0000 7fed837fe640 1 -- 172.21.15.53:0/3417236418 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c004250 con 0x7fed700039e0 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.510+0000 7fed8ab89640 1 -- 172.21.15.53:0/3417236418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed700039e0 msgr2=0x7fed70003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.510+0000 7fed8ab89640 1 --2- 172.21.15.53:0/3417236418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed700039e0 0x7fed70003e00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098a0 tx=0x7fed70005830 comp rx=0 tx=0).stop 2024-05-10T10:38:43.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.510+0000 7fed89126640 1 -- 172.21.15.53:0/3417236418 reap_dead start 2024-05-10T10:38:43.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.510+0000 7fed8ab89640 1 -- 172.21.15.53:0/3417236418 shutdown_connections 2024-05-10T10:38:43.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.510+0000 7fed8ab89640 1 -- 172.21.15.53:0/3417236418 >> 172.21.15.53:0/3417236418 conn(0x7fed8409ba20 msgr2=0x7fed8409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:43.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.510+0000 7fed8ab89640 1 -- 172.21.15.53:0/3417236418 shutdown_connections 2024-05-10T10:38:43.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.510+0000 7fed8ab89640 1 -- 172.21.15.53:0/3417236418 wait complete. 2024-05-10T10:38:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.511+0000 7fed8ab89640 1 Processor -- start 2024-05-10T10:38:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.511+0000 7fed8ab89640 1 -- start start 2024-05-10T10:38:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed8ab89640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed700039e0 0x7fed84156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed8ab89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8406e160 0x7fed840cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed8ab89640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8406e5e0 0x7fed840cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed8ab89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed840d1c20 con 0x7fed8406e160 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed8ab89640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed840d1aa0 con 0x7fed700039e0 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed8ab89640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fed840b5e80 con 0x7fed8406e5e0 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8406e160 0x7fed840cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8406e160 0x7fed840cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43604/0 (socket says 172.21.15.53:43604) 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 -- 172.21.15.53:0/3328183241 learned_addr learned my addr 172.21.15.53:0/3328183241 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed88925640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed700039e0 0x7fed84156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed89126640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8406e5e0 0x7fed840cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 -- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8406e5e0 msgr2=0x7fed840cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8406e5e0 0x7fed840cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 -- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed700039e0 msgr2=0x7fed84156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed700039e0 0x7fed84156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.512+0000 7fed83fff640 1 -- 172.21.15.53:0/3328183241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed840b6160 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.513+0000 7fed83fff640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8406e160 0x7fed840cead0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fed74007ab0 tx=0x7fed7400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.513+0000 7fed817fa640 1 -- 172.21.15.53:0/3328183241 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74013420 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.513+0000 7fed817fa640 1 -- 172.21.15.53:0/3328183241 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed74018070 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.513+0000 7fed817fa640 1 -- 172.21.15.53:0/3328183241 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740137c0 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.513+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed840b6440 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.513+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed840b6a00 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.514+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed840b6c00 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.514+0000 7fed817fa640 1 -- 172.21.15.53:0/3328183241 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fed74013960 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.515+0000 7fed817fa640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fed640762d0 0x7fed64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.515+0000 7fed817fa640 1 -- 172.21.15.53:0/3328183241 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fed74097ed0 con 0x7fed8406e160 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.515+0000 7fed88925640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fed640762d0 0x7fed64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.516+0000 7fed88925640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fed640762d0 0x7fed64078790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed7c033000 tx=0x7fed7c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.519+0000 7fed817fa640 1 -- 172.21.15.53:0/3328183241 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fed840b6c00 con 0x7fed8406e160 2024-05-10T10:38:43.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.747+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed840693f0 con 0x7fed8406e160 2024-05-10T10:38:43.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.747+0000 7fed817fa640 1 -- 172.21.15.53:0/3328183241 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7fed740616e0 con 0x7fed8406e160 2024-05-10T10:38:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:43.749 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:38:14.328418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":5}]} 2024-05-10T10:38:43.750 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2024-05-10T10:38:43.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fed640762d0 msgr2=0x7fed64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fed640762d0 0x7fed64078790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed7c033000 tx=0x7fed7c02e000 comp rx=0 tx=0).stop 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8406e160 msgr2=0x7fed840cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8406e160 0x7fed840cead0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fed74007ab0 tx=0x7fed7400e530 comp rx=0 tx=0).stop 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 shutdown_connections 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fed640762d0 0x7fed64078790 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed8406e5e0 0x7fed840cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8406e160 0x7fed840cead0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 --2- 172.21.15.53:0/3328183241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed700039e0 0x7fed84156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 >> 172.21.15.53:0/3328183241 conn(0x7fed8409ba20 msgr2=0x7fed8409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 shutdown_connections 2024-05-10T10:38:43.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.750+0000 7fed8ab89640 1 -- 172.21.15.53:0/3328183241 wait complete. 2024-05-10T10:38:43.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:38:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.864+0000 7f06dba46640 1 Processor -- start 2024-05-10T10:38:43.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.865+0000 7f06dba46640 1 -- start start 2024-05-10T10:38:43.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.865+0000 7f06dba46640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f06d415f3b0 0x7f06d40d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.865+0000 7f06dba46640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d415f790 0x7f06d40d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.865+0000 7f06dba46640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f06d40d5d30 0x7f06d40c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.865+0000 7f06dba46640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f06d40d30b0 con 0x7f06d40d5d30 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.865+0000 7f06dba46640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f06d40d3390 con 0x7f06d415f3b0 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.865+0000 7f06dba46640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f06d40c8c80 con 0x7f06d415f790 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f06d40d5d30 0x7f06d40c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f06d40d5d30 0x7f06d40c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43618/0 (socket says 172.21.15.53:43618) 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 -- 172.21.15.53:0/415992921 learned_addr learned my addr 172.21.15.53:0/415992921 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d97e2640 1 --2- 172.21.15.53:0/415992921 >> v2:172.21.15.135:3300/0 conn(0x7f06d415f3b0 0x7f06d40d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d8fe1640 1 --2- 172.21.15.53:0/415992921 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d415f790 0x7f06d40d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 -- 172.21.15.53:0/415992921 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d415f790 msgr2=0x7f06d40d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:43.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 --2- 172.21.15.53:0/415992921 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d415f790 0x7f06d40d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 -- 172.21.15.53:0/415992921 >> v2:172.21.15.135:3300/0 conn(0x7f06d415f3b0 msgr2=0x7f06d40d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:43.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 --2- 172.21.15.53:0/415992921 >> v2:172.21.15.135:3300/0 conn(0x7f06d415f3b0 0x7f06d40d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d9fe3640 1 -- 172.21.15.53:0/415992921 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d40c8f60 con 0x7f06d40d5d30 2024-05-10T10:38:43.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.866+0000 7f06d97e2640 1 --2- 172.21.15.53:0/415992921 >> v2:172.21.15.135:3300/0 conn(0x7f06d415f3b0 0x7f06d40d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:43.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06d9fe3640 1 --2- 172.21.15.53:0/415992921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d40d5d30 0x7f06d40c9760 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f06d408e820 tx=0x7f06cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed94578c15b1b84d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06cbfff640 1 -- 172.21.15.53:0/415992921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc036070 con 0x7f06d40d5d30 2024-05-10T10:38:43.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06cbfff640 1 -- 172.21.15.53:0/415992921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc031600 con 0x7f06d40d5d30 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06cbfff640 1 -- 172.21.15.53:0/415992921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0319d0 con 0x7f06d40d5d30 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06dba46640 1 -- 172.21.15.53:0/415992921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d40d5d30 msgr2=0x7f06d40c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06dba46640 1 --2- 172.21.15.53:0/415992921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d40d5d30 0x7f06d40c9760 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f06d408e820 tx=0x7f06cc02a6a0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06dba46640 1 -- 172.21.15.53:0/415992921 shutdown_connections 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06dba46640 1 --2- 172.21.15.53:0/415992921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d40d5d30 0x7f06d40c9760 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06dba46640 1 --2- 172.21.15.53:0/415992921 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d415f790 0x7f06d40d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06dba46640 1 --2- 172.21.15.53:0/415992921 >> v2:172.21.15.135:3300/0 conn(0x7f06d415f3b0 0x7f06d40d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.867+0000 7f06dba46640 1 -- 172.21.15.53:0/415992921 >> 172.21.15.53:0/415992921 conn(0x7f06d40d5130 msgr2=0x7f06d40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.868+0000 7f06dba46640 1 -- 172.21.15.53:0/415992921 shutdown_connections 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.868+0000 7f06dba46640 1 -- 172.21.15.53:0/415992921 wait complete. 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 Processor -- start 2024-05-10T10:38:43.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 -- start start 2024-05-10T10:38:43.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d40d5d30 0x7f06d41f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06d415f3b0 0x7f06d41f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d41575a0 0x7f06d4154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f06d40c9e20 con 0x7f06d41575a0 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06d40c9ca0 con 0x7f06d415f3b0 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.869+0000 7f06dba46640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f06d40c9fa0 con 0x7f06d40d5d30 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d41575a0 0x7f06d4154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d41575a0 0x7f06d4154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43620/0 (socket says 172.21.15.53:43620) 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 -- 172.21.15.53:0/601700913 learned_addr learned my addr 172.21.15.53:0/601700913 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d8fe1640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06d415f3b0 0x7f06d41f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d97e2640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d40d5d30 0x7f06d41f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 -- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d40d5d30 msgr2=0x7f06d41f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d40d5d30 0x7f06d41f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 -- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06d415f3b0 msgr2=0x7f06d41f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06d415f3b0 0x7f06d41f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:43.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 -- 172.21.15.53:0/601700913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d41f4930 con 0x7f06d41575a0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d8fe1640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06d415f3b0 0x7f06d41f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06d9fe3640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d41575a0 0x7f06d4154f40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009f40 tx=0x7f06cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06c9ffb640 1 -- 172.21.15.53:0/601700913 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc036050 con 0x7f06d41575a0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06c9ffb640 1 -- 172.21.15.53:0/601700913 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc042070 con 0x7f06d41575a0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d4157b60 con 0x7f06d41575a0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06c9ffb640 1 -- 172.21.15.53:0/601700913 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc009510 con 0x7f06d41575a0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.870+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d41f8120 con 0x7f06d41575a0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.871+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d406a220 con 0x7f06d41575a0 2024-05-10T10:38:43.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.872+0000 7f06c9ffb640 1 -- 172.21.15.53:0/601700913 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f06cc007bd0 con 0x7f06d41575a0 2024-05-10T10:38:43.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.873+0000 7f06c9ffb640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06b80762d0 0x7f06b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:43.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.873+0000 7f06c9ffb640 1 -- 172.21.15.53:0/601700913 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f06cc0b87c0 con 0x7f06d41575a0 2024-05-10T10:38:43.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.873+0000 7f06d97e2640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06b80762d0 0x7f06b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:43.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.873+0000 7f06d97e2640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06b80762d0 0x7f06b8078790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006fd0 tx=0x7f06d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:43.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:43.877+0000 7f06c9ffb640 1 -- 172.21.15.53:0/601700913 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f06cc081f30 con 0x7f06d41575a0 2024-05-10T10:38:44.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.106+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d40d1c00 con 0x7f06d41575a0 2024-05-10T10:38:44.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.107+0000 7f06c9ffb640 1 -- 172.21.15.53:0/601700913 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7f06cc081830 con 0x7f06d41575a0 2024-05-10T10:38:44.109 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:44.109 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:38:14.328418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":5}]} 2024-05-10T10:38:44.109 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06b80762d0 msgr2=0x7f06b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06b80762d0 0x7f06b8078790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006fd0 tx=0x7f06d0008040 comp rx=0 tx=0).stop 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d41575a0 msgr2=0x7f06d4154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d41575a0 0x7f06d4154f40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009f40 tx=0x7f06cc004a40 comp rx=0 tx=0).stop 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 shutdown_connections 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f06b80762d0 0x7f06b8078790 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06d41575a0 0x7f06d4154f40 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06d415f3b0 0x7f06d41f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 --2- 172.21.15.53:0/601700913 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06d40d5d30 0x7f06d41f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 >> 172.21.15.53:0/601700913 conn(0x7f06d40d5130 msgr2=0x7f06d415e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 shutdown_connections 2024-05-10T10:38:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.109+0000 7f06dba46640 1 -- 172.21.15.53:0/601700913 wait complete. 2024-05-10T10:38:44.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:38:44.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.224+0000 7f83c759e640 1 Processor -- start 2024-05-10T10:38:44.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.224+0000 7f83c759e640 1 -- start start 2024-05-10T10:38:44.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83c815f1f0 0x7f83c80d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c759e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83c80d5a00 0x7f83c80d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c759e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d3a00 0x7f83c80d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f83c80d2ff0 con 0x7f83c815f1f0 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c759e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f83c80d5e90 con 0x7f83c80d5a00 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c759e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83c80d3e20 con 0x7f83c80d3a00 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83c815f1f0 0x7f83c80d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83c815f1f0 0x7f83c80d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43630/0 (socket says 172.21.15.53:43630) 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 -- 172.21.15.53:0/1070774244 learned_addr learned my addr 172.21.15.53:0/1070774244 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 -- 172.21.15.53:0/1070774244 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d3a00 msgr2=0x7f83c80d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 --2- 172.21.15.53:0/1070774244 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d3a00 0x7f83c80d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 -- 172.21.15.53:0/1070774244 >> v2:172.21.15.135:3300/0 conn(0x7f83c80d5a00 msgr2=0x7f83c80d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:44.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83bdd9b640 1 --2- 172.21.15.53:0/1070774244 >> v2:172.21.15.135:3300/0 conn(0x7f83c80d5a00 0x7f83c80d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 --2- 172.21.15.53:0/1070774244 >> v2:172.21.15.135:3300/0 conn(0x7f83c80d5a00 0x7f83c80d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83c659c640 1 -- 172.21.15.53:0/1070774244 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c80d0b90 con 0x7f83c815f1f0 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.225+0000 7f83bdd9b640 1 --2- 172.21.15.53:0/1070774244 >> v2:172.21.15.135:3300/0 conn(0x7f83c80d5a00 0x7f83c80d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.226+0000 7f83c659c640 1 --2- 172.21.15.53:0/1070774244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c815f1f0 0x7f83c80d0500 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f83c8162cf0 tx=0x7f83b802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f148cf6c014eeff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.226+0000 7f83c5d9b640 1 -- 172.21.15.53:0/1070774244 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8037070 con 0x7f83c815f1f0 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.226+0000 7f83c5d9b640 1 -- 172.21.15.53:0/1070774244 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b8032440 con 0x7f83c815f1f0 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.226+0000 7f83c5d9b640 1 -- 172.21.15.53:0/1070774244 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8032810 con 0x7f83c815f1f0 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 -- 172.21.15.53:0/1070774244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c815f1f0 msgr2=0x7f83c80d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 --2- 172.21.15.53:0/1070774244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c815f1f0 0x7f83c80d0500 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f83c8162cf0 tx=0x7f83b802a140 comp rx=0 tx=0).stop 2024-05-10T10:38:44.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 -- 172.21.15.53:0/1070774244 shutdown_connections 2024-05-10T10:38:44.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 --2- 172.21.15.53:0/1070774244 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d3a00 0x7f83c80d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 --2- 172.21.15.53:0/1070774244 >> v2:172.21.15.135:3300/0 conn(0x7f83c80d5a00 0x7f83c80d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 --2- 172.21.15.53:0/1070774244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c815f1f0 0x7f83c80d0500 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 -- 172.21.15.53:0/1070774244 >> 172.21.15.53:0/1070774244 conn(0x7f83c80d4ee0 msgr2=0x7f83c80d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:44.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 -- 172.21.15.53:0/1070774244 shutdown_connections 2024-05-10T10:38:44.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.227+0000 7f83c759e640 1 -- 172.21.15.53:0/1070774244 wait complete. 2024-05-10T10:38:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.228+0000 7f83c759e640 1 Processor -- start 2024-05-10T10:38:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.228+0000 7f83c759e640 1 -- start start 2024-05-10T10:38:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c80d3a00 0x7f83c8153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c759e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d5a00 0x7f83c80ba780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83c815f1f0 0x7f83c8162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83c815f5d0 con 0x7f83c80d3a00 2024-05-10T10:38:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c759e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83c80b4750 con 0x7f83c815f1f0 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c759e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83c80d1a50 con 0x7f83c80d5a00 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c80d3a00 0x7f83c8153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c80d3a00 0x7f83c8153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43642/0 (socket says 172.21.15.53:43642) 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 -- 172.21.15.53:0/2822420385 learned_addr learned my addr 172.21.15.53:0/2822420385 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c6d9d640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83c815f1f0 0x7f83c8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83bdd9b640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d5a00 0x7f83c80ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 -- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d5a00 msgr2=0x7f83c80ba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d5a00 0x7f83c80ba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 -- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83c815f1f0 msgr2=0x7f83c8162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83c815f1f0 0x7f83c8162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.229+0000 7f83c659c640 1 -- 172.21.15.53:0/2822420385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c8161160 con 0x7f83c80d3a00 2024-05-10T10:38:44.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.230+0000 7f83c659c640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c80d3a00 0x7f83c8153360 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.230+0000 7f83bf7fe640 1 -- 172.21.15.53:0/2822420385 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8041020 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.230+0000 7f83bf7fe640 1 -- 172.21.15.53:0/2822420385 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b8030970 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.230+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c8161440 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.230+0000 7f83bf7fe640 1 -- 172.21.15.53:0/2822420385 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8037040 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.230+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c8161a00 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.231+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c806a220 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.231+0000 7f83bf7fe640 1 -- 172.21.15.53:0/2822420385 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f83b8044020 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.232+0000 7f83bf7fe640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83a80762d0 0x7f83a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.232+0000 7f83bf7fe640 1 -- 172.21.15.53:0/2822420385 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f83b80b8170 con 0x7f83c80d3a00 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.232+0000 7f83bdd9b640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83a80762d0 0x7f83a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.233+0000 7f83bdd9b640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83a80762d0 0x7f83a8078790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f83c80d1630 tx=0x7f83b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.236+0000 7f83bf7fe640 1 -- 172.21.15.53:0/2822420385 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f83b804b030 con 0x7f83c80d3a00 2024-05-10T10:38:44.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.431+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83c80d1380 con 0x7f83c80d3a00 2024-05-10T10:38:44.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.432+0000 7f83bf7fe640 1 -- 172.21.15.53:0/2822420385 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17074 (secure 0 0 0) 0x7f83b8081980 con 0x7f83c80d3a00 2024-05-10T10:38:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:44.434 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":75,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:37:31.312656+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:37:15.291160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T10:37:16.547933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6834/1984524370":"2024-05-11T10:37:31.312633+0000","172.21.15.53:6832/1984524370":"2024-05-11T10:37:31.312633+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.434+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83a80762d0 msgr2=0x7f83a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.434+0000 7f83c759e640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83a80762d0 0x7f83a8078790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f83c80d1630 tx=0x7f83b0008040 comp rx=0 tx=0).stop 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.434+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c80d3a00 msgr2=0x7f83c8153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.434+0000 7f83c759e640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c80d3a00 0x7f83c8153360 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b80046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 shutdown_connections 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83a80762d0 0x7f83a8078790 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83c815f1f0 0x7f83c8162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83c80d5a00 0x7f83c80ba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 --2- 172.21.15.53:0/2822420385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c80d3a00 0x7f83c8153360 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 >> 172.21.15.53:0/2822420385 conn(0x7f83c80d4ee0 msgr2=0x7f83c80b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:44.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 shutdown_connections 2024-05-10T10:38:44.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.435+0000 7f83c759e640 1 -- 172.21.15.53:0/2822420385 wait complete. 2024-05-10T10:38:44.448 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:38:44.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:38:44.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.550+0000 7fbf370e1640 1 Processor -- start 2024-05-10T10:38:44.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.550+0000 7fbf370e1640 1 -- start start 2024-05-10T10:38:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf370e1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf301598c0 0x7fbf3006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf370e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3006ec40 0x7fbf300c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf370e1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbf300c71c0 0x7fbf3015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf370e1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3006f1c0 con 0x7fbf301598c0 2024-05-10T10:38:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf370e1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3006f4a0 con 0x7fbf300c71c0 2024-05-10T10:38:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf370e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3006f780 con 0x7fbf3006ec40 2024-05-10T10:38:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf301598c0 0x7fbf3006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf301598c0 0x7fbf3006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43656/0 (socket says 172.21.15.53:43656) 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 -- 172.21.15.53:0/3664736696 learned_addr learned my addr 172.21.15.53:0/3664736696 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf3567e640 1 --2- 172.21.15.53:0/3664736696 >> v2:172.21.15.135:3300/0 conn(0x7fbf300c71c0 0x7fbf3015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 -- 172.21.15.53:0/3664736696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3006ec40 msgr2=0x7fbf300c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf2ffff640 1 --2- 172.21.15.53:0/3664736696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3006ec40 0x7fbf300c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 --2- 172.21.15.53:0/3664736696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3006ec40 0x7fbf300c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 -- 172.21.15.53:0/3664736696 >> v2:172.21.15.135:3300/0 conn(0x7fbf300c71c0 msgr2=0x7fbf3015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 --2- 172.21.15.53:0/3664736696 >> v2:172.21.15.135:3300/0 conn(0x7fbf300c71c0 0x7fbf3015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf34e7d640 1 -- 172.21.15.53:0/3664736696 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3006fa60 con 0x7fbf301598c0 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf3567e640 1 --2- 172.21.15.53:0/3664736696 >> v2:172.21.15.135:3300/0 conn(0x7fbf300c71c0 0x7fbf3015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.551+0000 7fbf2ffff640 1 --2- 172.21.15.53:0/3664736696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3006ec40 0x7fbf300c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.552+0000 7fbf34e7d640 1 --2- 172.21.15.53:0/3664736696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf301598c0 0x7fbf3006e700 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbf3015d5b0 tx=0x7fbf2002a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666f1dcc785ff3ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.552+0000 7fbf2f7fe640 1 -- 172.21.15.53:0/3664736696 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20036070 con 0x7fbf301598c0 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.552+0000 7fbf2f7fe640 1 -- 172.21.15.53:0/3664736696 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf20031600 con 0x7fbf301598c0 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.552+0000 7fbf2f7fe640 1 -- 172.21.15.53:0/3664736696 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200319d0 con 0x7fbf301598c0 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 -- 172.21.15.53:0/3664736696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf301598c0 msgr2=0x7fbf3006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 --2- 172.21.15.53:0/3664736696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf301598c0 0x7fbf3006e700 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbf3015d5b0 tx=0x7fbf2002a690 comp rx=0 tx=0).stop 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 -- 172.21.15.53:0/3664736696 shutdown_connections 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 --2- 172.21.15.53:0/3664736696 >> v2:172.21.15.135:3300/0 conn(0x7fbf300c71c0 0x7fbf3015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 --2- 172.21.15.53:0/3664736696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3006ec40 0x7fbf300c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 --2- 172.21.15.53:0/3664736696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf301598c0 0x7fbf3006e700 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 -- 172.21.15.53:0/3664736696 >> 172.21.15.53:0/3664736696 conn(0x7fbf300b2900 msgr2=0x7fbf300b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 -- 172.21.15.53:0/3664736696 shutdown_connections 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.553+0000 7fbf370e1640 1 -- 172.21.15.53:0/3664736696 wait complete. 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.554+0000 7fbf370e1640 1 Processor -- start 2024-05-10T10:38:44.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.554+0000 7fbf370e1640 1 -- start start 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf370e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3006ec40 0x7fbf3015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf370e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf300c71c0 0x7fbf30071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf370e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf301598c0 0x7fbf300ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf370e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30159ca0 con 0x7fbf300c71c0 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf370e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3015c6e0 con 0x7fbf3006ec40 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf370e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf30159e20 con 0x7fbf301598c0 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf300c71c0 0x7fbf30071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf300c71c0 0x7fbf30071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43668/0 (socket says 172.21.15.53:43668) 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 -- 172.21.15.53:0/1351589699 learned_addr learned my addr 172.21.15.53:0/1351589699 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf3567e640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf301598c0 0x7fbf300ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf34e7d640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3006ec40 0x7fbf3015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 -- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf301598c0 msgr2=0x7fbf300ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf301598c0 0x7fbf300ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 -- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3006ec40 msgr2=0x7fbf3015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3006ec40 0x7fbf3015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 -- 172.21.15.53:0/1351589699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf30073cd0 con 0x7fbf300c71c0 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2ffff640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf300c71c0 0x7fbf30071df0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c7f0 tx=0x7fbf1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2d7fa640 1 -- 172.21.15.53:0/1351589699 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18015070 con 0x7fbf300c71c0 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2d7fa640 1 -- 172.21.15.53:0/1351589699 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf18007820 con 0x7fbf300c71c0 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf2d7fa640 1 -- 172.21.15.53:0/1351589699 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18007bd0 con 0x7fbf300c71c0 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.555+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf30073fb0 con 0x7fbf300c71c0 2024-05-10T10:38:44.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.556+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf30153c60 con 0x7fbf300c71c0 2024-05-10T10:38:44.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.557+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf30074180 con 0x7fbf300c71c0 2024-05-10T10:38:44.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.557+0000 7fbf2d7fa640 1 -- 172.21.15.53:0/1351589699 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf18018070 con 0x7fbf300c71c0 2024-05-10T10:38:44.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.558+0000 7fbf2d7fa640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf100762d0 0x7fbf10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.558+0000 7fbf2d7fa640 1 -- 172.21.15.53:0/1351589699 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fbf18097850 con 0x7fbf300c71c0 2024-05-10T10:38:44.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.558+0000 7fbf34e7d640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf100762d0 0x7fbf10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.559+0000 7fbf34e7d640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf100762d0 0x7fbf10078790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf20033000 tx=0x7fbf20009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.562+0000 7fbf2d7fa640 1 -- 172.21.15.53:0/1351589699 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbf30074180 con 0x7fbf300c71c0 2024-05-10T10:38:44.654 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:38:44.657 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:38:44.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.793+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf3015dc00 con 0x7fbf300c71c0 2024-05-10T10:38:44.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.793+0000 7fbf2d7fa640 1 -- 172.21.15.53:0/1351589699 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7fbf18061060 con 0x7fbf300c71c0 2024-05-10T10:38:44.795 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:44.795 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:38:14.328418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":5}]} 2024-05-10T10:38:44.795 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2024-05-10T10:38:44.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf100762d0 msgr2=0x7fbf10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf100762d0 0x7fbf10078790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf20033000 tx=0x7fbf20009000 comp rx=0 tx=0).stop 2024-05-10T10:38:44.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf300c71c0 msgr2=0x7fbf30071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf300c71c0 0x7fbf30071df0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c7f0 tx=0x7fbf1800ccb0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 shutdown_connections 2024-05-10T10:38:44.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf100762d0 0x7fbf10078790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf301598c0 0x7fbf300ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf300c71c0 0x7fbf30071df0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 --2- 172.21.15.53:0/1351589699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3006ec40 0x7fbf3015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 >> 172.21.15.53:0/1351589699 conn(0x7fbf300b2900 msgr2=0x7fbf300b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:44.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 shutdown_connections 2024-05-10T10:38:44.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.795+0000 7fbf370e1640 1 -- 172.21.15.53:0/1351589699 wait complete. 2024-05-10T10:38:44.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:38:44.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.910+0000 7f5d12a45640 1 Processor -- start 2024-05-10T10:38:44.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.910+0000 7f5d12a45640 1 -- start start 2024-05-10T10:38:44.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.911+0000 7f5d12a45640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162b30 0x7f5d0c169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.911+0000 7f5d12a45640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0c0b45f0 con 0x7f5d0c0d5cb0 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.911+0000 7f5d12a45640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0c0b48d0 con 0x7f5d0c162750 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.911+0000 7f5d12a45640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c0b4f20 con 0x7f5d0c162b30 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.911+0000 7f5d0b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5d0c0d5cb0 0x7f5d0c0d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58320/0 (socket says 172.21.15.53:58320) 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.911+0000 7f5d0b7fe640 1 -- 172.21.15.53:0/1068711235 learned_addr learned my addr 172.21.15.53:0/1068711235 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d10fe2640 1 --2- 172.21.15.53:0/1068711235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162b30 0x7f5d0c169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d0affd640 1 -- 172.21.15.53:0/1068711235 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219899493 0 0) 0x7f5d0c0b45f0 con 0x7f5d0c0d5cb0 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d0affd640 1 -- 172.21.15.53:0/1068711235 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cf4003520 con 0x7f5d0c0d5cb0 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d0affd640 1 -- 172.21.15.53:0/1068711235 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2482807200 0 0) 0x7f5d0c0b48d0 con 0x7f5d0c162750 2024-05-10T10:38:44.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d0affd640 1 -- 172.21.15.53:0/1068711235 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d0c0b45f0 con 0x7f5d0c162750 2024-05-10T10:38:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d10fe2640 1 -- 172.21.15.53:0/1068711235 >> v1:172.21.15.135:6789/0 conn(0x7f5d0c162750 legacy=0x7f5d0c0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d10fe2640 1 -- 172.21.15.53:0/1068711235 >> v1:172.21.15.53:6789/0 conn(0x7f5d0c0d5cb0 legacy=0x7f5d0c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.912+0000 7f5d10fe2640 1 -- 172.21.15.53:0/1068711235 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c0b4bb0 con 0x7f5d0c162b30 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.913+0000 7f5d10fe2640 1 --2- 172.21.15.53:0/1068711235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162b30 0x7f5d0c169fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009dc0 tx=0x7f5d0402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c752b4bef08f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.913+0000 7f5d0affd640 1 -- 172.21.15.53:0/1068711235 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0402ae60 con 0x7f5d0c162b30 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.913+0000 7f5d0affd640 1 -- 172.21.15.53:0/1068711235 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d04007740 con 0x7f5d0c162b30 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.913+0000 7f5d0affd640 1 -- 172.21.15.53:0/1068711235 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04007af0 con 0x7f5d0c162b30 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.914+0000 7f5d12a45640 1 -- 172.21.15.53:0/1068711235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162b30 msgr2=0x7f5d0c169fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.914+0000 7f5d12a45640 1 --2- 172.21.15.53:0/1068711235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162b30 0x7f5d0c169fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009dc0 tx=0x7f5d0402a640 comp rx=0 tx=0).stop 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.914+0000 7f5d12a45640 1 -- 172.21.15.53:0/1068711235 shutdown_connections 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.914+0000 7f5d12a45640 1 --2- 172.21.15.53:0/1068711235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162b30 0x7f5d0c169fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.914+0000 7f5d12a45640 1 -- 172.21.15.53:0/1068711235 >> 172.21.15.53:0/1068711235 conn(0x7f5d0c09e3b0 msgr2=0x7f5d0c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:44.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.914+0000 7f5d12a45640 1 -- 172.21.15.53:0/1068711235 shutdown_connections 2024-05-10T10:38:44.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.914+0000 7f5d12a45640 1 -- 172.21.15.53:0/1068711235 wait complete. 2024-05-10T10:38:44.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.915+0000 7f5d12a45640 1 Processor -- start 2024-05-10T10:38:44.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.915+0000 7f5d12a45640 1 -- start start 2024-05-10T10:38:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d12a45640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162750 0x7f5d0c0b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d12a45640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d0c162b30 0x7f5d0c0d5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d12a45640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d0c0ca7c0 0x7f5d0c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d12a45640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c0d0fe0 con 0x7f5d0c162b30 2024-05-10T10:38:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d12a45640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c0d3f90 con 0x7f5d0c0ca7c0 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d12a45640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c0d0d80 con 0x7f5d0c162750 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d0c162b30 0x7f5d0c0d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d0c162b30 0x7f5d0c0d5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43680/0 (socket says 172.21.15.53:43680) 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 -- 172.21.15.53:0/596142033 learned_addr learned my addr 172.21.15.53:0/596142033 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d10fe2640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d0c0ca7c0 0x7f5d0c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0bfff640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162750 0x7f5d0c0b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 -- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162750 msgr2=0x7f5d0c0b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162750 0x7f5d0c0b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 -- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d0c0ca7c0 msgr2=0x7f5d0c0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d0c0ca7c0 0x7f5d0c0cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 -- 172.21.15.53:0/596142033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c0cee00 con 0x7f5d0c162b30 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.916+0000 7f5d0b7fe640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d0c162b30 0x7f5d0c0d5130 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c480 tx=0x7f5d0000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.917+0000 7f5d08ff9640 1 -- 172.21.15.53:0/596142033 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00017070 con 0x7f5d0c162b30 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.917+0000 7f5d08ff9640 1 -- 172.21.15.53:0/596142033 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d00005da0 con 0x7f5d0c162b30 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.917+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c0b7e90 con 0x7f5d0c162b30 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.917+0000 7f5d08ff9640 1 -- 172.21.15.53:0/596142033 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00007290 con 0x7f5d0c162b30 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.917+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c0b8eb0 con 0x7f5d0c162b30 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.918+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c06a220 con 0x7f5d0c162b30 2024-05-10T10:38:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.918+0000 7f5d08ff9640 1 -- 172.21.15.53:0/596142033 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5d0001a070 con 0x7f5d0c162b30 2024-05-10T10:38:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.919+0000 7f5d08ff9640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5cf80762d0 0x7f5cf8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.919+0000 7f5d08ff9640 1 -- 172.21.15.53:0/596142033 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f5d00096c70 con 0x7f5d0c162b30 2024-05-10T10:38:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.919+0000 7f5d0bfff640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5cf80762d0 0x7f5cf8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.920+0000 7f5d0bfff640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5cf80762d0 0x7f5cf8078790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001650 tx=0x7f5cfc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:44.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:44.923+0000 7f5d08ff9640 1 -- 172.21.15.53:0/596142033 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5d000603f0 con 0x7f5d0c162b30 2024-05-10T10:38:45.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.155+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d0c0d1340 con 0x7f5d0c162b30 2024-05-10T10:38:45.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.156+0000 7f5d08ff9640 1 -- 172.21.15.53:0/596142033 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7f5d0005fcf0 con 0x7f5d0c162b30 2024-05-10T10:38:45.157 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:45.157 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:37:17.592875+0000","modified":"2024-05-10T10:38:14.328418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6834/3202950630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3202950630},{"type":"v1","addr":"172.21.15.53:6834","nonce":3202950630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6291,"qdb_cluster":[6291]},"id":5}]} 2024-05-10T10:38:45.158 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5cf80762d0 msgr2=0x7f5cf8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5cf80762d0 0x7f5cf8078790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001650 tx=0x7f5cfc009040 comp rx=0 tx=0).stop 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d0c162b30 msgr2=0x7f5d0c0d5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d0c162b30 0x7f5d0c0d5130 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c480 tx=0x7f5d0000c940 comp rx=0 tx=0).stop 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 shutdown_connections 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5cf80762d0 0x7f5cf8078790 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d0c0ca7c0 0x7f5d0c0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d0c162b30 0x7f5d0c0d5130 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 --2- 172.21.15.53:0/596142033 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d0c162750 0x7f5d0c0b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 >> 172.21.15.53:0/596142033 conn(0x7f5d0c09e3b0 msgr2=0x7f5d0c06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:45.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 shutdown_connections 2024-05-10T10:38:45.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.158+0000 7f5d12a45640 1 -- 172.21.15.53:0/596142033 wait complete. 2024-05-10T10:38:45.171 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:38:45.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.274+0000 7f43c52e1640 1 Processor -- start 2024-05-10T10:38:45.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.274+0000 7f43c52e1640 1 -- start start 2024-05-10T10:38:45.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.274+0000 7f43c52e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c00c6c80 0x7f43c00c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.274+0000 7f43c52e1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f43c00cee40 0x7f43c00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.274+0000 7f43c52e1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c006f1c0 con 0x7f43c01598c0 2024-05-10T10:38:45.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.274+0000 7f43c52e1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f43c00c7620 con 0x7f43c00cee40 2024-05-10T10:38:45.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.274+0000 7f43c52e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f43c006f4a0 con 0x7f43c00c6c80 2024-05-10T10:38:45.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43bf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f43c00cee40 0x7f43c00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43beffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f43c01598c0 0x7f43c006e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58324/0 (socket says 172.21.15.53:58324) 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43beffd640 1 -- 172.21.15.53:0/2803862296 learned_addr learned my addr 172.21.15.53:0/2803862296 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43be7fc640 1 --2- 172.21.15.53:0/2803862296 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c00c6c80 0x7f43c00c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43bf7fe640 1 -- 172.21.15.53:0/2803862296 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c00c6c80 msgr2=0x7f43c00c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43bf7fe640 1 --2- 172.21.15.53:0/2803862296 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c00c6c80 0x7f43c00c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43bf7fe640 1 -- 172.21.15.53:0/2803862296 >> v1:172.21.15.53:6789/0 conn(0x7f43c01598c0 legacy=0x7f43c006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43bf7fe640 1 -- 172.21.15.53:0/2803862296 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c006f780 con 0x7f43c00cee40 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.275+0000 7f43bdffb640 1 -- 172.21.15.53:0/2803862296 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4213448142 0 0) 0x7f43c006f1c0 con 0x7f43c01598c0 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43bf7fe640 1 --2- 172.21.15.53:0/2803862296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00cee40 0x7f43c00cf2d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009d90 tx=0x7f43b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348691258b2cde84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43bdffb640 1 -- 172.21.15.53:0/2803862296 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8007740 con 0x7f43c00cee40 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43bdffb640 1 -- 172.21.15.53:0/2803862296 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b80078e0 con 0x7f43c00cee40 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43bdffb640 1 -- 172.21.15.53:0/2803862296 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8007cb0 con 0x7f43c00cee40 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 -- 172.21.15.53:0/2803862296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00cee40 msgr2=0x7f43c00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2803862296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00cee40 0x7f43c00cf2d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009d90 tx=0x7f43b802a660 comp rx=0 tx=0).stop 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 -- 172.21.15.53:0/2803862296 shutdown_connections 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2803862296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00cee40 0x7f43c00cf2d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2803862296 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c00c6c80 0x7f43c00c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 -- 172.21.15.53:0/2803862296 >> 172.21.15.53:0/2803862296 conn(0x7f43c00b2900 msgr2=0x7f43c00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 -- 172.21.15.53:0/2803862296 shutdown_connections 2024-05-10T10:38:45.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.276+0000 7f43c52e1640 1 -- 172.21.15.53:0/2803862296 wait complete. 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 Processor -- start 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 -- start start 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00c6c80 0x7f43c00aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43c00cee40 0x7f43c015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c01598c0 0x7f43c015f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f43c015c6e0 con 0x7f43c00c6c80 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43c00ce920 con 0x7f43c00cee40 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.278+0000 7f43c52e1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f43c006ee10 con 0x7f43c01598c0 2024-05-10T10:38:45.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00c6c80 0x7f43c00aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00c6c80 0x7f43c00aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43698/0 (socket says 172.21.15.53:43698) 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 -- 172.21.15.53:0/2679903012 learned_addr learned my addr 172.21.15.53:0/2679903012 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 -- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c01598c0 msgr2=0x7f43c015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43bf7fe640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c01598c0 0x7f43c015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43be7fc640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43c00cee40 0x7f43c015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c01598c0 0x7f43c015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 -- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43c00cee40 msgr2=0x7f43c015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43c00cee40 0x7f43c015eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 -- 172.21.15.53:0/2679903012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c00c95d0 con 0x7f43c00c6c80 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43be7fc640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43c00cee40 0x7f43c015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43bf7fe640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c01598c0 0x7f43c015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:45.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43beffd640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00c6c80 0x7f43c00aec10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c460 tx=0x7f43b000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43a77fe640 1 -- 172.21.15.53:0/2679903012 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0018070 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43a77fe640 1 -- 172.21.15.53:0/2679903012 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b0007500 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c0154810 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43a77fe640 1 -- 172.21.15.53:0/2679903012 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0004a20 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.279+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c0073d40 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.280+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c006a220 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.281+0000 7f43a77fe640 1 -- 172.21.15.53:0/2679903012 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f43b001b070 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.281+0000 7f43a77fe640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43ac076300 0x7f43ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.281+0000 7f43a77fe640 1 -- 172.21.15.53:0/2679903012 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f43b0096a10 con 0x7f43c00c6c80 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.282+0000 7f43be7fc640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43ac076300 0x7f43ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.282+0000 7f43be7fc640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43ac076300 0x7f43ac0787c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f43c0073a80 tx=0x7f43b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.286+0000 7f43a77fe640 1 -- 172.21.15.53:0/2679903012 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f43b0060180 con 0x7f43c00c6c80 2024-05-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.476+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43c006e4e0 con 0x7f43c00c6c80 2024-05-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.478+0000 7f43a77fe640 1 -- 172.21.15.53:0/2679903012 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17074 (secure 0 0 0) 0x7f43b005fa80 con 0x7f43c00c6c80 2024-05-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:45.480 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":75,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:37:31.312656+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:37:15.291160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T10:37:16.547933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6834/1984524370":"2024-05-11T10:37:31.312633+0000","172.21.15.53:6832/1984524370":"2024-05-11T10:37:31.312633+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43ac076300 msgr2=0x7f43ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43ac076300 0x7f43ac0787c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f43c0073a80 tx=0x7f43b4008040 comp rx=0 tx=0).stop 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00c6c80 msgr2=0x7f43c00aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00c6c80 0x7f43c00aec10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c460 tx=0x7f43b000c920 comp rx=0 tx=0).stop 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 shutdown_connections 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43ac076300 0x7f43ac0787c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43c01598c0 0x7f43c015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43c00cee40 0x7f43c015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 --2- 172.21.15.53:0/2679903012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00c6c80 0x7f43c00aec10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 >> 172.21.15.53:0/2679903012 conn(0x7f43c00b2900 msgr2=0x7f43c0158ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:45.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 shutdown_connections 2024-05-10T10:38:45.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.480+0000 7f43c52e1640 1 -- 172.21.15.53:0/2679903012 wait complete. 2024-05-10T10:38:45.493 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:38:45.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.596+0000 7fbacae54640 1 Processor -- start 2024-05-10T10:38:45.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.597+0000 7fbacae54640 1 -- start start 2024-05-10T10:38:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.597+0000 7fbacae54640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 0x7fbac4167f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.597+0000 7fbacae54640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac4160c90 con 0x7fbac41608b0 2024-05-10T10:38:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.597+0000 7fbacae54640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac40d1b10 con 0x7fbac40b3910 2024-05-10T10:38:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.597+0000 7fbacae54640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbac40ce5f0 con 0x7fbac40d03c0 2024-05-10T10:38:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.597+0000 7fbac8bf0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbac41608b0 0x7fbac40d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58334/0 (socket says 172.21.15.53:58334) 2024-05-10T10:38:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac8bf0640 1 -- 172.21.15.53:0/3922198019 learned_addr learned my addr 172.21.15.53:0/3922198019 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:45.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446573296 0 0) 0x7fbab4001110 con 0x7fbac41608b0 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaac003520 con 0x7fbac41608b0 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac93f1640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 0x7fbac4167f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1507417627 0 0) 0x7fbaac003520 con 0x7fbac41608b0 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 msgr2=0x7fbac4167f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 0x7fbac4167f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 >> v1:172.21.15.135:6789/0 conn(0x7fbac40b3910 legacy=0x7fbac40d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac40ce8d0 con 0x7fbac41608b0 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fbab4001840 con 0x7fbac41608b0 2024-05-10T10:38:45.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 >> v1:172.21.15.53:6789/0 conn(0x7fbac41608b0 legacy=0x7fbac40d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac0039e0 0x7fbaac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 0x7fbaac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaac004d40 0x7fbaac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbac40ce5f0 con 0x7fbaac004d40 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac00ba70 con 0x7fbaac0039e0 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbaac00bc70 con 0x7fbac40d03c0 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac93f1640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaac004d40 0x7fbaac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac8bf0640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac0039e0 0x7fbaac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.598+0000 7fbac3fff640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 0x7fbaac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.599+0000 7fbac93f1640 1 -- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 msgr2=0x7fbaac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.599+0000 7fbac93f1640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac40d03c0 0x7fbaac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.599+0000 7fbac93f1640 1 -- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac0039e0 msgr2=0x7fbaac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.599+0000 7fbac93f1640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac0039e0 0x7fbaac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.599+0000 7fbac93f1640 1 -- 172.21.15.53:0/3922198019 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac003520 con 0x7fbaac004d40 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.599+0000 7fbac93f1640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaac004d40 0x7fbaac0051c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009f90 tx=0x7fbaac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca3608de1a605db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc009c10 con 0x7fbaac004d40 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc036070 con 0x7fbaac004d40 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbac37fe640 1 -- 172.21.15.53:0/3922198019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc031850 con 0x7fbaac004d40 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbacae54640 1 -- 172.21.15.53:0/3922198019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaac004d40 msgr2=0x7fbaac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbacae54640 1 --2- 172.21.15.53:0/3922198019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaac004d40 0x7fbaac0051c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009f90 tx=0x7fbaac005830 comp rx=0 tx=0).stop 2024-05-10T10:38:45.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbac93f1640 1 -- 172.21.15.53:0/3922198019 reap_dead start 2024-05-10T10:38:45.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbacae54640 1 -- 172.21.15.53:0/3922198019 shutdown_connections 2024-05-10T10:38:45.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbacae54640 1 -- 172.21.15.53:0/3922198019 >> 172.21.15.53:0/3922198019 conn(0x7fbac40d2df0 msgr2=0x7fbac40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:45.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbacae54640 1 -- 172.21.15.53:0/3922198019 shutdown_connections 2024-05-10T10:38:45.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.600+0000 7fbacae54640 1 -- 172.21.15.53:0/3922198019 wait complete. 2024-05-10T10:38:45.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.601+0000 7fbacae54640 1 Processor -- start 2024-05-10T10:38:45.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.601+0000 7fbacae54640 1 -- start start 2024-05-10T10:38:45.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbacae54640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac004d40 0x7fbac406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbacae54640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac41f5370 0x7fbac40c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbacae54640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac415c420 0x7fbac40d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbacae54640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbac40d1d90 con 0x7fbac415c420 2024-05-10T10:38:45.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbacae54640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbac40b1f50 con 0x7fbaac004d40 2024-05-10T10:38:45.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbacae54640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbac40d1f10 con 0x7fbac41f5370 2024-05-10T10:38:45.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbac93f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac415c420 0x7fbac40d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbac93f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac415c420 0x7fbac40d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43722/0 (socket says 172.21.15.53:43722) 2024-05-10T10:38:45.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.602+0000 7fbac93f1640 1 -- 172.21.15.53:0/1797003092 learned_addr learned my addr 172.21.15.53:0/1797003092 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:45.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac3fff640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac41f5370 0x7fbac40c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac8bf0640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac004d40 0x7fbac406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac93f1640 1 -- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac41f5370 msgr2=0x7fbac40c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac93f1640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac41f5370 0x7fbac40c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac93f1640 1 -- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac004d40 msgr2=0x7fbac406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac93f1640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac004d40 0x7fbac406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac93f1640 1 -- 172.21.15.53:0/1797003092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac40c69c0 con 0x7fbac415c420 2024-05-10T10:38:45.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac3fff640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac41f5370 0x7fbac40c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:45.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac93f1640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac415c420 0x7fbac40d0cf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fbabc033040 tx=0x7fbabc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac17fa640 1 -- 172.21.15.53:0/1797003092 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc003990 con 0x7fbac415c420 2024-05-10T10:38:45.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac17fa640 1 -- 172.21.15.53:0/1797003092 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc003b50 con 0x7fbac415c420 2024-05-10T10:38:45.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac40c5e50 con 0x7fbac415c420 2024-05-10T10:38:45.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbac17fa640 1 -- 172.21.15.53:0/1797003092 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc004020 con 0x7fbac415c420 2024-05-10T10:38:45.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.603+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac41f5cf0 con 0x7fbac415c420 2024-05-10T10:38:45.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.604+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac406a220 con 0x7fbac415c420 2024-05-10T10:38:45.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.605+0000 7fbac17fa640 1 -- 172.21.15.53:0/1797003092 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbabc005d10 con 0x7fbac415c420 2024-05-10T10:38:45.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.605+0000 7fbac17fa640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab0076300 0x7fbab00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.606+0000 7fbac17fa640 1 -- 172.21.15.53:0/1797003092 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fbabc0b8bd0 con 0x7fbac415c420 2024-05-10T10:38:45.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.606+0000 7fbac8bf0640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab0076300 0x7fbab00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.606+0000 7fbac8bf0640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab0076300 0x7fbab00787c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbab4001eb0 tx=0x7fbab4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.610+0000 7fbac17fa640 1 -- 172.21.15.53:0/1797003092 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbabc04a060 con 0x7fbac415c420 2024-05-10T10:38:45.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.840+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbac41f5750 con 0x7fbac415c420 2024-05-10T10:38:45.842 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:38:45.841+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:38:45.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.843+0000 7fbac17fa640 1 -- 172.21.15.53:0/1797003092 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7fbabc02e050 con 0x7fbac415c420 2024-05-10T10:38:45.844 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:38:45.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab0076300 msgr2=0x7fbab00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab0076300 0x7fbab00787c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbab4001eb0 tx=0x7fbab4009040 comp rx=0 tx=0).stop 2024-05-10T10:38:45.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac415c420 msgr2=0x7fbac40d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac415c420 0x7fbac40d0cf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fbabc033040 tx=0x7fbabc009750 comp rx=0 tx=0).stop 2024-05-10T10:38:45.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 shutdown_connections 2024-05-10T10:38:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab0076300 0x7fbab00787c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac415c420 0x7fbac40d0cf0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac41f5370 0x7fbac40c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 --2- 172.21.15.53:0/1797003092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaac004d40 0x7fbac406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 >> 172.21.15.53:0/1797003092 conn(0x7fbac40d2df0 msgr2=0x7fbac4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 shutdown_connections 2024-05-10T10:38:45.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.846+0000 7fbacae54640 1 -- 172.21.15.53:0/1797003092 wait complete. 2024-05-10T10:38:45.851 INFO:tasks.ceph.mds.a.smithi053.stderr: -4023> 2024-05-10T10:37:37.323+0000 7f3e71eb3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:38:45.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:38:45.884 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:38:45.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.968+0000 7ff1d4ea7640 1 Processor -- start 2024-05-10T10:38:45.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.968+0000 7ff1d4ea7640 1 -- start start 2024-05-10T10:38:45.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.969+0000 7ff1d4ea7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1d006c2c0 0x7ff1d006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.969+0000 7ff1d4ea7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d006c6a0 0x7ff1d00739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.969+0000 7ff1d4ea7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d0157a40 con 0x7ff1d0073f20 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.969+0000 7ff1d4ea7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d0161990 con 0x7ff1d006c2c0 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.969+0000 7ff1d4ea7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d0160d80 con 0x7ff1d006c6a0 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1ce59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1d006c2c0 0x7ff1d006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1ced9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff1d0073f20 0x7ff1d015d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58352/0 (socket says 172.21.15.53:58352) 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1ced9d640 1 -- 172.21.15.53:0/2050801835 learned_addr learned my addr 172.21.15.53:0/2050801835 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1cdd9b640 1 --2- 172.21.15.53:0/2050801835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d006c6a0 0x7ff1d00739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1ce59c640 1 -- 172.21.15.53:0/2050801835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d006c6a0 msgr2=0x7ff1d00739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1ce59c640 1 --2- 172.21.15.53:0/2050801835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d006c6a0 0x7ff1d00739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1ce59c640 1 -- 172.21.15.53:0/2050801835 >> v1:172.21.15.53:6789/0 conn(0x7ff1d0073f20 legacy=0x7ff1d015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1cd59a640 1 -- 172.21.15.53:0/2050801835 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 426608187 0 0) 0x7ff1c8001490 con 0x7ff1d0073f20 2024-05-10T10:38:45.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.970+0000 7ff1ce59c640 1 -- 172.21.15.53:0/2050801835 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d0161060 con 0x7ff1d006c2c0 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1ce59c640 1 --2- 172.21.15.53:0/2050801835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c2c0 0x7ff1d006df20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00097a0 tx=0x7ff1c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd39176569801a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1cd59a640 1 -- 172.21.15.53:0/2050801835 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0037070 con 0x7ff1d006c2c0 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1cd59a640 1 -- 172.21.15.53:0/2050801835 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c0032440 con 0x7ff1d006c2c0 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1cd59a640 1 -- 172.21.15.53:0/2050801835 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0032810 con 0x7ff1d006c2c0 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/2050801835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c2c0 msgr2=0x7ff1d006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/2050801835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c2c0 0x7ff1d006df20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00097a0 tx=0x7ff1c002a180 comp rx=0 tx=0).stop 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/2050801835 shutdown_connections 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/2050801835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d006c6a0 0x7ff1d00739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/2050801835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c2c0 0x7ff1d006df20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.971+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/2050801835 >> 172.21.15.53:0/2050801835 conn(0x7ff1d00d0c90 msgr2=0x7ff1d00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.972+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/2050801835 shutdown_connections 2024-05-10T10:38:45.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.972+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/2050801835 wait complete. 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.973+0000 7ff1d4ea7640 1 Processor -- start 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.973+0000 7ff1d4ea7640 1 -- start start 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1d4ea7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d006c2c0 0x7ff1d00ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1d4ea7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c6a0 0x7ff1d00ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1d4ea7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d0073f20 0x7ff1d01f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1d4ea7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00cd440 con 0x7ff1d006c6a0 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1d4ea7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0161620 con 0x7ff1d006c2c0 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1d4ea7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d006cb20 con 0x7ff1d0073f20 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c6a0 0x7ff1d00ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c6a0 0x7ff1d00ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43766/0 (socket says 172.21.15.53:43766) 2024-05-10T10:38:45.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 -- 172.21.15.53:0/3730795748 learned_addr learned my addr 172.21.15.53:0/3730795748 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1ced9d640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d0073f20 0x7ff1d01f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 -- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d0073f20 msgr2=0x7ff1d01f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1ce59c640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d006c2c0 0x7ff1d00ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d0073f20 0x7ff1d01f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 -- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d006c2c0 msgr2=0x7ff1d00ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d006c2c0 0x7ff1d00ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 -- 172.21.15.53:0/3730795748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d015f510 con 0x7ff1d006c6a0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1ce59c640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d006c2c0 0x7ff1d00ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1cdd9b640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c6a0 0x7ff1d00ca570 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400d5f0 tx=0x7ff1c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.974+0000 7ff1beffd640 1 -- 172.21.15.53:0/3730795748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40125f0 con 0x7ff1d006c6a0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.975+0000 7ff1beffd640 1 -- 172.21.15.53:0/3730795748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4019070 con 0x7ff1d006c6a0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.975+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d015f7f0 con 0x7ff1d006c6a0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.975+0000 7ff1beffd640 1 -- 172.21.15.53:0/3730795748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40129d0 con 0x7ff1d006c6a0 2024-05-10T10:38:45.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.975+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d015fdb0 con 0x7ff1d006c6a0 2024-05-10T10:38:45.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.976+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d006a220 con 0x7ff1d006c6a0 2024-05-10T10:38:45.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.976+0000 7ff1beffd640 1 -- 172.21.15.53:0/3730795748 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff1c4012b70 con 0x7ff1d006c6a0 2024-05-10T10:38:45.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.977+0000 7ff1beffd640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1b0076300 0x7ff1b00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:45.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.977+0000 7ff1beffd640 1 -- 172.21.15.53:0/3730795748 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6273+0+0 (secure 0 0 0) 0x7ff1c4097b90 con 0x7ff1d006c6a0 2024-05-10T10:38:45.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.977+0000 7ff1ce59c640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1b0076300 0x7ff1b00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:45.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.978+0000 7ff1ce59c640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1b0076300 0x7ff1b00787c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0034000 tx=0x7ff1c002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:45.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:45.981+0000 7ff1beffd640 1 -- 172.21.15.53:0/3730795748 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff1c4061340 con 0x7ff1d006c6a0 2024-05-10T10:38:46.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.212+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff1d00d1350 con 0x7ff1d006c6a0 2024-05-10T10:38:46.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.849+0000 7ff1beffd640 1 -- 172.21.15.53:0/3730795748 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7ff1c4060c40 con 0x7ff1d006c6a0 2024-05-10T10:38:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1b0076300 msgr2=0x7ff1b00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1b0076300 0x7ff1b00787c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0034000 tx=0x7ff1c002e000 comp rx=0 tx=0).stop 2024-05-10T10:38:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c6a0 msgr2=0x7ff1d00ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c6a0 0x7ff1d00ca570 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400d5f0 tx=0x7ff1c400dab0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 shutdown_connections 2024-05-10T10:38:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff1b0076300 0x7ff1b00787c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1d0073f20 0x7ff1d01f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1d006c6a0 0x7ff1d00ca570 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 --2- 172.21.15.53:0/3730795748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1d006c2c0 0x7ff1d00ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:46.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 >> 172.21.15.53:0/3730795748 conn(0x7ff1d00d0c90 msgr2=0x7ff1d00b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:46.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 shutdown_connections 2024-05-10T10:38:46.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:46.852+0000 7ff1d4ea7640 1 -- 172.21.15.53:0/3730795748 wait complete. 2024-05-10T10:38:46.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:38:47.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.015+0000 7f1eca059640 1 Processor -- start 2024-05-10T10:38:47.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.015+0000 7f1eca059640 1 -- start start 2024-05-10T10:38:47.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1eca059640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1ec415f6e0 0x7f1ec40d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1eca059640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ec40d5cf0 0x7f1ec40b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1eca059640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec40b1d90 con 0x7f1ec40d3680 2024-05-10T10:38:47.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1eca059640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec40d1010 con 0x7f1ec40d5cf0 2024-05-10T10:38:47.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1eca059640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec40b26c0 con 0x7f1ec415f6e0 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ec40d5cf0 0x7f1ec40b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ec40d5cf0 0x7f1ec40b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43774/0 (socket says 172.21.15.53:43774) 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec3fff640 1 -- 172.21.15.53:0/3747814287 learned_addr learned my addr 172.21.15.53:0/3747814287 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec2ffd640 1 --2- 172.21.15.53:0/3747814287 >> v2:172.21.15.135:3300/0 conn(0x7f1ec415f6e0 0x7f1ec40d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec3fff640 1 -- 172.21.15.53:0/3747814287 >> v2:172.21.15.135:3300/0 conn(0x7f1ec415f6e0 msgr2=0x7f1ec40d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec3fff640 1 --2- 172.21.15.53:0/3747814287 >> v2:172.21.15.135:3300/0 conn(0x7f1ec415f6e0 0x7f1ec40d0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec3fff640 1 -- 172.21.15.53:0/3747814287 >> v1:172.21.15.135:6789/0 conn(0x7f1ec40d3680 legacy=0x7f1ec40d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:47.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.016+0000 7f1ec3fff640 1 -- 172.21.15.53:0/3747814287 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec40b2070 con 0x7f1ec40d5cf0 2024-05-10T10:38:47.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.017+0000 7f1ec3fff640 1 --2- 172.21.15.53:0/3747814287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d5cf0 0x7f1ec40b2b50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1ec408f280 tx=0x7f1ebc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78e792a1732e153 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:47.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.017+0000 7f1ec27fc640 1 -- 172.21.15.53:0/3747814287 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc036070 con 0x7f1ec40d5cf0 2024-05-10T10:38:47.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.017+0000 7f1ec27fc640 1 -- 172.21.15.53:0/3747814287 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ebc031600 con 0x7f1ec40d5cf0 2024-05-10T10:38:47.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.017+0000 7f1ec27fc640 1 -- 172.21.15.53:0/3747814287 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0319d0 con 0x7f1ec40d5cf0 2024-05-10T10:38:47.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 -- 172.21.15.53:0/3747814287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d5cf0 msgr2=0x7f1ec40b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 --2- 172.21.15.53:0/3747814287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d5cf0 0x7f1ec40b2b50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1ec408f280 tx=0x7f1ebc02a660 comp rx=0 tx=0).stop 2024-05-10T10:38:47.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 -- 172.21.15.53:0/3747814287 shutdown_connections 2024-05-10T10:38:47.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 --2- 172.21.15.53:0/3747814287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d5cf0 0x7f1ec40b2b50 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 --2- 172.21.15.53:0/3747814287 >> v2:172.21.15.135:3300/0 conn(0x7f1ec415f6e0 0x7f1ec40d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 -- 172.21.15.53:0/3747814287 >> 172.21.15.53:0/3747814287 conn(0x7f1ec409e370 msgr2=0x7f1ec40d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:47.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 -- 172.21.15.53:0/3747814287 shutdown_connections 2024-05-10T10:38:47.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.018+0000 7f1eca059640 1 -- 172.21.15.53:0/3747814287 wait complete. 2024-05-10T10:38:47.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.019+0000 7f1eca059640 1 Processor -- start 2024-05-10T10:38:47.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.019+0000 7f1eca059640 1 -- start start 2024-05-10T10:38:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1eca059640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d3680 0x7f1ec40ba870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1eca059640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec40d5cf0 0x7f1ec415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1eca059640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ec415f6e0 0x7f1ec40ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1eca059640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec40b3090 con 0x7f1ec40d3680 2024-05-10T10:38:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1eca059640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec40d0c90 con 0x7f1ec40d5cf0 2024-05-10T10:38:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1eca059640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec40b3210 con 0x7f1ec415f6e0 2024-05-10T10:38:47.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1ec37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d3680 0x7f1ec40ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1ec37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d3680 0x7f1ec40ba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43790/0 (socket says 172.21.15.53:43790) 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1ec37fe640 1 -- 172.21.15.53:0/1741482868 learned_addr learned my addr 172.21.15.53:0/1741482868 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.020+0000 7f1ec2ffd640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec40d5cf0 0x7f1ec415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1ec3fff640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ec415f6e0 0x7f1ec40ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1ec37fe640 1 -- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ec415f6e0 msgr2=0x7f1ec40ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1ec37fe640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ec415f6e0 0x7f1ec40ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1ec37fe640 1 -- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec40d5cf0 msgr2=0x7f1ec415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1ec37fe640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec40d5cf0 0x7f1ec415c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1ec37fe640 1 -- 172.21.15.53:0/1741482868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec415c930 con 0x7f1ec40d3680 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1ec37fe640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d3680 0x7f1ec40ba870 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400c460 tx=0x7f1eb400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1eabfff640 1 -- 172.21.15.53:0/1741482868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4016020 con 0x7f1ec40d3680 2024-05-10T10:38:47.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1eabfff640 1 -- 172.21.15.53:0/1741482868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb4012870 con 0x7f1ec40d3680 2024-05-10T10:38:47.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1eabfff640 1 -- 172.21.15.53:0/1741482868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4012c60 con 0x7f1ec40d3680 2024-05-10T10:38:47.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec40cf030 con 0x7f1ec40d3680 2024-05-10T10:38:47.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.021+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec409ee80 con 0x7f1ec40d3680 2024-05-10T10:38:47.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.022+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec40693f0 con 0x7f1ec40d3680 2024-05-10T10:38:47.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.023+0000 7f1eabfff640 1 -- 172.21.15.53:0/1741482868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1eb4019020 con 0x7f1ec40d3680 2024-05-10T10:38:47.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.023+0000 7f1eabfff640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eb00762d0 0x7f1eb0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.024+0000 7f1eabfff640 1 -- 172.21.15.53:0/1741482868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6273+0+0 (secure 0 0 0) 0x7f1eb40974e0 con 0x7f1ec40d3680 2024-05-10T10:38:47.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.024+0000 7f1ec2ffd640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eb00762d0 0x7f1eb0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.024+0000 7f1ec2ffd640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eb00762d0 0x7f1eb0078790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40b45d0 tx=0x7f1eb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:47.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.028+0000 7f1eabfff640 1 -- 172.21.15.53:0/1741482868 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1eb4020030 con 0x7f1ec40d3680 2024-05-10T10:38:47.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.218+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ec40d3460 con 0x7f1ec40d3680 2024-05-10T10:38:47.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.850+0000 7f1eabfff640 1 -- 172.21.15.53:0/1741482868 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) ==== 181+0+0 (secure 0 0 0) 0x7f1ec40cf030 con 0x7f1ec40d3680 2024-05-10T10:38:47.852 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eb00762d0 msgr2=0x7f1eb0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eb00762d0 0x7f1eb0078790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40b45d0 tx=0x7f1eb8008040 comp rx=0 tx=0).stop 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d3680 msgr2=0x7f1ec40ba870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d3680 0x7f1ec40ba870 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400c460 tx=0x7f1eb400c920 comp rx=0 tx=0).stop 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 shutdown_connections 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1eb00762d0 0x7f1eb0078790 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1ec415f6e0 0x7f1ec40ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ec40d5cf0 0x7f1ec415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 --2- 172.21.15.53:0/1741482868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec40d3680 0x7f1ec40ba870 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 >> 172.21.15.53:0/1741482868 conn(0x7f1ec409e370 msgr2=0x7f1ec4161b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:47.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 shutdown_connections 2024-05-10T10:38:47.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.853+0000 7f1eca059640 1 -- 172.21.15.53:0/1741482868 wait complete. 2024-05-10T10:38:47.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:38:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.980+0000 7f424be64640 1 Processor -- start 2024-05-10T10:38:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.980+0000 7f424be64640 1 -- start start 2024-05-10T10:38:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f424be64640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4244162990 0x7f42440b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f424be64640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f42440b5f30 0x7f42440d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f424be64640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42440d0830 con 0x7f42440d1aa0 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f424be64640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f42440d10d0 con 0x7f4244162990 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f424be64640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f42440d0b10 con 0x7f42440b5f30 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f4249c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4244162990 0x7f42440b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f4249c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4244162990 0x7f42440b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43798/0 (socket says 172.21.15.53:43798) 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.981+0000 7f4249c00640 1 -- 172.21.15.53:0/990746499 learned_addr learned my addr 172.21.15.53:0/990746499 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.982+0000 7f42493ff640 1 --2- 172.21.15.53:0/990746499 >> v2:172.21.15.135:3300/0 conn(0x7f42440b5f30 0x7f42440d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.982+0000 7f4249c00640 1 -- 172.21.15.53:0/990746499 >> v2:172.21.15.135:3300/0 conn(0x7f42440b5f30 msgr2=0x7f42440d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.982+0000 7f4249c00640 1 --2- 172.21.15.53:0/990746499 >> v2:172.21.15.135:3300/0 conn(0x7f42440b5f30 0x7f42440d59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.982+0000 7f4249c00640 1 -- 172.21.15.53:0/990746499 >> v1:172.21.15.53:6789/0 conn(0x7f42440d1aa0 legacy=0x7f42441630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.982+0000 7f4249c00640 1 -- 172.21.15.53:0/990746499 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440d0df0 con 0x7f4244162990 2024-05-10T10:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.982+0000 7f4248bfe640 1 -- 172.21.15.53:0/990746499 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134847123 0 0) 0x7f42440d0830 con 0x7f42440d1aa0 2024-05-10T10:38:47.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f4249c00640 1 --2- 172.21.15.53:0/990746499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b48a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f424408f750 tx=0x7f423402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3540507d5129b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:47.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f4248bfe640 1 -- 172.21.15.53:0/990746499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234036070 con 0x7f4244162990 2024-05-10T10:38:47.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f4248bfe640 1 -- 172.21.15.53:0/990746499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4234031600 con 0x7f4244162990 2024-05-10T10:38:47.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f4248bfe640 1 -- 172.21.15.53:0/990746499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42340319d0 con 0x7f4244162990 2024-05-10T10:38:47.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f424be64640 1 -- 172.21.15.53:0/990746499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 msgr2=0x7f42440b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f424be64640 1 --2- 172.21.15.53:0/990746499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b48a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f424408f750 tx=0x7f423402a730 comp rx=0 tx=0).stop 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f424be64640 1 -- 172.21.15.53:0/990746499 shutdown_connections 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f424be64640 1 --2- 172.21.15.53:0/990746499 >> v2:172.21.15.135:3300/0 conn(0x7f42440b5f30 0x7f42440d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f424be64640 1 --2- 172.21.15.53:0/990746499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b48a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.983+0000 7f424be64640 1 -- 172.21.15.53:0/990746499 >> 172.21.15.53:0/990746499 conn(0x7f424409e3f0 msgr2=0x7f42440d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.984+0000 7f424be64640 1 -- 172.21.15.53:0/990746499 shutdown_connections 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.984+0000 7f424be64640 1 -- 172.21.15.53:0/990746499 wait complete. 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.985+0000 7f424be64640 1 Processor -- start 2024-05-10T10:38:47.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.985+0000 7f424be64640 1 -- start start 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424be64640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42440b5f30 0x7f42441f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424be64640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42440d1aa0 0x7f42441f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424be64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424be64640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f42440d5ef0 con 0x7f4244162990 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424be64640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4244162d70 con 0x7f42440d1aa0 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424be64640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f42440d1f30 con 0x7f42440b5f30 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43800/0 (socket says 172.21.15.53:43800) 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 -- 172.21.15.53:0/110391516 learned_addr learned my addr 172.21.15.53:0/110391516 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f42493ff640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42440d1aa0 0x7f42441f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f4249c00640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42440b5f30 0x7f42441f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 -- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42440b5f30 msgr2=0x7f42441f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42440b5f30 0x7f42441f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 -- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42440d1aa0 msgr2=0x7f42441f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42440d1aa0 0x7f42441f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 -- 172.21.15.53:0/110391516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440b8d80 con 0x7f4244162990 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f4249c00640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42440b5f30 0x7f42441f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424a401640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b7c90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c4b0 tx=0x7f423c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f42427fc640 1 -- 172.21.15.53:0/110391516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c005480 con 0x7f4244162990 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f42427fc640 1 -- 172.21.15.53:0/110391516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423c0035f0 con 0x7f4244162990 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42440b84e0 con 0x7f4244162990 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.986+0000 7f42427fc640 1 -- 172.21.15.53:0/110391516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c0039c0 con 0x7f4244162990 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.987+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42440b8aa0 con 0x7f4244162990 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.988+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424406a220 con 0x7f4244162990 2024-05-10T10:38:47.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.988+0000 7f42427fc640 1 -- 172.21.15.53:0/110391516 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f423c003b60 con 0x7f4244162990 2024-05-10T10:38:47.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.989+0000 7f42427fc640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4224076470 0x7f4224078930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:47.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.989+0000 7f42427fc640 1 -- 172.21.15.53:0/110391516 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5835+0+0 (secure 0 0 0) 0x7f423c0971d0 con 0x7f4244162990 2024-05-10T10:38:47.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.989+0000 7f4249c00640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4224076470 0x7f4224078930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:47.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.990+0000 7f4249c00640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4224076470 0x7f4224078930 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4234033000 tx=0x7f4234009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:47.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:47.994+0000 7f42427fc640 1 -- 172.21.15.53:0/110391516 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f423c060ab0 con 0x7f4244162990 2024-05-10T10:38:48.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.184+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42440b8f50 con 0x7f4244162990 2024-05-10T10:38:48.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.856+0000 7f42427fc640 1 -- 172.21.15.53:0/110391516 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) ==== 169+0+0 (secure 0 0 0) 0x7f423c0603b0 con 0x7f4244162990 2024-05-10T10:38:48.858 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:38:48.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4224076470 msgr2=0x7f4224078930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:48.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4224076470 0x7f4224078930 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4234033000 tx=0x7f4234009000 comp rx=0 tx=0).stop 2024-05-10T10:38:48.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 msgr2=0x7f42440b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:48.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b7c90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c4b0 tx=0x7f423c00c970 comp rx=0 tx=0).stop 2024-05-10T10:38:48.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 shutdown_connections 2024-05-10T10:38:48.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4224076470 0x7f4224078930 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4244162990 0x7f42440b7c90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42440d1aa0 0x7f42441f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 --2- 172.21.15.53:0/110391516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42440b5f30 0x7f42441f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.859+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 >> 172.21.15.53:0/110391516 conn(0x7f424409e3f0 msgr2=0x7f42440b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:48.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.860+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 shutdown_connections 2024-05-10T10:38:48.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.860+0000 7f424be64640 1 -- 172.21.15.53:0/110391516 wait complete. 2024-05-10T10:38:48.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T10:38:48.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.987+0000 7f6baa03f640 1 Processor -- start 2024-05-10T10:38:48.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.987+0000 7f6baa03f640 1 -- start start 2024-05-10T10:38:48.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6baa03f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6ba4162ce0 0x7f6ba40b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6baa03f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6ba40b4e20 0x7f6ba40d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6baa03f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba40d4190 con 0x7f6ba4162900 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6baa03f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba40cfc40 con 0x7f6ba40b4e20 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6baa03f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba40d3eb0 con 0x7f6ba4162ce0 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6ba40b4e20 0x7f6ba40d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba37fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6ba4162900 0x7f6ba40d3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58374/0 (socket says 172.21.15.53:58374) 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba37fe640 1 -- 172.21.15.53:0/4062594880 learned_addr learned my addr 172.21.15.53:0/4062594880 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba2ffd640 1 --2- 172.21.15.53:0/4062594880 >> v2:172.21.15.135:3300/0 conn(0x7f6ba4162ce0 0x7f6ba40b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba3fff640 1 -- 172.21.15.53:0/4062594880 >> v2:172.21.15.135:3300/0 conn(0x7f6ba4162ce0 msgr2=0x7f6ba40b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba3fff640 1 --2- 172.21.15.53:0/4062594880 >> v2:172.21.15.135:3300/0 conn(0x7f6ba4162ce0 0x7f6ba40b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba3fff640 1 -- 172.21.15.53:0/4062594880 >> v1:172.21.15.53:6789/0 conn(0x7f6ba4162900 legacy=0x7f6ba40d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba27fc640 1 -- 172.21.15.53:0/4062594880 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067412323 0 0) 0x7f6b94001110 con 0x7f6ba4162900 2024-05-10T10:38:48.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.988+0000 7f6ba3fff640 1 -- 172.21.15.53:0/4062594880 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba40cfee0 con 0x7f6ba40b4e20 2024-05-10T10:38:48.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.989+0000 7f6ba3fff640 1 --2- 172.21.15.53:0/4062594880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba40b4e20 0x7f6ba40d09c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40c8bc0 tx=0x7f6b9c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9fa9d2e40ec217 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:48.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.989+0000 7f6ba27fc640 1 -- 172.21.15.53:0/4062594880 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c007740 con 0x7f6ba40b4e20 2024-05-10T10:38:48.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.989+0000 7f6ba27fc640 1 -- 172.21.15.53:0/4062594880 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0078e0 con 0x7f6ba40b4e20 2024-05-10T10:38:48.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.989+0000 7f6ba27fc640 1 -- 172.21.15.53:0/4062594880 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c007cb0 con 0x7f6ba40b4e20 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 -- 172.21.15.53:0/4062594880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba40b4e20 msgr2=0x7f6ba40d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4062594880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba40b4e20 0x7f6ba40d09c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40c8bc0 tx=0x7f6b9c02a5f0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 -- 172.21.15.53:0/4062594880 shutdown_connections 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4062594880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba40b4e20 0x7f6ba40d09c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4062594880 >> v2:172.21.15.135:3300/0 conn(0x7f6ba4162ce0 0x7f6ba40b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 -- 172.21.15.53:0/4062594880 >> 172.21.15.53:0/4062594880 conn(0x7f6ba409e430 msgr2=0x7f6ba40d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 -- 172.21.15.53:0/4062594880 shutdown_connections 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.990+0000 7f6baa03f640 1 -- 172.21.15.53:0/4062594880 wait complete. 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.991+0000 7f6baa03f640 1 Processor -- start 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.991+0000 7f6baa03f640 1 -- start start 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6baa03f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba40b4e20 0x7f6ba415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6baa03f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ba4162900 0x7f6ba415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:48.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6baa03f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba4162ce0 0x7f6ba415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6baa03f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40b4180 con 0x7f6ba4162ce0 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6baa03f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40d3b20 con 0x7f6ba40b4e20 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6baa03f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba40b4300 con 0x7f6ba4162900 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba4162ce0 0x7f6ba415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba4162ce0 0x7f6ba415dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43812/0 (socket says 172.21.15.53:43812) 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 -- 172.21.15.53:0/4163184400 learned_addr learned my addr 172.21.15.53:0/4163184400 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba37fe640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba40b4e20 0x7f6ba415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba2ffd640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ba4162900 0x7f6ba415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 -- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ba4162900 msgr2=0x7f6ba415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ba4162900 0x7f6ba415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 -- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba40b4e20 msgr2=0x7f6ba415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba40b4e20 0x7f6ba415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:48.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba3fff640 1 -- 172.21.15.53:0/4163184400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba40cd920 con 0x7f6ba4162ce0 2024-05-10T10:38:48.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.992+0000 7f6ba2ffd640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ba4162900 0x7f6ba415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:48.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.993+0000 7f6ba3fff640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba4162ce0 0x7f6ba415dbe0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009e50 tx=0x7f6b9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:48.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.993+0000 7f6b8bfff640 1 -- 172.21.15.53:0/4163184400 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c040070 con 0x7f6ba4162ce0 2024-05-10T10:38:48.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.993+0000 7f6b8bfff640 1 -- 172.21.15.53:0/4163184400 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c031cc0 con 0x7f6ba4162ce0 2024-05-10T10:38:48.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.993+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba40bad50 con 0x7f6ba4162ce0 2024-05-10T10:38:48.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.993+0000 7f6b8bfff640 1 -- 172.21.15.53:0/4163184400 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c045680 con 0x7f6ba4162ce0 2024-05-10T10:38:48.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.993+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba40cde80 con 0x7f6ba4162ce0 2024-05-10T10:38:48.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.994+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba40b3da0 con 0x7f6ba4162ce0 2024-05-10T10:38:48.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.995+0000 7f6b8bfff640 1 -- 172.21.15.53:0/4163184400 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6b9c039040 con 0x7f6ba4162ce0 2024-05-10T10:38:48.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.995+0000 7f6b8bfff640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6b900762d0 0x7f6b90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:48.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.995+0000 7f6b8bfff640 1 -- 172.21.15.53:0/4163184400 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f6b9c0b8e50 con 0x7f6ba4162ce0 2024-05-10T10:38:48.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.995+0000 7f6ba37fe640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6b900762d0 0x7f6b90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:48.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:48.996+0000 7f6ba37fe640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6b900762d0 0x7f6b90078790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b94000f60 tx=0x7f6b94007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.000+0000 7f6b8bfff640 1 -- 172.21.15.53:0/4163184400 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6ba40b3da0 con 0x7f6ba4162ce0 2024-05-10T10:38:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.193+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6ba406a220 con 0x7f6ba4162ce0 2024-05-10T10:38:49.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.196+0000 7f6b8bfff640 1 -- 172.21.15.53:0/4163184400 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f6ba40cd920 con 0x7f6ba4162ce0 2024-05-10T10:38:49.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6b900762d0 msgr2=0x7f6b90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6b900762d0 0x7f6b90078790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b94000f60 tx=0x7f6b94007b50 comp rx=0 tx=0).stop 2024-05-10T10:38:49.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba4162ce0 msgr2=0x7f6ba415dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba4162ce0 0x7f6ba415dbe0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009e50 tx=0x7f6b9c004a40 comp rx=0 tx=0).stop 2024-05-10T10:38:49.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 shutdown_connections 2024-05-10T10:38:49.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6b900762d0 0x7f6b90078790 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ba4162ce0 0x7f6ba415dbe0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6ba4162900 0x7f6ba415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 --2- 172.21.15.53:0/4163184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ba40b4e20 0x7f6ba415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 >> 172.21.15.53:0/4163184400 conn(0x7f6ba409e430 msgr2=0x7f6ba40d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:49.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 shutdown_connections 2024-05-10T10:38:49.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.198+0000 7f6baa03f640 1 -- 172.21.15.53:0/4163184400 wait complete. 2024-05-10T10:38:49.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-10T10:38:49.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.314+0000 7f5b5d589640 1 Processor -- start 2024-05-10T10:38:49.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.314+0000 7f5b5d589640 1 -- start start 2024-05-10T10:38:49.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.314+0000 7f5b5d589640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d5cb0 0x7f5b580d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b5d589640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b580b45f0 con 0x7f5b58162750 2024-05-10T10:38:49.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b5d589640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b580b48d0 con 0x7f5b58162b30 2024-05-10T10:38:49.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b5d589640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5b580b4f20 con 0x7f5b580d5cb0 2024-05-10T10:38:49.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b56ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5b58162750 0x7f5b580d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58382/0 (socket says 172.21.15.53:58382) 2024-05-10T10:38:49.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b56ffd640 1 -- 172.21.15.53:0/3921761318 learned_addr learned my addr 172.21.15.53:0/3921761318 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:49.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b4ffff640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d5cb0 0x7f5b580d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415922657 0 0) 0x7f5b580b45f0 con 0x7f5b58162750 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b44003520 con 0x7f5b58162750 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3158783679 0 0) 0x7f5b44003520 con 0x7f5b58162750 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d5cb0 msgr2=0x7f5b580d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d5cb0 0x7f5b580d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 >> v1:172.21.15.135:6789/0 conn(0x7f5b58162b30 legacy=0x7f5b580b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b580b4bb0 con 0x7f5b58162750 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f5b48001670 con 0x7f5b58162750 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 >> v1:172.21.15.53:6789/0 conn(0x7f5b58162750 legacy=0x7f5b580d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b440039e0 0x7f5b44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44004390 0x7f5b44004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b44005100 0x7f5b4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580b4f20 con 0x7f5b44004390 2024-05-10T10:38:49.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4400bf00 con 0x7f5b440039e0 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4400c080 con 0x7f5b44005100 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.315+0000 7f5b4ffff640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44004390 0x7f5b44004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b56ffd640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b440039e0 0x7f5b44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b577fe640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b44005100 0x7f5b4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b4ffff640 1 -- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b44005100 msgr2=0x7f5b4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b4ffff640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b44005100 0x7f5b4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b4ffff640 1 -- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b440039e0 msgr2=0x7f5b44003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b4ffff640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b440039e0 0x7f5b44003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b4ffff640 1 -- 172.21.15.53:0/3921761318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b44003520 con 0x7f5b44004390 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b577fe640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b44005100 0x7f5b4400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b56ffd640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b440039e0 0x7f5b44003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:49.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.316+0000 7f5b4ffff640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44004390 0x7f5b44004860 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009ac0 tx=0x7f5b44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90519d4b7b4939c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40036070 con 0x7f5b44004390 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b40031400 con 0x7f5b44004390 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b567fc640 1 -- 172.21.15.53:0/3921761318 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40009690 con 0x7f5b44004390 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b5d589640 1 -- 172.21.15.53:0/3921761318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44004390 msgr2=0x7f5b44004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b5d589640 1 --2- 172.21.15.53:0/3921761318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44004390 0x7f5b44004860 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009ac0 tx=0x7f5b44005830 comp rx=0 tx=0).stop 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b577fe640 1 -- 172.21.15.53:0/3921761318 reap_dead start 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b5d589640 1 -- 172.21.15.53:0/3921761318 shutdown_connections 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b5d589640 1 -- 172.21.15.53:0/3921761318 >> 172.21.15.53:0/3921761318 conn(0x7f5b5809e3b0 msgr2=0x7f5b580d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b5d589640 1 -- 172.21.15.53:0/3921761318 shutdown_connections 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.317+0000 7f5b5d589640 1 -- 172.21.15.53:0/3921761318 wait complete. 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.318+0000 7f5b5d589640 1 Processor -- start 2024-05-10T10:38:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b5d589640 1 -- start start 2024-05-10T10:38:49.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b5d589640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b44004390 0x7f5b58151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b5d589640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b581514b0 0x7f5b5806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b5d589640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5806dce0 0x7f5b580a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b5d589640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580d3a00 con 0x7f5b5806dce0 2024-05-10T10:38:49.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b5d589640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580d3880 con 0x7f5b44004390 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b5d589640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5b580d3b80 con 0x7f5b581514b0 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5806dce0 0x7f5b580a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5806dce0 0x7f5b580a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43822/0 (socket says 172.21.15.53:43822) 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b577fe640 1 -- 172.21.15.53:0/2083426163 learned_addr learned my addr 172.21.15.53:0/2083426163 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b56ffd640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b44004390 0x7f5b58151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.319+0000 7f5b4ffff640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b581514b0 0x7f5b5806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b577fe640 1 -- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b581514b0 msgr2=0x7f5b5806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b577fe640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b581514b0 0x7f5b5806d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b577fe640 1 -- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b44004390 msgr2=0x7f5b58151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b577fe640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b44004390 0x7f5b58151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b577fe640 1 -- 172.21.15.53:0/2083426163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b58152440 con 0x7f5b5806dce0 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b577fe640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5806dce0 0x7f5b580a00b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5b580d32f0 tx=0x7f5b5000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b4f7fe640 1 -- 172.21.15.53:0/2083426163 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50003b80 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b4f7fe640 1 -- 172.21.15.53:0/2083426163 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b50003d40 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b4f7fe640 1 -- 172.21.15.53:0/2083426163 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50006b50 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b580a0e30 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.320+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b580a13f0 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.321+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5806a220 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.322+0000 7f5b4f7fe640 1 -- 172.21.15.53:0/2083426163 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b5001a020 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.322+0000 7f5b4f7fe640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b38076300 0x7f5b380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.322+0000 7f5b4f7fe640 1 -- 172.21.15.53:0/2083426163 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f5b50096970 con 0x7f5b5806dce0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.323+0000 7f5b56ffd640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b38076300 0x7f5b380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.323+0000 7f5b56ffd640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b38076300 0x7f5b380787c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5b48000c00 tx=0x7f5b48007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.326+0000 7f5b4f7fe640 1 -- 172.21.15.53:0/2083426163 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5b5005fe30 con 0x7f5b5806dce0 2024-05-10T10:38:49.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.591+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f5b580d2420 con 0x7f5b5806dce0 2024-05-10T10:38:49.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.857+0000 7f5b4f7fe640 1 -- 172.21.15.53:0/2083426163 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v255) ==== 166+0+0 (secure 0 0 0) 0x7f5b50064130 con 0x7f5b5806dce0 2024-05-10T10:38:49.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b38076300 msgr2=0x7f5b380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b38076300 0x7f5b380787c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5b48000c00 tx=0x7f5b48007040 comp rx=0 tx=0).stop 2024-05-10T10:38:49.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5806dce0 msgr2=0x7f5b580a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5806dce0 0x7f5b580a00b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5b580d32f0 tx=0x7f5b5000ecd0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 shutdown_connections 2024-05-10T10:38:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b38076300 0x7f5b380787c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5806dce0 0x7f5b580a00b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b581514b0 0x7f5b5806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 --2- 172.21.15.53:0/2083426163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b44004390 0x7f5b58151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 >> 172.21.15.53:0/2083426163 conn(0x7f5b5809e3b0 msgr2=0x7f5b581603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 shutdown_connections 2024-05-10T10:38:49.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.859+0000 7f5b5d589640 1 -- 172.21.15.53:0/2083426163 wait complete. 2024-05-10T10:38:49.872 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-10T10:38:49.872 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-10T10:38:49.872 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-10T10:38:49.873 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:38:49.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.975+0000 7ff7ee59e640 1 Processor -- start 2024-05-10T10:38:49.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.975+0000 7ff7ee59e640 1 -- start start 2024-05-10T10:38:49.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.976+0000 7ff7ee59e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7e80d19b0 0x7ff7e815f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.976+0000 7ff7ee59e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 0x7ff7e815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.976+0000 7ff7ee59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7e80d0fd0 con 0x7ff7e80d1d90 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.976+0000 7ff7ee59e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e815fb10 con 0x7ff7e80d19b0 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.976+0000 7ff7ee59e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e80d0b40 con 0x7ff7e80b5b60 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.976+0000 7ff7ecd9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff7e80d1d90 0x7ff7e80d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58390/0 (socket says 172.21.15.53:58390) 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.976+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/607023989 learned_addr learned my addr 172.21.15.53:0/607023989 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814896165 0 0) 0x7ff7e80d0fd0 con 0x7ff7e80d1d90 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7edd9d640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 0x7ff7e815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d0003520 con 0x7ff7e80d1d90 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ed59c640 1 --2- 172.21.15.53:0/607023989 >> v2:172.21.15.135:3300/0 conn(0x7ff7e80d19b0 0x7ff7e815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2219356059 0 0) 0x7ff7d0003520 con 0x7ff7e80d1d90 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 msgr2=0x7ff7e815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 0x7ff7e815f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 >> v2:172.21.15.135:3300/0 conn(0x7ff7e80d19b0 msgr2=0x7ff7e815f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 --2- 172.21.15.53:0/607023989 >> v2:172.21.15.135:3300/0 conn(0x7ff7e80d19b0 0x7ff7e815f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e80d2310 con 0x7ff7e80d1d90 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ed59c640 1 --2- 172.21.15.53:0/607023989 >> v2:172.21.15.135:3300/0 conn(0x7ff7e80d19b0 0x7ff7e815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7ff7dc0019f0 con 0x7ff7e80d1d90 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 >> v1:172.21.15.53:6789/0 conn(0x7ff7e80d1d90 legacy=0x7ff7e80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d00039b0 0x7ff7d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0004800 0x7ff7d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 0x7ff7d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e80d0b40 con 0x7ff7d0004800 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d000baa0 con 0x7ff7d00039b0 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d000bca0 con 0x7ff7e80b5b60 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ecd9b640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0004800 0x7ff7d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ed59c640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d00039b0 0x7ff7d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7edd9d640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 0x7ff7d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 msgr2=0x7ff7d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ecd9b640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e80b5b60 0x7ff7d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d00039b0 msgr2=0x7ff7d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:49.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ecd9b640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d00039b0 0x7ff7d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/607023989 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d0003520 con 0x7ff7d0004800 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.977+0000 7ff7ed59c640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d00039b0 0x7ff7d0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.978+0000 7ff7ecd9b640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0004800 0x7ff7d0004c80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00a870 tx=0x7ff7d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d04503fb8506254 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.978+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00a4c0 con 0x7ff7d0004800 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.978+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7dc02f070 con 0x7ff7d0004800 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.978+0000 7ff7e7fff640 1 -- 172.21.15.53:0/607023989 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc02bcc0 con 0x7ff7d0004800 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.979+0000 7ff7ee59e640 1 -- 172.21.15.53:0/607023989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0004800 msgr2=0x7ff7d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.979+0000 7ff7ee59e640 1 --2- 172.21.15.53:0/607023989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0004800 0x7ff7d0004c80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00a870 tx=0x7ff7d0005830 comp rx=0 tx=0).stop 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.979+0000 7ff7edd9d640 1 -- 172.21.15.53:0/607023989 reap_dead start 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.979+0000 7ff7ee59e640 1 -- 172.21.15.53:0/607023989 shutdown_connections 2024-05-10T10:38:49.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.979+0000 7ff7ee59e640 1 -- 172.21.15.53:0/607023989 >> 172.21.15.53:0/607023989 conn(0x7ff7e80d4f60 msgr2=0x7ff7e80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:49.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.979+0000 7ff7ee59e640 1 -- 172.21.15.53:0/607023989 shutdown_connections 2024-05-10T10:38:49.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.979+0000 7ff7ee59e640 1 -- 172.21.15.53:0/607023989 wait complete. 2024-05-10T10:38:49.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.980+0000 7ff7ee59e640 1 Processor -- start 2024-05-10T10:38:49.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.980+0000 7ff7ee59e640 1 -- start start 2024-05-10T10:38:49.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ee59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d0004800 0x7ff7e815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ee59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e8158f70 0x7ff7e81f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ee59e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e81593f0 0x7ff7e81f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ee59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e80d07d0 con 0x7ff7e8158f70 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ee59e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e80d28d0 con 0x7ff7d0004800 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ee59e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e80c9ac0 con 0x7ff7e81593f0 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e8158f70 0x7ff7e81f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e8158f70 0x7ff7e81f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43852/0 (socket says 172.21.15.53:43852) 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/2011906183 learned_addr learned my addr 172.21.15.53:0/2011906183 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7edd9d640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e81593f0 0x7ff7e81f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ed59c640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d0004800 0x7ff7e815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e81593f0 msgr2=0x7ff7e81f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e81593f0 0x7ff7e81f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d0004800 msgr2=0x7ff7e815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d0004800 0x7ff7e815af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.981+0000 7ff7ecd9b640 1 -- 172.21.15.53:0/2011906183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e815b630 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.982+0000 7ff7ecd9b640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e8158f70 0x7ff7e81f8450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00a990 tx=0x7ff7dc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.982+0000 7ff7e5ffb640 1 -- 172.21.15.53:0/2011906183 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc044070 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.982+0000 7ff7e5ffb640 1 -- 172.21.15.53:0/2011906183 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7dc02bc70 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.982+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e815b910 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.982+0000 7ff7e5ffb640 1 -- 172.21.15.53:0/2011906183 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0346b0 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.982+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e815bed0 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.983+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e806a220 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.983+0000 7ff7e5ffb640 1 -- 172.21.15.53:0/2011906183 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff7dc047070 con 0x7ff7e8158f70 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.984+0000 7ff7e5ffb640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7d40762d0 0x7ff7d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:49.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.984+0000 7ff7e5ffb640 1 -- 172.21.15.53:0/2011906183 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7ff7dc0b75d0 con 0x7ff7e8158f70 2024-05-10T10:38:49.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.984+0000 7ff7ed59c640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7d40762d0 0x7ff7d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:49.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.985+0000 7ff7ed59c640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7d40762d0 0x7ff7d4078790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80d3f60 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:49.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:49.988+0000 7ff7e5ffb640 1 -- 172.21.15.53:0/2011906183 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff7dc080a00 con 0x7ff7e8158f70 2024-05-10T10:38:50.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.181+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7e80cf540 con 0x7ff7e8158f70 2024-05-10T10:38:50.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.182+0000 7ff7e5ffb640 1 -- 172.21.15.53:0/2011906183 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+13424 (secure 0 0 0) 0x7ff7dc02be20 con 0x7ff7e8158f70 2024-05-10T10:38:50.184 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:50.184 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:38:48.852111+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6834/3202950630":"2024-05-11T10:38:45.842639+0000","172.21.15.53:6832/3202950630":"2024-05-11T10:38:45.842639+0000","172.21.15.53:6834/1984524370":"2024-05-11T10:37:31.312633+0000","172.21.15.53:6832/1984524370":"2024-05-11T10:37:31.312633+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:38:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7d40762d0 msgr2=0x7ff7d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7d40762d0 0x7ff7d4078790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80d3f60 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2024-05-10T10:38:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e8158f70 msgr2=0x7ff7e81f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e8158f70 0x7ff7e81f8450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00a990 tx=0x7ff7dc001dc0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 shutdown_connections 2024-05-10T10:38:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff7d40762d0 0x7ff7d4078790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7e81593f0 0x7ff7e81f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7e8158f70 0x7ff7e81f8450 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 --2- 172.21.15.53:0/2011906183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7d0004800 0x7ff7e815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 >> 172.21.15.53:0/2011906183 conn(0x7ff7e80d4f60 msgr2=0x7ff7e80ba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:50.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 shutdown_connections 2024-05-10T10:38:50.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.184+0000 7ff7ee59e640 1 -- 172.21.15.53:0/2011906183 wait complete. 2024-05-10T10:38:50.197 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:33:28.171977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T10:38:50.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:38:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.303+0000 7f7ac6fb6640 1 Processor -- start 2024-05-10T10:38:50.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.304+0000 7f7ac6fb6640 1 -- start start 2024-05-10T10:38:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.304+0000 7f7ac6fb6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ac015f400 0x7f7ac006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.304+0000 7f7ac6fb6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ac006e290 0x7f7ac00d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.304+0000 7f7ac6fb6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac00d1ed0 con 0x7f7ac00d1620 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.304+0000 7f7ac6fb6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac006e6f0 con 0x7f7ac006e290 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.304+0000 7f7ac6fb6640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac00d21b0 con 0x7f7ac015f400 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.304+0000 7f7abffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ac006e290 0x7f7ac00d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7abffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ac006e290 0x7f7ac00d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43854/0 (socket says 172.21.15.53:43854) 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7abffff640 1 -- 172.21.15.53:0/84550097 learned_addr learned my addr 172.21.15.53:0/84550097 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7abffff640 1 -- 172.21.15.53:0/84550097 >> v2:172.21.15.135:3300/0 conn(0x7f7ac015f400 msgr2=0x7f7ac006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7ac4d52640 1 --2- 172.21.15.53:0/84550097 >> v2:172.21.15.135:3300/0 conn(0x7f7ac015f400 0x7f7ac006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7abffff640 1 --2- 172.21.15.53:0/84550097 >> v2:172.21.15.135:3300/0 conn(0x7f7ac015f400 0x7f7ac006dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7abffff640 1 -- 172.21.15.53:0/84550097 >> v1:172.21.15.135:6789/0 conn(0x7f7ac00d1620 legacy=0x7f7ac00cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:50.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7abffff640 1 -- 172.21.15.53:0/84550097 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac00d2490 con 0x7f7ac006e290 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.305+0000 7f7abffff640 1 --2- 172.21.15.53:0/84550097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac00d10e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0074680 tx=0x7f7aa802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8adfa76497bde6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7abf7fe640 1 -- 172.21.15.53:0/84550097 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8037070 con 0x7f7ac006e290 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7abf7fe640 1 -- 172.21.15.53:0/84550097 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa8032440 con 0x7f7ac006e290 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7abf7fe640 1 -- 172.21.15.53:0/84550097 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8032810 con 0x7f7ac006e290 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/84550097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 msgr2=0x7f7ac00d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/84550097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac00d10e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0074680 tx=0x7f7aa802a230 comp rx=0 tx=0).stop 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/84550097 shutdown_connections 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/84550097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac00d10e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/84550097 >> v2:172.21.15.135:3300/0 conn(0x7f7ac015f400 0x7f7ac006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/84550097 >> 172.21.15.53:0/84550097 conn(0x7f7ac009ba20 msgr2=0x7f7ac009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:50.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/84550097 shutdown_connections 2024-05-10T10:38:50.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.306+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/84550097 wait complete. 2024-05-10T10:38:50.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.307+0000 7f7ac6fb6640 1 Processor -- start 2024-05-10T10:38:50.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.307+0000 7f7ac6fb6640 1 -- start start 2024-05-10T10:38:50.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac6fb6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac0157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac6fb6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ac00d1620 0x7f7ac00b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac6fb6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ac015f400 0x7f7ac00b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac6fb6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac015f7e0 con 0x7f7ac006e290 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac6fb6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac00d1a70 con 0x7f7ac015f400 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac6fb6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac00c7e10 con 0x7f7ac00d1620 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac4d52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac0157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac4d52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac0157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43866/0 (socket says 172.21.15.53:43866) 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.308+0000 7f7ac4d52640 1 -- 172.21.15.53:0/3713542744 learned_addr learned my addr 172.21.15.53:0/3713542744 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac4d52640 1 -- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ac00d1620 msgr2=0x7f7ac00b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac4d52640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ac00d1620 0x7f7ac00b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac4d52640 1 -- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ac015f400 msgr2=0x7f7ac00b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac4d52640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ac015f400 0x7f7ac00b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac4d52640 1 -- 172.21.15.53:0/3713542744 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac00b56a0 con 0x7f7ac006e290 2024-05-10T10:38:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac4d52640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac0157d90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c7f0 tx=0x7f7ab800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7abd7fa640 1 -- 172.21.15.53:0/3713542744 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8015070 con 0x7f7ac006e290 2024-05-10T10:38:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7abd7fa640 1 -- 172.21.15.53:0/3713542744 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab8004220 con 0x7f7ac006e290 2024-05-10T10:38:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac00b4e00 con 0x7f7ac006e290 2024-05-10T10:38:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7abd7fa640 1 -- 172.21.15.53:0/3713542744 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8007620 con 0x7f7ac006e290 2024-05-10T10:38:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.309+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac00b53c0 con 0x7f7ac006e290 2024-05-10T10:38:50.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.310+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac00693f0 con 0x7f7ac006e290 2024-05-10T10:38:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.311+0000 7f7abd7fa640 1 -- 172.21.15.53:0/3713542744 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7ab80079d0 con 0x7f7ac006e290 2024-05-10T10:38:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.311+0000 7f7abd7fa640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7aa00762d0 0x7f7aa0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.312+0000 7f7abd7fa640 1 -- 172.21.15.53:0/3713542744 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f7ab80977e0 con 0x7f7ac006e290 2024-05-10T10:38:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.312+0000 7f7abffff640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7aa00762d0 0x7f7aa0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.312+0000 7f7abffff640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7aa00762d0 0x7f7aa0078790 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8034000 tx=0x7f7aa802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:50.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.316+0000 7f7abd7fa640 1 -- 172.21.15.53:0/3713542744 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7ab8061290 con 0x7f7ac006e290 2024-05-10T10:38:50.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.505+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ac00b1de0 con 0x7f7ac006e290 2024-05-10T10:38:50.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.506+0000 7f7abd7fa640 1 -- 172.21.15.53:0/3713542744 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f7ab8069280 con 0x7f7ac006e290 2024-05-10T10:38:50.507 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-05-10T10:38:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.507+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7aa00762d0 msgr2=0x7f7aa0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.507+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7aa00762d0 0x7f7aa0078790 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8034000 tx=0x7f7aa802c000 comp rx=0 tx=0).stop 2024-05-10T10:38:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.507+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 msgr2=0x7f7ac0157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.507+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac0157d90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c7f0 tx=0x7f7ab800ccb0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 shutdown_connections 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7aa00762d0 0x7f7aa0078790 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ac015f400 0x7f7ac00b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ac00d1620 0x7f7ac00b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 --2- 172.21.15.53:0/3713542744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ac006e290 0x7f7ac0157d90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 >> 172.21.15.53:0/3713542744 conn(0x7f7ac009ba20 msgr2=0x7f7ac00710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 shutdown_connections 2024-05-10T10:38:50.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.508+0000 7f7ac6fb6640 1 -- 172.21.15.53:0/3713542744 wait complete. 2024-05-10T10:38:50.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-10T10:38:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.622+0000 7f220a10a640 1 Processor -- start 2024-05-10T10:38:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.622+0000 7f220a10a640 1 -- start start 2024-05-10T10:38:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f220a10a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2204162750 0x7f22040d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f220a10a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2204162b30 0x7f22040b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f220a10a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22040b45f0 con 0x7f22040d5cb0 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f220a10a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f22040b4f20 con 0x7f2204162750 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f220a10a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f22040d0fe0 con 0x7f2204162b30 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f22037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2204162750 0x7f22040d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f22037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2204162750 0x7f22040d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43878/0 (socket says 172.21.15.53:43878) 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f22037fe640 1 -- 172.21.15.53:0/1749719925 learned_addr learned my addr 172.21.15.53:0/1749719925 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.623+0000 7f2203fff640 1 --2- 172.21.15.53:0/1749719925 >> v2:172.21.15.135:3300/0 conn(0x7f2204162b30 0x7f22040b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.624+0000 7f22037fe640 1 -- 172.21.15.53:0/1749719925 >> v2:172.21.15.135:3300/0 conn(0x7f2204162b30 msgr2=0x7f22040b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.624+0000 7f22037fe640 1 --2- 172.21.15.53:0/1749719925 >> v2:172.21.15.135:3300/0 conn(0x7f2204162b30 0x7f22040b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.624+0000 7f22037fe640 1 -- 172.21.15.53:0/1749719925 >> v1:172.21.15.135:6789/0 conn(0x7f22040d5cb0 legacy=0x7f22040d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.624+0000 7f22037fe640 1 -- 172.21.15.53:0/1749719925 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22040b48d0 con 0x7f2204162750 2024-05-10T10:38:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.624+0000 7f22037fe640 1 --2- 172.21.15.53:0/1749719925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040d0670 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f22040d32c0 tx=0x7f21f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70544edb9ced1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.624+0000 7f22027fc640 1 -- 172.21.15.53:0/1749719925 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037070 con 0x7f2204162750 2024-05-10T10:38:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f22027fc640 1 -- 172.21.15.53:0/1749719925 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f4032440 con 0x7f2204162750 2024-05-10T10:38:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f22027fc640 1 -- 172.21.15.53:0/1749719925 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4032810 con 0x7f2204162750 2024-05-10T10:38:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 -- 172.21.15.53:0/1749719925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 msgr2=0x7f22040d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 --2- 172.21.15.53:0/1749719925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040d0670 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f22040d32c0 tx=0x7f21f402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 -- 172.21.15.53:0/1749719925 shutdown_connections 2024-05-10T10:38:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 --2- 172.21.15.53:0/1749719925 >> v2:172.21.15.135:3300/0 conn(0x7f2204162b30 0x7f22040b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 --2- 172.21.15.53:0/1749719925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040d0670 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 -- 172.21.15.53:0/1749719925 >> 172.21.15.53:0/1749719925 conn(0x7f220409e3b0 msgr2=0x7f22040d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 -- 172.21.15.53:0/1749719925 shutdown_connections 2024-05-10T10:38:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.625+0000 7f220a10a640 1 -- 172.21.15.53:0/1749719925 wait complete. 2024-05-10T10:38:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.626+0000 7f220a10a640 1 Processor -- start 2024-05-10T10:38:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f220a10a640 1 -- start start 2024-05-10T10:38:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f220a10a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040ba720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f220a10a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2204162b30 0x7f2204156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f220a10a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22041f5300 0x7f22041f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f220a10a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f22040d0d30 con 0x7f2204162750 2024-05-10T10:38:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f220a10a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22040d0bb0 con 0x7f2204162b30 2024-05-10T10:38:50.631 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f220a10a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f22040d3d40 con 0x7f22041f5300 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f22037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f22037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040ba720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43886/0 (socket says 172.21.15.53:43886) 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f22037fe640 1 -- 172.21.15.53:0/2919730694 learned_addr learned my addr 172.21.15.53:0/2919730694 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.627+0000 7f2203fff640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22041f5300 0x7f22041f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f2202ffd640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2204162b30 0x7f2204156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f22037fe640 1 -- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22041f5300 msgr2=0x7f22041f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f22037fe640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22041f5300 0x7f22041f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f22037fe640 1 -- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2204162b30 msgr2=0x7f2204156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f22037fe640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2204162b30 0x7f2204156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f22037fe640 1 -- 172.21.15.53:0/2919730694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22040cd470 con 0x7f2204162750 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f22037fe640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040ba720 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f21f402a220 tx=0x7f21f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f21ebfff640 1 -- 172.21.15.53:0/2919730694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037040 con 0x7f2204162750 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f21ebfff640 1 -- 172.21.15.53:0/2919730694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f4042070 con 0x7f2204162750 2024-05-10T10:38:50.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22041576a0 con 0x7f2204162750 2024-05-10T10:38:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f21ebfff640 1 -- 172.21.15.53:0/2919730694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4030880 con 0x7f2204162750 2024-05-10T10:38:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.628+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22040cceb0 con 0x7f2204162750 2024-05-10T10:38:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.629+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220406a220 con 0x7f2204162750 2024-05-10T10:38:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.630+0000 7f21ebfff640 1 -- 172.21.15.53:0/2919730694 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f21f4030a20 con 0x7f2204162750 2024-05-10T10:38:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.630+0000 7f21ebfff640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f21f00762d0 0x7f21f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.630+0000 7f21ebfff640 1 -- 172.21.15.53:0/2919730694 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f21f40b7d90 con 0x7f2204162750 2024-05-10T10:38:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.631+0000 7f2202ffd640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f21f00762d0 0x7f21f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.631+0000 7f2202ffd640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f21f00762d0 0x7f21f0078790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22040d2350 tx=0x7f21f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.635+0000 7f21ebfff640 1 -- 172.21.15.53:0/2919730694 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f21f4081810 con 0x7f2204162750 2024-05-10T10:38:50.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:50.900+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f2204157160 con 0x7f2204162750 2024-05-10T10:38:51.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.862+0000 7f21ebfff640 1 -- 172.21.15.53:0/2919730694 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v257) ==== 170+0+0 (secure 0 0 0) 0x7f21f4081110 con 0x7f2204162750 2024-05-10T10:38:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.865+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f21f00762d0 msgr2=0x7f21f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.865+0000 7f220a10a640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f21f00762d0 0x7f21f0078790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22040d2350 tx=0x7f21f8009040 comp rx=0 tx=0).stop 2024-05-10T10:38:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.865+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 msgr2=0x7f22040ba720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.865+0000 7f220a10a640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040ba720 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f21f402a220 tx=0x7f21f40046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 shutdown_connections 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f21f00762d0 0x7f21f0078790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22041f5300 0x7f22041f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2204162b30 0x7f2204156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 --2- 172.21.15.53:0/2919730694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2204162750 0x7f22040ba720 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 >> 172.21.15.53:0/2919730694 conn(0x7f220409e3b0 msgr2=0x7f220406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 shutdown_connections 2024-05-10T10:38:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.866+0000 7f220a10a640 1 -- 172.21.15.53:0/2919730694 wait complete. 2024-05-10T10:38:51.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T10:38:51.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.992+0000 7eff01d4e640 1 Processor -- start 2024-05-10T10:38:51.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.992+0000 7eff01d4e640 1 -- start start 2024-05-10T10:38:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7eff01d4e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efefc06f030 0x7efefc160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7eff01d4e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efefc06f410 0x7efefc15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7eff01d4e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc15c9e0 0x7efefc0cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7eff01d4e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7efefc0ccb60 con 0x7efefc06f030 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7eff01d4e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efefc15cfb0 con 0x7efefc06f410 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7eff01d4e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efefc15d290 con 0x7efefc15c9e0 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7efefb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efefc06f030 0x7efefc160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7efefb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efefc06f030 0x7efefc160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43900/0 (socket says 172.21.15.53:43900) 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.993+0000 7efefb7fe640 1 -- 172.21.15.53:0/3393674071 learned_addr learned my addr 172.21.15.53:0/3393674071 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefaffd640 1 --2- 172.21.15.53:0/3393674071 >> v2:172.21.15.135:3300/0 conn(0x7efefc06f410 0x7efefc15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefbfff640 1 --2- 172.21.15.53:0/3393674071 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc15c9e0 0x7efefc0cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefb7fe640 1 -- 172.21.15.53:0/3393674071 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc15c9e0 msgr2=0x7efefc0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefb7fe640 1 --2- 172.21.15.53:0/3393674071 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc15c9e0 0x7efefc0cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefb7fe640 1 -- 172.21.15.53:0/3393674071 >> v2:172.21.15.135:3300/0 conn(0x7efefc06f410 msgr2=0x7efefc15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefb7fe640 1 --2- 172.21.15.53:0/3393674071 >> v2:172.21.15.135:3300/0 conn(0x7efefc06f410 0x7efefc15c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefb7fe640 1 -- 172.21.15.53:0/3393674071 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc15fd30 con 0x7efefc06f030 2024-05-10T10:38:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefbfff640 1 --2- 172.21.15.53:0/3393674071 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc15c9e0 0x7efefc0cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.994+0000 7efefaffd640 1 --2- 172.21.15.53:0/3393674071 >> v2:172.21.15.135:3300/0 conn(0x7efefc06f410 0x7efefc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7efefb7fe640 1 --2- 172.21.15.53:0/3393674071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc06f030 0x7efefc160530 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7efefc06ffd0 tx=0x7efeec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf302961e91beda1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7efefa7fc640 1 -- 172.21.15.53:0/3393674071 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec037070 con 0x7efefc06f030 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7efefa7fc640 1 -- 172.21.15.53:0/3393674071 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec032440 con 0x7efefc06f030 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7efefa7fc640 1 -- 172.21.15.53:0/3393674071 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec032810 con 0x7efefc06f030 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 -- 172.21.15.53:0/3393674071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc06f030 msgr2=0x7efefc160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3393674071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc06f030 0x7efefc160530 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7efefc06ffd0 tx=0x7efeec02a1e0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 -- 172.21.15.53:0/3393674071 shutdown_connections 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3393674071 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc15c9e0 0x7efefc0cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3393674071 >> v2:172.21.15.135:3300/0 conn(0x7efefc06f410 0x7efefc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3393674071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc06f030 0x7efefc160530 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 -- 172.21.15.53:0/3393674071 >> 172.21.15.53:0/3393674071 conn(0x7efefc0d0c90 msgr2=0x7efefc0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 -- 172.21.15.53:0/3393674071 shutdown_connections 2024-05-10T10:38:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.995+0000 7eff01d4e640 1 -- 172.21.15.53:0/3393674071 wait complete. 2024-05-10T10:38:52.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 Processor -- start 2024-05-10T10:38:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 -- start start 2024-05-10T10:38:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc06f030 0x7efefc14b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc06f410 0x7efefc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc15c9e0 0x7efefc147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efefc0cb910 con 0x7efefc15c9e0 2024-05-10T10:38:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efefc15fa40 con 0x7efefc06f410 2024-05-10T10:38:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.997+0000 7eff01d4e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efefc159d70 con 0x7efefc06f030 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc15c9e0 0x7efefc147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc15c9e0 0x7efefc147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43914/0 (socket says 172.21.15.53:43914) 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 -- 172.21.15.53:0/3288283936 learned_addr learned my addr 172.21.15.53:0/3288283936 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefaffd640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc06f410 0x7efefc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefb7fe640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc06f030 0x7efefc14b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 -- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc06f030 msgr2=0x7efefc14b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc06f030 0x7efefc14b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 -- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc06f410 msgr2=0x7efefc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc06f410 0x7efefc0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 -- 172.21.15.53:0/3288283936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc14beb0 con 0x7efefc15c9e0 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efefbfff640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc15c9e0 0x7efefc147dc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efef4007d40 tx=0x7efef400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efee3fff640 1 -- 172.21.15.53:0/3288283936 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef401a070 con 0x7efefc15c9e0 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efee3fff640 1 -- 172.21.15.53:0/3288283936 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef4003990 con 0x7efefc15c9e0 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7efee3fff640 1 -- 172.21.15.53:0/3288283936 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4003d80 con 0x7efefc15c9e0 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.998+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc0c4ac0 con 0x7efefc15c9e0 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.999+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc1f69a0 con 0x7efefc15c9e0 2024-05-10T10:38:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:51.999+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc06a220 con 0x7efefc15c9e0 2024-05-10T10:38:52.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.000+0000 7efee3fff640 1 -- 172.21.15.53:0/3288283936 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efef401d070 con 0x7efefc15c9e0 2024-05-10T10:38:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.001+0000 7efee3fff640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efee80762d0 0x7efee8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.001+0000 7efee3fff640 1 -- 172.21.15.53:0/3288283936 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7efef4097600 con 0x7efefc15c9e0 2024-05-10T10:38:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.001+0000 7efefb7fe640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efee80762d0 0x7efee8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.001+0000 7efefb7fe640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efee80762d0 0x7efee8078790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efeec034000 tx=0x7efeec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.005+0000 7efee3fff640 1 -- 172.21.15.53:0/3288283936 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7efefc14beb0 con 0x7efefc15c9e0 2024-05-10T10:38:52.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.195+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efefc0cf020 con 0x7efefc15c9e0 2024-05-10T10:38:52.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.199+0000 7efee3fff640 1 -- 172.21.15.53:0/3288283936 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7efef4010070 con 0x7efefc15c9e0 2024-05-10T10:38:52.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efee80762d0 msgr2=0x7efee8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efee80762d0 0x7efee8078790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efeec034000 tx=0x7efeec02e000 comp rx=0 tx=0).stop 2024-05-10T10:38:52.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc15c9e0 msgr2=0x7efefc147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc15c9e0 0x7efefc147dc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efef4007d40 tx=0x7efef400d4a0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 shutdown_connections 2024-05-10T10:38:52.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efee80762d0 0x7efee8078790 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc15c9e0 0x7efefc147dc0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc06f410 0x7efefc0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 --2- 172.21.15.53:0/3288283936 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc06f030 0x7efefc14b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 >> 172.21.15.53:0/3288283936 conn(0x7efefc0d0c90 msgr2=0x7efefc0ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:52.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 shutdown_connections 2024-05-10T10:38:52.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.201+0000 7eff01d4e640 1 -- 172.21.15.53:0/3288283936 wait complete. 2024-05-10T10:38:52.214 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:38:52.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:38:52.267 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:38:52.313 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:38:52.313 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:38:52.314 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:52.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:38:52.330 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:38:52.330 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:52.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:38:52.348 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:38:52.349 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:52.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:38:52.387 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:38:52.387 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:38:52.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:38:52.406 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T10:38:52.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:38:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.528+0000 7f1fe6d19640 1 Processor -- start 2024-05-10T10:38:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.528+0000 7f1fe6d19640 1 -- start start 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe6d19640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fe015f3b0 0x7f1fe00d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe6d19640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe00d5d30 0x7f1fe00d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe6d19640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe00d30b0 con 0x7f1fe015f790 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe6d19640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe00d3390 con 0x7f1fe015f3b0 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe6d19640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe00c8c80 con 0x7f1fe00d5d30 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe4ab5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fe015f3b0 0x7f1fe00d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe4ab5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fe015f3b0 0x7f1fe00d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43928/0 (socket says 172.21.15.53:43928) 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe4ab5640 1 -- 172.21.15.53:0/3830945694 learned_addr learned my addr 172.21.15.53:0/3830945694 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe52b6640 1 --2- 172.21.15.53:0/3830945694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe00d5d30 0x7f1fe00d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe4ab5640 1 -- 172.21.15.53:0/3830945694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe00d5d30 msgr2=0x7f1fe00d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe4ab5640 1 --2- 172.21.15.53:0/3830945694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe00d5d30 0x7f1fe00d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe4ab5640 1 -- 172.21.15.53:0/3830945694 >> v1:172.21.15.135:6789/0 conn(0x7f1fe015f790 legacy=0x7f1fe00c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:52.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.529+0000 7f1fe4ab5640 1 -- 172.21.15.53:0/3830945694 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe00c8f60 con 0x7f1fe015f3b0 2024-05-10T10:38:52.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.530+0000 7f1fe4ab5640 1 --2- 172.21.15.53:0/3830945694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe015f3b0 0x7f1fe00d36f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009870 tx=0x7f1fd002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec6d85f66353464 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.530+0000 7f1fdf7fe640 1 -- 172.21.15.53:0/3830945694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0037070 con 0x7f1fe015f3b0 2024-05-10T10:38:52.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.530+0000 7f1fdf7fe640 1 -- 172.21.15.53:0/3830945694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd0032440 con 0x7f1fe015f3b0 2024-05-10T10:38:52.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.530+0000 7f1fdf7fe640 1 -- 172.21.15.53:0/3830945694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0032810 con 0x7f1fe015f3b0 2024-05-10T10:38:52.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 -- 172.21.15.53:0/3830945694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe015f3b0 msgr2=0x7f1fe00d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/3830945694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe015f3b0 0x7f1fe00d36f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009870 tx=0x7f1fd002a210 comp rx=0 tx=0).stop 2024-05-10T10:38:52.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 -- 172.21.15.53:0/3830945694 shutdown_connections 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/3830945694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe00d5d30 0x7f1fe00d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/3830945694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe015f3b0 0x7f1fe00d36f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 -- 172.21.15.53:0/3830945694 >> 172.21.15.53:0/3830945694 conn(0x7f1fe00d5130 msgr2=0x7f1fe00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 -- 172.21.15.53:0/3830945694 shutdown_connections 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.531+0000 7f1fe6d19640 1 -- 172.21.15.53:0/3830945694 wait complete. 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.532+0000 7f1fe6d19640 1 Processor -- start 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.532+0000 7f1fe6d19640 1 -- start start 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe6d19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe00d5d30 0x7f1fe015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe6d19640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe015f3b0 0x7f1fe014c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe6d19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe00ce950 0x7f1fe0161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe6d19640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00d0e60 con 0x7f1fe00ce950 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe6d19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00d0ce0 con 0x7f1fe00d5d30 2024-05-10T10:38:52.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe6d19640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe00d0fe0 con 0x7f1fe015f3b0 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe00ce950 0x7f1fe0161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe00ce950 0x7f1fe0161b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43936/0 (socket says 172.21.15.53:43936) 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 -- 172.21.15.53:0/1168552330 learned_addr learned my addr 172.21.15.53:0/1168552330 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe4ab5640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe00d5d30 0x7f1fe015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fdffff640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe015f3b0 0x7f1fe014c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 -- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe015f3b0 msgr2=0x7f1fe014c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe015f3b0 0x7f1fe014c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 -- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe00d5d30 msgr2=0x7f1fe015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe00d5d30 0x7f1fe015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.533+0000 7f1fe52b6640 1 -- 172.21.15.53:0/1168552330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe014ca80 con 0x7f1fe00ce950 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.534+0000 7f1fe52b6640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe00ce950 0x7f1fe0161b90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8007e70 tx=0x7f1fd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.534+0000 7f1fdd7fa640 1 -- 172.21.15.53:0/1168552330 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8014620 con 0x7f1fe00ce950 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.534+0000 7f1fdd7fa640 1 -- 172.21.15.53:0/1168552330 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd8019070 con 0x7f1fe00ce950 2024-05-10T10:38:52.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.534+0000 7f1fdd7fa640 1 -- 172.21.15.53:0/1168552330 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8014a20 con 0x7f1fe00ce950 2024-05-10T10:38:52.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.534+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe0159410 con 0x7f1fe00ce950 2024-05-10T10:38:52.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.534+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01599d0 con 0x7f1fe00ce950 2024-05-10T10:38:52.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.535+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe006a220 con 0x7f1fe00ce950 2024-05-10T10:38:52.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.536+0000 7f1fdd7fa640 1 -- 172.21.15.53:0/1168552330 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1fd8014bc0 con 0x7f1fe00ce950 2024-05-10T10:38:52.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.536+0000 7f1fdd7fa640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fcc0762d0 0x7f1fcc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.536+0000 7f1fdd7fa640 1 -- 172.21.15.53:0/1168552330 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f1fd8097d20 con 0x7f1fe00ce950 2024-05-10T10:38:52.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.537+0000 7f1fe4ab5640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fcc0762d0 0x7f1fcc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.537+0000 7f1fe4ab5640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fcc0762d0 0x7f1fcc078790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0034000 tx=0x7f1fd002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.540+0000 7f1fdd7fa640 1 -- 172.21.15.53:0/1168552330 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1fe014ca80 con 0x7f1fe00ce950 2024-05-10T10:38:52.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.771+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fe00d2540 con 0x7f1fe00ce950 2024-05-10T10:38:52.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.772+0000 7f1fdd7fa640 1 -- 172.21.15.53:0/1168552330 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3872 (secure 0 0 0) 0x7f1fd8061830 con 0x7f1fe00ce950 2024-05-10T10:38:52.774 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:52.774 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2959245807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2959245807},{"type":"v1","addr":"172.21.15.53:6835","nonce":2959245807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/191914787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":191914787},{"type":"v1","addr":"172.21.15.135:6836","nonce":191914787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1633639207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1633639207},{"type":"v1","addr":"172.21.15.135:6837","nonce":1633639207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1791560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1791560229},{"type":"v1","addr":"172.21.15.53:6834","nonce":1791560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[]} 2024-05-10T10:38:52.774 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 55 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fcc0762d0 msgr2=0x7f1fcc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fcc0762d0 0x7f1fcc078790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0034000 tx=0x7f1fd002e000 comp rx=0 tx=0).stop 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe00ce950 msgr2=0x7f1fe0161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe00ce950 0x7f1fe0161b90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8007e70 tx=0x7f1fd800e4b0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 shutdown_connections 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1fcc0762d0 0x7f1fcc078790 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fe00ce950 0x7f1fe0161b90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe015f3b0 0x7f1fe014c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 --2- 172.21.15.53:0/1168552330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe00d5d30 0x7f1fe015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 >> 172.21.15.53:0/1168552330 conn(0x7f1fe00d5130 msgr2=0x7f1fe00c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:52.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 shutdown_connections 2024-05-10T10:38:52.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.774+0000 7f1fe6d19640 1 -- 172.21.15.53:0/1168552330 wait complete. 2024-05-10T10:38:52.787 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:38:52.787 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T10:38:52.788 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:38:52.788 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:38:52.788 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:38:52.788 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T10:38:52.788 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:38:52.788 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:38:52.788 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:38:52.788 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T10:38:52.789 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:38:52.789 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:38:52.789 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:38:52.789 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T10:38:52.789 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:38:52.789 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:38:52.796 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:38:52.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:38:52.798 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:38:52.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:38:52.800 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:38:52.801 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:38:52.801 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:38:52.801 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:38:52.840 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:38:52.840 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:52.840 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:38:52.841 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:52.843 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:38:52.843 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:52.844 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:38:52.844 DEBUG:teuthology.parallel:result is None 2024-05-10T10:38:52.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T10:38:52.918 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:38:52.919 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:38:52.926 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:38:52.926 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:38:52.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.953+0000 7f1f26c6d640 1 Processor -- start 2024-05-10T10:38:52.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.953+0000 7f1f26c6d640 1 -- start start 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f26c6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f2015f790 0x7f1f200cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f26c6d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1f200d5d30 0x7f1f200d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f26c6d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f200d30b0 con 0x7f1f2015f3b0 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f26c6d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1f200d3390 con 0x7f1f200d5d30 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f26c6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1f200c8c80 con 0x7f1f2015f790 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f2015f790 0x7f1f200cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f2015f790 0x7f1f200cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:36988/0 (socket says 172.21.15.53:36988) 2024-05-10T10:38:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f1ffff640 1 -- 172.21.15.53:0/4199440104 learned_addr learned my addr 172.21.15.53:0/4199440104 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:52.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.954+0000 7f1f2520a640 1 --2- 172.21.15.53:0/4199440104 >> v2:172.21.15.135:3300/0 conn(0x7f1f200d5d30 0x7f1f200d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1ffff640 1 -- 172.21.15.53:0/4199440104 >> v2:172.21.15.135:3300/0 conn(0x7f1f200d5d30 msgr2=0x7f1f200d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:52.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1ffff640 1 --2- 172.21.15.53:0/4199440104 >> v2:172.21.15.135:3300/0 conn(0x7f1f200d5d30 0x7f1f200d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1ffff640 1 -- 172.21.15.53:0/4199440104 >> v1:172.21.15.135:6789/0 conn(0x7f1f2015f3b0 legacy=0x7f1f200d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1ffff640 1 -- 172.21.15.53:0/4199440104 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f200c8f60 con 0x7f1f2015f790 2024-05-10T10:38:52.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1ffff640 1 --2- 172.21.15.53:0/4199440104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f2015f790 0x7f1f200cfcd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b36540c6abf22a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1f7fe640 1 -- 172.21.15.53:0/4199440104 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c037070 con 0x7f1f2015f790 2024-05-10T10:38:52.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1f7fe640 1 -- 172.21.15.53:0/4199440104 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0c032440 con 0x7f1f2015f790 2024-05-10T10:38:52.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.955+0000 7f1f1f7fe640 1 -- 172.21.15.53:0/4199440104 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c032810 con 0x7f1f2015f790 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 -- 172.21.15.53:0/4199440104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f2015f790 msgr2=0x7f1f200cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/4199440104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f2015f790 0x7f1f200cfcd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c02a250 comp rx=0 tx=0).stop 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 -- 172.21.15.53:0/4199440104 shutdown_connections 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/4199440104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f2015f790 0x7f1f200cfcd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/4199440104 >> v2:172.21.15.135:3300/0 conn(0x7f1f200d5d30 0x7f1f200d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 -- 172.21.15.53:0/4199440104 >> 172.21.15.53:0/4199440104 conn(0x7f1f200d5130 msgr2=0x7f1f200d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 -- 172.21.15.53:0/4199440104 shutdown_connections 2024-05-10T10:38:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.956+0000 7f1f26c6d640 1 -- 172.21.15.53:0/4199440104 wait complete. 2024-05-10T10:38:52.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.958+0000 7f1f26c6d640 1 Processor -- start 2024-05-10T10:38:52.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.958+0000 7f1f26c6d640 1 -- start start 2024-05-10T10:38:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f26c6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f200d5d30 0x7f1f200ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f26c6d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f2015f3b0 0x7f1f201510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f26c6d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f20151610 0x7f1f2014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f26c6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200d3f80 con 0x7f1f200d5d30 2024-05-10T10:38:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f26c6d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200d3e00 con 0x7f1f2015f3b0 2024-05-10T10:38:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f26c6d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1f200ca770 con 0x7f1f20151610 2024-05-10T10:38:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f24a09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f200d5d30 0x7f1f200ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f2015f3b0 0x7f1f201510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f2015f3b0 0x7f1f201510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:45704/0 (socket says 172.21.15.53:45704) 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f24a09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f200d5d30 0x7f1f200ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35062/0 (socket says 172.21.15.53:35062) 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 -- 172.21.15.53:0/3719395918 learned_addr learned my addr 172.21.15.53:0/3719395918 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f2520a640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f20151610 0x7f1f2014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 -- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f20151610 msgr2=0x7f1f2014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f20151610 0x7f1f2014f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 -- 172.21.15.53:0/3719395918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f200d5d30 msgr2=0x7f1f200ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f200d5d30 0x7f1f200ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 -- 172.21.15.53:0/3719395918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f20160d70 con 0x7f1f2015f3b0 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f24a09640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f200d5d30 0x7f1f200ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.959+0000 7f1f1ffff640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f2015f3b0 0x7f1f201510d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c02a290 tx=0x7f1f0c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.960+0000 7f1f1d7fa640 1 -- 172.21.15.53:0/3719395918 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c037040 con 0x7f1f2015f3b0 2024-05-10T10:38:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.960+0000 7f1f1d7fa640 1 -- 172.21.15.53:0/3719395918 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0c042070 con 0x7f1f2015f3b0 2024-05-10T10:38:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.960+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f20161050 con 0x7f1f2015f3b0 2024-05-10T10:38:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.960+0000 7f1f1d7fa640 1 -- 172.21.15.53:0/3719395918 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c030880 con 0x7f1f2015f3b0 2024-05-10T10:38:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.960+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f200ceb80 con 0x7f1f2015f3b0 2024-05-10T10:38:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.961+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2006a220 con 0x7f1f2015f3b0 2024-05-10T10:38:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.961+0000 7f1f1d7fa640 1 -- 172.21.15.53:0/3719395918 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f0c030a20 con 0x7f1f2015f3b0 2024-05-10T10:38:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.962+0000 7f1f1d7fa640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f040762d0 0x7f1f04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.962+0000 7f1f1d7fa640 1 -- 172.21.15.53:0/3719395918 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f1f0c0b7bc0 con 0x7f1f2015f3b0 2024-05-10T10:38:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.962+0000 7f1f24a09640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f040762d0 0x7f1f04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.963+0000 7f1f24a09640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f040762d0 0x7f1f04078790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001b70 tx=0x7f1f14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:52.967+0000 7f1f1d7fa640 1 -- 172.21.15.53:0/3719395918 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1f0c080ff0 con 0x7f1f2015f3b0 2024-05-10T10:38:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:53.170+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1f200d4720 con 0x7f1f2015f3b0 2024-05-10T10:38:54.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.220+0000 7f1f1d7fa640 1 -- 172.21.15.53:0/3719395918 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) ==== 97+0+0 (secure 0 0 0) 0x7f1f0c0852f0 con 0x7f1f2015f3b0 2024-05-10T10:38:54.222 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2024-05-10T10:38:54.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f040762d0 msgr2=0x7f1f04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f040762d0 0x7f1f04078790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001b70 tx=0x7f1f14009040 comp rx=0 tx=0).stop 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f2015f3b0 msgr2=0x7f1f201510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f2015f3b0 0x7f1f201510d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c02a290 tx=0x7f1f0c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 shutdown_connections 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1f040762d0 0x7f1f04078790 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f20151610 0x7f1f2014f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f2015f3b0 0x7f1f201510d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 --2- 172.21.15.53:0/3719395918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f200d5d30 0x7f1f200ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 >> 172.21.15.53:0/3719395918 conn(0x7f1f200d5130 msgr2=0x7f1f200baa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 shutdown_connections 2024-05-10T10:38:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.223+0000 7f1f26c6d640 1 -- 172.21.15.53:0/3719395918 wait complete. 2024-05-10T10:38:54.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T10:38:54.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.347+0000 7fdfcba74640 1 Processor -- start 2024-05-10T10:38:54.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.347+0000 7fdfcba74640 1 -- start start 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.348+0000 7fdfcba74640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 0x7fdfc406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.348+0000 7fdfcba74640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdfc40d0230 0x7fdfc40cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.348+0000 7fdfcba74640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfc415fc90 con 0x7fdfc40cfbb0 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.348+0000 7fdfcba74640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc406ee50 con 0x7fdfc40d0230 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.348+0000 7fdfcba74640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc40b8c40 con 0x7fdfc40b8860 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.348+0000 7fdfca011640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdfc40cfbb0 0x7fdfc40d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60618/0 (socket says 172.21.15.53:60618) 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.348+0000 7fdfca011640 1 -- 172.21.15.53:0/1856822409 learned_addr learned my addr 172.21.15.53:0/1856822409 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc9810640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 0x7fdfc406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc900f640 1 --2- 172.21.15.53:0/1856822409 >> v2:172.21.15.135:3300/0 conn(0x7fdfc40d0230 0x7fdfc40cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16079945 0 0) 0x7fdfc415fc90 con 0x7fdfc40cfbb0 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfb0003520 con 0x7fdfc40cfbb0 2024-05-10T10:38:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2102553916 0 0) 0x7fdfb0003520 con 0x7fdfc40cfbb0 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 msgr2=0x7fdfc406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 0x7fdfc406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 >> v2:172.21.15.135:3300/0 conn(0x7fdfc40d0230 msgr2=0x7fdfc40cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 --2- 172.21.15.53:0/1856822409 >> v2:172.21.15.135:3300/0 conn(0x7fdfc40d0230 0x7fdfc40cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc40cd570 con 0x7fdfc40cfbb0 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fdfbc001670 con 0x7fdfc40cfbb0 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 >> v1:172.21.15.53:6789/0 conn(0x7fdfc40cfbb0 legacy=0x7fdfc40d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb00039b0 0x7fdfb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 0x7fdfb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb0004d40 0x7fdfb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc406ee50 con 0x7fdfb00039b0 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40b8c40 con 0x7fdfb0004d40 2024-05-10T10:38:54.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb000b9c0 con 0x7fdfc40b8860 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc9810640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb00039b0 0x7fdfb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfca011640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb0004d40 0x7fdfb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc900f640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 0x7fdfb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc9810640 1 -- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 msgr2=0x7fdfb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.349+0000 7fdfc9810640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc40b8860 0x7fdfb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfc9810640 1 -- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb0004d40 msgr2=0x7fdfb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfc9810640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb0004d40 0x7fdfb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfc9810640 1 -- 172.21.15.53:0/1856822409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb0003520 con 0x7fdfb00039b0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfca011640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb0004d40 0x7fdfb00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfc9810640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb00039b0 0x7fdfb0003dd0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009fc0 tx=0x7fdfb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291ef6c2bb398933 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0009c10 con 0x7fdfb00039b0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc0036070 con 0x7fdfb00039b0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.350+0000 7fdfc880e640 1 -- 172.21.15.53:0/1856822409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0031880 con 0x7fdfb00039b0 2024-05-10T10:38:54.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.351+0000 7fdfcba74640 1 -- 172.21.15.53:0/1856822409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb00039b0 msgr2=0x7fdfb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.351+0000 7fdfcba74640 1 --2- 172.21.15.53:0/1856822409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfb00039b0 0x7fdfb0003dd0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009fc0 tx=0x7fdfb0005830 comp rx=0 tx=0).stop 2024-05-10T10:38:54.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.351+0000 7fdfca011640 1 -- 172.21.15.53:0/1856822409 reap_dead start 2024-05-10T10:38:54.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.351+0000 7fdfcba74640 1 -- 172.21.15.53:0/1856822409 shutdown_connections 2024-05-10T10:38:54.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.351+0000 7fdfcba74640 1 -- 172.21.15.53:0/1856822409 >> 172.21.15.53:0/1856822409 conn(0x7fdfc40d2df0 msgr2=0x7fdfc40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:54.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.351+0000 7fdfcba74640 1 -- 172.21.15.53:0/1856822409 shutdown_connections 2024-05-10T10:38:54.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.351+0000 7fdfcba74640 1 -- 172.21.15.53:0/1856822409 wait complete. 2024-05-10T10:38:54.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.352+0000 7fdfcba74640 1 Processor -- start 2024-05-10T10:38:54.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.352+0000 7fdfcba74640 1 -- start start 2024-05-10T10:38:54.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfcba74640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfb00039b0 0x7fdfc4153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfcba74640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc40ca6b0 0x7fdfc40cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfcba74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfc40cc800 0x7fdfc40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfcba74640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40cde10 con 0x7fdfc40cc800 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfcba74640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40cdb30 con 0x7fdfc40ca6b0 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfcba74640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc40b5c40 con 0x7fdfb00039b0 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfc40cc800 0x7fdfc40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfc40cc800 0x7fdfc40ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35082/0 (socket says 172.21.15.53:35082) 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 -- 172.21.15.53:0/691845176 learned_addr learned my addr 172.21.15.53:0/691845176 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfc900f640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc40ca6b0 0x7fdfc40cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfc9810640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfb00039b0 0x7fdfc4153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 -- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfb00039b0 msgr2=0x7fdfc4153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfb00039b0 0x7fdfc4153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 -- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc40ca6b0 msgr2=0x7fdfc40cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc40ca6b0 0x7fdfc40cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfca011640 1 -- 172.21.15.53:0/691845176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc4153c70 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.353+0000 7fdfc900f640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc40ca6b0 0x7fdfc40cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.354+0000 7fdfca011640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfc40cc800 0x7fdfc40ccc80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000fb0 tx=0x7fdfbc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.354+0000 7fdfba7fc640 1 -- 172.21.15.53:0/691845176 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00c5b0 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.354+0000 7fdfba7fc640 1 -- 172.21.15.53:0/691845176 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc018070 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.354+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc4154160 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.354+0000 7fdfba7fc640 1 -- 172.21.15.53:0/691845176 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00c990 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.354+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc40c6d30 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.355+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc406a220 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.356+0000 7fdfba7fc640 1 -- 172.21.15.53:0/691845176 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdfbc00cb30 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.356+0000 7fdfba7fc640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdfa80762d0 0x7fdfa8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.356+0000 7fdfba7fc640 1 -- 172.21.15.53:0/691845176 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7fdfbc096f90 con 0x7fdfc40cc800 2024-05-10T10:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.356+0000 7fdfc9810640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdfa80762d0 0x7fdfa8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.357+0000 7fdfc9810640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdfa80762d0 0x7fdfa8078790 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0035000 tx=0x7fdfc002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.360+0000 7fdfba7fc640 1 -- 172.21.15.53:0/691845176 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdfbc060b40 con 0x7fdfc40cc800 2024-05-10T10:38:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.634+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdfc4163080 con 0x7fdfc40cc800 2024-05-10T10:38:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.634+0000 7fdfba7fc640 1 -- 172.21.15.53:0/691845176 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fdfbc060440 con 0x7fdfc40cc800 2024-05-10T10:38:54.637 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:54.637 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:38:54.637 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2024-05-10T10:38:54.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.637 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2024-05-10T10:38:54.637 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg==", 2024-05-10T10:38:54.637 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.638 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2024-05-10T10:38:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ==", 2024-05-10T10:38:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:38:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:38:54.639 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2024-05-10T10:38:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA==", 2024-05-10T10:38:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.640 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:38:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:38:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew==", 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:38:54.642 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA==", 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA==", 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw==", 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.644 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA==", 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.645 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g==", 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A==", 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA==", 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.647 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg==", 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg==", 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2024-05-10T10:38:54.649 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g==", 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw==", 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw==", 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.651 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1mKY2tIhAAzPSpOQLOFy1GEKdl5bPBEw==", 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:38:54.652 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw==", 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ==", 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:38:54.654 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:38:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdfa80762d0 msgr2=0x7fdfa8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdfa80762d0 0x7fdfa8078790 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0035000 tx=0x7fdfc002f000 comp rx=0 tx=0).stop 2024-05-10T10:38:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfc40cc800 msgr2=0x7fdfc40ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfc40cc800 0x7fdfc40ccc80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000fb0 tx=0x7fdfbc00eb90 comp rx=0 tx=0).stop 2024-05-10T10:38:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 shutdown_connections 2024-05-10T10:38:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdfa80762d0 0x7fdfa8078790 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfc40cc800 0x7fdfc40ccc80 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc40ca6b0 0x7fdfc40cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 --2- 172.21.15.53:0/691845176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfb00039b0 0x7fdfc4153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 >> 172.21.15.53:0/691845176 conn(0x7fdfc40d2df0 msgr2=0x7fdfc4071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 shutdown_connections 2024-05-10T10:38:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.637+0000 7fdfcba74640 1 -- 172.21.15.53:0/691845176 wait complete. 2024-05-10T10:38:54.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:38:54.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.759+0000 7fcfdffc1640 1 Processor -- start 2024-05-10T10:38:54.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.759+0000 7fcfdffc1640 1 -- start start 2024-05-10T10:38:54.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfdffc1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfd815f4e0 0x7fcfd806d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfdffc1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd806e620 0x7fcfd80d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfdffc1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd80b0a90 con 0x7fcfd815f8c0 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfdffc1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd80b23f0 con 0x7fcfd815f4e0 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfdffc1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd80b4150 con 0x7fcfd806e620 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfddd5d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfd815f4e0 0x7fcfd806d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfddd5d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfd815f4e0 0x7fcfd806d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35088/0 (socket says 172.21.15.53:35088) 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.760+0000 7fcfddd5d640 1 -- 172.21.15.53:0/1699126019 learned_addr learned my addr 172.21.15.53:0/1699126019 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.761+0000 7fcfde55e640 1 --2- 172.21.15.53:0/1699126019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd806e620 0x7fcfd80d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.761+0000 7fcfddd5d640 1 -- 172.21.15.53:0/1699126019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd806e620 msgr2=0x7fcfd80d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.761+0000 7fcfddd5d640 1 --2- 172.21.15.53:0/1699126019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd806e620 0x7fcfd80d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.761+0000 7fcfddd5d640 1 -- 172.21.15.53:0/1699126019 >> v1:172.21.15.135:6789/0 conn(0x7fcfd815f8c0 legacy=0x7fcfd806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.761+0000 7fcfddd5d640 1 -- 172.21.15.53:0/1699126019 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd80b3540 con 0x7fcfd815f4e0 2024-05-10T10:38:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.761+0000 7fcfddd5d640 1 --2- 172.21.15.53:0/1699126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f4e0 0x7fcfd806d9d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009870 tx=0x7fcfd002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682c113d2990e5bb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdcd5b640 1 -- 172.21.15.53:0/1699126019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0037070 con 0x7fcfd815f4e0 2024-05-10T10:38:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdcd5b640 1 -- 172.21.15.53:0/1699126019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd0032440 con 0x7fcfd815f4e0 2024-05-10T10:38:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdcd5b640 1 -- 172.21.15.53:0/1699126019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0032810 con 0x7fcfd815f4e0 2024-05-10T10:38:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 -- 172.21.15.53:0/1699126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f4e0 msgr2=0x7fcfd806d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/1699126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f4e0 0x7fcfd806d9d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009870 tx=0x7fcfd002a1c0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 -- 172.21.15.53:0/1699126019 shutdown_connections 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/1699126019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd806e620 0x7fcfd80d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/1699126019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f4e0 0x7fcfd806d9d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 -- 172.21.15.53:0/1699126019 >> 172.21.15.53:0/1699126019 conn(0x7fcfd80d2df0 msgr2=0x7fcfd80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 -- 172.21.15.53:0/1699126019 shutdown_connections 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.762+0000 7fcfdffc1640 1 -- 172.21.15.53:0/1699126019 wait complete. 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.763+0000 7fcfdffc1640 1 Processor -- start 2024-05-10T10:38:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.763+0000 7fcfdffc1640 1 -- start start 2024-05-10T10:38:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfdffc1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd806e620 0x7fcfd8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfdffc1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd815f4e0 0x7fcfd81606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfdffc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f8c0 0x7fcfd815bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfdffc1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd80b3de0 con 0x7fcfd815f8c0 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfdffc1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd80b3b00 con 0x7fcfd806e620 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfdffc1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd80b45e0 con 0x7fcfd815f4e0 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfde55e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f8c0 0x7fcfd815bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfde55e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f8c0 0x7fcfd815bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35098/0 (socket says 172.21.15.53:35098) 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.764+0000 7fcfde55e640 1 -- 172.21.15.53:0/3655831750 learned_addr learned my addr 172.21.15.53:0/3655831750 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfddd5d640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd806e620 0x7fcfd8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfdd55c640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd815f4e0 0x7fcfd81606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfde55e640 1 -- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd815f4e0 msgr2=0x7fcfd81606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfde55e640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd815f4e0 0x7fcfd81606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfde55e640 1 -- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd806e620 msgr2=0x7fcfd8151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfde55e640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd806e620 0x7fcfd8151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfde55e640 1 -- 172.21.15.53:0/3655831750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd81585b0 con 0x7fcfd815f8c0 2024-05-10T10:38:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfddd5d640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd806e620 0x7fcfd8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfde55e640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f8c0 0x7fcfd815bda0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8007e90 tx=0x7fcfc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfd67fc640 1 -- 172.21.15.53:0/3655831750 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8014420 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfd67fc640 1 -- 172.21.15.53:0/3655831750 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc8019070 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd8158850 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfd67fc640 1 -- 172.21.15.53:0/3655831750 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8014800 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.765+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd8158e10 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.766+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd806a220 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.767+0000 7fcfd67fc640 1 -- 172.21.15.53:0/3655831750 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcfc80149a0 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.767+0000 7fcfd67fc640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcfc40762d0 0x7fcfc4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.767+0000 7fcfd67fc640 1 -- 172.21.15.53:0/3655831750 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7fcfc8097c30 con 0x7fcfd815f8c0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.768+0000 7fcfddd5d640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcfc40762d0 0x7fcfc4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:54.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.768+0000 7fcfddd5d640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcfc40762d0 0x7fcfc4078790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0034000 tx=0x7fcfd002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:54.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.772+0000 7fcfd67fc640 1 -- 172.21.15.53:0/3655831750 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fcfc8061100 con 0x7fcfd815f8c0 2024-05-10T10:38:54.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.965+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcfd806e290 con 0x7fcfd815f8c0 2024-05-10T10:38:54.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.965+0000 7fcfd67fc640 1 -- 172.21.15.53:0/3655831750 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7fcfc8065400 con 0x7fcfd815f8c0 2024-05-10T10:38:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:54.967 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:38:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:38:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:38:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:38:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:38:54.968 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:38:54.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcfc40762d0 msgr2=0x7fcfc4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcfc40762d0 0x7fcfc4078790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0034000 tx=0x7fcfd002e000 comp rx=0 tx=0).stop 2024-05-10T10:38:54.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f8c0 msgr2=0x7fcfd815bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:54.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f8c0 0x7fcfd815bda0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8007e90 tx=0x7fcfc800e4b0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 shutdown_connections 2024-05-10T10:38:54.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcfc40762d0 0x7fcfc4078790 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd815f8c0 0x7fcfd815bda0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfd815f4e0 0x7fcfd81606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 --2- 172.21.15.53:0/3655831750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd806e620 0x7fcfd8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:54.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 >> 172.21.15.53:0/3655831750 conn(0x7fcfd80d2df0 msgr2=0x7fcfd806ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:54.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 shutdown_connections 2024-05-10T10:38:54.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:54.967+0000 7fcfdffc1640 1 -- 172.21.15.53:0/3655831750 wait complete. 2024-05-10T10:38:54.980 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:38:54.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:38:55.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.086+0000 7f17a00a7640 1 Processor -- start 2024-05-10T10:38:55.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.086+0000 7f17a00a7640 1 -- start start 2024-05-10T10:38:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f17a00a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 0x7f17980d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f17a00a7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17980d3f10 con 0x7f179815f6e0 2024-05-10T10:38:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f17a00a7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17980d5d80 con 0x7f17980d2df0 2024-05-10T10:38:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f17a00a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f17980d5b30 con 0x7f17980d35f0 2024-05-10T10:38:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f179d642640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f179815f6e0 0x7f17980cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60630/0 (socket says 172.21.15.53:60630) 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f179d642640 1 -- 172.21.15.53:0/2388268229 learned_addr learned my addr 172.21.15.53:0/2388268229 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f179de43640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 0x7f17980d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232535356 0 0) 0x7f17980d3f10 con 0x7f179815f6e0 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.087+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1784003520 con 0x7f179815f6e0 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3146300450 0 0) 0x7f1784003520 con 0x7f179815f6e0 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 msgr2=0x7f17980d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 0x7f17980d39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 >> v1:172.21.15.135:6789/0 conn(0x7f17980d2df0 legacy=0x7f17980d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17980b1eb0 con 0x7f179815f6e0 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f178c001670 con 0x7f179815f6e0 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 >> v1:172.21.15.53:6789/0 conn(0x7f179815f6e0 legacy=0x7f17980cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17840039e0 0x7f1784003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1784004800 0x7f1784004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 0x7f1784005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17980d5b30 con 0x7f1784004800 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f178400baf0 con 0x7f17840039e0 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f178400bcf0 con 0x7f17980d35f0 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179d642640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1784004800 0x7f1784004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179de43640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17840039e0 0x7f1784003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179e644640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 0x7f1784005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179d642640 1 -- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 msgr2=0x7f1784005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179d642640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17980d35f0 0x7f1784005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179d642640 1 -- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17840039e0 msgr2=0x7f1784003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179d642640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17840039e0 0x7f1784003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.088+0000 7f179d642640 1 -- 172.21.15.53:0/2388268229 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1784003520 con 0x7f1784004800 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.089+0000 7f179d642640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1784004800 0x7f1784004c80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c210 tx=0x7f1784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe92c70cb277db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.089+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c007ce0 con 0x7f1784004800 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.089+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c005af0 con 0x7f1784004800 2024-05-10T10:38:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.089+0000 7f179ce41640 1 -- 172.21.15.53:0/2388268229 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c007070 con 0x7f1784004800 2024-05-10T10:38:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.090+0000 7f17a00a7640 1 -- 172.21.15.53:0/2388268229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1784004800 msgr2=0x7f1784004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.090+0000 7f17a00a7640 1 --2- 172.21.15.53:0/2388268229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1784004800 0x7f1784004c80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c210 tx=0x7f1784005830 comp rx=0 tx=0).stop 2024-05-10T10:38:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.090+0000 7f179e644640 1 -- 172.21.15.53:0/2388268229 reap_dead start 2024-05-10T10:38:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.090+0000 7f17a00a7640 1 -- 172.21.15.53:0/2388268229 shutdown_connections 2024-05-10T10:38:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.090+0000 7f17a00a7640 1 -- 172.21.15.53:0/2388268229 >> 172.21.15.53:0/2388268229 conn(0x7f17980d5010 msgr2=0x7f17980d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.090+0000 7f17a00a7640 1 -- 172.21.15.53:0/2388268229 shutdown_connections 2024-05-10T10:38:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.090+0000 7f17a00a7640 1 -- 172.21.15.53:0/2388268229 wait complete. 2024-05-10T10:38:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.091+0000 7f17a00a7640 1 Processor -- start 2024-05-10T10:38:55.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.091+0000 7f17a00a7640 1 -- start start 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f17a00a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1784004800 0x7f179814e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f17a00a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17980b5d10 0x7f17981550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f17a00a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1798161ca0 0x7f1798162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f17a00a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17980d2200 con 0x7f17980b5d10 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f17a00a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17980b1c60 con 0x7f1784004800 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f17a00a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f17980d2380 con 0x7f1798161ca0 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17980b5d10 0x7f17981550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17980b5d10 0x7f17981550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35118/0 (socket says 172.21.15.53:35118) 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 -- 172.21.15.53:0/1228406279 learned_addr learned my addr 172.21.15.53:0/1228406279 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179de43640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1784004800 0x7f179814e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179e644640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1798161ca0 0x7f1798162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 -- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1798161ca0 msgr2=0x7f1798162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1798161ca0 0x7f1798162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 -- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1784004800 msgr2=0x7f179814e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1784004800 0x7f179814e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 -- 172.21.15.53:0/1228406279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17980b6210 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f179d642640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17980b5d10 0x7f17981550a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c330 tx=0x7f17980d5f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.092+0000 7f178a7fc640 1 -- 172.21.15.53:0/1228406279 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c041020 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.093+0000 7f178a7fc640 1 -- 172.21.15.53:0/1228406279 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c0013b0 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.093+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17981578b0 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.093+0000 7f178a7fc640 1 -- 172.21.15.53:0/1228406279 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c007070 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.093+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1798158390 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.094+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179806a220 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.094+0000 7f178a7fc640 1 -- 172.21.15.53:0/1228406279 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f178c044020 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.095+0000 7f178a7fc640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f177c076300 0x7f177c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.095+0000 7f178a7fc640 1 -- 172.21.15.53:0/1228406279 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7f178c0b7600 con 0x7f17980b5d10 2024-05-10T10:38:55.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.095+0000 7f179de43640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f177c076300 0x7f177c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:55.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.096+0000 7f179de43640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f177c076300 0x7f177c0787c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1794009a80 tx=0x7f1794008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:55.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.099+0000 7f178a7fc640 1 -- 172.21.15.53:0/1228406279 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f178c0811b0 con 0x7f17980b5d10 2024-05-10T10:38:55.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:55.290+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f17980d5420 con 0x7f17980b5d10 2024-05-10T10:38:56.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.220+0000 7f178a7fc640 1 -- 172.21.15.53:0/1228406279 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f178c080ab0 con 0x7f17980b5d10 2024-05-10T10:38:56.223 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:38:56.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f177c076300 msgr2=0x7f177c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f177c076300 0x7f177c0787c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1794009a80 tx=0x7f1794008040 comp rx=0 tx=0).stop 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17980b5d10 msgr2=0x7f17981550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17980b5d10 0x7f17981550a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c330 tx=0x7f17980d5f50 comp rx=0 tx=0).stop 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 shutdown_connections 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f177c076300 0x7f177c0787c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1798161ca0 0x7f1798162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17980b5d10 0x7f17981550a0 secure :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c330 tx=0x7f17980d5f50 comp rx=0 tx=0).stop 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 --2- 172.21.15.53:0/1228406279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1784004800 0x7f179814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.223+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 >> 172.21.15.53:0/1228406279 conn(0x7f17980d5010 msgr2=0x7f17980b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.224+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 shutdown_connections 2024-05-10T10:38:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.225+0000 7f17a00a7640 1 -- 172.21.15.53:0/1228406279 wait complete. 2024-05-10T10:38:56.242 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:38:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.357+0000 7fc77d87e640 1 Processor -- start 2024-05-10T10:38:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.357+0000 7fc77d87e640 1 -- start start 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc77d87e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc778162ce0 0x7fc7780b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc77d87e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7780d4190 con 0x7fc7780b4e20 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc77d87e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7780cfc40 con 0x7fc778162900 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc77d87e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc7780d3eb0 con 0x7fc778162ce0 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc7767fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc778162ce0 0x7fc7780b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc7777fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc7780b4e20 0x7fc7780d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60640/0 (socket says 172.21.15.53:60640) 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc7777fe640 1 -- 172.21.15.53:0/363700780 learned_addr learned my addr 172.21.15.53:0/363700780 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc7767fc640 1 -- 172.21.15.53:0/363700780 >> v1:172.21.15.135:6789/0 conn(0x7fc778162900 legacy=0x7fc7780d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc7767fc640 1 -- 172.21.15.53:0/363700780 >> v1:172.21.15.53:6789/0 conn(0x7fc7780b4e20 legacy=0x7fc7780d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.358+0000 7fc7767fc640 1 -- 172.21.15.53:0/363700780 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7780cfee0 con 0x7fc778162ce0 2024-05-10T10:38:56.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.359+0000 7fc7767fc640 1 --2- 172.21.15.53:0/363700780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780b48e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097a0 tx=0x7fc76002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a81940ff83a600f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:56.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.359+0000 7fc775ffb640 1 -- 172.21.15.53:0/363700780 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760037070 con 0x7fc778162ce0 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.359+0000 7fc775ffb640 1 -- 172.21.15.53:0/363700780 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76002ae50 con 0x7fc778162ce0 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.359+0000 7fc775ffb640 1 -- 172.21.15.53:0/363700780 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760032660 con 0x7fc778162ce0 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.360+0000 7fc77d87e640 1 -- 172.21.15.53:0/363700780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 msgr2=0x7fc7780b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.360+0000 7fc77d87e640 1 --2- 172.21.15.53:0/363700780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780b48e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097a0 tx=0x7fc76002a140 comp rx=0 tx=0).stop 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.360+0000 7fc77d87e640 1 -- 172.21.15.53:0/363700780 shutdown_connections 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.360+0000 7fc77d87e640 1 --2- 172.21.15.53:0/363700780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780b48e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.360+0000 7fc77d87e640 1 -- 172.21.15.53:0/363700780 >> 172.21.15.53:0/363700780 conn(0x7fc77809e430 msgr2=0x7fc7780d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.360+0000 7fc77d87e640 1 -- 172.21.15.53:0/363700780 shutdown_connections 2024-05-10T10:38:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.360+0000 7fc77d87e640 1 -- 172.21.15.53:0/363700780 wait complete. 2024-05-10T10:38:56.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.361+0000 7fc77d87e640 1 Processor -- start 2024-05-10T10:38:56.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.361+0000 7fc77d87e640 1 -- start start 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc77d87e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7780b4e20 0x7fc7780baa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc77d87e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc778162900 0x7fc77815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc77d87e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc77d87e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc7780b4180 con 0x7fc778162ce0 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc77d87e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7780d3b20 con 0x7fc778162900 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc77d87e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc7780b4300 con 0x7fc7780b4e20 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35132/0 (socket says 172.21.15.53:35132) 2024-05-10T10:38:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc7777fe640 1 -- 172.21.15.53:0/1643356811 learned_addr learned my addr 172.21.15.53:0/1643356811 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc7767fc640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc778162900 0x7fc77815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.362+0000 7fc776ffd640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7780b4e20 0x7fc7780baa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc7777fe640 1 -- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7780b4e20 msgr2=0x7fc7780baa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc7777fe640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7780b4e20 0x7fc7780baa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc7777fe640 1 -- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc778162900 msgr2=0x7fc77815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc7777fe640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc778162900 0x7fc77815ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc7777fe640 1 -- 172.21.15.53:0/1643356811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7780b8880 con 0x7fc778162ce0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc7767fc640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc778162900 0x7fc77815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc7777fe640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780cc1b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c450 tx=0x7fc77000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc76ffff640 1 -- 172.21.15.53:0/1643356811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770017070 con 0x7fc778162ce0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc76ffff640 1 -- 172.21.15.53:0/1643356811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770005ce0 con 0x7fc778162ce0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7780b98e0 con 0x7fc778162ce0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc76ffff640 1 -- 172.21.15.53:0/1643356811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770007270 con 0x7fc778162ce0 2024-05-10T10:38:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.363+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7780b8b20 con 0x7fc778162ce0 2024-05-10T10:38:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.364+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77806a220 con 0x7fc778162ce0 2024-05-10T10:38:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.365+0000 7fc76ffff640 1 -- 172.21.15.53:0/1643356811 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc77001a070 con 0x7fc778162ce0 2024-05-10T10:38:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.365+0000 7fc76ffff640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7580762d0 0x7fc758078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.365+0000 7fc76ffff640 1 -- 172.21.15.53:0/1643356811 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5619+0+0 (secure 0 0 0) 0x7fc770096740 con 0x7fc778162ce0 2024-05-10T10:38:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.366+0000 7fc776ffd640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7580762d0 0x7fc758078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.366+0000 7fc776ffd640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7580762d0 0x7fc758078790 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc7780ba9c0 tx=0x7fc768009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.370+0000 7fc76ffff640 1 -- 172.21.15.53:0/1643356811 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc7780d4190 con 0x7fc778162ce0 2024-05-10T10:38:56.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:56.559+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc77809e840 con 0x7fc778162ce0 2024-05-10T10:38:57.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.222+0000 7fc76ffff640 1 -- 172.21.15.53:0/1643356811 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7fc77005fb30 con 0x7fc778162ce0 2024-05-10T10:38:57.224 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:38:57.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.225+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7580762d0 msgr2=0x7fc758078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:57.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.225+0000 7fc77d87e640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7580762d0 0x7fc758078790 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc7780ba9c0 tx=0x7fc768009040 comp rx=0 tx=0).stop 2024-05-10T10:38:57.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.225+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 msgr2=0x7fc7780cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:57.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.225+0000 7fc77d87e640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780cc1b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c450 tx=0x7fc77000c910 comp rx=0 tx=0).stop 2024-05-10T10:38:57.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.226+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 shutdown_connections 2024-05-10T10:38:57.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.226+0000 7fc77d87e640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc7580762d0 0x7fc758078790 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.226+0000 7fc77d87e640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc778162ce0 0x7fc7780cc1b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.226+0000 7fc77d87e640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc778162900 0x7fc77815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.226+0000 7fc77d87e640 1 --2- 172.21.15.53:0/1643356811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7780b4e20 0x7fc7780baa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.226+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 >> 172.21.15.53:0/1643356811 conn(0x7fc77809e430 msgr2=0x7fc77806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:57.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.227+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 shutdown_connections 2024-05-10T10:38:57.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.228+0000 7fc77d87e640 1 -- 172.21.15.53:0/1643356811 wait complete. 2024-05-10T10:38:57.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:38:57.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.363+0000 7faa208f8640 1 Processor -- start 2024-05-10T10:38:57.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.363+0000 7faa208f8640 1 -- start start 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa208f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d2d10 0x7faa1c0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa208f8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa1c0d3170 con 0x7faa1c0d1f50 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa208f8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa1c0d41d0 con 0x7faa1c0d3df0 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa208f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7faa1c0cfd50 con 0x7faa1c0d2d10 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa1ad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faa1c0d1f50 0x7faa1c0d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60644/0 (socket says 172.21.15.53:60644) 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa1ad9d640 1 -- 172.21.15.53:0/3463258508 learned_addr learned my addr 172.21.15.53:0/3463258508 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa19d9b640 1 --2- 172.21.15.53:0/3463258508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d2d10 0x7faa1c0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637670572 0 0) 0x7faa1c0d3170 con 0x7faa1c0d1f50 2024-05-10T10:38:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa04003520 con 0x7faa1c0d1f50 2024-05-10T10:38:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 370730350 0 0) 0x7faa1c0d41d0 con 0x7faa1c0d3df0 2024-05-10T10:38:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa1c0d3170 con 0x7faa1c0d3df0 2024-05-10T10:38:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa19d9b640 1 -- 172.21.15.53:0/3463258508 >> v1:172.21.15.135:6789/0 conn(0x7faa1c0d3df0 legacy=0x7faa1c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1974520661 0 0) 0x7faa04003520 con 0x7faa1c0d1f50 2024-05-10T10:38:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa19d9b640 1 -- 172.21.15.53:0/3463258508 >> v1:172.21.15.53:6789/0 conn(0x7faa1c0d1f50 legacy=0x7faa1c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.364+0000 7faa19d9b640 1 -- 172.21.15.53:0/3463258508 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c0b54f0 con 0x7faa1c0d2d10 2024-05-10T10:38:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.365+0000 7faa19d9b640 1 --2- 172.21.15.53:0/3463258508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d2d10 0x7faa1c0d1a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa100097d0 tx=0x7faa1002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38244d1ece2d48f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.365+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10037070 con 0x7faa1c0d2d10 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.365+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1002ac30 con 0x7faa1c0d2d10 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.365+0000 7faa1959a640 1 -- 172.21.15.53:0/3463258508 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140019f0 con 0x7faa1c0d2d10 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.366+0000 7faa208f8640 1 -- 172.21.15.53:0/3463258508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d2d10 msgr2=0x7faa1c0d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.366+0000 7faa208f8640 1 --2- 172.21.15.53:0/3463258508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d2d10 0x7faa1c0d1a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa100097d0 tx=0x7faa1002a1a0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.366+0000 7faa208f8640 1 -- 172.21.15.53:0/3463258508 shutdown_connections 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.366+0000 7faa208f8640 1 --2- 172.21.15.53:0/3463258508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d2d10 0x7faa1c0d1a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.366+0000 7faa208f8640 1 -- 172.21.15.53:0/3463258508 >> 172.21.15.53:0/3463258508 conn(0x7faa1c09e450 msgr2=0x7faa1c0d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.366+0000 7faa208f8640 1 -- 172.21.15.53:0/3463258508 shutdown_connections 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.366+0000 7faa208f8640 1 -- 172.21.15.53:0/3463258508 wait complete. 2024-05-10T10:38:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.367+0000 7faa208f8640 1 Processor -- start 2024-05-10T10:38:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.367+0000 7faa208f8640 1 -- start start 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa208f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa1c0d1f50 0x7faa1c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa208f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa1c0d2d10 0x7faa1c153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa208f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d3df0 0x7faa1c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa208f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c0b5ab0 con 0x7faa1c0d2d10 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa208f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c0b5d90 con 0x7faa1c0d1f50 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa208f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7faa1c0cff80 con 0x7faa1c0d3df0 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa19d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa1c0d2d10 0x7faa1c153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa19d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa1c0d2d10 0x7faa1c153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35146/0 (socket says 172.21.15.53:35146) 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa19d9b640 1 -- 172.21.15.53:0/3205748147 learned_addr learned my addr 172.21.15.53:0/3205748147 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa1a59c640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa1c0d1f50 0x7faa1c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa1ad9d640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d3df0 0x7faa1c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa19d9b640 1 -- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d3df0 msgr2=0x7faa1c0b8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa19d9b640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d3df0 0x7faa1c0b8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.368+0000 7faa19d9b640 1 -- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa1c0d1f50 msgr2=0x7faa1c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa19d9b640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa1c0d1f50 0x7faa1c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa19d9b640 1 -- 172.21.15.53:0/3205748147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c0ca8a0 con 0x7faa1c0d2d10 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa1ad9d640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d3df0 0x7faa1c0b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa19d9b640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa1c0d2d10 0x7faa1c153550 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faa10035040 tx=0x7faa100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa0affd640 1 -- 172.21.15.53:0/3205748147 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10041020 con 0x7faa1c0d2d10 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa0affd640 1 -- 172.21.15.53:0/3205748147 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1003d640 con 0x7faa1c0d2d10 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c0cab80 con 0x7faa1c0d2d10 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa0affd640 1 -- 172.21.15.53:0/3205748147 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10037040 con 0x7faa1c0d2d10 2024-05-10T10:38:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.369+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c1e8ec0 con 0x7faa1c0d2d10 2024-05-10T10:38:57.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.370+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1c06a220 con 0x7faa1c0d2d10 2024-05-10T10:38:57.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.371+0000 7faa0affd640 1 -- 172.21.15.53:0/3205748147 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faa10044020 con 0x7faa1c0d2d10 2024-05-10T10:38:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.371+0000 7faa0affd640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9fc0762d0 0x7fa9fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.371+0000 7faa0affd640 1 -- 172.21.15.53:0/3205748147 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6001+0+0 (secure 0 0 0) 0x7faa100b7fa0 con 0x7faa1c0d2d10 2024-05-10T10:38:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.372+0000 7faa1a59c640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9fc0762d0 0x7fa9fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.372+0000 7faa1a59c640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9fc0762d0 0x7fa9fc078790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0011e0 tx=0x7faa0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.375+0000 7faa0affd640 1 -- 172.21.15.53:0/3205748147 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7faa1004b030 con 0x7faa1c0d2d10 2024-05-10T10:38:57.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:57.606+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faa1c08f200 con 0x7faa1c0d2d10 2024-05-10T10:38:58.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.232+0000 7faa0affd640 1 -- 172.21.15.53:0/3205748147 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v57) ==== 174+0+0 (secure 0 0 0) 0x7faa100818b0 con 0x7faa1c0d2d10 2024-05-10T10:38:58.234 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 13 and data pool 14 2024-05-10T10:38:58.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.237+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9fc0762d0 msgr2=0x7fa9fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.237+0000 7faa208f8640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9fc0762d0 0x7fa9fc078790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0011e0 tx=0x7faa0c009040 comp rx=0 tx=0).stop 2024-05-10T10:38:58.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.237+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa1c0d2d10 msgr2=0x7faa1c153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.237+0000 7faa208f8640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa1c0d2d10 0x7faa1c153550 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faa10035040 tx=0x7faa100046c0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.238+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 shutdown_connections 2024-05-10T10:38:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.238+0000 7faa208f8640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa9fc0762d0 0x7fa9fc078790 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.238+0000 7faa208f8640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faa1c0d3df0 0x7faa1c0b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.238+0000 7faa208f8640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa1c0d2d10 0x7faa1c153550 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.238+0000 7faa208f8640 1 --2- 172.21.15.53:0/3205748147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa1c0d1f50 0x7faa1c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.238+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 >> 172.21.15.53:0/3205748147 conn(0x7faa1c09e450 msgr2=0x7faa1c0d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:58.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.241+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 shutdown_connections 2024-05-10T10:38:58.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.242+0000 7faa208f8640 1 -- 172.21.15.53:0/3205748147 wait complete. 2024-05-10T10:38:58.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:38:58.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.376+0000 7fd077122640 1 Processor -- start 2024-05-10T10:38:58.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.376+0000 7fd077122640 1 -- start start 2024-05-10T10:38:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd077122640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd0700d5cb0 0x7fd0700d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd077122640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0700b45f0 con 0x7fd070162750 2024-05-10T10:38:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd077122640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0700b48d0 con 0x7fd070162b30 2024-05-10T10:38:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd077122640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd0700b4f20 con 0x7fd0700d5cb0 2024-05-10T10:38:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd074ebe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd070162750 0x7fd0700d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60648/0 (socket says 172.21.15.53:60648) 2024-05-10T10:38:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd074ebe640 1 -- 172.21.15.53:0/1560746617 learned_addr learned my addr 172.21.15.53:0/1560746617 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd067fff640 1 --2- 172.21.15.53:0/1560746617 >> v2:172.21.15.135:3300/0 conn(0x7fd0700d5cb0 0x7fd0700d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058995248 0 0) 0x7fd0700b45f0 con 0x7fd070162750 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd058003520 con 0x7fd070162750 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1659894384 0 0) 0x7fd058003520 con 0x7fd070162750 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 >> v2:172.21.15.135:3300/0 conn(0x7fd0700d5cb0 msgr2=0x7fd0700d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd0677fe640 1 --2- 172.21.15.53:0/1560746617 >> v2:172.21.15.135:3300/0 conn(0x7fd0700d5cb0 0x7fd0700d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 >> v1:172.21.15.135:6789/0 conn(0x7fd070162b30 legacy=0x7fd0700b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.377+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0700b4bb0 con 0x7fd070162750 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fd06c001670 con 0x7fd070162750 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 >> v1:172.21.15.53:6789/0 conn(0x7fd070162750 legacy=0x7fd0700d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0580039b0 0x7fd058003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd058004800 0x7fd058004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0580051c0 0x7fd05800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700b4f20 con 0x7fd058004800 2024-05-10T10:38:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd05800c160 con 0x7fd0580051c0 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd05800c360 con 0x7fd0580039b0 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd067fff640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd058004800 0x7fd058004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd074ebe640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0580039b0 0x7fd058003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0756bf640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0580051c0 0x7fd05800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd067fff640 1 -- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0580039b0 msgr2=0x7fd058003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd067fff640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0580039b0 0x7fd058003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd067fff640 1 -- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0580051c0 msgr2=0x7fd05800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd067fff640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0580051c0 0x7fd05800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd067fff640 1 -- 172.21.15.53:0/1560746617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058003520 con 0x7fd058004800 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0756bf640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0580051c0 0x7fd05800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.378+0000 7fd0756bf640 1 -- 172.21.15.53:0/1560746617 reap_dead start 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd067fff640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd058004800 0x7fd058004c80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd07006cd40 tx=0x7fd058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9375bdaa60197596 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060036070 con 0x7fd058004800 2024-05-10T10:38:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd060031400 con 0x7fd058004800 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd0677fe640 1 -- 172.21.15.53:0/1560746617 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060009690 con 0x7fd058004800 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd077122640 1 -- 172.21.15.53:0/1560746617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd058004800 msgr2=0x7fd058004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd077122640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd058004800 0x7fd058004c80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd07006cd40 tx=0x7fd058005830 comp rx=0 tx=0).stop 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd077122640 1 -- 172.21.15.53:0/1560746617 shutdown_connections 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.379+0000 7fd077122640 1 --2- 172.21.15.53:0/1560746617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd058004800 0x7fd058004c80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.380+0000 7fd077122640 1 -- 172.21.15.53:0/1560746617 >> 172.21.15.53:0/1560746617 conn(0x7fd07009e3b0 msgr2=0x7fd0700d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.380+0000 7fd077122640 1 -- 172.21.15.53:0/1560746617 shutdown_connections 2024-05-10T10:38:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.380+0000 7fd077122640 1 -- 172.21.15.53:0/1560746617 wait complete. 2024-05-10T10:38:58.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.381+0000 7fd077122640 1 Processor -- start 2024-05-10T10:38:58.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.381+0000 7fd077122640 1 -- start start 2024-05-10T10:38:58.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd058004800 0x7fd0701f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0701582a0 0x7fd070158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0700cdd90 0x7fd0700ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700d0a00 con 0x7fd0701582a0 2024-05-10T10:38:58.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700d0880 con 0x7fd058004800 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd0700d0b80 con 0x7fd0700cdd90 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0701582a0 0x7fd070158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0701582a0 0x7fd070158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35172/0 (socket says 172.21.15.53:35172) 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 -- 172.21.15.53:0/2440136619 learned_addr learned my addr 172.21.15.53:0/2440136619 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd074ebe640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd058004800 0x7fd0701f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd0756bf640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0700cdd90 0x7fd0700ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 -- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0700cdd90 msgr2=0x7fd0700ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0700cdd90 0x7fd0700ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 -- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd058004800 msgr2=0x7fd0701f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd058004800 0x7fd0701f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 -- 172.21.15.53:0/2440136619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0700ccd30 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd0756bf640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0700cdd90 0x7fd0700ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd067fff640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0701582a0 0x7fd070158700 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd060034040 tx=0x7fd060004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd0657fa640 1 -- 172.21.15.53:0/2440136619 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06003f030 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd0657fa640 1 -- 172.21.15.53:0/2440136619 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd060037040 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0700ccfd0 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd0657fa640 1 -- 172.21.15.53:0/2440136619 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060036070 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.382+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0700cd590 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.384+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07006a220 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.384+0000 7fd0657fa640 1 -- 172.21.15.53:0/2440136619 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd060042070 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.385+0000 7fd0657fa640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.385+0000 7fd0657fa640 1 -- 172.21.15.53:0/2440136619 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fd0600b6cb0 con 0x7fd0701582a0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.385+0000 7fd074ebe640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.385+0000 7fd074ebe640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd05c0762d0 0x7fd05c078790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0017e0 tx=0x7fd06c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.389+0000 7fd0657fa640 1 -- 172.21.15.53:0/2440136619 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd06007fec0 con 0x7fd0701582a0 2024-05-10T10:38:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.583+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0700b3fb0 con 0x7fd0701582a0 2024-05-10T10:38:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.584+0000 7fd0657fa640 1 -- 172.21.15.53:0/2440136619 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16898 (secure 0 0 0) 0x7fd0600841c0 con 0x7fd0701582a0 2024-05-10T10:38:58.586 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:58.586 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:38:57.607895+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:38:55.295910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T10:38:56.564675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd05c0762d0 msgr2=0x7fd05c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd05c0762d0 0x7fd05c078790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0017e0 tx=0x7fd06c007040 comp rx=0 tx=0).stop 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0701582a0 msgr2=0x7fd070158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0701582a0 0x7fd070158700 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd060034040 tx=0x7fd060004730 comp rx=0 tx=0).stop 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 shutdown_connections 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0700cdd90 0x7fd0700ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0701582a0 0x7fd070158700 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 --2- 172.21.15.53:0/2440136619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd058004800 0x7fd0701f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 >> 172.21.15.53:0/2440136619 conn(0x7fd07009e3b0 msgr2=0x7fd0700b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 shutdown_connections 2024-05-10T10:38:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.586+0000 7fd077122640 1 -- 172.21.15.53:0/2440136619 wait complete. 2024-05-10T10:38:58.599 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:38:58.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.701+0000 7f993e29b640 1 Processor -- start 2024-05-10T10:38:58.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.701+0000 7f993e29b640 1 -- start start 2024-05-10T10:38:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f993e29b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99380d5cb0 0x7f99380d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f993e29b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99380b45f0 con 0x7f9938162b30 2024-05-10T10:38:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f993e29b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99380b48d0 con 0x7f9938162750 2024-05-10T10:38:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f993e29b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f99380b4f20 con 0x7f99380d5cb0 2024-05-10T10:38:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f993c838640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9938162b30 0x7f99380b53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60660/0 (socket says 172.21.15.53:60660) 2024-05-10T10:38:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f993c838640 1 -- 172.21.15.53:0/3232135024 learned_addr learned my addr 172.21.15.53:0/3232135024 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f99377fe640 1 --2- 172.21.15.53:0/3232135024 >> v2:172.21.15.135:3300/0 conn(0x7f99380d5cb0 0x7f99380d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f9936ffd640 1 -- 172.21.15.53:0/3232135024 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2452178029 0 0) 0x7f99380b45f0 con 0x7f9938162b30 2024-05-10T10:38:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.702+0000 7f9936ffd640 1 -- 172.21.15.53:0/3232135024 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9920003520 con 0x7f9938162b30 2024-05-10T10:38:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.703+0000 7f9936ffd640 1 -- 172.21.15.53:0/3232135024 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1920545782 0 0) 0x7f9920003520 con 0x7f9938162b30 2024-05-10T10:38:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.703+0000 7f99377fe640 1 -- 172.21.15.53:0/3232135024 >> v1:172.21.15.135:6789/0 conn(0x7f9938162750 legacy=0x7f99380d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.703+0000 7f99377fe640 1 -- 172.21.15.53:0/3232135024 >> v1:172.21.15.53:6789/0 conn(0x7f9938162b30 legacy=0x7f99380b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.703+0000 7f99377fe640 1 -- 172.21.15.53:0/3232135024 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99380b4bb0 con 0x7f99380d5cb0 2024-05-10T10:38:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f99377fe640 1 --2- 172.21.15.53:0/3232135024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99380d5cb0 0x7f99380d3630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f992c0097a0 tx=0x7f992c02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=771a5c2a6aca2633 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f9936ffd640 1 -- 172.21.15.53:0/3232135024 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c037070 con 0x7f99380d5cb0 2024-05-10T10:38:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f9936ffd640 1 -- 172.21.15.53:0/3232135024 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c02aad0 con 0x7f99380d5cb0 2024-05-10T10:38:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f9936ffd640 1 -- 172.21.15.53:0/3232135024 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99300019f0 con 0x7f99380d5cb0 2024-05-10T10:38:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f993e29b640 1 -- 172.21.15.53:0/3232135024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99380d5cb0 msgr2=0x7f99380d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f993e29b640 1 --2- 172.21.15.53:0/3232135024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99380d5cb0 0x7f99380d3630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f992c0097a0 tx=0x7f992c02a040 comp rx=0 tx=0).stop 2024-05-10T10:38:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f993e29b640 1 -- 172.21.15.53:0/3232135024 shutdown_connections 2024-05-10T10:38:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f993e29b640 1 --2- 172.21.15.53:0/3232135024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99380d5cb0 0x7f99380d3630 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f993e29b640 1 -- 172.21.15.53:0/3232135024 >> 172.21.15.53:0/3232135024 conn(0x7f993809e3b0 msgr2=0x7f99380d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f993e29b640 1 -- 172.21.15.53:0/3232135024 shutdown_connections 2024-05-10T10:38:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.704+0000 7f993e29b640 1 -- 172.21.15.53:0/3232135024 wait complete. 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.705+0000 7f993e29b640 1 Processor -- start 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.706+0000 7f993e29b640 1 -- start start 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.706+0000 7f993e29b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9938162750 0x7f9938157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.706+0000 7f993e29b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9938162b30 0x7f99381e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.706+0000 7f993e29b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99381e9300 0x7f99380cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.706+0000 7f993e29b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99380d3f00 con 0x7f99381e9300 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.706+0000 7f993e29b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99380d3d80 con 0x7f9938162b30 2024-05-10T10:38:58.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.706+0000 7f993e29b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f99380d0fe0 con 0x7f9938162750 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f99377fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9938162b30 0x7f99381e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f9937fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9938162750 0x7f9938157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f99377fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9938162b30 0x7f99381e8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:45854/0 (socket says 172.21.15.53:45854) 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f99377fe640 1 -- 172.21.15.53:0/209779729 learned_addr learned my addr 172.21.15.53:0/209779729 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993c838640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99381e9300 0x7f99380cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993c838640 1 -- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9938162750 msgr2=0x7f9938157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993c838640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9938162750 0x7f9938157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993c838640 1 -- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9938162b30 msgr2=0x7f99381e8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993c838640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9938162b30 0x7f99381e8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993c838640 1 -- 172.21.15.53:0/209779729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99381484e0 con 0x7f99381e9300 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993c838640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99381e9300 0x7f99380cc300 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f993000c7a0 tx=0x7f993000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f9934ff9640 1 -- 172.21.15.53:0/209779729 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930018070 con 0x7f99381e9300 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f9934ff9640 1 -- 172.21.15.53:0/209779729 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993000a5e0 con 0x7f99381e9300 2024-05-10T10:38:58.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99381487c0 con 0x7f99381e9300 2024-05-10T10:38:58.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f9934ff9640 1 -- 172.21.15.53:0/209779729 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000aa10 con 0x7f99381e9300 2024-05-10T10:38:58.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.707+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9938148d80 con 0x7f99381e9300 2024-05-10T10:38:58.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.709+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993806a220 con 0x7f99381e9300 2024-05-10T10:38:58.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.709+0000 7f9934ff9640 1 -- 172.21.15.53:0/209779729 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f993001b070 con 0x7f99381e9300 2024-05-10T10:38:58.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.709+0000 7f9934ff9640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f99240762d0 0x7f9924078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:58.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.710+0000 7f9934ff9640 1 -- 172.21.15.53:0/209779729 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f9930096da0 con 0x7f99381e9300 2024-05-10T10:38:58.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.710+0000 7f9937fff640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f99240762d0 0x7f9924078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:58.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.710+0000 7f9937fff640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f99240762d0 0x7f9924078790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9928001900 tx=0x7f9928009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.714+0000 7f9934ff9640 1 -- 172.21.15.53:0/209779729 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f993005fe90 con 0x7f99381e9300 2024-05-10T10:38:58.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.906+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f993808d0f0 con 0x7f99381e9300 2024-05-10T10:38:58.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.907+0000 7f9934ff9640 1 -- 172.21.15.53:0/209779729 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16898 (secure 0 0 0) 0x7f9930064190 con 0x7f99381e9300 2024-05-10T10:38:58.909 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:38:58.909 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:38:57.607895+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:38:55.295910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T10:38:56.564675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:38:58.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f99240762d0 msgr2=0x7f9924078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f99240762d0 0x7f9924078790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9928001900 tx=0x7f9928009040 comp rx=0 tx=0).stop 2024-05-10T10:38:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99381e9300 msgr2=0x7f99380cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99381e9300 0x7f99380cc300 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f993000c7a0 tx=0x7f993000cc60 comp rx=0 tx=0).stop 2024-05-10T10:38:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 shutdown_connections 2024-05-10T10:38:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f99240762d0 0x7f9924078790 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99381e9300 0x7f99380cc300 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9938162b30 0x7f99381e8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 --2- 172.21.15.53:0/209779729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9938162750 0x7f9938157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:58.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 >> 172.21.15.53:0/209779729 conn(0x7f993809e3b0 msgr2=0x7f99380d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:58.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 shutdown_connections 2024-05-10T10:38:58.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:58.909+0000 7f993e29b640 1 -- 172.21.15.53:0/209779729 wait complete. 2024-05-10T10:38:58.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:38:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.028+0000 7fb4a6911640 1 Processor -- start 2024-05-10T10:38:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.028+0000 7fb4a6911640 1 -- start start 2024-05-10T10:38:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb4a6911640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb4a00d3df0 0x7fb4a00d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb4a6911640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4a00b4aa0 0x7fb4a00d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb4a6911640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a00d41d0 con 0x7fb4a00b4f30 2024-05-10T10:38:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb4a6911640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a00cfd50 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb4a6911640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a00b5d90 con 0x7fb4a00d3df0 2024-05-10T10:38:59.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb497fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4a00b4aa0 0x7fb4a00d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb4a4eae640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb4a00b4f30 0x7fb4a00d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60662/0 (socket says 172.21.15.53:60662) 2024-05-10T10:38:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb4a4eae640 1 -- 172.21.15.53:0/73924995 learned_addr learned my addr 172.21.15.53:0/73924995 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:38:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.029+0000 7fb49ffff640 1 --2- 172.21.15.53:0/73924995 >> v2:172.21.15.135:3300/0 conn(0x7fb4a00d3df0 0x7fb4a00d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb497fff640 1 -- 172.21.15.53:0/73924995 >> v2:172.21.15.135:3300/0 conn(0x7fb4a00d3df0 msgr2=0x7fb4a00d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:38:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb497fff640 1 --2- 172.21.15.53:0/73924995 >> v2:172.21.15.135:3300/0 conn(0x7fb4a00d3df0 0x7fb4a00d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb497fff640 1 -- 172.21.15.53:0/73924995 >> v1:172.21.15.53:6789/0 conn(0x7fb4a00b4f30 legacy=0x7fb4a00d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb497fff640 1 -- 172.21.15.53:0/73924995 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a00b54f0 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb49f7fe640 1 -- 172.21.15.53:0/73924995 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140157077 0 0) 0x7fb4a00d41d0 con 0x7fb4a00b4f30 2024-05-10T10:38:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb497fff640 1 --2- 172.21.15.53:0/73924995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a00d1a10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb4a006cbe0 tx=0x7fb48802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31650393f8087abf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb49f7fe640 1 -- 172.21.15.53:0/73924995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488037070 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb49f7fe640 1 -- 172.21.15.53:0/73924995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb488032440 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.030+0000 7fb49f7fe640 1 -- 172.21.15.53:0/73924995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488032810 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 -- 172.21.15.53:0/73924995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 msgr2=0x7fb4a00d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 --2- 172.21.15.53:0/73924995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a00d1a10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb4a006cbe0 tx=0x7fb48802a1a0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 -- 172.21.15.53:0/73924995 shutdown_connections 2024-05-10T10:38:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 --2- 172.21.15.53:0/73924995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a00d1a10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 --2- 172.21.15.53:0/73924995 >> v2:172.21.15.135:3300/0 conn(0x7fb4a00d3df0 0x7fb4a00d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 -- 172.21.15.53:0/73924995 >> 172.21.15.53:0/73924995 conn(0x7fb4a009e450 msgr2=0x7fb4a00d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:38:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 -- 172.21.15.53:0/73924995 shutdown_connections 2024-05-10T10:38:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.031+0000 7fb4a6911640 1 -- 172.21.15.53:0/73924995 wait complete. 2024-05-10T10:38:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.032+0000 7fb4a6911640 1 Processor -- start 2024-05-10T10:38:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.032+0000 7fb4a6911640 1 -- start start 2024-05-10T10:38:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb4a6911640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a01d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb4a6911640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a00b4f30 0x7fb4a00cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb4a6911640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a00d3df0 0x7fb4a01e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb4a6911640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a00cff80 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb4a6911640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a00b5ab0 con 0x7fb4a00d3df0 2024-05-10T10:38:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb4a6911640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a00b6120 con 0x7fb4a00b4f30 2024-05-10T10:38:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a01d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a01d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35204/0 (socket says 172.21.15.53:35204) 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb49ffff640 1 -- 172.21.15.53:0/2333813747 learned_addr learned my addr 172.21.15.53:0/2333813747 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.033+0000 7fb4a4eae640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a00d3df0 0x7fb4a01e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb497fff640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a00b4f30 0x7fb4a00cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49ffff640 1 -- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a00b4f30 msgr2=0x7fb4a00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49ffff640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a00b4f30 0x7fb4a00cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49ffff640 1 -- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a00d3df0 msgr2=0x7fb4a01e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49ffff640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a00d3df0 0x7fb4a01e2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49ffff640 1 -- 172.21.15.53:0/2333813747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a00c9c90 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb4a4eae640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a00d3df0 0x7fb4a01e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb497fff640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a00b4f30 0x7fb4a00cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49ffff640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a01d0970 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb49000e8a0 tx=0x7fb49000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49d7fa640 1 -- 172.21.15.53:0/2333813747 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490017070 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49d7fa640 1 -- 172.21.15.53:0/2333813747 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49000c940 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a00c9f70 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb49d7fa640 1 -- 172.21.15.53:0/2333813747 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49000cd30 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.034+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a00ca530 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.035+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a006a220 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.036+0000 7fb49d7fa640 1 -- 172.21.15.53:0/2333813747 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb49001a070 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.036+0000 7fb49d7fa640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb4800762d0 0x7fb480078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:38:59.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.036+0000 7fb49d7fa640 1 -- 172.21.15.53:0/2333813747 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fb4900979e0 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.036+0000 7fb497fff640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb4800762d0 0x7fb480078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:38:59.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.037+0000 7fb497fff640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb4800762d0 0x7fb480078790 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb488034000 tx=0x7fb48802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:38:59.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.041+0000 7fb49d7fa640 1 -- 172.21.15.53:0/2333813747 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb4900611b0 con 0x7fb4a00b4aa0 2024-05-10T10:38:59.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:38:59.270+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb4a008f200 con 0x7fb4a00b4aa0 2024-05-10T10:39:00.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.253+0000 7fb49d7fa640 1 -- 172.21.15.53:0/2333813747 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) ==== 118+0+0 (secure 0 0 0) 0x7fb490060ab0 con 0x7fb4a00b4aa0 2024-05-10T10:39:00.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb4800762d0 msgr2=0x7fb480078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:00.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb4800762d0 0x7fb480078790 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb488034000 tx=0x7fb48802e000 comp rx=0 tx=0).stop 2024-05-10T10:39:00.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 msgr2=0x7fb4a01d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:00.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a01d0970 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb49000e8a0 tx=0x7fb49000ed60 comp rx=0 tx=0).stop 2024-05-10T10:39:00.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 shutdown_connections 2024-05-10T10:39:00.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb4800762d0 0x7fb480078790 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a00d3df0 0x7fb4a01e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a00b4f30 0x7fb4a00cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 --2- 172.21.15.53:0/2333813747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4a00b4aa0 0x7fb4a01d0970 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.256+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 >> 172.21.15.53:0/2333813747 conn(0x7fb4a009e450 msgr2=0x7fb4a00b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:00.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.257+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 shutdown_connections 2024-05-10T10:39:00.259 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.257+0000 7fb4a6911640 1 -- 172.21.15.53:0/2333813747 wait complete. 2024-05-10T10:39:00.272 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T10:39:00.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:39:00.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.383+0000 7f01c702c640 1 Processor -- start 2024-05-10T10:39:00.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.383+0000 7f01c702c640 1 -- start start 2024-05-10T10:39:00.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c702c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f01c015f190 0x7f01c00cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:00.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c702c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01c006d3e0 0x7f01c00cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:00.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c702c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c006d920 con 0x7f01c006e920 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c702c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f01c006dc00 con 0x7f01c006d3e0 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c702c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f01c00c6b80 con 0x7f01c015f190 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c55c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01c006d3e0 0x7f01c00cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c55c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01c006d3e0 0x7f01c00cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35214/0 (socket says 172.21.15.53:35214) 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c55c9640 1 -- 172.21.15.53:0/3408974482 learned_addr learned my addr 172.21.15.53:0/3408974482 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c4dc8640 1 --2- 172.21.15.53:0/3408974482 >> v2:172.21.15.135:3300/0 conn(0x7f01c015f190 0x7f01c00cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c55c9640 1 -- 172.21.15.53:0/3408974482 >> v2:172.21.15.135:3300/0 conn(0x7f01c015f190 msgr2=0x7f01c00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c55c9640 1 --2- 172.21.15.53:0/3408974482 >> v2:172.21.15.135:3300/0 conn(0x7f01c015f190 0x7f01c00cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c55c9640 1 -- 172.21.15.53:0/3408974482 >> v1:172.21.15.135:6789/0 conn(0x7f01c006e920 legacy=0x7f01c006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c55c9640 1 -- 172.21.15.53:0/3408974482 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c00c6e60 con 0x7f01c006d3e0 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.384+0000 7f01c4dc8640 1 --2- 172.21.15.53:0/3408974482 >> v2:172.21.15.135:3300/0 conn(0x7f01c015f190 0x7f01c00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.385+0000 7f01c55c9640 1 --2- 172.21.15.53:0/3408974482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c006d3e0 0x7f01c00cfeb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f01c0073bc0 tx=0x7f01b802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db2fb36ac4c3034 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.385+0000 7f01bf7fe640 1 -- 172.21.15.53:0/3408974482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8036070 con 0x7f01c006d3e0 2024-05-10T10:39:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.385+0000 7f01bf7fe640 1 -- 172.21.15.53:0/3408974482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8031600 con 0x7f01c006d3e0 2024-05-10T10:39:00.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.385+0000 7f01bf7fe640 1 -- 172.21.15.53:0/3408974482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80319d0 con 0x7f01c006d3e0 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 -- 172.21.15.53:0/3408974482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c006d3e0 msgr2=0x7f01c00cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 --2- 172.21.15.53:0/3408974482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c006d3e0 0x7f01c00cfeb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f01c0073bc0 tx=0x7f01b802a730 comp rx=0 tx=0).stop 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 -- 172.21.15.53:0/3408974482 shutdown_connections 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 --2- 172.21.15.53:0/3408974482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c006d3e0 0x7f01c00cfeb0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 --2- 172.21.15.53:0/3408974482 >> v2:172.21.15.135:3300/0 conn(0x7f01c015f190 0x7f01c00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 -- 172.21.15.53:0/3408974482 >> 172.21.15.53:0/3408974482 conn(0x7f01c00ce0a0 msgr2=0x7f01c00ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 -- 172.21.15.53:0/3408974482 shutdown_connections 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.386+0000 7f01c702c640 1 -- 172.21.15.53:0/3408974482 wait complete. 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.387+0000 7f01c702c640 1 Processor -- start 2024-05-10T10:39:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.387+0000 7f01c702c640 1 -- start start 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c702c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01c006d3e0 0x7f01c015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c702c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01c006e920 0x7f01c0158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c702c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c015f190 0x7f01c0156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c702c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0168df0 con 0x7f01c015f190 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c702c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01c015f570 con 0x7f01c006d3e0 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c702c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f01c0168f70 con 0x7f01c006e920 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c015f190 0x7f01c0156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c015f190 0x7f01c0156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35230/0 (socket says 172.21.15.53:35230) 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 -- 172.21.15.53:0/1678073619 learned_addr learned my addr 172.21.15.53:0/1678073619 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c4dc8640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01c006d3e0 0x7f01c015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01bffff640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01c006e920 0x7f01c0158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 -- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01c006e920 msgr2=0x7f01c0158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01c006e920 0x7f01c0158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 -- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01c006d3e0 msgr2=0x7f01c015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01c006d3e0 0x7f01c015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 -- 172.21.15.53:0/1678073619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c01555d0 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.388+0000 7f01c55c9640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c015f190 0x7f01c0156070 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f01b802ad20 tx=0x7f01b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.389+0000 7f01bd7fa640 1 -- 172.21.15.53:0/1678073619 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b803f020 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.389+0000 7f01bd7fa640 1 -- 172.21.15.53:0/1678073619 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8037040 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.389+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c015df60 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.389+0000 7f01bd7fa640 1 -- 172.21.15.53:0/1678073619 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8036070 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.389+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c0155b50 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.390+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c0068480 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.390+0000 7f01bd7fa640 1 -- 172.21.15.53:0/1678073619 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f01b8042020 con 0x7f01c015f190 2024-05-10T10:39:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.391+0000 7f01bd7fa640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f01a40762d0 0x7f01a4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:00.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.391+0000 7f01bd7fa640 1 -- 172.21.15.53:0/1678073619 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f01b80b7a50 con 0x7f01c015f190 2024-05-10T10:39:00.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.391+0000 7f01c4dc8640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f01a40762d0 0x7f01a4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:00.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.391+0000 7f01c4dc8640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f01a40762d0 0x7f01a4078790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f01c0158980 tx=0x7f01b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.395+0000 7f01bd7fa640 1 -- 172.21.15.53:0/1678073619 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f01c0068480 con 0x7f01c015f190 2024-05-10T10:39:00.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:00.625+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f01c006a220 con 0x7f01c015f190 2024-05-10T10:39:01.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.254+0000 7f01bd7fa640 1 -- 172.21.15.53:0/1678073619 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) ==== 122+0+0 (secure 0 0 0) 0x7f01b8049030 con 0x7f01c015f190 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f01a40762d0 msgr2=0x7f01a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f01a40762d0 0x7f01a4078790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f01c0158980 tx=0x7f01b4008040 comp rx=0 tx=0).stop 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c015f190 msgr2=0x7f01c0156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c015f190 0x7f01c0156070 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f01b802ad20 tx=0x7f01b8004a40 comp rx=0 tx=0).stop 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 shutdown_connections 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f01a40762d0 0x7f01a4078790 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01c015f190 0x7f01c0156070 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01c006e920 0x7f01c0158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 --2- 172.21.15.53:0/1678073619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01c006d3e0 0x7f01c015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 >> 172.21.15.53:0/1678073619 conn(0x7f01c00ce0a0 msgr2=0x7f01c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 shutdown_connections 2024-05-10T10:39:01.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.258+0000 7f01c702c640 1 -- 172.21.15.53:0/1678073619 wait complete. 2024-05-10T10:39:01.273 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:39:01.273 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:39:01.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.381+0000 7fbace2cf640 1 Processor -- start 2024-05-10T10:39:01.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.381+0000 7fbace2cf640 1 -- start start 2024-05-10T10:39:01.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbace2cf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbac8162750 0x7fbac80d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbace2cf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbac80d5cb0 0x7fbac80d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbace2cf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162b30 0x7fbac80d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbace2cf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbac80b4f20 con 0x7fbac80d5cb0 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbace2cf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbac80d0fe0 con 0x7fbac8162750 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbace2cf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbac80b45f0 con 0x7fbac8162b30 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbac80d5cb0 0x7fbac80d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbac80d5cb0 0x7fbac80d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35236/0 (socket says 172.21.15.53:35236) 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 -- 172.21.15.53:0/3416790917 learned_addr learned my addr 172.21.15.53:0/3416790917 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac7fff640 1 --2- 172.21.15.53:0/3416790917 >> v2:172.21.15.135:3300/0 conn(0x7fbac8162750 0x7fbac80d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbacc86c640 1 --2- 172.21.15.53:0/3416790917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162b30 0x7fbac80d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 -- 172.21.15.53:0/3416790917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162b30 msgr2=0x7fbac80d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 --2- 172.21.15.53:0/3416790917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162b30 0x7fbac80d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 -- 172.21.15.53:0/3416790917 >> v2:172.21.15.135:3300/0 conn(0x7fbac8162750 msgr2=0x7fbac80d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 --2- 172.21.15.53:0/3416790917 >> v2:172.21.15.135:3300/0 conn(0x7fbac8162750 0x7fbac80d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.382+0000 7fbac77fe640 1 -- 172.21.15.53:0/3416790917 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac80b48d0 con 0x7fbac80d5cb0 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.383+0000 7fbac7fff640 1 --2- 172.21.15.53:0/3416790917 >> v2:172.21.15.135:3300/0 conn(0x7fbac8162750 0x7fbac80d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:01.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.383+0000 7fbacc86c640 1 --2- 172.21.15.53:0/3416790917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162b30 0x7fbac80d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.383+0000 7fbac77fe640 1 --2- 172.21.15.53:0/3416790917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac80d5cb0 0x7fbac80d3630 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbac80b7c40 tx=0x7fbabc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5f7204f4f34a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.383+0000 7fbac6ffd640 1 -- 172.21.15.53:0/3416790917 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc037070 con 0x7fbac80d5cb0 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbac6ffd640 1 -- 172.21.15.53:0/3416790917 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc032440 con 0x7fbac80d5cb0 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbac6ffd640 1 -- 172.21.15.53:0/3416790917 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc032810 con 0x7fbac80d5cb0 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 -- 172.21.15.53:0/3416790917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac80d5cb0 msgr2=0x7fbac80d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 --2- 172.21.15.53:0/3416790917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac80d5cb0 0x7fbac80d3630 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbac80b7c40 tx=0x7fbabc02a140 comp rx=0 tx=0).stop 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 -- 172.21.15.53:0/3416790917 shutdown_connections 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 --2- 172.21.15.53:0/3416790917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162b30 0x7fbac80d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 --2- 172.21.15.53:0/3416790917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac80d5cb0 0x7fbac80d3630 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 --2- 172.21.15.53:0/3416790917 >> v2:172.21.15.135:3300/0 conn(0x7fbac8162750 0x7fbac80d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 -- 172.21.15.53:0/3416790917 >> 172.21.15.53:0/3416790917 conn(0x7fbac809e3b0 msgr2=0x7fbac80d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:01.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 -- 172.21.15.53:0/3416790917 shutdown_connections 2024-05-10T10:39:01.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.384+0000 7fbace2cf640 1 -- 172.21.15.53:0/3416790917 wait complete. 2024-05-10T10:39:01.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.385+0000 7fbace2cf640 1 Processor -- start 2024-05-10T10:39:01.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbace2cf640 1 -- start start 2024-05-10T10:39:01.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbace2cf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162750 0x7fbac815d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbace2cf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbac8162b30 0x7fbac81e8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbace2cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac81e8540 0x7fbac80b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbace2cf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbac80d0d30 con 0x7fbac81e8540 2024-05-10T10:39:01.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbace2cf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbac80d0bb0 con 0x7fbac8162b30 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbace2cf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbac80d1660 con 0x7fbac8162750 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbacc86c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac81e8540 0x7fbac80b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbacc86c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac81e8540 0x7fbac80b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35246/0 (socket says 172.21.15.53:35246) 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.386+0000 7fbacc86c640 1 -- 172.21.15.53:0/2960882165 learned_addr learned my addr 172.21.15.53:0/2960882165 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbac7fff640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162750 0x7fbac815d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbac77fe640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbac8162b30 0x7fbac81e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbacc86c640 1 -- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162750 msgr2=0x7fbac815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbacc86c640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162750 0x7fbac815d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbacc86c640 1 -- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbac8162b30 msgr2=0x7fbac81e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbacc86c640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbac8162b30 0x7fbac81e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbacc86c640 1 -- 172.21.15.53:0/2960882165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac80cd7d0 con 0x7fbac81e8540 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbac7fff640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162750 0x7fbac815d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbac77fe640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbac8162b30 0x7fbac81e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:01.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbacc86c640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac81e8540 0x7fbac80b7c40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbac000cd50 tx=0x7fbac000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbac4ff9640 1 -- 172.21.15.53:0/2960882165 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac000f680 con 0x7fbac81e8540 2024-05-10T10:39:01.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbac4ff9640 1 -- 172.21.15.53:0/2960882165 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac000f840 con 0x7fbac81e8540 2024-05-10T10:39:01.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac80cdab0 con 0x7fbac81e8540 2024-05-10T10:39:01.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbac4ff9640 1 -- 172.21.15.53:0/2960882165 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac000fc30 con 0x7fbac81e8540 2024-05-10T10:39:01.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.387+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac80ce070 con 0x7fbac81e8540 2024-05-10T10:39:01.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.388+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac806a220 con 0x7fbac81e8540 2024-05-10T10:39:01.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.389+0000 7fbac4ff9640 1 -- 172.21.15.53:0/2960882165 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbac00213f0 con 0x7fbac81e8540 2024-05-10T10:39:01.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.389+0000 7fbac4ff9640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab40762d0 0x7fbab4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.389+0000 7fbac4ff9640 1 -- 172.21.15.53:0/2960882165 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fbac00984e0 con 0x7fbac81e8540 2024-05-10T10:39:01.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.389+0000 7fbac7fff640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab40762d0 0x7fbab4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.390+0000 7fbac7fff640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab40762d0 0x7fbab4078790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbac80d2180 tx=0x7fbab8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.394+0000 7fbac4ff9640 1 -- 172.21.15.53:0/2960882165 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbac0014030 con 0x7fbac81e8540 2024-05-10T10:39:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.624+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbac80cb750 con 0x7fbac81e8540 2024-05-10T10:39:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.624+0000 7fbac4ff9640 1 -- 172.21.15.53:0/2960882165 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5186 (secure 0 0 0) 0x7fbac0061d40 con 0x7fbac81e8540 2024-05-10T10:39:01.626 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:01.626 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:00.626907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6833/2934725760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2934725760},{"type":"v1","addr":"172.21.15.53:6833","nonce":2934725760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6693,"qdb_cluster":[6693]},"id":6}]} 2024-05-10T10:39:01.626 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab40762d0 msgr2=0x7fbab4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab40762d0 0x7fbab4078790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbac80d2180 tx=0x7fbab8008040 comp rx=0 tx=0).stop 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac81e8540 msgr2=0x7fbac80b7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac81e8540 0x7fbac80b7c40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbac000cd50 tx=0x7fbac000a650 comp rx=0 tx=0).stop 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 shutdown_connections 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbab40762d0 0x7fbab4078790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbac81e8540 0x7fbac80b7c40 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbac8162b30 0x7fbac81e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 --2- 172.21.15.53:0/2960882165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbac8162750 0x7fbac815d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 >> 172.21.15.53:0/2960882165 conn(0x7fbac809e3b0 msgr2=0x7fbac806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:01.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 shutdown_connections 2024-05-10T10:39:01.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.626+0000 7fbace2cf640 1 -- 172.21.15.53:0/2960882165 wait complete. 2024-05-10T10:39:01.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:39:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff49aba4640 1 Processor -- start 2024-05-10T10:39:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff49aba4640 1 -- start start 2024-05-10T10:39:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff49aba4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff4940d1aa0 0x7ff4940d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff49aba4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff4940d3340 0x7ff4940ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff49aba4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff494072330 con 0x7ff49415d290 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff49aba4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff4940d37d0 con 0x7ff4940d1aa0 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff49aba4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff494071d70 con 0x7ff4940d3340 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.741+0000 7ff498940640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff49415d290 0x7ff494071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60670/0 (socket says 172.21.15.53:60670) 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff498940640 1 -- 172.21.15.53:0/3810166145 learned_addr learned my addr 172.21.15.53:0/3810166145 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff493fff640 1 --2- 172.21.15.53:0/3810166145 >> v2:172.21.15.53:3300/0 conn(0x7ff4940d1aa0 0x7ff4940d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff499141640 1 --2- 172.21.15.53:0/3810166145 >> v2:172.21.15.135:3300/0 conn(0x7ff4940d3340 0x7ff4940ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff4937fe640 1 -- 172.21.15.53:0/3810166145 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908756575 0 0) 0x7ff494072330 con 0x7ff49415d290 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff493fff640 1 -- 172.21.15.53:0/3810166145 >> v2:172.21.15.135:3300/0 conn(0x7ff4940d3340 msgr2=0x7ff4940ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff493fff640 1 --2- 172.21.15.53:0/3810166145 >> v2:172.21.15.135:3300/0 conn(0x7ff4940d3340 0x7ff4940ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff493fff640 1 -- 172.21.15.53:0/3810166145 >> v1:172.21.15.53:6789/0 conn(0x7ff49415d290 legacy=0x7ff494071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff493fff640 1 -- 172.21.15.53:0/3810166145 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff494072050 con 0x7ff4940d1aa0 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.742+0000 7ff499141640 1 --2- 172.21.15.53:0/3810166145 >> v2:172.21.15.135:3300/0 conn(0x7ff4940d3340 0x7ff4940ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:01.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff493fff640 1 --2- 172.21.15.53:0/3810166145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d1aa0 0x7ff4940d2e00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff4940b2900 tx=0x7ff47c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d314ad3a9a87dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff4937fe640 1 -- 172.21.15.53:0/3810166145 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c037070 con 0x7ff4940d1aa0 2024-05-10T10:39:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff4937fe640 1 -- 172.21.15.53:0/3810166145 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c032440 con 0x7ff4940d1aa0 2024-05-10T10:39:01.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff4937fe640 1 -- 172.21.15.53:0/3810166145 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c032810 con 0x7ff4940d1aa0 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 -- 172.21.15.53:0/3810166145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d1aa0 msgr2=0x7ff4940d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 --2- 172.21.15.53:0/3810166145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d1aa0 0x7ff4940d2e00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff4940b2900 tx=0x7ff47c02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 -- 172.21.15.53:0/3810166145 shutdown_connections 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 --2- 172.21.15.53:0/3810166145 >> v2:172.21.15.135:3300/0 conn(0x7ff4940d3340 0x7ff4940ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 --2- 172.21.15.53:0/3810166145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d1aa0 0x7ff4940d2e00 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 -- 172.21.15.53:0/3810166145 >> 172.21.15.53:0/3810166145 conn(0x7ff49409ba20 msgr2=0x7ff49409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 -- 172.21.15.53:0/3810166145 shutdown_connections 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.743+0000 7ff49aba4640 1 -- 172.21.15.53:0/3810166145 wait complete. 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.745+0000 7ff49aba4640 1 Processor -- start 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.745+0000 7ff49aba4640 1 -- start start 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.745+0000 7ff49aba4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4940d1aa0 0x7ff49415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.745+0000 7ff49aba4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d3340 0x7ff4940c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff49aba4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49415d290 0x7ff4940cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff49aba4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff494071a50 con 0x7ff4940d3340 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff49aba4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4940cdd50 con 0x7ff49415d290 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff49aba4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff4940ce7b0 con 0x7ff4940d1aa0 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d3340 0x7ff4940c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d3340 0x7ff4940c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35272/0 (socket says 172.21.15.53:35272) 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 -- 172.21.15.53:0/1191258287 learned_addr learned my addr 172.21.15.53:0/1191258287 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff499141640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49415d290 0x7ff4940cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff498940640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4940d1aa0 0x7ff49415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 -- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4940d1aa0 msgr2=0x7ff49415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4940d1aa0 0x7ff49415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 -- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49415d290 msgr2=0x7ff4940cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49415d290 0x7ff4940cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 -- 172.21.15.53:0/1191258287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4941f2bb0 con 0x7ff4940d3340 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff499141640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49415d290 0x7ff4940cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff493fff640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d3340 0x7ff4940c93a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff47c000c00 tx=0x7ff47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff4917fa640 1 -- 172.21.15.53:0/1191258287 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c037040 con 0x7ff4940d3340 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff4917fa640 1 -- 172.21.15.53:0/1191258287 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c042070 con 0x7ff4940d3340 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4941f2e90 con 0x7ff4940d3340 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff4917fa640 1 -- 172.21.15.53:0/1191258287 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c030bd0 con 0x7ff4940d3340 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.746+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4941f3450 con 0x7ff4940d3340 2024-05-10T10:39:01.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.747+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4940693f0 con 0x7ff4940d3340 2024-05-10T10:39:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.748+0000 7ff4917fa640 1 -- 172.21.15.53:0/1191258287 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff47c047460 con 0x7ff4940d3340 2024-05-10T10:39:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.748+0000 7ff4917fa640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.748+0000 7ff4917fa640 1 -- 172.21.15.53:0/1191258287 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7ff47c0b8330 con 0x7ff4940d3340 2024-05-10T10:39:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.749+0000 7ff498940640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:01.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.749+0000 7ff498940640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000f60 tx=0x7ff48c007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:01.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.753+0000 7ff4917fa640 1 -- 172.21.15.53:0/1191258287 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff47c032e20 con 0x7ff4940d3340 2024-05-10T10:39:01.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.951+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4940b48a0 con 0x7ff4940d3340 2024-05-10T10:39:01.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.952+0000 7ff4917fa640 1 -- 172.21.15.53:0/1191258287 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16885 (secure 0 0 0) 0x7ff47c081b00 con 0x7ff4940d3340 2024-05-10T10:39:01.954 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:01.954 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":87,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:38:59.244126+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:38:55.295910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T10:38:56.564675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 msgr2=0x7ff474078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000f60 tx=0x7ff48c007c00 comp rx=0 tx=0).stop 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d3340 msgr2=0x7ff4940c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d3340 0x7ff4940c93a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff47c000c00 tx=0x7ff47c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 shutdown_connections 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff4740762d0 0x7ff474078790 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49415d290 0x7ff4940cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4940d3340 0x7ff4940c93a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 --2- 172.21.15.53:0/1191258287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4940d1aa0 0x7ff49415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:01.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 >> 172.21.15.53:0/1191258287 conn(0x7ff49409ba20 msgr2=0x7ff4940b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:01.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 shutdown_connections 2024-05-10T10:39:01.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:01.954+0000 7ff49aba4640 1 -- 172.21.15.53:0/1191258287 wait complete. 2024-05-10T10:39:01.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:39:01.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:39:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.071+0000 7efc2a893640 1 Processor -- start 2024-05-10T10:39:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.071+0000 7efc2a893640 1 -- start start 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc2a893640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240d5cb0 0x7efc240d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc2a893640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efc24162b30 0x7efc240d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc2a893640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc240b45f0 con 0x7efc24162750 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc2a893640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efc240b4f20 con 0x7efc24162b30 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc2a893640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efc240d0fe0 con 0x7efc240d5cb0 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc23fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efc24162750 0x7efc240d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60674/0 (socket says 172.21.15.53:60674) 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc23fff640 1 -- 172.21.15.53:0/829924421 learned_addr learned my addr 172.21.15.53:0/829924421 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc237fe640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240d5cb0 0x7efc240d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc28e30640 1 --2- 172.21.15.53:0/829924421 >> v2:172.21.15.135:3300/0 conn(0x7efc24162b30 0x7efc240d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1227513364 0 0) 0x7efc240b45f0 con 0x7efc24162750 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc0c003520 con 0x7efc24162750 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 467109893 0 0) 0x7efc0c003520 con 0x7efc24162750 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240d5cb0 msgr2=0x7efc240d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240d5cb0 0x7efc240d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 >> v2:172.21.15.135:3300/0 conn(0x7efc24162b30 msgr2=0x7efc240d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 --2- 172.21.15.53:0/829924421 >> v2:172.21.15.135:3300/0 conn(0x7efc24162b30 0x7efc240d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc240b48d0 con 0x7efc24162750 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.072+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7efc14001670 con 0x7efc24162750 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 >> v1:172.21.15.53:6789/0 conn(0x7efc24162750 legacy=0x7efc240d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc22ffd640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc0c0039b0 0x7efc0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc22ffd640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c004800 0x7efc0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc22ffd640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc0c0051c0 0x7efc0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc240b4f20 con 0x7efc0c0039b0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d0fe0 con 0x7efc0c004800 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c00bf00 con 0x7efc0c0051c0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc23fff640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc0c0039b0 0x7efc0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc237fe640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c004800 0x7efc0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc28e30640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc0c0051c0 0x7efc0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc23fff640 1 -- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc0c0051c0 msgr2=0x7efc0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc23fff640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc0c0051c0 0x7efc0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc23fff640 1 -- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c004800 msgr2=0x7efc0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc23fff640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c004800 0x7efc0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.073+0000 7efc23fff640 1 -- 172.21.15.53:0/829924421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c003520 con 0x7efc0c0039b0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc23fff640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc0c0039b0 0x7efc0c003dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c210 tx=0x7efc0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c92413ad9cbef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007d50 con 0x7efc0c0039b0 2024-05-10T10:39:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc14005af0 con 0x7efc0c0039b0 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc22ffd640 1 -- 172.21.15.53:0/829924421 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007070 con 0x7efc0c0039b0 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc2a893640 1 -- 172.21.15.53:0/829924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc0c0039b0 msgr2=0x7efc0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc2a893640 1 --2- 172.21.15.53:0/829924421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc0c0039b0 0x7efc0c003dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c210 tx=0x7efc0c005830 comp rx=0 tx=0).stop 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc28e30640 1 -- 172.21.15.53:0/829924421 reap_dead start 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc2a893640 1 -- 172.21.15.53:0/829924421 shutdown_connections 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.074+0000 7efc2a893640 1 -- 172.21.15.53:0/829924421 >> 172.21.15.53:0/829924421 conn(0x7efc2409e3b0 msgr2=0x7efc240d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.075+0000 7efc2a893640 1 -- 172.21.15.53:0/829924421 shutdown_connections 2024-05-10T10:39:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.075+0000 7efc2a893640 1 -- 172.21.15.53:0/829924421 wait complete. 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.076+0000 7efc2a893640 1 Processor -- start 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.076+0000 7efc2a893640 1 -- start start 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c0039b0 0x7efc240d5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc24159f10 0x7efc2415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240ce9a0 0x7efc240cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d3630 con 0x7efc24159f10 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d0d80 con 0x7efc0c0039b0 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efc240d37b0 con 0x7efc240ce9a0 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc24159f10 0x7efc2415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc24159f10 0x7efc2415a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35292/0 (socket says 172.21.15.53:35292) 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 -- 172.21.15.53:0/2658572184 learned_addr learned my addr 172.21.15.53:0/2658572184 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc28e30640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240ce9a0 0x7efc240cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc23fff640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c0039b0 0x7efc240d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 -- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240ce9a0 msgr2=0x7efc240cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240ce9a0 0x7efc240cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 -- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c0039b0 msgr2=0x7efc240d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c0039b0 0x7efc240d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 -- 172.21.15.53:0/2658572184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc241f90c0 con 0x7efc24159f10 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc23fff640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c0039b0 0x7efc240d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc237fe640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc24159f10 0x7efc2415a390 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7efc18011910 tx=0x7efc18011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc20ff9640 1 -- 172.21.15.53:0/2658572184 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1800faf0 con 0x7efc24159f10 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc20ff9640 1 -- 172.21.15.53:0/2658572184 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1800fc90 con 0x7efc24159f10 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc24159810 con 0x7efc24159f10 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc20ff9640 1 -- 172.21.15.53:0/2658572184 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18014440 con 0x7efc24159f10 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.077+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241f85e0 con 0x7efc24159f10 2024-05-10T10:39:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.078+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc2406a220 con 0x7efc24159f10 2024-05-10T10:39:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.079+0000 7efc20ff9640 1 -- 172.21.15.53:0/2658572184 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efc18012040 con 0x7efc24159f10 2024-05-10T10:39:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.080+0000 7efc20ff9640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efc100762d0 0x7efc10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.080+0000 7efc20ff9640 1 -- 172.21.15.53:0/2658572184 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7efc18097d10 con 0x7efc24159f10 2024-05-10T10:39:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.080+0000 7efc23fff640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efc100762d0 0x7efc10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.080+0000 7efc23fff640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efc100762d0 0x7efc10078790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efc14033000 tx=0x7efc14009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.084+0000 7efc20ff9640 1 -- 172.21.15.53:0/2658572184 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7efc18019030 con 0x7efc24159f10 2024-05-10T10:39:02.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.274+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc240d1440 con 0x7efc100762d0 2024-05-10T10:39:02.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.281+0000 7efc20ff9640 1 -- 172.21.15.53:0/2658572184 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246754 (secure 0 0 0) 0x7efc240d1440 con 0x7efc100762d0 2024-05-10T10:39:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:02.286 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efc100762d0 msgr2=0x7efc10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efc100762d0 0x7efc10078790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efc14033000 tx=0x7efc14009030 comp rx=0 tx=0).stop 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc24159f10 msgr2=0x7efc2415a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc24159f10 0x7efc2415a390 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7efc18011910 tx=0x7efc18011dd0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 shutdown_connections 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efc100762d0 0x7efc10078790 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc240ce9a0 0x7efc240cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc24159f10 0x7efc2415a390 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 --2- 172.21.15.53:0/2658572184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc0c0039b0 0x7efc240d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 >> 172.21.15.53:0/2658572184 conn(0x7efc2409e3b0 msgr2=0x7efc241605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 shutdown_connections 2024-05-10T10:39:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.284+0000 7efc2a893640 1 -- 172.21.15.53:0/2658572184 wait complete. 2024-05-10T10:39:02.297 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-05-10T10:39:02.235941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254640,"kb_used_data":9352,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720080,"statfs":{"total":773094113280,"available":772833361920,"internally_reserved":0,"allocated":9576448,"data_stored":4992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232129+0000","last_change":"2024-05-10T10:38:57.232129+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232129+0000","last_undegraded":"2024-05-10T10:38:57.232129+0000","last_fullsized":"2024-05-10T10:38:57.232129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227450+0000","last_change":"2024-05-10T10:38:57.227450+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227450+0000","last_undegraded":"2024-05-10T10:38:57.227450+0000","last_fullsized":"2024-05-10T10:38:57.227450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232404+0000","last_change":"2024-05-10T10:38:57.232404+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232404+0000","last_undegraded":"2024-05-10T10:38:57.232404+0000","last_fullsized":"2024-05-10T10:38:57.232404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.244373+0000","last_change":"2024-05-10T10:38:57.244373+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.244373+0000","last_undegraded":"2024-05-10T10:38:57.244373+0000","last_fullsized":"2024-05-10T10:38:57.244373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235333+0000","last_change":"2024-05-10T10:38:57.235333+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235333+0000","last_undegraded":"2024-05-10T10:38:57.235333+0000","last_fullsized":"2024-05-10T10:38:57.235333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.236760+0000","last_change":"2024-05-10T10:38:57.236760+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.236760+0000","last_undegraded":"2024-05-10T10:38:57.236760+0000","last_fullsized":"2024-05-10T10:38:57.236760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227454+0000","last_change":"2024-05-10T10:38:57.227454+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227454+0000","last_undegraded":"2024-05-10T10:38:57.227454+0000","last_fullsized":"2024-05-10T10:38:57.227454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243738+0000","last_change":"2024-05-10T10:38:57.243738+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243738+0000","last_undegraded":"2024-05-10T10:38:57.243738+0000","last_fullsized":"2024-05-10T10:38:57.243738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234548+0000","last_change":"2024-05-10T10:38:57.234548+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234548+0000","last_undegraded":"2024-05-10T10:38:57.234548+0000","last_fullsized":"2024-05-10T10:38:57.234548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234335+0000","last_change":"2024-05-10T10:38:57.234335+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234335+0000","last_undegraded":"2024-05-10T10:38:57.234335+0000","last_fullsized":"2024-05-10T10:38:57.234335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234094+0000","last_change":"2024-05-10T10:38:57.234094+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234094+0000","last_undegraded":"2024-05-10T10:38:57.234094+0000","last_fullsized":"2024-05-10T10:38:57.234094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234550+0000","last_change":"2024-05-10T10:38:57.234550+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234550+0000","last_undegraded":"2024-05-10T10:38:57.234550+0000","last_fullsized":"2024-05-10T10:38:57.234550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243276+0000","last_change":"2024-05-10T10:38:57.243276+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243276+0000","last_undegraded":"2024-05-10T10:38:57.243276+0000","last_fullsized":"2024-05-10T10:38:57.243276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233211+0000","last_change":"2024-05-10T10:38:57.233211+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233211+0000","last_undegraded":"2024-05-10T10:38:57.233211+0000","last_fullsized":"2024-05-10T10:38:57.233211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239867+0000","last_change":"2024-05-10T10:38:57.239867+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239867+0000","last_undegraded":"2024-05-10T10:38:57.239867+0000","last_fullsized":"2024-05-10T10:38:57.239867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235604+0000","last_change":"2024-05-10T10:38:57.235604+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235604+0000","last_undegraded":"2024-05-10T10:38:57.235604+0000","last_fullsized":"2024-05-10T10:38:57.235604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235059+0000","last_change":"2024-05-10T10:38:57.235059+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235059+0000","last_undegraded":"2024-05-10T10:38:57.235059+0000","last_fullsized":"2024-05-10T10:38:57.235059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231848+0000","last_change":"2024-05-10T10:38:57.231848+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231848+0000","last_undegraded":"2024-05-10T10:38:57.231848+0000","last_fullsized":"2024-05-10T10:38:57.231848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233571+0000","last_change":"2024-05-10T10:38:57.233571+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233571+0000","last_undegraded":"2024-05-10T10:38:57.233571+0000","last_fullsized":"2024-05-10T10:38:57.233571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240887+0000","last_change":"2024-05-10T10:38:57.240887+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240887+0000","last_undegraded":"2024-05-10T10:38:57.240887+0000","last_fullsized":"2024-05-10T10:38:57.240887+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232634+0000","last_change":"2024-05-10T10:38:57.232634+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232634+0000","last_undegraded":"2024-05-10T10:38:57.232634+0000","last_fullsized":"2024-05-10T10:38:57.232634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232606+0000","last_change":"2024-05-10T10:38:57.232606+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232606+0000","last_undegraded":"2024-05-10T10:38:57.232606+0000","last_fullsized":"2024-05-10T10:38:57.232606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235045+0000","last_change":"2024-05-10T10:38:57.235045+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235045+0000","last_undegraded":"2024-05-10T10:38:57.235045+0000","last_fullsized":"2024-05-10T10:38:57.235045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.229984+0000","last_change":"2024-05-10T10:38:57.229984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.229984+0000","last_undegraded":"2024-05-10T10:38:57.229984+0000","last_fullsized":"2024-05-10T10:38:57.229984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228435+0000","last_change":"2024-05-10T10:38:57.228435+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228435+0000","last_undegraded":"2024-05-10T10:38:57.228435+0000","last_fullsized":"2024-05-10T10:38:57.228435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233929+0000","last_change":"2024-05-10T10:38:57.233929+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233929+0000","last_undegraded":"2024-05-10T10:38:57.233929+0000","last_fullsized":"2024-05-10T10:38:57.233929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233882+0000","last_change":"2024-05-10T10:38:57.233882+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233882+0000","last_undegraded":"2024-05-10T10:38:57.233882+0000","last_fullsized":"2024-05-10T10:38:57.233882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240407+0000","last_change":"2024-05-10T10:38:57.240407+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240407+0000","last_undegraded":"2024-05-10T10:38:57.240407+0000","last_fullsized":"2024-05-10T10:38:57.240407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228010+0000","last_change":"2024-05-10T10:38:57.228010+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228010+0000","last_undegraded":"2024-05-10T10:38:57.228010+0000","last_fullsized":"2024-05-10T10:38:57.228010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232153+0000","last_change":"2024-05-10T10:38:57.232153+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232153+0000","last_undegraded":"2024-05-10T10:38:57.232153+0000","last_fullsized":"2024-05-10T10:38:57.232153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228694+0000","last_change":"2024-05-10T10:38:57.228694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228694+0000","last_undegraded":"2024-05-10T10:38:57.228694+0000","last_fullsized":"2024-05-10T10:38:57.228694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234543+0000","last_change":"2024-05-10T10:38:57.234543+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234543+0000","last_undegraded":"2024-05-10T10:38:57.234543+0000","last_fullsized":"2024-05-10T10:38:57.234543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233533+0000","last_change":"2024-05-10T10:38:57.233533+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233533+0000","last_undegraded":"2024-05-10T10:38:57.233533+0000","last_fullsized":"2024-05-10T10:38:57.233533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245963+0000","last_change":"2024-05-10T10:38:57.244889+0000","last_active":"2024-05-10T10:38:57.245963+0000","last_peered":"2024-05-10T10:38:57.245963+0000","last_clean":"2024-05-10T10:38:57.245963+0000","last_became_active":"2024-05-10T10:38:57.244439+0000","last_became_peered":"2024-05-10T10:38:57.244439+0000","last_unstale":"2024-05-10T10:38:57.245963+0000","last_undegraded":"2024-05-10T10:38:57.245963+0000","last_fullsized":"2024-05-10T10:38:57.245963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234020+0000","last_change":"2024-05-10T10:38:57.234020+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234020+0000","last_undegraded":"2024-05-10T10:38:57.234020+0000","last_fullsized":"2024-05-10T10:38:57.234020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243113+0000","last_change":"2024-05-10T10:38:57.241164+0000","last_active":"2024-05-10T10:38:57.243113+0000","last_peered":"2024-05-10T10:38:57.243113+0000","last_clean":"2024-05-10T10:38:57.243113+0000","last_became_active":"2024-05-10T10:38:57.240896+0000","last_became_peered":"2024-05-10T10:38:57.240896+0000","last_unstale":"2024-05-10T10:38:57.243113+0000","last_undegraded":"2024-05-10T10:38:57.243113+0000","last_fullsized":"2024-05-10T10:38:57.243113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:16.909583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227680+0000","last_change":"2024-05-10T10:38:57.227680+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227680+0000","last_undegraded":"2024-05-10T10:38:57.227680+0000","last_fullsized":"2024-05-10T10:38:57.227680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241017+0000","last_change":"2024-05-10T10:38:57.238335+0000","last_active":"2024-05-10T10:38:57.241017+0000","last_peered":"2024-05-10T10:38:57.241017+0000","last_clean":"2024-05-10T10:38:57.241017+0000","last_became_active":"2024-05-10T10:38:57.238013+0000","last_became_peered":"2024-05-10T10:38:57.238013+0000","last_unstale":"2024-05-10T10:38:57.241017+0000","last_undegraded":"2024-05-10T10:38:57.241017+0000","last_fullsized":"2024-05-10T10:38:57.241017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234527+0000","last_change":"2024-05-10T10:38:57.234527+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234527+0000","last_undegraded":"2024-05-10T10:38:57.234527+0000","last_fullsized":"2024-05-10T10:38:57.234527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245980+0000","last_change":"2024-05-10T10:38:57.245097+0000","last_active":"2024-05-10T10:38:57.245980+0000","last_peered":"2024-05-10T10:38:57.245980+0000","last_clean":"2024-05-10T10:38:57.245980+0000","last_became_active":"2024-05-10T10:38:57.244782+0000","last_became_peered":"2024-05-10T10:38:57.244782+0000","last_unstale":"2024-05-10T10:38:57.245980+0000","last_undegraded":"2024-05-10T10:38:57.245980+0000","last_fullsized":"2024-05-10T10:38:57.245980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:57.598796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239613+0000","last_change":"2024-05-10T10:38:57.239613+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239613+0000","last_undegraded":"2024-05-10T10:38:57.239613+0000","last_fullsized":"2024-05-10T10:38:57.239613+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251314+0000","last_change":"2024-05-10T10:38:57.249546+0000","last_active":"2024-05-10T10:38:57.251314+0000","last_peered":"2024-05-10T10:38:57.251314+0000","last_clean":"2024-05-10T10:38:57.251314+0000","last_became_active":"2024-05-10T10:38:57.249316+0000","last_became_peered":"2024-05-10T10:38:57.249316+0000","last_unstale":"2024-05-10T10:38:57.251314+0000","last_undegraded":"2024-05-10T10:38:57.251314+0000","last_fullsized":"2024-05-10T10:38:57.251314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:22.781845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232578+0000","last_change":"2024-05-10T10:38:57.232578+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232578+0000","last_undegraded":"2024-05-10T10:38:57.232578+0000","last_fullsized":"2024-05-10T10:38:57.232578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251627+0000","last_change":"2024-05-10T10:38:57.250917+0000","last_active":"2024-05-10T10:38:57.251627+0000","last_peered":"2024-05-10T10:38:57.251627+0000","last_clean":"2024-05-10T10:38:57.251627+0000","last_became_active":"2024-05-10T10:38:57.250631+0000","last_became_peered":"2024-05-10T10:38:57.250631+0000","last_unstale":"2024-05-10T10:38:57.251627+0000","last_undegraded":"2024-05-10T10:38:57.251627+0000","last_fullsized":"2024-05-10T10:38:57.251627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233065+0000","last_change":"2024-05-10T10:38:57.233065+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233065+0000","last_undegraded":"2024-05-10T10:38:57.233065+0000","last_fullsized":"2024-05-10T10:38:57.233065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235166+0000","last_change":"2024-05-10T10:38:57.234005+0000","last_active":"2024-05-10T10:38:57.235166+0000","last_peered":"2024-05-10T10:38:57.235166+0000","last_clean":"2024-05-10T10:38:57.235166+0000","last_became_active":"2024-05-10T10:38:57.233624+0000","last_became_peered":"2024-05-10T10:38:57.233624+0000","last_unstale":"2024-05-10T10:38:57.235166+0000","last_undegraded":"2024-05-10T10:38:57.235166+0000","last_fullsized":"2024-05-10T10:38:57.235166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.241217+0000","last_change":"2024-05-10T10:38:57.241217+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.241217+0000","last_undegraded":"2024-05-10T10:38:57.241217+0000","last_fullsized":"2024-05-10T10:38:57.241217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245553+0000","last_change":"2024-05-10T10:38:57.243368+0000","last_active":"2024-05-10T10:38:57.245553+0000","last_peered":"2024-05-10T10:38:57.245553+0000","last_clean":"2024-05-10T10:38:57.245553+0000","last_became_active":"2024-05-10T10:38:57.243101+0000","last_became_peered":"2024-05-10T10:38:57.243101+0000","last_unstale":"2024-05-10T10:38:57.245553+0000","last_undegraded":"2024-05-10T10:38:57.245553+0000","last_fullsized":"2024-05-10T10:38:57.245553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231694+0000","last_change":"2024-05-10T10:38:57.231694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231694+0000","last_undegraded":"2024-05-10T10:38:57.231694+0000","last_fullsized":"2024-05-10T10:38:57.231694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245621+0000","last_change":"2024-05-10T10:38:57.244172+0000","last_active":"2024-05-10T10:38:57.245621+0000","last_peered":"2024-05-10T10:38:57.245621+0000","last_clean":"2024-05-10T10:38:57.245621+0000","last_became_active":"2024-05-10T10:38:57.243827+0000","last_became_peered":"2024-05-10T10:38:57.243827+0000","last_unstale":"2024-05-10T10:38:57.245621+0000","last_undegraded":"2024-05-10T10:38:57.245621+0000","last_fullsized":"2024-05-10T10:38:57.245621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:14.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.226934+0000","last_change":"2024-05-10T10:38:57.226934+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.226934+0000","last_undegraded":"2024-05-10T10:38:57.226934+0000","last_fullsized":"2024-05-10T10:38:57.226934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243252+0000","last_change":"2024-05-10T10:38:57.242608+0000","last_active":"2024-05-10T10:38:57.243252+0000","last_peered":"2024-05-10T10:38:57.243252+0000","last_clean":"2024-05-10T10:38:57.243252+0000","last_became_active":"2024-05-10T10:38:57.242307+0000","last_became_peered":"2024-05-10T10:38:57.242307+0000","last_unstale":"2024-05-10T10:38:57.243252+0000","last_undegraded":"2024-05-10T10:38:57.243252+0000","last_fullsized":"2024-05-10T10:38:57.243252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230903+0000","last_change":"2024-05-10T10:38:57.230903+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230903+0000","last_undegraded":"2024-05-10T10:38:57.230903+0000","last_fullsized":"2024-05-10T10:38:57.230903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247838+0000","last_change":"2024-05-10T10:38:57.246632+0000","last_active":"2024-05-10T10:38:57.247838+0000","last_peered":"2024-05-10T10:38:57.247838+0000","last_clean":"2024-05-10T10:38:57.247838+0000","last_became_active":"2024-05-10T10:38:57.246417+0000","last_became_peered":"2024-05-10T10:38:57.246417+0000","last_unstale":"2024-05-10T10:38:57.247838+0000","last_undegraded":"2024-05-10T10:38:57.247838+0000","last_fullsized":"2024-05-10T10:38:57.247838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:12.230139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239277+0000","last_change":"2024-05-10T10:38:57.239277+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239277+0000","last_undegraded":"2024-05-10T10:38:57.239277+0000","last_fullsized":"2024-05-10T10:38:57.239277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244891+0000","last_change":"2024-05-10T10:38:57.238700+0000","last_active":"2024-05-10T10:38:57.244891+0000","last_peered":"2024-05-10T10:38:57.244891+0000","last_clean":"2024-05-10T10:38:57.244891+0000","last_became_active":"2024-05-10T10:38:57.238416+0000","last_became_peered":"2024-05-10T10:38:57.238416+0000","last_unstale":"2024-05-10T10:38:57.244891+0000","last_undegraded":"2024-05-10T10:38:57.244891+0000","last_fullsized":"2024-05-10T10:38:57.244891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231354+0000","last_change":"2024-05-10T10:38:57.231354+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231354+0000","last_undegraded":"2024-05-10T10:38:57.231354+0000","last_fullsized":"2024-05-10T10:38:57.231354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235157+0000","last_change":"2024-05-10T10:38:57.233569+0000","last_active":"2024-05-10T10:38:57.235157+0000","last_peered":"2024-05-10T10:38:57.235157+0000","last_clean":"2024-05-10T10:38:57.235157+0000","last_became_active":"2024-05-10T10:38:57.233304+0000","last_became_peered":"2024-05-10T10:38:57.233304+0000","last_unstale":"2024-05-10T10:38:57.235157+0000","last_undegraded":"2024-05-10T10:38:57.235157+0000","last_fullsized":"2024-05-10T10:38:57.235157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:19.208256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231109+0000","last_change":"2024-05-10T10:38:57.231109+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231109+0000","last_undegraded":"2024-05-10T10:38:57.231109+0000","last_fullsized":"2024-05-10T10:38:57.231109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244857+0000","last_change":"2024-05-10T10:38:57.242521+0000","last_active":"2024-05-10T10:38:57.244857+0000","last_peered":"2024-05-10T10:38:57.244857+0000","last_clean":"2024-05-10T10:38:57.244857+0000","last_became_active":"2024-05-10T10:38:57.242195+0000","last_became_peered":"2024-05-10T10:38:57.242195+0000","last_unstale":"2024-05-10T10:38:57.244857+0000","last_undegraded":"2024-05-10T10:38:57.244857+0000","last_fullsized":"2024-05-10T10:38:57.244857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232630+0000","last_change":"2024-05-10T10:38:57.232630+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232630+0000","last_undegraded":"2024-05-10T10:38:57.232630+0000","last_fullsized":"2024-05-10T10:38:57.232630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244065+0000","last_change":"2024-05-10T10:38:57.240790+0000","last_active":"2024-05-10T10:38:57.244065+0000","last_peered":"2024-05-10T10:38:57.244065+0000","last_clean":"2024-05-10T10:38:57.244065+0000","last_became_active":"2024-05-10T10:38:57.240455+0000","last_became_peered":"2024-05-10T10:38:57.240455+0000","last_unstale":"2024-05-10T10:38:57.244065+0000","last_undegraded":"2024-05-10T10:38:57.244065+0000","last_fullsized":"2024-05-10T10:38:57.244065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234758+0000","last_change":"2024-05-10T10:38:57.234758+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234758+0000","last_undegraded":"2024-05-10T10:38:57.234758+0000","last_fullsized":"2024-05-10T10:38:57.234758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.250429+0000","last_change":"2024-05-10T10:38:57.249734+0000","last_active":"2024-05-10T10:38:57.250429+0000","last_peered":"2024-05-10T10:38:57.250429+0000","last_clean":"2024-05-10T10:38:57.250429+0000","last_became_active":"2024-05-10T10:38:57.249452+0000","last_became_peered":"2024-05-10T10:38:57.249452+0000","last_unstale":"2024-05-10T10:38:57.250429+0000","last_undegraded":"2024-05-10T10:38:57.250429+0000","last_fullsized":"2024-05-10T10:38:57.250429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232904+0000","last_change":"2024-05-10T10:38:57.232904+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232904+0000","last_undegraded":"2024-05-10T10:38:57.232904+0000","last_fullsized":"2024-05-10T10:38:57.232904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242899+0000","last_change":"2024-05-10T10:38:57.241984+0000","last_active":"2024-05-10T10:38:57.242899+0000","last_peered":"2024-05-10T10:38:57.242899+0000","last_clean":"2024-05-10T10:38:57.242899+0000","last_became_active":"2024-05-10T10:38:57.241709+0000","last_became_peered":"2024-05-10T10:38:57.241709+0000","last_unstale":"2024-05-10T10:38:57.242899+0000","last_undegraded":"2024-05-10T10:38:57.242899+0000","last_fullsized":"2024-05-10T10:38:57.242899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234942+0000","last_change":"2024-05-10T10:38:57.234942+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234942+0000","last_undegraded":"2024-05-10T10:38:57.234942+0000","last_fullsized":"2024-05-10T10:38:57.234942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242330+0000","last_change":"2024-05-10T10:38:57.240601+0000","last_active":"2024-05-10T10:38:57.242330+0000","last_peered":"2024-05-10T10:38:57.242330+0000","last_clean":"2024-05-10T10:38:57.242330+0000","last_became_active":"2024-05-10T10:38:57.240018+0000","last_became_peered":"2024-05-10T10:38:57.240018+0000","last_unstale":"2024-05-10T10:38:57.242330+0000","last_undegraded":"2024-05-10T10:38:57.242330+0000","last_fullsized":"2024-05-10T10:38:57.242330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233300+0000","last_change":"2024-05-10T10:38:57.233300+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233300+0000","last_undegraded":"2024-05-10T10:38:57.233300+0000","last_fullsized":"2024-05-10T10:38:57.233300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249361+0000","last_change":"2024-05-10T10:38:57.248178+0000","last_active":"2024-05-10T10:38:57.249361+0000","last_peered":"2024-05-10T10:38:57.249361+0000","last_clean":"2024-05-10T10:38:57.249361+0000","last_became_active":"2024-05-10T10:38:57.247953+0000","last_became_peered":"2024-05-10T10:38:57.247953+0000","last_unstale":"2024-05-10T10:38:57.249361+0000","last_undegraded":"2024-05-10T10:38:57.249361+0000","last_fullsized":"2024-05-10T10:38:57.249361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:30.018605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231497+0000","last_change":"2024-05-10T10:38:57.231497+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231497+0000","last_undegraded":"2024-05-10T10:38:57.231497+0000","last_fullsized":"2024-05-10T10:38:57.231497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240504+0000","last_change":"2024-05-10T10:38:57.235875+0000","last_active":"2024-05-10T10:38:57.240504+0000","last_peered":"2024-05-10T10:38:57.240504+0000","last_clean":"2024-05-10T10:38:57.240504+0000","last_became_active":"2024-05-10T10:38:57.235601+0000","last_became_peered":"2024-05-10T10:38:57.235601+0000","last_unstale":"2024-05-10T10:38:57.240504+0000","last_undegraded":"2024-05-10T10:38:57.240504+0000","last_fullsized":"2024-05-10T10:38:57.240504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:28:51.797568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227049+0000","last_change":"2024-05-10T10:38:57.227049+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227049+0000","last_undegraded":"2024-05-10T10:38:57.227049+0000","last_fullsized":"2024-05-10T10:38:57.227049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249243+0000","last_change":"2024-05-10T10:38:57.248738+0000","last_active":"2024-05-10T10:38:57.249243+0000","last_peered":"2024-05-10T10:38:57.249243+0000","last_clean":"2024-05-10T10:38:57.249243+0000","last_became_active":"2024-05-10T10:38:57.248459+0000","last_became_peered":"2024-05-10T10:38:57.248459+0000","last_unstale":"2024-05-10T10:38:57.249243+0000","last_undegraded":"2024-05-10T10:38:57.249243+0000","last_fullsized":"2024-05-10T10:38:57.249243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:40:18.359499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227165+0000","last_change":"2024-05-10T10:38:57.227165+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227165+0000","last_undegraded":"2024-05-10T10:38:57.227165+0000","last_fullsized":"2024-05-10T10:38:57.227165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.239432+0000","last_change":"2024-05-10T10:38:57.230406+0000","last_active":"2024-05-10T10:38:57.239432+0000","last_peered":"2024-05-10T10:38:57.239432+0000","last_clean":"2024-05-10T10:38:57.239432+0000","last_became_active":"2024-05-10T10:38:57.230065+0000","last_became_peered":"2024-05-10T10:38:57.230065+0000","last_unstale":"2024-05-10T10:38:57.239432+0000","last_undegraded":"2024-05-10T10:38:57.239432+0000","last_fullsized":"2024-05-10T10:38:57.239432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:37:46.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232029+0000","last_change":"2024-05-10T10:38:57.232029+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232029+0000","last_undegraded":"2024-05-10T10:38:57.232029+0000","last_fullsized":"2024-05-10T10:38:57.232029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243287+0000","last_change":"2024-05-10T10:38:57.241714+0000","last_active":"2024-05-10T10:38:57.243287+0000","last_peered":"2024-05-10T10:38:57.243287+0000","last_clean":"2024-05-10T10:38:57.243287+0000","last_became_active":"2024-05-10T10:38:57.241428+0000","last_became_peered":"2024-05-10T10:38:57.241428+0000","last_unstale":"2024-05-10T10:38:57.243287+0000","last_undegraded":"2024-05-10T10:38:57.243287+0000","last_fullsized":"2024-05-10T10:38:57.243287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:08.883771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234030+0000","last_change":"2024-05-10T10:38:57.234030+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234030+0000","last_undegraded":"2024-05-10T10:38:57.234030+0000","last_fullsized":"2024-05-10T10:38:57.234030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241462+0000","last_change":"2024-05-10T10:38:57.239328+0000","last_active":"2024-05-10T10:38:57.241462+0000","last_peered":"2024-05-10T10:38:57.241462+0000","last_clean":"2024-05-10T10:38:57.241462+0000","last_became_active":"2024-05-10T10:38:57.239042+0000","last_became_peered":"2024-05-10T10:38:57.239042+0000","last_unstale":"2024-05-10T10:38:57.241462+0000","last_undegraded":"2024-05-10T10:38:57.241462+0000","last_fullsized":"2024-05-10T10:38:57.241462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231035+0000","last_change":"2024-05-10T10:38:57.231035+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231035+0000","last_undegraded":"2024-05-10T10:38:57.231035+0000","last_fullsized":"2024-05-10T10:38:57.231035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243561+0000","last_change":"2024-05-10T10:38:57.242120+0000","last_active":"2024-05-10T10:38:57.243561+0000","last_peered":"2024-05-10T10:38:57.243561+0000","last_clean":"2024-05-10T10:38:57.243561+0000","last_became_active":"2024-05-10T10:38:57.241862+0000","last_became_peered":"2024-05-10T10:38:57.241862+0000","last_unstale":"2024-05-10T10:38:57.243561+0000","last_undegraded":"2024-05-10T10:38:57.243561+0000","last_fullsized":"2024-05-10T10:38:57.243561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:32.971721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230683+0000","last_change":"2024-05-10T10:38:57.230683+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230683+0000","last_undegraded":"2024-05-10T10:38:57.230683+0000","last_fullsized":"2024-05-10T10:38:57.230683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247818+0000","last_change":"2024-05-10T10:38:57.246576+0000","last_active":"2024-05-10T10:38:57.247818+0000","last_peered":"2024-05-10T10:38:57.247818+0000","last_clean":"2024-05-10T10:38:57.247818+0000","last_became_active":"2024-05-10T10:38:57.246353+0000","last_became_peered":"2024-05-10T10:38:57.246353+0000","last_unstale":"2024-05-10T10:38:57.247818+0000","last_undegraded":"2024-05-10T10:38:57.247818+0000","last_fullsized":"2024-05-10T10:38:57.247818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:14.665243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227984+0000","last_change":"2024-05-10T10:38:57.227984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227984+0000","last_undegraded":"2024-05-10T10:38:57.227984+0000","last_fullsized":"2024-05-10T10:38:57.227984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245546+0000","last_change":"2024-05-10T10:38:57.243728+0000","last_active":"2024-05-10T10:38:57.245546+0000","last_peered":"2024-05-10T10:38:57.245546+0000","last_clean":"2024-05-10T10:38:57.245546+0000","last_became_active":"2024-05-10T10:38:57.243406+0000","last_became_peered":"2024-05-10T10:38:57.243406+0000","last_unstale":"2024-05-10T10:38:57.245546+0000","last_undegraded":"2024-05-10T10:38:57.245546+0000","last_fullsized":"2024-05-10T10:38:57.245546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232998+0000","last_change":"2024-05-10T10:38:57.232998+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232998+0000","last_undegraded":"2024-05-10T10:38:57.232998+0000","last_fullsized":"2024-05-10T10:38:57.232998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.248051+0000","last_change":"2024-05-10T10:38:57.244580+0000","last_active":"2024-05-10T10:38:57.248051+0000","last_peered":"2024-05-10T10:38:57.248051+0000","last_clean":"2024-05-10T10:38:57.248051+0000","last_became_active":"2024-05-10T10:38:57.244271+0000","last_became_peered":"2024-05-10T10:38:57.244271+0000","last_unstale":"2024-05-10T10:38:57.248051+0000","last_undegraded":"2024-05-10T10:38:57.248051+0000","last_fullsized":"2024-05-10T10:38:57.248051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231123+0000","last_change":"2024-05-10T10:38:57.231123+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231123+0000","last_undegraded":"2024-05-10T10:38:57.231123+0000","last_fullsized":"2024-05-10T10:38:57.231123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240531+0000","last_change":"2024-05-10T10:38:57.236911+0000","last_active":"2024-05-10T10:38:57.240531+0000","last_peered":"2024-05-10T10:38:57.240531+0000","last_clean":"2024-05-10T10:38:57.240531+0000","last_became_active":"2024-05-10T10:38:57.236571+0000","last_became_peered":"2024-05-10T10:38:57.236571+0000","last_unstale":"2024-05-10T10:38:57.240531+0000","last_undegraded":"2024-05-10T10:38:57.240531+0000","last_fullsized":"2024-05-10T10:38:57.240531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:14.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233921+0000","last_change":"2024-05-10T10:38:57.233921+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233921+0000","last_undegraded":"2024-05-10T10:38:57.233921+0000","last_fullsized":"2024-05-10T10:38:57.233921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244349+0000","last_change":"2024-05-10T10:38:57.243643+0000","last_active":"2024-05-10T10:38:57.244349+0000","last_peered":"2024-05-10T10:38:57.244349+0000","last_clean":"2024-05-10T10:38:57.244349+0000","last_became_active":"2024-05-10T10:38:57.243364+0000","last_became_peered":"2024-05-10T10:38:57.243364+0000","last_unstale":"2024-05-10T10:38:57.244349+0000","last_undegraded":"2024-05-10T10:38:57.244349+0000","last_fullsized":"2024-05-10T10:38:57.244349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230113+0000","last_change":"2024-05-10T10:38:57.230113+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230113+0000","last_undegraded":"2024-05-10T10:38:57.230113+0000","last_fullsized":"2024-05-10T10:38:57.230113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241495+0000","last_change":"2024-05-10T10:38:57.239379+0000","last_active":"2024-05-10T10:38:57.241495+0000","last_peered":"2024-05-10T10:38:57.241495+0000","last_clean":"2024-05-10T10:38:57.241495+0000","last_became_active":"2024-05-10T10:38:57.239057+0000","last_became_peered":"2024-05-10T10:38:57.239057+0000","last_unstale":"2024-05-10T10:38:57.241495+0000","last_undegraded":"2024-05-10T10:38:57.241495+0000","last_fullsized":"2024-05-10T10:38:57.241495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228473+0000","last_change":"2024-05-10T10:38:57.228473+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228473+0000","last_undegraded":"2024-05-10T10:38:57.228473+0000","last_fullsized":"2024-05-10T10:38:57.228473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244095+0000","last_change":"2024-05-10T10:38:57.242780+0000","last_active":"2024-05-10T10:38:57.244095+0000","last_peered":"2024-05-10T10:38:57.244095+0000","last_clean":"2024-05-10T10:38:57.244095+0000","last_became_active":"2024-05-10T10:38:57.242494+0000","last_became_peered":"2024-05-10T10:38:57.242494+0000","last_unstale":"2024-05-10T10:38:57.244095+0000","last_undegraded":"2024-05-10T10:38:57.244095+0000","last_fullsized":"2024-05-10T10:38:57.244095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":242,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.228637+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:38:57.228637+0000","last_peered":"2024-05-10T10:38:57.228637+0000","last_clean":"2024-05-10T10:38:57.228637+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:38:57.228637+0000","last_undegraded":"2024-05-10T10:38:57.228637+0000","last_fullsized":"2024-05-10T10:38:57.228637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:39:02.301 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:39:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.439+0000 7f676f8c4640 1 Processor -- start 2024-05-10T10:39:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.439+0000 7f676f8c4640 1 -- start start 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676f8c4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67680d0230 0x7f67680cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676f8c4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 0x7f67680ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676f8c4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676815fc90 con 0x7f67680b8860 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676f8c4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f676806ec30 con 0x7f67680d0230 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676f8c4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f676806ee50 con 0x7f67680cfbb0 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676d660640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f67680b8860 0x7f676806e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60676/0 (socket says 172.21.15.53:60676) 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676d660640 1 -- 172.21.15.53:0/1754266626 learned_addr learned my addr 172.21.15.53:0/1754266626 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676ce5f640 1 --2- 172.21.15.53:0/1754266626 >> v2:172.21.15.135:3300/0 conn(0x7f67680d0230 0x7f67680cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676de61640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 0x7f67680ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 307996516 0 0) 0x7f676815fc90 con 0x7f67680b8860 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6754003520 con 0x7f67680b8860 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3493160528 0 0) 0x7f6754003520 con 0x7f67680b8860 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 msgr2=0x7f67680ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 0x7f67680ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 >> v2:172.21.15.135:3300/0 conn(0x7f67680d0230 msgr2=0x7f67680cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 --2- 172.21.15.53:0/1754266626 >> v2:172.21.15.135:3300/0 conn(0x7f67680d0230 0x7f67680cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67680b8c40 con 0x7f67680b8860 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f675c001670 con 0x7f67680b8860 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.440+0000 7f676ce5f640 1 --2- 172.21.15.53:0/1754266626 >> v2:172.21.15.135:3300/0 conn(0x7f67680d0230 0x7f67680cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 >> v1:172.21.15.53:6789/0 conn(0x7f67680b8860 legacy=0x7f676806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f675bfff640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6754003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f675bfff640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 0x7f6754004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f675bfff640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6754004d40 0x7f67540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f676806ec30 con 0x7f67540039e0 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f676806ee50 con 0x7f6754004d40 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f675400b9f0 con 0x7f67680cfbb0 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676d660640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6754003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676ce5f640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 0x7f6754004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676de61640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6754004d40 0x7f67540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676d660640 1 -- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 msgr2=0x7f6754004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676d660640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cfbb0 0x7f6754004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676d660640 1 -- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6754004d40 msgr2=0x7f67540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676d660640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6754004d40 0x7f67540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.441+0000 7f676d660640 1 -- 172.21.15.53:0/1754266626 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6754003520 con 0x7f67540039e0 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f676d660640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6754003e00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c210 tx=0x7f6754005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd982e3eedae22d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c007d30 con 0x7f67540039e0 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c005ad0 con 0x7f67540039e0 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f675bfff640 1 -- 172.21.15.53:0/1754266626 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c007070 con 0x7f67540039e0 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f676f8c4640 1 -- 172.21.15.53:0/1754266626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 msgr2=0x7f6754003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f676f8c4640 1 --2- 172.21.15.53:0/1754266626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6754003e00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c210 tx=0x7f6754005830 comp rx=0 tx=0).stop 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f676de61640 1 -- 172.21.15.53:0/1754266626 reap_dead start 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f676f8c4640 1 -- 172.21.15.53:0/1754266626 shutdown_connections 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.442+0000 7f676f8c4640 1 -- 172.21.15.53:0/1754266626 >> 172.21.15.53:0/1754266626 conn(0x7f67680d2df0 msgr2=0x7f67680d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.443+0000 7f676f8c4640 1 -- 172.21.15.53:0/1754266626 shutdown_connections 2024-05-10T10:39:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.443+0000 7f676f8c4640 1 -- 172.21.15.53:0/1754266626 wait complete. 2024-05-10T10:39:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 Processor -- start 2024-05-10T10:39:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 -- start start 2024-05-10T10:39:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6768160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6768154310 0x7f67681f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67681f5430 0x7f67680c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67680cdb30 con 0x7f67540039e0 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67680cd850 con 0x7f6768154310 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.444+0000 7f676f8c4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f67680cde10 con 0x7f67681f5430 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6768160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6768160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35300/0 (socket says 172.21.15.53:35300) 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 -- 172.21.15.53:0/4054251325 learned_addr learned my addr 172.21.15.53:0/4054251325 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676de61640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67681f5430 0x7f67680c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676ce5f640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6768154310 0x7f67681f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 -- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67681f5430 msgr2=0x7f67680c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67681f5430 0x7f67680c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 -- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6768154310 msgr2=0x7f67681f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6768154310 0x7f67681f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 -- 172.21.15.53:0/4054251325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67680cafe0 con 0x7f67540039e0 2024-05-10T10:39:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676ce5f640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6768154310 0x7f67681f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676d660640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6768160ce0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c330 tx=0x7f675c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f6759ffb640 1 -- 172.21.15.53:0/4054251325 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c007d10 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f6759ffb640 1 -- 172.21.15.53:0/4054251325 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c0049f0 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6768154750 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.445+0000 7f6759ffb640 1 -- 172.21.15.53:0/4054251325 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c004da0 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.446+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67681f5b80 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.447+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676806a220 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.447+0000 7f6759ffb640 1 -- 172.21.15.53:0/4054251325 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f675c045030 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.448+0000 7f6759ffb640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f674c0762d0 0x7f674c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.448+0000 7f6759ffb640 1 -- 172.21.15.53:0/4054251325 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f675c0b77b0 con 0x7f67540039e0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.448+0000 7f676ce5f640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f674c0762d0 0x7f674c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.449+0000 7f676ce5f640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f674c0762d0 0x7f674c078790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f67680745b0 tx=0x7f6764008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.452+0000 7f6759ffb640 1 -- 172.21.15.53:0/4054251325 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f675c04c020 con 0x7f67540039e0 2024-05-10T10:39:02.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.640+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67681f5e60 con 0x7f674c0762d0 2024-05-10T10:39:02.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.647+0000 7f6759ffb640 1 -- 172.21.15.53:0/4054251325 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246754 (secure 0 0 0) 0x7f67681f5e60 con 0x7f674c0762d0 2024-05-10T10:39:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:02.652 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f674c0762d0 msgr2=0x7f674c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f674c0762d0 0x7f674c078790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f67680745b0 tx=0x7f6764008040 comp rx=0 tx=0).stop 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 msgr2=0x7f6768160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6768160ce0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c330 tx=0x7f675c001a40 comp rx=0 tx=0).stop 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 shutdown_connections 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f674c0762d0 0x7f674c078790 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67681f5430 0x7f67680c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6768154310 0x7f67681f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 --2- 172.21.15.53:0/4054251325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67540039e0 0x7f6768160ce0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 >> 172.21.15.53:0/4054251325 conn(0x7f67680d2df0 msgr2=0x7f6768070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 shutdown_connections 2024-05-10T10:39:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.650+0000 7f676f8c4640 1 -- 172.21.15.53:0/4054251325 wait complete. 2024-05-10T10:39:02.662 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-05-10T10:39:02.235941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254640,"kb_used_data":9352,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720080,"statfs":{"total":773094113280,"available":772833361920,"internally_reserved":0,"allocated":9576448,"data_stored":4992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232129+0000","last_change":"2024-05-10T10:38:57.232129+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232129+0000","last_undegraded":"2024-05-10T10:38:57.232129+0000","last_fullsized":"2024-05-10T10:38:57.232129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227450+0000","last_change":"2024-05-10T10:38:57.227450+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227450+0000","last_undegraded":"2024-05-10T10:38:57.227450+0000","last_fullsized":"2024-05-10T10:38:57.227450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232404+0000","last_change":"2024-05-10T10:38:57.232404+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232404+0000","last_undegraded":"2024-05-10T10:38:57.232404+0000","last_fullsized":"2024-05-10T10:38:57.232404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.244373+0000","last_change":"2024-05-10T10:38:57.244373+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.244373+0000","last_undegraded":"2024-05-10T10:38:57.244373+0000","last_fullsized":"2024-05-10T10:38:57.244373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235333+0000","last_change":"2024-05-10T10:38:57.235333+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235333+0000","last_undegraded":"2024-05-10T10:38:57.235333+0000","last_fullsized":"2024-05-10T10:38:57.235333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.236760+0000","last_change":"2024-05-10T10:38:57.236760+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.236760+0000","last_undegraded":"2024-05-10T10:38:57.236760+0000","last_fullsized":"2024-05-10T10:38:57.236760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227454+0000","last_change":"2024-05-10T10:38:57.227454+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227454+0000","last_undegraded":"2024-05-10T10:38:57.227454+0000","last_fullsized":"2024-05-10T10:38:57.227454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243738+0000","last_change":"2024-05-10T10:38:57.243738+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243738+0000","last_undegraded":"2024-05-10T10:38:57.243738+0000","last_fullsized":"2024-05-10T10:38:57.243738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234548+0000","last_change":"2024-05-10T10:38:57.234548+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234548+0000","last_undegraded":"2024-05-10T10:38:57.234548+0000","last_fullsized":"2024-05-10T10:38:57.234548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234335+0000","last_change":"2024-05-10T10:38:57.234335+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234335+0000","last_undegraded":"2024-05-10T10:38:57.234335+0000","last_fullsized":"2024-05-10T10:38:57.234335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234094+0000","last_change":"2024-05-10T10:38:57.234094+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234094+0000","last_undegraded":"2024-05-10T10:38:57.234094+0000","last_fullsized":"2024-05-10T10:38:57.234094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234550+0000","last_change":"2024-05-10T10:38:57.234550+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234550+0000","last_undegraded":"2024-05-10T10:38:57.234550+0000","last_fullsized":"2024-05-10T10:38:57.234550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243276+0000","last_change":"2024-05-10T10:38:57.243276+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243276+0000","last_undegraded":"2024-05-10T10:38:57.243276+0000","last_fullsized":"2024-05-10T10:38:57.243276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233211+0000","last_change":"2024-05-10T10:38:57.233211+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233211+0000","last_undegraded":"2024-05-10T10:38:57.233211+0000","last_fullsized":"2024-05-10T10:38:57.233211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239867+0000","last_change":"2024-05-10T10:38:57.239867+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239867+0000","last_undegraded":"2024-05-10T10:38:57.239867+0000","last_fullsized":"2024-05-10T10:38:57.239867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235604+0000","last_change":"2024-05-10T10:38:57.235604+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235604+0000","last_undegraded":"2024-05-10T10:38:57.235604+0000","last_fullsized":"2024-05-10T10:38:57.235604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235059+0000","last_change":"2024-05-10T10:38:57.235059+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235059+0000","last_undegraded":"2024-05-10T10:38:57.235059+0000","last_fullsized":"2024-05-10T10:38:57.235059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231848+0000","last_change":"2024-05-10T10:38:57.231848+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231848+0000","last_undegraded":"2024-05-10T10:38:57.231848+0000","last_fullsized":"2024-05-10T10:38:57.231848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233571+0000","last_change":"2024-05-10T10:38:57.233571+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233571+0000","last_undegraded":"2024-05-10T10:38:57.233571+0000","last_fullsized":"2024-05-10T10:38:57.233571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240887+0000","last_change":"2024-05-10T10:38:57.240887+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240887+0000","last_undegraded":"2024-05-10T10:38:57.240887+0000","last_fullsized":"2024-05-10T10:38:57.240887+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232634+0000","last_change":"2024-05-10T10:38:57.232634+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232634+0000","last_undegraded":"2024-05-10T10:38:57.232634+0000","last_fullsized":"2024-05-10T10:38:57.232634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232606+0000","last_change":"2024-05-10T10:38:57.232606+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232606+0000","last_undegraded":"2024-05-10T10:38:57.232606+0000","last_fullsized":"2024-05-10T10:38:57.232606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235045+0000","last_change":"2024-05-10T10:38:57.235045+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235045+0000","last_undegraded":"2024-05-10T10:38:57.235045+0000","last_fullsized":"2024-05-10T10:38:57.235045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.229984+0000","last_change":"2024-05-10T10:38:57.229984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.229984+0000","last_undegraded":"2024-05-10T10:38:57.229984+0000","last_fullsized":"2024-05-10T10:38:57.229984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228435+0000","last_change":"2024-05-10T10:38:57.228435+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228435+0000","last_undegraded":"2024-05-10T10:38:57.228435+0000","last_fullsized":"2024-05-10T10:38:57.228435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233929+0000","last_change":"2024-05-10T10:38:57.233929+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233929+0000","last_undegraded":"2024-05-10T10:38:57.233929+0000","last_fullsized":"2024-05-10T10:38:57.233929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233882+0000","last_change":"2024-05-10T10:38:57.233882+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233882+0000","last_undegraded":"2024-05-10T10:38:57.233882+0000","last_fullsized":"2024-05-10T10:38:57.233882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240407+0000","last_change":"2024-05-10T10:38:57.240407+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240407+0000","last_undegraded":"2024-05-10T10:38:57.240407+0000","last_fullsized":"2024-05-10T10:38:57.240407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228010+0000","last_change":"2024-05-10T10:38:57.228010+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228010+0000","last_undegraded":"2024-05-10T10:38:57.228010+0000","last_fullsized":"2024-05-10T10:38:57.228010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232153+0000","last_change":"2024-05-10T10:38:57.232153+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232153+0000","last_undegraded":"2024-05-10T10:38:57.232153+0000","last_fullsized":"2024-05-10T10:38:57.232153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228694+0000","last_change":"2024-05-10T10:38:57.228694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228694+0000","last_undegraded":"2024-05-10T10:38:57.228694+0000","last_fullsized":"2024-05-10T10:38:57.228694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234543+0000","last_change":"2024-05-10T10:38:57.234543+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234543+0000","last_undegraded":"2024-05-10T10:38:57.234543+0000","last_fullsized":"2024-05-10T10:38:57.234543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233533+0000","last_change":"2024-05-10T10:38:57.233533+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233533+0000","last_undegraded":"2024-05-10T10:38:57.233533+0000","last_fullsized":"2024-05-10T10:38:57.233533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245963+0000","last_change":"2024-05-10T10:38:57.244889+0000","last_active":"2024-05-10T10:38:57.245963+0000","last_peered":"2024-05-10T10:38:57.245963+0000","last_clean":"2024-05-10T10:38:57.245963+0000","last_became_active":"2024-05-10T10:38:57.244439+0000","last_became_peered":"2024-05-10T10:38:57.244439+0000","last_unstale":"2024-05-10T10:38:57.245963+0000","last_undegraded":"2024-05-10T10:38:57.245963+0000","last_fullsized":"2024-05-10T10:38:57.245963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234020+0000","last_change":"2024-05-10T10:38:57.234020+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234020+0000","last_undegraded":"2024-05-10T10:38:57.234020+0000","last_fullsized":"2024-05-10T10:38:57.234020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243113+0000","last_change":"2024-05-10T10:38:57.241164+0000","last_active":"2024-05-10T10:38:57.243113+0000","last_peered":"2024-05-10T10:38:57.243113+0000","last_clean":"2024-05-10T10:38:57.243113+0000","last_became_active":"2024-05-10T10:38:57.240896+0000","last_became_peered":"2024-05-10T10:38:57.240896+0000","last_unstale":"2024-05-10T10:38:57.243113+0000","last_undegraded":"2024-05-10T10:38:57.243113+0000","last_fullsized":"2024-05-10T10:38:57.243113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:16.909583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227680+0000","last_change":"2024-05-10T10:38:57.227680+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227680+0000","last_undegraded":"2024-05-10T10:38:57.227680+0000","last_fullsized":"2024-05-10T10:38:57.227680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241017+0000","last_change":"2024-05-10T10:38:57.238335+0000","last_active":"2024-05-10T10:38:57.241017+0000","last_peered":"2024-05-10T10:38:57.241017+0000","last_clean":"2024-05-10T10:38:57.241017+0000","last_became_active":"2024-05-10T10:38:57.238013+0000","last_became_peered":"2024-05-10T10:38:57.238013+0000","last_unstale":"2024-05-10T10:38:57.241017+0000","last_undegraded":"2024-05-10T10:38:57.241017+0000","last_fullsized":"2024-05-10T10:38:57.241017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234527+0000","last_change":"2024-05-10T10:38:57.234527+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234527+0000","last_undegraded":"2024-05-10T10:38:57.234527+0000","last_fullsized":"2024-05-10T10:38:57.234527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245980+0000","last_change":"2024-05-10T10:38:57.245097+0000","last_active":"2024-05-10T10:38:57.245980+0000","last_peered":"2024-05-10T10:38:57.245980+0000","last_clean":"2024-05-10T10:38:57.245980+0000","last_became_active":"2024-05-10T10:38:57.244782+0000","last_became_peered":"2024-05-10T10:38:57.244782+0000","last_unstale":"2024-05-10T10:38:57.245980+0000","last_undegraded":"2024-05-10T10:38:57.245980+0000","last_fullsized":"2024-05-10T10:38:57.245980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:57.598796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239613+0000","last_change":"2024-05-10T10:38:57.239613+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239613+0000","last_undegraded":"2024-05-10T10:38:57.239613+0000","last_fullsized":"2024-05-10T10:38:57.239613+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251314+0000","last_change":"2024-05-10T10:38:57.249546+0000","last_active":"2024-05-10T10:38:57.251314+0000","last_peered":"2024-05-10T10:38:57.251314+0000","last_clean":"2024-05-10T10:38:57.251314+0000","last_became_active":"2024-05-10T10:38:57.249316+0000","last_became_peered":"2024-05-10T10:38:57.249316+0000","last_unstale":"2024-05-10T10:38:57.251314+0000","last_undegraded":"2024-05-10T10:38:57.251314+0000","last_fullsized":"2024-05-10T10:38:57.251314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:22.781845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232578+0000","last_change":"2024-05-10T10:38:57.232578+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232578+0000","last_undegraded":"2024-05-10T10:38:57.232578+0000","last_fullsized":"2024-05-10T10:38:57.232578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251627+0000","last_change":"2024-05-10T10:38:57.250917+0000","last_active":"2024-05-10T10:38:57.251627+0000","last_peered":"2024-05-10T10:38:57.251627+0000","last_clean":"2024-05-10T10:38:57.251627+0000","last_became_active":"2024-05-10T10:38:57.250631+0000","last_became_peered":"2024-05-10T10:38:57.250631+0000","last_unstale":"2024-05-10T10:38:57.251627+0000","last_undegraded":"2024-05-10T10:38:57.251627+0000","last_fullsized":"2024-05-10T10:38:57.251627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233065+0000","last_change":"2024-05-10T10:38:57.233065+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233065+0000","last_undegraded":"2024-05-10T10:38:57.233065+0000","last_fullsized":"2024-05-10T10:38:57.233065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235166+0000","last_change":"2024-05-10T10:38:57.234005+0000","last_active":"2024-05-10T10:38:57.235166+0000","last_peered":"2024-05-10T10:38:57.235166+0000","last_clean":"2024-05-10T10:38:57.235166+0000","last_became_active":"2024-05-10T10:38:57.233624+0000","last_became_peered":"2024-05-10T10:38:57.233624+0000","last_unstale":"2024-05-10T10:38:57.235166+0000","last_undegraded":"2024-05-10T10:38:57.235166+0000","last_fullsized":"2024-05-10T10:38:57.235166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.241217+0000","last_change":"2024-05-10T10:38:57.241217+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.241217+0000","last_undegraded":"2024-05-10T10:38:57.241217+0000","last_fullsized":"2024-05-10T10:38:57.241217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245553+0000","last_change":"2024-05-10T10:38:57.243368+0000","last_active":"2024-05-10T10:38:57.245553+0000","last_peered":"2024-05-10T10:38:57.245553+0000","last_clean":"2024-05-10T10:38:57.245553+0000","last_became_active":"2024-05-10T10:38:57.243101+0000","last_became_peered":"2024-05-10T10:38:57.243101+0000","last_unstale":"2024-05-10T10:38:57.245553+0000","last_undegraded":"2024-05-10T10:38:57.245553+0000","last_fullsized":"2024-05-10T10:38:57.245553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231694+0000","last_change":"2024-05-10T10:38:57.231694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231694+0000","last_undegraded":"2024-05-10T10:38:57.231694+0000","last_fullsized":"2024-05-10T10:38:57.231694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245621+0000","last_change":"2024-05-10T10:38:57.244172+0000","last_active":"2024-05-10T10:38:57.245621+0000","last_peered":"2024-05-10T10:38:57.245621+0000","last_clean":"2024-05-10T10:38:57.245621+0000","last_became_active":"2024-05-10T10:38:57.243827+0000","last_became_peered":"2024-05-10T10:38:57.243827+0000","last_unstale":"2024-05-10T10:38:57.245621+0000","last_undegraded":"2024-05-10T10:38:57.245621+0000","last_fullsized":"2024-05-10T10:38:57.245621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:14.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.226934+0000","last_change":"2024-05-10T10:38:57.226934+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.226934+0000","last_undegraded":"2024-05-10T10:38:57.226934+0000","last_fullsized":"2024-05-10T10:38:57.226934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243252+0000","last_change":"2024-05-10T10:38:57.242608+0000","last_active":"2024-05-10T10:38:57.243252+0000","last_peered":"2024-05-10T10:38:57.243252+0000","last_clean":"2024-05-10T10:38:57.243252+0000","last_became_active":"2024-05-10T10:38:57.242307+0000","last_became_peered":"2024-05-10T10:38:57.242307+0000","last_unstale":"2024-05-10T10:38:57.243252+0000","last_undegraded":"2024-05-10T10:38:57.243252+0000","last_fullsized":"2024-05-10T10:38:57.243252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230903+0000","last_change":"2024-05-10T10:38:57.230903+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230903+0000","last_undegraded":"2024-05-10T10:38:57.230903+0000","last_fullsized":"2024-05-10T10:38:57.230903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247838+0000","last_change":"2024-05-10T10:38:57.246632+0000","last_active":"2024-05-10T10:38:57.247838+0000","last_peered":"2024-05-10T10:38:57.247838+0000","last_clean":"2024-05-10T10:38:57.247838+0000","last_became_active":"2024-05-10T10:38:57.246417+0000","last_became_peered":"2024-05-10T10:38:57.246417+0000","last_unstale":"2024-05-10T10:38:57.247838+0000","last_undegraded":"2024-05-10T10:38:57.247838+0000","last_fullsized":"2024-05-10T10:38:57.247838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:12.230139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239277+0000","last_change":"2024-05-10T10:38:57.239277+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239277+0000","last_undegraded":"2024-05-10T10:38:57.239277+0000","last_fullsized":"2024-05-10T10:38:57.239277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244891+0000","last_change":"2024-05-10T10:38:57.238700+0000","last_active":"2024-05-10T10:38:57.244891+0000","last_peered":"2024-05-10T10:38:57.244891+0000","last_clean":"2024-05-10T10:38:57.244891+0000","last_became_active":"2024-05-10T10:38:57.238416+0000","last_became_peered":"2024-05-10T10:38:57.238416+0000","last_unstale":"2024-05-10T10:38:57.244891+0000","last_undegraded":"2024-05-10T10:38:57.244891+0000","last_fullsized":"2024-05-10T10:38:57.244891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231354+0000","last_change":"2024-05-10T10:38:57.231354+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231354+0000","last_undegraded":"2024-05-10T10:38:57.231354+0000","last_fullsized":"2024-05-10T10:38:57.231354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235157+0000","last_change":"2024-05-10T10:38:57.233569+0000","last_active":"2024-05-10T10:38:57.235157+0000","last_peered":"2024-05-10T10:38:57.235157+0000","last_clean":"2024-05-10T10:38:57.235157+0000","last_became_active":"2024-05-10T10:38:57.233304+0000","last_became_peered":"2024-05-10T10:38:57.233304+0000","last_unstale":"2024-05-10T10:38:57.235157+0000","last_undegraded":"2024-05-10T10:38:57.235157+0000","last_fullsized":"2024-05-10T10:38:57.235157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:19.208256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231109+0000","last_change":"2024-05-10T10:38:57.231109+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231109+0000","last_undegraded":"2024-05-10T10:38:57.231109+0000","last_fullsized":"2024-05-10T10:38:57.231109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244857+0000","last_change":"2024-05-10T10:38:57.242521+0000","last_active":"2024-05-10T10:38:57.244857+0000","last_peered":"2024-05-10T10:38:57.244857+0000","last_clean":"2024-05-10T10:38:57.244857+0000","last_became_active":"2024-05-10T10:38:57.242195+0000","last_became_peered":"2024-05-10T10:38:57.242195+0000","last_unstale":"2024-05-10T10:38:57.244857+0000","last_undegraded":"2024-05-10T10:38:57.244857+0000","last_fullsized":"2024-05-10T10:38:57.244857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232630+0000","last_change":"2024-05-10T10:38:57.232630+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232630+0000","last_undegraded":"2024-05-10T10:38:57.232630+0000","last_fullsized":"2024-05-10T10:38:57.232630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244065+0000","last_change":"2024-05-10T10:38:57.240790+0000","last_active":"2024-05-10T10:38:57.244065+0000","last_peered":"2024-05-10T10:38:57.244065+0000","last_clean":"2024-05-10T10:38:57.244065+0000","last_became_active":"2024-05-10T10:38:57.240455+0000","last_became_peered":"2024-05-10T10:38:57.240455+0000","last_unstale":"2024-05-10T10:38:57.244065+0000","last_undegraded":"2024-05-10T10:38:57.244065+0000","last_fullsized":"2024-05-10T10:38:57.244065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234758+0000","last_change":"2024-05-10T10:38:57.234758+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234758+0000","last_undegraded":"2024-05-10T10:38:57.234758+0000","last_fullsized":"2024-05-10T10:38:57.234758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.250429+0000","last_change":"2024-05-10T10:38:57.249734+0000","last_active":"2024-05-10T10:38:57.250429+0000","last_peered":"2024-05-10T10:38:57.250429+0000","last_clean":"2024-05-10T10:38:57.250429+0000","last_became_active":"2024-05-10T10:38:57.249452+0000","last_became_peered":"2024-05-10T10:38:57.249452+0000","last_unstale":"2024-05-10T10:38:57.250429+0000","last_undegraded":"2024-05-10T10:38:57.250429+0000","last_fullsized":"2024-05-10T10:38:57.250429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232904+0000","last_change":"2024-05-10T10:38:57.232904+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232904+0000","last_undegraded":"2024-05-10T10:38:57.232904+0000","last_fullsized":"2024-05-10T10:38:57.232904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242899+0000","last_change":"2024-05-10T10:38:57.241984+0000","last_active":"2024-05-10T10:38:57.242899+0000","last_peered":"2024-05-10T10:38:57.242899+0000","last_clean":"2024-05-10T10:38:57.242899+0000","last_became_active":"2024-05-10T10:38:57.241709+0000","last_became_peered":"2024-05-10T10:38:57.241709+0000","last_unstale":"2024-05-10T10:38:57.242899+0000","last_undegraded":"2024-05-10T10:38:57.242899+0000","last_fullsized":"2024-05-10T10:38:57.242899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234942+0000","last_change":"2024-05-10T10:38:57.234942+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234942+0000","last_undegraded":"2024-05-10T10:38:57.234942+0000","last_fullsized":"2024-05-10T10:38:57.234942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242330+0000","last_change":"2024-05-10T10:38:57.240601+0000","last_active":"2024-05-10T10:38:57.242330+0000","last_peered":"2024-05-10T10:38:57.242330+0000","last_clean":"2024-05-10T10:38:57.242330+0000","last_became_active":"2024-05-10T10:38:57.240018+0000","last_became_peered":"2024-05-10T10:38:57.240018+0000","last_unstale":"2024-05-10T10:38:57.242330+0000","last_undegraded":"2024-05-10T10:38:57.242330+0000","last_fullsized":"2024-05-10T10:38:57.242330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233300+0000","last_change":"2024-05-10T10:38:57.233300+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233300+0000","last_undegraded":"2024-05-10T10:38:57.233300+0000","last_fullsized":"2024-05-10T10:38:57.233300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249361+0000","last_change":"2024-05-10T10:38:57.248178+0000","last_active":"2024-05-10T10:38:57.249361+0000","last_peered":"2024-05-10T10:38:57.249361+0000","last_clean":"2024-05-10T10:38:57.249361+0000","last_became_active":"2024-05-10T10:38:57.247953+0000","last_became_peered":"2024-05-10T10:38:57.247953+0000","last_unstale":"2024-05-10T10:38:57.249361+0000","last_undegraded":"2024-05-10T10:38:57.249361+0000","last_fullsized":"2024-05-10T10:38:57.249361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:30.018605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231497+0000","last_change":"2024-05-10T10:38:57.231497+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231497+0000","last_undegraded":"2024-05-10T10:38:57.231497+0000","last_fullsized":"2024-05-10T10:38:57.231497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240504+0000","last_change":"2024-05-10T10:38:57.235875+0000","last_active":"2024-05-10T10:38:57.240504+0000","last_peered":"2024-05-10T10:38:57.240504+0000","last_clean":"2024-05-10T10:38:57.240504+0000","last_became_active":"2024-05-10T10:38:57.235601+0000","last_became_peered":"2024-05-10T10:38:57.235601+0000","last_unstale":"2024-05-10T10:38:57.240504+0000","last_undegraded":"2024-05-10T10:38:57.240504+0000","last_fullsized":"2024-05-10T10:38:57.240504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:28:51.797568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227049+0000","last_change":"2024-05-10T10:38:57.227049+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227049+0000","last_undegraded":"2024-05-10T10:38:57.227049+0000","last_fullsized":"2024-05-10T10:38:57.227049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249243+0000","last_change":"2024-05-10T10:38:57.248738+0000","last_active":"2024-05-10T10:38:57.249243+0000","last_peered":"2024-05-10T10:38:57.249243+0000","last_clean":"2024-05-10T10:38:57.249243+0000","last_became_active":"2024-05-10T10:38:57.248459+0000","last_became_peered":"2024-05-10T10:38:57.248459+0000","last_unstale":"2024-05-10T10:38:57.249243+0000","last_undegraded":"2024-05-10T10:38:57.249243+0000","last_fullsized":"2024-05-10T10:38:57.249243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:40:18.359499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227165+0000","last_change":"2024-05-10T10:38:57.227165+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227165+0000","last_undegraded":"2024-05-10T10:38:57.227165+0000","last_fullsized":"2024-05-10T10:38:57.227165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.239432+0000","last_change":"2024-05-10T10:38:57.230406+0000","last_active":"2024-05-10T10:38:57.239432+0000","last_peered":"2024-05-10T10:38:57.239432+0000","last_clean":"2024-05-10T10:38:57.239432+0000","last_became_active":"2024-05-10T10:38:57.230065+0000","last_became_peered":"2024-05-10T10:38:57.230065+0000","last_unstale":"2024-05-10T10:38:57.239432+0000","last_undegraded":"2024-05-10T10:38:57.239432+0000","last_fullsized":"2024-05-10T10:38:57.239432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:37:46.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232029+0000","last_change":"2024-05-10T10:38:57.232029+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232029+0000","last_undegraded":"2024-05-10T10:38:57.232029+0000","last_fullsized":"2024-05-10T10:38:57.232029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243287+0000","last_change":"2024-05-10T10:38:57.241714+0000","last_active":"2024-05-10T10:38:57.243287+0000","last_peered":"2024-05-10T10:38:57.243287+0000","last_clean":"2024-05-10T10:38:57.243287+0000","last_became_active":"2024-05-10T10:38:57.241428+0000","last_became_peered":"2024-05-10T10:38:57.241428+0000","last_unstale":"2024-05-10T10:38:57.243287+0000","last_undegraded":"2024-05-10T10:38:57.243287+0000","last_fullsized":"2024-05-10T10:38:57.243287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:08.883771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234030+0000","last_change":"2024-05-10T10:38:57.234030+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234030+0000","last_undegraded":"2024-05-10T10:38:57.234030+0000","last_fullsized":"2024-05-10T10:38:57.234030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241462+0000","last_change":"2024-05-10T10:38:57.239328+0000","last_active":"2024-05-10T10:38:57.241462+0000","last_peered":"2024-05-10T10:38:57.241462+0000","last_clean":"2024-05-10T10:38:57.241462+0000","last_became_active":"2024-05-10T10:38:57.239042+0000","last_became_peered":"2024-05-10T10:38:57.239042+0000","last_unstale":"2024-05-10T10:38:57.241462+0000","last_undegraded":"2024-05-10T10:38:57.241462+0000","last_fullsized":"2024-05-10T10:38:57.241462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231035+0000","last_change":"2024-05-10T10:38:57.231035+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231035+0000","last_undegraded":"2024-05-10T10:38:57.231035+0000","last_fullsized":"2024-05-10T10:38:57.231035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243561+0000","last_change":"2024-05-10T10:38:57.242120+0000","last_active":"2024-05-10T10:38:57.243561+0000","last_peered":"2024-05-10T10:38:57.243561+0000","last_clean":"2024-05-10T10:38:57.243561+0000","last_became_active":"2024-05-10T10:38:57.241862+0000","last_became_peered":"2024-05-10T10:38:57.241862+0000","last_unstale":"2024-05-10T10:38:57.243561+0000","last_undegraded":"2024-05-10T10:38:57.243561+0000","last_fullsized":"2024-05-10T10:38:57.243561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:32.971721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230683+0000","last_change":"2024-05-10T10:38:57.230683+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230683+0000","last_undegraded":"2024-05-10T10:38:57.230683+0000","last_fullsized":"2024-05-10T10:38:57.230683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247818+0000","last_change":"2024-05-10T10:38:57.246576+0000","last_active":"2024-05-10T10:38:57.247818+0000","last_peered":"2024-05-10T10:38:57.247818+0000","last_clean":"2024-05-10T10:38:57.247818+0000","last_became_active":"2024-05-10T10:38:57.246353+0000","last_became_peered":"2024-05-10T10:38:57.246353+0000","last_unstale":"2024-05-10T10:38:57.247818+0000","last_undegraded":"2024-05-10T10:38:57.247818+0000","last_fullsized":"2024-05-10T10:38:57.247818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:14.665243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227984+0000","last_change":"2024-05-10T10:38:57.227984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227984+0000","last_undegraded":"2024-05-10T10:38:57.227984+0000","last_fullsized":"2024-05-10T10:38:57.227984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245546+0000","last_change":"2024-05-10T10:38:57.243728+0000","last_active":"2024-05-10T10:38:57.245546+0000","last_peered":"2024-05-10T10:38:57.245546+0000","last_clean":"2024-05-10T10:38:57.245546+0000","last_became_active":"2024-05-10T10:38:57.243406+0000","last_became_peered":"2024-05-10T10:38:57.243406+0000","last_unstale":"2024-05-10T10:38:57.245546+0000","last_undegraded":"2024-05-10T10:38:57.245546+0000","last_fullsized":"2024-05-10T10:38:57.245546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232998+0000","last_change":"2024-05-10T10:38:57.232998+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232998+0000","last_undegraded":"2024-05-10T10:38:57.232998+0000","last_fullsized":"2024-05-10T10:38:57.232998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.248051+0000","last_change":"2024-05-10T10:38:57.244580+0000","last_active":"2024-05-10T10:38:57.248051+0000","last_peered":"2024-05-10T10:38:57.248051+0000","last_clean":"2024-05-10T10:38:57.248051+0000","last_became_active":"2024-05-10T10:38:57.244271+0000","last_became_peered":"2024-05-10T10:38:57.244271+0000","last_unstale":"2024-05-10T10:38:57.248051+0000","last_undegraded":"2024-05-10T10:38:57.248051+0000","last_fullsized":"2024-05-10T10:38:57.248051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231123+0000","last_change":"2024-05-10T10:38:57.231123+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231123+0000","last_undegraded":"2024-05-10T10:38:57.231123+0000","last_fullsized":"2024-05-10T10:38:57.231123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240531+0000","last_change":"2024-05-10T10:38:57.236911+0000","last_active":"2024-05-10T10:38:57.240531+0000","last_peered":"2024-05-10T10:38:57.240531+0000","last_clean":"2024-05-10T10:38:57.240531+0000","last_became_active":"2024-05-10T10:38:57.236571+0000","last_became_peered":"2024-05-10T10:38:57.236571+0000","last_unstale":"2024-05-10T10:38:57.240531+0000","last_undegraded":"2024-05-10T10:38:57.240531+0000","last_fullsized":"2024-05-10T10:38:57.240531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:14.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233921+0000","last_change":"2024-05-10T10:38:57.233921+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233921+0000","last_undegraded":"2024-05-10T10:38:57.233921+0000","last_fullsized":"2024-05-10T10:38:57.233921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244349+0000","last_change":"2024-05-10T10:38:57.243643+0000","last_active":"2024-05-10T10:38:57.244349+0000","last_peered":"2024-05-10T10:38:57.244349+0000","last_clean":"2024-05-10T10:38:57.244349+0000","last_became_active":"2024-05-10T10:38:57.243364+0000","last_became_peered":"2024-05-10T10:38:57.243364+0000","last_unstale":"2024-05-10T10:38:57.244349+0000","last_undegraded":"2024-05-10T10:38:57.244349+0000","last_fullsized":"2024-05-10T10:38:57.244349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230113+0000","last_change":"2024-05-10T10:38:57.230113+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230113+0000","last_undegraded":"2024-05-10T10:38:57.230113+0000","last_fullsized":"2024-05-10T10:38:57.230113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241495+0000","last_change":"2024-05-10T10:38:57.239379+0000","last_active":"2024-05-10T10:38:57.241495+0000","last_peered":"2024-05-10T10:38:57.241495+0000","last_clean":"2024-05-10T10:38:57.241495+0000","last_became_active":"2024-05-10T10:38:57.239057+0000","last_became_peered":"2024-05-10T10:38:57.239057+0000","last_unstale":"2024-05-10T10:38:57.241495+0000","last_undegraded":"2024-05-10T10:38:57.241495+0000","last_fullsized":"2024-05-10T10:38:57.241495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228473+0000","last_change":"2024-05-10T10:38:57.228473+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228473+0000","last_undegraded":"2024-05-10T10:38:57.228473+0000","last_fullsized":"2024-05-10T10:38:57.228473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244095+0000","last_change":"2024-05-10T10:38:57.242780+0000","last_active":"2024-05-10T10:38:57.244095+0000","last_peered":"2024-05-10T10:38:57.244095+0000","last_clean":"2024-05-10T10:38:57.244095+0000","last_became_active":"2024-05-10T10:38:57.242494+0000","last_became_peered":"2024-05-10T10:38:57.242494+0000","last_unstale":"2024-05-10T10:38:57.244095+0000","last_undegraded":"2024-05-10T10:38:57.244095+0000","last_fullsized":"2024-05-10T10:38:57.244095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":242,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.228637+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:38:57.228637+0000","last_peered":"2024-05-10T10:38:57.228637+0000","last_clean":"2024-05-10T10:38:57.228637+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:38:57.228637+0000","last_undegraded":"2024-05-10T10:38:57.228637+0000","last_fullsized":"2024-05-10T10:38:57.228637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:39:02.666 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:39:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.805+0000 7fee11319640 1 Processor -- start 2024-05-10T10:39:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee11319640 1 -- start start 2024-05-10T10:39:02.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee11319640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fee0c15f290 0x7fee0c0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee11319640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0702e0 0x7fee0c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee11319640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fee0c0d3280 0x7fee0c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee11319640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c0d14c0 con 0x7fee0c15f290 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee11319640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c0d2300 con 0x7fee0c0d3280 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee11319640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c06f3c0 con 0x7fee0c0702e0 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee0affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fee0c15f290 0x7fee0c0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee0affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fee0c15f290 0x7fee0c0d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60490/0 (socket says 172.21.15.53:60490) 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.806+0000 7fee0affd640 1 -- 172.21.15.53:0/3931490708 learned_addr learned my addr 172.21.15.53:0/3931490708 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0a7fc640 1 --2- 172.21.15.53:0/3931490708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0702e0 0x7fee0c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0b7fe640 1 --2- 172.21.15.53:0/3931490708 >> v2:172.21.15.135:3300/0 conn(0x7fee0c0d3280 0x7fee0c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0affd640 1 -- 172.21.15.53:0/3931490708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0702e0 msgr2=0x7fee0c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0affd640 1 --2- 172.21.15.53:0/3931490708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0702e0 0x7fee0c0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0affd640 1 -- 172.21.15.53:0/3931490708 >> v2:172.21.15.135:3300/0 conn(0x7fee0c0d3280 msgr2=0x7fee0c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0affd640 1 --2- 172.21.15.53:0/3931490708 >> v2:172.21.15.135:3300/0 conn(0x7fee0c0d3280 0x7fee0c0cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0affd640 1 -- 172.21.15.53:0/3931490708 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c0d16f0 con 0x7fee0c15f290 2024-05-10T10:39:02.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0b7fe640 1 --2- 172.21.15.53:0/3931490708 >> v2:172.21.15.135:3300/0 conn(0x7fee0c0d3280 0x7fee0c0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:02.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0a7fc640 1 --2- 172.21.15.53:0/3931490708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0702e0 0x7fee0c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.807+0000 7fee0affd640 1 --2- 172.21.15.53:0/3931490708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0d0f80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fee0c06d120 tx=0x7fee0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85487a6c9d9b3d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee09ffb640 1 -- 172.21.15.53:0/3931490708 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04037070 con 0x7fee0c15f290 2024-05-10T10:39:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee09ffb640 1 -- 172.21.15.53:0/3931490708 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee04032440 con 0x7fee0c15f290 2024-05-10T10:39:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee09ffb640 1 -- 172.21.15.53:0/3931490708 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04032810 con 0x7fee0c15f290 2024-05-10T10:39:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 -- 172.21.15.53:0/3931490708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 msgr2=0x7fee0c0d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 --2- 172.21.15.53:0/3931490708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0d0f80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fee0c06d120 tx=0x7fee0402a270 comp rx=0 tx=0).stop 2024-05-10T10:39:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 -- 172.21.15.53:0/3931490708 shutdown_connections 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 --2- 172.21.15.53:0/3931490708 >> v2:172.21.15.135:3300/0 conn(0x7fee0c0d3280 0x7fee0c0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 --2- 172.21.15.53:0/3931490708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0702e0 0x7fee0c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 --2- 172.21.15.53:0/3931490708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0d0f80 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 -- 172.21.15.53:0/3931490708 >> 172.21.15.53:0/3931490708 conn(0x7fee0c09ba20 msgr2=0x7fee0c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 -- 172.21.15.53:0/3931490708 shutdown_connections 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.808+0000 7fee11319640 1 -- 172.21.15.53:0/3931490708 wait complete. 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 Processor -- start 2024-05-10T10:39:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 -- start start 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee0c0702e0 0x7fee0c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0d3280 0x7fee0c0b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0d1f90 con 0x7fee0c15f290 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0d1cb0 con 0x7fee0c0702e0 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee11319640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c15f670 con 0x7fee0c0d3280 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.810+0000 7fee0b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60498/0 (socket says 172.21.15.53:60498) 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 -- 172.21.15.53:0/2368025945 learned_addr learned my addr 172.21.15.53:0/2368025945 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0affd640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee0c0702e0 0x7fee0c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0a7fc640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0d3280 0x7fee0c0b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 -- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0d3280 msgr2=0x7fee0c0b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0d3280 0x7fee0c0b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 -- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee0c0702e0 msgr2=0x7fee0c14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee0c0702e0 0x7fee0c14e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 -- 172.21.15.53:0/2368025945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c1f4c10 con 0x7fee0c15f290 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee0b7fe640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0b6080 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00cef0 tx=0x7fedfc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee037fe640 1 -- 172.21.15.53:0/2368025945 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc01a070 con 0x7fee0c15f290 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee037fe640 1 -- 172.21.15.53:0/2368025945 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedfc00d860 con 0x7fee0c15f290 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee037fe640 1 -- 172.21.15.53:0/2368025945 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00dc50 con 0x7fee0c15f290 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c1f8f70 con 0x7fee0c15f290 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.811+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1f51b0 con 0x7fee0c15f290 2024-05-10T10:39:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.812+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c0693f0 con 0x7fee0c15f290 2024-05-10T10:39:02.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.813+0000 7fee037fe640 1 -- 172.21.15.53:0/2368025945 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fedfc01d070 con 0x7fee0c15f290 2024-05-10T10:39:02.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.813+0000 7fee037fe640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fedec0762d0 0x7fedec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:02.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.814+0000 7fee037fe640 1 -- 172.21.15.53:0/2368025945 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fedfc097f40 con 0x7fee0c15f290 2024-05-10T10:39:02.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.814+0000 7fee0affd640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fedec0762d0 0x7fedec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:02.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.814+0000 7fee0affd640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fedec0762d0 0x7fedec078790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee04034000 tx=0x7fee0402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:02.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:02.818+0000 7fee037fe640 1 -- 172.21.15.53:0/2368025945 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fedfc061840 con 0x7fee0c15f290 2024-05-10T10:39:03.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.008+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee0c075220 con 0x7fedec0762d0 2024-05-10T10:39:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.015+0000 7fee037fe640 1 -- 172.21.15.53:0/2368025945 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246754 (secure 0 0 0) 0x7fee0c075220 con 0x7fedec0762d0 2024-05-10T10:39:03.019 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:03.020 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:39:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fedec0762d0 msgr2=0x7fedec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fedec0762d0 0x7fedec078790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee04034000 tx=0x7fee0402e000 comp rx=0 tx=0).stop 2024-05-10T10:39:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 msgr2=0x7fee0c0b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0b6080 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00cef0 tx=0x7fedfc00a710 comp rx=0 tx=0).stop 2024-05-10T10:39:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 shutdown_connections 2024-05-10T10:39:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fedec0762d0 0x7fedec078790 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee0c15f290 0x7fee0c0b6080 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fee0c0d3280 0x7fee0c0b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 --2- 172.21.15.53:0/2368025945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee0c0702e0 0x7fee0c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 >> 172.21.15.53:0/2368025945 conn(0x7fee0c09ba20 msgr2=0x7fee0c0d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 shutdown_connections 2024-05-10T10:39:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.018+0000 7fee11319640 1 -- 172.21.15.53:0/2368025945 wait complete. 2024-05-10T10:39:03.030 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-05-10T10:39:02.235941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254640,"kb_used_data":9352,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720080,"statfs":{"total":773094113280,"available":772833361920,"internally_reserved":0,"allocated":9576448,"data_stored":4992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232129+0000","last_change":"2024-05-10T10:38:57.232129+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232129+0000","last_undegraded":"2024-05-10T10:38:57.232129+0000","last_fullsized":"2024-05-10T10:38:57.232129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227450+0000","last_change":"2024-05-10T10:38:57.227450+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227450+0000","last_undegraded":"2024-05-10T10:38:57.227450+0000","last_fullsized":"2024-05-10T10:38:57.227450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232404+0000","last_change":"2024-05-10T10:38:57.232404+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232404+0000","last_undegraded":"2024-05-10T10:38:57.232404+0000","last_fullsized":"2024-05-10T10:38:57.232404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.244373+0000","last_change":"2024-05-10T10:38:57.244373+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.244373+0000","last_undegraded":"2024-05-10T10:38:57.244373+0000","last_fullsized":"2024-05-10T10:38:57.244373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235333+0000","last_change":"2024-05-10T10:38:57.235333+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235333+0000","last_undegraded":"2024-05-10T10:38:57.235333+0000","last_fullsized":"2024-05-10T10:38:57.235333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.236760+0000","last_change":"2024-05-10T10:38:57.236760+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.236760+0000","last_undegraded":"2024-05-10T10:38:57.236760+0000","last_fullsized":"2024-05-10T10:38:57.236760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227454+0000","last_change":"2024-05-10T10:38:57.227454+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227454+0000","last_undegraded":"2024-05-10T10:38:57.227454+0000","last_fullsized":"2024-05-10T10:38:57.227454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243738+0000","last_change":"2024-05-10T10:38:57.243738+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243738+0000","last_undegraded":"2024-05-10T10:38:57.243738+0000","last_fullsized":"2024-05-10T10:38:57.243738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234548+0000","last_change":"2024-05-10T10:38:57.234548+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234548+0000","last_undegraded":"2024-05-10T10:38:57.234548+0000","last_fullsized":"2024-05-10T10:38:57.234548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234335+0000","last_change":"2024-05-10T10:38:57.234335+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234335+0000","last_undegraded":"2024-05-10T10:38:57.234335+0000","last_fullsized":"2024-05-10T10:38:57.234335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234094+0000","last_change":"2024-05-10T10:38:57.234094+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234094+0000","last_undegraded":"2024-05-10T10:38:57.234094+0000","last_fullsized":"2024-05-10T10:38:57.234094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234550+0000","last_change":"2024-05-10T10:38:57.234550+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234550+0000","last_undegraded":"2024-05-10T10:38:57.234550+0000","last_fullsized":"2024-05-10T10:38:57.234550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243276+0000","last_change":"2024-05-10T10:38:57.243276+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243276+0000","last_undegraded":"2024-05-10T10:38:57.243276+0000","last_fullsized":"2024-05-10T10:38:57.243276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233211+0000","last_change":"2024-05-10T10:38:57.233211+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233211+0000","last_undegraded":"2024-05-10T10:38:57.233211+0000","last_fullsized":"2024-05-10T10:38:57.233211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239867+0000","last_change":"2024-05-10T10:38:57.239867+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239867+0000","last_undegraded":"2024-05-10T10:38:57.239867+0000","last_fullsized":"2024-05-10T10:38:57.239867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235604+0000","last_change":"2024-05-10T10:38:57.235604+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235604+0000","last_undegraded":"2024-05-10T10:38:57.235604+0000","last_fullsized":"2024-05-10T10:38:57.235604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235059+0000","last_change":"2024-05-10T10:38:57.235059+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235059+0000","last_undegraded":"2024-05-10T10:38:57.235059+0000","last_fullsized":"2024-05-10T10:38:57.235059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231848+0000","last_change":"2024-05-10T10:38:57.231848+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231848+0000","last_undegraded":"2024-05-10T10:38:57.231848+0000","last_fullsized":"2024-05-10T10:38:57.231848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233571+0000","last_change":"2024-05-10T10:38:57.233571+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233571+0000","last_undegraded":"2024-05-10T10:38:57.233571+0000","last_fullsized":"2024-05-10T10:38:57.233571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240887+0000","last_change":"2024-05-10T10:38:57.240887+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240887+0000","last_undegraded":"2024-05-10T10:38:57.240887+0000","last_fullsized":"2024-05-10T10:38:57.240887+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232634+0000","last_change":"2024-05-10T10:38:57.232634+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232634+0000","last_undegraded":"2024-05-10T10:38:57.232634+0000","last_fullsized":"2024-05-10T10:38:57.232634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232606+0000","last_change":"2024-05-10T10:38:57.232606+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232606+0000","last_undegraded":"2024-05-10T10:38:57.232606+0000","last_fullsized":"2024-05-10T10:38:57.232606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235045+0000","last_change":"2024-05-10T10:38:57.235045+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235045+0000","last_undegraded":"2024-05-10T10:38:57.235045+0000","last_fullsized":"2024-05-10T10:38:57.235045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.229984+0000","last_change":"2024-05-10T10:38:57.229984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.229984+0000","last_undegraded":"2024-05-10T10:38:57.229984+0000","last_fullsized":"2024-05-10T10:38:57.229984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228435+0000","last_change":"2024-05-10T10:38:57.228435+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228435+0000","last_undegraded":"2024-05-10T10:38:57.228435+0000","last_fullsized":"2024-05-10T10:38:57.228435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233929+0000","last_change":"2024-05-10T10:38:57.233929+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233929+0000","last_undegraded":"2024-05-10T10:38:57.233929+0000","last_fullsized":"2024-05-10T10:38:57.233929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233882+0000","last_change":"2024-05-10T10:38:57.233882+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233882+0000","last_undegraded":"2024-05-10T10:38:57.233882+0000","last_fullsized":"2024-05-10T10:38:57.233882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240407+0000","last_change":"2024-05-10T10:38:57.240407+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240407+0000","last_undegraded":"2024-05-10T10:38:57.240407+0000","last_fullsized":"2024-05-10T10:38:57.240407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228010+0000","last_change":"2024-05-10T10:38:57.228010+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228010+0000","last_undegraded":"2024-05-10T10:38:57.228010+0000","last_fullsized":"2024-05-10T10:38:57.228010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232153+0000","last_change":"2024-05-10T10:38:57.232153+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232153+0000","last_undegraded":"2024-05-10T10:38:57.232153+0000","last_fullsized":"2024-05-10T10:38:57.232153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228694+0000","last_change":"2024-05-10T10:38:57.228694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228694+0000","last_undegraded":"2024-05-10T10:38:57.228694+0000","last_fullsized":"2024-05-10T10:38:57.228694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234543+0000","last_change":"2024-05-10T10:38:57.234543+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234543+0000","last_undegraded":"2024-05-10T10:38:57.234543+0000","last_fullsized":"2024-05-10T10:38:57.234543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233533+0000","last_change":"2024-05-10T10:38:57.233533+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233533+0000","last_undegraded":"2024-05-10T10:38:57.233533+0000","last_fullsized":"2024-05-10T10:38:57.233533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245963+0000","last_change":"2024-05-10T10:38:57.244889+0000","last_active":"2024-05-10T10:38:57.245963+0000","last_peered":"2024-05-10T10:38:57.245963+0000","last_clean":"2024-05-10T10:38:57.245963+0000","last_became_active":"2024-05-10T10:38:57.244439+0000","last_became_peered":"2024-05-10T10:38:57.244439+0000","last_unstale":"2024-05-10T10:38:57.245963+0000","last_undegraded":"2024-05-10T10:38:57.245963+0000","last_fullsized":"2024-05-10T10:38:57.245963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234020+0000","last_change":"2024-05-10T10:38:57.234020+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234020+0000","last_undegraded":"2024-05-10T10:38:57.234020+0000","last_fullsized":"2024-05-10T10:38:57.234020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243113+0000","last_change":"2024-05-10T10:38:57.241164+0000","last_active":"2024-05-10T10:38:57.243113+0000","last_peered":"2024-05-10T10:38:57.243113+0000","last_clean":"2024-05-10T10:38:57.243113+0000","last_became_active":"2024-05-10T10:38:57.240896+0000","last_became_peered":"2024-05-10T10:38:57.240896+0000","last_unstale":"2024-05-10T10:38:57.243113+0000","last_undegraded":"2024-05-10T10:38:57.243113+0000","last_fullsized":"2024-05-10T10:38:57.243113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:16.909583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227680+0000","last_change":"2024-05-10T10:38:57.227680+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227680+0000","last_undegraded":"2024-05-10T10:38:57.227680+0000","last_fullsized":"2024-05-10T10:38:57.227680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241017+0000","last_change":"2024-05-10T10:38:57.238335+0000","last_active":"2024-05-10T10:38:57.241017+0000","last_peered":"2024-05-10T10:38:57.241017+0000","last_clean":"2024-05-10T10:38:57.241017+0000","last_became_active":"2024-05-10T10:38:57.238013+0000","last_became_peered":"2024-05-10T10:38:57.238013+0000","last_unstale":"2024-05-10T10:38:57.241017+0000","last_undegraded":"2024-05-10T10:38:57.241017+0000","last_fullsized":"2024-05-10T10:38:57.241017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234527+0000","last_change":"2024-05-10T10:38:57.234527+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234527+0000","last_undegraded":"2024-05-10T10:38:57.234527+0000","last_fullsized":"2024-05-10T10:38:57.234527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245980+0000","last_change":"2024-05-10T10:38:57.245097+0000","last_active":"2024-05-10T10:38:57.245980+0000","last_peered":"2024-05-10T10:38:57.245980+0000","last_clean":"2024-05-10T10:38:57.245980+0000","last_became_active":"2024-05-10T10:38:57.244782+0000","last_became_peered":"2024-05-10T10:38:57.244782+0000","last_unstale":"2024-05-10T10:38:57.245980+0000","last_undegraded":"2024-05-10T10:38:57.245980+0000","last_fullsized":"2024-05-10T10:38:57.245980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:57.598796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239613+0000","last_change":"2024-05-10T10:38:57.239613+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239613+0000","last_undegraded":"2024-05-10T10:38:57.239613+0000","last_fullsized":"2024-05-10T10:38:57.239613+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251314+0000","last_change":"2024-05-10T10:38:57.249546+0000","last_active":"2024-05-10T10:38:57.251314+0000","last_peered":"2024-05-10T10:38:57.251314+0000","last_clean":"2024-05-10T10:38:57.251314+0000","last_became_active":"2024-05-10T10:38:57.249316+0000","last_became_peered":"2024-05-10T10:38:57.249316+0000","last_unstale":"2024-05-10T10:38:57.251314+0000","last_undegraded":"2024-05-10T10:38:57.251314+0000","last_fullsized":"2024-05-10T10:38:57.251314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:22.781845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232578+0000","last_change":"2024-05-10T10:38:57.232578+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232578+0000","last_undegraded":"2024-05-10T10:38:57.232578+0000","last_fullsized":"2024-05-10T10:38:57.232578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251627+0000","last_change":"2024-05-10T10:38:57.250917+0000","last_active":"2024-05-10T10:38:57.251627+0000","last_peered":"2024-05-10T10:38:57.251627+0000","last_clean":"2024-05-10T10:38:57.251627+0000","last_became_active":"2024-05-10T10:38:57.250631+0000","last_became_peered":"2024-05-10T10:38:57.250631+0000","last_unstale":"2024-05-10T10:38:57.251627+0000","last_undegraded":"2024-05-10T10:38:57.251627+0000","last_fullsized":"2024-05-10T10:38:57.251627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233065+0000","last_change":"2024-05-10T10:38:57.233065+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233065+0000","last_undegraded":"2024-05-10T10:38:57.233065+0000","last_fullsized":"2024-05-10T10:38:57.233065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235166+0000","last_change":"2024-05-10T10:38:57.234005+0000","last_active":"2024-05-10T10:38:57.235166+0000","last_peered":"2024-05-10T10:38:57.235166+0000","last_clean":"2024-05-10T10:38:57.235166+0000","last_became_active":"2024-05-10T10:38:57.233624+0000","last_became_peered":"2024-05-10T10:38:57.233624+0000","last_unstale":"2024-05-10T10:38:57.235166+0000","last_undegraded":"2024-05-10T10:38:57.235166+0000","last_fullsized":"2024-05-10T10:38:57.235166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.241217+0000","last_change":"2024-05-10T10:38:57.241217+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.241217+0000","last_undegraded":"2024-05-10T10:38:57.241217+0000","last_fullsized":"2024-05-10T10:38:57.241217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245553+0000","last_change":"2024-05-10T10:38:57.243368+0000","last_active":"2024-05-10T10:38:57.245553+0000","last_peered":"2024-05-10T10:38:57.245553+0000","last_clean":"2024-05-10T10:38:57.245553+0000","last_became_active":"2024-05-10T10:38:57.243101+0000","last_became_peered":"2024-05-10T10:38:57.243101+0000","last_unstale":"2024-05-10T10:38:57.245553+0000","last_undegraded":"2024-05-10T10:38:57.245553+0000","last_fullsized":"2024-05-10T10:38:57.245553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231694+0000","last_change":"2024-05-10T10:38:57.231694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231694+0000","last_undegraded":"2024-05-10T10:38:57.231694+0000","last_fullsized":"2024-05-10T10:38:57.231694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245621+0000","last_change":"2024-05-10T10:38:57.244172+0000","last_active":"2024-05-10T10:38:57.245621+0000","last_peered":"2024-05-10T10:38:57.245621+0000","last_clean":"2024-05-10T10:38:57.245621+0000","last_became_active":"2024-05-10T10:38:57.243827+0000","last_became_peered":"2024-05-10T10:38:57.243827+0000","last_unstale":"2024-05-10T10:38:57.245621+0000","last_undegraded":"2024-05-10T10:38:57.245621+0000","last_fullsized":"2024-05-10T10:38:57.245621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:14.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.226934+0000","last_change":"2024-05-10T10:38:57.226934+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.226934+0000","last_undegraded":"2024-05-10T10:38:57.226934+0000","last_fullsized":"2024-05-10T10:38:57.226934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243252+0000","last_change":"2024-05-10T10:38:57.242608+0000","last_active":"2024-05-10T10:38:57.243252+0000","last_peered":"2024-05-10T10:38:57.243252+0000","last_clean":"2024-05-10T10:38:57.243252+0000","last_became_active":"2024-05-10T10:38:57.242307+0000","last_became_peered":"2024-05-10T10:38:57.242307+0000","last_unstale":"2024-05-10T10:38:57.243252+0000","last_undegraded":"2024-05-10T10:38:57.243252+0000","last_fullsized":"2024-05-10T10:38:57.243252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230903+0000","last_change":"2024-05-10T10:38:57.230903+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230903+0000","last_undegraded":"2024-05-10T10:38:57.230903+0000","last_fullsized":"2024-05-10T10:38:57.230903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247838+0000","last_change":"2024-05-10T10:38:57.246632+0000","last_active":"2024-05-10T10:38:57.247838+0000","last_peered":"2024-05-10T10:38:57.247838+0000","last_clean":"2024-05-10T10:38:57.247838+0000","last_became_active":"2024-05-10T10:38:57.246417+0000","last_became_peered":"2024-05-10T10:38:57.246417+0000","last_unstale":"2024-05-10T10:38:57.247838+0000","last_undegraded":"2024-05-10T10:38:57.247838+0000","last_fullsized":"2024-05-10T10:38:57.247838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:12.230139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239277+0000","last_change":"2024-05-10T10:38:57.239277+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239277+0000","last_undegraded":"2024-05-10T10:38:57.239277+0000","last_fullsized":"2024-05-10T10:38:57.239277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244891+0000","last_change":"2024-05-10T10:38:57.238700+0000","last_active":"2024-05-10T10:38:57.244891+0000","last_peered":"2024-05-10T10:38:57.244891+0000","last_clean":"2024-05-10T10:38:57.244891+0000","last_became_active":"2024-05-10T10:38:57.238416+0000","last_became_peered":"2024-05-10T10:38:57.238416+0000","last_unstale":"2024-05-10T10:38:57.244891+0000","last_undegraded":"2024-05-10T10:38:57.244891+0000","last_fullsized":"2024-05-10T10:38:57.244891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231354+0000","last_change":"2024-05-10T10:38:57.231354+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231354+0000","last_undegraded":"2024-05-10T10:38:57.231354+0000","last_fullsized":"2024-05-10T10:38:57.231354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235157+0000","last_change":"2024-05-10T10:38:57.233569+0000","last_active":"2024-05-10T10:38:57.235157+0000","last_peered":"2024-05-10T10:38:57.235157+0000","last_clean":"2024-05-10T10:38:57.235157+0000","last_became_active":"2024-05-10T10:38:57.233304+0000","last_became_peered":"2024-05-10T10:38:57.233304+0000","last_unstale":"2024-05-10T10:38:57.235157+0000","last_undegraded":"2024-05-10T10:38:57.235157+0000","last_fullsized":"2024-05-10T10:38:57.235157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:19.208256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231109+0000","last_change":"2024-05-10T10:38:57.231109+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231109+0000","last_undegraded":"2024-05-10T10:38:57.231109+0000","last_fullsized":"2024-05-10T10:38:57.231109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244857+0000","last_change":"2024-05-10T10:38:57.242521+0000","last_active":"2024-05-10T10:38:57.244857+0000","last_peered":"2024-05-10T10:38:57.244857+0000","last_clean":"2024-05-10T10:38:57.244857+0000","last_became_active":"2024-05-10T10:38:57.242195+0000","last_became_peered":"2024-05-10T10:38:57.242195+0000","last_unstale":"2024-05-10T10:38:57.244857+0000","last_undegraded":"2024-05-10T10:38:57.244857+0000","last_fullsized":"2024-05-10T10:38:57.244857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232630+0000","last_change":"2024-05-10T10:38:57.232630+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232630+0000","last_undegraded":"2024-05-10T10:38:57.232630+0000","last_fullsized":"2024-05-10T10:38:57.232630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244065+0000","last_change":"2024-05-10T10:38:57.240790+0000","last_active":"2024-05-10T10:38:57.244065+0000","last_peered":"2024-05-10T10:38:57.244065+0000","last_clean":"2024-05-10T10:38:57.244065+0000","last_became_active":"2024-05-10T10:38:57.240455+0000","last_became_peered":"2024-05-10T10:38:57.240455+0000","last_unstale":"2024-05-10T10:38:57.244065+0000","last_undegraded":"2024-05-10T10:38:57.244065+0000","last_fullsized":"2024-05-10T10:38:57.244065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234758+0000","last_change":"2024-05-10T10:38:57.234758+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234758+0000","last_undegraded":"2024-05-10T10:38:57.234758+0000","last_fullsized":"2024-05-10T10:38:57.234758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.250429+0000","last_change":"2024-05-10T10:38:57.249734+0000","last_active":"2024-05-10T10:38:57.250429+0000","last_peered":"2024-05-10T10:38:57.250429+0000","last_clean":"2024-05-10T10:38:57.250429+0000","last_became_active":"2024-05-10T10:38:57.249452+0000","last_became_peered":"2024-05-10T10:38:57.249452+0000","last_unstale":"2024-05-10T10:38:57.250429+0000","last_undegraded":"2024-05-10T10:38:57.250429+0000","last_fullsized":"2024-05-10T10:38:57.250429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232904+0000","last_change":"2024-05-10T10:38:57.232904+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232904+0000","last_undegraded":"2024-05-10T10:38:57.232904+0000","last_fullsized":"2024-05-10T10:38:57.232904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242899+0000","last_change":"2024-05-10T10:38:57.241984+0000","last_active":"2024-05-10T10:38:57.242899+0000","last_peered":"2024-05-10T10:38:57.242899+0000","last_clean":"2024-05-10T10:38:57.242899+0000","last_became_active":"2024-05-10T10:38:57.241709+0000","last_became_peered":"2024-05-10T10:38:57.241709+0000","last_unstale":"2024-05-10T10:38:57.242899+0000","last_undegraded":"2024-05-10T10:38:57.242899+0000","last_fullsized":"2024-05-10T10:38:57.242899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234942+0000","last_change":"2024-05-10T10:38:57.234942+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234942+0000","last_undegraded":"2024-05-10T10:38:57.234942+0000","last_fullsized":"2024-05-10T10:38:57.234942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242330+0000","last_change":"2024-05-10T10:38:57.240601+0000","last_active":"2024-05-10T10:38:57.242330+0000","last_peered":"2024-05-10T10:38:57.242330+0000","last_clean":"2024-05-10T10:38:57.242330+0000","last_became_active":"2024-05-10T10:38:57.240018+0000","last_became_peered":"2024-05-10T10:38:57.240018+0000","last_unstale":"2024-05-10T10:38:57.242330+0000","last_undegraded":"2024-05-10T10:38:57.242330+0000","last_fullsized":"2024-05-10T10:38:57.242330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233300+0000","last_change":"2024-05-10T10:38:57.233300+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233300+0000","last_undegraded":"2024-05-10T10:38:57.233300+0000","last_fullsized":"2024-05-10T10:38:57.233300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249361+0000","last_change":"2024-05-10T10:38:57.248178+0000","last_active":"2024-05-10T10:38:57.249361+0000","last_peered":"2024-05-10T10:38:57.249361+0000","last_clean":"2024-05-10T10:38:57.249361+0000","last_became_active":"2024-05-10T10:38:57.247953+0000","last_became_peered":"2024-05-10T10:38:57.247953+0000","last_unstale":"2024-05-10T10:38:57.249361+0000","last_undegraded":"2024-05-10T10:38:57.249361+0000","last_fullsized":"2024-05-10T10:38:57.249361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:30.018605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231497+0000","last_change":"2024-05-10T10:38:57.231497+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231497+0000","last_undegraded":"2024-05-10T10:38:57.231497+0000","last_fullsized":"2024-05-10T10:38:57.231497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240504+0000","last_change":"2024-05-10T10:38:57.235875+0000","last_active":"2024-05-10T10:38:57.240504+0000","last_peered":"2024-05-10T10:38:57.240504+0000","last_clean":"2024-05-10T10:38:57.240504+0000","last_became_active":"2024-05-10T10:38:57.235601+0000","last_became_peered":"2024-05-10T10:38:57.235601+0000","last_unstale":"2024-05-10T10:38:57.240504+0000","last_undegraded":"2024-05-10T10:38:57.240504+0000","last_fullsized":"2024-05-10T10:38:57.240504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:28:51.797568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227049+0000","last_change":"2024-05-10T10:38:57.227049+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227049+0000","last_undegraded":"2024-05-10T10:38:57.227049+0000","last_fullsized":"2024-05-10T10:38:57.227049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249243+0000","last_change":"2024-05-10T10:38:57.248738+0000","last_active":"2024-05-10T10:38:57.249243+0000","last_peered":"2024-05-10T10:38:57.249243+0000","last_clean":"2024-05-10T10:38:57.249243+0000","last_became_active":"2024-05-10T10:38:57.248459+0000","last_became_peered":"2024-05-10T10:38:57.248459+0000","last_unstale":"2024-05-10T10:38:57.249243+0000","last_undegraded":"2024-05-10T10:38:57.249243+0000","last_fullsized":"2024-05-10T10:38:57.249243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:40:18.359499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227165+0000","last_change":"2024-05-10T10:38:57.227165+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227165+0000","last_undegraded":"2024-05-10T10:38:57.227165+0000","last_fullsized":"2024-05-10T10:38:57.227165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.239432+0000","last_change":"2024-05-10T10:38:57.230406+0000","last_active":"2024-05-10T10:38:57.239432+0000","last_peered":"2024-05-10T10:38:57.239432+0000","last_clean":"2024-05-10T10:38:57.239432+0000","last_became_active":"2024-05-10T10:38:57.230065+0000","last_became_peered":"2024-05-10T10:38:57.230065+0000","last_unstale":"2024-05-10T10:38:57.239432+0000","last_undegraded":"2024-05-10T10:38:57.239432+0000","last_fullsized":"2024-05-10T10:38:57.239432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:37:46.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232029+0000","last_change":"2024-05-10T10:38:57.232029+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232029+0000","last_undegraded":"2024-05-10T10:38:57.232029+0000","last_fullsized":"2024-05-10T10:38:57.232029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243287+0000","last_change":"2024-05-10T10:38:57.241714+0000","last_active":"2024-05-10T10:38:57.243287+0000","last_peered":"2024-05-10T10:38:57.243287+0000","last_clean":"2024-05-10T10:38:57.243287+0000","last_became_active":"2024-05-10T10:38:57.241428+0000","last_became_peered":"2024-05-10T10:38:57.241428+0000","last_unstale":"2024-05-10T10:38:57.243287+0000","last_undegraded":"2024-05-10T10:38:57.243287+0000","last_fullsized":"2024-05-10T10:38:57.243287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:08.883771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234030+0000","last_change":"2024-05-10T10:38:57.234030+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234030+0000","last_undegraded":"2024-05-10T10:38:57.234030+0000","last_fullsized":"2024-05-10T10:38:57.234030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241462+0000","last_change":"2024-05-10T10:38:57.239328+0000","last_active":"2024-05-10T10:38:57.241462+0000","last_peered":"2024-05-10T10:38:57.241462+0000","last_clean":"2024-05-10T10:38:57.241462+0000","last_became_active":"2024-05-10T10:38:57.239042+0000","last_became_peered":"2024-05-10T10:38:57.239042+0000","last_unstale":"2024-05-10T10:38:57.241462+0000","last_undegraded":"2024-05-10T10:38:57.241462+0000","last_fullsized":"2024-05-10T10:38:57.241462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231035+0000","last_change":"2024-05-10T10:38:57.231035+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231035+0000","last_undegraded":"2024-05-10T10:38:57.231035+0000","last_fullsized":"2024-05-10T10:38:57.231035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243561+0000","last_change":"2024-05-10T10:38:57.242120+0000","last_active":"2024-05-10T10:38:57.243561+0000","last_peered":"2024-05-10T10:38:57.243561+0000","last_clean":"2024-05-10T10:38:57.243561+0000","last_became_active":"2024-05-10T10:38:57.241862+0000","last_became_peered":"2024-05-10T10:38:57.241862+0000","last_unstale":"2024-05-10T10:38:57.243561+0000","last_undegraded":"2024-05-10T10:38:57.243561+0000","last_fullsized":"2024-05-10T10:38:57.243561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:32.971721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230683+0000","last_change":"2024-05-10T10:38:57.230683+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230683+0000","last_undegraded":"2024-05-10T10:38:57.230683+0000","last_fullsized":"2024-05-10T10:38:57.230683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247818+0000","last_change":"2024-05-10T10:38:57.246576+0000","last_active":"2024-05-10T10:38:57.247818+0000","last_peered":"2024-05-10T10:38:57.247818+0000","last_clean":"2024-05-10T10:38:57.247818+0000","last_became_active":"2024-05-10T10:38:57.246353+0000","last_became_peered":"2024-05-10T10:38:57.246353+0000","last_unstale":"2024-05-10T10:38:57.247818+0000","last_undegraded":"2024-05-10T10:38:57.247818+0000","last_fullsized":"2024-05-10T10:38:57.247818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:14.665243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227984+0000","last_change":"2024-05-10T10:38:57.227984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227984+0000","last_undegraded":"2024-05-10T10:38:57.227984+0000","last_fullsized":"2024-05-10T10:38:57.227984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245546+0000","last_change":"2024-05-10T10:38:57.243728+0000","last_active":"2024-05-10T10:38:57.245546+0000","last_peered":"2024-05-10T10:38:57.245546+0000","last_clean":"2024-05-10T10:38:57.245546+0000","last_became_active":"2024-05-10T10:38:57.243406+0000","last_became_peered":"2024-05-10T10:38:57.243406+0000","last_unstale":"2024-05-10T10:38:57.245546+0000","last_undegraded":"2024-05-10T10:38:57.245546+0000","last_fullsized":"2024-05-10T10:38:57.245546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232998+0000","last_change":"2024-05-10T10:38:57.232998+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232998+0000","last_undegraded":"2024-05-10T10:38:57.232998+0000","last_fullsized":"2024-05-10T10:38:57.232998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.248051+0000","last_change":"2024-05-10T10:38:57.244580+0000","last_active":"2024-05-10T10:38:57.248051+0000","last_peered":"2024-05-10T10:38:57.248051+0000","last_clean":"2024-05-10T10:38:57.248051+0000","last_became_active":"2024-05-10T10:38:57.244271+0000","last_became_peered":"2024-05-10T10:38:57.244271+0000","last_unstale":"2024-05-10T10:38:57.248051+0000","last_undegraded":"2024-05-10T10:38:57.248051+0000","last_fullsized":"2024-05-10T10:38:57.248051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231123+0000","last_change":"2024-05-10T10:38:57.231123+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231123+0000","last_undegraded":"2024-05-10T10:38:57.231123+0000","last_fullsized":"2024-05-10T10:38:57.231123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240531+0000","last_change":"2024-05-10T10:38:57.236911+0000","last_active":"2024-05-10T10:38:57.240531+0000","last_peered":"2024-05-10T10:38:57.240531+0000","last_clean":"2024-05-10T10:38:57.240531+0000","last_became_active":"2024-05-10T10:38:57.236571+0000","last_became_peered":"2024-05-10T10:38:57.236571+0000","last_unstale":"2024-05-10T10:38:57.240531+0000","last_undegraded":"2024-05-10T10:38:57.240531+0000","last_fullsized":"2024-05-10T10:38:57.240531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:14.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233921+0000","last_change":"2024-05-10T10:38:57.233921+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233921+0000","last_undegraded":"2024-05-10T10:38:57.233921+0000","last_fullsized":"2024-05-10T10:38:57.233921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244349+0000","last_change":"2024-05-10T10:38:57.243643+0000","last_active":"2024-05-10T10:38:57.244349+0000","last_peered":"2024-05-10T10:38:57.244349+0000","last_clean":"2024-05-10T10:38:57.244349+0000","last_became_active":"2024-05-10T10:38:57.243364+0000","last_became_peered":"2024-05-10T10:38:57.243364+0000","last_unstale":"2024-05-10T10:38:57.244349+0000","last_undegraded":"2024-05-10T10:38:57.244349+0000","last_fullsized":"2024-05-10T10:38:57.244349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230113+0000","last_change":"2024-05-10T10:38:57.230113+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230113+0000","last_undegraded":"2024-05-10T10:38:57.230113+0000","last_fullsized":"2024-05-10T10:38:57.230113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241495+0000","last_change":"2024-05-10T10:38:57.239379+0000","last_active":"2024-05-10T10:38:57.241495+0000","last_peered":"2024-05-10T10:38:57.241495+0000","last_clean":"2024-05-10T10:38:57.241495+0000","last_became_active":"2024-05-10T10:38:57.239057+0000","last_became_peered":"2024-05-10T10:38:57.239057+0000","last_unstale":"2024-05-10T10:38:57.241495+0000","last_undegraded":"2024-05-10T10:38:57.241495+0000","last_fullsized":"2024-05-10T10:38:57.241495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228473+0000","last_change":"2024-05-10T10:38:57.228473+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228473+0000","last_undegraded":"2024-05-10T10:38:57.228473+0000","last_fullsized":"2024-05-10T10:38:57.228473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244095+0000","last_change":"2024-05-10T10:38:57.242780+0000","last_active":"2024-05-10T10:38:57.244095+0000","last_peered":"2024-05-10T10:38:57.244095+0000","last_clean":"2024-05-10T10:38:57.244095+0000","last_became_active":"2024-05-10T10:38:57.242494+0000","last_became_peered":"2024-05-10T10:38:57.242494+0000","last_unstale":"2024-05-10T10:38:57.244095+0000","last_undegraded":"2024-05-10T10:38:57.244095+0000","last_fullsized":"2024-05-10T10:38:57.244095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":242,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.228637+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:38:57.228637+0000","last_peered":"2024-05-10T10:38:57.228637+0000","last_clean":"2024-05-10T10:38:57.228637+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:38:57.228637+0000","last_undegraded":"2024-05-10T10:38:57.228637+0000","last_fullsized":"2024-05-10T10:38:57.228637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:39:03.035 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3f is not active+clean 2024-05-10T10:39:03.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232129+0000', 'last_change': '2024-05-10T10:38:57.232129+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232129+0000', 'last_undegraded': '2024-05-10T10:38:57.232129+0000', 'last_fullsized': '2024-05-10T10:38:57.232129+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:39:03.035 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3e is not active+clean 2024-05-10T10:39:03.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.227450+0000', 'last_change': '2024-05-10T10:38:57.227450+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.227450+0000', 'last_undegraded': '2024-05-10T10:38:57.227450+0000', 'last_fullsized': '2024-05-10T10:38:57.227450+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:39:03.035 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3d is not active+clean 2024-05-10T10:39:03.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232404+0000', 'last_change': '2024-05-10T10:38:57.232404+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232404+0000', 'last_undegraded': '2024-05-10T10:38:57.232404+0000', 'last_fullsized': '2024-05-10T10:38:57.232404+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:39:03.035 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3c is not active+clean 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.244373+0000', 'last_change': '2024-05-10T10:38:57.244373+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.244373+0000', 'last_undegraded': '2024-05-10T10:38:57.244373+0000', 'last_fullsized': '2024-05-10T10:38:57.244373+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3b is not active+clean 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.235333+0000', 'last_change': '2024-05-10T10:38:57.235333+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.235333+0000', 'last_undegraded': '2024-05-10T10:38:57.235333+0000', 'last_fullsized': '2024-05-10T10:38:57.235333+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3a is not active+clean 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.236760+0000', 'last_change': '2024-05-10T10:38:57.236760+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.236760+0000', 'last_undegraded': '2024-05-10T10:38:57.236760+0000', 'last_fullsized': '2024-05-10T10:38:57.236760+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:PG 14.39 is not active+clean 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.227454+0000', 'last_change': '2024-05-10T10:38:57.227454+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.227454+0000', 'last_undegraded': '2024-05-10T10:38:57.227454+0000', 'last_fullsized': '2024-05-10T10:38:57.227454+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:PG 14.38 is not active+clean 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.243738+0000', 'last_change': '2024-05-10T10:38:57.243738+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.243738+0000', 'last_undegraded': '2024-05-10T10:38:57.243738+0000', 'last_fullsized': '2024-05-10T10:38:57.243738+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:PG 14.37 is not active+clean 2024-05-10T10:39:03.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234548+0000', 'last_change': '2024-05-10T10:38:57.234548+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234548+0000', 'last_undegraded': '2024-05-10T10:38:57.234548+0000', 'last_fullsized': '2024-05-10T10:38:57.234548+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:PG 14.36 is not active+clean 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234335+0000', 'last_change': '2024-05-10T10:38:57.234335+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234335+0000', 'last_undegraded': '2024-05-10T10:38:57.234335+0000', 'last_fullsized': '2024-05-10T10:38:57.234335+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:PG 14.35 is not active+clean 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234094+0000', 'last_change': '2024-05-10T10:38:57.234094+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234094+0000', 'last_undegraded': '2024-05-10T10:38:57.234094+0000', 'last_fullsized': '2024-05-10T10:38:57.234094+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:PG 14.34 is not active+clean 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234550+0000', 'last_change': '2024-05-10T10:38:57.234550+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234550+0000', 'last_undegraded': '2024-05-10T10:38:57.234550+0000', 'last_fullsized': '2024-05-10T10:38:57.234550+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:PG 14.33 is not active+clean 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.243276+0000', 'last_change': '2024-05-10T10:38:57.243276+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.243276+0000', 'last_undegraded': '2024-05-10T10:38:57.243276+0000', 'last_fullsized': '2024-05-10T10:38:57.243276+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:PG 14.32 is not active+clean 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233211+0000', 'last_change': '2024-05-10T10:38:57.233211+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233211+0000', 'last_undegraded': '2024-05-10T10:38:57.233211+0000', 'last_fullsized': '2024-05-10T10:38:57.233211+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:PG 14.31 is not active+clean 2024-05-10T10:39:03.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.239867+0000', 'last_change': '2024-05-10T10:38:57.239867+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.239867+0000', 'last_undegraded': '2024-05-10T10:38:57.239867+0000', 'last_fullsized': '2024-05-10T10:38:57.239867+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:PG 14.30 is not active+clean 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.235604+0000', 'last_change': '2024-05-10T10:38:57.235604+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.235604+0000', 'last_undegraded': '2024-05-10T10:38:57.235604+0000', 'last_fullsized': '2024-05-10T10:38:57.235604+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2f is not active+clean 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.235059+0000', 'last_change': '2024-05-10T10:38:57.235059+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.235059+0000', 'last_undegraded': '2024-05-10T10:38:57.235059+0000', 'last_fullsized': '2024-05-10T10:38:57.235059+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2e is not active+clean 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.231848+0000', 'last_change': '2024-05-10T10:38:57.231848+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.231848+0000', 'last_undegraded': '2024-05-10T10:38:57.231848+0000', 'last_fullsized': '2024-05-10T10:38:57.231848+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2d is not active+clean 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233571+0000', 'last_change': '2024-05-10T10:38:57.233571+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233571+0000', 'last_undegraded': '2024-05-10T10:38:57.233571+0000', 'last_fullsized': '2024-05-10T10:38:57.233571+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2c is not active+clean 2024-05-10T10:39:03.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.240887+0000', 'last_change': '2024-05-10T10:38:57.240887+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.240887+0000', 'last_undegraded': '2024-05-10T10:38:57.240887+0000', 'last_fullsized': '2024-05-10T10:38:57.240887+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2b is not active+clean 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232634+0000', 'last_change': '2024-05-10T10:38:57.232634+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232634+0000', 'last_undegraded': '2024-05-10T10:38:57.232634+0000', 'last_fullsized': '2024-05-10T10:38:57.232634+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2a is not active+clean 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232606+0000', 'last_change': '2024-05-10T10:38:57.232606+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232606+0000', 'last_undegraded': '2024-05-10T10:38:57.232606+0000', 'last_fullsized': '2024-05-10T10:38:57.232606+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:PG 14.29 is not active+clean 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.235045+0000', 'last_change': '2024-05-10T10:38:57.235045+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.235045+0000', 'last_undegraded': '2024-05-10T10:38:57.235045+0000', 'last_fullsized': '2024-05-10T10:38:57.235045+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:PG 14.28 is not active+clean 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.229984+0000', 'last_change': '2024-05-10T10:38:57.229984+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.229984+0000', 'last_undegraded': '2024-05-10T10:38:57.229984+0000', 'last_fullsized': '2024-05-10T10:38:57.229984+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:PG 14.27 is not active+clean 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.228435+0000', 'last_change': '2024-05-10T10:38:57.228435+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.228435+0000', 'last_undegraded': '2024-05-10T10:38:57.228435+0000', 'last_fullsized': '2024-05-10T10:38:57.228435+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:PG 14.26 is not active+clean 2024-05-10T10:39:03.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233929+0000', 'last_change': '2024-05-10T10:38:57.233929+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233929+0000', 'last_undegraded': '2024-05-10T10:38:57.233929+0000', 'last_fullsized': '2024-05-10T10:38:57.233929+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:PG 14.25 is not active+clean 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233882+0000', 'last_change': '2024-05-10T10:38:57.233882+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233882+0000', 'last_undegraded': '2024-05-10T10:38:57.233882+0000', 'last_fullsized': '2024-05-10T10:38:57.233882+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:PG 14.24 is not active+clean 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.240407+0000', 'last_change': '2024-05-10T10:38:57.240407+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.240407+0000', 'last_undegraded': '2024-05-10T10:38:57.240407+0000', 'last_fullsized': '2024-05-10T10:38:57.240407+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:PG 14.23 is not active+clean 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.228010+0000', 'last_change': '2024-05-10T10:38:57.228010+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.228010+0000', 'last_undegraded': '2024-05-10T10:38:57.228010+0000', 'last_fullsized': '2024-05-10T10:38:57.228010+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:PG 14.22 is not active+clean 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232153+0000', 'last_change': '2024-05-10T10:38:57.232153+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232153+0000', 'last_undegraded': '2024-05-10T10:38:57.232153+0000', 'last_fullsized': '2024-05-10T10:38:57.232153+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:PG 14.21 is not active+clean 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.228694+0000', 'last_change': '2024-05-10T10:38:57.228694+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.228694+0000', 'last_undegraded': '2024-05-10T10:38:57.228694+0000', 'last_fullsized': '2024-05-10T10:38:57.228694+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:39:03.040 INFO:tasks.ceph.ceph_manager.ceph:PG 14.20 is not active+clean 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234543+0000', 'last_change': '2024-05-10T10:38:57.234543+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234543+0000', 'last_undegraded': '2024-05-10T10:38:57.234543+0000', 'last_fullsized': '2024-05-10T10:38:57.234543+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1c is not active+clean 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233533+0000', 'last_change': '2024-05-10T10:38:57.233533+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233533+0000', 'last_undegraded': '2024-05-10T10:38:57.233533+0000', 'last_fullsized': '2024-05-10T10:38:57.233533+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1d is not active+clean 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234020+0000', 'last_change': '2024-05-10T10:38:57.234020+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234020+0000', 'last_undegraded': '2024-05-10T10:38:57.234020+0000', 'last_fullsized': '2024-05-10T10:38:57.234020+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1e is not active+clean 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.227680+0000', 'last_change': '2024-05-10T10:38:57.227680+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.227680+0000', 'last_undegraded': '2024-05-10T10:38:57.227680+0000', 'last_fullsized': '2024-05-10T10:38:57.227680+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1f is not active+clean 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234527+0000', 'last_change': '2024-05-10T10:38:57.234527+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234527+0000', 'last_undegraded': '2024-05-10T10:38:57.234527+0000', 'last_fullsized': '2024-05-10T10:38:57.234527+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:PG 14.18 is not active+clean 2024-05-10T10:39:03.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.239613+0000', 'last_change': '2024-05-10T10:38:57.239613+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.239613+0000', 'last_undegraded': '2024-05-10T10:38:57.239613+0000', 'last_fullsized': '2024-05-10T10:38:57.239613+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:PG 14.19 is not active+clean 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232578+0000', 'last_change': '2024-05-10T10:38:57.232578+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232578+0000', 'last_undegraded': '2024-05-10T10:38:57.232578+0000', 'last_fullsized': '2024-05-10T10:38:57.232578+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1a is not active+clean 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233065+0000', 'last_change': '2024-05-10T10:38:57.233065+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233065+0000', 'last_undegraded': '2024-05-10T10:38:57.233065+0000', 'last_fullsized': '2024-05-10T10:38:57.233065+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1b is not active+clean 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.241217+0000', 'last_change': '2024-05-10T10:38:57.241217+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.241217+0000', 'last_undegraded': '2024-05-10T10:38:57.241217+0000', 'last_fullsized': '2024-05-10T10:38:57.241217+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:PG 14.14 is not active+clean 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.231694+0000', 'last_change': '2024-05-10T10:38:57.231694+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.231694+0000', 'last_undegraded': '2024-05-10T10:38:57.231694+0000', 'last_fullsized': '2024-05-10T10:38:57.231694+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.042 INFO:tasks.ceph.ceph_manager.ceph:PG 14.15 is not active+clean 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.226934+0000', 'last_change': '2024-05-10T10:38:57.226934+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.226934+0000', 'last_undegraded': '2024-05-10T10:38:57.226934+0000', 'last_fullsized': '2024-05-10T10:38:57.226934+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:PG 14.16 is not active+clean 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.230903+0000', 'last_change': '2024-05-10T10:38:57.230903+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.230903+0000', 'last_undegraded': '2024-05-10T10:38:57.230903+0000', 'last_fullsized': '2024-05-10T10:38:57.230903+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:PG 14.17 is not active+clean 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.239277+0000', 'last_change': '2024-05-10T10:38:57.239277+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.239277+0000', 'last_undegraded': '2024-05-10T10:38:57.239277+0000', 'last_fullsized': '2024-05-10T10:38:57.239277+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:PG 14.10 is not active+clean 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.231354+0000', 'last_change': '2024-05-10T10:38:57.231354+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.231354+0000', 'last_undegraded': '2024-05-10T10:38:57.231354+0000', 'last_fullsized': '2024-05-10T10:38:57.231354+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:PG 14.11 is not active+clean 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.231109+0000', 'last_change': '2024-05-10T10:38:57.231109+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.231109+0000', 'last_undegraded': '2024-05-10T10:38:57.231109+0000', 'last_fullsized': '2024-05-10T10:38:57.231109+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:PG 14.12 is not active+clean 2024-05-10T10:39:03.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232630+0000', 'last_change': '2024-05-10T10:38:57.232630+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232630+0000', 'last_undegraded': '2024-05-10T10:38:57.232630+0000', 'last_fullsized': '2024-05-10T10:38:57.232630+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:PG 14.13 is not active+clean 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234758+0000', 'last_change': '2024-05-10T10:38:57.234758+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234758+0000', 'last_undegraded': '2024-05-10T10:38:57.234758+0000', 'last_fullsized': '2024-05-10T10:38:57.234758+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:PG 14.c is not active+clean 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232904+0000', 'last_change': '2024-05-10T10:38:57.232904+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232904+0000', 'last_undegraded': '2024-05-10T10:38:57.232904+0000', 'last_fullsized': '2024-05-10T10:38:57.232904+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:PG 14.d is not active+clean 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234942+0000', 'last_change': '2024-05-10T10:38:57.234942+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234942+0000', 'last_undegraded': '2024-05-10T10:38:57.234942+0000', 'last_fullsized': '2024-05-10T10:38:57.234942+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:PG 14.e is not active+clean 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233300+0000', 'last_change': '2024-05-10T10:38:57.233300+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233300+0000', 'last_undegraded': '2024-05-10T10:38:57.233300+0000', 'last_fullsized': '2024-05-10T10:38:57.233300+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:PG 14.8 is not active+clean 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.231497+0000', 'last_change': '2024-05-10T10:38:57.231497+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.231497+0000', 'last_undegraded': '2024-05-10T10:38:57.231497+0000', 'last_fullsized': '2024-05-10T10:38:57.231497+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.044 INFO:tasks.ceph.ceph_manager.ceph:PG 14.9 is not active+clean 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.227049+0000', 'last_change': '2024-05-10T10:38:57.227049+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.227049+0000', 'last_undegraded': '2024-05-10T10:38:57.227049+0000', 'last_fullsized': '2024-05-10T10:38:57.227049+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:PG 14.a is not active+clean 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.227165+0000', 'last_change': '2024-05-10T10:38:57.227165+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.227165+0000', 'last_undegraded': '2024-05-10T10:38:57.227165+0000', 'last_fullsized': '2024-05-10T10:38:57.227165+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:PG 14.b is not active+clean 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232029+0000', 'last_change': '2024-05-10T10:38:57.232029+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232029+0000', 'last_undegraded': '2024-05-10T10:38:57.232029+0000', 'last_fullsized': '2024-05-10T10:38:57.232029+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:PG 14.4 is not active+clean 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.234030+0000', 'last_change': '2024-05-10T10:38:57.234030+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.234030+0000', 'last_undegraded': '2024-05-10T10:38:57.234030+0000', 'last_fullsized': '2024-05-10T10:38:57.234030+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:PG 14.5 is not active+clean 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.231035+0000', 'last_change': '2024-05-10T10:38:57.231035+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.231035+0000', 'last_undegraded': '2024-05-10T10:38:57.231035+0000', 'last_fullsized': '2024-05-10T10:38:57.231035+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.045 INFO:tasks.ceph.ceph_manager.ceph:PG 14.6 is not active+clean 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.230683+0000', 'last_change': '2024-05-10T10:38:57.230683+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.230683+0000', 'last_undegraded': '2024-05-10T10:38:57.230683+0000', 'last_fullsized': '2024-05-10T10:38:57.230683+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:PG 14.7 is not active+clean 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.227984+0000', 'last_change': '2024-05-10T10:38:57.227984+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.227984+0000', 'last_undegraded': '2024-05-10T10:38:57.227984+0000', 'last_fullsized': '2024-05-10T10:38:57.227984+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:PG 14.0 is not active+clean 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.232998+0000', 'last_change': '2024-05-10T10:38:57.232998+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.232998+0000', 'last_undegraded': '2024-05-10T10:38:57.232998+0000', 'last_fullsized': '2024-05-10T10:38:57.232998+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1 is not active+clean 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.231123+0000', 'last_change': '2024-05-10T10:38:57.231123+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.231123+0000', 'last_undegraded': '2024-05-10T10:38:57.231123+0000', 'last_fullsized': '2024-05-10T10:38:57.231123+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2 is not active+clean 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.233921+0000', 'last_change': '2024-05-10T10:38:57.233921+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.233921+0000', 'last_undegraded': '2024-05-10T10:38:57.233921+0000', 'last_fullsized': '2024-05-10T10:38:57.233921+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-10T10:39:03.046 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3 is not active+clean 2024-05-10T10:39:03.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.230113+0000', 'last_change': '2024-05-10T10:38:57.230113+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.230113+0000', 'last_undegraded': '2024-05-10T10:38:57.230113+0000', 'last_fullsized': '2024-05-10T10:38:57.230113+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-10T10:39:03.047 INFO:tasks.ceph.ceph_manager.ceph:PG 14.f is not active+clean 2024-05-10T10:39:03.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-05-10T10:38:57.228473+0000', 'last_change': '2024-05-10T10:38:57.228473+0000', 'last_active': '2024-05-10T10:38:57.221344+0000', 'last_peered': '2024-05-10T10:38:57.221344+0000', 'last_clean': '2024-05-10T10:38:57.221344+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-10T10:38:57.228473+0000', 'last_undegraded': '2024-05-10T10:38:57.228473+0000', 'last_fullsized': '2024-05-10T10:38:57.228473+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'last_clean_scrub_stamp': '2024-05-10T10:38:57.221344+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-10T10:39:03.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-10T10:39:03.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.171+0000 7f3f35693640 1 Processor -- start 2024-05-10T10:39:03.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.172+0000 7f3f35693640 1 -- start start 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.172+0000 7f3f35693640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300c8740 0x7f3f300c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.172+0000 7f3f35693640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3f300d21c0 0x7f3f300c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.172+0000 7f3f35693640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f300d0960 con 0x7f3f300b64a0 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.172+0000 7f3f35693640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3f300d0c40 con 0x7f3f300d21c0 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.172+0000 7f3f35693640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3f300d0f20 con 0x7f3f300c8740 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3f300d21c0 0x7f3f300c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2e7fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300c8740 0x7f3f300c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3f300d21c0 0x7f3f300c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:34386/0 (socket says 172.21.15.53:34386) 2024-05-10T10:39:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/2050941463 learned_addr learned my addr 172.21.15.53:0/2050941463 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300c8740 msgr2=0x7f3f300c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2f7fe640 1 --2- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300c8740 0x7f3f300c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/2050941463 >> v1:172.21.15.135:6789/0 conn(0x7f3f300b64a0 legacy=0x7f3f300d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2dffb640 1 -- 172.21.15.53:0/2050941463 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911321557 0 0) 0x7f3f300d0960 con 0x7f3f300b64a0 2024-05-10T10:39:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/2050941463 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f300b6880 con 0x7f3f300d21c0 2024-05-10T10:39:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.173+0000 7f3f2e7fc640 1 --2- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300c8740 0x7f3f300c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.174+0000 7f3f2f7fe640 1 --2- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300d21c0 0x7f3f300c8d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009870 tx=0x7f3f2802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae14b12ae8ce751 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.174+0000 7f3f2dffb640 1 -- 172.21.15.53:0/2050941463 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28037070 con 0x7f3f300d21c0 2024-05-10T10:39:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.174+0000 7f3f2dffb640 1 -- 172.21.15.53:0/2050941463 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f28032440 con 0x7f3f300d21c0 2024-05-10T10:39:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.174+0000 7f3f2dffb640 1 -- 172.21.15.53:0/2050941463 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28032810 con 0x7f3f300d21c0 2024-05-10T10:39:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 -- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300d21c0 msgr2=0x7f3f300c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 --2- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300d21c0 0x7f3f300c8d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009870 tx=0x7f3f2802a250 comp rx=0 tx=0).stop 2024-05-10T10:39:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 -- 172.21.15.53:0/2050941463 shutdown_connections 2024-05-10T10:39:03.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 --2- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300c8740 0x7f3f300c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 --2- 172.21.15.53:0/2050941463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300d21c0 0x7f3f300c8d80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 -- 172.21.15.53:0/2050941463 >> 172.21.15.53:0/2050941463 conn(0x7f3f300d5130 msgr2=0x7f3f300d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:03.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 -- 172.21.15.53:0/2050941463 shutdown_connections 2024-05-10T10:39:03.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.175+0000 7f3f35693640 1 -- 172.21.15.53:0/2050941463 wait complete. 2024-05-10T10:39:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.176+0000 7f3f35693640 1 Processor -- start 2024-05-10T10:39:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.176+0000 7f3f35693640 1 -- start start 2024-05-10T10:39:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f35693640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300b64a0 0x7f3f301612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f35693640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300c8740 0x7f3f301ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f35693640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f300d21c0 0x7f3f30152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f35693640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3f300c92c0 con 0x7f3f300d21c0 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f35693640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f300d5ea0 con 0x7f3f300c8740 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f35693640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3f300c9440 con 0x7f3f300b64a0 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f2f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f300d21c0 0x7f3f30152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f2f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f300d21c0 0x7f3f30152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60510/0 (socket says 172.21.15.53:60510) 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/1329696079 learned_addr learned my addr 172.21.15.53:0/1329696079 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f2e7fc640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300c8740 0x7f3f301ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.177+0000 7f3f2effd640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300b64a0 0x7f3f301612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300b64a0 msgr2=0x7f3f301612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f2f7fe640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300b64a0 0x7f3f301612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300c8740 msgr2=0x7f3f301ed7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f2f7fe640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300c8740 0x7f3f301ed7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/1329696079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f301ecd00 con 0x7f3f300d21c0 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f2f7fe640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f300d21c0 0x7f3f30152390 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009870 tx=0x7f3f280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f277fe640 1 -- 172.21.15.53:0/1329696079 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28037040 con 0x7f3f300d21c0 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f277fe640 1 -- 172.21.15.53:0/1329696079 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f28004230 con 0x7f3f300d21c0 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f277fe640 1 -- 172.21.15.53:0/1329696079 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28003610 con 0x7f3f300d21c0 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f30163ff0 con 0x7f3f300d21c0 2024-05-10T10:39:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.178+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301645b0 con 0x7f3f300d21c0 2024-05-10T10:39:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.179+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f30164780 con 0x7f3f300d21c0 2024-05-10T10:39:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.180+0000 7f3f277fe640 1 -- 172.21.15.53:0/1329696079 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f280037b0 con 0x7f3f300d21c0 2024-05-10T10:39:03.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.180+0000 7f3f277fe640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f10076300 0x7f3f100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.180+0000 7f3f277fe640 1 -- 172.21.15.53:0/1329696079 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f3f280b82d0 con 0x7f3f300d21c0 2024-05-10T10:39:03.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.180+0000 7f3f2effd640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f10076300 0x7f3f100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.181+0000 7f3f2effd640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f10076300 0x7f3f100787c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3f200012e0 tx=0x7f3f20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:03.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.184+0000 7f3f277fe640 1 -- 172.21.15.53:0/1329696079 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3f28081450 con 0x7f3f300d21c0 2024-05-10T10:39:03.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.450+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3f3006a220 con 0x7f3f300d21c0 2024-05-10T10:39:03.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.450+0000 7f3f277fe640 1 -- 172.21.15.53:0/1329696079 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7f3f28085750 con 0x7f3f300d21c0 2024-05-10T10:39:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:03.456 INFO:teuthology.orchestra.run.smithi053.stdout:{"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":345,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1715337207,"num_in_osds":8,"osd_in_since":1715337199,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":260751360,"bytes_avail":772833361920,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":61,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":6693}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-05-10T10:38:46.232671+0000","services":{}},"progress_events":{}} 2024-05-10T10:39:03.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f10076300 msgr2=0x7f3f100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f10076300 0x7f3f100787c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3f200012e0 tx=0x7f3f20009040 comp rx=0 tx=0).stop 2024-05-10T10:39:03.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f300d21c0 msgr2=0x7f3f30152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f300d21c0 0x7f3f30152390 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009870 tx=0x7f3f280046c0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 shutdown_connections 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3f10076300 0x7f3f100787c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f300d21c0 0x7f3f30152390 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f300c8740 0x7f3f301ed7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 --2- 172.21.15.53:0/1329696079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3f300b64a0 0x7f3f301612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 >> 172.21.15.53:0/1329696079 conn(0x7f3f300d5130 msgr2=0x7f3f300d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 shutdown_connections 2024-05-10T10:39:03.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.452+0000 7f3f35693640 1 -- 172.21.15.53:0/1329696079 wait complete. 2024-05-10T10:39:03.465 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-10T10:39:03.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:39:03.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.567+0000 7f7a0f37f640 1 Processor -- start 2024-05-10T10:39:03.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.568+0000 7f7a0f37f640 1 -- start start 2024-05-10T10:39:03.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.568+0000 7f7a0f37f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a080702e0 0x7f7a080d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.568+0000 7f7a0f37f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0806f3c0 con 0x7f7a0815f290 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.568+0000 7f7a0f37f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a080d16f0 con 0x7f7a080d3450 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.568+0000 7f7a0f37f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7a080d14c0 con 0x7f7a080702e0 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.568+0000 7f7a0d11b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7a0815f290 0x7f7a080d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37430/0 (socket says 172.21.15.53:37430) 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.568+0000 7f7a0d11b640 1 -- 172.21.15.53:0/1344808667 learned_addr learned my addr 172.21.15.53:0/1344808667 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a0d91c640 1 --2- 172.21.15.53:0/1344808667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a080702e0 0x7f7a080d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309278427 0 0) 0x7f7a0806f3c0 con 0x7f7a0815f290 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f0003520 con 0x7f7a0815f290 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087613158 0 0) 0x7f7a080d16f0 con 0x7f7a080d3450 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a0806f3c0 con 0x7f7a080d3450 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3627842010 0 0) 0x7f79f0003520 con 0x7f7a0815f290 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a0d91c640 1 -- 172.21.15.53:0/1344808667 >> v1:172.21.15.135:6789/0 conn(0x7f7a080d3450 legacy=0x7f7a0806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a0d91c640 1 -- 172.21.15.53:0/1344808667 >> v1:172.21.15.53:6789/0 conn(0x7f7a0815f290 legacy=0x7f7a080d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.569+0000 7f7a0d91c640 1 -- 172.21.15.53:0/1344808667 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a080d19d0 con 0x7f7a080702e0 2024-05-10T10:39:03.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.570+0000 7f7a0d91c640 1 --2- 172.21.15.53:0/1344808667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a080702e0 0x7f7a080d0f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009dc0 tx=0x7f79f802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d56e706b35925d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:03.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.570+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00001670 con 0x7f7a080702e0 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.570+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f802ae50 con 0x7f7a080702e0 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.570+0000 7f7a07fff640 1 -- 172.21.15.53:0/1344808667 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8007760 con 0x7f7a080702e0 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.571+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1344808667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a080702e0 msgr2=0x7f7a080d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.571+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1344808667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a080702e0 0x7f7a080d0f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009dc0 tx=0x7f79f802a6f0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.571+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1344808667 shutdown_connections 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.571+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1344808667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a080702e0 0x7f7a080d0f80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.571+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1344808667 >> 172.21.15.53:0/1344808667 conn(0x7f7a0809ba20 msgr2=0x7f7a0809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.571+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1344808667 shutdown_connections 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.571+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1344808667 wait complete. 2024-05-10T10:39:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.572+0000 7f7a0f37f640 1 Processor -- start 2024-05-10T10:39:03.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.572+0000 7f7a0f37f640 1 -- start start 2024-05-10T10:39:03.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0f37f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a080702e0 0x7f7a0815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0f37f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a080d3450 0x7f7a0809ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0f37f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0815f290 0x7f7a081f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0f37f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a080d1f90 con 0x7f7a080702e0 2024-05-10T10:39:03.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0f37f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a080d2300 con 0x7f7a080d3450 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0f37f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0815f670 con 0x7f7a0815f290 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a080702e0 0x7f7a0815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a080702e0 0x7f7a0815b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60514/0 (socket says 172.21.15.53:60514) 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 -- 172.21.15.53:0/1830498278 learned_addr learned my addr 172.21.15.53:0/1830498278 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0c91a640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a080d3450 0x7f7a0809ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d91c640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0815f290 0x7f7a081f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 -- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0815f290 msgr2=0x7f7a081f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0815f290 0x7f7a081f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 -- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a080d3450 msgr2=0x7f7a0809ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a080d3450 0x7f7a0809ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 -- 172.21.15.53:0/1830498278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0815a0f0 con 0x7f7a080702e0 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0c91a640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a080d3450 0x7f7a0809ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0d11b640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a080702e0 0x7f7a0815b160 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000c750 tx=0x7f7a0000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:03.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a05ffb640 1 -- 172.21.15.53:0/1830498278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00005ad0 con 0x7f7a080702e0 2024-05-10T10:39:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a05ffb640 1 -- 172.21.15.53:0/1830498278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a00005c70 con 0x7f7a080702e0 2024-05-10T10:39:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0815a680 con 0x7f7a080702e0 2024-05-10T10:39:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.573+0000 7f7a05ffb640 1 -- 172.21.15.53:0/1830498278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00005e60 con 0x7f7a080702e0 2024-05-10T10:39:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.574+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0815ac40 con 0x7f7a080702e0 2024-05-10T10:39:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.575+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a0815ae40 con 0x7f7a080702e0 2024-05-10T10:39:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.575+0000 7f7a05ffb640 1 -- 172.21.15.53:0/1830498278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7a000072b0 con 0x7f7a080702e0 2024-05-10T10:39:03.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.576+0000 7f7a05ffb640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f79f40762d0 0x7f79f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:03.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.576+0000 7f7a05ffb640 1 -- 172.21.15.53:0/1830498278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f7a000974e0 con 0x7f7a080702e0 2024-05-10T10:39:03.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.576+0000 7f7a0c91a640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f79f40762d0 0x7f79f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:03.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.577+0000 7f7a0c91a640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f79f40762d0 0x7f79f4078790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7a080c9a10 tx=0x7f79fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:03.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.580+0000 7f7a05ffb640 1 -- 172.21.15.53:0/1830498278 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7a0815ae40 con 0x7f7a080702e0 2024-05-10T10:39:03.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.765+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a080693f0 con 0x7f79f40762d0 2024-05-10T10:39:03.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.772+0000 7f7a05ffb640 1 -- 172.21.15.53:0/1830498278 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246754 (secure 0 0 0) 0x7f7a080693f0 con 0x7f79f40762d0 2024-05-10T10:39:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:03.777 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f79f40762d0 msgr2=0x7f79f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f79f40762d0 0x7f79f4078790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7a080c9a10 tx=0x7f79fc009040 comp rx=0 tx=0).stop 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a080702e0 msgr2=0x7f7a0815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a080702e0 0x7f7a0815b160 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000c750 tx=0x7f7a0000cc10 comp rx=0 tx=0).stop 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 shutdown_connections 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f79f40762d0 0x7f79f4078790 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0815f290 0x7f7a081f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a080d3450 0x7f7a0809ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 --2- 172.21.15.53:0/1830498278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a080702e0 0x7f7a0815b160 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 >> 172.21.15.53:0/1830498278 conn(0x7f7a0809ba20 msgr2=0x7f7a080c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 shutdown_connections 2024-05-10T10:39:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:03.775+0000 7f7a0f37f640 1 -- 172.21.15.53:0/1830498278 wait complete. 2024-05-10T10:39:03.787 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-05-10T10:39:02.235941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254640,"kb_used_data":9352,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720080,"statfs":{"total":773094113280,"available":772833361920,"internally_reserved":0,"allocated":9576448,"data_stored":4992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232129+0000","last_change":"2024-05-10T10:38:57.232129+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232129+0000","last_undegraded":"2024-05-10T10:38:57.232129+0000","last_fullsized":"2024-05-10T10:38:57.232129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227450+0000","last_change":"2024-05-10T10:38:57.227450+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227450+0000","last_undegraded":"2024-05-10T10:38:57.227450+0000","last_fullsized":"2024-05-10T10:38:57.227450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232404+0000","last_change":"2024-05-10T10:38:57.232404+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232404+0000","last_undegraded":"2024-05-10T10:38:57.232404+0000","last_fullsized":"2024-05-10T10:38:57.232404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.244373+0000","last_change":"2024-05-10T10:38:57.244373+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.244373+0000","last_undegraded":"2024-05-10T10:38:57.244373+0000","last_fullsized":"2024-05-10T10:38:57.244373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235333+0000","last_change":"2024-05-10T10:38:57.235333+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235333+0000","last_undegraded":"2024-05-10T10:38:57.235333+0000","last_fullsized":"2024-05-10T10:38:57.235333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.236760+0000","last_change":"2024-05-10T10:38:57.236760+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.236760+0000","last_undegraded":"2024-05-10T10:38:57.236760+0000","last_fullsized":"2024-05-10T10:38:57.236760+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227454+0000","last_change":"2024-05-10T10:38:57.227454+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227454+0000","last_undegraded":"2024-05-10T10:38:57.227454+0000","last_fullsized":"2024-05-10T10:38:57.227454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243738+0000","last_change":"2024-05-10T10:38:57.243738+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243738+0000","last_undegraded":"2024-05-10T10:38:57.243738+0000","last_fullsized":"2024-05-10T10:38:57.243738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234548+0000","last_change":"2024-05-10T10:38:57.234548+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234548+0000","last_undegraded":"2024-05-10T10:38:57.234548+0000","last_fullsized":"2024-05-10T10:38:57.234548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234335+0000","last_change":"2024-05-10T10:38:57.234335+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234335+0000","last_undegraded":"2024-05-10T10:38:57.234335+0000","last_fullsized":"2024-05-10T10:38:57.234335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234094+0000","last_change":"2024-05-10T10:38:57.234094+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234094+0000","last_undegraded":"2024-05-10T10:38:57.234094+0000","last_fullsized":"2024-05-10T10:38:57.234094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234550+0000","last_change":"2024-05-10T10:38:57.234550+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234550+0000","last_undegraded":"2024-05-10T10:38:57.234550+0000","last_fullsized":"2024-05-10T10:38:57.234550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.243276+0000","last_change":"2024-05-10T10:38:57.243276+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.243276+0000","last_undegraded":"2024-05-10T10:38:57.243276+0000","last_fullsized":"2024-05-10T10:38:57.243276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233211+0000","last_change":"2024-05-10T10:38:57.233211+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233211+0000","last_undegraded":"2024-05-10T10:38:57.233211+0000","last_fullsized":"2024-05-10T10:38:57.233211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239867+0000","last_change":"2024-05-10T10:38:57.239867+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239867+0000","last_undegraded":"2024-05-10T10:38:57.239867+0000","last_fullsized":"2024-05-10T10:38:57.239867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235604+0000","last_change":"2024-05-10T10:38:57.235604+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235604+0000","last_undegraded":"2024-05-10T10:38:57.235604+0000","last_fullsized":"2024-05-10T10:38:57.235604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235059+0000","last_change":"2024-05-10T10:38:57.235059+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235059+0000","last_undegraded":"2024-05-10T10:38:57.235059+0000","last_fullsized":"2024-05-10T10:38:57.235059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231848+0000","last_change":"2024-05-10T10:38:57.231848+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231848+0000","last_undegraded":"2024-05-10T10:38:57.231848+0000","last_fullsized":"2024-05-10T10:38:57.231848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233571+0000","last_change":"2024-05-10T10:38:57.233571+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233571+0000","last_undegraded":"2024-05-10T10:38:57.233571+0000","last_fullsized":"2024-05-10T10:38:57.233571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240887+0000","last_change":"2024-05-10T10:38:57.240887+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240887+0000","last_undegraded":"2024-05-10T10:38:57.240887+0000","last_fullsized":"2024-05-10T10:38:57.240887+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232634+0000","last_change":"2024-05-10T10:38:57.232634+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232634+0000","last_undegraded":"2024-05-10T10:38:57.232634+0000","last_fullsized":"2024-05-10T10:38:57.232634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232606+0000","last_change":"2024-05-10T10:38:57.232606+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232606+0000","last_undegraded":"2024-05-10T10:38:57.232606+0000","last_fullsized":"2024-05-10T10:38:57.232606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.235045+0000","last_change":"2024-05-10T10:38:57.235045+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.235045+0000","last_undegraded":"2024-05-10T10:38:57.235045+0000","last_fullsized":"2024-05-10T10:38:57.235045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.229984+0000","last_change":"2024-05-10T10:38:57.229984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.229984+0000","last_undegraded":"2024-05-10T10:38:57.229984+0000","last_fullsized":"2024-05-10T10:38:57.229984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228435+0000","last_change":"2024-05-10T10:38:57.228435+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228435+0000","last_undegraded":"2024-05-10T10:38:57.228435+0000","last_fullsized":"2024-05-10T10:38:57.228435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233929+0000","last_change":"2024-05-10T10:38:57.233929+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233929+0000","last_undegraded":"2024-05-10T10:38:57.233929+0000","last_fullsized":"2024-05-10T10:38:57.233929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233882+0000","last_change":"2024-05-10T10:38:57.233882+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233882+0000","last_undegraded":"2024-05-10T10:38:57.233882+0000","last_fullsized":"2024-05-10T10:38:57.233882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.240407+0000","last_change":"2024-05-10T10:38:57.240407+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.240407+0000","last_undegraded":"2024-05-10T10:38:57.240407+0000","last_fullsized":"2024-05-10T10:38:57.240407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228010+0000","last_change":"2024-05-10T10:38:57.228010+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228010+0000","last_undegraded":"2024-05-10T10:38:57.228010+0000","last_fullsized":"2024-05-10T10:38:57.228010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232153+0000","last_change":"2024-05-10T10:38:57.232153+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232153+0000","last_undegraded":"2024-05-10T10:38:57.232153+0000","last_fullsized":"2024-05-10T10:38:57.232153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228694+0000","last_change":"2024-05-10T10:38:57.228694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228694+0000","last_undegraded":"2024-05-10T10:38:57.228694+0000","last_fullsized":"2024-05-10T10:38:57.228694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234543+0000","last_change":"2024-05-10T10:38:57.234543+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234543+0000","last_undegraded":"2024-05-10T10:38:57.234543+0000","last_fullsized":"2024-05-10T10:38:57.234543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233533+0000","last_change":"2024-05-10T10:38:57.233533+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233533+0000","last_undegraded":"2024-05-10T10:38:57.233533+0000","last_fullsized":"2024-05-10T10:38:57.233533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245963+0000","last_change":"2024-05-10T10:38:57.244889+0000","last_active":"2024-05-10T10:38:57.245963+0000","last_peered":"2024-05-10T10:38:57.245963+0000","last_clean":"2024-05-10T10:38:57.245963+0000","last_became_active":"2024-05-10T10:38:57.244439+0000","last_became_peered":"2024-05-10T10:38:57.244439+0000","last_unstale":"2024-05-10T10:38:57.245963+0000","last_undegraded":"2024-05-10T10:38:57.245963+0000","last_fullsized":"2024-05-10T10:38:57.245963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234020+0000","last_change":"2024-05-10T10:38:57.234020+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234020+0000","last_undegraded":"2024-05-10T10:38:57.234020+0000","last_fullsized":"2024-05-10T10:38:57.234020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243113+0000","last_change":"2024-05-10T10:38:57.241164+0000","last_active":"2024-05-10T10:38:57.243113+0000","last_peered":"2024-05-10T10:38:57.243113+0000","last_clean":"2024-05-10T10:38:57.243113+0000","last_became_active":"2024-05-10T10:38:57.240896+0000","last_became_peered":"2024-05-10T10:38:57.240896+0000","last_unstale":"2024-05-10T10:38:57.243113+0000","last_undegraded":"2024-05-10T10:38:57.243113+0000","last_fullsized":"2024-05-10T10:38:57.243113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:12:16.909583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227680+0000","last_change":"2024-05-10T10:38:57.227680+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227680+0000","last_undegraded":"2024-05-10T10:38:57.227680+0000","last_fullsized":"2024-05-10T10:38:57.227680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241017+0000","last_change":"2024-05-10T10:38:57.238335+0000","last_active":"2024-05-10T10:38:57.241017+0000","last_peered":"2024-05-10T10:38:57.241017+0000","last_clean":"2024-05-10T10:38:57.241017+0000","last_became_active":"2024-05-10T10:38:57.238013+0000","last_became_peered":"2024-05-10T10:38:57.238013+0000","last_unstale":"2024-05-10T10:38:57.241017+0000","last_undegraded":"2024-05-10T10:38:57.241017+0000","last_fullsized":"2024-05-10T10:38:57.241017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234527+0000","last_change":"2024-05-10T10:38:57.234527+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234527+0000","last_undegraded":"2024-05-10T10:38:57.234527+0000","last_fullsized":"2024-05-10T10:38:57.234527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245980+0000","last_change":"2024-05-10T10:38:57.245097+0000","last_active":"2024-05-10T10:38:57.245980+0000","last_peered":"2024-05-10T10:38:57.245980+0000","last_clean":"2024-05-10T10:38:57.245980+0000","last_became_active":"2024-05-10T10:38:57.244782+0000","last_became_peered":"2024-05-10T10:38:57.244782+0000","last_unstale":"2024-05-10T10:38:57.245980+0000","last_undegraded":"2024-05-10T10:38:57.245980+0000","last_fullsized":"2024-05-10T10:38:57.245980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:57.598796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239613+0000","last_change":"2024-05-10T10:38:57.239613+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239613+0000","last_undegraded":"2024-05-10T10:38:57.239613+0000","last_fullsized":"2024-05-10T10:38:57.239613+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251314+0000","last_change":"2024-05-10T10:38:57.249546+0000","last_active":"2024-05-10T10:38:57.251314+0000","last_peered":"2024-05-10T10:38:57.251314+0000","last_clean":"2024-05-10T10:38:57.251314+0000","last_became_active":"2024-05-10T10:38:57.249316+0000","last_became_peered":"2024-05-10T10:38:57.249316+0000","last_unstale":"2024-05-10T10:38:57.251314+0000","last_undegraded":"2024-05-10T10:38:57.251314+0000","last_fullsized":"2024-05-10T10:38:57.251314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:54:22.781845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232578+0000","last_change":"2024-05-10T10:38:57.232578+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232578+0000","last_undegraded":"2024-05-10T10:38:57.232578+0000","last_fullsized":"2024-05-10T10:38:57.232578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.251627+0000","last_change":"2024-05-10T10:38:57.250917+0000","last_active":"2024-05-10T10:38:57.251627+0000","last_peered":"2024-05-10T10:38:57.251627+0000","last_clean":"2024-05-10T10:38:57.251627+0000","last_became_active":"2024-05-10T10:38:57.250631+0000","last_became_peered":"2024-05-10T10:38:57.250631+0000","last_unstale":"2024-05-10T10:38:57.251627+0000","last_undegraded":"2024-05-10T10:38:57.251627+0000","last_fullsized":"2024-05-10T10:38:57.251627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233065+0000","last_change":"2024-05-10T10:38:57.233065+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233065+0000","last_undegraded":"2024-05-10T10:38:57.233065+0000","last_fullsized":"2024-05-10T10:38:57.233065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235166+0000","last_change":"2024-05-10T10:38:57.234005+0000","last_active":"2024-05-10T10:38:57.235166+0000","last_peered":"2024-05-10T10:38:57.235166+0000","last_clean":"2024-05-10T10:38:57.235166+0000","last_became_active":"2024-05-10T10:38:57.233624+0000","last_became_peered":"2024-05-10T10:38:57.233624+0000","last_unstale":"2024-05-10T10:38:57.235166+0000","last_undegraded":"2024-05-10T10:38:57.235166+0000","last_fullsized":"2024-05-10T10:38:57.235166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.241217+0000","last_change":"2024-05-10T10:38:57.241217+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.241217+0000","last_undegraded":"2024-05-10T10:38:57.241217+0000","last_fullsized":"2024-05-10T10:38:57.241217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245553+0000","last_change":"2024-05-10T10:38:57.243368+0000","last_active":"2024-05-10T10:38:57.245553+0000","last_peered":"2024-05-10T10:38:57.245553+0000","last_clean":"2024-05-10T10:38:57.245553+0000","last_became_active":"2024-05-10T10:38:57.243101+0000","last_became_peered":"2024-05-10T10:38:57.243101+0000","last_unstale":"2024-05-10T10:38:57.245553+0000","last_undegraded":"2024-05-10T10:38:57.245553+0000","last_fullsized":"2024-05-10T10:38:57.245553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231694+0000","last_change":"2024-05-10T10:38:57.231694+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231694+0000","last_undegraded":"2024-05-10T10:38:57.231694+0000","last_fullsized":"2024-05-10T10:38:57.231694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245621+0000","last_change":"2024-05-10T10:38:57.244172+0000","last_active":"2024-05-10T10:38:57.245621+0000","last_peered":"2024-05-10T10:38:57.245621+0000","last_clean":"2024-05-10T10:38:57.245621+0000","last_became_active":"2024-05-10T10:38:57.243827+0000","last_became_peered":"2024-05-10T10:38:57.243827+0000","last_unstale":"2024-05-10T10:38:57.245621+0000","last_undegraded":"2024-05-10T10:38:57.245621+0000","last_fullsized":"2024-05-10T10:38:57.245621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:14.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.226934+0000","last_change":"2024-05-10T10:38:57.226934+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.226934+0000","last_undegraded":"2024-05-10T10:38:57.226934+0000","last_fullsized":"2024-05-10T10:38:57.226934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243252+0000","last_change":"2024-05-10T10:38:57.242608+0000","last_active":"2024-05-10T10:38:57.243252+0000","last_peered":"2024-05-10T10:38:57.243252+0000","last_clean":"2024-05-10T10:38:57.243252+0000","last_became_active":"2024-05-10T10:38:57.242307+0000","last_became_peered":"2024-05-10T10:38:57.242307+0000","last_unstale":"2024-05-10T10:38:57.243252+0000","last_undegraded":"2024-05-10T10:38:57.243252+0000","last_fullsized":"2024-05-10T10:38:57.243252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230903+0000","last_change":"2024-05-10T10:38:57.230903+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230903+0000","last_undegraded":"2024-05-10T10:38:57.230903+0000","last_fullsized":"2024-05-10T10:38:57.230903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247838+0000","last_change":"2024-05-10T10:38:57.246632+0000","last_active":"2024-05-10T10:38:57.247838+0000","last_peered":"2024-05-10T10:38:57.247838+0000","last_clean":"2024-05-10T10:38:57.247838+0000","last_became_active":"2024-05-10T10:38:57.246417+0000","last_became_peered":"2024-05-10T10:38:57.246417+0000","last_unstale":"2024-05-10T10:38:57.247838+0000","last_undegraded":"2024-05-10T10:38:57.247838+0000","last_fullsized":"2024-05-10T10:38:57.247838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:52:12.230139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.239277+0000","last_change":"2024-05-10T10:38:57.239277+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.239277+0000","last_undegraded":"2024-05-10T10:38:57.239277+0000","last_fullsized":"2024-05-10T10:38:57.239277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244891+0000","last_change":"2024-05-10T10:38:57.238700+0000","last_active":"2024-05-10T10:38:57.244891+0000","last_peered":"2024-05-10T10:38:57.244891+0000","last_clean":"2024-05-10T10:38:57.244891+0000","last_became_active":"2024-05-10T10:38:57.238416+0000","last_became_peered":"2024-05-10T10:38:57.238416+0000","last_unstale":"2024-05-10T10:38:57.244891+0000","last_undegraded":"2024-05-10T10:38:57.244891+0000","last_fullsized":"2024-05-10T10:38:57.244891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231354+0000","last_change":"2024-05-10T10:38:57.231354+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231354+0000","last_undegraded":"2024-05-10T10:38:57.231354+0000","last_fullsized":"2024-05-10T10:38:57.231354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.235157+0000","last_change":"2024-05-10T10:38:57.233569+0000","last_active":"2024-05-10T10:38:57.235157+0000","last_peered":"2024-05-10T10:38:57.235157+0000","last_clean":"2024-05-10T10:38:57.235157+0000","last_became_active":"2024-05-10T10:38:57.233304+0000","last_became_peered":"2024-05-10T10:38:57.233304+0000","last_unstale":"2024-05-10T10:38:57.235157+0000","last_undegraded":"2024-05-10T10:38:57.235157+0000","last_fullsized":"2024-05-10T10:38:57.235157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:19.208256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231109+0000","last_change":"2024-05-10T10:38:57.231109+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231109+0000","last_undegraded":"2024-05-10T10:38:57.231109+0000","last_fullsized":"2024-05-10T10:38:57.231109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244857+0000","last_change":"2024-05-10T10:38:57.242521+0000","last_active":"2024-05-10T10:38:57.244857+0000","last_peered":"2024-05-10T10:38:57.244857+0000","last_clean":"2024-05-10T10:38:57.244857+0000","last_became_active":"2024-05-10T10:38:57.242195+0000","last_became_peered":"2024-05-10T10:38:57.242195+0000","last_unstale":"2024-05-10T10:38:57.244857+0000","last_undegraded":"2024-05-10T10:38:57.244857+0000","last_fullsized":"2024-05-10T10:38:57.244857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232630+0000","last_change":"2024-05-10T10:38:57.232630+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232630+0000","last_undegraded":"2024-05-10T10:38:57.232630+0000","last_fullsized":"2024-05-10T10:38:57.232630+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244065+0000","last_change":"2024-05-10T10:38:57.240790+0000","last_active":"2024-05-10T10:38:57.244065+0000","last_peered":"2024-05-10T10:38:57.244065+0000","last_clean":"2024-05-10T10:38:57.244065+0000","last_became_active":"2024-05-10T10:38:57.240455+0000","last_became_peered":"2024-05-10T10:38:57.240455+0000","last_unstale":"2024-05-10T10:38:57.244065+0000","last_undegraded":"2024-05-10T10:38:57.244065+0000","last_fullsized":"2024-05-10T10:38:57.244065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234758+0000","last_change":"2024-05-10T10:38:57.234758+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234758+0000","last_undegraded":"2024-05-10T10:38:57.234758+0000","last_fullsized":"2024-05-10T10:38:57.234758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.250429+0000","last_change":"2024-05-10T10:38:57.249734+0000","last_active":"2024-05-10T10:38:57.250429+0000","last_peered":"2024-05-10T10:38:57.250429+0000","last_clean":"2024-05-10T10:38:57.250429+0000","last_became_active":"2024-05-10T10:38:57.249452+0000","last_became_peered":"2024-05-10T10:38:57.249452+0000","last_unstale":"2024-05-10T10:38:57.250429+0000","last_undegraded":"2024-05-10T10:38:57.250429+0000","last_fullsized":"2024-05-10T10:38:57.250429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232904+0000","last_change":"2024-05-10T10:38:57.232904+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232904+0000","last_undegraded":"2024-05-10T10:38:57.232904+0000","last_fullsized":"2024-05-10T10:38:57.232904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242899+0000","last_change":"2024-05-10T10:38:57.241984+0000","last_active":"2024-05-10T10:38:57.242899+0000","last_peered":"2024-05-10T10:38:57.242899+0000","last_clean":"2024-05-10T10:38:57.242899+0000","last_became_active":"2024-05-10T10:38:57.241709+0000","last_became_peered":"2024-05-10T10:38:57.241709+0000","last_unstale":"2024-05-10T10:38:57.242899+0000","last_undegraded":"2024-05-10T10:38:57.242899+0000","last_fullsized":"2024-05-10T10:38:57.242899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234942+0000","last_change":"2024-05-10T10:38:57.234942+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234942+0000","last_undegraded":"2024-05-10T10:38:57.234942+0000","last_fullsized":"2024-05-10T10:38:57.234942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.242330+0000","last_change":"2024-05-10T10:38:57.240601+0000","last_active":"2024-05-10T10:38:57.242330+0000","last_peered":"2024-05-10T10:38:57.242330+0000","last_clean":"2024-05-10T10:38:57.242330+0000","last_became_active":"2024-05-10T10:38:57.240018+0000","last_became_peered":"2024-05-10T10:38:57.240018+0000","last_unstale":"2024-05-10T10:38:57.242330+0000","last_undegraded":"2024-05-10T10:38:57.242330+0000","last_fullsized":"2024-05-10T10:38:57.242330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233300+0000","last_change":"2024-05-10T10:38:57.233300+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233300+0000","last_undegraded":"2024-05-10T10:38:57.233300+0000","last_fullsized":"2024-05-10T10:38:57.233300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249361+0000","last_change":"2024-05-10T10:38:57.248178+0000","last_active":"2024-05-10T10:38:57.249361+0000","last_peered":"2024-05-10T10:38:57.249361+0000","last_clean":"2024-05-10T10:38:57.249361+0000","last_became_active":"2024-05-10T10:38:57.247953+0000","last_became_peered":"2024-05-10T10:38:57.247953+0000","last_unstale":"2024-05-10T10:38:57.249361+0000","last_undegraded":"2024-05-10T10:38:57.249361+0000","last_fullsized":"2024-05-10T10:38:57.249361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:30.018605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231497+0000","last_change":"2024-05-10T10:38:57.231497+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231497+0000","last_undegraded":"2024-05-10T10:38:57.231497+0000","last_fullsized":"2024-05-10T10:38:57.231497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240504+0000","last_change":"2024-05-10T10:38:57.235875+0000","last_active":"2024-05-10T10:38:57.240504+0000","last_peered":"2024-05-10T10:38:57.240504+0000","last_clean":"2024-05-10T10:38:57.240504+0000","last_became_active":"2024-05-10T10:38:57.235601+0000","last_became_peered":"2024-05-10T10:38:57.235601+0000","last_unstale":"2024-05-10T10:38:57.240504+0000","last_undegraded":"2024-05-10T10:38:57.240504+0000","last_fullsized":"2024-05-10T10:38:57.240504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:28:51.797568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227049+0000","last_change":"2024-05-10T10:38:57.227049+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227049+0000","last_undegraded":"2024-05-10T10:38:57.227049+0000","last_fullsized":"2024-05-10T10:38:57.227049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.249243+0000","last_change":"2024-05-10T10:38:57.248738+0000","last_active":"2024-05-10T10:38:57.249243+0000","last_peered":"2024-05-10T10:38:57.249243+0000","last_clean":"2024-05-10T10:38:57.249243+0000","last_became_active":"2024-05-10T10:38:57.248459+0000","last_became_peered":"2024-05-10T10:38:57.248459+0000","last_unstale":"2024-05-10T10:38:57.249243+0000","last_undegraded":"2024-05-10T10:38:57.249243+0000","last_fullsized":"2024-05-10T10:38:57.249243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:40:18.359499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227165+0000","last_change":"2024-05-10T10:38:57.227165+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227165+0000","last_undegraded":"2024-05-10T10:38:57.227165+0000","last_fullsized":"2024-05-10T10:38:57.227165+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.239432+0000","last_change":"2024-05-10T10:38:57.230406+0000","last_active":"2024-05-10T10:38:57.239432+0000","last_peered":"2024-05-10T10:38:57.239432+0000","last_clean":"2024-05-10T10:38:57.239432+0000","last_became_active":"2024-05-10T10:38:57.230065+0000","last_became_peered":"2024-05-10T10:38:57.230065+0000","last_unstale":"2024-05-10T10:38:57.239432+0000","last_undegraded":"2024-05-10T10:38:57.239432+0000","last_fullsized":"2024-05-10T10:38:57.239432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:37:46.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232029+0000","last_change":"2024-05-10T10:38:57.232029+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232029+0000","last_undegraded":"2024-05-10T10:38:57.232029+0000","last_fullsized":"2024-05-10T10:38:57.232029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243287+0000","last_change":"2024-05-10T10:38:57.241714+0000","last_active":"2024-05-10T10:38:57.243287+0000","last_peered":"2024-05-10T10:38:57.243287+0000","last_clean":"2024-05-10T10:38:57.243287+0000","last_became_active":"2024-05-10T10:38:57.241428+0000","last_became_peered":"2024-05-10T10:38:57.241428+0000","last_unstale":"2024-05-10T10:38:57.243287+0000","last_undegraded":"2024-05-10T10:38:57.243287+0000","last_fullsized":"2024-05-10T10:38:57.243287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:08.883771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.234030+0000","last_change":"2024-05-10T10:38:57.234030+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.234030+0000","last_undegraded":"2024-05-10T10:38:57.234030+0000","last_fullsized":"2024-05-10T10:38:57.234030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241462+0000","last_change":"2024-05-10T10:38:57.239328+0000","last_active":"2024-05-10T10:38:57.241462+0000","last_peered":"2024-05-10T10:38:57.241462+0000","last_clean":"2024-05-10T10:38:57.241462+0000","last_became_active":"2024-05-10T10:38:57.239042+0000","last_became_peered":"2024-05-10T10:38:57.239042+0000","last_unstale":"2024-05-10T10:38:57.241462+0000","last_undegraded":"2024-05-10T10:38:57.241462+0000","last_fullsized":"2024-05-10T10:38:57.241462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231035+0000","last_change":"2024-05-10T10:38:57.231035+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231035+0000","last_undegraded":"2024-05-10T10:38:57.231035+0000","last_fullsized":"2024-05-10T10:38:57.231035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.243561+0000","last_change":"2024-05-10T10:38:57.242120+0000","last_active":"2024-05-10T10:38:57.243561+0000","last_peered":"2024-05-10T10:38:57.243561+0000","last_clean":"2024-05-10T10:38:57.243561+0000","last_became_active":"2024-05-10T10:38:57.241862+0000","last_became_peered":"2024-05-10T10:38:57.241862+0000","last_unstale":"2024-05-10T10:38:57.243561+0000","last_undegraded":"2024-05-10T10:38:57.243561+0000","last_fullsized":"2024-05-10T10:38:57.243561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:50:32.971721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230683+0000","last_change":"2024-05-10T10:38:57.230683+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230683+0000","last_undegraded":"2024-05-10T10:38:57.230683+0000","last_fullsized":"2024-05-10T10:38:57.230683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.247818+0000","last_change":"2024-05-10T10:38:57.246576+0000","last_active":"2024-05-10T10:38:57.247818+0000","last_peered":"2024-05-10T10:38:57.247818+0000","last_clean":"2024-05-10T10:38:57.247818+0000","last_became_active":"2024-05-10T10:38:57.246353+0000","last_became_peered":"2024-05-10T10:38:57.246353+0000","last_unstale":"2024-05-10T10:38:57.247818+0000","last_undegraded":"2024-05-10T10:38:57.247818+0000","last_fullsized":"2024-05-10T10:38:57.247818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:14.665243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.227984+0000","last_change":"2024-05-10T10:38:57.227984+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.227984+0000","last_undegraded":"2024-05-10T10:38:57.227984+0000","last_fullsized":"2024-05-10T10:38:57.227984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.245546+0000","last_change":"2024-05-10T10:38:57.243728+0000","last_active":"2024-05-10T10:38:57.245546+0000","last_peered":"2024-05-10T10:38:57.245546+0000","last_clean":"2024-05-10T10:38:57.245546+0000","last_became_active":"2024-05-10T10:38:57.243406+0000","last_became_peered":"2024-05-10T10:38:57.243406+0000","last_unstale":"2024-05-10T10:38:57.245546+0000","last_undegraded":"2024-05-10T10:38:57.245546+0000","last_fullsized":"2024-05-10T10:38:57.245546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.232998+0000","last_change":"2024-05-10T10:38:57.232998+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.232998+0000","last_undegraded":"2024-05-10T10:38:57.232998+0000","last_fullsized":"2024-05-10T10:38:57.232998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.248051+0000","last_change":"2024-05-10T10:38:57.244580+0000","last_active":"2024-05-10T10:38:57.248051+0000","last_peered":"2024-05-10T10:38:57.248051+0000","last_clean":"2024-05-10T10:38:57.248051+0000","last_became_active":"2024-05-10T10:38:57.244271+0000","last_became_peered":"2024-05-10T10:38:57.244271+0000","last_unstale":"2024-05-10T10:38:57.248051+0000","last_undegraded":"2024-05-10T10:38:57.248051+0000","last_fullsized":"2024-05-10T10:38:57.248051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:37.653905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.231123+0000","last_change":"2024-05-10T10:38:57.231123+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.231123+0000","last_undegraded":"2024-05-10T10:38:57.231123+0000","last_fullsized":"2024-05-10T10:38:57.231123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.240531+0000","last_change":"2024-05-10T10:38:57.236911+0000","last_active":"2024-05-10T10:38:57.240531+0000","last_peered":"2024-05-10T10:38:57.240531+0000","last_clean":"2024-05-10T10:38:57.240531+0000","last_became_active":"2024-05-10T10:38:57.236571+0000","last_became_peered":"2024-05-10T10:38:57.236571+0000","last_unstale":"2024-05-10T10:38:57.240531+0000","last_undegraded":"2024-05-10T10:38:57.240531+0000","last_fullsized":"2024-05-10T10:38:57.240531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:14.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.233921+0000","last_change":"2024-05-10T10:38:57.233921+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.233921+0000","last_undegraded":"2024-05-10T10:38:57.233921+0000","last_fullsized":"2024-05-10T10:38:57.233921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244349+0000","last_change":"2024-05-10T10:38:57.243643+0000","last_active":"2024-05-10T10:38:57.244349+0000","last_peered":"2024-05-10T10:38:57.244349+0000","last_clean":"2024-05-10T10:38:57.244349+0000","last_became_active":"2024-05-10T10:38:57.243364+0000","last_became_peered":"2024-05-10T10:38:57.243364+0000","last_unstale":"2024-05-10T10:38:57.244349+0000","last_undegraded":"2024-05-10T10:38:57.244349+0000","last_fullsized":"2024-05-10T10:38:57.244349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.230113+0000","last_change":"2024-05-10T10:38:57.230113+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.230113+0000","last_undegraded":"2024-05-10T10:38:57.230113+0000","last_fullsized":"2024-05-10T10:38:57.230113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.241495+0000","last_change":"2024-05-10T10:38:57.239379+0000","last_active":"2024-05-10T10:38:57.241495+0000","last_peered":"2024-05-10T10:38:57.241495+0000","last_clean":"2024-05-10T10:38:57.241495+0000","last_became_active":"2024-05-10T10:38:57.239057+0000","last_became_peered":"2024-05-10T10:38:57.239057+0000","last_unstale":"2024-05-10T10:38:57.241495+0000","last_undegraded":"2024-05-10T10:38:57.241495+0000","last_fullsized":"2024-05-10T10:38:57.241495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:06.038345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-10T10:38:57.228473+0000","last_change":"2024-05-10T10:38:57.228473+0000","last_active":"2024-05-10T10:38:57.221344+0000","last_peered":"2024-05-10T10:38:57.221344+0000","last_clean":"2024-05-10T10:38:57.221344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:38:57.228473+0000","last_undegraded":"2024-05-10T10:38:57.228473+0000","last_fullsized":"2024-05-10T10:38:57.228473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.244095+0000","last_change":"2024-05-10T10:38:57.242780+0000","last_active":"2024-05-10T10:38:57.244095+0000","last_peered":"2024-05-10T10:38:57.244095+0000","last_clean":"2024-05-10T10:38:57.244095+0000","last_became_active":"2024-05-10T10:38:57.242494+0000","last_became_peered":"2024-05-10T10:38:57.242494+0000","last_unstale":"2024-05-10T10:38:57.244095+0000","last_undegraded":"2024-05-10T10:38:57.244095+0000","last_fullsized":"2024-05-10T10:38:57.244095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":242,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T10:38:57.228637+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:38:57.228637+0000","last_peered":"2024-05-10T10:38:57.228637+0000","last_clean":"2024-05-10T10:38:57.228637+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:38:57.228637+0000","last_undegraded":"2024-05-10T10:38:57.228637+0000","last_fullsized":"2024-05-10T10:38:57.228637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":476416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1066784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:39:03.874 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:39:03.873+0000 7f9f05921640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:39:06.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:39:06.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.908+0000 7fecd6d2c640 1 Processor -- start 2024-05-10T10:39:06.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.908+0000 7fecd6d2c640 1 -- start start 2024-05-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.909+0000 7fecd6d2c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fecd00d5cb0 0x7fecd00d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.909+0000 7fecd6d2c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00b45f0 con 0x7fecd0162750 2024-05-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.909+0000 7fecd6d2c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00b48d0 con 0x7fecd0162b30 2024-05-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.909+0000 7fecd6d2c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fecd00b4f20 con 0x7fecd00d5cb0 2024-05-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.909+0000 7feccffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fecd00d5cb0 0x7fecd00d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.909+0000 7fecd4ac8640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fecd0162750 0x7fecd00d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37442/0 (socket says 172.21.15.53:37442) 2024-05-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.909+0000 7fecd4ac8640 1 -- 172.21.15.53:0/2679779502 learned_addr learned my addr 172.21.15.53:0/2679779502 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.910+0000 7feccffff640 1 -- 172.21.15.53:0/2679779502 >> v1:172.21.15.135:6789/0 conn(0x7fecd0162b30 legacy=0x7fecd00b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.910+0000 7feccffff640 1 -- 172.21.15.53:0/2679779502 >> v1:172.21.15.53:6789/0 conn(0x7fecd0162750 legacy=0x7fecd00d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.910+0000 7feccf7fe640 1 -- 172.21.15.53:0/2679779502 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114709924 0 0) 0x7fecd00b45f0 con 0x7fecd0162750 2024-05-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.910+0000 7feccffff640 1 -- 172.21.15.53:0/2679779502 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00b4bb0 con 0x7fecd00d5cb0 2024-05-10T10:39:06.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7feccffff640 1 --2- 172.21.15.53:0/2679779502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd00d5cb0 0x7fecd00d3630 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fecb80097a0 tx=0x7fecb802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43332eb51db36ad9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:06.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7feccf7fe640 1 -- 172.21.15.53:0/2679779502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8037070 con 0x7fecd00d5cb0 2024-05-10T10:39:06.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7feccf7fe640 1 -- 172.21.15.53:0/2679779502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb802ae50 con 0x7fecd00d5cb0 2024-05-10T10:39:06.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7feccf7fe640 1 -- 172.21.15.53:0/2679779502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8032660 con 0x7fecd00d5cb0 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7fecd6d2c640 1 -- 172.21.15.53:0/2679779502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd00d5cb0 msgr2=0x7fecd00d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/2679779502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd00d5cb0 0x7fecd00d3630 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fecb80097a0 tx=0x7fecb802a140 comp rx=0 tx=0).stop 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7fecd6d2c640 1 -- 172.21.15.53:0/2679779502 shutdown_connections 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/2679779502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd00d5cb0 0x7fecd00d3630 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7fecd6d2c640 1 -- 172.21.15.53:0/2679779502 >> 172.21.15.53:0/2679779502 conn(0x7fecd009e3b0 msgr2=0x7fecd00d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7fecd6d2c640 1 -- 172.21.15.53:0/2679779502 shutdown_connections 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.911+0000 7fecd6d2c640 1 -- 172.21.15.53:0/2679779502 wait complete. 2024-05-10T10:39:06.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.913+0000 7fecd6d2c640 1 Processor -- start 2024-05-10T10:39:06.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.913+0000 7fecd6d2c640 1 -- start start 2024-05-10T10:39:06.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd6d2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd0162750 0x7fecd00cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd6d2c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd0162b30 0x7fecd00cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd6d2c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00cc660 0x7fecd00cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd6d2c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fecd00d3f40 con 0x7fecd0162750 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd6d2c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecd00d3dc0 con 0x7fecd00cc660 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd6d2c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fecd00d0fe0 con 0x7fecd0162b30 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd0162750 0x7fecd00cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd0162750 0x7fecd00cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60528/0 (socket says 172.21.15.53:60528) 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 -- 172.21.15.53:0/1045231315 learned_addr learned my addr 172.21.15.53:0/1045231315 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd52c9640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00cc660 0x7fecd00cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7feccffff640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd0162b30 0x7fecd00cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 -- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd0162b30 msgr2=0x7fecd00cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd0162b30 0x7fecd00cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:06.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 -- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00cc660 msgr2=0x7fecd00cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00cc660 0x7fecd00cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 -- 172.21.15.53:0/1045231315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00ca0f0 con 0x7fecd0162750 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd52c9640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00cc660 0x7fecd00cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd4ac8640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd0162750 0x7fecd00cc120 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fecc800c500 tx=0x7fecc800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7feccd7fa640 1 -- 172.21.15.53:0/1045231315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8018070 con 0x7fecd0162750 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7feccd7fa640 1 -- 172.21.15.53:0/1045231315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc8007580 con 0x7fecd0162750 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd00ca360 con 0x7fecd0162750 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.914+0000 7feccd7fa640 1 -- 172.21.15.53:0/1045231315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8004a20 con 0x7fecd0162750 2024-05-10T10:39:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.915+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd00ca920 con 0x7fecd0162750 2024-05-10T10:39:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.916+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd006a220 con 0x7fecd0162750 2024-05-10T10:39:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.916+0000 7feccd7fa640 1 -- 172.21.15.53:0/1045231315 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fecc801b070 con 0x7fecd0162750 2024-05-10T10:39:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.917+0000 7feccd7fa640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecb0076300 0x7fecb00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.917+0000 7feccd7fa640 1 -- 172.21.15.53:0/1045231315 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fecc8096a70 con 0x7fecd0162750 2024-05-10T10:39:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.917+0000 7feccffff640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecb0076300 0x7fecb00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.918+0000 7feccffff640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecb0076300 0x7fecb00787c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fecb8034000 tx=0x7fecb802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:06.921+0000 7feccd7fa640 1 -- 172.21.15.53:0/1045231315 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fecc80602d0 con 0x7fecd0162750 2024-05-10T10:39:07.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.110+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fecd00b42a0 con 0x7fecb0076300 2024-05-10T10:39:07.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.117+0000 7feccd7fa640 1 -- 172.21.15.53:0/1045231315 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252063 (secure 0 0 0) 0x7fecd00b42a0 con 0x7fecb0076300 2024-05-10T10:39:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:07.121 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecb0076300 msgr2=0x7fecb00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecb0076300 0x7fecb00787c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fecb8034000 tx=0x7fecb802e000 comp rx=0 tx=0).stop 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd0162750 msgr2=0x7fecd00cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd0162750 0x7fecd00cc120 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fecc800c500 tx=0x7fecc800c9c0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 shutdown_connections 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecb0076300 0x7fecb00787c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00cc660 0x7fecd00cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd0162b30 0x7fecd00cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 --2- 172.21.15.53:0/1045231315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecd0162750 0x7fecd00cc120 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 >> 172.21.15.53:0/1045231315 conn(0x7fecd009e3b0 msgr2=0x7fecd00b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 shutdown_connections 2024-05-10T10:39:07.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.120+0000 7fecd6d2c640 1 -- 172.21.15.53:0/1045231315 wait complete. 2024-05-10T10:39:07.132 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-05-10T10:39:06.237106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259100,"kb_used_data":9608,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715620,"statfs":{"total":773094113280,"available":772828794880,"internally_reserved":0,"allocated":9838592,"data_stored":5107756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002209"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742864+0000","last_change":"2024-05-10T10:38:58.271744+0000","last_active":"2024-05-10T10:38:59.742864+0000","last_peered":"2024-05-10T10:38:59.742864+0000","last_clean":"2024-05-10T10:38:59.742864+0000","last_became_active":"2024-05-10T10:38:58.271440+0000","last_became_peered":"2024-05-10T10:38:58.271440+0000","last_unstale":"2024-05-10T10:38:59.742864+0000","last_undegraded":"2024-05-10T10:38:59.742864+0000","last_fullsized":"2024-05-10T10:38:59.742864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:35:47.913221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252938+0000","last_change":"2024-05-10T10:38:58.267656+0000","last_active":"2024-05-10T10:38:59.252938+0000","last_peered":"2024-05-10T10:38:59.252938+0000","last_clean":"2024-05-10T10:38:59.252938+0000","last_became_active":"2024-05-10T10:38:58.267367+0000","last_became_peered":"2024-05-10T10:38:58.267367+0000","last_unstale":"2024-05-10T10:38:59.252938+0000","last_undegraded":"2024-05-10T10:38:59.252938+0000","last_fullsized":"2024-05-10T10:38:59.252938+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:19:49.391063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743142+0000","last_change":"2024-05-10T10:38:58.261787+0000","last_active":"2024-05-10T10:38:59.743142+0000","last_peered":"2024-05-10T10:38:59.743142+0000","last_clean":"2024-05-10T10:38:59.743142+0000","last_became_active":"2024-05-10T10:38:58.261486+0000","last_became_peered":"2024-05-10T10:38:58.261486+0000","last_unstale":"2024-05-10T10:38:59.743142+0000","last_undegraded":"2024-05-10T10:38:59.743142+0000","last_fullsized":"2024-05-10T10:38:59.743142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:19.294869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742170+0000","last_change":"2024-05-10T10:38:58.256249+0000","last_active":"2024-05-10T10:38:59.742170+0000","last_peered":"2024-05-10T10:38:59.742170+0000","last_clean":"2024-05-10T10:38:59.742170+0000","last_became_active":"2024-05-10T10:38:58.255462+0000","last_became_peered":"2024-05-10T10:38:58.255462+0000","last_unstale":"2024-05-10T10:38:59.742170+0000","last_undegraded":"2024-05-10T10:38:59.742170+0000","last_fullsized":"2024-05-10T10:38:59.742170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:20.210006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741886+0000","last_change":"2024-05-10T10:38:58.248598+0000","last_active":"2024-05-10T10:38:59.741886+0000","last_peered":"2024-05-10T10:38:59.741886+0000","last_clean":"2024-05-10T10:38:59.741886+0000","last_became_active":"2024-05-10T10:38:58.248315+0000","last_became_peered":"2024-05-10T10:38:58.248315+0000","last_unstale":"2024-05-10T10:38:59.741886+0000","last_undegraded":"2024-05-10T10:38:59.741886+0000","last_fullsized":"2024-05-10T10:38:59.741886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:42.944065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742558+0000","last_change":"2024-05-10T10:38:58.270661+0000","last_active":"2024-05-10T10:38:59.742558+0000","last_peered":"2024-05-10T10:38:59.742558+0000","last_clean":"2024-05-10T10:38:59.742558+0000","last_became_active":"2024-05-10T10:38:58.270100+0000","last_became_peered":"2024-05-10T10:38:58.270100+0000","last_unstale":"2024-05-10T10:38:59.742558+0000","last_undegraded":"2024-05-10T10:38:59.742558+0000","last_fullsized":"2024-05-10T10:38:59.742558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:54.746830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252807+0000","last_change":"2024-05-10T10:38:58.267494+0000","last_active":"2024-05-10T10:38:59.252807+0000","last_peered":"2024-05-10T10:38:59.252807+0000","last_clean":"2024-05-10T10:38:59.252807+0000","last_became_active":"2024-05-10T10:38:58.267199+0000","last_became_peered":"2024-05-10T10:38:58.267199+0000","last_unstale":"2024-05-10T10:38:59.252807+0000","last_undegraded":"2024-05-10T10:38:59.252807+0000","last_fullsized":"2024-05-10T10:38:59.252807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:19.294869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741876+0000","last_change":"2024-05-10T10:38:58.255432+0000","last_active":"2024-05-10T10:38:59.741876+0000","last_peered":"2024-05-10T10:38:59.741876+0000","last_clean":"2024-05-10T10:38:59.741876+0000","last_became_active":"2024-05-10T10:38:58.255158+0000","last_became_peered":"2024-05-10T10:38:58.255158+0000","last_unstale":"2024-05-10T10:38:59.741876+0000","last_undegraded":"2024-05-10T10:38:59.741876+0000","last_fullsized":"2024-05-10T10:38:59.741876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:06:54.131965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741888+0000","last_change":"2024-05-10T10:38:58.247820+0000","last_active":"2024-05-10T10:38:59.741888+0000","last_peered":"2024-05-10T10:38:59.741888+0000","last_clean":"2024-05-10T10:38:59.741888+0000","last_became_active":"2024-05-10T10:38:58.247525+0000","last_became_peered":"2024-05-10T10:38:58.247525+0000","last_unstale":"2024-05-10T10:38:59.741888+0000","last_undegraded":"2024-05-10T10:38:59.741888+0000","last_fullsized":"2024-05-10T10:38:59.741888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:26.933519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252187+0000","last_change":"2024-05-10T10:38:58.268442+0000","last_active":"2024-05-10T10:38:59.252187+0000","last_peered":"2024-05-10T10:38:59.252187+0000","last_clean":"2024-05-10T10:38:59.252187+0000","last_became_active":"2024-05-10T10:38:58.268183+0000","last_became_peered":"2024-05-10T10:38:58.268183+0000","last_unstale":"2024-05-10T10:38:59.252187+0000","last_undegraded":"2024-05-10T10:38:59.252187+0000","last_fullsized":"2024-05-10T10:38:59.252187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:37.647524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742202+0000","last_change":"2024-05-10T10:38:58.269065+0000","last_active":"2024-05-10T10:38:59.742202+0000","last_peered":"2024-05-10T10:38:59.742202+0000","last_clean":"2024-05-10T10:38:59.742202+0000","last_became_active":"2024-05-10T10:38:58.268881+0000","last_became_peered":"2024-05-10T10:38:58.268881+0000","last_unstale":"2024-05-10T10:38:59.742202+0000","last_undegraded":"2024-05-10T10:38:59.742202+0000","last_fullsized":"2024-05-10T10:38:59.742202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:40.839086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.251960+0000","last_change":"2024-05-10T10:38:58.258650+0000","last_active":"2024-05-10T10:38:59.251960+0000","last_peered":"2024-05-10T10:38:59.251960+0000","last_clean":"2024-05-10T10:38:59.251960+0000","last_became_active":"2024-05-10T10:38:58.258318+0000","last_became_peered":"2024-05-10T10:38:58.258318+0000","last_unstale":"2024-05-10T10:38:59.251960+0000","last_undegraded":"2024-05-10T10:38:59.251960+0000","last_fullsized":"2024-05-10T10:38:59.251960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:38:25.292893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742272+0000","last_change":"2024-05-10T10:38:58.255319+0000","last_active":"2024-05-10T10:38:59.742272+0000","last_peered":"2024-05-10T10:38:59.742272+0000","last_clean":"2024-05-10T10:38:59.742272+0000","last_became_active":"2024-05-10T10:38:58.255036+0000","last_became_peered":"2024-05-10T10:38:58.255036+0000","last_unstale":"2024-05-10T10:38:59.742272+0000","last_undegraded":"2024-05-10T10:38:59.742272+0000","last_fullsized":"2024-05-10T10:38:59.742272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:07.040095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741318+0000","last_change":"2024-05-10T10:38:58.256387+0000","last_active":"2024-05-10T10:38:59.741318+0000","last_peered":"2024-05-10T10:38:59.741318+0000","last_clean":"2024-05-10T10:38:59.741318+0000","last_became_active":"2024-05-10T10:38:58.254395+0000","last_became_peered":"2024-05-10T10:38:58.254395+0000","last_unstale":"2024-05-10T10:38:59.741318+0000","last_undegraded":"2024-05-10T10:38:59.741318+0000","last_fullsized":"2024-05-10T10:38:59.741318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:15.037318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742474+0000","last_change":"2024-05-10T10:38:58.267625+0000","last_active":"2024-05-10T10:38:59.742474+0000","last_peered":"2024-05-10T10:38:59.742474+0000","last_clean":"2024-05-10T10:38:59.742474+0000","last_became_active":"2024-05-10T10:38:58.267349+0000","last_became_peered":"2024-05-10T10:38:58.267349+0000","last_unstale":"2024-05-10T10:38:59.742474+0000","last_undegraded":"2024-05-10T10:38:59.742474+0000","last_fullsized":"2024-05-10T10:38:59.742474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:42.944065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743272+0000","last_change":"2024-05-10T10:38:58.262607+0000","last_active":"2024-05-10T10:38:59.743272+0000","last_peered":"2024-05-10T10:38:59.743272+0000","last_clean":"2024-05-10T10:38:59.743272+0000","last_became_active":"2024-05-10T10:38:58.262118+0000","last_became_peered":"2024-05-10T10:38:58.262118+0000","last_unstale":"2024-05-10T10:38:59.743272+0000","last_undegraded":"2024-05-10T10:38:59.743272+0000","last_fullsized":"2024-05-10T10:38:59.743272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:27.271826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742440+0000","last_change":"2024-05-10T10:38:58.260998+0000","last_active":"2024-05-10T10:38:59.742440+0000","last_peered":"2024-05-10T10:38:59.742440+0000","last_clean":"2024-05-10T10:38:59.742440+0000","last_became_active":"2024-05-10T10:38:58.260743+0000","last_became_peered":"2024-05-10T10:38:58.260743+0000","last_unstale":"2024-05-10T10:38:59.742440+0000","last_undegraded":"2024-05-10T10:38:59.742440+0000","last_fullsized":"2024-05-10T10:38:59.742440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:23:07.372563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742934+0000","last_change":"2024-05-10T10:38:58.264697+0000","last_active":"2024-05-10T10:38:59.742934+0000","last_peered":"2024-05-10T10:38:59.742934+0000","last_clean":"2024-05-10T10:38:59.742934+0000","last_became_active":"2024-05-10T10:38:58.264431+0000","last_became_peered":"2024-05-10T10:38:58.264431+0000","last_unstale":"2024-05-10T10:38:59.742934+0000","last_undegraded":"2024-05-10T10:38:59.742934+0000","last_fullsized":"2024-05-10T10:38:59.742934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:15.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742250+0000","last_change":"2024-05-10T10:38:58.254113+0000","last_active":"2024-05-10T10:38:59.742250+0000","last_peered":"2024-05-10T10:38:59.742250+0000","last_clean":"2024-05-10T10:38:59.742250+0000","last_became_active":"2024-05-10T10:38:58.251621+0000","last_became_peered":"2024-05-10T10:38:58.251621+0000","last_unstale":"2024-05-10T10:38:59.742250+0000","last_undegraded":"2024-05-10T10:38:59.742250+0000","last_fullsized":"2024-05-10T10:38:59.742250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:05.217908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742355+0000","last_change":"2024-05-10T10:38:58.259601+0000","last_active":"2024-05-10T10:38:59.742355+0000","last_peered":"2024-05-10T10:38:59.742355+0000","last_clean":"2024-05-10T10:38:59.742355+0000","last_became_active":"2024-05-10T10:38:58.259335+0000","last_became_peered":"2024-05-10T10:38:58.259335+0000","last_unstale":"2024-05-10T10:38:59.742355+0000","last_undegraded":"2024-05-10T10:38:59.742355+0000","last_fullsized":"2024-05-10T10:38:59.742355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:38.655656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741708+0000","last_change":"2024-05-10T10:38:58.246945+0000","last_active":"2024-05-10T10:38:59.741708+0000","last_peered":"2024-05-10T10:38:59.741708+0000","last_clean":"2024-05-10T10:38:59.741708+0000","last_became_active":"2024-05-10T10:38:58.246018+0000","last_became_peered":"2024-05-10T10:38:58.246018+0000","last_unstale":"2024-05-10T10:38:59.741708+0000","last_undegraded":"2024-05-10T10:38:59.741708+0000","last_fullsized":"2024-05-10T10:38:59.741708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:20.210006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741492+0000","last_change":"2024-05-10T10:38:58.254667+0000","last_active":"2024-05-10T10:38:59.741492+0000","last_peered":"2024-05-10T10:38:59.741492+0000","last_clean":"2024-05-10T10:38:59.741492+0000","last_became_active":"2024-05-10T10:38:58.254379+0000","last_became_peered":"2024-05-10T10:38:58.254379+0000","last_unstale":"2024-05-10T10:38:59.741492+0000","last_undegraded":"2024-05-10T10:38:59.741492+0000","last_fullsized":"2024-05-10T10:38:59.741492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:06:54.131965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741217+0000","last_change":"2024-05-10T10:38:58.256718+0000","last_active":"2024-05-10T10:38:59.741217+0000","last_peered":"2024-05-10T10:38:59.741217+0000","last_clean":"2024-05-10T10:38:59.741217+0000","last_became_active":"2024-05-10T10:38:58.256424+0000","last_became_peered":"2024-05-10T10:38:58.256424+0000","last_unstale":"2024-05-10T10:38:59.741217+0000","last_undegraded":"2024-05-10T10:38:59.741217+0000","last_fullsized":"2024-05-10T10:38:59.741217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:28:52.799319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741925+0000","last_change":"2024-05-10T10:38:58.259679+0000","last_active":"2024-05-10T10:38:59.741925+0000","last_peered":"2024-05-10T10:38:59.741925+0000","last_clean":"2024-05-10T10:38:59.741925+0000","last_became_active":"2024-05-10T10:38:58.257935+0000","last_became_peered":"2024-05-10T10:38:58.257935+0000","last_unstale":"2024-05-10T10:38:59.741925+0000","last_undegraded":"2024-05-10T10:38:59.741925+0000","last_fullsized":"2024-05-10T10:38:59.741925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:48:05.257978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253076+0000","last_change":"2024-05-10T10:38:58.254245+0000","last_active":"2024-05-10T10:38:59.253076+0000","last_peered":"2024-05-10T10:38:59.253076+0000","last_clean":"2024-05-10T10:38:59.253076+0000","last_became_active":"2024-05-10T10:38:58.253980+0000","last_became_peered":"2024-05-10T10:38:58.253980+0000","last_unstale":"2024-05-10T10:38:59.253076+0000","last_undegraded":"2024-05-10T10:38:59.253076+0000","last_fullsized":"2024-05-10T10:38:59.253076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:30.109999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741678+0000","last_change":"2024-05-10T10:38:58.251028+0000","last_active":"2024-05-10T10:38:59.741678+0000","last_peered":"2024-05-10T10:38:59.741678+0000","last_clean":"2024-05-10T10:38:59.741678+0000","last_became_active":"2024-05-10T10:38:58.250744+0000","last_became_peered":"2024-05-10T10:38:58.250744+0000","last_unstale":"2024-05-10T10:38:59.741678+0000","last_undegraded":"2024-05-10T10:38:59.741678+0000","last_fullsized":"2024-05-10T10:38:59.741678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:24:06.243530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.251841+0000","last_change":"2024-05-10T10:38:58.252593+0000","last_active":"2024-05-10T10:38:59.251841+0000","last_peered":"2024-05-10T10:38:59.251841+0000","last_clean":"2024-05-10T10:38:59.251841+0000","last_became_active":"2024-05-10T10:38:58.252312+0000","last_became_peered":"2024-05-10T10:38:58.252312+0000","last_unstale":"2024-05-10T10:38:59.251841+0000","last_undegraded":"2024-05-10T10:38:59.251841+0000","last_fullsized":"2024-05-10T10:38:59.251841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:31.432456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742748+0000","last_change":"2024-05-10T10:38:58.251155+0000","last_active":"2024-05-10T10:38:59.742748+0000","last_peered":"2024-05-10T10:38:59.742748+0000","last_clean":"2024-05-10T10:38:59.742748+0000","last_became_active":"2024-05-10T10:38:58.250863+0000","last_became_peered":"2024-05-10T10:38:58.250863+0000","last_unstale":"2024-05-10T10:38:59.742748+0000","last_undegraded":"2024-05-10T10:38:59.742748+0000","last_fullsized":"2024-05-10T10:38:59.742748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:05.217908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253415+0000","last_change":"2024-05-10T10:38:58.247045+0000","last_active":"2024-05-10T10:38:59.253415+0000","last_peered":"2024-05-10T10:38:59.253415+0000","last_clean":"2024-05-10T10:38:59.253415+0000","last_became_active":"2024-05-10T10:38:58.246789+0000","last_became_peered":"2024-05-10T10:38:58.246789+0000","last_unstale":"2024-05-10T10:38:59.253415+0000","last_undegraded":"2024-05-10T10:38:59.253415+0000","last_fullsized":"2024-05-10T10:38:59.253415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:33:47.218640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741684+0000","last_change":"2024-05-10T10:38:58.248557+0000","last_active":"2024-05-10T10:38:59.741684+0000","last_peered":"2024-05-10T10:38:59.741684+0000","last_clean":"2024-05-10T10:38:59.741684+0000","last_became_active":"2024-05-10T10:38:58.248276+0000","last_became_peered":"2024-05-10T10:38:58.248276+0000","last_unstale":"2024-05-10T10:38:59.741684+0000","last_undegraded":"2024-05-10T10:38:59.741684+0000","last_fullsized":"2024-05-10T10:38:59.741684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:00:30.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252705+0000","last_change":"2024-05-10T10:38:58.252297+0000","last_active":"2024-05-10T10:38:59.252705+0000","last_peered":"2024-05-10T10:38:59.252705+0000","last_clean":"2024-05-10T10:38:59.252705+0000","last_became_active":"2024-05-10T10:38:58.252008+0000","last_became_peered":"2024-05-10T10:38:58.252008+0000","last_unstale":"2024-05-10T10:38:59.252705+0000","last_undegraded":"2024-05-10T10:38:59.252705+0000","last_fullsized":"2024-05-10T10:38:59.252705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:45:56.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743366+0000","last_change":"2024-05-10T10:38:58.262374+0000","last_active":"2024-05-10T10:38:59.743366+0000","last_peered":"2024-05-10T10:38:59.743366+0000","last_clean":"2024-05-10T10:38:59.743366+0000","last_became_active":"2024-05-10T10:38:58.261754+0000","last_became_peered":"2024-05-10T10:38:58.261754+0000","last_unstale":"2024-05-10T10:38:59.743366+0000","last_undegraded":"2024-05-10T10:38:59.743366+0000","last_fullsized":"2024-05-10T10:38:59.743366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:34:33.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741893+0000","last_change":"2024-05-10T10:38:58.265913+0000","last_active":"2024-05-10T10:38:59.741893+0000","last_peered":"2024-05-10T10:38:59.741893+0000","last_clean":"2024-05-10T10:38:59.741893+0000","last_became_active":"2024-05-10T10:38:58.265601+0000","last_became_peered":"2024-05-10T10:38:58.265601+0000","last_unstale":"2024-05-10T10:38:59.741893+0000","last_undegraded":"2024-05-10T10:38:59.741893+0000","last_fullsized":"2024-05-10T10:38:59.741893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:33:47.218640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741623+0000","last_change":"2024-05-10T10:38:57.244889+0000","last_active":"2024-05-10T10:38:59.741623+0000","last_peered":"2024-05-10T10:38:59.741623+0000","last_clean":"2024-05-10T10:38:59.741623+0000","last_became_active":"2024-05-10T10:38:57.244439+0000","last_became_peered":"2024-05-10T10:38:57.244439+0000","last_unstale":"2024-05-10T10:38:59.741623+0000","last_undegraded":"2024-05-10T10:38:59.741623+0000","last_fullsized":"2024-05-10T10:38:59.741623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:22.836570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742893+0000","last_change":"2024-05-10T10:38:58.267086+0000","last_active":"2024-05-10T10:38:59.742893+0000","last_peered":"2024-05-10T10:38:59.742893+0000","last_clean":"2024-05-10T10:38:59.742893+0000","last_became_active":"2024-05-10T10:38:58.265319+0000","last_became_peered":"2024-05-10T10:38:58.265319+0000","last_unstale":"2024-05-10T10:38:59.742893+0000","last_undegraded":"2024-05-10T10:38:59.742893+0000","last_fullsized":"2024-05-10T10:38:59.742893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:16:03.769037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741970+0000","last_change":"2024-05-10T10:38:57.241164+0000","last_active":"2024-05-10T10:38:59.741970+0000","last_peered":"2024-05-10T10:38:59.741970+0000","last_clean":"2024-05-10T10:38:59.741970+0000","last_became_active":"2024-05-10T10:38:57.240896+0000","last_became_peered":"2024-05-10T10:38:57.240896+0000","last_unstale":"2024-05-10T10:38:59.741970+0000","last_undegraded":"2024-05-10T10:38:59.741970+0000","last_fullsized":"2024-05-10T10:38:59.741970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:00:23.703664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252988+0000","last_change":"2024-05-10T10:38:58.255467+0000","last_active":"2024-05-10T10:38:59.252988+0000","last_peered":"2024-05-10T10:38:59.252988+0000","last_clean":"2024-05-10T10:38:59.252988+0000","last_became_active":"2024-05-10T10:38:58.255175+0000","last_became_peered":"2024-05-10T10:38:58.255175+0000","last_unstale":"2024-05-10T10:38:59.252988+0000","last_undegraded":"2024-05-10T10:38:59.252988+0000","last_fullsized":"2024-05-10T10:38:59.252988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:38:04.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253328+0000","last_change":"2024-05-10T10:38:57.238335+0000","last_active":"2024-05-10T10:38:59.253328+0000","last_peered":"2024-05-10T10:38:59.253328+0000","last_clean":"2024-05-10T10:38:59.253328+0000","last_became_active":"2024-05-10T10:38:57.238013+0000","last_became_peered":"2024-05-10T10:38:57.238013+0000","last_unstale":"2024-05-10T10:38:59.253328+0000","last_undegraded":"2024-05-10T10:38:59.253328+0000","last_fullsized":"2024-05-10T10:38:59.253328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:07.321406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742603+0000","last_change":"2024-05-10T10:38:58.259854+0000","last_active":"2024-05-10T10:38:59.742603+0000","last_peered":"2024-05-10T10:38:59.742603+0000","last_clean":"2024-05-10T10:38:59.742603+0000","last_became_active":"2024-05-10T10:38:58.259525+0000","last_became_peered":"2024-05-10T10:38:58.259525+0000","last_unstale":"2024-05-10T10:38:59.742603+0000","last_undegraded":"2024-05-10T10:38:59.742603+0000","last_fullsized":"2024-05-10T10:38:59.742603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:03:55.971693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741910+0000","last_change":"2024-05-10T10:38:57.245097+0000","last_active":"2024-05-10T10:38:59.741910+0000","last_peered":"2024-05-10T10:38:59.741910+0000","last_clean":"2024-05-10T10:38:59.741910+0000","last_became_active":"2024-05-10T10:38:57.244782+0000","last_became_peered":"2024-05-10T10:38:57.244782+0000","last_unstale":"2024-05-10T10:38:59.741910+0000","last_undegraded":"2024-05-10T10:38:59.741910+0000","last_fullsized":"2024-05-10T10:38:59.741910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:14.022836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742053+0000","last_change":"2024-05-10T10:38:58.259023+0000","last_active":"2024-05-10T10:38:59.742053+0000","last_peered":"2024-05-10T10:38:59.742053+0000","last_clean":"2024-05-10T10:38:59.742053+0000","last_became_active":"2024-05-10T10:38:58.258762+0000","last_became_peered":"2024-05-10T10:38:58.258762+0000","last_unstale":"2024-05-10T10:38:59.742053+0000","last_undegraded":"2024-05-10T10:38:59.742053+0000","last_fullsized":"2024-05-10T10:38:59.742053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:24:06.243530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742115+0000","last_change":"2024-05-10T10:38:57.249546+0000","last_active":"2024-05-10T10:38:59.742115+0000","last_peered":"2024-05-10T10:38:59.742115+0000","last_clean":"2024-05-10T10:38:59.742115+0000","last_became_active":"2024-05-10T10:38:57.249316+0000","last_became_peered":"2024-05-10T10:38:57.249316+0000","last_unstale":"2024-05-10T10:38:59.742115+0000","last_undegraded":"2024-05-10T10:38:59.742115+0000","last_fullsized":"2024-05-10T10:38:59.742115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:30.296069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741354+0000","last_change":"2024-05-10T10:38:58.257488+0000","last_active":"2024-05-10T10:38:59.741354+0000","last_peered":"2024-05-10T10:38:59.741354+0000","last_clean":"2024-05-10T10:38:59.741354+0000","last_became_active":"2024-05-10T10:38:58.257123+0000","last_became_peered":"2024-05-10T10:38:58.257123+0000","last_unstale":"2024-05-10T10:38:59.741354+0000","last_undegraded":"2024-05-10T10:38:59.741354+0000","last_fullsized":"2024-05-10T10:38:59.741354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:15.435237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743470+0000","last_change":"2024-05-10T10:38:57.250917+0000","last_active":"2024-05-10T10:38:59.743470+0000","last_peered":"2024-05-10T10:38:59.743470+0000","last_clean":"2024-05-10T10:38:59.743470+0000","last_became_active":"2024-05-10T10:38:57.250631+0000","last_became_peered":"2024-05-10T10:38:57.250631+0000","last_unstale":"2024-05-10T10:38:59.743470+0000","last_undegraded":"2024-05-10T10:38:59.743470+0000","last_fullsized":"2024-05-10T10:38:59.743470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:57:14.022836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742714+0000","last_change":"2024-05-10T10:38:58.270437+0000","last_active":"2024-05-10T10:38:59.742714+0000","last_peered":"2024-05-10T10:38:59.742714+0000","last_clean":"2024-05-10T10:38:59.742714+0000","last_became_active":"2024-05-10T10:38:58.269761+0000","last_became_peered":"2024-05-10T10:38:58.269761+0000","last_unstale":"2024-05-10T10:38:59.742714+0000","last_undegraded":"2024-05-10T10:38:59.742714+0000","last_fullsized":"2024-05-10T10:38:59.742714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:51:04.283164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742682+0000","last_change":"2024-05-10T10:38:57.234005+0000","last_active":"2024-05-10T10:38:59.742682+0000","last_peered":"2024-05-10T10:38:59.742682+0000","last_clean":"2024-05-10T10:38:59.742682+0000","last_became_active":"2024-05-10T10:38:57.233624+0000","last_became_peered":"2024-05-10T10:38:57.233624+0000","last_unstale":"2024-05-10T10:38:59.742682+0000","last_undegraded":"2024-05-10T10:38:59.742682+0000","last_fullsized":"2024-05-10T10:38:59.742682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:01.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742598+0000","last_change":"2024-05-10T10:38:58.255959+0000","last_active":"2024-05-10T10:38:59.742598+0000","last_peered":"2024-05-10T10:38:59.742598+0000","last_clean":"2024-05-10T10:38:59.742598+0000","last_became_active":"2024-05-10T10:38:58.255692+0000","last_became_peered":"2024-05-10T10:38:58.255692+0000","last_unstale":"2024-05-10T10:38:59.742598+0000","last_undegraded":"2024-05-10T10:38:59.742598+0000","last_fullsized":"2024-05-10T10:38:59.742598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:40.839086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253061+0000","last_change":"2024-05-10T10:38:57.243368+0000","last_active":"2024-05-10T10:38:59.253061+0000","last_peered":"2024-05-10T10:38:59.253061+0000","last_clean":"2024-05-10T10:38:59.253061+0000","last_became_active":"2024-05-10T10:38:57.243101+0000","last_became_peered":"2024-05-10T10:38:57.243101+0000","last_unstale":"2024-05-10T10:38:59.253061+0000","last_undegraded":"2024-05-10T10:38:59.253061+0000","last_fullsized":"2024-05-10T10:38:59.253061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:54.482891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741936+0000","last_change":"2024-05-10T10:38:58.258487+0000","last_active":"2024-05-10T10:38:59.741936+0000","last_peered":"2024-05-10T10:38:59.741936+0000","last_clean":"2024-05-10T10:38:59.741936+0000","last_became_active":"2024-05-10T10:38:58.258219+0000","last_became_peered":"2024-05-10T10:38:58.258219+0000","last_unstale":"2024-05-10T10:38:59.741936+0000","last_undegraded":"2024-05-10T10:38:59.741936+0000","last_fullsized":"2024-05-10T10:38:59.741936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:12:07.040095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252974+0000","last_change":"2024-05-10T10:38:57.244172+0000","last_active":"2024-05-10T10:38:59.252974+0000","last_peered":"2024-05-10T10:38:59.252974+0000","last_clean":"2024-05-10T10:38:59.252974+0000","last_became_active":"2024-05-10T10:38:57.243827+0000","last_became_peered":"2024-05-10T10:38:57.243827+0000","last_unstale":"2024-05-10T10:38:59.252974+0000","last_undegraded":"2024-05-10T10:38:59.252974+0000","last_fullsized":"2024-05-10T10:38:59.252974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:00:29.286197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253278+0000","last_change":"2024-05-10T10:38:58.253839+0000","last_active":"2024-05-10T10:38:59.253278+0000","last_peered":"2024-05-10T10:38:59.253278+0000","last_clean":"2024-05-10T10:38:59.253278+0000","last_became_active":"2024-05-10T10:38:58.253516+0000","last_became_peered":"2024-05-10T10:38:58.253516+0000","last_unstale":"2024-05-10T10:38:59.253278+0000","last_undegraded":"2024-05-10T10:38:59.253278+0000","last_fullsized":"2024-05-10T10:38:59.253278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:48.181211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743145+0000","last_change":"2024-05-10T10:38:57.242608+0000","last_active":"2024-05-10T10:38:59.743145+0000","last_peered":"2024-05-10T10:38:59.743145+0000","last_clean":"2024-05-10T10:38:59.743145+0000","last_became_active":"2024-05-10T10:38:57.242307+0000","last_became_peered":"2024-05-10T10:38:57.242307+0000","last_unstale":"2024-05-10T10:38:59.743145+0000","last_undegraded":"2024-05-10T10:38:59.743145+0000","last_fullsized":"2024-05-10T10:38:59.743145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:04.216157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743294+0000","last_change":"2024-05-10T10:38:58.269804+0000","last_active":"2024-05-10T10:38:59.743294+0000","last_peered":"2024-05-10T10:38:59.743294+0000","last_clean":"2024-05-10T10:38:59.743294+0000","last_became_active":"2024-05-10T10:38:58.269552+0000","last_became_peered":"2024-05-10T10:38:58.269552+0000","last_unstale":"2024-05-10T10:38:59.743294+0000","last_undegraded":"2024-05-10T10:38:59.743294+0000","last_fullsized":"2024-05-10T10:38:59.743294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:38:04.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742325+0000","last_change":"2024-05-10T10:38:57.246632+0000","last_active":"2024-05-10T10:38:59.742325+0000","last_peered":"2024-05-10T10:38:59.742325+0000","last_clean":"2024-05-10T10:38:59.742325+0000","last_became_active":"2024-05-10T10:38:57.246417+0000","last_became_peered":"2024-05-10T10:38:57.246417+0000","last_unstale":"2024-05-10T10:38:59.742325+0000","last_undegraded":"2024-05-10T10:38:59.742325+0000","last_fullsized":"2024-05-10T10:38:59.742325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:49:46.486328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742111+0000","last_change":"2024-05-10T10:38:58.253319+0000","last_active":"2024-05-10T10:38:59.742111+0000","last_peered":"2024-05-10T10:38:59.742111+0000","last_clean":"2024-05-10T10:38:59.742111+0000","last_became_active":"2024-05-10T10:38:58.253021+0000","last_became_peered":"2024-05-10T10:38:58.253021+0000","last_unstale":"2024-05-10T10:38:59.742111+0000","last_undegraded":"2024-05-10T10:38:59.742111+0000","last_fullsized":"2024-05-10T10:38:59.742111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:00:30.287948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743530+0000","last_change":"2024-05-10T10:38:57.238700+0000","last_active":"2024-05-10T10:38:59.743530+0000","last_peered":"2024-05-10T10:38:59.743530+0000","last_clean":"2024-05-10T10:38:59.743530+0000","last_became_active":"2024-05-10T10:38:57.238416+0000","last_became_peered":"2024-05-10T10:38:57.238416+0000","last_unstale":"2024-05-10T10:38:59.743530+0000","last_undegraded":"2024-05-10T10:38:59.743530+0000","last_fullsized":"2024-05-10T10:38:59.743530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:04.216157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742265+0000","last_change":"2024-05-10T10:38:58.254762+0000","last_active":"2024-05-10T10:38:59.742265+0000","last_peered":"2024-05-10T10:38:59.742265+0000","last_clean":"2024-05-10T10:38:59.742265+0000","last_became_active":"2024-05-10T10:38:58.254441+0000","last_became_peered":"2024-05-10T10:38:58.254441+0000","last_unstale":"2024-05-10T10:38:59.742265+0000","last_undegraded":"2024-05-10T10:38:59.742265+0000","last_fullsized":"2024-05-10T10:38:59.742265+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:26.933519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253387+0000","last_change":"2024-05-10T10:38:57.233569+0000","last_active":"2024-05-10T10:38:59.253387+0000","last_peered":"2024-05-10T10:38:59.253387+0000","last_clean":"2024-05-10T10:38:59.253387+0000","last_became_active":"2024-05-10T10:38:57.233304+0000","last_became_peered":"2024-05-10T10:38:57.233304+0000","last_unstale":"2024-05-10T10:38:59.253387+0000","last_undegraded":"2024-05-10T10:38:59.253387+0000","last_fullsized":"2024-05-10T10:38:59.253387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:23:04.216157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741452+0000","last_change":"2024-05-10T10:38:58.249578+0000","last_active":"2024-05-10T10:38:59.741452+0000","last_peered":"2024-05-10T10:38:59.741452+0000","last_clean":"2024-05-10T10:38:59.741452+0000","last_became_active":"2024-05-10T10:38:58.249300+0000","last_became_peered":"2024-05-10T10:38:58.249300+0000","last_unstale":"2024-05-10T10:38:59.741452+0000","last_undegraded":"2024-05-10T10:38:59.741452+0000","last_fullsized":"2024-05-10T10:38:59.741452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:15.435237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742807+0000","last_change":"2024-05-10T10:38:57.242521+0000","last_active":"2024-05-10T10:38:59.742807+0000","last_peered":"2024-05-10T10:38:59.742807+0000","last_clean":"2024-05-10T10:38:59.742807+0000","last_became_active":"2024-05-10T10:38:57.242195+0000","last_became_peered":"2024-05-10T10:38:57.242195+0000","last_unstale":"2024-05-10T10:38:59.742807+0000","last_undegraded":"2024-05-10T10:38:59.742807+0000","last_fullsized":"2024-05-10T10:38:59.742807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:39.837335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.251756+0000","last_change":"2024-05-10T10:38:58.266286+0000","last_active":"2024-05-10T10:38:59.251756+0000","last_peered":"2024-05-10T10:38:59.251756+0000","last_clean":"2024-05-10T10:38:59.251756+0000","last_became_active":"2024-05-10T10:38:58.266020+0000","last_became_peered":"2024-05-10T10:38:58.266020+0000","last_unstale":"2024-05-10T10:38:59.251756+0000","last_undegraded":"2024-05-10T10:38:59.251756+0000","last_fullsized":"2024-05-10T10:38:59.251756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:34:20.925827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742616+0000","last_change":"2024-05-10T10:38:57.240790+0000","last_active":"2024-05-10T10:38:59.742616+0000","last_peered":"2024-05-10T10:38:59.742616+0000","last_clean":"2024-05-10T10:38:59.742616+0000","last_became_active":"2024-05-10T10:38:57.240455+0000","last_became_peered":"2024-05-10T10:38:57.240455+0000","last_unstale":"2024-05-10T10:38:59.742616+0000","last_undegraded":"2024-05-10T10:38:59.742616+0000","last_fullsized":"2024-05-10T10:38:59.742616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:06.542298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.251969+0000","last_change":"2024-05-10T10:38:58.253374+0000","last_active":"2024-05-10T10:38:59.251969+0000","last_peered":"2024-05-10T10:38:59.251969+0000","last_clean":"2024-05-10T10:38:59.251969+0000","last_became_active":"2024-05-10T10:38:58.253121+0000","last_became_peered":"2024-05-10T10:38:58.253121+0000","last_unstale":"2024-05-10T10:38:59.251969+0000","last_undegraded":"2024-05-10T10:38:59.251969+0000","last_fullsized":"2024-05-10T10:38:59.251969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:00.861051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742445+0000","last_change":"2024-05-10T10:38:57.249734+0000","last_active":"2024-05-10T10:38:59.742445+0000","last_peered":"2024-05-10T10:38:59.742445+0000","last_clean":"2024-05-10T10:38:59.742445+0000","last_became_active":"2024-05-10T10:38:57.249452+0000","last_became_peered":"2024-05-10T10:38:57.249452+0000","last_unstale":"2024-05-10T10:38:59.742445+0000","last_undegraded":"2024-05-10T10:38:59.742445+0000","last_fullsized":"2024-05-10T10:38:59.742445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:57.598796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743576+0000","last_change":"2024-05-10T10:38:58.262691+0000","last_active":"2024-05-10T10:38:59.743576+0000","last_peered":"2024-05-10T10:38:59.743576+0000","last_clean":"2024-05-10T10:38:59.743576+0000","last_became_active":"2024-05-10T10:38:58.262421+0000","last_became_peered":"2024-05-10T10:38:58.262421+0000","last_unstale":"2024-05-10T10:38:59.743576+0000","last_undegraded":"2024-05-10T10:38:59.743576+0000","last_fullsized":"2024-05-10T10:38:59.743576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:45:08.706118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741894+0000","last_change":"2024-05-10T10:38:57.241984+0000","last_active":"2024-05-10T10:38:59.741894+0000","last_peered":"2024-05-10T10:38:59.741894+0000","last_clean":"2024-05-10T10:38:59.741894+0000","last_became_active":"2024-05-10T10:38:57.241709+0000","last_became_peered":"2024-05-10T10:38:57.241709+0000","last_unstale":"2024-05-10T10:38:59.741894+0000","last_undegraded":"2024-05-10T10:38:59.741894+0000","last_fullsized":"2024-05-10T10:38:59.741894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:41:13.191495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741430+0000","last_change":"2024-05-10T10:38:58.253239+0000","last_active":"2024-05-10T10:38:59.741430+0000","last_peered":"2024-05-10T10:38:59.741430+0000","last_clean":"2024-05-10T10:38:59.741430+0000","last_became_active":"2024-05-10T10:38:58.252967+0000","last_became_peered":"2024-05-10T10:38:58.252967+0000","last_unstale":"2024-05-10T10:38:59.741430+0000","last_undegraded":"2024-05-10T10:38:59.741430+0000","last_fullsized":"2024-05-10T10:38:59.741430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:06:54.131965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253087+0000","last_change":"2024-05-10T10:38:57.240601+0000","last_active":"2024-05-10T10:38:59.253087+0000","last_peered":"2024-05-10T10:38:59.253087+0000","last_clean":"2024-05-10T10:38:59.253087+0000","last_became_active":"2024-05-10T10:38:57.240018+0000","last_became_peered":"2024-05-10T10:38:57.240018+0000","last_unstale":"2024-05-10T10:38:59.253087+0000","last_undegraded":"2024-05-10T10:38:59.253087+0000","last_fullsized":"2024-05-10T10:38:59.253087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:51:01.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741862+0000","last_change":"2024-05-10T10:38:58.250770+0000","last_active":"2024-05-10T10:38:59.741862+0000","last_peered":"2024-05-10T10:38:59.741862+0000","last_clean":"2024-05-10T10:38:59.741862+0000","last_became_active":"2024-05-10T10:38:58.250134+0000","last_became_peered":"2024-05-10T10:38:58.250134+0000","last_unstale":"2024-05-10T10:38:59.741862+0000","last_undegraded":"2024-05-10T10:38:59.741862+0000","last_fullsized":"2024-05-10T10:38:59.741862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:38.655656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742344+0000","last_change":"2024-05-10T10:38:57.248178+0000","last_active":"2024-05-10T10:38:59.742344+0000","last_peered":"2024-05-10T10:38:59.742344+0000","last_clean":"2024-05-10T10:38:59.742344+0000","last_became_active":"2024-05-10T10:38:57.247953+0000","last_became_peered":"2024-05-10T10:38:57.247953+0000","last_unstale":"2024-05-10T10:38:59.742344+0000","last_undegraded":"2024-05-10T10:38:59.742344+0000","last_fullsized":"2024-05-10T10:38:59.742344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:59:51.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743418+0000","last_change":"2024-05-10T10:38:58.267839+0000","last_active":"2024-05-10T10:38:59.743418+0000","last_peered":"2024-05-10T10:38:59.743418+0000","last_clean":"2024-05-10T10:38:59.743418+0000","last_became_active":"2024-05-10T10:38:58.267583+0000","last_became_peered":"2024-05-10T10:38:58.267583+0000","last_unstale":"2024-05-10T10:38:59.743418+0000","last_undegraded":"2024-05-10T10:38:59.743418+0000","last_fullsized":"2024-05-10T10:38:59.743418+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:38.857008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252353+0000","last_change":"2024-05-10T10:38:57.235875+0000","last_active":"2024-05-10T10:38:59.252353+0000","last_peered":"2024-05-10T10:38:59.252353+0000","last_clean":"2024-05-10T10:38:59.252353+0000","last_became_active":"2024-05-10T10:38:57.235601+0000","last_became_peered":"2024-05-10T10:38:57.235601+0000","last_unstale":"2024-05-10T10:38:59.252353+0000","last_undegraded":"2024-05-10T10:38:59.252353+0000","last_fullsized":"2024-05-10T10:38:59.252353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:24:05.241780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252379+0000","last_change":"2024-05-10T10:38:58.258469+0000","last_active":"2024-05-10T10:38:59.252379+0000","last_peered":"2024-05-10T10:38:59.252379+0000","last_clean":"2024-05-10T10:38:59.252379+0000","last_became_active":"2024-05-10T10:38:58.258186+0000","last_became_peered":"2024-05-10T10:38:58.258186+0000","last_unstale":"2024-05-10T10:38:59.252379+0000","last_undegraded":"2024-05-10T10:38:59.252379+0000","last_fullsized":"2024-05-10T10:38:59.252379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:08.323156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.251819+0000","last_change":"2024-05-10T10:38:57.248738+0000","last_active":"2024-05-10T10:38:59.251819+0000","last_peered":"2024-05-10T10:38:59.251819+0000","last_clean":"2024-05-10T10:38:59.251819+0000","last_became_active":"2024-05-10T10:38:57.248459+0000","last_became_peered":"2024-05-10T10:38:57.248459+0000","last_unstale":"2024-05-10T10:38:59.251819+0000","last_undegraded":"2024-05-10T10:38:59.251819+0000","last_fullsized":"2024-05-10T10:38:59.251819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:58:20.965008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252558+0000","last_change":"2024-05-10T10:38:58.256198+0000","last_active":"2024-05-10T10:38:59.252558+0000","last_peered":"2024-05-10T10:38:59.252558+0000","last_clean":"2024-05-10T10:38:59.252558+0000","last_became_active":"2024-05-10T10:38:58.255913+0000","last_became_peered":"2024-05-10T10:38:58.255913+0000","last_unstale":"2024-05-10T10:38:59.252558+0000","last_undegraded":"2024-05-10T10:38:59.252558+0000","last_fullsized":"2024-05-10T10:38:59.252558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:33:55.484642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252487+0000","last_change":"2024-05-10T10:38:57.230406+0000","last_active":"2024-05-10T10:38:59.252487+0000","last_peered":"2024-05-10T10:38:59.252487+0000","last_clean":"2024-05-10T10:38:59.252487+0000","last_became_active":"2024-05-10T10:38:57.230065+0000","last_became_peered":"2024-05-10T10:38:57.230065+0000","last_unstale":"2024-05-10T10:38:59.252487+0000","last_undegraded":"2024-05-10T10:38:59.252487+0000","last_fullsized":"2024-05-10T10:38:59.252487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:57:25.931769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742242+0000","last_change":"2024-05-10T10:38:58.255086+0000","last_active":"2024-05-10T10:38:59.742242+0000","last_peered":"2024-05-10T10:38:59.742242+0000","last_clean":"2024-05-10T10:38:59.742242+0000","last_became_active":"2024-05-10T10:38:58.254750+0000","last_became_peered":"2024-05-10T10:38:58.254750+0000","last_unstale":"2024-05-10T10:38:59.742242+0000","last_undegraded":"2024-05-10T10:38:59.742242+0000","last_fullsized":"2024-05-10T10:38:59.742242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:40:36.930877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.251644+0000","last_change":"2024-05-10T10:38:57.241714+0000","last_active":"2024-05-10T10:38:59.251644+0000","last_peered":"2024-05-10T10:38:59.251644+0000","last_clean":"2024-05-10T10:38:59.251644+0000","last_became_active":"2024-05-10T10:38:57.241428+0000","last_became_peered":"2024-05-10T10:38:57.241428+0000","last_unstale":"2024-05-10T10:38:59.251644+0000","last_undegraded":"2024-05-10T10:38:59.251644+0000","last_fullsized":"2024-05-10T10:38:59.251644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:56:43.397551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741737+0000","last_change":"2024-05-10T10:38:58.254606+0000","last_active":"2024-05-10T10:38:59.741737+0000","last_peered":"2024-05-10T10:38:59.741737+0000","last_clean":"2024-05-10T10:38:59.741737+0000","last_became_active":"2024-05-10T10:38:58.254358+0000","last_became_peered":"2024-05-10T10:38:58.254358+0000","last_unstale":"2024-05-10T10:38:59.741737+0000","last_undegraded":"2024-05-10T10:38:59.741737+0000","last_fullsized":"2024-05-10T10:38:59.741737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:24:06.243530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253616+0000","last_change":"2024-05-10T10:38:57.239328+0000","last_active":"2024-05-10T10:38:59.253616+0000","last_peered":"2024-05-10T10:38:59.253616+0000","last_clean":"2024-05-10T10:38:59.253616+0000","last_became_active":"2024-05-10T10:38:57.239042+0000","last_became_peered":"2024-05-10T10:38:57.239042+0000","last_unstale":"2024-05-10T10:38:59.253616+0000","last_undegraded":"2024-05-10T10:38:59.253616+0000","last_fullsized":"2024-05-10T10:38:59.253616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:45:55.031773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.743104+0000","last_change":"2024-05-10T10:38:58.265046+0000","last_active":"2024-05-10T10:38:59.743104+0000","last_peered":"2024-05-10T10:38:59.743104+0000","last_clean":"2024-05-10T10:38:59.743104+0000","last_became_active":"2024-05-10T10:38:58.264257+0000","last_became_peered":"2024-05-10T10:38:58.264257+0000","last_unstale":"2024-05-10T10:38:59.743104+0000","last_undegraded":"2024-05-10T10:38:59.743104+0000","last_fullsized":"2024-05-10T10:38:59.743104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:29.038277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"87'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742131+0000","last_change":"2024-05-10T10:38:57.242120+0000","last_active":"2024-05-10T10:38:59.742131+0000","last_peered":"2024-05-10T10:38:59.742131+0000","last_clean":"2024-05-10T10:38:59.742131+0000","last_became_active":"2024-05-10T10:38:57.241862+0000","last_became_peered":"2024-05-10T10:38:57.241862+0000","last_unstale":"2024-05-10T10:38:59.742131+0000","last_undegraded":"2024-05-10T10:38:59.742131+0000","last_fullsized":"2024-05-10T10:38:59.742131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:33:49.869889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742072+0000","last_change":"2024-05-10T10:38:58.261810+0000","last_active":"2024-05-10T10:38:59.742072+0000","last_peered":"2024-05-10T10:38:59.742072+0000","last_clean":"2024-05-10T10:38:59.742072+0000","last_became_active":"2024-05-10T10:38:58.261515+0000","last_became_peered":"2024-05-10T10:38:58.261515+0000","last_unstale":"2024-05-10T10:38:59.742072+0000","last_undegraded":"2024-05-10T10:38:59.742072+0000","last_fullsized":"2024-05-10T10:38:59.742072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:11.794588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742390+0000","last_change":"2024-05-10T10:38:57.246576+0000","last_active":"2024-05-10T10:38:59.742390+0000","last_peered":"2024-05-10T10:38:59.742390+0000","last_clean":"2024-05-10T10:38:59.742390+0000","last_became_active":"2024-05-10T10:38:57.246353+0000","last_became_peered":"2024-05-10T10:38:57.246353+0000","last_unstale":"2024-05-10T10:38:59.742390+0000","last_undegraded":"2024-05-10T10:38:59.742390+0000","last_fullsized":"2024-05-10T10:38:59.742390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:24:50.126035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252879+0000","last_change":"2024-05-10T10:38:58.243438+0000","last_active":"2024-05-10T10:38:59.252879+0000","last_peered":"2024-05-10T10:38:59.252879+0000","last_clean":"2024-05-10T10:38:59.252879+0000","last_became_active":"2024-05-10T10:38:58.243151+0000","last_became_peered":"2024-05-10T10:38:58.243151+0000","last_unstale":"2024-05-10T10:38:59.252879+0000","last_undegraded":"2024-05-10T10:38:59.252879+0000","last_fullsized":"2024-05-10T10:38:59.252879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:09:03.878776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742016+0000","last_change":"2024-05-10T10:38:57.243728+0000","last_active":"2024-05-10T10:38:59.742016+0000","last_peered":"2024-05-10T10:38:59.742016+0000","last_clean":"2024-05-10T10:38:59.742016+0000","last_became_active":"2024-05-10T10:38:57.243406+0000","last_became_peered":"2024-05-10T10:38:57.243406+0000","last_unstale":"2024-05-10T10:38:59.742016+0000","last_undegraded":"2024-05-10T10:38:59.742016+0000","last_fullsized":"2024-05-10T10:38:59.742016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:24:06.542298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.251641+0000","last_change":"2024-05-10T10:38:58.269077+0000","last_active":"2024-05-10T10:38:59.251641+0000","last_peered":"2024-05-10T10:38:59.251641+0000","last_clean":"2024-05-10T10:38:59.251641+0000","last_became_active":"2024-05-10T10:38:58.268834+0000","last_became_peered":"2024-05-10T10:38:58.268834+0000","last_unstale":"2024-05-10T10:38:59.251641+0000","last_undegraded":"2024-05-10T10:38:59.251641+0000","last_fullsized":"2024-05-10T10:38:59.251641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:45:50.906404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742580+0000","last_change":"2024-05-10T10:38:57.244580+0000","last_active":"2024-05-10T10:38:59.742580+0000","last_peered":"2024-05-10T10:38:59.742580+0000","last_clean":"2024-05-10T10:38:59.742580+0000","last_became_active":"2024-05-10T10:38:57.244271+0000","last_became_peered":"2024-05-10T10:38:57.244271+0000","last_unstale":"2024-05-10T10:38:59.742580+0000","last_undegraded":"2024-05-10T10:38:59.742580+0000","last_fullsized":"2024-05-10T10:38:59.742580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:10:41.942314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742426+0000","last_change":"2024-05-10T10:38:58.267337+0000","last_active":"2024-05-10T10:38:59.742426+0000","last_peered":"2024-05-10T10:38:59.742426+0000","last_clean":"2024-05-10T10:38:59.742426+0000","last_became_active":"2024-05-10T10:38:58.267055+0000","last_became_peered":"2024-05-10T10:38:58.267055+0000","last_unstale":"2024-05-10T10:38:59.742426+0000","last_undegraded":"2024-05-10T10:38:59.742426+0000","last_fullsized":"2024-05-10T10:38:59.742426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:49:52.677913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.252586+0000","last_change":"2024-05-10T10:38:57.236911+0000","last_active":"2024-05-10T10:38:59.252586+0000","last_peered":"2024-05-10T10:38:59.252586+0000","last_clean":"2024-05-10T10:38:59.252586+0000","last_became_active":"2024-05-10T10:38:57.236571+0000","last_became_peered":"2024-05-10T10:38:57.236571+0000","last_unstale":"2024-05-10T10:38:59.252586+0000","last_undegraded":"2024-05-10T10:38:59.252586+0000","last_fullsized":"2024-05-10T10:38:59.252586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:04:19.208256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741299+0000","last_change":"2024-05-10T10:38:58.257503+0000","last_active":"2024-05-10T10:38:59.741299+0000","last_peered":"2024-05-10T10:38:59.741299+0000","last_clean":"2024-05-10T10:38:59.741299+0000","last_became_active":"2024-05-10T10:38:58.256800+0000","last_became_peered":"2024-05-10T10:38:58.256800+0000","last_unstale":"2024-05-10T10:38:59.741299+0000","last_undegraded":"2024-05-10T10:38:59.741299+0000","last_fullsized":"2024-05-10T10:38:59.741299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:15.037318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.741323+0000","last_change":"2024-05-10T10:38:57.243643+0000","last_active":"2024-05-10T10:38:59.741323+0000","last_peered":"2024-05-10T10:38:59.741323+0000","last_clean":"2024-05-10T10:38:59.741323+0000","last_became_active":"2024-05-10T10:38:57.243364+0000","last_became_peered":"2024-05-10T10:38:57.243364+0000","last_unstale":"2024-05-10T10:38:59.741323+0000","last_undegraded":"2024-05-10T10:38:59.741323+0000","last_fullsized":"2024-05-10T10:38:59.741323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:08:59.506424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742395+0000","last_change":"2024-05-10T10:38:58.260221+0000","last_active":"2024-05-10T10:38:59.742395+0000","last_peered":"2024-05-10T10:38:59.742395+0000","last_clean":"2024-05-10T10:38:59.742395+0000","last_became_active":"2024-05-10T10:38:58.259943+0000","last_became_peered":"2024-05-10T10:38:58.259943+0000","last_unstale":"2024-05-10T10:38:59.742395+0000","last_undegraded":"2024-05-10T10:38:59.742395+0000","last_fullsized":"2024-05-10T10:38:59.742395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:32:37.757780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253086+0000","last_change":"2024-05-10T10:38:57.239379+0000","last_active":"2024-05-10T10:38:59.253086+0000","last_peered":"2024-05-10T10:38:59.253086+0000","last_clean":"2024-05-10T10:38:59.253086+0000","last_became_active":"2024-05-10T10:38:57.239057+0000","last_became_peered":"2024-05-10T10:38:57.239057+0000","last_unstale":"2024-05-10T10:38:59.253086+0000","last_undegraded":"2024-05-10T10:38:59.253086+0000","last_fullsized":"2024-05-10T10:38:59.253086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:09:02.877025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253572+0000","last_change":"2024-05-10T10:38:58.258403+0000","last_active":"2024-05-10T10:38:59.253572+0000","last_peered":"2024-05-10T10:38:59.253572+0000","last_clean":"2024-05-10T10:38:59.253572+0000","last_became_active":"2024-05-10T10:38:58.258116+0000","last_became_peered":"2024-05-10T10:38:58.258116+0000","last_unstale":"2024-05-10T10:38:59.253572+0000","last_undegraded":"2024-05-10T10:38:59.253572+0000","last_fullsized":"2024-05-10T10:38:59.253572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:57.221344+0000","last_clean_scrub_stamp":"2024-05-10T10:38:57.221344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:11.794588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.742912+0000","last_change":"2024-05-10T10:38:57.242780+0000","last_active":"2024-05-10T10:38:59.742912+0000","last_peered":"2024-05-10T10:38:59.742912+0000","last_clean":"2024-05-10T10:38:59.742912+0000","last_became_active":"2024-05-10T10:38:57.242494+0000","last_became_peered":"2024-05-10T10:38:57.242494+0000","last_unstale":"2024-05-10T10:38:59.742912+0000","last_undegraded":"2024-05-10T10:38:59.742912+0000","last_fullsized":"2024-05-10T10:38:59.742912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:38:56.219594+0000","last_clean_scrub_stamp":"2024-05-10T10:38:56.219594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:08:59.506424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":246,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T10:38:59.253186+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:38:59.253186+0000","last_peered":"2024-05-10T10:38:59.253186+0000","last_clean":"2024-05-10T10:38:59.253186+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:38:59.253186+0000","last_undegraded":"2024-05-10T10:38:59.253186+0000","last_fullsized":"2024-05-10T10:38:59.253186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1675264,"data_stored":1080699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1093632,"data_stored":492491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":490275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":490435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":490275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":490275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":490345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1082961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:39:07.135 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:39:07.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T10:39:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.282+0000 7fb23eacf640 1 Processor -- start 2024-05-10T10:39:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.282+0000 7fb23eacf640 1 -- start start 2024-05-10T10:39:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.283+0000 7fb23eacf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb23815f410 0x7fb2380d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.283+0000 7fb23eacf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2380d0f10 0x7fb2380b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.283+0000 7fb23eacf640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2380d2800 con 0x7fb2380b54d0 2024-05-10T10:39:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.283+0000 7fb23eacf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb2380d2ae0 con 0x7fb23815f410 2024-05-10T10:39:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.283+0000 7fb23eacf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb2380d01d0 con 0x7fb2380d0f10 2024-05-10T10:39:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb23c86b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb23815f410 0x7fb2380d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb237fff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2380d0f10 0x7fb2380b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb23d06c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb2380b54d0 0x7fb2380d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:39678/0 (socket says 172.21.15.53:39678) 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb23d06c640 1 -- 172.21.15.53:0/2507485120 learned_addr learned my addr 172.21.15.53:0/2507485120 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb237fff640 1 -- 172.21.15.53:0/2507485120 >> v2:172.21.15.135:3300/0 conn(0x7fb23815f410 msgr2=0x7fb2380d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb237fff640 1 --2- 172.21.15.53:0/2507485120 >> v2:172.21.15.135:3300/0 conn(0x7fb23815f410 0x7fb2380d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb237fff640 1 -- 172.21.15.53:0/2507485120 >> v1:172.21.15.135:6789/0 conn(0x7fb2380b54d0 legacy=0x7fb2380d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb237fff640 1 -- 172.21.15.53:0/2507485120 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2380d04b0 con 0x7fb2380d0f10 2024-05-10T10:39:07.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.284+0000 7fb23c86b640 1 --2- 172.21.15.53:0/2507485120 >> v2:172.21.15.135:3300/0 conn(0x7fb23815f410 0x7fb2380d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:07.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.285+0000 7fb237fff640 1 --2- 172.21.15.53:0/2507485120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2380d0f10 0x7fb2380b4f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0097a0 tx=0x7fb22c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11c8a3dcd0072c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.285+0000 7fb2377fe640 1 -- 172.21.15.53:0/2507485120 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c037070 con 0x7fb2380d0f10 2024-05-10T10:39:07.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.285+0000 7fb2377fe640 1 -- 172.21.15.53:0/2507485120 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c02ae50 con 0x7fb2380d0f10 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.285+0000 7fb2377fe640 1 -- 172.21.15.53:0/2507485120 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c032660 con 0x7fb2380d0f10 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 -- 172.21.15.53:0/2507485120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2380d0f10 msgr2=0x7fb2380b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 --2- 172.21.15.53:0/2507485120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2380d0f10 0x7fb2380b4f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0097a0 tx=0x7fb22c02a100 comp rx=0 tx=0).stop 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 -- 172.21.15.53:0/2507485120 shutdown_connections 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 --2- 172.21.15.53:0/2507485120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2380d0f10 0x7fb2380b4f90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 --2- 172.21.15.53:0/2507485120 >> v2:172.21.15.135:3300/0 conn(0x7fb23815f410 0x7fb2380d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 -- 172.21.15.53:0/2507485120 >> 172.21.15.53:0/2507485120 conn(0x7fb23808ed30 msgr2=0x7fb23809e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 -- 172.21.15.53:0/2507485120 shutdown_connections 2024-05-10T10:39:07.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.286+0000 7fb23eacf640 1 -- 172.21.15.53:0/2507485120 wait complete. 2024-05-10T10:39:07.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.287+0000 7fb23eacf640 1 Processor -- start 2024-05-10T10:39:07.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.287+0000 7fb23eacf640 1 -- start start 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23eacf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2380b54d0 0x7fb2381f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23eacf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2380d0f10 0x7fb2380b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23eacf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb23815f410 0x7fb2380cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23eacf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb2380cffb0 con 0x7fb2380d0f10 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23eacf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2380cfe30 con 0x7fb2380b54d0 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23eacf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb2380d3de0 con 0x7fb23815f410 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2380d0f10 0x7fb2380b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2380d0f10 0x7fb2380b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60538/0 (socket says 172.21.15.53:60538) 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 -- 172.21.15.53:0/3813459170 learned_addr learned my addr 172.21.15.53:0/3813459170 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23d06c640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb23815f410 0x7fb2380cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb23c86b640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2380b54d0 0x7fb2381f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 -- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb23815f410 msgr2=0x7fb2380cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb23815f410 0x7fb2380cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 -- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2380b54d0 msgr2=0x7fb2381f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2380b54d0 0x7fb2381f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.288+0000 7fb237fff640 1 -- 172.21.15.53:0/3813459170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2381f6e00 con 0x7fb2380d0f10 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.289+0000 7fb237fff640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2380d0f10 0x7fb2380b2730 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb22c035040 tx=0x7fb22c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.289+0000 7fb2357fa640 1 -- 172.21.15.53:0/3813459170 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c041070 con 0x7fb2380d0f10 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.289+0000 7fb2357fa640 1 -- 172.21.15.53:0/3813459170 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c03c460 con 0x7fb2380d0f10 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.289+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2381f70e0 con 0x7fb2380d0f10 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.289+0000 7fb2357fa640 1 -- 172.21.15.53:0/3813459170 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c037040 con 0x7fb2380d0f10 2024-05-10T10:39:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.289+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2381f7980 con 0x7fb2380d0f10 2024-05-10T10:39:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.290+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23806a220 con 0x7fb2380d0f10 2024-05-10T10:39:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.290+0000 7fb2357fa640 1 -- 172.21.15.53:0/3813459170 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb22c03a070 con 0x7fb2380d0f10 2024-05-10T10:39:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.291+0000 7fb2357fa640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2240762d0 0x7fb224078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.291+0000 7fb2357fa640 1 -- 172.21.15.53:0/3813459170 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fb22c0b7980 con 0x7fb2380d0f10 2024-05-10T10:39:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.292+0000 7fb23c86b640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2240762d0 0x7fb224078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.292+0000 7fb23c86b640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2240762d0 0x7fb224078790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb228009560 tx=0x7fb228009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.296+0000 7fb2357fa640 1 -- 172.21.15.53:0/3813459170 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb22c080b00 con 0x7fb2380d0f10 2024-05-10T10:39:07.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.562+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb2380d6090 con 0x7fb2380d0f10 2024-05-10T10:39:07.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.564+0000 7fb2357fa640 1 -- 172.21.15.53:0/3813459170 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fb22c084e00 con 0x7fb2380d0f10 2024-05-10T10:39:07.565 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:39:07.565 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:39:07.565 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2024-05-10T10:39:07.565 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2024-05-10T10:39:07.565 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-10T10:39:07.565 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2240762d0 msgr2=0x7fb224078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2240762d0 0x7fb224078790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb228009560 tx=0x7fb228009190 comp rx=0 tx=0).stop 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2380d0f10 msgr2=0x7fb2380b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2380d0f10 0x7fb2380b2730 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb22c035040 tx=0x7fb22c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 shutdown_connections 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2240762d0 0x7fb224078790 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb23815f410 0x7fb2380cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2380d0f10 0x7fb2380b2730 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 --2- 172.21.15.53:0/3813459170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2380b54d0 0x7fb2381f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 >> 172.21.15.53:0/3813459170 conn(0x7fb23808ed30 msgr2=0x7fb2380757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 shutdown_connections 2024-05-10T10:39:07.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.566+0000 7fb23eacf640 1 -- 172.21.15.53:0/3813459170 wait complete. 2024-05-10T10:39:07.578 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:39:07.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.680+0000 7f3d030a0640 1 Processor -- start 2024-05-10T10:39:07.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.680+0000 7f3d030a0640 1 -- start start 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d030a0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3cfc15f3b0 0x7f3cfc0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d030a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc0d5d30 0x7f3cfc0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d030a0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cfc0d30b0 con 0x7f3cfc15f790 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d030a0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3cfc0d3390 con 0x7f3cfc15f3b0 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d030a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3cfc0c8c80 con 0x7f3cfc0d5d30 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d00e3c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3cfc15f3b0 0x7f3cfc0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d00e3c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3cfc15f3b0 0x7f3cfc0d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60544/0 (socket says 172.21.15.53:60544) 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.681+0000 7f3d00e3c640 1 -- 172.21.15.53:0/3008435517 learned_addr learned my addr 172.21.15.53:0/3008435517 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.682+0000 7f3d00e3c640 1 -- 172.21.15.53:0/3008435517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc0d5d30 msgr2=0x7f3cfc0d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.682+0000 7f3d00e3c640 1 --2- 172.21.15.53:0/3008435517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc0d5d30 0x7f3cfc0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.682+0000 7f3d00e3c640 1 -- 172.21.15.53:0/3008435517 >> v1:172.21.15.135:6789/0 conn(0x7f3cfc15f790 legacy=0x7f3cfc0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:07.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.682+0000 7f3d00e3c640 1 -- 172.21.15.53:0/3008435517 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc0c8f60 con 0x7f3cfc15f3b0 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.682+0000 7f3d00e3c640 1 --2- 172.21.15.53:0/3008435517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc15f3b0 0x7f3cfc0d36f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009870 tx=0x7f3cf402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c669c93d7f275c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3cfb7fe640 1 -- 172.21.15.53:0/3008435517 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4037070 con 0x7f3cfc15f3b0 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3cfb7fe640 1 -- 172.21.15.53:0/3008435517 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf4032440 con 0x7f3cfc15f3b0 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3cfb7fe640 1 -- 172.21.15.53:0/3008435517 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4032810 con 0x7f3cfc15f3b0 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 -- 172.21.15.53:0/3008435517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc15f3b0 msgr2=0x7f3cfc0d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 --2- 172.21.15.53:0/3008435517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc15f3b0 0x7f3cfc0d36f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009870 tx=0x7f3cf402a210 comp rx=0 tx=0).stop 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 -- 172.21.15.53:0/3008435517 shutdown_connections 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 --2- 172.21.15.53:0/3008435517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc0d5d30 0x7f3cfc0d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 --2- 172.21.15.53:0/3008435517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc15f3b0 0x7f3cfc0d36f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 -- 172.21.15.53:0/3008435517 >> 172.21.15.53:0/3008435517 conn(0x7f3cfc0d5130 msgr2=0x7f3cfc0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 -- 172.21.15.53:0/3008435517 shutdown_connections 2024-05-10T10:39:07.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.683+0000 7f3d030a0640 1 -- 172.21.15.53:0/3008435517 wait complete. 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.684+0000 7f3d030a0640 1 Processor -- start 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.684+0000 7f3d030a0640 1 -- start start 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d030a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cfc0d5d30 0x7f3cfc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d030a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc15f3b0 0x7f3cfc14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d030a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc14c7a0 0x7f3cfc0b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d030a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc0d0e60 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d030a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc0d0ce0 con 0x7f3cfc0d5d30 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d030a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3cfc0d0fe0 con 0x7f3cfc15f3b0 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d0163d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc14c7a0 0x7f3cfc0b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d0163d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc14c7a0 0x7f3cfc0b96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60548/0 (socket says 172.21.15.53:60548) 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.685+0000 7f3d0163d640 1 -- 172.21.15.53:0/1700133880 learned_addr learned my addr 172.21.15.53:0/1700133880 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:07.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d00e3c640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cfc0d5d30 0x7f3cfc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3cfbfff640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc15f3b0 0x7f3cfc14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d0163d640 1 -- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc15f3b0 msgr2=0x7f3cfc14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d0163d640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc15f3b0 0x7f3cfc14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d0163d640 1 -- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cfc0d5d30 msgr2=0x7f3cfc0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d0163d640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cfc0d5d30 0x7f3cfc0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d0163d640 1 -- 172.21.15.53:0/1700133880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc0ba430 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d00e3c640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cfc0d5d30 0x7f3cfc0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d0163d640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc14c7a0 0x7f3cfc0b96c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009fd0 tx=0x7f3ce400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3cf97fa640 1 -- 172.21.15.53:0/1700133880 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4011830 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3cf97fa640 1 -- 172.21.15.53:0/1700133880 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce4016070 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3cf97fa640 1 -- 172.21.15.53:0/1700133880 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4011bf0 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cfc0ba710 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.686+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cfc14cc30 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.687+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cfc06a220 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.688+0000 7f3cf97fa640 1 -- 172.21.15.53:0/1700133880 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3ce401b460 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.688+0000 7f3cf97fa640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3cdc0762d0 0x7f3cdc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:07.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.688+0000 7f3cf97fa640 1 -- 172.21.15.53:0/1700133880 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f3ce40974f0 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.688+0000 7f3d00e3c640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3cdc0762d0 0x7f3cdc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:07.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.689+0000 7f3d00e3c640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3cdc0762d0 0x7f3cdc078790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4034000 tx=0x7f3cf402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:07.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.693+0000 7f3cf97fa640 1 -- 172.21.15.53:0/1700133880 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3ce4060cd0 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.922+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cfc0d1fa0 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.923+0000 7f3cf97fa640 1 -- 172.21.15.53:0/1700133880 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5186 (secure 0 0 0) 0x7f3ce40605d0 con 0x7f3cfc14c7a0 2024-05-10T10:39:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:07.924 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:00.626907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6833/2934725760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2934725760},{"type":"v1","addr":"172.21.15.53:6833","nonce":2934725760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6693,"qdb_cluster":[6693]},"id":6}]} 2024-05-10T10:39:07.924 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3cdc0762d0 msgr2=0x7f3cdc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3cdc0762d0 0x7f3cdc078790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4034000 tx=0x7f3cf402e000 comp rx=0 tx=0).stop 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc14c7a0 msgr2=0x7f3cfc0b96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc14c7a0 0x7f3cfc0b96c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009fd0 tx=0x7f3ce400b8e0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 shutdown_connections 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3cdc0762d0 0x7f3cdc078790 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cfc14c7a0 0x7f3cfc0b96c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3cfc15f3b0 0x7f3cfc14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 --2- 172.21.15.53:0/1700133880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cfc0d5d30 0x7f3cfc0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 >> 172.21.15.53:0/1700133880 conn(0x7f3cfc0d5130 msgr2=0x7f3cfc15e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 shutdown_connections 2024-05-10T10:39:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:07.925+0000 7f3d030a0640 1 -- 172.21.15.53:0/1700133880 wait complete. 2024-05-10T10:39:08.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:38:57.607866+0000', 'modified': '2024-05-10T10:39:00.626907+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6693}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6693': {'gid': 6693, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6833/2934725760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 2934725760}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 2934725760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6693, 'qdb_cluster': [6693]} 2024-05-10T10:39:08.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:39:08.339 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T10:39:08.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.450+0000 7fe7db4da640 1 Processor -- start 2024-05-10T10:39:08.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.450+0000 7fe7db4da640 1 -- start start 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7db4da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe7d415ce60 0x7fe7d40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7db4da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d41611e0 0x7fe7d415d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7db4da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d40c6a60 con 0x7fe7d40c65f0 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7db4da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d40cd090 con 0x7fe7d415ce60 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7db4da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d40cd370 con 0x7fe7d41611e0 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7d8a75640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe7d40c65f0 0x7fe7d4160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37450/0 (socket says 172.21.15.53:37450) 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7d8a75640 1 -- 172.21.15.53:0/1345785539 learned_addr learned my addr 172.21.15.53:0/1345785539 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d41611e0 0x7fe7d415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1345785539 >> v2:172.21.15.135:3300/0 conn(0x7fe7d415ce60 0x7fe7d40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727645776 0 0) 0x7fe7d40c6a60 con 0x7fe7d40c65f0 2024-05-10T10:39:08.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.451+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7bc003520 con 0x7fe7d40c65f0 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1577073752 0 0) 0x7fe7bc003520 con 0x7fe7d40c65f0 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d41611e0 msgr2=0x7fe7d415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d41611e0 0x7fe7d415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 >> v2:172.21.15.135:3300/0 conn(0x7fe7d415ce60 msgr2=0x7fe7d40cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 --2- 172.21.15.53:0/1345785539 >> v2:172.21.15.135:3300/0 conn(0x7fe7d415ce60 0x7fe7d40cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d40cd650 con 0x7fe7d40c65f0 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fe7c8001690 con 0x7fe7d40c65f0 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1345785539 >> v2:172.21.15.135:3300/0 conn(0x7fe7d415ce60 0x7fe7d40cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 >> v1:172.21.15.53:6789/0 conn(0x7fe7d40c65f0 legacy=0x7fe7d4160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7bc0039b0 0x7fe7bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7bc0043a0 0x7fe7bc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc004f90 0x7fe7bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40cd090 con 0x7fe7bc0043a0 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40cd370 con 0x7fe7bc004f90 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe7bc00bf30 con 0x7fe7bc0039b0 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d41611e0 0x7fe7d415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7bc0043a0 0x7fe7bc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7bc0039b0 0x7fe7bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc004f90 0x7fe7bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d8a75640 1 -- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7bc0039b0 msgr2=0x7fe7bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7bc0039b0 0x7fe7bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d8a75640 1 -- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc004f90 msgr2=0x7fe7bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc004f90 0x7fe7bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.452+0000 7fe7d8a75640 1 -- 172.21.15.53:0/1345785539 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc003520 con 0x7fe7bc0043a0 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.453+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7bc0043a0 0x7fe7bc004880 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c210 tx=0x7fe7bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb736aa0bf085fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.453+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8007d60 con 0x7fe7bc0043a0 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.453+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c8005af0 con 0x7fe7bc0043a0 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.453+0000 7fe7d3fff640 1 -- 172.21.15.53:0/1345785539 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8007070 con 0x7fe7bc0043a0 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.454+0000 7fe7db4da640 1 -- 172.21.15.53:0/1345785539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7bc0043a0 msgr2=0x7fe7bc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.454+0000 7fe7db4da640 1 --2- 172.21.15.53:0/1345785539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7bc0043a0 0x7fe7bc004880 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c210 tx=0x7fe7bc005830 comp rx=0 tx=0).stop 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.454+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1345785539 reap_dead start 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.454+0000 7fe7db4da640 1 -- 172.21.15.53:0/1345785539 shutdown_connections 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.454+0000 7fe7db4da640 1 -- 172.21.15.53:0/1345785539 >> 172.21.15.53:0/1345785539 conn(0x7fe7d40b2900 msgr2=0x7fe7d40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.454+0000 7fe7db4da640 1 -- 172.21.15.53:0/1345785539 shutdown_connections 2024-05-10T10:39:08.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.454+0000 7fe7db4da640 1 -- 172.21.15.53:0/1345785539 wait complete. 2024-05-10T10:39:08.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.455+0000 7fe7db4da640 1 Processor -- start 2024-05-10T10:39:08.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.455+0000 7fe7db4da640 1 -- start start 2024-05-10T10:39:08.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7db4da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc0043a0 0x7fe7d41571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7db4da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d415ea60 0x7fe7d415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7db4da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d41f6880 0x7fe7d41f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7db4da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40cde50 con 0x7fe7d41f6880 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7db4da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40ccd60 con 0x7fe7bc0043a0 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7db4da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d40cdfd0 con 0x7fe7d415ea60 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d41f6880 0x7fe7d41f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d41f6880 0x7fe7d41f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60568/0 (socket says 172.21.15.53:60568) 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 -- 172.21.15.53:0/3295733468 learned_addr learned my addr 172.21.15.53:0/3295733468 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9276640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc0043a0 0x7fe7d41571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d415ea60 0x7fe7d415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 -- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d415ea60 msgr2=0x7fe7d415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d415ea60 0x7fe7d415eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 -- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc0043a0 msgr2=0x7fe7d41571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc0043a0 0x7fe7d41571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 -- 172.21.15.53:0/3295733468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d41578c0 con 0x7fe7d41f6880 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d41f6880 0x7fe7d41f6d00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4074260 tx=0x7fe7c4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.456+0000 7fe7d1ffb640 1 -- 172.21.15.53:0/3295733468 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4003990 con 0x7fe7d41f6880 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.457+0000 7fe7d1ffb640 1 -- 172.21.15.53:0/3295733468 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c401b070 con 0x7fe7d41f6880 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.457+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d4157ba0 con 0x7fe7d41f6880 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.457+0000 7fe7d1ffb640 1 -- 172.21.15.53:0/3295733468 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4003b70 con 0x7fe7d41f6880 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.457+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d4158160 con 0x7fe7d41f6880 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.459+0000 7fe7d1ffb640 1 -- 172.21.15.53:0/3295733468 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe7c4005d10 con 0x7fe7d41f6880 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.459+0000 7fe7d1ffb640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe7c0076300 0x7fe7c00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.460+0000 7fe7d1ffb640 1 -- 172.21.15.53:0/3295733468 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fe7c4097e30 con 0x7fe7d41f6880 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.460+0000 7fe7d9276640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe7c0076300 0x7fe7c00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.460+0000 7fe7de1f9740 1 Processor -- start 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.460+0000 7fe7de1f9740 1 -- start start 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.460+0000 7fe7d9276640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe7c0076300 0x7fe7c00787c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0093d0 tx=0x7fe7cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d2147a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d214771bb0 0x55d214777be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d2146db8d0 con 0x55d2147bd8b0 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d214694c10 con 0x55d214771bb0 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d2146d7890 con 0x55d2147a1950 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147bdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60572/0 (socket says 172.21.15.53:60572) 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1733180695 learned_addr learned my addr 172.21.15.53:0/1733180695 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d2147a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d214771bb0 0x55d214777be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 msgr2=0x55d2147a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d2147a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d214771bb0 msgr2=0x55d214777be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d214771bb0 0x55d214777be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1733180695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21469c5d0 con 0x55d2147bd8b0 2024-05-10T10:39:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147bdd30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4096070 tx=0x7fe7c405cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18212b32f2b1f025 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.461+0000 7fe7baffd640 1 -- 172.21.15.53:0/1733180695 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4065210 con 0x55d2147bd8b0 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7baffd640 1 -- 172.21.15.53:0/1733180695 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c4069c30 con 0x55d2147bd8b0 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7baffd640 1 -- 172.21.15.53:0/1733180695 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c407a370 con 0x55d2147bd8b0 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1733180695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 msgr2=0x55d2147bdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147bdd30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4096070 tx=0x7fe7c405cf00 comp rx=0 tx=0).stop 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1733180695 shutdown_connections 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147bdd30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d214771bb0 0x55d214777be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1733180695 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d2147a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1733180695 >> 172.21.15.53:0/1733180695 conn(0x55d214773e20 msgr2=0x55d2147bc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1733180695 shutdown_connections 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.462+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1733180695 wait complete. 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.463+0000 7fe7de1f9740 1 -- start start 2024-05-10T10:39:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.463+0000 7fe7de1f9740 1 Processor -- start 2024-05-10T10:39:08.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d21479dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d2147b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147d3450 0x55d2147d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d214778120 con 0x55d2147d3450 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d214698760 con 0x55d2147bd8b0 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d214777750 con 0x55d2147a1950 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 10 client.0 _ll_register_callbacks cb 0x55d2146d1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147d3450 0x55d2147d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147d3450 0x55d2147d38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60584/0 (socket says 172.21.15.53:60584) 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1601419313 learned_addr learned my addr 172.21.15.53:0/1601419313 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d2147b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d21479dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 msgr2=0x55d21479dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d21479dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 msgr2=0x55d2147b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d2147b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 -- 172.21.15.53:0/1601419313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2143ed2d0 con 0x55d2147d3450 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d21479dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147d3450 0x55d2147d38b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe7c405d500 tx=0x7fe7c4077000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de0a2d616443c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7b3fff640 1 -- 172.21.15.53:0/1601419313 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c406e060 con 0x55d2147d3450 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7b3fff640 1 -- 172.21.15.53:0/1601419313 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fe7c40699b0 con 0x55d2147d3450 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.464+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c4069c30 con 0x55d2147d3450 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7b3fff640 1 -- 172.21.15.53:0/1601419313 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe7c407e070 con 0x55d2147d3450 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 10 client.6843 fetch_fsmap learned FSMap version 61 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55d2144b76b0 con 0x55d2147d3450 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7b3fff640 1 -- 172.21.15.53:0/1601419313 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c4069c30 con 0x55d2147d3450 2024-05-10T10:39:08.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7b3fff640 1 -- 172.21.15.53:0/1601419313 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7fe7c4060170 con 0x55d2147d3450 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 10 client.6843 fetch_fsmap finished waiting for FSMap version 61 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 10 client.6843 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 20 client.6843 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d2147d10c0 0x55d2147d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 4 client.6843 mds_command: new command op to 6693 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 --> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d21465f760 con 0x55d2147d10c0 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d2147d10c0 0x55d2147d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7b3fff640 10 client.6843 ms_handle_connect on v2:172.21.15.53:6832/2934725760 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.465+0000 7fe7d9276640 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d2147d10c0 0x55d2147d14a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08b0d0c2fe2b257 server_cookie=996bcf7f045a60a0 in_seq=0 out_seq=0 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7b3fff640 1 -- 172.21.15.53:0/1601419313 <== mds.0 v2:172.21.15.53:6832/2934725760 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d21465f760 con 0x55d2147d10c0 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7b3fff640 10 client.6843 handle_command_reply: tid=0 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 client.6843 shutdown 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147d3450 msgr2=0x55d2147d38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147d3450 0x55d2147d38b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe7c405d500 tx=0x7fe7c4077000 comp rx=0 tx=0).stop 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 shutdown_connections 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d2147d10c0 0x55d2147d14a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147d3450 0x55d2147d38b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d2147b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/1601419313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d21479dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7b3fff640 0 client.6843 ms_handle_reset on v2:172.21.15.53:6832/2934725760 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.466+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 >> 172.21.15.53:0/1601419313 conn(0x55d214771bb0 msgr2=0x55d214775d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.467+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 shutdown_connections 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.467+0000 7fe7de1f9740 1 -- 172.21.15.53:0/1601419313 wait complete. 2024-05-10T10:39:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.467+0000 7fe7de1f9740 20 client.6843 trim_cache size 0 max 16384 2024-05-10T10:39:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.485+0000 7fe7de1f9740 1 Processor -- start 2024-05-10T10:39:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7de1f9740 1 -- start start 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147a1950 0x55d2147b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d214776800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147d3450 0x55d214776d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d406f7e0 con 0x55d2147a1950 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40c90d0 con 0x55d2147bd8b0 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d406f190 con 0x55d2147d3450 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7d9276640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147a1950 0x55d2147b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7d9276640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147a1950 0x55d2147b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60598/0 (socket says 172.21.15.53:60598) 2024-05-10T10:39:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7d9276640 1 -- 172.21.15.53:0/934331129 learned_addr learned my addr 172.21.15.53:0/934331129 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.486+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d214776800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147d3450 0x55d214776d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7d9276640 1 -- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147d3450 msgr2=0x55d214776d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7d9276640 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147d3450 0x55d214776d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7d9276640 1 -- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 msgr2=0x55d214776800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7d9276640 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d214776800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7d9276640 1 -- 172.21.15.53:0/934331129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2147a2020 con 0x55d2147a1950 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7d9276640 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147a1950 0x55d2147b5340 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55d214777840 tx=0x7fe7cc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fed32461b6a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7ba7fc640 1 -- 172.21.15.53:0/934331129 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc01b070 con 0x55d2147a1950 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7ba7fc640 1 -- 172.21.15.53:0/934331129 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7cc015c40 con 0x55d2147a1950 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7ba7fc640 1 -- 172.21.15.53:0/934331129 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc004020 con 0x55d2147a1950 2024-05-10T10:39:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7de1f9740 1 -- 172.21.15.53:0/934331129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147a1950 msgr2=0x55d2147b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.487+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147a1950 0x55d2147b5340 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55d214777840 tx=0x7fe7cc01b000 comp rx=0 tx=0).stop 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 -- 172.21.15.53:0/934331129 shutdown_connections 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147d3450 0x55d214776d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147bd8b0 0x55d214776800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/934331129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147a1950 0x55d2147b5340 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 -- 172.21.15.53:0/934331129 >> 172.21.15.53:0/934331129 conn(0x55d214771bb0 msgr2=0x55d214772d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 -- 172.21.15.53:0/934331129 shutdown_connections 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 -- 172.21.15.53:0/934331129 wait complete. 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 -- start start 2024-05-10T10:39:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.488+0000 7fe7de1f9740 1 Processor -- start 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d214840f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147ba3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7de1f9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147d3450 0x55d2147e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d214778120 con 0x55d2147bd8b0 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d214698760 con 0x55d2147d3450 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7de1f9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d2147a49d0 con 0x55d2147a1950 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7de1f9740 10 client.0 _ll_register_callbacks cb 0x7fe7c4060d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7d8a75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147ba3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7d8a75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147ba3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60612/0 (socket says 172.21.15.53:60612) 2024-05-10T10:39:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7d8a75640 1 -- 172.21.15.53:0/932039153 learned_addr learned my addr 172.21.15.53:0/932039153 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7d9276640 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d214840f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.489+0000 7fe7d9a77640 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147d3450 0x55d2147e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7d8a75640 1 -- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 msgr2=0x55d214840f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d214840f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7d8a75640 1 -- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147d3450 msgr2=0x55d2147e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147d3450 0x55d2147e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7d8a75640 1 -- 172.21.15.53:0/932039153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2147b5620 con 0x55d2147bd8b0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7d8a75640 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147ba3b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c8001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22691f81bdacd34 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7b3fff640 1 -- 172.21.15.53:0/932039153 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80049e0 con 0x55d2147bd8b0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7b3fff640 1 -- 172.21.15.53:0/932039153 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fe7c8004b80 con 0x55d2147bd8b0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7cc015c40 con 0x55d2147bd8b0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7b3fff640 1 -- 172.21.15.53:0/932039153 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe7c8050050 con 0x55d2147bd8b0 2024-05-10T10:39:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 10 client.6849 fetch_fsmap learned FSMap version 61 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55d2147ba8f0 con 0x55d2147bd8b0 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7b3fff640 1 -- 172.21.15.53:0/932039153 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7cc015c40 con 0x55d2147bd8b0 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7b3fff640 1 -- 172.21.15.53:0/932039153 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7fe7c8053020 con 0x55d2147bd8b0 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 10 client.6849 fetch_fsmap finished waiting for FSMap version 61 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 10 client.6849 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 20 client.6849 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d214776cf0 0x55d2147bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 4 client.6849 mds_command: new command op to 6693 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.490+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 --> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] -- command(tid 0: {"prefix": "status"}) -- 0x55d2146e94e0 con 0x55d214776cf0 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7d9276640 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d214776cf0 0x55d2147bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7d9276640 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d214776cf0 0x55d2147bfc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a26ce20caccab2 server_cookie=7605168d3804aeb5 in_seq=0 out_seq=0 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7b3fff640 10 client.6849 ms_handle_connect on v2:172.21.15.53:6832/2934725760 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7b3fff640 1 -- 172.21.15.53:0/932039153 <== mds.0 v2:172.21.15.53:6832/2934725760 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55d2146e94e0 con 0x55d214776cf0 2024-05-10T10:39:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7b3fff640 10 client.6849 handle_command_reply: tid=0 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 client.6849 shutdown 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 msgr2=0x55d2147ba3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147ba3b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c8001a60 comp rx=0 tx=0).stop 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 shutdown_connections 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:6832/2934725760,v1:172.21.15.53:6833/2934725760] conn(0x55d214776cf0 0x55d2147bfc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2147d3450 0x55d2147e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2147bd8b0 0x55d2147ba3b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7b3fff640 0 client.6849 ms_handle_reset on v2:172.21.15.53:6832/2934725760 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 --2- 172.21.15.53:0/932039153 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d2147a1950 0x55d214840f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 >> 172.21.15.53:0/932039153 conn(0x55d214771bb0 msgr2=0x55d2147cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 shutdown_connections 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.491+0000 7fe7de1f9740 1 -- 172.21.15.53:0/932039153 wait complete. 2024-05-10T10:39:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7de1f9740 20 client.6849 trim_cache size 0 max 16384 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 6693, 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 10.257295245, 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 61, 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 87, 2024-05-10T10:39:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 87, 2024-05-10T10:39:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 15.571885299 2024-05-10T10:39:08.500 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:39:08.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe7c0076300 msgr2=0x7fe7c00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe7c0076300 0x7fe7c00787c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0093d0 tx=0x7fe7cc009190 comp rx=0 tx=0).stop 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d41f6880 msgr2=0x7fe7d41f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d41f6880 0x7fe7d41f6d00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4074260 tx=0x7fe7c4011890 comp rx=0 tx=0).stop 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 shutdown_connections 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe7c0076300 0x7fe7c00787c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d41f6880 0x7fe7d41f6d00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7d415ea60 0x7fe7d415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 --2- 172.21.15.53:0/3295733468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7bc0043a0 0x7fe7d41571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 >> 172.21.15.53:0/3295733468 conn(0x7fe7d40b2900 msgr2=0x7fe7d40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 shutdown_connections 2024-05-10T10:39:08.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.492+0000 7fe7db4da640 1 -- 172.21.15.53:0/3295733468 wait complete. 2024-05-10T10:39:08.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:39:08.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.606+0000 7f57fc2a7640 1 Processor -- start 2024-05-10T10:39:08.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.607+0000 7f57fc2a7640 1 -- start start 2024-05-10T10:39:08.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57fc2a7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f57f415f3b0 0x7f57f40d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57fc2a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57f415f790 0x7f57f40d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57fc2a7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f40d30b0 con 0x7f57f40d5d30 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57fc2a7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f57f40d3390 con 0x7f57f415f790 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57fc2a7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f57f40c8c80 con 0x7f57f415f3b0 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57f9842640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57f415f790 0x7f57f40d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57f9842640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57f415f790 0x7f57f40d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60618/0 (socket says 172.21.15.53:60618) 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57f9842640 1 -- 172.21.15.53:0/2975109092 learned_addr learned my addr 172.21.15.53:0/2975109092 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57fa043640 1 --2- 172.21.15.53:0/2975109092 >> v2:172.21.15.135:3300/0 conn(0x7f57f415f3b0 0x7f57f40d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57f9842640 1 -- 172.21.15.53:0/2975109092 >> v2:172.21.15.135:3300/0 conn(0x7f57f415f3b0 msgr2=0x7f57f40d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:08.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57f9842640 1 --2- 172.21.15.53:0/2975109092 >> v2:172.21.15.135:3300/0 conn(0x7f57f415f3b0 0x7f57f40d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57f9842640 1 -- 172.21.15.53:0/2975109092 >> v1:172.21.15.135:6789/0 conn(0x7f57f40d5d30 legacy=0x7f57f40c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.608+0000 7f57f9842640 1 -- 172.21.15.53:0/2975109092 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f40c8f60 con 0x7f57f415f790 2024-05-10T10:39:08.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.609+0000 7f57f9842640 1 --2- 172.21.15.53:0/2975109092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f415f790 0x7f57f40d3c30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f57f408e820 tx=0x7f57e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990538576759784c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.609+0000 7f57f9041640 1 -- 172.21.15.53:0/2975109092 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4037070 con 0x7f57f415f790 2024-05-10T10:39:08.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.609+0000 7f57f9041640 1 -- 172.21.15.53:0/2975109092 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e4032440 con 0x7f57f415f790 2024-05-10T10:39:08.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.609+0000 7f57f9041640 1 -- 172.21.15.53:0/2975109092 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4032810 con 0x7f57f415f790 2024-05-10T10:39:08.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.609+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2975109092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f415f790 msgr2=0x7f57f40d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.609+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2975109092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f415f790 0x7f57f40d3c30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f57f408e820 tx=0x7f57e402a270 comp rx=0 tx=0).stop 2024-05-10T10:39:08.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.610+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2975109092 shutdown_connections 2024-05-10T10:39:08.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.610+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2975109092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f415f790 0x7f57f40d3c30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.610+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2975109092 >> v2:172.21.15.135:3300/0 conn(0x7f57f415f3b0 0x7f57f40d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.610+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2975109092 >> 172.21.15.53:0/2975109092 conn(0x7f57f40d5130 msgr2=0x7f57f40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.610+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2975109092 shutdown_connections 2024-05-10T10:39:08.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.610+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2975109092 wait complete. 2024-05-10T10:39:08.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.611+0000 7f57fc2a7640 1 Processor -- start 2024-05-10T10:39:08.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.611+0000 7f57fc2a7640 1 -- start start 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.611+0000 7f57fc2a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57f40d5d30 0x7f57f4160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fc2a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57f415f3b0 0x7f57f4157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fc2a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f41622e0 0x7f57f4153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fc2a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40d0e60 con 0x7f57f41622e0 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fc2a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40d0ce0 con 0x7f57f40d5d30 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fc2a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f57f40d0fe0 con 0x7f57f415f3b0 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f41622e0 0x7f57f4153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f41622e0 0x7f57f4153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60632/0 (socket says 172.21.15.53:60632) 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 -- 172.21.15.53:0/2155239218 learned_addr learned my addr 172.21.15.53:0/2155239218 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa043640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57f40d5d30 0x7f57f4160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57f9842640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57f415f3b0 0x7f57f4157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 -- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57f415f3b0 msgr2=0x7f57f4157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57f415f3b0 0x7f57f4157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 -- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57f40d5d30 msgr2=0x7f57f4160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57f40d5d30 0x7f57f4160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 -- 172.21.15.53:0/2155239218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4154480 con 0x7f57f41622e0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fa844640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f41622e0 0x7f57f4153f40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c7d0 tx=0x7f57ec00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57eaffd640 1 -- 172.21.15.53:0/2155239218 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec003b70 con 0x7f57f41622e0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57eaffd640 1 -- 172.21.15.53:0/2155239218 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec003d30 con 0x7f57f41622e0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57eaffd640 1 -- 172.21.15.53:0/2155239218 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00aa90 con 0x7f57f41622e0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f4161240 con 0x7f57f41622e0 2024-05-10T10:39:08.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.612+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f4161800 con 0x7f57f41622e0 2024-05-10T10:39:08.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.613+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f406a220 con 0x7f57f41622e0 2024-05-10T10:39:08.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.614+0000 7f57eaffd640 1 -- 172.21.15.53:0/2155239218 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f57ec00ac30 con 0x7f57f41622e0 2024-05-10T10:39:08.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.615+0000 7f57eaffd640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57d80762d0 0x7f57d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:08.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.615+0000 7f57eaffd640 1 -- 172.21.15.53:0/2155239218 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f57ec0974a0 con 0x7f57f41622e0 2024-05-10T10:39:08.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.615+0000 7f57fa043640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57d80762d0 0x7f57d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:08.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.616+0000 7f57fa043640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57d80762d0 0x7f57d8078790 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f57f0006fd0 tx=0x7f57f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:08.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.619+0000 7f57eaffd640 1 -- 172.21.15.53:0/2155239218 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f57ec0606b0 con 0x7f57f41622e0 2024-05-10T10:39:08.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.847+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f57f4075d00 con 0x7f57f41622e0 2024-05-10T10:39:08.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.847+0000 7f57eaffd640 1 -- 172.21.15.53:0/2155239218 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7f57ec0649b0 con 0x7f57f41622e0 2024-05-10T10:39:08.848 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:39:08.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57d80762d0 msgr2=0x7f57d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57d80762d0 0x7f57d8078790 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f57f0006fd0 tx=0x7f57f0008040 comp rx=0 tx=0).stop 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f41622e0 msgr2=0x7f57f4153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f41622e0 0x7f57f4153f40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c7d0 tx=0x7f57ec00cc90 comp rx=0 tx=0).stop 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 shutdown_connections 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f57d80762d0 0x7f57d8078790 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57f41622e0 0x7f57f4153f40 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57f415f3b0 0x7f57f4157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 --2- 172.21.15.53:0/2155239218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57f40d5d30 0x7f57f4160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 >> 172.21.15.53:0/2155239218 conn(0x7f57f40d5130 msgr2=0x7f57f406cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 shutdown_connections 2024-05-10T10:39:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:08.849+0000 7f57fc2a7640 1 -- 172.21.15.53:0/2155239218 wait complete. 2024-05-10T10:39:08.860 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:39:08.861 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:39:08.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:08.861 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:39:08.878 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42420sec preferred_lft 42420sec 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:08.879 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:39:08.880 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:39:08.881 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:08.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:08.881 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:39:08.943 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:39:08.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:08.944 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:39:09.001 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:39:09.002 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:39:09.002 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:39:09.002 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:39:09.002 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:39:09.002 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:39:09.002 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:39:09.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:39:09.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:39:09.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:39:09.003 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:39:09.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:39:09.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:39:09.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.132 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:39:09.148 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:39:09.148 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:39:09.148 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:39:09.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:39:09.214 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:39:09.215 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:09.270 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:39:09.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:39:09.271 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:09.327 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:39:09.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:09.413 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:09.413 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:39:09.413 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:39:09.422 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:09.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:39:09.422 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:39:09.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:39:09.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:39:09.427 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:39:09.428 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:39:09.428 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:39:09.428 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:39:09.428 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:39:09.429 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:39:09.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:09.462 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:39:09.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:39:09.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:39:09.564 INFO:teuthology.orchestra.run.smithi053.stdout:client6858 2024-05-10T10:39:09.565 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:39:09.565 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:39:09.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.582 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-05-10T10:39:09.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.642 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-05-10T10:39:09.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.700 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-05-10T10:39:09.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:09.758 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-05-10T10:39:09.819 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:39:09.819 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:39:09.819 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:39:09.819 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:39:09.819 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-05-10T10:39:09.820 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:39:09.928 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2024-05-10T10:39:09.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:39:09.937 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:39:09.938 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:39:09.994 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:39:09.994 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:39:10.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:39:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.195+0000 7f05d00fc640 1 Processor -- start 2024-05-10T10:39:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.195+0000 7f05d00fc640 1 -- start start 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05d00fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c81598c0 0x7f05c806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05d00fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f05c806ec40 0x7f05c80c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05d00fc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c806f1c0 con 0x7f05c80c71c0 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05d00fc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f05c806f4a0 con 0x7f05c806ec40 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05d00fc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f05c806f780 con 0x7f05c81598c0 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cd697640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f05c806ec40 0x7f05c80c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cd697640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f05c806ec40 0x7f05c80c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60648/0 (socket says 172.21.15.53:60648) 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cd697640 1 -- 172.21.15.53:0/4267637229 learned_addr learned my addr 172.21.15.53:0/4267637229 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cde98640 1 --2- 172.21.15.53:0/4267637229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c81598c0 0x7f05c806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cd697640 1 -- 172.21.15.53:0/4267637229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c81598c0 msgr2=0x7f05c806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cd697640 1 --2- 172.21.15.53:0/4267637229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c81598c0 0x7f05c806e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cd697640 1 -- 172.21.15.53:0/4267637229 >> v1:172.21.15.135:6789/0 conn(0x7f05c80c71c0 legacy=0x7f05c815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.196+0000 7f05cd697640 1 -- 172.21.15.53:0/4267637229 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c806fa60 con 0x7f05c806ec40 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.197+0000 7f05cd697640 1 --2- 172.21.15.53:0/4267637229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c806ec40 0x7f05c80c6c80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009870 tx=0x7f05b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6996b3f6a3ddb0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.197+0000 7f05cce96640 1 -- 172.21.15.53:0/4267637229 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8037070 con 0x7f05c806ec40 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.197+0000 7f05cce96640 1 -- 172.21.15.53:0/4267637229 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b8032440 con 0x7f05c806ec40 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.197+0000 7f05cce96640 1 -- 172.21.15.53:0/4267637229 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8032810 con 0x7f05c806ec40 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.197+0000 7f05d00fc640 1 -- 172.21.15.53:0/4267637229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c806ec40 msgr2=0x7f05c80c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.197+0000 7f05d00fc640 1 --2- 172.21.15.53:0/4267637229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c806ec40 0x7f05c80c6c80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009870 tx=0x7f05b802a270 comp rx=0 tx=0).stop 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.198+0000 7f05d00fc640 1 -- 172.21.15.53:0/4267637229 shutdown_connections 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.198+0000 7f05d00fc640 1 --2- 172.21.15.53:0/4267637229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c806ec40 0x7f05c80c6c80 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.198+0000 7f05d00fc640 1 --2- 172.21.15.53:0/4267637229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c81598c0 0x7f05c806e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.198+0000 7f05d00fc640 1 -- 172.21.15.53:0/4267637229 >> 172.21.15.53:0/4267637229 conn(0x7f05c80b2900 msgr2=0x7f05c80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.198+0000 7f05d00fc640 1 -- 172.21.15.53:0/4267637229 shutdown_connections 2024-05-10T10:39:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.198+0000 7f05d00fc640 1 -- 172.21.15.53:0/4267637229 wait complete. 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.199+0000 7f05d00fc640 1 Processor -- start 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.199+0000 7f05d00fc640 1 -- start start 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c806ec40 0x7f05c815ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c71c0 0x7f05c80c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c81598c0 0x7f05c81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8159ca0 con 0x7f05c81598c0 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05c815c6e0 con 0x7f05c80c71c0 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f05c8159e20 con 0x7f05c806ec40 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c81598c0 0x7f05c81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c81598c0 0x7f05c81563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60652/0 (socket says 172.21.15.53:60652) 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 -- 172.21.15.53:0/2960194838 learned_addr learned my addr 172.21.15.53:0/2960194838 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05cde98640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c806ec40 0x7f05c815ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05cd697640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c71c0 0x7f05c80c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 -- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c806ec40 msgr2=0x7f05c815ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c806ec40 0x7f05c815ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 -- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c71c0 msgr2=0x7f05c80c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c71c0 0x7f05c80c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 -- 172.21.15.53:0/2960194838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c815f510 con 0x7f05c81598c0 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05cde98640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c806ec40 0x7f05c815ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05cd697640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c71c0 0x7f05c80c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05ce699640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c81598c0 0x7f05c81563b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c7a0 tx=0x7f05c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05be7fc640 1 -- 172.21.15.53:0/2960194838 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0003b70 con 0x7f05c81598c0 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05be7fc640 1 -- 172.21.15.53:0/2960194838 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0003d30 con 0x7f05c81598c0 2024-05-10T10:39:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c815d680 con 0x7f05c81598c0 2024-05-10T10:39:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05be7fc640 1 -- 172.21.15.53:0/2960194838 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000aa90 con 0x7f05c81598c0 2024-05-10T10:39:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.200+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c815dc40 con 0x7f05c81598c0 2024-05-10T10:39:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.201+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c806a220 con 0x7f05c81598c0 2024-05-10T10:39:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.202+0000 7f05be7fc640 1 -- 172.21.15.53:0/2960194838 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f05c000ac30 con 0x7f05c81598c0 2024-05-10T10:39:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.203+0000 7f05be7fc640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f05ac076300 0x7f05ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.203+0000 7f05be7fc640 1 -- 172.21.15.53:0/2960194838 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f05c00976a0 con 0x7f05c81598c0 2024-05-10T10:39:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.203+0000 7f05cde98640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f05ac076300 0x7f05ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.203+0000 7f05cde98640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f05ac076300 0x7f05ac0787c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006fd0 tx=0x7f05c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.207+0000 7f05be7fc640 1 -- 172.21.15.53:0/2960194838 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f05c0060e60 con 0x7f05c81598c0 2024-05-10T10:39:10.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.410+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f05c806e4e0 con 0x7f05c81598c0 2024-05-10T10:39:10.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.410+0000 7f05be7fc640 1 -- 172.21.15.53:0/2960194838 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) ==== 80+0+0 (secure 0 0 0) 0x7f05c0060760 con 0x7f05c81598c0 2024-05-10T10:39:10.411 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:39:10.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f05ac076300 msgr2=0x7f05ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f05ac076300 0x7f05ac0787c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006fd0 tx=0x7f05c4008040 comp rx=0 tx=0).stop 2024-05-10T10:39:10.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c81598c0 msgr2=0x7f05c81563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c81598c0 0x7f05c81563b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c7a0 tx=0x7f05c000cc60 comp rx=0 tx=0).stop 2024-05-10T10:39:10.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 shutdown_connections 2024-05-10T10:39:10.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f05ac076300 0x7f05ac0787c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c81598c0 0x7f05c81563b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05c80c71c0 0x7f05c80c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 --2- 172.21.15.53:0/2960194838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05c806ec40 0x7f05c815ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 >> 172.21.15.53:0/2960194838 conn(0x7f05c80b2900 msgr2=0x7f05c8157270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 shutdown_connections 2024-05-10T10:39:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.413+0000 7f05d00fc640 1 -- 172.21.15.53:0/2960194838 wait complete. 2024-05-10T10:39:10.424 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:39:10.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:10.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:10.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:39:10.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:10.473 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:39:10.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:39:10.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.647+0000 7f72fc483640 1 Processor -- start 2024-05-10T10:39:10.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.647+0000 7f72fc483640 1 -- start start 2024-05-10T10:39:10.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72fc483640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f72f415f790 0x7f72f40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72fc483640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72f40d30b0 con 0x7f72f40d5d30 2024-05-10T10:39:10.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72fc483640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72f40d3390 con 0x7f72f415f3b0 2024-05-10T10:39:10.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72fc483640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f72f40c8c80 con 0x7f72f415f790 2024-05-10T10:39:10.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72f9a1e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f72f415f790 0x7f72f40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72faa20640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f72f40d5d30 0x7f72f40d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37454/0 (socket says 172.21.15.53:37454) 2024-05-10T10:39:10.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72faa20640 1 -- 172.21.15.53:0/2963920246 learned_addr learned my addr 172.21.15.53:0/2963920246 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:10.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72f9a1e640 1 -- 172.21.15.53:0/2963920246 >> v1:172.21.15.135:6789/0 conn(0x7f72f415f3b0 legacy=0x7f72f40d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72f9a1e640 1 -- 172.21.15.53:0/2963920246 >> v1:172.21.15.53:6789/0 conn(0x7f72f40d5d30 legacy=0x7f72f40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.649+0000 7f72f9a1e640 1 -- 172.21.15.53:0/2963920246 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f40c8f60 con 0x7f72f415f790 2024-05-10T10:39:10.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.648+0000 7f72f921d640 1 -- 172.21.15.53:0/2963920246 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839803065 0 0) 0x7f72f40d30b0 con 0x7f72f40d5d30 2024-05-10T10:39:10.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.649+0000 7f72f9a1e640 1 --2- 172.21.15.53:0/2963920246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f415f790 0x7f72f40cfcd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f72e40097a0 tx=0x7f72e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3586bf3f6d62e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.649+0000 7f72f921d640 1 -- 172.21.15.53:0/2963920246 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4037070 con 0x7f72f415f790 2024-05-10T10:39:10.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.649+0000 7f72f921d640 1 -- 172.21.15.53:0/2963920246 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e4032440 con 0x7f72f415f790 2024-05-10T10:39:10.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.649+0000 7f72f921d640 1 -- 172.21.15.53:0/2963920246 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4032810 con 0x7f72f415f790 2024-05-10T10:39:10.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.650+0000 7f72fc483640 1 -- 172.21.15.53:0/2963920246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f415f790 msgr2=0x7f72f40cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.650+0000 7f72fc483640 1 --2- 172.21.15.53:0/2963920246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f415f790 0x7f72f40cfcd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f72e40097a0 tx=0x7f72e402a230 comp rx=0 tx=0).stop 2024-05-10T10:39:10.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.650+0000 7f72fc483640 1 -- 172.21.15.53:0/2963920246 shutdown_connections 2024-05-10T10:39:10.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.650+0000 7f72fc483640 1 --2- 172.21.15.53:0/2963920246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f415f790 0x7f72f40cfcd0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.650+0000 7f72fc483640 1 -- 172.21.15.53:0/2963920246 >> 172.21.15.53:0/2963920246 conn(0x7f72f40d5130 msgr2=0x7f72f40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.650+0000 7f72fc483640 1 -- 172.21.15.53:0/2963920246 shutdown_connections 2024-05-10T10:39:10.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.650+0000 7f72fc483640 1 -- 172.21.15.53:0/2963920246 wait complete. 2024-05-10T10:39:10.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.651+0000 7f72fc483640 1 Processor -- start 2024-05-10T10:39:10.654 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.651+0000 7f72fc483640 1 -- start start 2024-05-10T10:39:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72fc483640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72f40d5d30 0x7f72f4153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72fc483640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f415f3b0 0x7f72f41f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72fc483640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f40ccc10 0x7f72f4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72fc483640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f72f40d3f80 con 0x7f72f40ccc10 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72fc483640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72f40d3e00 con 0x7f72f415f3b0 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72fc483640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f72f40ca770 con 0x7f72f40d5d30 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72faa20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f40ccc10 0x7f72f4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72faa20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f40ccc10 0x7f72f4161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60666/0 (socket says 172.21.15.53:60666) 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72faa20640 1 -- 172.21.15.53:0/1992868053 learned_addr learned my addr 172.21.15.53:0/1992868053 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72f9a1e640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f415f3b0 0x7f72f41f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.652+0000 7f72fa21f640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72f40d5d30 0x7f72f4153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72faa20640 1 -- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72f40d5d30 msgr2=0x7f72f4153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72faa20640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72f40d5d30 0x7f72f4153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72faa20640 1 -- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f415f3b0 msgr2=0x7f72f41f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72faa20640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f415f3b0 0x7f72f41f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72faa20640 1 -- 172.21.15.53:0/1992868053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f40b8e60 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72fa21f640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72f40d5d30 0x7f72f4153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72faa20640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f40ccc10 0x7f72f4161d50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00c480 tx=0x7f72ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72eaffd640 1 -- 172.21.15.53:0/1992868053 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec018070 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72eaffd640 1 -- 172.21.15.53:0/1992868053 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ec007500 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f40b9140 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72eaffd640 1 -- 172.21.15.53:0/1992868053 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec004a20 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.653+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f40b9700 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.654+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f406a220 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.655+0000 7f72eaffd640 1 -- 172.21.15.53:0/1992868053 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f72ec01b070 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.655+0000 7f72eaffd640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72d80762d0 0x7f72d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.655+0000 7f72eaffd640 1 -- 172.21.15.53:0/1992868053 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f72ec096990 con 0x7f72f40ccc10 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.656+0000 7f72fa21f640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72d80762d0 0x7f72d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:10.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.656+0000 7f72fa21f640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72d80762d0 0x7f72d8078790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f72f00012e0 tx=0x7f72f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:10.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.660+0000 7f72eaffd640 1 -- 172.21.15.53:0/1992868053 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f72ec05fba0 con 0x7f72f40ccc10 2024-05-10T10:39:10.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.888+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72f40cb110 con 0x7f72f40ccc10 2024-05-10T10:39:10.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.888+0000 7f72eaffd640 1 -- 172.21.15.53:0/1992868053 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5186 (secure 0 0 0) 0x7f72ec060e20 con 0x7f72f40ccc10 2024-05-10T10:39:10.889 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:10.889 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:00.626907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6833/2934725760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2934725760},{"type":"v1","addr":"172.21.15.53:6833","nonce":2934725760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6693,"qdb_cluster":[6693]},"id":6}]} 2024-05-10T10:39:10.889 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2024-05-10T10:39:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.890+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72d80762d0 msgr2=0x7f72d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.890+0000 7f72fc483640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72d80762d0 0x7f72d8078790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f72f00012e0 tx=0x7f72f0009040 comp rx=0 tx=0).stop 2024-05-10T10:39:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.890+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f40ccc10 msgr2=0x7f72f4161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.890+0000 7f72fc483640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f40ccc10 0x7f72f4161d50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00c480 tx=0x7f72ec00c940 comp rx=0 tx=0).stop 2024-05-10T10:39:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.890+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 shutdown_connections 2024-05-10T10:39:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.891+0000 7f72fc483640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72d80762d0 0x7f72d8078790 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.891+0000 7f72fc483640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72f40ccc10 0x7f72f4161d50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.891+0000 7f72fc483640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f415f3b0 0x7f72f41f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.891+0000 7f72fc483640 1 --2- 172.21.15.53:0/1992868053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72f40d5d30 0x7f72f4153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.891+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 >> 172.21.15.53:0/1992868053 conn(0x7f72f40d5130 msgr2=0x7f72f40cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.891+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 shutdown_connections 2024-05-10T10:39:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:10.891+0000 7f72fc483640 1 -- 172.21.15.53:0/1992868053 wait complete. 2024-05-10T10:39:10.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-10T10:39:11.033 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:39:11.033 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:39:11.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:39:11.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.137+0000 7fe860771640 1 Processor -- start 2024-05-10T10:39:11.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.137+0000 7fe860771640 1 -- start start 2024-05-10T10:39:11.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe860771640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8580b8860 0x7fe85806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe860771640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe8580d0230 0x7fe8580cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe860771640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe8580d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe860771640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe85806ee50 con 0x7fe8580b8860 2024-05-10T10:39:11.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe860771640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe85815fc90 con 0x7fe8580d0230 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe860771640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe8580b8c40 con 0x7fe8580cfbb0 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe85e50d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8580b8860 0x7fe85806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe85e50d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8580b8860 0x7fe85806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60680/0 (socket says 172.21.15.53:60680) 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe85e50d640 1 -- 172.21.15.53:0/4119891284 learned_addr learned my addr 172.21.15.53:0/4119891284 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe85ed0e640 1 --2- 172.21.15.53:0/4119891284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe8580d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.138+0000 7fe85dd0c640 1 --2- 172.21.15.53:0/4119891284 >> v2:172.21.15.135:3300/0 conn(0x7fe8580d0230 0x7fe8580cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85e50d640 1 -- 172.21.15.53:0/4119891284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 msgr2=0x7fe8580d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85e50d640 1 --2- 172.21.15.53:0/4119891284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe8580d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85e50d640 1 -- 172.21.15.53:0/4119891284 >> v2:172.21.15.135:3300/0 conn(0x7fe8580d0230 msgr2=0x7fe8580cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85e50d640 1 --2- 172.21.15.53:0/4119891284 >> v2:172.21.15.135:3300/0 conn(0x7fe8580d0230 0x7fe8580cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85e50d640 1 -- 172.21.15.53:0/4119891284 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8580cd570 con 0x7fe8580b8860 2024-05-10T10:39:11.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85e50d640 1 --2- 172.21.15.53:0/4119891284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580b8860 0x7fe85806e520 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe8580b48b0 tx=0x7fe85002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ec2eab70299811 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85d50b640 1 -- 172.21.15.53:0/4119891284 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850037070 con 0x7fe8580b8860 2024-05-10T10:39:11.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.139+0000 7fe85d50b640 1 -- 172.21.15.53:0/4119891284 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe850032440 con 0x7fe8580b8860 2024-05-10T10:39:11.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe85d50b640 1 -- 172.21.15.53:0/4119891284 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850032810 con 0x7fe8580b8860 2024-05-10T10:39:11.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 -- 172.21.15.53:0/4119891284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580b8860 msgr2=0x7fe85806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 --2- 172.21.15.53:0/4119891284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580b8860 0x7fe85806e520 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe8580b48b0 tx=0x7fe85002a180 comp rx=0 tx=0).stop 2024-05-10T10:39:11.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 -- 172.21.15.53:0/4119891284 shutdown_connections 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 --2- 172.21.15.53:0/4119891284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe8580d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 --2- 172.21.15.53:0/4119891284 >> v2:172.21.15.135:3300/0 conn(0x7fe8580d0230 0x7fe8580cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 --2- 172.21.15.53:0/4119891284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580b8860 0x7fe85806e520 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 -- 172.21.15.53:0/4119891284 >> 172.21.15.53:0/4119891284 conn(0x7fe8580d2df0 msgr2=0x7fe8580d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 -- 172.21.15.53:0/4119891284 shutdown_connections 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.140+0000 7fe860771640 1 -- 172.21.15.53:0/4119891284 wait complete. 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.141+0000 7fe860771640 1 Processor -- start 2024-05-10T10:39:11.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.141+0000 7fe860771640 1 -- start start 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe860771640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8580b8860 0x7fe858149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe860771640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe858162230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe860771640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580d0230 0x7fe85815c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe860771640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580cde10 con 0x7fe8580d0230 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe860771640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8580cdb30 con 0x7fe8580b8860 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe860771640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe85806ea60 con 0x7fe8580cfbb0 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe85ed0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580d0230 0x7fe85815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe85ed0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580d0230 0x7fe85815c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60690/0 (socket says 172.21.15.53:60690) 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe85ed0e640 1 -- 172.21.15.53:0/4269337669 learned_addr learned my addr 172.21.15.53:0/4269337669 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe85dd0c640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe858162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.142+0000 7fe85e50d640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8580b8860 0x7fe858149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe85ed0e640 1 -- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 msgr2=0x7fe858162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe85ed0e640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe858162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe85ed0e640 1 -- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8580b8860 msgr2=0x7fe858149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe85ed0e640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8580b8860 0x7fe858149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe85ed0e640 1 -- 172.21.15.53:0/4269337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe858160eb0 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe85ed0e640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580d0230 0x7fe85815c580 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe848007e60 tx=0x7fe84800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe856ffd640 1 -- 172.21.15.53:0/4269337669 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848014620 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe856ffd640 1 -- 172.21.15.53:0/4269337669 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe848019070 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe858161190 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe856ffd640 1 -- 172.21.15.53:0/4269337669 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848014a20 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.143+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe858161750 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.144+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe85806a220 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.145+0000 7fe856ffd640 1 -- 172.21.15.53:0/4269337669 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe848014bc0 con 0x7fe8580d0230 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.145+0000 7fe856ffd640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8440762d0 0x7fe844078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.145+0000 7fe856ffd640 1 -- 172.21.15.53:0/4269337669 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fe848097e00 con 0x7fe8580d0230 2024-05-10T10:39:11.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.145+0000 7fe85e50d640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8440762d0 0x7fe844078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.146+0000 7fe85e50d640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8440762d0 0x7fe844078790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe850034000 tx=0x7fe85002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.149+0000 7fe856ffd640 1 -- 172.21.15.53:0/4269337669 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe848061010 con 0x7fe8580d0230 2024-05-10T10:39:11.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.379+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe8580d2320 con 0x7fe8580d0230 2024-05-10T10:39:11.380 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:39:11.380+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:39:11.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.381+0000 7fe856ffd640 1 -- 172.21.15.53:0/4269337669 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7fe848065310 con 0x7fe8580d0230 2024-05-10T10:39:11.382 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8440762d0 msgr2=0x7fe844078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8440762d0 0x7fe844078790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe850034000 tx=0x7fe85002e000 comp rx=0 tx=0).stop 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580d0230 msgr2=0x7fe85815c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580d0230 0x7fe85815c580 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe848007e60 tx=0x7fe84800e4b0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 shutdown_connections 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8440762d0 0x7fe844078790 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8580d0230 0x7fe85815c580 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8580cfbb0 0x7fe858162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 --2- 172.21.15.53:0/4269337669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8580b8860 0x7fe858149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 >> 172.21.15.53:0/4269337669 conn(0x7fe8580d2df0 msgr2=0x7fe858070740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 shutdown_connections 2024-05-10T10:39:11.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.384+0000 7fe860771640 1 -- 172.21.15.53:0/4269337669 wait complete. 2024-05-10T10:39:11.388 INFO:tasks.ceph.mds.a.smithi053.stderr: -4981> 2024-05-10T10:39:03.873+0000 7f9f05921640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:39:11.400 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-05-10T10:39:11.423 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:39:11.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 Processor -- start 2024-05-10T10:39:11.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 -- start start 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75db85b0 0x558c75d8a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5b180 0x558c75d8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75e5ea30 0x558c75e5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558c75d2ff50 con 0x558c75e5ea30 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c75d33f90 con 0x558c75db85b0 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb214828f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558c75e5df50 con 0x558c75e5b180 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb212b23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75e5ea30 0x558c75e5ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb212b23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75e5ea30 0x558c75e5ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60718/0 (socket says 172.21.15.53:60718) 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.444+0000 7fb212b23640 1 -- 172.21.15.53:0/1523246763 learned_addr learned my addr 172.21.15.53:0/1523246763 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb212322640 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75db85b0 0x558c75d8a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb211b21640 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5b180 0x558c75d8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb212b23640 1 -- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5b180 msgr2=0x558c75d8aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb212b23640 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5b180 0x558c75d8aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb212b23640 1 -- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75db85b0 msgr2=0x558c75d8a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb212b23640 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75db85b0 0x558c75d8a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb212b23640 1 -- 172.21.15.53:0/1523246763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c75db20d0 con 0x558c75e5ea30 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb212b23640 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75e5ea30 0x558c75e5ba90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb20c025a00 tx=0x7fb20c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad1d150fec54e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb210b1f640 1 -- 172.21.15.53:0/1523246763 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c056040 con 0x558c75e5ea30 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb210b1f640 1 -- 172.21.15.53:0/1523246763 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c052920 con 0x558c75e5ea30 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb210b1f640 1 -- 172.21.15.53:0/1523246763 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c052cf0 con 0x558c75e5ea30 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb214828f40 1 -- 172.21.15.53:0/1523246763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75e5ea30 msgr2=0x558c75e5ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.445+0000 7fb214828f40 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75e5ea30 0x558c75e5ba90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb20c025a00 tx=0x7fb20c051000 comp rx=0 tx=0).stop 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 -- 172.21.15.53:0/1523246763 shutdown_connections 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75e5ea30 0x558c75e5ba90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5b180 0x558c75d8aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 --2- 172.21.15.53:0/1523246763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75db85b0 0x558c75d8a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 -- 172.21.15.53:0/1523246763 >> 172.21.15.53:0/1523246763 conn(0x558c75db2800 msgr2=0x558c75d86c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 -- 172.21.15.53:0/1523246763 shutdown_connections 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 -- 172.21.15.53:0/1523246763 wait complete. 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 Processor -- start 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.446+0000 7fb214828f40 1 -- start start 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb214828f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75db85b0 0x558c75e4dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb214828f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75e5b180 0x558c75e606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb214828f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5ea30 0x558c75ef97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb214828f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558c75e55c90 con 0x558c75db85b0 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb214828f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c75db7a30 con 0x558c75e5b180 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb214828f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558c75e50760 con 0x558c75e5ea30 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75db85b0 0x558c75e4dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75db85b0 0x558c75e4dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60732/0 (socket says 172.21.15.53:60732) 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 -- 172.21.15.53:0/1621360440 learned_addr learned my addr 172.21.15.53:0/1621360440 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb211b21640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75e5b180 0x558c75e606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212b23640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5ea30 0x558c75ef97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 -- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5ea30 msgr2=0x558c75ef97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c75e5ea30 0x558c75ef97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 -- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75e5b180 msgr2=0x558c75e606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c75e5b180 0x558c75e606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 -- 172.21.15.53:0/1621360440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c75d8a6e0 con 0x558c75db85b0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.447+0000 7fb212322640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75db85b0 0x558c75e4dd70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb20400df70 tx=0x7fb20400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.448+0000 7fb2037fe640 1 -- 172.21.15.53:0/1621360440 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20400be60 con 0x558c75db85b0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.448+0000 7fb2037fe640 1 -- 172.21.15.53:0/1621360440 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb204016070 con 0x558c75db85b0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.448+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558c75d8a400 con 0x558c75db85b0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.448+0000 7fb2037fe640 1 -- 172.21.15.53:0/1621360440 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2040116a0 con 0x558c75db85b0 2024-05-10T10:39:11.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.448+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c75d871f0 con 0x558c75db85b0 2024-05-10T10:39:11.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.449+0000 7fb2037fe640 1 -- 172.21.15.53:0/1621360440 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb204011840 con 0x558c75db85b0 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.450+0000 7fb2037fe640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb1f4074cb0 0x7fb1f4077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.450+0000 7fb2037fe640 1 -- 172.21.15.53:0/1621360440 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7fb204096220 con 0x558c75db85b0 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.450+0000 7fb211b21640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb1f4074cb0 0x7fb1f4077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.450+0000 7fb214828f40 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558c75e60bf0 0x558c75e4ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.450+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e88) -- 0x558c75e5cb00 con 0x558c75e60bf0 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.450+0000 7fb212b23640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558c75e60bf0 0x558c75e4ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.451+0000 7fb212b23640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558c75e60bf0 0x558c75e4ee40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.451+0000 7fb211b21640 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb1f4074cb0 0x7fb1f4077170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x558c75d8aee0 tx=0x7fb208002860 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb212b23640 1 -- 172.21.15.53:0/1621360440 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v88'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb20c066480 con 0x558c75e60bf0 2024-05-10T10:39:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558c75e60bf0 msgr2=0x558c75e4ee40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x558c75e60bf0 0x558c75e4ee40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb1f4074cb0 msgr2=0x7fb1f4077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb1f4074cb0 0x7fb1f4077170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x558c75d8aee0 tx=0x7fb208002860 comp rx=0 tx=0).stop 2024-05-10T10:39:11.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75db85b0 msgr2=0x558c75e4dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 --2- 172.21.15.53:0/1621360440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c75db85b0 0x558c75e4dd70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb20400df70 tx=0x7fb20400b5b0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb212b23640 1 -- 172.21.15.53:0/1621360440 reap_dead start 2024-05-10T10:39:11.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 shutdown_connections 2024-05-10T10:39:11.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 >> 172.21.15.53:0/1621360440 conn(0x558c75db2800 msgr2=0x558c75e5d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 shutdown_connections 2024-05-10T10:39:11.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.454+0000 7fb214828f40 1 -- 172.21.15.53:0/1621360440 wait complete. 2024-05-10T10:39:11.458 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-05-10T10:39:11.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.540+0000 7fae54f57280 1 -- start start 2024-05-10T10:39:11.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.540+0000 7fae54f57280 1 Processor -- start 2024-05-10T10:39:11.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a470c00 0x56240b248000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae54f57280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240a471800 0x56240b248580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240a471c00 0x56240b248b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae54f57280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56240b20e180 con 0x56240a471800 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56240b20e300 con 0x56240a471c00 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56240b20e480 con 0x56240a470c00 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240a471800 0x56240b248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240a471800 0x56240b248580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60738/0 (socket says 172.21.15.53:60738) 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 -- 172.21.15.53:0/334014334 learned_addr learned my addr 172.21.15.53:0/334014334 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae53f04640 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240a471c00 0x56240b248b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae53703640 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a470c00 0x56240b248000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 -- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a470c00 msgr2=0x56240b248000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a470c00 0x56240b248000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 -- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240a471c00 msgr2=0x56240b248b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240a471c00 0x56240b248b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 -- 172.21.15.53:0/334014334 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240a4a6780 con 0x56240a471800 2024-05-10T10:39:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae53f04640 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240a471c00 0x56240b248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.541+0000 7fae52f02640 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240a471800 0x56240b248580 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x56240b29f710 tx=0x56240b211f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5135a209eb96569 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae50efe640 1 -- 172.21.15.53:0/334014334 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240a4969c0 con 0x56240a471800 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae50efe640 1 -- 172.21.15.53:0/334014334 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56240a49cfc0 con 0x56240a471800 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae50efe640 1 -- 172.21.15.53:0/334014334 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240a496680 con 0x56240a471800 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae54f57280 1 -- 172.21.15.53:0/334014334 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240a4a63c0 con 0x56240a471800 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae54f57280 1 -- 172.21.15.53:0/334014334 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56240a4a7680 con 0x56240a471800 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae54f57280 4 waiting for MDS map... 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae50efe640 1 -- 172.21.15.53:0/334014334 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x56240b358200 con 0x56240a471800 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae50efe640 1 -- 172.21.15.53:0/334014334 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x56240a4cb180 con 0x56240a471800 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae54f57280 4 Got MDS map 62 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae54f57280 10 main: JournalTool::main 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae54f57280 4 main: JournalTool: connecting to RADOS... 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.542+0000 7fae54f57280 1 Processor -- start 2024-05-10T10:39:11.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae54f57280 1 -- start start 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae54f57280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae54f57280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56240b20e600 con 0x56240b3a6800 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56240b20e780 con 0x56240b3a6000 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56240b20e900 con 0x56240b3a6400 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60754/0 (socket says 172.21.15.53:60754) 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 -- 172.21.15.53:0/3563494729 learned_addr learned my addr 172.21.15.53:0/3563494729 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae53f04640 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae53703640 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 -- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6400 msgr2=0x56240b249600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 -- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6000 msgr2=0x56240b249080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae52f02640 1 -- 172.21.15.53:0/3563494729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240a4a74a0 con 0x56240b3a6800 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.543+0000 7fae53703640 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae52f02640 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6800 0x56240b249b80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x56240b29fd10 tx=0x56240b210660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd159ac3d6746b83 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae4eefa640 1 -- 172.21.15.53:0/3563494729 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240a496b60 con 0x56240b3a6800 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae4eefa640 1 -- 172.21.15.53:0/3563494729 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56240a49ce00 con 0x56240b3a6800 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae4eefa640 1 -- 172.21.15.53:0/3563494729 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240a496820 con 0x56240b3a6800 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 -- 172.21.15.53:0/3563494729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6800 msgr2=0x56240b249b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6800 0x56240b249b80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x56240b29fd10 tx=0x56240b210660 comp rx=0 tx=0).stop 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 -- 172.21.15.53:0/3563494729 shutdown_connections 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 --2- 172.21.15.53:0/3563494729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 -- 172.21.15.53:0/3563494729 >> 172.21.15.53:0/3563494729 conn(0x56240b26bc00 msgr2=0x56240a56b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 -- 172.21.15.53:0/3563494729 shutdown_connections 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.544+0000 7fae54f57280 1 -- 172.21.15.53:0/3563494729 wait complete. 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 Processor -- start 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 -- start start 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56240b20f500 con 0x56240b3a6000 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56240b20ea80 con 0x56240b3a6400 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56240b20f680 con 0x56240b3a6800 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae53f04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae53f04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60762/0 (socket says 172.21.15.53:60762) 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae53f04640 1 -- 172.21.15.53:0/1769368319 learned_addr learned my addr 172.21.15.53:0/1769368319 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.545+0000 7fae53703640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae52f02640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae53f04640 1 -- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6800 msgr2=0x56240b249b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae53f04640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae53f04640 1 -- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6400 msgr2=0x56240b249600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae53f04640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae53f04640 1 -- 172.21.15.53:0/1769368319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240a4a6000 con 0x56240b3a6000 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae52f02640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae53f04640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6000 0x56240b249080 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x56240b283bf0 tx=0x56240b34cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae4d6f7640 1 -- 172.21.15.53:0/1769368319 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240a496b60 con 0x56240b3a6000 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae4d6f7640 1 -- 172.21.15.53:0/1769368319 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56240a49ca80 con 0x56240b3a6000 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56240a4a6960 con 0x56240b3a6000 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae4d6f7640 1 -- 172.21.15.53:0/1769368319 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240a496820 con 0x56240b3a6000 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240a4a7860 con 0x56240b3a6000 2024-05-10T10:39:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.546+0000 7fae54f57280 4 main: JournalTool: resolving pool 13 2024-05-10T10:39:11.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.547+0000 7fae4d6f7640 1 -- 172.21.15.53:0/1769368319 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x56240b3aec00 con 0x56240b3a6000 2024-05-10T10:39:11.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae4d6f7640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x56240b3b0c00 0x56240b24a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae4d6f7640 1 -- 172.21.15.53:0/1769368319 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x56240b3ad000 con 0x56240b3a6000 2024-05-10T10:39:11.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae54f57280 4 main: JournalTool: creating IoCtx.. 2024-05-10T10:39:11.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae54f57280 4 main: Executing for rank 0 2024-05-10T10:39:11.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae53703640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x56240b3b0c00 0x56240b24a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae53703640 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x56240b3b0c00 0x56240b24a100 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56240b1fede0 tx=0x56240a789800 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae54f57280 1 -- start start 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.548+0000 7fae54f57280 1 Processor -- start 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a7000 0x56240b24a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae54f57280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3b1c00 0x56240b24ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae54f57280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a86a000 0x56240b24b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae54f57280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56240b20f980 con 0x56240b3b1c00 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56240b20fb00 con 0x56240b3a7000 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae54f57280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56240b20fc80 con 0x56240a86a000 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae53703640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3b1c00 0x56240b24ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae53703640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3b1c00 0x56240b24ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60774/0 (socket says 172.21.15.53:60774) 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae53703640 1 -- 172.21.15.53:0/3157750314 learned_addr learned my addr 172.21.15.53:0/3157750314 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae53f04640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a7000 0x56240b24a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae52f02640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a86a000 0x56240b24b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae53703640 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a86a000 msgr2=0x56240b24b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.549+0000 7fae53703640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a86a000 0x56240b24b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae53703640 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a7000 msgr2=0x56240b24a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae53703640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a7000 0x56240b24a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae53703640 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240a4a7a40 con 0x56240b3b1c00 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae53703640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3b1c00 0x56240b24ac00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x56240a7a7ef0 tx=0x56240b210a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d1379b7df15cba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae4a6f1640 1 -- 172.21.15.53:0/3157750314 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240b401860 con 0x56240b3b1c00 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae4a6f1640 1 -- 172.21.15.53:0/3157750314 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56240a49cc40 con 0x56240b3b1c00 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae4a6f1640 1 -- 172.21.15.53:0/3157750314 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56240b402820 con 0x56240b3b1c00 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240a4a6000 con 0x56240b3b1c00 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56240a4a6960 con 0x56240b3b1c00 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae54f57280 4 waiting for MDS map... 2024-05-10T10:39:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae4a6f1640 1 -- 172.21.15.53:0/3157750314 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x56240a870600 con 0x56240b3b1c00 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae4a6f1640 1 -- 172.21.15.53:0/3157750314 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x56240a4cb500 con 0x56240b3b1c00 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae54f57280 4 Got MDS map 62 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x56240a86a400 0x56240b24b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x56240a86a800 con 0x56240a86a400 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.550+0000 7fae53f04640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x56240a86a400 0x56240b24b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.551+0000 7fae53f04640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x56240a86a400 0x56240b24b700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=f66ed5f5f38a8a06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.551+0000 7fae53f04640 1 -- 172.21.15.53:0/3157750314 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x56240a57e000 con 0x56240a86a400 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.551+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x56240a86ac00 0x56240a890000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.551+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x56240a86b000 con 0x56240a86ac00 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.551+0000 7fae52f02640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x56240a86ac00 0x56240a890000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.552+0000 7fae52f02640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x56240a86ac00 0x56240a890000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=284dcc6f761ad35f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.554+0000 7fae52f02640 1 -- 172.21.15.53:0/3157750314 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x56240a57e500 con 0x56240a86ac00 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.554+0000 7fae48eee640 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88) -- 0x56240a873800 con 0x56240a86ac00 2024-05-10T10:39:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.554+0000 7fae48eee640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x56240a873c00 0x56240a891b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.554+0000 7fae48eee640 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88) -- 0x56240a898000 con 0x56240a873c00 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.554+0000 7fae52f02640 1 -- 172.21.15.53:0/3157750314 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x56240a57e500 con 0x56240a86ac00 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.554+0000 7fae53f04640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x56240a873c00 0x56240a891b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.555+0000 7fae53f04640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x56240a873c00 0x56240a891b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=5cb59be73b47ed1d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.555+0000 7fae53f04640 1 -- 172.21.15.53:0/3157750314 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56240a57e000 con 0x56240a873c00 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stdout:old journal was 4212139~2605 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stdout:new journal start will be 8388608 (4173864 bytes past old end) 2024-05-10T10:39:11.562 INFO:teuthology.orchestra.run.smithi053.stdout:writing journal head 2024-05-10T10:39:11.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.555+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240a86b400 con 0x56240a86ac00 2024-05-10T10:39:11.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae52f02640 1 -- 172.21.15.53:0/3157750314 <== osd.0 v2:172.21.15.53:6802/1634417257 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v88'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56240a57e500 con 0x56240a86ac00 2024-05-10T10:39:11.563 INFO:teuthology.orchestra.run.smithi053.stdout:writing EResetJournal entry 2024-05-10T10:39:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240a86b800 con 0x56240a873c00 2024-05-10T10:39:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x56240a898c00 0x56240a890580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240a899000 con 0x56240a898c00 2024-05-10T10:39:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240a86b000 con 0x56240a86a400 2024-05-10T10:39:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x56240a86b400 0x56240a890b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240a873800 con 0x56240a86b400 2024-05-10T10:39:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240b3a7c00 con 0x56240a873c00 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.556+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x56240b3a7400 0x56240a891080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.557+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240b3a7800 con 0x56240b3a7400 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.557+0000 7fae52f02640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x56240a86b400 0x56240a890b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.557+0000 7fae53703640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x56240a898c00 0x56240a890580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.557+0000 7fae53f04640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x56240b3a7400 0x56240a891080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.557+0000 7fae52f02640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x56240a86b400 0x56240a890b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=222d90573208e481 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.557+0000 7fae53f04640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x56240b3a7400 0x56240a891080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=a4a9c8adfac7c264 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.557+0000 7fae53703640 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x56240a898c00 0x56240a890580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=7006aaf04b68362d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.558+0000 7fae53f04640 1 -- 172.21.15.53:0/3157750314 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(8 200.00000004 [delete] v88'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56240a57e000 con 0x56240a86a400 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.558+0000 7fae52f02640 1 -- 172.21.15.53:0/3157750314 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(9 200.00000005 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56240a57e500 con 0x56240a86b400 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.558+0000 7fae53f04640 1 -- 172.21.15.53:0/3157750314 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== osd_op_reply(6 200.00000002 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56240a57e000 con 0x56240a873c00 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.558+0000 7fae53f04640 1 -- 172.21.15.53:0/3157750314 <== osd.4 v2:172.21.15.135:6800/3517077655 3 ==== osd_op_reply(10 200.00000006 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56240a57e000 con 0x56240a873c00 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.559+0000 7fae53703640 1 -- 172.21.15.53:0/3157750314 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(7 200.00000003 [delete] v88'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56240a57e780 con 0x56240a898c00 2024-05-10T10:39:11.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.559+0000 7fae48eee640 1 -- 172.21.15.53:0/3157750314 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x56240a898400 con 0x56240a873c00 2024-05-10T10:39:11.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.559+0000 7fae53f04640 1 -- 172.21.15.53:0/3157750314 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(11 200.00000007 [delete] v88'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56240a57e000 con 0x56240b3a7400 2024-05-10T10:39:11.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae53f04640 1 -- 172.21.15.53:0/3157750314 <== osd.4 v2:172.21.15.135:6800/3517077655 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v88'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56240a57e000 con 0x56240a873c00 2024-05-10T10:39:11.567 INFO:teuthology.orchestra.run.smithi053.stdout:done 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x56240a86ac00 msgr2=0x56240a890000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x56240a86ac00 0x56240a890000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x56240a86b400 msgr2=0x56240a890b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x56240a86b400 0x56240a890b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x56240a873c00 msgr2=0x56240a891b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x56240a873c00 0x56240a891b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x56240a898c00 msgr2=0x56240a890580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x56240a898c00 0x56240a890580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x56240a86a400 msgr2=0x56240b24b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x56240a86a400 0x56240b24b700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x56240b3a7400 msgr2=0x56240a891080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x56240b3a7400 0x56240a891080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3b1c00 msgr2=0x56240b24ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3b1c00 0x56240b24ac00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x56240a7a7ef0 tx=0x56240b210a80 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 shutdown_connections 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x56240b3a7400 0x56240a891080 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x56240a873c00 0x56240a891b80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x56240a86ac00 0x56240a890000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x56240a898c00 0x56240a890580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae52f02640 1 -- 172.21.15.53:0/3157750314 reap_dead start 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a86a000 0x56240b24b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x56240a86a400 0x56240b24b700 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3b1c00 0x56240b24ac00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x56240a86b400 0x56240a890b00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 --2- 172.21.15.53:0/3157750314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a7000 0x56240b24a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 >> 172.21.15.53:0/3157750314 conn(0x56240b3a6c00 msgr2=0x56240a56b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 shutdown_connections 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.560+0000 7fae54f57280 1 -- 172.21.15.53:0/3157750314 wait complete. 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x56240b3b0c00 msgr2=0x56240b24a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x56240b3b0c00 0x56240b24a100 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56240b1fede0 tx=0x56240a789800 comp rx=0 tx=0).stop 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6000 msgr2=0x56240b249080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6000 0x56240b249080 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x56240b283bf0 tx=0x56240b34cc00 comp rx=0 tx=0).stop 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 shutdown_connections 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x56240b3b0c00 0x56240b24a100 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240b3a6800 0x56240b249b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240b3a6400 0x56240b249600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/1769368319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240b3a6000 0x56240b249080 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 >> 172.21.15.53:0/1769368319 conn(0x56240b26bc00 msgr2=0x56240a56b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 shutdown_connections 2024-05-10T10:39:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/1769368319 wait complete. 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/334014334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240a471800 msgr2=0x56240b248580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240a471800 0x56240b248580 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x56240b29f710 tx=0x56240b211f20 comp rx=0 tx=0).stop 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/334014334 shutdown_connections 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56240a471c00 0x56240b248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56240a471800 0x56240b248580 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 --2- 172.21.15.53:0/334014334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56240a470c00 0x56240b248000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/334014334 >> 172.21.15.53:0/334014334 conn(0x56240a471000 msgr2=0x56240a56a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/334014334 shutdown_connections 2024-05-10T10:39:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.561+0000 7fae54f57280 1 -- 172.21.15.53:0/334014334 wait complete. 2024-05-10T10:39:11.573 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.115778, result: old journal was 4212139~2605 new journal start will be 8388608 (4173864 bytes past old end) writing journal head writing EResetJournal entry done 2024-05-10T10:39:11.573 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-10T10:39:11.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.716+0000 7f4c22946640 1 Processor -- start 2024-05-10T10:39:11.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.717+0000 7f4c22946640 1 -- start start 2024-05-10T10:39:11.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.717+0000 7f4c22946640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c1c06e460 0x7f4c1c0d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.717+0000 7f4c22946640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c1c0d1ed0 con 0x7f4c1c0d1620 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.717+0000 7f4c22946640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c1c0d21b0 con 0x7f4c1c15f400 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.717+0000 7f4c22946640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4c1c0d2490 con 0x7f4c1c06e460 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.718+0000 7f4c1b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c1c06e460 0x7f4c1c0d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.718+0000 7f4c20ee3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4c1c0d1620 0x7f4c1c0b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37498/0 (socket says 172.21.15.53:37498) 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.718+0000 7f4c20ee3640 1 -- 172.21.15.53:0/1403698019 learned_addr learned my addr 172.21.15.53:0/1403698019 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.718+0000 7f4c1b7fe640 1 -- 172.21.15.53:0/1403698019 >> v1:172.21.15.135:6789/0 conn(0x7f4c1c15f400 legacy=0x7f4c1c06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.718+0000 7f4c1b7fe640 1 -- 172.21.15.53:0/1403698019 >> v1:172.21.15.53:6789/0 conn(0x7f4c1c0d1620 legacy=0x7f4c1c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.718+0000 7f4c1b7fe640 1 -- 172.21.15.53:0/1403698019 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c0d2770 con 0x7f4c1c06e460 2024-05-10T10:39:11.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.718+0000 7f4c1affd640 1 -- 172.21.15.53:0/1403698019 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1401613475 0 0) 0x7f4c1c0d1ed0 con 0x7f4c1c0d1620 2024-05-10T10:39:11.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c1b7fe640 1 --2- 172.21.15.53:0/1403698019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c06e460 0x7f4c1c0d10e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4c100097a0 tx=0x7f4c1002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c006d08301705d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c1affd640 1 -- 172.21.15.53:0/1403698019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10037070 con 0x7f4c1c06e460 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c1affd640 1 -- 172.21.15.53:0/1403698019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c10032440 con 0x7f4c1c06e460 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c1affd640 1 -- 172.21.15.53:0/1403698019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10032810 con 0x7f4c1c06e460 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c22946640 1 -- 172.21.15.53:0/1403698019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c06e460 msgr2=0x7f4c1c0d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c22946640 1 --2- 172.21.15.53:0/1403698019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c06e460 0x7f4c1c0d10e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4c100097a0 tx=0x7f4c1002a230 comp rx=0 tx=0).stop 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c22946640 1 -- 172.21.15.53:0/1403698019 shutdown_connections 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c22946640 1 --2- 172.21.15.53:0/1403698019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c06e460 0x7f4c1c0d10e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c22946640 1 -- 172.21.15.53:0/1403698019 >> 172.21.15.53:0/1403698019 conn(0x7f4c1c09ba20 msgr2=0x7f4c1c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c22946640 1 -- 172.21.15.53:0/1403698019 shutdown_connections 2024-05-10T10:39:11.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.719+0000 7f4c22946640 1 -- 172.21.15.53:0/1403698019 wait complete. 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.721+0000 7f4c22946640 1 Processor -- start 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.721+0000 7f4c22946640 1 -- start start 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.721+0000 7f4c22946640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c1c06e460 0x7f4c1c14cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c22946640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c0d1620 0x7f4c1c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c22946640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c1c15f400 0x7f4c1c161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c22946640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c0d1bd0 con 0x7f4c1c0d1620 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c22946640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c0d1a50 con 0x7f4c1c06e460 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c22946640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1c0b2de0 con 0x7f4c1c15f400 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c0d1620 0x7f4c1c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c0d1620 0x7f4c1c14c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60784/0 (socket says 172.21.15.53:60784) 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 -- 172.21.15.53:0/810445167 learned_addr learned my addr 172.21.15.53:0/810445167 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:11.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1bfff640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c1c06e460 0x7f4c1c14cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c20ee3640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c1c15f400 0x7f4c1c161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 -- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c1c15f400 msgr2=0x7f4c1c161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c1c15f400 0x7f4c1c161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 -- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c1c06e460 msgr2=0x7f4c1c14cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c1c06e460 0x7f4c1c14cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 -- 172.21.15.53:0/810445167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c14d1c0 con 0x7f4c1c0d1620 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c20ee3640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c1c15f400 0x7f4c1c161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1bfff640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c1c06e460 0x7f4c1c14cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c1b7fe640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c0d1620 0x7f4c1c14c460 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4c100098c0 tx=0x7f4c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c18ff9640 1 -- 172.21.15.53:0/810445167 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10037040 con 0x7f4c1c0d1620 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c18ff9640 1 -- 172.21.15.53:0/810445167 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c10042070 con 0x7f4c1c0d1620 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1c0c6d60 con 0x7f4c1c0d1620 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c18ff9640 1 -- 172.21.15.53:0/810445167 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10030a70 con 0x7f4c1c0d1620 2024-05-10T10:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.722+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1c0c7320 con 0x7f4c1c0d1620 2024-05-10T10:39:11.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.723+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1c0693f0 con 0x7f4c1c0d1620 2024-05-10T10:39:11.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.724+0000 7f4c18ff9640 1 -- 172.21.15.53:0/810445167 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c10030c10 con 0x7f4c1c0d1620 2024-05-10T10:39:11.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.725+0000 7f4c18ff9640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c080762d0 0x7f4c08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:11.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.725+0000 7f4c18ff9640 1 -- 172.21.15.53:0/810445167 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f4c100b8180 con 0x7f4c1c0d1620 2024-05-10T10:39:11.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.725+0000 7f4c1bfff640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c080762d0 0x7f4c08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:11.727 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.725+0000 7f4c1bfff640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c080762d0 0x7f4c08078790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c000fb0 tx=0x7f4c0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:11.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.729+0000 7f4c18ff9640 1 -- 172.21.15.53:0/810445167 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4c10081990 con 0x7f4c1c0d1620 2024-05-10T10:39:11.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:11.963+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4c1c15fb00 con 0x7f4c1c0d1620 2024-05-10T10:39:12.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.383+0000 7f4c18ff9640 1 -- 172.21.15.53:0/810445167 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v63) ==== 162+0+0 (secure 0 0 0) 0x7f4c10081290 con 0x7f4c1c0d1620 2024-05-10T10:39:12.384 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-10T10:39:12.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c080762d0 msgr2=0x7f4c08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c080762d0 0x7f4c08078790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c000fb0 tx=0x7f4c0c009040 comp rx=0 tx=0).stop 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c0d1620 msgr2=0x7f4c1c14c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c0d1620 0x7f4c1c14c460 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4c100098c0 tx=0x7f4c100046c0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 shutdown_connections 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4c080762d0 0x7f4c08078790 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c1c15f400 0x7f4c1c161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c1c0d1620 0x7f4c1c14c460 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 --2- 172.21.15.53:0/810445167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c1c06e460 0x7f4c1c14cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.385+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 >> 172.21.15.53:0/810445167 conn(0x7f4c1c09ba20 msgr2=0x7f4c1c070250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.386+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 shutdown_connections 2024-05-10T10:39:12.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.386+0000 7f4c22946640 1 -- 172.21.15.53:0/810445167 wait complete. 2024-05-10T10:39:12.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:39:12.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.507+0000 7f7b3a7af640 1 Processor -- start 2024-05-10T10:39:12.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.507+0000 7f7b3a7af640 1 -- start start 2024-05-10T10:39:12.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.508+0000 7f7b3a7af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7b340b8860 0x7f7b3406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.508+0000 7f7b3a7af640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b3415fc90 con 0x7f7b340d0230 2024-05-10T10:39:12.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.508+0000 7f7b3a7af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b340b8c40 con 0x7f7b340cfd80 2024-05-10T10:39:12.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.508+0000 7f7b3a7af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7b3406ee50 con 0x7f7b340b8860 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.508+0000 7f7b2bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7b340d0230 0x7f7b340cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37502/0 (socket says 172.21.15.53:37502) 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.508+0000 7f7b2bfff640 1 -- 172.21.15.53:0/866470424 learned_addr learned my addr 172.21.15.53:0/866470424 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.508+0000 7f7b33fff640 1 --2- 172.21.15.53:0/866470424 >> v2:172.21.15.135:3300/0 conn(0x7f7b340b8860 0x7f7b3406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 937027057 0 0) 0x7f7b3415fc90 con 0x7f7b340d0230 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b20003520 con 0x7f7b340d0230 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210568421 0 0) 0x7f7b340b8c40 con 0x7f7b340cfd80 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b3415fc90 con 0x7f7b340cfd80 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2670585690 0 0) 0x7f7b20003520 con 0x7f7b340d0230 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 >> v2:172.21.15.135:3300/0 conn(0x7f7b340b8860 msgr2=0x7f7b3406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 --2- 172.21.15.53:0/866470424 >> v2:172.21.15.135:3300/0 conn(0x7f7b340b8860 0x7f7b3406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 >> v1:172.21.15.135:6789/0 conn(0x7f7b340cfd80 legacy=0x7f7b340ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b340cd570 con 0x7f7b340d0230 2024-05-10T10:39:12.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f7b1c0015f0 con 0x7f7b340d0230 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 >> v1:172.21.15.53:6789/0 conn(0x7f7b340d0230 legacy=0x7f7b340cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20003b90 0x7f7b20003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b20004800 0x7f7b20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b20005210 0x7f7b2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3406ee50 con 0x7f7b20004800 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2000c140 con 0x7f7b20005210 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2000c340 con 0x7f7b20003b90 2024-05-10T10:39:12.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b2bfff640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b20004800 0x7f7b20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b33fff640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20003b90 0x7f7b20003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b38d4c640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b20005210 0x7f7b2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b2bfff640 1 -- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20003b90 msgr2=0x7f7b20003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b2bfff640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20003b90 0x7f7b20003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b2bfff640 1 -- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b20005210 msgr2=0x7f7b2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b2bfff640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b20005210 0x7f7b2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b2bfff640 1 -- 172.21.15.53:0/866470424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b20003520 con 0x7f7b20004800 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b33fff640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20003b90 0x7f7b20003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b38d4c640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b20005210 0x7f7b2000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:12.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.509+0000 7f7b38d4c640 1 -- 172.21.15.53:0/866470424 reap_dead start 2024-05-10T10:39:12.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.510+0000 7f7b2bfff640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b20004800 0x7f7b20004c80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c007f80 tx=0x7f7b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb1fa12dc6bbd4f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:12.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.510+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c007ba0 con 0x7f7b20004800 2024-05-10T10:39:12.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.510+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1c005bb0 con 0x7f7b20004800 2024-05-10T10:39:12.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.510+0000 7f7b337fe640 1 -- 172.21.15.53:0/866470424 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c007070 con 0x7f7b20004800 2024-05-10T10:39:12.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.511+0000 7f7b3a7af640 1 -- 172.21.15.53:0/866470424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b20004800 msgr2=0x7f7b20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.511+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b20004800 0x7f7b20004c80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c007f80 tx=0x7f7b20005830 comp rx=0 tx=0).stop 2024-05-10T10:39:12.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.511+0000 7f7b3a7af640 1 -- 172.21.15.53:0/866470424 shutdown_connections 2024-05-10T10:39:12.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.511+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/866470424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b20004800 0x7f7b20004c80 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.511+0000 7f7b3a7af640 1 -- 172.21.15.53:0/866470424 >> 172.21.15.53:0/866470424 conn(0x7f7b340d2df0 msgr2=0x7f7b340d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:12.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.511+0000 7f7b3a7af640 1 -- 172.21.15.53:0/866470424 shutdown_connections 2024-05-10T10:39:12.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.511+0000 7f7b3a7af640 1 -- 172.21.15.53:0/866470424 wait complete. 2024-05-10T10:39:12.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.512+0000 7f7b3a7af640 1 Processor -- start 2024-05-10T10:39:12.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.512+0000 7f7b3a7af640 1 -- start start 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b3a7af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20004800 0x7f7b341f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b3a7af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b3415d0e0 0x7f7b340cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b3a7af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b3415d560 0x7f7b340cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b3a7af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b340cde10 con 0x7f7b3415d560 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b3a7af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b340cdb30 con 0x7f7b3415d0e0 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b3a7af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7b340b5c40 con 0x7f7b20004800 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b3415d560 0x7f7b340cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b3415d560 0x7f7b340cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60808/0 (socket says 172.21.15.53:60808) 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 -- 172.21.15.53:0/832743497 learned_addr learned my addr 172.21.15.53:0/832743497 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b33fff640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20004800 0x7f7b341f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b2bfff640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b3415d0e0 0x7f7b340cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 -- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20004800 msgr2=0x7f7b341f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20004800 0x7f7b341f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 -- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b3415d0e0 msgr2=0x7f7b340cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b3415d0e0 0x7f7b340cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 -- 172.21.15.53:0/832743497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b340b6390 con 0x7f7b3415d560 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.513+0000 7f7b38d4c640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b3415d560 0x7f7b340cbd20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7b3406d220 tx=0x7f7b2c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.514+0000 7f7b317fa640 1 -- 172.21.15.53:0/832743497 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c006b50 con 0x7f7b3415d560 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.514+0000 7f7b317fa640 1 -- 172.21.15.53:0/832743497 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b2c006cf0 con 0x7f7b3415d560 2024-05-10T10:39:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.514+0000 7f7b317fa640 1 -- 172.21.15.53:0/832743497 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c003970 con 0x7f7b3415d560 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.514+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b340b6670 con 0x7f7b3415d560 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.514+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b340b6c30 con 0x7f7b3415d560 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.515+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3406a220 con 0x7f7b3415d560 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.515+0000 7f7b317fa640 1 -- 172.21.15.53:0/832743497 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7b2c01a020 con 0x7f7b3415d560 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.516+0000 7f7b317fa640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b140762d0 0x7f7b14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.516+0000 7f7b317fa640 1 -- 172.21.15.53:0/832743497 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f7b2c096c80 con 0x7f7b3415d560 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.516+0000 7f7b33fff640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b140762d0 0x7f7b14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.516+0000 7f7b33fff640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b140762d0 0x7f7b14078790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009ae0 tx=0x7f7b24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.520+0000 7f7b317fa640 1 -- 172.21.15.53:0/832743497 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7b2c05fe40 con 0x7f7b3415d560 2024-05-10T10:39:12.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.752+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b340d1e20 con 0x7f7b3415d560 2024-05-10T10:39:12.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.753+0000 7f7b317fa640 1 -- 172.21.15.53:0/832743497 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5177 (secure 0 0 0) 0x7f7b2c064140 con 0x7f7b3415d560 2024-05-10T10:39:12.753 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:12.753 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:12.386237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"a","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.53:6833/285942496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":285942496},{"type":"v1","addr":"172.21.15.53:6833","nonce":285942496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-10T10:39:12.754 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 64 2024-05-10T10:39:12.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b140762d0 msgr2=0x7f7b14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b140762d0 0x7f7b14078790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009ae0 tx=0x7f7b24008040 comp rx=0 tx=0).stop 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b3415d560 msgr2=0x7f7b340cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b3415d560 0x7f7b340cbd20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7b3406d220 tx=0x7f7b2c00eb90 comp rx=0 tx=0).stop 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 shutdown_connections 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7b140762d0 0x7f7b14078790 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b3415d560 0x7f7b340cbd20 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b3415d0e0 0x7f7b340cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 --2- 172.21.15.53:0/832743497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7b20004800 0x7f7b341f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 >> 172.21.15.53:0/832743497 conn(0x7f7b340d2df0 msgr2=0x7f7b34070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 shutdown_connections 2024-05-10T10:39:12.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:12.755+0000 7f7b3a7af640 1 -- 172.21.15.53:0/832743497 wait complete. 2024-05-10T10:39:13.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:38:57.607866+0000', 'modified': '2024-05-10T10:39:12.386237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6888': {'gid': 6888, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/285942496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 285942496}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 285942496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:39:13.169 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6888:up:replay 2024-05-10T10:39:14.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:39:14.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.288+0000 7fda78eb4640 1 Processor -- start 2024-05-10T10:39:14.289 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.288+0000 7fda78eb4640 1 -- start start 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda78eb4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fda74162940 0x7fda740b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda78eb4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda740d59d0 0x7fda740d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda78eb4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda740b47a0 0x7fda740d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda78eb4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fda74162d20 con 0x7fda740d59d0 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda78eb4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fda740d2d20 con 0x7fda74162940 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda78eb4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fda740d3000 con 0x7fda740b47a0 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda71d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda740d59d0 0x7fda740d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda71d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda740d59d0 0x7fda740d3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49284/0 (socket says 172.21.15.53:49284) 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda71d9b640 1 -- 172.21.15.53:0/4109531261 learned_addr learned my addr 172.21.15.53:0/4109531261 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda7259c640 1 --2- 172.21.15.53:0/4109531261 >> v2:172.21.15.135:3300/0 conn(0x7fda74162940 0x7fda740b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:14.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.289+0000 7fda72d9d640 1 --2- 172.21.15.53:0/4109531261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda740b47a0 0x7fda740d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda71d9b640 1 -- 172.21.15.53:0/4109531261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda740b47a0 msgr2=0x7fda740d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda71d9b640 1 --2- 172.21.15.53:0/4109531261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda740b47a0 0x7fda740d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda71d9b640 1 -- 172.21.15.53:0/4109531261 >> v2:172.21.15.135:3300/0 conn(0x7fda74162940 msgr2=0x7fda740b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda71d9b640 1 --2- 172.21.15.53:0/4109531261 >> v2:172.21.15.135:3300/0 conn(0x7fda74162940 0x7fda740b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda71d9b640 1 -- 172.21.15.53:0/4109531261 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda740d32e0 con 0x7fda740d59d0 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda72d9d640 1 --2- 172.21.15.53:0/4109531261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda740b47a0 0x7fda740d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda71d9b640 1 --2- 172.21.15.53:0/4109531261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda740d3670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fda7408d970 tx=0x7fda6802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b26b1786625207 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.290+0000 7fda7159a640 1 -- 172.21.15.53:0/4109531261 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68037070 con 0x7fda740d59d0 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda7159a640 1 -- 172.21.15.53:0/4109531261 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda68032440 con 0x7fda740d59d0 2024-05-10T10:39:14.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda7159a640 1 -- 172.21.15.53:0/4109531261 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68032810 con 0x7fda740d59d0 2024-05-10T10:39:14.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 -- 172.21.15.53:0/4109531261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 msgr2=0x7fda740d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:14.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 --2- 172.21.15.53:0/4109531261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda740d3670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fda7408d970 tx=0x7fda6802a140 comp rx=0 tx=0).stop 2024-05-10T10:39:14.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 -- 172.21.15.53:0/4109531261 shutdown_connections 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 --2- 172.21.15.53:0/4109531261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda740b47a0 0x7fda740d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 --2- 172.21.15.53:0/4109531261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda740d3670 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 --2- 172.21.15.53:0/4109531261 >> v2:172.21.15.135:3300/0 conn(0x7fda74162940 0x7fda740b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 -- 172.21.15.53:0/4109531261 >> 172.21.15.53:0/4109531261 conn(0x7fda7409e380 msgr2=0x7fda740d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 -- 172.21.15.53:0/4109531261 shutdown_connections 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.291+0000 7fda78eb4640 1 -- 172.21.15.53:0/4109531261 wait complete. 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 Processor -- start 2024-05-10T10:39:14.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 -- start start 2024-05-10T10:39:14.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda740b47a0 0x7fda740c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:14.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda7415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:14.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda74162940 0x7fda7415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:14.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda740d2390 con 0x7fda740d59d0 2024-05-10T10:39:14.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda740cfc50 con 0x7fda740b47a0 2024-05-10T10:39:14.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.293+0000 7fda78eb4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fda740d2510 con 0x7fda74162940 2024-05-10T10:39:14.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda7415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda7415fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49298/0 (socket says 172.21.15.53:49298) 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 -- 172.21.15.53:0/2157865141 learned_addr learned my addr 172.21.15.53:0/2157865141 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda7259c640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda740b47a0 0x7fda740c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda72d9d640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda74162940 0x7fda7415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 -- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda74162940 msgr2=0x7fda7415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda74162940 0x7fda7415cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 -- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda740b47a0 msgr2=0x7fda740c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda740b47a0 0x7fda740c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 -- 172.21.15.53:0/2157865141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda740c9aa0 con 0x7fda740d59d0 2024-05-10T10:39:14.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda72d9d640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda74162940 0x7fda7415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda71d9b640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda7415fec0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fda6802a180 tx=0x7fda680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda62ffd640 1 -- 172.21.15.53:0/2157865141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68041020 con 0x7fda740d59d0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda62ffd640 1 -- 172.21.15.53:0/2157865141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda680306c0 con 0x7fda740d59d0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda740ca960 con 0x7fda740d59d0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda62ffd640 1 -- 172.21.15.53:0/2157865141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68037040 con 0x7fda740d59d0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.294+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda740caf20 con 0x7fda740d59d0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.296+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7406a220 con 0x7fda740d59d0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.296+0000 7fda62ffd640 1 -- 172.21.15.53:0/2157865141 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fda68044020 con 0x7fda740d59d0 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.297+0000 7fda62ffd640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda54076300 0x7fda540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:14.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.297+0000 7fda62ffd640 1 -- 172.21.15.53:0/2157865141 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7fda680b7ff0 con 0x7fda740d59d0 2024-05-10T10:39:14.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.297+0000 7fda7259c640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda54076300 0x7fda540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:14.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.298+0000 7fda7259c640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda54076300 0x7fda540787c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fda64006fd0 tx=0x7fda64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:14.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.301+0000 7fda62ffd640 1 -- 172.21.15.53:0/2157865141 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fda6804b030 con 0x7fda740d59d0 2024-05-10T10:39:14.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.530+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda740b2160 con 0x7fda740d59d0 2024-05-10T10:39:14.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.531+0000 7fda62ffd640 1 -- 172.21.15.53:0/2157865141 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5177 (secure 0 0 0) 0x7fda68081770 con 0x7fda740d59d0 2024-05-10T10:39:14.531 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:14.531 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:13.394302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"a","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.53:6833/285942496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":285942496},{"type":"v1","addr":"172.21.15.53:6833","nonce":285942496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-10T10:39:14.532 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 66 2024-05-10T10:39:14.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda54076300 msgr2=0x7fda540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:14.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda54076300 0x7fda540787c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fda64006fd0 tx=0x7fda64008040 comp rx=0 tx=0).stop 2024-05-10T10:39:14.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 msgr2=0x7fda7415fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:14.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda7415fec0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fda6802a180 tx=0x7fda680046c0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 shutdown_connections 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fda54076300 0x7fda540787c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fda74162940 0x7fda7415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda740d59d0 0x7fda7415fec0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 --2- 172.21.15.53:0/2157865141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda740b47a0 0x7fda740c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.532+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 >> 172.21.15.53:0/2157865141 conn(0x7fda7409e380 msgr2=0x7fda740d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.533+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 shutdown_connections 2024-05-10T10:39:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:14.533+0000 7fda78eb4640 1 -- 172.21.15.53:0/2157865141 wait complete. 2024-05-10T10:39:14.760 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:39:14.763 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:39:14.946 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:38:57.607866+0000', 'modified': '2024-05-10T10:39:13.394302+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6888': {'gid': 6888, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.53:6833/285942496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 285942496}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 285942496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:39:14.946 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6888:up:rejoin 2024-05-10T10:39:15.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:39:16.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.064+0000 7f7fff191640 1 Processor -- start 2024-05-10T10:39:16.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.064+0000 7f7fff191640 1 -- start start 2024-05-10T10:39:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7fff191640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ff815f3b0 0x7f7ff80d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7fff191640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815f790 0x7f7ff80d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7fff191640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff80d30b0 con 0x7f7ff80d5d30 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7fff191640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff80d3390 con 0x7f7ff815f3b0 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7fff191640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff80c8c80 con 0x7f7ff815f790 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ffd72e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7ff80d5d30 0x7f7ff80c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39726/0 (socket says 172.21.15.53:39726) 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ffd72e640 1 -- 172.21.15.53:0/1351448777 learned_addr learned my addr 172.21.15.53:0/1351448777 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ffcf2d640 1 --2- 172.21.15.53:0/1351448777 >> v2:172.21.15.135:3300/0 conn(0x7f7ff815f3b0 0x7f7ff80d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815f790 0x7f7ff80d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804196815 0 0) 0x7f7ff80d30b0 con 0x7f7ff80d5d30 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe4003520 con 0x7f7ff80d5d30 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1629070839 0 0) 0x7f7fe4003520 con 0x7f7ff80d5d30 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815f790 msgr2=0x7f7ff80d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815f790 0x7f7ff80d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 >> v2:172.21.15.135:3300/0 conn(0x7f7ff815f3b0 msgr2=0x7f7ff80d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 --2- 172.21.15.53:0/1351448777 >> v2:172.21.15.135:3300/0 conn(0x7f7ff815f3b0 0x7f7ff80d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff80c8f60 con 0x7f7ff80d5d30 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f7ff0001670 con 0x7f7ff80d5d30 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.065+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 >> v1:172.21.15.53:6789/0 conn(0x7f7ff80d5d30 legacy=0x7f7ff80c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff77fe640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7fe40039b0 0x7f7fe4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff77fe640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7fe4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff77fe640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe4004e50 0x7f7fe40052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80d3390 con 0x7f7fe4004310 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80c8c80 con 0x7f7fe4004e50 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe400bd40 con 0x7f7fe40039b0 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7fe4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ffcf2d640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7fe40039b0 0x7f7fe4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ffd72e640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe4004e50 0x7f7fe40052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff7fff640 1 -- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7fe40039b0 msgr2=0x7f7fe4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7fe40039b0 0x7f7fe4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff7fff640 1 -- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe4004e50 msgr2=0x7f7fe40052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe4004e50 0x7f7fe40052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.066+0000 7f7ff7fff640 1 -- 172.21.15.53:0/1351448777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe4003520 con 0x7f7fe4004310 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.067+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7fe4004800 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009f90 tx=0x7f7fe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cddbde590465838 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.067+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8009c10 con 0x7f7fe4004310 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.067+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe8036070 con 0x7f7fe4004310 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.067+0000 7f7ff77fe640 1 -- 172.21.15.53:0/1351448777 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8031810 con 0x7f7fe4004310 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.067+0000 7f7fff191640 1 -- 172.21.15.53:0/1351448777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 msgr2=0x7f7fe4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.067+0000 7f7fff191640 1 --2- 172.21.15.53:0/1351448777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7fe4004800 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009f90 tx=0x7f7fe4005830 comp rx=0 tx=0).stop 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.067+0000 7f7ffd72e640 1 -- 172.21.15.53:0/1351448777 reap_dead start 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.068+0000 7f7fff191640 1 -- 172.21.15.53:0/1351448777 shutdown_connections 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.068+0000 7f7fff191640 1 -- 172.21.15.53:0/1351448777 >> 172.21.15.53:0/1351448777 conn(0x7f7ff80d5130 msgr2=0x7f7ff80d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.068+0000 7f7fff191640 1 -- 172.21.15.53:0/1351448777 shutdown_connections 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.068+0000 7f7fff191640 1 -- 172.21.15.53:0/1351448777 wait complete. 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.069+0000 7f7fff191640 1 Processor -- start 2024-05-10T10:39:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.069+0000 7f7fff191640 1 -- start start 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7fff191640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7ff80b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7fff191640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff8157680 0x7f7ff8161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7fff191640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815c830 0x7f7ff815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7fff191640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80d3db0 con 0x7f7fe4004310 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7fff191640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80d3c30 con 0x7f7ff8157680 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7fff191640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff80d3f30 con 0x7f7ff815c830 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7ff80b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7ff80b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49316/0 (socket says 172.21.15.53:49316) 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 -- 172.21.15.53:0/4287867935 learned_addr learned my addr 172.21.15.53:0/4287867935 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff8157680 0x7f7ff8161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffd72e640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815c830 0x7f7ff815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 -- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815c830 msgr2=0x7f7ff815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815c830 0x7f7ff815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 -- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff8157680 msgr2=0x7f7ff8161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff8157680 0x7f7ff8161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 -- 172.21.15.53:0/4287867935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff80b8d10 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffd72e640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815c830 0x7f7ff815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.070+0000 7f7ffcf2d640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7ff80b8730 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0011990 tx=0x7f7fe0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.071+0000 7f7ff57fa640 1 -- 172.21.15.53:0/4287867935 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000fa10 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.071+0000 7f7ff57fa640 1 -- 172.21.15.53:0/4287867935 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe000fbb0 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.071+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff80b8ff0 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.071+0000 7f7ff57fa640 1 -- 172.21.15.53:0/4287867935 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0014440 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.071+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff80b95b0 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.072+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff806a220 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.072+0000 7f7ff57fa640 1 -- 172.21.15.53:0/4287867935 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7fe0012040 con 0x7f7fe4004310 2024-05-10T10:39:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.073+0000 7f7ff57fa640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fd80762d0 0x7f7fd8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.073+0000 7f7ff57fa640 1 -- 172.21.15.53:0/4287867935 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f7fe0097dd0 con 0x7f7fe4004310 2024-05-10T10:39:16.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.073+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fd80762d0 0x7f7fd8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.074+0000 7f7ff7fff640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fd80762d0 0x7f7fd8078790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8035000 tx=0x7f7fe802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.077+0000 7f7ff57fa640 1 -- 172.21.15.53:0/4287867935 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7fe0019030 con 0x7f7fe4004310 2024-05-10T10:39:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.307+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ff806eb50 con 0x7f7fe4004310 2024-05-10T10:39:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.308+0000 7f7ff57fa640 1 -- 172.21.15.53:0/4287867935 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5184 (secure 0 0 0) 0x7f7fe0061550 con 0x7f7fe4004310 2024-05-10T10:39:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:16.309 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:15.393521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/285942496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":285942496},{"type":"v1","addr":"172.21.15.53:6833","nonce":285942496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6888,"qdb_cluster":[6888]},"id":6}]} 2024-05-10T10:39:16.309 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 67 2024-05-10T10:39:16.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fd80762d0 msgr2=0x7f7fd8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fd80762d0 0x7f7fd8078790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8035000 tx=0x7f7fe802f000 comp rx=0 tx=0).stop 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 msgr2=0x7f7ff80b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7ff80b8730 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0011990 tx=0x7f7fe0011e50 comp rx=0 tx=0).stop 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 shutdown_connections 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7fd80762d0 0x7f7fd8078790 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ff815c830 0x7f7ff815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff8157680 0x7f7ff8161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 --2- 172.21.15.53:0/4287867935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4004310 0x7f7ff80b8730 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 >> 172.21.15.53:0/4287867935 conn(0x7f7ff80d5130 msgr2=0x7f7ff815e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 shutdown_connections 2024-05-10T10:39:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.310+0000 7f7fff191640 1 -- 172.21.15.53:0/4287867935 wait complete. 2024-05-10T10:39:16.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:38:57.607866+0000', 'modified': '2024-05-10T10:39:15.393521+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6888': {'gid': 6888, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6833/285942496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 285942496}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 285942496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6888, 'qdb_cluster': [6888]} 2024-05-10T10:39:16.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:39:16.724 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T10:39:16.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.832+0000 7f6257990640 1 Processor -- start 2024-05-10T10:39:16.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.832+0000 7f6257990640 1 -- start start 2024-05-10T10:39:16.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6257990640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 0x7f62500cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6257990640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f62500b43a0 0x7f62500b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6257990640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62500ce590 con 0x7f62500d3980 2024-05-10T10:39:16.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6257990640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f625015da30 con 0x7f62500b43a0 2024-05-10T10:39:16.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6257990640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f62500d0500 con 0x7f625015d650 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6254f2b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f62500d3980 0x7f62500cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39742/0 (socket says 172.21.15.53:39742) 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6254f2b640 1 -- 172.21.15.53:0/1387052798 learned_addr learned my addr 172.21.15.53:0/1387052798 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6255f2d640 1 --2- 172.21.15.53:0/1387052798 >> v2:172.21.15.135:3300/0 conn(0x7f62500b43a0 0x7f62500b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f625572c640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 0x7f62500cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2019129143 0 0) 0x7f6240001110 con 0x7f62500d3980 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f623c003520 con 0x7f62500d3980 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 905326262 0 0) 0x7f623c003520 con 0x7f62500d3980 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 msgr2=0x7f62500cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6247fff640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 0x7f62500cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 >> v2:172.21.15.135:3300/0 conn(0x7f62500b43a0 msgr2=0x7f62500b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.833+0000 7f6247fff640 1 --2- 172.21.15.53:0/1387052798 >> v2:172.21.15.135:3300/0 conn(0x7f62500b43a0 0x7f62500b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62500ce810 con 0x7f62500d3980 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f6240001840 con 0x7f62500d3980 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 >> v1:172.21.15.53:6789/0 conn(0x7f62500d3980 legacy=0x7f62500cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 0x7f623c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c003f30 0x7f623c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f623c004d40 0x7f623c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62500d0500 con 0x7f623c003f30 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f623c00b9c0 con 0x7f623c004d40 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f623c00bb40 con 0x7f625015d650 2024-05-10T10:39:16.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6254f2b640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c003f30 0x7f623c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f625572c640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 0x7f623c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6255f2d640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f623c004d40 0x7f623c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6254f2b640 1 -- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 msgr2=0x7f623c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6254f2b640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f625015d650 0x7f623c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6254f2b640 1 -- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f623c004d40 msgr2=0x7f623c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6254f2b640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f623c004d40 0x7f623c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6254f2b640 1 -- 172.21.15.53:0/1387052798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c003520 con 0x7f623c003f30 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.834+0000 7f6255f2d640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f623c004d40 0x7f623c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6254f2b640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c003f30 0x7f623c004800 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f624000c3b0 tx=0x7f623c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e085054df82de956 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624000c0e0 con 0x7f623c003f30 2024-05-10T10:39:16.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240034070 con 0x7f623c003f30 2024-05-10T10:39:16.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6247fff640 1 -- 172.21.15.53:0/1387052798 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240005d10 con 0x7f623c003f30 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6257990640 1 -- 172.21.15.53:0/1387052798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c003f30 msgr2=0x7f623c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6257990640 1 --2- 172.21.15.53:0/1387052798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c003f30 0x7f623c004800 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f624000c3b0 tx=0x7f623c005830 comp rx=0 tx=0).stop 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6255f2d640 1 -- 172.21.15.53:0/1387052798 reap_dead start 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6257990640 1 -- 172.21.15.53:0/1387052798 shutdown_connections 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.835+0000 7f6257990640 1 -- 172.21.15.53:0/1387052798 >> 172.21.15.53:0/1387052798 conn(0x7f62500d2df0 msgr2=0x7f62500d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.836+0000 7f6257990640 1 -- 172.21.15.53:0/1387052798 shutdown_connections 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.836+0000 7f6257990640 1 -- 172.21.15.53:0/1387052798 wait complete. 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.837+0000 7f6257990640 1 Processor -- start 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.837+0000 7f6257990640 1 -- start start 2024-05-10T10:39:16.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6257990640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f623c003f30 0x7f62501502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6257990640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f625006dde0 0x7f62500c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6257990640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62500c8910 0x7f62500b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6257990640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62500d0190 con 0x7f625006dde0 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6257990640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62500cedd0 con 0x7f62500c8910 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6257990640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f62500cf540 con 0x7f623c003f30 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f625006dde0 0x7f62500c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f625006dde0 0x7f62500c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49336/0 (socket says 172.21.15.53:49336) 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 -- 172.21.15.53:0/2163820035 learned_addr learned my addr 172.21.15.53:0/2163820035 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f625572c640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f623c003f30 0x7f62501502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6255f2d640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62500c8910 0x7f62500b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 -- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f623c003f30 msgr2=0x7f62501502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f623c003f30 0x7f62501502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 -- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62500c8910 msgr2=0x7f62500b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62500c8910 0x7f62500b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 -- 172.21.15.53:0/2163820035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62500c5b40 con 0x7f625006dde0 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f625572c640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f623c003f30 0x7f62501502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f625006dde0 0x7f62500c83d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f624002afe0 tx=0x7f62500cd860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6245ffb640 1 -- 172.21.15.53:0/2163820035 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240007270 con 0x7f625006dde0 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6245ffb640 1 -- 172.21.15.53:0/2163820035 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240007410 con 0x7f625006dde0 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625006f090 con 0x7f625006dde0 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.838+0000 7f6245ffb640 1 -- 172.21.15.53:0/2163820035 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240007800 con 0x7f625006dde0 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.839+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62500c78f0 con 0x7f625006dde0 2024-05-10T10:39:16.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.840+0000 7f6245ffb640 1 -- 172.21.15.53:0/2163820035 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f624002f460 con 0x7f625006dde0 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.841+0000 7f6245ffb640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6234076300 0x7f62340787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.841+0000 7f6245ffb640 1 -- 172.21.15.53:0/2163820035 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f62400b7f40 con 0x7f625006dde0 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.841+0000 7f625572c640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6234076300 0x7f62340787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.841+0000 7f625572c640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6234076300 0x7f62340787c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f624c013370 tx=0x7f624c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 Processor -- start 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 -- start start 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09afabb0 0x558d09afe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558d09a648d0 con 0x558d09b28330 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558d09a1dc10 con 0x558d09afabb0 2024-05-10T10:39:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558d09a60890 con 0x558d09b27f50 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f6254f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.842+0000 7f6254f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b2aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49338/0 (socket says 172.21.15.53:49338) 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6254f2b640 1 -- 172.21.15.53:0/2320838888 learned_addr learned my addr 172.21.15.53:0/2320838888 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6255f2d640 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09afabb0 0x558d09afe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f625572c640 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6254f2b640 1 -- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 msgr2=0x558d09b2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b2a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6254f2b640 1 -- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09afabb0 msgr2=0x558d09afe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09afabb0 0x558d09afe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6254f2b640 1 -- 172.21.15.53:0/2320838888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d09a255d0 con 0x558d09b28330 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6255f2d640 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09afabb0 0x558d09afe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b2aca0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f624007d5a0 tx=0x7f6240080ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71790b4fbe042e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f623affd640 1 -- 172.21.15.53:0/2320838888 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240085360 con 0x558d09b28330 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f623affd640 1 -- 172.21.15.53:0/2320838888 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240034030 con 0x558d09b28330 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.843+0000 7f623affd640 1 -- 172.21.15.53:0/2320838888 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624002cdf0 con 0x558d09b28330 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 -- 172.21.15.53:0/2320838888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 msgr2=0x558d09b2aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b2aca0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f624007d5a0 tx=0x7f6240080ac0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 -- 172.21.15.53:0/2320838888 shutdown_connections 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09afabb0 0x558d09afe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b2aca0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 --2- 172.21.15.53:0/2320838888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 -- 172.21.15.53:0/2320838888 >> 172.21.15.53:0/2320838888 conn(0x558d09afce20 msgr2=0x558d09b45760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 -- 172.21.15.53:0/2320838888 shutdown_connections 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 -- 172.21.15.53:0/2320838888 wait complete. 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 -- start start 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.844+0000 7f625a785740 1 Processor -- start 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625a785740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b27f50 0x558d09b490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09b4bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b4c460 0x558d09b49e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625a785740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558d09a60890 con 0x558d09b27f50 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558d09a21760 con 0x558d09b28330 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558d09a648d0 con 0x558d09b4c460 2024-05-10T10:39:16.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625a785740 10 client.0 _ll_register_callbacks cb 0x558d09a5acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625572c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b27f50 0x558d09b490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625572c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b27f50 0x558d09b490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49342/0 (socket says 172.21.15.53:49342) 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.845+0000 7f625572c640 1 -- 172.21.15.53:0/2617335873 learned_addr learned my addr 172.21.15.53:0/2617335873 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09b4bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f6255f2d640 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b4c460 0x558d09b49e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625572c640 1 -- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b4c460 msgr2=0x558d09b49e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625572c640 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b4c460 0x558d09b49e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625572c640 1 -- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 msgr2=0x558d09b4bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625572c640 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09b4bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625572c640 1 -- 172.21.15.53:0/2617335873 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d097762d0 con 0x558d09b27f50 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625572c640 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b27f50 0x558d09b490b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f624c016ea0 tx=0x7f624c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87838224f79ee43e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f622ffff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f622ffff640 1 -- 172.21.15.53:0/2617335873 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c027070 con 0x558d09b27f50 2024-05-10T10:39:16.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f622ffff640 1 -- 172.21.15.53:0/2617335873 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f624c023610 con 0x558d09b27f50 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6240034030 con 0x558d09b27f50 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f622ffff640 1 -- 172.21.15.53:0/2617335873 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f624c023810 con 0x558d09b27f50 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625a785740 10 client.6954 fetch_fsmap learned FSMap version 67 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558d098406b0 con 0x558d09b27f50 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f622ffff640 1 -- 172.21.15.53:0/2617335873 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240034030 con 0x558d09b27f50 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f622ffff640 1 -- 172.21.15.53:0/2617335873 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f624c035080 con 0x558d09b27f50 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625a785740 10 client.6954 fetch_fsmap finished waiting for FSMap version 67 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.846+0000 7f625a785740 10 client.6954 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.847+0000 7f625a785740 20 client.6954 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.847+0000 7f625a785740 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b59410 0x558d09b597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.847+0000 7f625a785740 4 client.6954 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.847+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d099e8760 con 0x558d09b59410 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.847+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b59410 0x558d09b597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.847+0000 7f6254f2b640 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b59410 0x558d09b597f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c3ce5669335236 server_cookie=2cd38f70378e77a7 in_seq=0 out_seq=0 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.847+0000 7f622ffff640 10 client.6954 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f622ffff640 1 -- 172.21.15.53:0/2617335873 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d099e8760 con 0x558d09b59410 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f622ffff640 10 client.6954 handle_command_reply: tid=0 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 client.6954 shutdown 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b27f50 msgr2=0x558d09b490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b27f50 0x558d09b490b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f624c016ea0 tx=0x7f624c004ab0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 shutdown_connections 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b4c460 0x558d09b49e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09b4bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b59410 0x558d09b597f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 --2- 172.21.15.53:0/2617335873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b27f50 0x558d09b490b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f622ffff640 0 client.6954 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:16.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 >> 172.21.15.53:0/2617335873 conn(0x558d09afabb0 msgr2=0x558d09b003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 shutdown_connections 2024-05-10T10:39:16.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 1 -- 172.21.15.53:0/2617335873 wait complete. 2024-05-10T10:39:16.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.848+0000 7f625a785740 20 client.6954 trim_cache size 0 max 16384 2024-05-10T10:39:16.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.867+0000 7f625a785740 1 Processor -- start 2024-05-10T10:39:16.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.867+0000 7f625a785740 1 -- start start 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b5f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09af9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f625a785740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09afb2c0 0x558d09af9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f625a785740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6250201750 con 0x558d09afb2c0 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6250200ae0 con 0x558d09b28330 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6250201100 con 0x558d09b27f50 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09afb2c0 0x558d09af9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09afb2c0 0x558d09af9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49350/0 (socket says 172.21.15.53:49350) 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 -- 172.21.15.53:0/165681880 learned_addr learned my addr 172.21.15.53:0/165681880 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6254f2b640 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09af9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 -- 172.21.15.53:0/165681880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 msgr2=0x558d09b5f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b5f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 -- 172.21.15.53:0/165681880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 msgr2=0x558d09af9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09af9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 -- 172.21.15.53:0/165681880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d09b488b0 con 0x558d09afb2c0 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f6255f2d640 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09afb2c0 0x558d09af9f80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6248009a70 tx=0x7f6248009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4313fdd01eaa1987 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f623a7fc640 1 -- 172.21.15.53:0/165681880 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624801c070 con 0x558d09afb2c0 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f623a7fc640 1 -- 172.21.15.53:0/165681880 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6248016b30 con 0x558d09afb2c0 2024-05-10T10:39:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.868+0000 7f623a7fc640 1 -- 172.21.15.53:0/165681880 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248021460 con 0x558d09afb2c0 2024-05-10T10:39:16.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 -- 172.21.15.53:0/165681880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09afb2c0 msgr2=0x558d09af9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09afb2c0 0x558d09af9f80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6248009a70 tx=0x7f6248009190 comp rx=0 tx=0).stop 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 -- 172.21.15.53:0/165681880 shutdown_connections 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09afb2c0 0x558d09af9f80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b28330 0x558d09af9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 --2- 172.21.15.53:0/165681880 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b5f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 -- 172.21.15.53:0/165681880 >> 172.21.15.53:0/165681880 conn(0x558d09afabb0 msgr2=0x558d09b4ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 -- 172.21.15.53:0/165681880 shutdown_connections 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.869+0000 7f625a785740 1 -- 172.21.15.53:0/165681880 wait complete. 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.870+0000 7f625a785740 1 -- start start 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.870+0000 7f625a785740 1 Processor -- start 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b4ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b17910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b17e50 0x558d09b14190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558d09b495f0 con 0x558d09b28330 2024-05-10T10:39:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558d09a1dc10 con 0x558d09b17e50 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558d09b27400 con 0x558d09b27f50 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 10 client.0 _ll_register_callbacks cb 0x7f624c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b17910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b17910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49352/0 (socket says 172.21.15.53:49352) 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 -- 172.21.15.53:0/3834647517 learned_addr learned my addr 172.21.15.53:0/3834647517 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625572c640 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b4ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6255f2d640 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b17e50 0x558d09b14190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 -- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 msgr2=0x558d09b4ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b4ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 -- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b17e50 msgr2=0x558d09b14190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b17e50 0x558d09b14190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 -- 172.21.15.53:0/3834647517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d09b4b1d0 con 0x558d09b28330 2024-05-10T10:39:16.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f6254f2b640 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b17910 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f624007e220 tx=0x7f624000c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35364ec0ac0040ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f622ffff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f622ffff640 1 -- 172.21.15.53:0/3834647517 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624007ce70 con 0x558d09b28330 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f622ffff640 1 -- 172.21.15.53:0/3834647517 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f624002cbf0 con 0x558d09b28330 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6248016b30 con 0x558d09b28330 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f622ffff640 1 -- 172.21.15.53:0/3834647517 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f624002cdf0 con 0x558d09b28330 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 10 client.6960 fetch_fsmap learned FSMap version 67 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.871+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558d09b4b1d0 con 0x558d09b28330 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f622ffff640 1 -- 172.21.15.53:0/3834647517 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6248016b30 con 0x558d09b28330 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f622ffff640 1 -- 172.21.15.53:0/3834647517 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f6240081210 con 0x558d09b28330 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 10 client.6960 fetch_fsmap finished waiting for FSMap version 67 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 10 client.6960 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:39:16.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 20 client.6960 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b146d0 0x558d09b14ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 4 client.6960 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "status"}) -- 0x558d09a724e0 con 0x558d09b146d0 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625572c640 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b146d0 0x558d09b14ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f622ffff640 10 client.6960 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625572c640 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b146d0 0x558d09b14ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4520f541d98517 server_cookie=af5a8158005c2c7f in_seq=0 out_seq=0 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f622ffff640 1 -- 172.21.15.53:0/3834647517 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558d09a724e0 con 0x558d09b146d0 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f622ffff640 10 client.6960 handle_command_reply: tid=0 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 client.6960 shutdown 2024-05-10T10:39:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 msgr2=0x558d09b17910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b17910 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f624007e220 tx=0x7f624000c210 comp rx=0 tx=0).stop 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 shutdown_connections 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558d09b17e50 0x558d09b14190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x558d09b146d0 0x558d09b14ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d09b28330 0x558d09b17910 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 --2- 172.21.15.53:0/3834647517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558d09b27f50 0x558d09b4ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f622ffff640 0 client.6960 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.872+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 >> 172.21.15.53:0/3834647517 conn(0x558d09afabb0 msgr2=0x558d09af9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 shutdown_connections 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f625a785740 1 -- 172.21.15.53:0/3834647517 wait complete. 2024-05-10T10:39:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f625a785740 20 client.6960 trim_cache size 0 max 16384 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 6888, 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:39:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 4.4822156030000002, 2024-05-10T10:39:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 67, 2024-05-10T10:39:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 88, 2024-05-10T10:39:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 88, 2024-05-10T10:39:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 5.4477664069999996 2024-05-10T10:39:16.881 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:39:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6234076300 msgr2=0x7f62340787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6234076300 0x7f62340787c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f624c013370 tx=0x7f624c00b040 comp rx=0 tx=0).stop 2024-05-10T10:39:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f625006dde0 msgr2=0x7f62500c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f625006dde0 0x7f62500c83d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f624002afe0 tx=0x7f62500cd860 comp rx=0 tx=0).stop 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 shutdown_connections 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6234076300 0x7f62340787c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62500c8910 0x7f62500b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f625006dde0 0x7f62500c83d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 --2- 172.21.15.53:0/2163820035 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f623c003f30 0x7f62501502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 >> 172.21.15.53:0/2163820035 conn(0x7f62500d2df0 msgr2=0x7f62500cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 shutdown_connections 2024-05-10T10:39:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.873+0000 7f6257990640 1 -- 172.21.15.53:0/2163820035 wait complete. 2024-05-10T10:39:16.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:39:16.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.987+0000 7f0c411a5640 1 Processor -- start 2024-05-10T10:39:16.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.987+0000 7f0c411a5640 1 -- start start 2024-05-10T10:39:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c411a5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0c3c15f790 0x7f0c3c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c411a5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c3c0d5d30 0x7f0c3c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c411a5640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c3c0d30b0 con 0x7f0c3c15f3b0 2024-05-10T10:39:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c411a5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c0d3390 con 0x7f0c3c0d5d30 2024-05-10T10:39:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c411a5640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c0c8c80 con 0x7f0c3c15f790 2024-05-10T10:39:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c3b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c3c0d5d30 0x7f0c3c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c3b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c3c0d5d30 0x7f0c3c0d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49360/0 (socket says 172.21.15.53:49360) 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.988+0000 7f0c3b59e640 1 -- 172.21.15.53:0/2576322628 learned_addr learned my addr 172.21.15.53:0/2576322628 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.989+0000 7f0c3a59c640 1 --2- 172.21.15.53:0/2576322628 >> v2:172.21.15.135:3300/0 conn(0x7f0c3c15f790 0x7f0c3c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.989+0000 7f0c3b59e640 1 -- 172.21.15.53:0/2576322628 >> v2:172.21.15.135:3300/0 conn(0x7f0c3c15f790 msgr2=0x7f0c3c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.989+0000 7f0c3b59e640 1 --2- 172.21.15.53:0/2576322628 >> v2:172.21.15.135:3300/0 conn(0x7f0c3c15f790 0x7f0c3c0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.989+0000 7f0c3b59e640 1 -- 172.21.15.53:0/2576322628 >> v1:172.21.15.135:6789/0 conn(0x7f0c3c15f3b0 legacy=0x7f0c3c0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.989+0000 7f0c3b59e640 1 -- 172.21.15.53:0/2576322628 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c0c8f60 con 0x7f0c3c0d5d30 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.989+0000 7f0c3b59e640 1 --2- 172.21.15.53:0/2576322628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0d0210 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0c8c00 tx=0x7f0c2c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f0f2d681ed6485 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c39d9b640 1 -- 172.21.15.53:0/2576322628 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c036070 con 0x7f0c3c0d5d30 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c39d9b640 1 -- 172.21.15.53:0/2576322628 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c031600 con 0x7f0c3c0d5d30 2024-05-10T10:39:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c39d9b640 1 -- 172.21.15.53:0/2576322628 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0319d0 con 0x7f0c3c0d5d30 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 -- 172.21.15.53:0/2576322628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0d5d30 msgr2=0x7f0c3c0d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 --2- 172.21.15.53:0/2576322628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0d0210 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0c8c00 tx=0x7f0c2c02a6f0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 -- 172.21.15.53:0/2576322628 shutdown_connections 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 --2- 172.21.15.53:0/2576322628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0d0210 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 --2- 172.21.15.53:0/2576322628 >> v2:172.21.15.135:3300/0 conn(0x7f0c3c15f790 0x7f0c3c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 -- 172.21.15.53:0/2576322628 >> 172.21.15.53:0/2576322628 conn(0x7f0c3c0d5130 msgr2=0x7f0c3c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 -- 172.21.15.53:0/2576322628 shutdown_connections 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.990+0000 7f0c411a5640 1 -- 172.21.15.53:0/2576322628 wait complete. 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.992+0000 7f0c411a5640 1 Processor -- start 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.992+0000 7f0c411a5640 1 -- start start 2024-05-10T10:39:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.992+0000 7f0c411a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.992+0000 7f0c411a5640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c3c15f3b0 0x7f0c3c1f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c411a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0b49e0 0x7f0c3c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c411a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c0d3f80 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c411a5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c0d3e00 con 0x7f0c3c0d5d30 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c411a5640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c0ca770 con 0x7f0c3c15f3b0 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0b49e0 0x7f0c3c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0b49e0 0x7f0c3c0b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49366/0 (socket says 172.21.15.53:49366) 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 -- 172.21.15.53:0/1830174721 learned_addr learned my addr 172.21.15.53:0/1830174721 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3ad9d640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3a59c640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c3c15f3b0 0x7f0c3c1f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 -- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c3c15f3b0 msgr2=0x7f0c3c1f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c3c15f3b0 0x7f0c3c1f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 -- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c0d5d30 msgr2=0x7f0c3c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 -- 172.21.15.53:0/1830174721 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c0b1f50 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3ad9d640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c3b59e640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0b49e0 0x7f0c3c0b4e70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009f40 tx=0x7f0c2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c337fe640 1 -- 172.21.15.53:0/1830174721 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c036050 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c337fe640 1 -- 172.21.15.53:0/1830174721 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c042070 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c0b2230 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c337fe640 1 -- 172.21.15.53:0/1830174721 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c009510 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.993+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c0b27f0 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.995+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3c06a220 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.995+0000 7f0c337fe640 1 -- 172.21.15.53:0/1830174721 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0c2c007c30 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.996+0000 7f0c337fe640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0c1c0762d0 0x7f0c1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:16.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.996+0000 7f0c337fe640 1 -- 172.21.15.53:0/1830174721 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f0c2c0b86c0 con 0x7f0c3c0b49e0 2024-05-10T10:39:16.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.996+0000 7f0c3ad9d640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0c1c0762d0 0x7f0c1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:16.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:16.996+0000 7f0c3ad9d640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0c1c0762d0 0x7f0c1c078790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003eb0 tx=0x7f0c34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.000+0000 7f0c337fe640 1 -- 172.21.15.53:0/1830174721 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0c2c081ed0 con 0x7f0c3c0b49e0 2024-05-10T10:39:17.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.238+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c3c06e140 con 0x7f0c3c0b49e0 2024-05-10T10:39:17.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.238+0000 7f0c337fe640 1 -- 172.21.15.53:0/1830174721 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f0c2c0817d0 con 0x7f0c3c0b49e0 2024-05-10T10:39:17.239 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:39:17.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0c1c0762d0 msgr2=0x7f0c1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:17.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0c1c0762d0 0x7f0c1c078790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003eb0 tx=0x7f0c34009040 comp rx=0 tx=0).stop 2024-05-10T10:39:17.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0b49e0 msgr2=0x7f0c3c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0b49e0 0x7f0c3c0b4e70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009f40 tx=0x7f0c2c004a40 comp rx=0 tx=0).stop 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 shutdown_connections 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0c1c0762d0 0x7f0c1c078790 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0b49e0 0x7f0c3c0b4e70 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c3c15f3b0 0x7f0c3c1f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 --2- 172.21.15.53:0/1830174721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c0d5d30 0x7f0c3c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.240+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 >> 172.21.15.53:0/1830174721 conn(0x7f0c3c0d5130 msgr2=0x7f0c3c06e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.241+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 shutdown_connections 2024-05-10T10:39:17.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17.241+0000 7f0c411a5640 1 -- 172.21.15.53:0/1830174721 wait complete. 2024-05-10T10:39:17.255 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:39:17.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:39:17.256 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:39:17.256 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:39:17.256 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:17.256 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:39:17.256 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:39:17.256 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:39:17.256 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:39:17.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.256 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:39:17.274 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:39:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:39:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:39:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:39:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:39:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:39:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42412sec preferred_lft 42412sec 2024-05-10T10:39:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:39:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:17.276 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:39:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:39:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:39:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:17.278 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:39:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:39:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:39:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:39:17.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.279 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:39:17.338 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:39:17.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.340 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:39:17.390 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:39:17.390+0000 7f54656aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:39:17.397 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:39:17.397 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:39:17.397 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:39:17.397 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:39:17.397 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:39:17.398 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:39:17.398 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:39:17.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:39:17.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:39:17.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:39:17.398 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:39:17.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:39:17.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:39:17.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.528 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:39:17.544 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:39:17.544 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:39:17.544 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:39:17.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:39:17.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:39:17.610 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:17.666 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:39:17.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:39:17.667 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:17.723 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:39:17.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:17.808 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:17.808 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:39:17.808 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:39:17.817 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:17.817 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:39:17.817 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:39:17.817 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:39:17.817 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:39:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:39:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:39:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:39:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:39:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:39:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:39:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:39:17.834 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:39:17.834 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:39:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:39:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:39:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:39:17.834 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:39:17.836 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:39:17.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:39:17.862 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:39:17.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:39:17.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:39:17.961 INFO:teuthology.orchestra.run.smithi053.stdout:client6406 2024-05-10T10:39:17.961 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:39:17.961 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:39:17.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:17.978 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-05-10T10:39:18.036 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:39:18.036+0000 7f54616a2640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-10T10:39:18.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:39:18.038 INFO:teuthology.orchestra.run.smithi053.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-05-10T10:39:18.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:18.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-05-10T10:39:18.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.185+0000 7f44a8675640 1 Processor -- start 2024-05-10T10:39:18.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.185+0000 7f44a8675640 1 -- start start 2024-05-10T10:39:18.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.186+0000 7f44a8675640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f44a015ced0 0x7f44a0068150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.186+0000 7f44a8675640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 0x7f44a00cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.186+0000 7f44a8675640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a0072ec0 con 0x7f44a00727e0 2024-05-10T10:39:18.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.186+0000 7f44a8675640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f44a00cfd40 con 0x7f44a015ced0 2024-05-10T10:39:18.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.186+0000 7f44a8675640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f44a0072c70 con 0x7f44a00cf3b0 2024-05-10T10:39:18.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.186+0000 7f44a5c10640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f44a00727e0 0x7f44a0160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39752/0 (socket says 172.21.15.53:39752) 2024-05-10T10:39:18.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.186+0000 7f44a5c10640 1 -- 172.21.15.53:0/1071672877 learned_addr learned my addr 172.21.15.53:0/1071672877 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 0x7f44a00cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6411640 1 --2- 172.21.15.53:0/1071672877 >> v2:172.21.15.135:3300/0 conn(0x7f44a015ced0 0x7f44a0068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369437756 0 0) 0x7f44a0072ec0 con 0x7f44a00727e0 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4488003520 con 0x7f44a00727e0 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2847596422 0 0) 0x7f4488003520 con 0x7f44a00727e0 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 msgr2=0x7f44a00cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 0x7f44a00cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 >> v2:172.21.15.135:3300/0 conn(0x7f44a015ced0 msgr2=0x7f44a0068150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 --2- 172.21.15.53:0/1071672877 >> v2:172.21.15.135:3300/0 conn(0x7f44a015ced0 0x7f44a0068150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a00731a0 con 0x7f44a00727e0 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4494001670 con 0x7f44a00727e0 2024-05-10T10:39:18.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 >> v1:172.21.15.53:6789/0 conn(0x7f44a00727e0 legacy=0x7f44a0160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44880039b0 0x7f4488003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 0x7f4488004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4488004d40 0x7f44880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44a00cfd40 con 0x7f4488004d40 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0072c70 con 0x7f44880039b0 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f448800b9c0 con 0x7f44a00cf3b0 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4488004d40 0x7f44880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6411640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44880039b0 0x7f4488003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 0x7f4488004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6c12640 1 -- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 msgr2=0x7f4488004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00cf3b0 0x7f4488004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6c12640 1 -- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44880039b0 msgr2=0x7f4488003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44880039b0 0x7f4488003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.187+0000 7f44a6c12640 1 -- 172.21.15.53:0/1071672877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4488003520 con 0x7f4488004d40 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.188+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4488004d40 0x7f44880051c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4490009fc0 tx=0x7f4488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7910db829f6deda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.188+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490009c10 con 0x7f4488004d40 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.188+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4490036070 con 0x7f4488004d40 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.188+0000 7f44a540f640 1 -- 172.21.15.53:0/1071672877 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490031880 con 0x7f4488004d40 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.189+0000 7f44a8675640 1 -- 172.21.15.53:0/1071672877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4488004d40 msgr2=0x7f44880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.189+0000 7f44a8675640 1 --2- 172.21.15.53:0/1071672877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4488004d40 0x7f44880051c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4490009fc0 tx=0x7f4488005830 comp rx=0 tx=0).stop 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.189+0000 7f44a6c12640 1 -- 172.21.15.53:0/1071672877 reap_dead start 2024-05-10T10:39:18.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.189+0000 7f44a8675640 1 -- 172.21.15.53:0/1071672877 shutdown_connections 2024-05-10T10:39:18.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.189+0000 7f44a8675640 1 -- 172.21.15.53:0/1071672877 >> 172.21.15.53:0/1071672877 conn(0x7f44a00d14b0 msgr2=0x7f44a00d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.189+0000 7f44a8675640 1 -- 172.21.15.53:0/1071672877 shutdown_connections 2024-05-10T10:39:18.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.189+0000 7f44a8675640 1 -- 172.21.15.53:0/1071672877 wait complete. 2024-05-10T10:39:18.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.190+0000 7f44a8675640 1 Processor -- start 2024-05-10T10:39:18.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.190+0000 7f44a8675640 1 -- start start 2024-05-10T10:39:18.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a8675640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4488004d40 0x7f44a0159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a8675640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a006d220 0x7f44a015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a8675640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00b52e0 0x7f44a00b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a8675640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44a015fd60 con 0x7f44a006d220 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a8675640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0160990 con 0x7f4488004d40 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a8675640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f44a015fa50 con 0x7f44a00b52e0 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a006d220 0x7f44a015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a006d220 0x7f44a015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49380/0 (socket says 172.21.15.53:49380) 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 -- 172.21.15.53:0/1482609881 learned_addr learned my addr 172.21.15.53:0/1482609881 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 -- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00b52e0 msgr2=0x7f44a00b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00b52e0 0x7f44a00b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 -- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4488004d40 msgr2=0x7f44a0159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4488004d40 0x7f44a0159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.191+0000 7f44a5c10640 1 -- 172.21.15.53:0/1482609881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a015e7e0 con 0x7f44a006d220 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.192+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a006d220 0x7f44a015de70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4494001ba0 tx=0x7f4494007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.192+0000 7f449effd640 1 -- 172.21.15.53:0/1482609881 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449400f9c0 con 0x7f44a006d220 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.192+0000 7f449effd640 1 -- 172.21.15.53:0/1482609881 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494018070 con 0x7f44a006d220 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.192+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a0159bc0 con 0x7f44a006d220 2024-05-10T10:39:18.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.192+0000 7f449effd640 1 -- 172.21.15.53:0/1482609881 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449400fde0 con 0x7f44a006d220 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.192+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a0159690 con 0x7f44a006d220 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.193+0000 7f449effd640 1 -- 172.21.15.53:0/1482609881 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f449401d460 con 0x7f44a006d220 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.194+0000 7f449effd640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c0762d0 0x7f448c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.194+0000 7f449effd640 1 -- 172.21.15.53:0/1482609881 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f44940977d0 con 0x7f44a006d220 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.194+0000 7f44a6411640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c0762d0 0x7f448c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.194+0000 7f44ab398740 1 Processor -- start 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44ab398740 1 -- start start 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44a6411640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c0762d0 0x7f448c078790 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4498006e80 tx=0x7f4498006df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44ab398740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6f890 0x563196b720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b424f0 0x563196b45d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44ab398740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563196aabbc0 con 0x563196b6f890 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563196a64d30 con 0x563196b424f0 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563196aa7b80 con 0x563196b6fc70 2024-05-10T10:39:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44a6411640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6f890 0x563196b720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44a6411640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6f890 0x563196b720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49390/0 (socket says 172.21.15.53:49390) 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.195+0000 7f44a6411640 1 -- 172.21.15.53:0/1534021923 learned_addr learned my addr 172.21.15.53:0/1534021923 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b424f0 0x563196b45d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6411640 1 -- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 msgr2=0x563196b725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6411640 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6411640 1 -- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b424f0 msgr2=0x563196b45d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6411640 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b424f0 0x563196b45d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6411640 1 -- 172.21.15.53:0/1534021923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563196a49100 con 0x563196b6f890 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b424f0 0x563196b45d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f44a6411640 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6f890 0x563196b720a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f44980044c0 tx=0x7f4498014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694792fb0ee8a1ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f4487fff640 1 -- 172.21.15.53:0/1534021923 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449801d050 con 0x563196b6f890 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f4487fff640 1 -- 172.21.15.53:0/1534021923 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498027070 con 0x563196b6f890 2024-05-10T10:39:18.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.196+0000 7f4487fff640 1 -- 172.21.15.53:0/1534021923 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498003a20 con 0x563196b6f890 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 -- 172.21.15.53:0/1534021923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6f890 msgr2=0x563196b720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6f890 0x563196b720a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f44980044c0 tx=0x7f4498014000 comp rx=0 tx=0).stop 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 -- 172.21.15.53:0/1534021923 shutdown_connections 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b424f0 0x563196b45d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 --2- 172.21.15.53:0/1534021923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6f890 0x563196b720a0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 -- 172.21.15.53:0/1534021923 >> 172.21.15.53:0/1534021923 conn(0x563196b44760 msgr2=0x563196b8d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 -- 172.21.15.53:0/1534021923 shutdown_connections 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 -- 172.21.15.53:0/1534021923 wait complete. 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.197+0000 7f44ab398740 1 -- start start 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.198+0000 7f44ab398740 1 Processor -- start 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.198+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6f890 0x563196b747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.198+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b74d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.198+0000 7f44ab398740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b660c0 0x563196b8eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.198+0000 7f44ab398740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563196b47910 con 0x563196b660c0 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.198+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563196a68870 con 0x563196b6f890 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563196b47bf0 con 0x563196b6fc70 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44ab398740 10 client.0 _ll_register_callbacks cb 0x563196ab8b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b660c0 0x563196b8eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b660c0 0x563196b8eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49404/0 (socket says 172.21.15.53:49404) 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 -- 172.21.15.53:0/104323943 learned_addr learned my addr 172.21.15.53:0/104323943 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6411640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6f890 0x563196b747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a5c10640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b74d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 -- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 msgr2=0x563196b74d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b74d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 -- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6f890 msgr2=0x563196b747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6f890 0x563196b747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 -- 172.21.15.53:0/104323943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631968897b0 con 0x563196b660c0 2024-05-10T10:39:18.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6411640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6f890 0x563196b747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44a6c12640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b660c0 0x563196b8eec0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f449000e230 tx=0x7f4490009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1e9c546ada2513 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44857fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44857fa640 1 -- 172.21.15.53:0/104323943 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490031c30 con 0x563196b660c0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44857fa640 1 -- 172.21.15.53:0/104323943 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f4490031dd0 con 0x563196b660c0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.199+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4498027070 con 0x563196b660c0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44857fa640 1 -- 172.21.15.53:0/104323943 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f449004d050 con 0x563196b660c0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 10 client.6978 fetch_fsmap learned FSMap version 67 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5631968e9f40 con 0x563196b660c0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44857fa640 1 -- 172.21.15.53:0/104323943 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498027070 con 0x563196b660c0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44857fa640 1 -- 172.21.15.53:0/104323943 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f4490050020 con 0x563196b660c0 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 10 client.6978 fetch_fsmap finished waiting for FSMap version 67 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 10 client.6978 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 20 client.6978 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b65af0 0x563196b8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 4 client.6978 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563196ab97d0 con 0x563196b65af0 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44a6411640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b65af0 0x563196b8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44857fa640 10 client.6978 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.200+0000 7f44a6411640 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b65af0 0x563196b8f400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47312b9cbd89005c server_cookie=b9d45b4062172df3 in_seq=0 out_seq=0 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44857fa640 1 -- 172.21.15.53:0/104323943 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563196ab97d0 con 0x563196b65af0 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44857fa640 10 client.6978 handle_command_reply: tid=0 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 client.6978 shutdown 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b660c0 msgr2=0x563196b8eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b660c0 0x563196b8eec0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f449000e230 tx=0x7f4490009750 comp rx=0 tx=0).stop 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 shutdown_connections 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b65af0 0x563196b8f400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b660c0 0x563196b8eec0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44857fa640 0 client.6978 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6fc70 0x563196b74d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 --2- 172.21.15.53:0/104323943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6f890 0x563196b747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 >> 172.21.15.53:0/104323943 conn(0x563196b424f0 msgr2=0x563196b723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.201+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 shutdown_connections 2024-05-10T10:39:18.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.202+0000 7f44ab398740 1 -- 172.21.15.53:0/104323943 wait complete. 2024-05-10T10:39:18.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.202+0000 7f44ab398740 20 client.6978 trim_cache size 0 max 16384 2024-05-10T10:39:18.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.220+0000 7f44ab398740 1 Processor -- start 2024-05-10T10:39:18.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.220+0000 7f44ab398740 1 -- start start 2024-05-10T10:39:18.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 0x563196b43230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44ab398740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6fc70 0x563196b629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b43770 0x563196b62f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44ab398740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44a02023d0 con 0x563196b6fc70 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0201410 con 0x563196b43770 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f44a0200770 con 0x563196b6f890 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6fc70 0x563196b629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6fc70 0x563196b629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49408/0 (socket says 172.21.15.53:49408) 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 -- 172.21.15.53:0/3833568995 learned_addr learned my addr 172.21.15.53:0/3833568995 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 -- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 msgr2=0x563196b43230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 0x563196b43230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 -- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b43770 msgr2=0x563196b62f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a6c12640 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b43770 0x563196b62f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b43770 0x563196b62f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 -- 172.21.15.53:0/3833568995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319677d0a0 con 0x563196b6fc70 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a6c12640 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b43770 0x563196b62f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44a5c10640 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6fc70 0x563196b629f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f449405dac0 tx=0x7f449405c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a217f7891e9ce60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44877fe640 1 -- 172.21.15.53:0/3833568995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494079020 con 0x563196b6fc70 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44877fe640 1 -- 172.21.15.53:0/3833568995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494060360 con 0x563196b6fc70 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.221+0000 7f44877fe640 1 -- 172.21.15.53:0/3833568995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449405ffc0 con 0x563196b6fc70 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 -- 172.21.15.53:0/3833568995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6fc70 msgr2=0x563196b629f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6fc70 0x563196b629f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f449405dac0 tx=0x7f449405c780 comp rx=0 tx=0).stop 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 -- 172.21.15.53:0/3833568995 shutdown_connections 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b43770 0x563196b62f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b6fc70 0x563196b629f0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 --2- 172.21.15.53:0/3833568995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 0x563196b43230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 -- 172.21.15.53:0/3833568995 >> 172.21.15.53:0/3833568995 conn(0x563196b424f0 msgr2=0x563196b3d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 -- 172.21.15.53:0/3833568995 shutdown_connections 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.222+0000 7f44ab398740 1 -- 172.21.15.53:0/3833568995 wait complete. 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.223+0000 7f44ab398740 1 -- start start 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.223+0000 7f44ab398740 1 Processor -- start 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b43770 0x563196b83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 0x563196b83580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6fc70 0x563196bb00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563196b65970 con 0x563196b43770 2024-05-10T10:39:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0200dc0 con 0x563196b6fc70 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563196b6d2d0 con 0x563196b6f890 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 10 client.0 _ll_register_callbacks cb 0x7f44900095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b43770 0x563196b83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b43770 0x563196b83040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49412/0 (socket says 172.21.15.53:49412) 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 -- 172.21.15.53:0/1243440590 learned_addr learned my addr 172.21.15.53:0/1243440590 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6c12640 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6fc70 0x563196bb00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 0x563196b83580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 -- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 msgr2=0x563196b83580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 0x563196b83580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 -- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6fc70 msgr2=0x563196bb00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6fc70 0x563196bb00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 -- 172.21.15.53:0/1243440590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563196b89b50 con 0x563196b43770 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44a6411640 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b43770 0x563196b83040 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4498011d20 tx=0x7f4498004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dc9e54b4cec2eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44857fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44857fa640 1 -- 172.21.15.53:0/1243440590 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449801d070 con 0x563196b43770 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44857fa640 1 -- 172.21.15.53:0/1243440590 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7f4498003ba0 con 0x563196b43770 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4494060360 con 0x563196b43770 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44857fa640 1 -- 172.21.15.53:0/1243440590 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f449803e050 con 0x563196b43770 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.224+0000 7f44ab398740 10 client.6984 fetch_fsmap learned FSMap version 67 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x563196b74dc0 con 0x563196b43770 2024-05-10T10:39:18.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44857fa640 1 -- 172.21.15.53:0/1243440590 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494060360 con 0x563196b43770 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44857fa640 1 -- 172.21.15.53:0/1243440590 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f4498041020 con 0x563196b43770 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44ab398740 10 client.6984 fetch_fsmap finished waiting for FSMap version 67 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44ab398740 10 client.6984 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44ab398740 20 client.6984 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44ab398740 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b42b90 0x563196bb0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44ab398740 4 client.6984 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x563196ab3640 con 0x563196b42b90 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b42b90 0x563196bb0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44857fa640 10 client.6984 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.225+0000 7f44a5c10640 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b42b90 0x563196bb0610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3102bd189224ed server_cookie=7831ce77bb0eda39 in_seq=0 out_seq=0 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44857fa640 1 -- 172.21.15.53:0/1243440590 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563196ab3640 con 0x563196b42b90 2024-05-10T10:39:18.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44857fa640 10 client.6984 handle_command_reply: tid=0 2024-05-10T10:39:18.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 client.6984 shutdown 2024-05-10T10:39:18.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b43770 msgr2=0x563196b83040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b43770 0x563196b83040 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4498011d20 tx=0x7f4498004730 comp rx=0 tx=0).stop 2024-05-10T10:39:18.234 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f54616a2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0) n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x559c49899b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-10T10:39:18.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 shutdown_connections 2024-05-10T10:39:18.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563196b6fc70 0x563196bb00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563196b6f890 0x563196b83580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x563196b42b90 0x563196bb0610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 --2- 172.21.15.53:0/1243440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563196b43770 0x563196b83040 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44857fa640 0 client.6984 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 >> 172.21.15.53:0/1243440590 conn(0x563196b424f0 msgr2=0x563196b8ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 shutdown_connections 2024-05-10T10:39:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 1 -- 172.21.15.53:0/1243440590 wait complete. 2024-05-10T10:39:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.226+0000 7f44ab398740 20 client.6984 trim_cache size 0 max 16384 2024-05-10T10:39:18.237 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:39:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:39:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "0d96928a-abf1-4ec2-9dac-809eef267363", 2024-05-10T10:39:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:39:18.237 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:39:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c0762d0 msgr2=0x7f448c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c0762d0 0x7f448c078790 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4498006e80 tx=0x7f4498006df0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a006d220 msgr2=0x7f44a015de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a006d220 0x7f44a015de70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4494001ba0 tx=0x7f4494007660 comp rx=0 tx=0).stop 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 shutdown_connections 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f448c0762d0 0x7f448c078790 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44a00b52e0 0x7f44a00b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a006d220 0x7f44a015de70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 --2- 172.21.15.53:0/1482609881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4488004d40 0x7f44a0159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 >> 172.21.15.53:0/1482609881 conn(0x7f44a00d14b0 msgr2=0x7f44a00b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 shutdown_connections 2024-05-10T10:39:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.227+0000 7f44a8675640 1 -- 172.21.15.53:0/1482609881 wait complete. 2024-05-10T10:39:18.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:18.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-10T10:39:18.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.345+0000 7fecbe6b4640 1 Processor -- start 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.346+0000 7fecbe6b4640 1 -- start start 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.346+0000 7fecbe6b4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fecb815f410 0x7fecb80d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.346+0000 7fecbe6b4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80d0f10 0x7fecb80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.346+0000 7fecbe6b4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fecb80b54d0 0x7fecb80d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.346+0000 7fecbe6b4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fecb80d2800 con 0x7fecb815f410 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.346+0000 7fecbe6b4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fecb80d2ae0 con 0x7fecb80b54d0 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.346+0000 7fecbe6b4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fecb80d01d0 con 0x7fecb80d0f10 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fecb815f410 0x7fecb80d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fecb815f410 0x7fecb80d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49414/0 (socket says 172.21.15.53:49414) 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 -- 172.21.15.53:0/2060197703 learned_addr learned my addr 172.21.15.53:0/2060197703 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecbcc51640 1 --2- 172.21.15.53:0/2060197703 >> v2:172.21.15.135:3300/0 conn(0x7fecb80b54d0 0x7fecb80d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb77fe640 1 --2- 172.21.15.53:0/2060197703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80d0f10 0x7fecb80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 -- 172.21.15.53:0/2060197703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80d0f10 msgr2=0x7fecb80b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 --2- 172.21.15.53:0/2060197703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80d0f10 0x7fecb80b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 -- 172.21.15.53:0/2060197703 >> v2:172.21.15.135:3300/0 conn(0x7fecb80b54d0 msgr2=0x7fecb80d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 --2- 172.21.15.53:0/2060197703 >> v2:172.21.15.135:3300/0 conn(0x7fecb80b54d0 0x7fecb80d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.347+0000 7fecb7fff640 1 -- 172.21.15.53:0/2060197703 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb80d04b0 con 0x7fecb815f410 2024-05-10T10:39:18.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecb7fff640 1 --2- 172.21.15.53:0/2060197703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb815f410 0x7fecb80d09d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fecb806c840 tx=0x7feca802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4a22b9b1349f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecb6ffd640 1 -- 172.21.15.53:0/2060197703 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8037070 con 0x7fecb815f410 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecb6ffd640 1 -- 172.21.15.53:0/2060197703 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8032440 con 0x7fecb815f410 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecb6ffd640 1 -- 172.21.15.53:0/2060197703 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8032810 con 0x7fecb815f410 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 -- 172.21.15.53:0/2060197703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb815f410 msgr2=0x7fecb80d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/2060197703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb815f410 0x7fecb80d09d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fecb806c840 tx=0x7feca802a140 comp rx=0 tx=0).stop 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 -- 172.21.15.53:0/2060197703 shutdown_connections 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/2060197703 >> v2:172.21.15.135:3300/0 conn(0x7fecb80b54d0 0x7fecb80d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/2060197703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80d0f10 0x7fecb80b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/2060197703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb815f410 0x7fecb80d09d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 -- 172.21.15.53:0/2060197703 >> 172.21.15.53:0/2060197703 conn(0x7fecb808ed30 msgr2=0x7fecb809e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.348+0000 7fecbe6b4640 1 -- 172.21.15.53:0/2060197703 shutdown_connections 2024-05-10T10:39:18.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.349+0000 7fecbe6b4640 1 -- 172.21.15.53:0/2060197703 wait complete. 2024-05-10T10:39:18.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.350+0000 7fecbe6b4640 1 Processor -- start 2024-05-10T10:39:18.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.350+0000 7fecbe6b4640 1 -- start start 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb80b54d0 0x7fecb815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80d0f10 0x7fecb81f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb815f410 0x7fecb80ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80cffb0 con 0x7fecb80b54d0 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80cfe30 con 0x7fecb80d0f10 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fecb80d3c10 con 0x7fecb815f410 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb80b54d0 0x7fecb815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb80b54d0 0x7fecb815c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49422/0 (socket says 172.21.15.53:49422) 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 -- 172.21.15.53:0/4019112542 learned_addr learned my addr 172.21.15.53:0/4019112542 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb77fe640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80d0f10 0x7fecb81f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbcc51640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb815f410 0x7fecb80ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 -- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb815f410 msgr2=0x7fecb80ba5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb815f410 0x7fecb80ba5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 -- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80d0f10 msgr2=0x7fecb81f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80d0f10 0x7fecb81f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 -- 172.21.15.53:0/4019112542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb80cc640 con 0x7fecb80b54d0 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb7fff640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb80b54d0 0x7fecb815c470 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7feca802a180 tx=0x7feca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb4ff9640 1 -- 172.21.15.53:0/4019112542 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8041020 con 0x7fecb80b54d0 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb4ff9640 1 -- 172.21.15.53:0/4019112542 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca803d620 con 0x7fecb80b54d0 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecb4ff9640 1 -- 172.21.15.53:0/4019112542 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8037040 con 0x7fecb80b54d0 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb80cc920 con 0x7fecb80b54d0 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.351+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb80b9800 con 0x7fecb80b54d0 2024-05-10T10:39:18.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.353+0000 7fecb4ff9640 1 -- 172.21.15.53:0/4019112542 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feca8044020 con 0x7fecb80b54d0 2024-05-10T10:39:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.354+0000 7fecb4ff9640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feca4076300 0x7feca40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.354+0000 7fecb4ff9640 1 -- 172.21.15.53:0/4019112542 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7feca80b7ef0 con 0x7fecb80b54d0 2024-05-10T10:39:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.354+0000 7fecb77fe640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feca4076300 0x7feca40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.354+0000 7fecc13cf740 1 Processor -- start 2024-05-10T10:39:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.354+0000 7fecc13cf740 1 -- start start 2024-05-10T10:39:18.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.354+0000 7fecb77fe640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feca4076300 0x7feca40787c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fecb80a2930 tx=0x7fecac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646447faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644b0280 0x5564644af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644afb40 0x556464484950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecc13cf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55646444c690 con 0x5564644cd160 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5564643e9f30 con 0x5564644afb40 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5564643a2b20 con 0x5564644b0280 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646447faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646447faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49432/0 (socket says 172.21.15.53:49432) 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 -- 172.21.15.53:0/1562676427 learned_addr learned my addr 172.21.15.53:0/1562676427 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecbcc51640 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644afb40 0x556464484950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb77fe640 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644b0280 0x5564644af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 -- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644b0280 msgr2=0x5564644af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644b0280 0x5564644af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 -- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644afb40 msgr2=0x556464484950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644afb40 0x556464484950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb7fff640 1 -- 172.21.15.53:0/1562676427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646439ae60 con 0x5564644cd160 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecbcc51640 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644afb40 0x556464484950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.355+0000 7fecb77fe640 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644b0280 0x5564644af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecb7fff640 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646447faf0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7feca807cf50 tx=0x7feca8080820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa488e13d9b009a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fec9dffb640 1 -- 172.21.15.53:0/1562676427 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca80815d0 con 0x5564644cd160 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fec9dffb640 1 -- 172.21.15.53:0/1562676427 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8080ed0 con 0x5564644cd160 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fec9dffb640 1 -- 172.21.15.53:0/1562676427 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8089a10 con 0x5564644cd160 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 -- 172.21.15.53:0/1562676427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 msgr2=0x55646447faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646447faf0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7feca807cf50 tx=0x7feca8080820 comp rx=0 tx=0).stop 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 -- 172.21.15.53:0/1562676427 shutdown_connections 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644afb40 0x556464484950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644b0280 0x5564644af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1562676427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646447faf0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 -- 172.21.15.53:0/1562676427 >> 172.21.15.53:0/1562676427 conn(0x556464481d60 msgr2=0x5564644ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 -- 172.21.15.53:0/1562676427 shutdown_connections 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.356+0000 7fecc13cf740 1 -- 172.21.15.53:0/1562676427 wait complete. 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.357+0000 7fecc13cf740 1 -- start start 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.357+0000 7fecc13cf740 1 Processor -- start 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644cd160 0x5564644cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644c1530 0x5564644b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecc13cf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5564643a65e0 con 0x5564644b0280 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5564643e5ef0 con 0x5564644c1530 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5564643e9f30 con 0x5564644cd160 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecc13cf740 10 client.0 _ll_register_callbacks cb 0x5564643a1320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:18.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644c1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49438/0 (socket says 172.21.15.53:49438) 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 -- 172.21.15.53:0/1285714950 learned_addr learned my addr 172.21.15.53:0/1285714950 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb77fe640 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644cd160 0x5564644cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecbcc51640 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644c1530 0x5564644b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 -- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644cd160 msgr2=0x5564644cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644cd160 0x5564644cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 -- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644c1530 msgr2=0x5564644b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644c1530 0x5564644b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.358+0000 7fecb7fff640 1 -- 172.21.15.53:0/1285714950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564641c57d0 con 0x5564644b0280 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecb7fff640 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644c1e50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7feca807cf50 tx=0x7feca8008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b88704866e63e69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fec96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fec96ffd640 1 -- 172.21.15.53:0/1285714950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca80a1060 con 0x5564644b0280 2024-05-10T10:39:18.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fec96ffd640 1 -- 172.21.15.53:0/1285714950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7feca8031c10 con 0x5564644b0280 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca8080ed0 con 0x5564644b0280 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fec96ffd640 1 -- 172.21.15.53:0/1285714950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7feca8085280 con 0x5564644b0280 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 10 client.6996 fetch_fsmap learned FSMap version 67 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5564640fb3f0 con 0x5564644b0280 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fec96ffd640 1 -- 172.21.15.53:0/1285714950 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8080ed0 con 0x5564644b0280 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fec96ffd640 1 -- 172.21.15.53:0/1285714950 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7feca8098070 con 0x5564644b0280 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 10 client.6996 fetch_fsmap finished waiting for FSMap version 67 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 10 client.6996 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 20 client.6996 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x5564644da5f0 0x5564644b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 4 client.6996 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564643f7b40 con 0x5564644da5f0 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.359+0000 7fecb77fe640 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x5564644da5f0 0x5564644b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fec96ffd640 10 client.6996 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecb77fe640 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x5564644da5f0 0x5564644b0770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a300048223b653 server_cookie=3d55a8350fd6c22e in_seq=0 out_seq=0 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fec96ffd640 1 -- 172.21.15.53:0/1285714950 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564643f7b40 con 0x5564644da5f0 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fec96ffd640 10 client.6996 handle_command_reply: tid=0 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 client.6996 shutdown 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 msgr2=0x5564644c1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644c1e50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7feca807cf50 tx=0x7feca8008070 comp rx=0 tx=0).stop 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 shutdown_connections 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644c1530 0x5564644b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644cd160 0x5564644cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x5564644da5f0 0x5564644b0770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 --2- 172.21.15.53:0/1285714950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644c1e50 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 >> 172.21.15.53:0/1285714950 conn(0x5564644afb40 msgr2=0x55646447f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.360+0000 7fec96ffd640 0 client.6996 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.361+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 shutdown_connections 2024-05-10T10:39:18.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.361+0000 7fecc13cf740 1 -- 172.21.15.53:0/1285714950 wait complete. 2024-05-10T10:39:18.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.361+0000 7fecc13cf740 20 client.6996 trim_cache size 0 max 16384 2024-05-10T10:39:18.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 Processor -- start 2024-05-10T10:39:18.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 -- start start 2024-05-10T10:39:18.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644cd160 0x5564644bff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8201f40 con 0x5564644b0280 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8200fa0 con 0x5564644cd160 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fecb80b8850 con 0x5564644c1530 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.379+0000 7fecb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49450/0 (socket says 172.21.15.53:49450) 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb7fff640 1 -- 172.21.15.53:0/3896441265 learned_addr learned my addr 172.21.15.53:0/3896441265 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb7fff640 1 -- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 msgr2=0x5564644cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb77fe640 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecbcc51640 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644cd160 0x5564644bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb7fff640 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb7fff640 1 -- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644cd160 msgr2=0x5564644bff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb7fff640 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644cd160 0x5564644bff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb7fff640 1 -- 172.21.15.53:0/3896441265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564640bb0f0 con 0x5564644b0280 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecbcc51640 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644cd160 0x5564644bff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fecb7fff640 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644cda20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7feca8000c00 tx=0x7feca80b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9f0f59b4d7471e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fec9d7fa640 1 -- 172.21.15.53:0/3896441265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8080640 con 0x5564644b0280 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fec9d7fa640 1 -- 172.21.15.53:0/3896441265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8080ed0 con 0x5564644b0280 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.380+0000 7fec9d7fa640 1 -- 172.21.15.53:0/3896441265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8080250 con 0x5564644b0280 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 -- 172.21.15.53:0/3896441265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 msgr2=0x5564644cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644cda20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7feca8000c00 tx=0x7feca80b4040 comp rx=0 tx=0).stop 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 -- 172.21.15.53:0/3896441265 shutdown_connections 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644cd160 0x5564644bff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 --2- 172.21.15.53:0/3896441265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644b0280 0x5564644cda20 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 -- 172.21.15.53:0/3896441265 >> 172.21.15.53:0/3896441265 conn(0x5564644afb40 msgr2=0x556464486be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 -- 172.21.15.53:0/3896441265 shutdown_connections 2024-05-10T10:39:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 -- 172.21.15.53:0/3896441265 wait complete. 2024-05-10T10:39:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 -- start start 2024-05-10T10:39:18.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.381+0000 7fecc13cf740 1 Processor -- start 2024-05-10T10:39:18.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644b0280 0x5564644cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644c0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecc13cf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646452d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecc13cf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55646444c690 con 0x5564644cd160 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8200af0 con 0x5564644b0280 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecc13cf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5564644edde0 con 0x5564644c1530 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecc13cf740 10 client.0 _ll_register_callbacks cb 0x7feca807ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecbcc51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646452d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecbcc51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646452d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49454/0 (socket says 172.21.15.53:49454) 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecbcc51640 1 -- 172.21.15.53:0/927555221 learned_addr learned my addr 172.21.15.53:0/927555221 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecb7fff640 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644b0280 0x5564644cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.382+0000 7fecb77fe640 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644c0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecbcc51640 1 -- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 msgr2=0x5564644c0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecbcc51640 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644c0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecbcc51640 1 -- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644b0280 msgr2=0x5564644cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecbcc51640 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644b0280 0x5564644cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecbcc51640 1 -- 172.21.15.53:0/927555221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564644ecd40 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecbcc51640 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646452d2c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x556464487380 tx=0x7fecb0007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c4663c7f1835ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fec96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fec96ffd640 1 -- 172.21.15.53:0/927555221 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0021410 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fec96ffd640 1 -- 172.21.15.53:0/927555221 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6187+0+0 (secure 0 0 0) 0x7fecb00215b0 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca8080ed0 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fec96ffd640 1 -- 172.21.15.53:0/927555221 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fecb0029050 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 10 client.7002 fetch_fsmap learned FSMap version 67 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55646447f960 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fec96ffd640 1 -- 172.21.15.53:0/927555221 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8080ed0 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fec96ffd640 1 -- 172.21.15.53:0/927555221 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fecb002c020 con 0x5564644cd160 2024-05-10T10:39:18.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 10 client.7002 fetch_fsmap finished waiting for FSMap version 67 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 10 client.7002 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 20 client.7002 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x556464483670 0x55646447e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 4 client.7002 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.383+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564643f19b0 con 0x556464483670 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecb7fff640 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x556464483670 0x55646447e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fec96ffd640 10 client.7002 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecb7fff640 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x556464483670 0x55646447e9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5141e91d37589354 server_cookie=568756a6c477e0a1 in_seq=0 out_seq=0 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fec96ffd640 1 -- 172.21.15.53:0/927555221 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x5564643f19b0 con 0x556464483670 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fec96ffd640 10 client.7002 handle_command_reply: tid=0 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 client.7002 shutdown 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 msgr2=0x55646452d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646452d2c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x556464487380 tx=0x7fecb0007330 comp rx=0 tx=0).stop 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 shutdown_connections 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x556464483670 0x55646447e9a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5564644cd160 0x55646452d2c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5564644c1530 0x5564644c0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fec96ffd640 0 client.7002 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 --2- 172.21.15.53:0/927555221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564644b0280 0x5564644cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 >> 172.21.15.53:0/927555221 conn(0x5564644afb40 msgr2=0x5564644cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 shutdown_connections 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.384+0000 7fecc13cf740 1 -- 172.21.15.53:0/927555221 wait complete. 2024-05-10T10:39:18.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecc13cf740 20 client.7002 trim_cache size 0 max 16384 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "0d96928a-abf1-4ec2-9dac-809eef267363": { 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/dir", 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "0d96928a-abf1-4ec2-9dac-809eef267363", 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive" 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:39:18.392 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:39:18.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feca4076300 msgr2=0x7feca40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feca4076300 0x7feca40787c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fecb80a2930 tx=0x7fecac008040 comp rx=0 tx=0).stop 2024-05-10T10:39:18.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb80b54d0 msgr2=0x7fecb815c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:18.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb80b54d0 0x7fecb815c470 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7feca802a180 tx=0x7feca80046c0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 shutdown_connections 2024-05-10T10:39:18.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7feca4076300 0x7feca40787c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb815f410 0x7fecb80ba5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80d0f10 0x7fecb81f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 --2- 172.21.15.53:0/4019112542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb80b54d0 0x7fecb815c470 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:18.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 >> 172.21.15.53:0/4019112542 conn(0x7fecb808ed30 msgr2=0x7fecb806d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:18.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 shutdown_connections 2024-05-10T10:39:18.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:18.385+0000 7fecbe6b4640 1 -- 172.21.15.53:0/4019112542 wait complete. 2024-05-10T10:39:18.396 INFO:tasks.cephfs.filesystem:scrub status for tag:0d96928a-abf1-4ec2-9dac-809eef267363 - {'path': '/dir', 'tag': '0d96928a-abf1-4ec2-9dac-809eef267363', 'options': 'recursive'} 2024-05-10T10:39:22.406 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:39:22.405+0000 7f39f976c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-10T10:39:44.892 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:39:44.896 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:39:48.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:48.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-10T10:39:48.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.517+0000 7f2940366640 1 Processor -- start 2024-05-10T10:39:48.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.517+0000 7f2940366640 1 -- start start 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f2940366640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f293806f030 0x7f2938160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f2940366640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 0x7f293815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f2940366640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f293815cfb0 con 0x7f293815c9e0 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f2940366640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f29380ccb60 con 0x7f293806f030 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f2940366640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f293815d290 con 0x7f293806f410 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293e903640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f293815c9e0 0x7f29380cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49062/0 (socket says 172.21.15.53:49062) 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293e903640 1 -- 172.21.15.53:0/2517248209 learned_addr learned my addr 172.21.15.53:0/2517248209 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d901640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 0x7f293815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293e102640 1 --2- 172.21.15.53:0/2517248209 >> v2:172.21.15.135:3300/0 conn(0x7f293806f030 0x7f2938160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670147962 0 0) 0x7f293815cfb0 con 0x7f293815c9e0 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2924003520 con 0x7f293815c9e0 2024-05-10T10:39:48.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 994980787 0 0) 0x7f2924003520 con 0x7f293815c9e0 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 msgr2=0x7f293815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 0x7f293815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 >> v2:172.21.15.135:3300/0 conn(0x7f293806f030 msgr2=0x7f2938160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 --2- 172.21.15.53:0/2517248209 >> v2:172.21.15.135:3300/0 conn(0x7f293806f030 0x7f2938160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293815fd30 con 0x7f293815c9e0 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f29300019f0 con 0x7f293815c9e0 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 >> v1:172.21.15.53:6789/0 conn(0x7f293815c9e0 legacy=0x7f29380cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.518+0000 7f293d100640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293d100640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 0x7f2924004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293d100640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2924004310 0x7f2924004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f293815d290 con 0x7f29240039b0 2024-05-10T10:39:48.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29240054e0 con 0x7f2924004310 2024-05-10T10:39:48.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f292400ffd0 con 0x7f293806f410 2024-05-10T10:39:48.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293e102640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293d901640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 0x7f2924004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293e903640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2924004310 0x7f2924004d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293e102640 1 -- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 msgr2=0x7f2924004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293e102640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f293806f410 0x7f2924004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293e102640 1 -- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2924004310 msgr2=0x7f2924004d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293e102640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2924004310 0x7f2924004d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.519+0000 7f293e102640 1 -- 172.21.15.53:0/2517248209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2924003520 con 0x7f29240039b0 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.520+0000 7f293e102640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2934009ac0 tx=0x7f2924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b452e1788cc354 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.520+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934036070 con 0x7f29240039b0 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.520+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2934031400 con 0x7f29240039b0 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.520+0000 7f293d100640 1 -- 172.21.15.53:0/2517248209 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934009690 con 0x7f29240039b0 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.520+0000 7f2940366640 1 -- 172.21.15.53:0/2517248209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29240039b0 msgr2=0x7f2924003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.520+0000 7f2940366640 1 --2- 172.21.15.53:0/2517248209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2934009ac0 tx=0x7f2924005830 comp rx=0 tx=0).stop 2024-05-10T10:39:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.520+0000 7f293e903640 1 -- 172.21.15.53:0/2517248209 reap_dead start 2024-05-10T10:39:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.521+0000 7f2940366640 1 -- 172.21.15.53:0/2517248209 shutdown_connections 2024-05-10T10:39:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.521+0000 7f2940366640 1 -- 172.21.15.53:0/2517248209 >> 172.21.15.53:0/2517248209 conn(0x7f29380d0c90 msgr2=0x7f29380d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.521+0000 7f2940366640 1 -- 172.21.15.53:0/2517248209 shutdown_connections 2024-05-10T10:39:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.521+0000 7f2940366640 1 -- 172.21.15.53:0/2517248209 wait complete. 2024-05-10T10:39:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.522+0000 7f2940366640 1 Processor -- start 2024-05-10T10:39:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.522+0000 7f2940366640 1 -- start start 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f2940366640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29240039b0 0x7f2938150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f2940366640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29380b3890 0x7f2938154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f2940366640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29380b3d10 0x7f2938072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f2940366640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29380cb910 con 0x7f29380b3890 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f2940366640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f293815fa40 con 0x7f29240039b0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f2940366640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2938159d70 con 0x7f29380b3d10 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29380b3890 0x7f2938154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29380b3890 0x7f2938154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43402/0 (socket says 172.21.15.53:43402) 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 -- 172.21.15.53:0/3735546474 learned_addr learned my addr 172.21.15.53:0/3735546474 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293e102640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29240039b0 0x7f2938150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293e903640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29380b3d10 0x7f2938072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 -- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29380b3d10 msgr2=0x7f2938072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29380b3d10 0x7f2938072870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 -- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29240039b0 msgr2=0x7f2938150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29240039b0 0x7f2938150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 -- 172.21.15.53:0/3735546474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2938152bb0 con 0x7f29380b3890 2024-05-10T10:39:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293e903640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29380b3d10 0x7f2938072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.523+0000 7f293d901640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29380b3890 0x7f2938154790 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f29280119a0 tx=0x7f2928011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.524+0000 7f292effd640 1 -- 172.21.15.53:0/3735546474 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292800faf0 con 0x7f29380b3890 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.524+0000 7f292effd640 1 -- 172.21.15.53:0/3735546474 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292800fc90 con 0x7f29380b3890 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.524+0000 7f292effd640 1 -- 172.21.15.53:0/3735546474 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928014440 con 0x7f29380b3890 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.524+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29380c8b30 con 0x7f29380b3890 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.524+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2938153100 con 0x7f29380b3890 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.525+0000 7f292effd640 1 -- 172.21.15.53:0/3735546474 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2928012040 con 0x7f29380b3890 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.526+0000 7f292effd640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f291c0762d0 0x7f291c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.526+0000 7f292effd640 1 -- 172.21.15.53:0/3735546474 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f2928097ef0 con 0x7f29380b3890 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.526+0000 7f293e102640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f291c0762d0 0x7f291c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.527+0000 7f294307e740 1 Processor -- start 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.527+0000 7f294307e740 1 -- start start 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.527+0000 7f293e102640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f291c0762d0 0x7f291c078790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2934033000 tx=0x7f2934008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f294307e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c9964d9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c996509600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996509b40 0x55c9964de950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f294307e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9964a6690 con 0x55c996527160 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c996443f30 con 0x55c996509b40 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c9963fcb20 con 0x55c99650a280 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c9964d9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c9964d9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43410/0 (socket says 172.21.15.53:43410) 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 -- 172.21.15.53:0/722199023 learned_addr learned my addr 172.21.15.53:0/722199023 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e903640 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996509b40 0x55c9964de950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293d901640 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c996509600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 -- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 msgr2=0x55c996509600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c996509600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 -- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996509b40 msgr2=0x55c9964de950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996509b40 0x55c9964de950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 -- 172.21.15.53:0/722199023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9963f4e60 con 0x55c996527160 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.528+0000 7f293e102640 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c9964d9af0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2934009be0 tx=0x7f293402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73194f63b8f2dd37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f291bfff640 1 -- 172.21.15.53:0/722199023 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934036040 con 0x55c996527160 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f291bfff640 1 -- 172.21.15.53:0/722199023 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293404c070 con 0x55c996527160 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f291bfff640 1 -- 172.21.15.53:0/722199023 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934003d00 con 0x55c996527160 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 -- 172.21.15.53:0/722199023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 msgr2=0x55c9964d9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c9964d9af0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2934009be0 tx=0x7f293402e000 comp rx=0 tx=0).stop 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 -- 172.21.15.53:0/722199023 shutdown_connections 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996509b40 0x55c9964de950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c996509600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 --2- 172.21.15.53:0/722199023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c9964d9af0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 -- 172.21.15.53:0/722199023 >> 172.21.15.53:0/722199023 conn(0x55c9964dbd60 msgr2=0x55c996524600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 -- 172.21.15.53:0/722199023 shutdown_connections 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.529+0000 7f294307e740 1 -- 172.21.15.53:0/722199023 wait complete. 2024-05-10T10:39:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.530+0000 7f294307e740 1 -- start start 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.530+0000 7f294307e740 1 Processor -- start 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99650a280 0x55c9964f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f294307e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c99650c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c996524600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f294307e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9964005e0 con 0x55c996527160 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c99643fef0 con 0x55c99650a280 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c9964ddf10 con 0x55c99650c8c0 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f294307e740 10 client.0 _ll_register_callbacks cb 0x55c9963fb320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f293d901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c99650c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f293d901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c99650c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43414/0 (socket says 172.21.15.53:43414) 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f293d901640 1 -- 172.21.15.53:0/1518803280 learned_addr learned my addr 172.21.15.53:0/1518803280 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f293e102640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99650a280 0x55c9964f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.531+0000 7f293e903640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c996524600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293d901640 1 -- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 msgr2=0x55c996524600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293d901640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c996524600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293d901640 1 -- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99650a280 msgr2=0x55c9964f9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293d901640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99650a280 0x55c9964f9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293d901640 1 -- 172.21.15.53:0/1518803280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99621f7d0 con 0x55c996527160 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293e903640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c996524600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293d901640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c99650c380 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f29280961d0 tx=0x7f29280614a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e288664fc7cbfbfb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f29197fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f29197fa640 1 -- 172.21.15.53:0/1518803280 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928022040 con 0x55c996527160 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f29197fa640 1 -- 172.21.15.53:0/1518803280 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f29280600e0 con 0x55c996527160 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293404c070 con 0x55c996527160 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f29197fa640 1 -- 172.21.15.53:0/1518803280 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f292807a3e0 con 0x55c996527160 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 10 client.7014 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c9961553f0 con 0x55c996527160 2024-05-10T10:39:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f29197fa640 1 -- 172.21.15.53:0/1518803280 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293404c070 con 0x55c996527160 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f29197fa640 1 -- 172.21.15.53:0/1518803280 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f2928079070 con 0x55c996527160 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 10 client.7014 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 10 client.7014 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 20 client.7014 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c996537aa0 0x55c996537eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 4 client.7014 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c996451b40 con 0x55c996537aa0 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.532+0000 7f293e102640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c996537aa0 0x55c996537eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.533+0000 7f293e102640 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c996537aa0 0x55c996537eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abad7cbd6722f3a server_cookie=db22ac8f965702e5 in_seq=0 out_seq=0 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.533+0000 7f29197fa640 10 client.7014 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.533+0000 7f29197fa640 1 -- 172.21.15.53:0/1518803280 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c996451b40 con 0x55c996537aa0 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.533+0000 7f29197fa640 10 client.7014 handle_command_reply: tid=0 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.533+0000 7f294307e740 1 client.7014 shutdown 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 msgr2=0x55c99650c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c99650c380 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f29280961d0 tx=0x7f29280614a0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 shutdown_connections 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c996524600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c996537aa0 0x55c996537eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c996527160 0x55c99650c380 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 --2- 172.21.15.53:0/1518803280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99650a280 0x55c9964f9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f29197fa640 0 client.7014 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 >> 172.21.15.53:0/1518803280 conn(0x55c996509b40 msgr2=0x55c996525730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 shutdown_connections 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 1 -- 172.21.15.53:0/1518803280 wait complete. 2024-05-10T10:39:48.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.534+0000 7f294307e740 20 client.7014 trim_cache size 0 max 16384 2024-05-10T10:39:48.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.552+0000 7f294307e740 1 Processor -- start 2024-05-10T10:39:48.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.552+0000 7f294307e740 1 -- start start 2024-05-10T10:39:48.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f294307e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650a280 0x55c99650d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c99650d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 0x55c996547f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f294307e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29381fe210 con 0x55c99650a280 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29381fd1f0 con 0x55c996527160 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29381fc5d0 con 0x55c99650c8c0 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650a280 0x55c99650d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650a280 0x55c99650d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43430/0 (socket says 172.21.15.53:43430) 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 -- 172.21.15.53:0/481170817 learned_addr learned my addr 172.21.15.53:0/481170817 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 -- 172.21.15.53:0/481170817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 msgr2=0x55c99650d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 --2- 172.21.15.53:0/481170817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c99650d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 -- 172.21.15.53:0/481170817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 msgr2=0x55c996547f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 --2- 172.21.15.53:0/481170817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 0x55c996547f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 -- 172.21.15.53:0/481170817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9964dcea0 con 0x55c99650a280 2024-05-10T10:39:48.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f293e102640 1 --2- 172.21.15.53:0/481170817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650a280 0x55c99650d050 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f293402abe0 tx=0x7f293403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07d7586976a7e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.553+0000 7f291b7fe640 1 -- 172.21.15.53:0/481170817 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934036040 con 0x55c99650a280 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f291b7fe640 1 -- 172.21.15.53:0/481170817 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293404c040 con 0x55c99650a280 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f291b7fe640 1 -- 172.21.15.53:0/481170817 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293405a3f0 con 0x55c99650a280 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 -- 172.21.15.53:0/481170817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650a280 msgr2=0x55c99650d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 --2- 172.21.15.53:0/481170817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650a280 0x55c99650d050 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f293402abe0 tx=0x7f293403c000 comp rx=0 tx=0).stop 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 -- 172.21.15.53:0/481170817 shutdown_connections 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 --2- 172.21.15.53:0/481170817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 0x55c996547f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 --2- 172.21.15.53:0/481170817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650c8c0 0x55c99650d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 --2- 172.21.15.53:0/481170817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650a280 0x55c99650d050 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 -- 172.21.15.53:0/481170817 >> 172.21.15.53:0/481170817 conn(0x55c996509b40 msgr2=0x55c9964f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 -- 172.21.15.53:0/481170817 shutdown_connections 2024-05-10T10:39:48.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.554+0000 7f294307e740 1 -- 172.21.15.53:0/481170817 wait complete. 2024-05-10T10:39:48.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.555+0000 7f294307e740 1 -- start start 2024-05-10T10:39:48.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.555+0000 7f294307e740 1 Processor -- start 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c9965a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650c8c0 0x55c9965a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 0x55c9965229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29381fcba0 con 0x55c99650c8c0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c99653ea20 con 0x55c996527160 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55c99653e860 con 0x55c99650a280 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 10 client.0 _ll_register_callbacks cb 0x7f292805d4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650c8c0 0x55c9965a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650c8c0 0x55c9965a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43444/0 (socket says 172.21.15.53:43444) 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 -- 172.21.15.53:0/2787744555 learned_addr learned my addr 172.21.15.53:0/2787744555 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293e102640 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c9965a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293e903640 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 0x55c9965229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 -- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 msgr2=0x55c9965a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c9965a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 -- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 msgr2=0x55c9965229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 0x55c9965229c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 -- 172.21.15.53:0/2787744555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9961150f0 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f293d901640 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650c8c0 0x55c9965a25a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2928060b80 tx=0x7f2928000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54c4c5b93243fd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f29197fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f29197fa640 1 -- 172.21.15.53:0/2787744555 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928022070 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f29197fa640 1 -- 172.21.15.53:0/2787744555 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f2928067650 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.556+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293404c040 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f29197fa640 1 -- 172.21.15.53:0/2787744555 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f2928071050 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 10 client.7020 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55c9965232a0 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f29197fa640 1 -- 172.21.15.53:0/2787744555 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293404c040 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f29197fa640 1 -- 172.21.15.53:0/2787744555 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f2928074020 con 0x55c99650c8c0 2024-05-10T10:39:48.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 10 client.7020 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 10 client.7020 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 20 client.7020 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c9964f9120 0x55c996532d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 4 client.7020 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c99644b9b0 con 0x55c9964f9120 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f293e102640 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c9964f9120 0x55c996532d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f29197fa640 10 client.7020 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f293e102640 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c9964f9120 0x55c996532d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49794a5e4e1eeb3 server_cookie=706fdb37bd84801e in_seq=0 out_seq=0 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f29197fa640 1 -- 172.21.15.53:0/2787744555 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c99644b9b0 con 0x55c9964f9120 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f29197fa640 10 client.7020 handle_command_reply: tid=0 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.557+0000 7f294307e740 1 client.7020 shutdown 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650c8c0 msgr2=0x55c9965a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650c8c0 0x55c9965a25a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2928060b80 tx=0x7f2928000bc0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 shutdown_connections 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c996527160 0x55c9965229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55c9964f9120 0x55c996532d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c99650c8c0 0x55c9965a25a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 --2- 172.21.15.53:0/2787744555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55c99650a280 0x55c9965a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f29197fa640 0 client.7020 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 >> 172.21.15.53:0/2787744555 conn(0x55c996509b40 msgr2=0x55c99651d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 shutdown_connections 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 1 -- 172.21.15.53:0/2787744555 wait complete. 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f294307e740 20 client.7020 trim_cache size 0 max 16384 2024-05-10T10:39:48.565 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:39:48.566 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f291c0762d0 msgr2=0x7f291c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f2940366640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f291c0762d0 0x7f291c078790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2934033000 tx=0x7f2934008030 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29380b3890 msgr2=0x7f2938154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.558+0000 7f2940366640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29380b3890 0x7f2938154790 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f29280119a0 tx=0x7f2928011e60 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 shutdown_connections 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f291c0762d0 0x7f291c078790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29380b3d10 0x7f2938072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29380b3890 0x7f2938154790 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 --2- 172.21.15.53:0/3735546474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29240039b0 0x7f2938150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 >> 172.21.15.53:0/3735546474 conn(0x7f29380d0c90 msgr2=0x7f2938073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 shutdown_connections 2024-05-10T10:39:48.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.559+0000 7f2940366640 1 -- 172.21.15.53:0/3735546474 wait complete. 2024-05-10T10:39:48.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:39:48.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:39:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.673+0000 7eff009dd640 1 Processor -- start 2024-05-10T10:39:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.673+0000 7eff009dd640 1 -- start start 2024-05-10T10:39:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7eff009dd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efefc0d0230 0x7efefc0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7eff009dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0cfbb0 0x7efefc0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7eff009dd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efefc15fc90 con 0x7efefc0b8860 2024-05-10T10:39:48.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7eff009dd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efefc06ec30 con 0x7efefc0d0230 2024-05-10T10:39:48.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7eff009dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efefc06ee50 con 0x7efefc0cfbb0 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7efef9d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efefc0d0230 0x7efefc0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7efefad9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0cfbb0 0x7efefc0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7efef9d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efefc0d0230 0x7efefc0cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:49252/0 (socket says 172.21.15.53:49252) 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7efef9d9b640 1 -- 172.21.15.53:0/2284408099 learned_addr learned my addr 172.21.15.53:0/2284408099 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7efefa59c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7efefc0b8860 0x7efefc06e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:37642/0 (socket says 172.21.15.53:37642) 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.674+0000 7efefad9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0cfbb0 0x7efefc0ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:57208/0 (socket says 172.21.15.53:57208) 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.675+0000 7efef9d9b640 1 -- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0cfbb0 msgr2=0x7efefc0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:48.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.675+0000 7efef9d9b640 1 --2- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0cfbb0 0x7efefc0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.675+0000 7efef9d9b640 1 -- 172.21.15.53:0/2284408099 >> v1:172.21.15.135:6789/0 conn(0x7efefc0b8860 legacy=0x7efefc06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.675+0000 7efef9d9b640 1 -- 172.21.15.53:0/2284408099 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc0b8c40 con 0x7efefc0d0230 2024-05-10T10:39:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.675+0000 7efefad9d640 1 --2- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0cfbb0 0x7efefc0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:39:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7efef9d9b640 1 --2- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0d0230 0x7efefc0cf670 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efef00098a0 tx=0x7efef002a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34dc1136646c96c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7efef959a640 1 -- 172.21.15.53:0/2284408099 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0037070 con 0x7efefc0d0230 2024-05-10T10:39:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7efef959a640 1 -- 172.21.15.53:0/2284408099 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef0032440 con 0x7efefc0d0230 2024-05-10T10:39:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7efef959a640 1 -- 172.21.15.53:0/2284408099 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0032810 con 0x7efefc0d0230 2024-05-10T10:39:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 -- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0d0230 msgr2=0x7efefc0cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 --2- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0d0230 0x7efefc0cf670 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efef00098a0 tx=0x7efef002a200 comp rx=0 tx=0).stop 2024-05-10T10:39:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 -- 172.21.15.53:0/2284408099 shutdown_connections 2024-05-10T10:39:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 --2- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0cfbb0 0x7efefc0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 --2- 172.21.15.53:0/2284408099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0d0230 0x7efefc0cf670 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 -- 172.21.15.53:0/2284408099 >> 172.21.15.53:0/2284408099 conn(0x7efefc0d2df0 msgr2=0x7efefc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 -- 172.21.15.53:0/2284408099 shutdown_connections 2024-05-10T10:39:48.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.676+0000 7eff009dd640 1 -- 172.21.15.53:0/2284408099 wait complete. 2024-05-10T10:39:48.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 Processor -- start 2024-05-10T10:39:48.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 -- start start 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0b8860 0x7efefc149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0cfbb0 0x7efefc0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc0d0230 0x7efefc0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efefc0cdb30 con 0x7efefc0d0230 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efefc0cd850 con 0x7efefc0cfbb0 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.678+0000 7eff009dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efefc0cde10 con 0x7efefc0b8860 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc0d0230 0x7efefc0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc0d0230 0x7efefc0c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43456/0 (socket says 172.21.15.53:43456) 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 -- 172.21.15.53:0/2916668958 learned_addr learned my addr 172.21.15.53:0/2916668958 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efef9d9b640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0cfbb0 0x7efefc0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefa59c640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0b8860 0x7efefc149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 -- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0b8860 msgr2=0x7efefc149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0b8860 0x7efefc149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 -- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0cfbb0 msgr2=0x7efefc0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0cfbb0 0x7efefc0cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 -- 172.21.15.53:0/2916668958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc0cd180 con 0x7efefc0d0230 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efef9d9b640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0cfbb0 0x7efefc0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efefad9d640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc0d0230 0x7efefc0c9ec0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efef400efc0 tx=0x7efef4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efeeaffd640 1 -- 172.21.15.53:0/2916668958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4018030 con 0x7efefc0d0230 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efeeaffd640 1 -- 172.21.15.53:0/2916668958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef400cde0 con 0x7efefc0d0230 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7efeeaffd640 1 -- 172.21.15.53:0/2916668958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4003bb0 con 0x7efefc0d0230 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc0cc8e0 con 0x7efefc0d0230 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.679+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc0ccea0 con 0x7efefc0d0230 2024-05-10T10:39:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.680+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc06a220 con 0x7efefc0d0230 2024-05-10T10:39:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.681+0000 7efeeaffd640 1 -- 172.21.15.53:0/2916668958 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efef4005d10 con 0x7efefc0d0230 2024-05-10T10:39:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.681+0000 7efeeaffd640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efedc0762d0 0x7efedc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.682+0000 7efeeaffd640 1 -- 172.21.15.53:0/2916668958 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7efef4098280 con 0x7efefc0d0230 2024-05-10T10:39:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.682+0000 7efefa59c640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efedc0762d0 0x7efedc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:48.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.682+0000 7efefa59c640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efedc0762d0 0x7efedc078790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efeec000f60 tx=0x7efeec007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:48.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.686+0000 7efeeaffd640 1 -- 172.21.15.53:0/2916668958 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7efef4022070 con 0x7efefc0d0230 2024-05-10T10:39:48.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.915+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efefc070970 con 0x7efefc0d0230 2024-05-10T10:39:48.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.916+0000 7efeeaffd640 1 -- 172.21.15.53:0/2916668958 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5184 (secure 0 0 0) 0x7efef4010040 con 0x7efefc0d0230 2024-05-10T10:39:48.917 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:48.917 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:22.393011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6833/285942496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":285942496},{"type":"v1","addr":"172.21.15.53:6833","nonce":285942496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6888,"qdb_cluster":[6888]},"id":6}]} 2024-05-10T10:39:48.918 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 68 2024-05-10T10:39:48.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efedc0762d0 msgr2=0x7efedc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efedc0762d0 0x7efedc078790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efeec000f60 tx=0x7efeec007cc0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc0d0230 msgr2=0x7efefc0c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc0d0230 0x7efefc0c9ec0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efef400efc0 tx=0x7efef4009f90 comp rx=0 tx=0).stop 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 shutdown_connections 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efedc0762d0 0x7efedc078790 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efefc0d0230 0x7efefc0c9ec0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efefc0cfbb0 0x7efefc0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 --2- 172.21.15.53:0/2916668958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efefc0b8860 0x7efefc149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 >> 172.21.15.53:0/2916668958 conn(0x7efefc0d2df0 msgr2=0x7efefc070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 shutdown_connections 2024-05-10T10:39:48.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:48.918+0000 7eff009dd640 1 -- 172.21.15.53:0/2916668958 wait complete. 2024-05-10T10:39:48.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-10T10:39:49.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.034+0000 7f2303379640 1 Processor -- start 2024-05-10T10:39:49.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.034+0000 7f2303379640 1 -- start start 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2303379640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f22fc15f190 0x7f22fc0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2303379640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22fc06d3e0 0x7f22fc0cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2303379640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22fc06d920 con 0x7f22fc06e920 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2303379640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f22fc06dc00 con 0x7f22fc06d3e0 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2303379640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f22fc0c6b80 con 0x7f22fc15f190 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301916640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22fc06d3e0 0x7f22fc0cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301916640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22fc06d3e0 0x7f22fc0cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43458/0 (socket says 172.21.15.53:43458) 2024-05-10T10:39:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301916640 1 -- 172.21.15.53:0/3191697052 learned_addr learned my addr 172.21.15.53:0/3191697052 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301115640 1 --2- 172.21.15.53:0/3191697052 >> v2:172.21.15.135:3300/0 conn(0x7f22fc15f190 0x7f22fc0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301916640 1 -- 172.21.15.53:0/3191697052 >> v2:172.21.15.135:3300/0 conn(0x7f22fc15f190 msgr2=0x7f22fc0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301916640 1 --2- 172.21.15.53:0/3191697052 >> v2:172.21.15.135:3300/0 conn(0x7f22fc15f190 0x7f22fc0cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301916640 1 -- 172.21.15.53:0/3191697052 >> v1:172.21.15.135:6789/0 conn(0x7f22fc06e920 legacy=0x7f22fc06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.035+0000 7f2301916640 1 -- 172.21.15.53:0/3191697052 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc0c6e60 con 0x7f22fc06d3e0 2024-05-10T10:39:49.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.036+0000 7f2301916640 1 --2- 172.21.15.53:0/3191697052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc0cfeb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f22fc073bc0 tx=0x7f22f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb2efe8127fbdf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.036+0000 7f22f3fff640 1 -- 172.21.15.53:0/3191697052 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4037070 con 0x7f22fc06d3e0 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.036+0000 7f22f3fff640 1 -- 172.21.15.53:0/3191697052 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f4032440 con 0x7f22fc06d3e0 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.036+0000 7f22f3fff640 1 -- 172.21.15.53:0/3191697052 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4032810 con 0x7f22fc06d3e0 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 -- 172.21.15.53:0/3191697052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 msgr2=0x7f22fc0cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 --2- 172.21.15.53:0/3191697052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc0cfeb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f22fc073bc0 tx=0x7f22f402a270 comp rx=0 tx=0).stop 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 -- 172.21.15.53:0/3191697052 shutdown_connections 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 --2- 172.21.15.53:0/3191697052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc0cfeb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 --2- 172.21.15.53:0/3191697052 >> v2:172.21.15.135:3300/0 conn(0x7f22fc15f190 0x7f22fc0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 -- 172.21.15.53:0/3191697052 >> 172.21.15.53:0/3191697052 conn(0x7f22fc0ce0a0 msgr2=0x7f22fc0ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 -- 172.21.15.53:0/3191697052 shutdown_connections 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.037+0000 7f2303379640 1 -- 172.21.15.53:0/3191697052 wait complete. 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.038+0000 7f2303379640 1 Processor -- start 2024-05-10T10:39:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.038+0000 7f2303379640 1 -- start start 2024-05-10T10:39:49.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2303379640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc14da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2303379640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22fc06e920 0x7f22fc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2303379640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22fc15f190 0x7f22fc072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2303379640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc168df0 con 0x7f22fc06d3e0 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2303379640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc15f570 con 0x7f22fc06e920 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2303379640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc168f70 con 0x7f22fc15f190 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc14da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc14da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43464/0 (socket says 172.21.15.53:43464) 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 -- 172.21.15.53:0/1748453433 learned_addr learned my addr 172.21.15.53:0/1748453433 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301916640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22fc15f190 0x7f22fc072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2300914640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22fc06e920 0x7f22fc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 -- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22fc15f190 msgr2=0x7f22fc072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22fc15f190 0x7f22fc072ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 -- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22fc06e920 msgr2=0x7f22fc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22fc06e920 0x7f22fc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 -- 172.21.15.53:0/1748453433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc15e030 con 0x7f22fc06d3e0 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f2301115640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc14da70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009db0 tx=0x7f22f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f22f1ffb640 1 -- 172.21.15.53:0/1748453433 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8018020 con 0x7f22fc06d3e0 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.039+0000 7f22f1ffb640 1 -- 172.21.15.53:0/1748453433 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f8014480 con 0x7f22fc06d3e0 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.040+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22fc15e310 con 0x7f22fc06d3e0 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.040+0000 7f22f1ffb640 1 -- 172.21.15.53:0/1748453433 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8014850 con 0x7f22fc06d3e0 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.040+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22fc15a7a0 con 0x7f22fc06d3e0 2024-05-10T10:39:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.041+0000 7f22f1ffb640 1 -- 172.21.15.53:0/1748453433 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f22f8005d10 con 0x7f22fc06d3e0 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.042+0000 7f22f1ffb640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22e0076300 0x7f22e00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.042+0000 7f22f1ffb640 1 -- 172.21.15.53:0/1748453433 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f22f8098210 con 0x7f22fc06d3e0 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.042+0000 7f2300914640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22e0076300 0x7f22e00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.042+0000 7f2306168740 1 Processor -- start 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.042+0000 7f2300914640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22e0076300 0x7f22e00787c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b7830 tx=0x7f22ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2306168740 1 -- start start 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d7752f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2306168740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a50b0 0x56233d779ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2306168740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56233d665920 con 0x56233d7a57f0 2024-05-10T10:39:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56233d6dadd0 con 0x56233d7a50b0 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56233d68b3e0 con 0x56233d7c26d0 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2300914640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2300914640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43470/0 (socket says 172.21.15.53:43470) 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.043+0000 7f2300914640 1 -- 172.21.15.53:0/3760718987 learned_addr learned my addr 172.21.15.53:0/3760718987 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2301916640 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a50b0 0x56233d779ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2301115640 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d7752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2300914640 1 -- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 msgr2=0x56233d7752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2300914640 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d7752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2300914640 1 -- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a50b0 msgr2=0x56233d779ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2300914640 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a50b0 0x56233d779ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2300914640 1 -- 172.21.15.53:0/3760718987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233d4bc7c0 con 0x56233d7a57f0 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2301916640 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a50b0 0x56233d779ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f2300914640 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a4b70 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00fba0 tx=0x7f22ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ccab8ebcf42ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f22e6ffd640 1 -- 172.21.15.53:0/3760718987 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec026070 con 0x56233d7a57f0 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f22e6ffd640 1 -- 172.21.15.53:0/3760718987 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec007030 con 0x56233d7a57f0 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.044+0000 7f22e6ffd640 1 -- 172.21.15.53:0/3760718987 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0073e0 con 0x56233d7a57f0 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 -- 172.21.15.53:0/3760718987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 msgr2=0x56233d7a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a4b70 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00fba0 tx=0x7f22ec012000 comp rx=0 tx=0).stop 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 -- 172.21.15.53:0/3760718987 shutdown_connections 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a50b0 0x56233d779ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a4b70 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 --2- 172.21.15.53:0/3760718987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d7752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 -- 172.21.15.53:0/3760718987 >> 172.21.15.53:0/3760718987 conn(0x56233d777560 msgr2=0x56233d7bfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 -- 172.21.15.53:0/3760718987 shutdown_connections 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 -- 172.21.15.53:0/3760718987 wait complete. 2024-05-10T10:39:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 -- start start 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.045+0000 7f2306168740 1 Processor -- start 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2306168740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d78f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7a1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2306168740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56233d6e3440 con 0x56233d7a57f0 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56233d6de120 con 0x56233d7a11f0 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56233d6ec210 con 0x56233d7c26d0 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2306168740 10 client.0 _ll_register_callbacks cb 0x56233d683690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2301115640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2301115640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43486/0 (socket says 172.21.15.53:43486) 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2301115640 1 -- 172.21.15.53:0/1131813912 learned_addr learned my addr 172.21.15.53:0/1131813912 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2300914640 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d78f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.046+0000 7f2301916640 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2301115640 1 -- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 msgr2=0x56233d78f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2301115640 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d78f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2301115640 1 -- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 msgr2=0x56233d7a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2301115640 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2301115640 1 -- 172.21.15.53:0/1131813912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233d51cf50 con 0x56233d7a57f0 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2301115640 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a5ce0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f22f80964f0 tx=0x7f22f80618f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42a9e6803645bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f22dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f22dbfff640 1 -- 172.21.15.53:0/1131813912 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f80612d0 con 0x56233d7a57f0 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f22dbfff640 1 -- 172.21.15.53:0/1131813912 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f22f805cc40 con 0x56233d7a57f0 2024-05-10T10:39:49.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22ec007030 con 0x56233d7a57f0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f22dbfff640 1 -- 172.21.15.53:0/1131813912 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f22f80655d0 con 0x56233d7a57f0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 10 client.7035 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x56233d3b00b0 con 0x56233d7a57f0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f22dbfff640 1 -- 172.21.15.53:0/1131813912 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec007030 con 0x56233d7a57f0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f22dbfff640 1 -- 172.21.15.53:0/1131813912 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f22f805ce40 con 0x56233d7a57f0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 10 client.7035 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 10 client.7035 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 20 client.7035 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7919f0 0x56233d791dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 4 client.7035 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.047+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233d6eca20 con 0x56233d7919f0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f2300914640 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7919f0 0x56233d791dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f2300914640 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7919f0 0x56233d791dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e63eb8b2ab4d6 server_cookie=818708520a4cae1c in_seq=0 out_seq=0 2024-05-10T10:39:49.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f22dbfff640 10 client.7035 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f22dbfff640 1 -- 172.21.15.53:0/1131813912 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56233d6eca20 con 0x56233d7919f0 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f22dbfff640 10 client.7035 handle_command_reply: tid=0 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f2306168740 1 client.7035 shutdown 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 msgr2=0x56233d7a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.048+0000 7f2306168740 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a5ce0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f22f80964f0 tx=0x7f22f80618f0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 shutdown_connections 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7c26d0 0x56233d78f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7919f0 0x56233d791dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 --2- 172.21.15.53:0/1131813912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7a57f0 0x56233d7a5ce0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f22dbfff640 0 client.7035 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 >> 172.21.15.53:0/1131813912 conn(0x56233d7a50b0 msgr2=0x56233d774a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 shutdown_connections 2024-05-10T10:39:49.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 1 -- 172.21.15.53:0/1131813912 wait complete. 2024-05-10T10:39:49.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.049+0000 7f2306168740 20 client.7035 trim_cache size 0 max 16384 2024-05-10T10:39:49.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.062+0000 7f2306168740 1 Processor -- start 2024-05-10T10:39:49.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2306168740 1 -- start start 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a11f0 0x56233d7cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a57f0 0x56233d7a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2306168740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2306168740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc200f60 con 0x56233d7c26d0 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc156010 con 0x56233d7a57f0 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc200940 con 0x56233d7a11f0 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2301916640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2301916640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43498/0 (socket says 172.21.15.53:43498) 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.063+0000 7f2301916640 1 -- 172.21.15.53:0/1854468908 learned_addr learned my addr 172.21.15.53:0/1854468908 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2300914640 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a57f0 0x56233d7a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301115640 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a11f0 0x56233d7cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301916640 1 -- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a11f0 msgr2=0x56233d7cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301916640 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a11f0 0x56233d7cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301916640 1 -- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a57f0 msgr2=0x56233d7a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301916640 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a57f0 0x56233d7a02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301916640 1 -- 172.21.15.53:0/1854468908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233d77c010 con 0x56233d7c26d0 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2300914640 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a57f0 0x56233d7a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301115640 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a11f0 0x56233d7cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:49.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2301916640 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7a0810 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009930 tx=0x7f22f402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a927258feed278 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f22e67fc640 1 -- 172.21.15.53:0/1854468908 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4037040 con 0x56233d7c26d0 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f22e67fc640 1 -- 172.21.15.53:0/1854468908 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f40327a0 con 0x56233d7c26d0 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f22e67fc640 1 -- 172.21.15.53:0/1854468908 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4032b90 con 0x56233d7c26d0 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2306168740 1 -- 172.21.15.53:0/1854468908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 msgr2=0x56233d7a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.064+0000 7f2306168740 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7a0810 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009930 tx=0x7f22f402c000 comp rx=0 tx=0).stop 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 -- 172.21.15.53:0/1854468908 shutdown_connections 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7a0810 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a57f0 0x56233d7a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 --2- 172.21.15.53:0/1854468908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a11f0 0x56233d7cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 -- 172.21.15.53:0/1854468908 >> 172.21.15.53:0/1854468908 conn(0x56233d7a50b0 msgr2=0x56233d7a70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 -- 172.21.15.53:0/1854468908 shutdown_connections 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 -- 172.21.15.53:0/1854468908 wait complete. 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 -- start start 2024-05-10T10:39:49.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.065+0000 7f2306168740 1 Processor -- start 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2306168740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a57f0 0x56233d7bee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2306168740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2306168740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56233d7b68c0 con 0x56233d7c26d0 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56233d694a20 con 0x56233d7a11f0 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2306168740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56233d7a85c0 con 0x56233d7a57f0 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2306168740 10 client.0 _ll_register_callbacks cb 0x7f22f8098130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7bf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43508/0 (socket says 172.21.15.53:43508) 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 -- 172.21.15.53:0/325423065 learned_addr learned my addr 172.21.15.53:0/325423065 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301115640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2300914640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a57f0 0x56233d7bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 -- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a57f0 msgr2=0x56233d7bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a57f0 0x56233d7bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 -- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 msgr2=0x56233d7c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.066+0000 7f2301916640 1 -- 172.21.15.53:0/325423065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233d6926a0 con 0x56233d7c26d0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2300914640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a57f0 0x56233d7bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2301916640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7bf3a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f22f4033000 tx=0x7f22f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46de6749cb25c709 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f22dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f22dbfff640 1 -- 172.21.15.53:0/325423065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4037040 con 0x56233d7c26d0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f22dbfff640 1 -- 172.21.15.53:0/325423065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f22f404fd40 con 0x56233d7c26d0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f40327a0 con 0x56233d7c26d0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f22dbfff640 1 -- 172.21.15.53:0/325423065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f22f4032b50 con 0x56233d7c26d0 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 10 client.7041 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:49.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x56233d7b99c0 con 0x56233d7c26d0 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f22dbfff640 1 -- 172.21.15.53:0/325423065 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f40327a0 con 0x56233d7c26d0 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f22dbfff640 1 -- 172.21.15.53:0/325423065 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f22f4058080 con 0x56233d7c26d0 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 10 client.7041 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 10 client.7041 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 20 client.7041 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7c0f30 0x56233d7c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 4 client.7041 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56233d6e6890 con 0x56233d7c0f30 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.067+0000 7f2301115640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7c0f30 0x56233d7c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f22dbfff640 10 client.7041 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2301115640 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7c0f30 0x56233d7c1310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b66b19c7899f8b server_cookie=ec0d26a9f699ef68 in_seq=0 out_seq=0 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f22dbfff640 1 -- 172.21.15.53:0/325423065 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x56233d6e6890 con 0x56233d7c0f30 2024-05-10T10:39:49.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f22dbfff640 10 client.7041 handle_command_reply: tid=0 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 client.7041 shutdown 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 msgr2=0x56233d7bf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7bf3a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f22f4033000 tx=0x7f22f4004730 comp rx=0 tx=0).stop 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 shutdown_connections 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x56233d7c0f30 0x56233d7c1310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56233d7c26d0 0x56233d7bf3a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56233d7a57f0 0x56233d7bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f22dbfff640 0 client.7041 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 --2- 172.21.15.53:0/325423065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233d7a11f0 0x56233d7c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 >> 172.21.15.53:0/325423065 conn(0x56233d7a50b0 msgr2=0x56233d78e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 shutdown_connections 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 1 -- 172.21.15.53:0/325423065 wait complete. 2024-05-10T10:39:49.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.068+0000 7f2306168740 20 client.7041 trim_cache size 0 max 16384 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "damage_type": "backtrace", 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 3300899879, 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627776, 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/dir" 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:39:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "damage_type": "backtrace", 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 3565970136, 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/testdir/hardlink" 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "damage_type": "dir_frag", 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 3705477333, 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627776, 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "frag": "*", 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/dir" 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:39:49.079 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:39:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22e0076300 msgr2=0x7f22e00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22e0076300 0x7f22e00787c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b7830 tx=0x7f22ec009040 comp rx=0 tx=0).stop 2024-05-10T10:39:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 msgr2=0x7f22fc14da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc14da70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009db0 tx=0x7f22f800c490 comp rx=0 tx=0).stop 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 shutdown_connections 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f22e0076300 0x7f22e00787c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22fc15f190 0x7f22fc072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22fc06e920 0x7f22fc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 --2- 172.21.15.53:0/1748453433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22fc06d3e0 0x7f22fc14da70 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 >> 172.21.15.53:0/1748453433 conn(0x7f22fc0ce0a0 msgr2=0x7f22fc0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 shutdown_connections 2024-05-10T10:39:49.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.069+0000 7f2303379640 1 -- 172.21.15.53:0/1748453433 wait complete. 2024-05-10T10:39:49.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T10:39:49.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.182+0000 7f0d8bf4e640 1 Processor -- start 2024-05-10T10:39:49.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.183+0000 7f0d8bf4e640 1 -- start start 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d8bf4e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d841611e0 0x7f0d840cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d8bf4e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d840c6a80 con 0x7f0d840c65f0 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d8bf4e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d840cd090 con 0x7f0d8415ce60 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d8bf4e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0d840cd370 con 0x7f0d841611e0 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d8a4eb640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0d840c65f0 0x7f0d8415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49072/0 (socket says 172.21.15.53:49072) 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d8a4eb640 1 -- 172.21.15.53:0/510175111 learned_addr learned my addr 172.21.15.53:0/510175111 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d894e9640 1 --2- 172.21.15.53:0/510175111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d841611e0 0x7f0d840cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704771115 0 0) 0x7f0d7c001490 con 0x7f0d840c65f0 2024-05-10T10:39:49.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d70003520 con 0x7f0d840c65f0 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651051894 0 0) 0x7f0d840cd090 con 0x7f0d8415ce60 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d7c001490 con 0x7f0d8415ce60 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d894e9640 1 -- 172.21.15.53:0/510175111 >> v1:172.21.15.135:6789/0 conn(0x7f0d8415ce60 legacy=0x7f0d84160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4169081562 0 0) 0x7f0d70003520 con 0x7f0d840c65f0 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d894e9640 1 -- 172.21.15.53:0/510175111 >> v1:172.21.15.53:6789/0 conn(0x7f0d840c65f0 legacy=0x7f0d8415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.184+0000 7f0d894e9640 1 -- 172.21.15.53:0/510175111 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d840cd650 con 0x7f0d841611e0 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.185+0000 7f0d894e9640 1 --2- 172.21.15.53:0/510175111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d841611e0 0x7f0d840cb300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0d740097d0 tx=0x7f0d7402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70cbb36257aefdeb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.185+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74037070 con 0x7f0d841611e0 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.185+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7402ab90 con 0x7f0d841611e0 2024-05-10T10:39:49.188 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d88ce8640 1 -- 172.21.15.53:0/510175111 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c001bc0 con 0x7f0d841611e0 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/510175111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d841611e0 msgr2=0x7f0d840cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/510175111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d841611e0 0x7f0d840cb300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0d740097d0 tx=0x7f0d7402a100 comp rx=0 tx=0).stop 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/510175111 shutdown_connections 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/510175111 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d841611e0 0x7f0d840cb300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/510175111 >> 172.21.15.53:0/510175111 conn(0x7f0d840b2900 msgr2=0x7f0d840b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/510175111 shutdown_connections 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.186+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/510175111 wait complete. 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.187+0000 7f0d8bf4e640 1 Processor -- start 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.187+0000 7f0d8bf4e640 1 -- start start 2024-05-10T10:39:49.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8bf4e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d840c65f0 0x7f0d841f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8bf4e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d8415ce60 0x7f0d840b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8bf4e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d841611e0 0x7f0d841f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8bf4e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d840cde50 con 0x7f0d841611e0 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8bf4e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d840ccd60 con 0x7f0d840c65f0 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8bf4e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0d840cdfd0 con 0x7f0d8415ce60 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8a4eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d841611e0 0x7f0d841f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8a4eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d841611e0 0x7f0d841f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43518/0 (socket says 172.21.15.53:43518) 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d8a4eb640 1 -- 172.21.15.53:0/1217598793 learned_addr learned my addr 172.21.15.53:0/1217598793 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d894e9640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d8415ce60 0x7f0d840b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.188+0000 7f0d89cea640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d840c65f0 0x7f0d841f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8a4eb640 1 -- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d8415ce60 msgr2=0x7f0d840b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8a4eb640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d8415ce60 0x7f0d840b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.192 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8a4eb640 1 -- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d840c65f0 msgr2=0x7f0d841f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8a4eb640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d840c65f0 0x7f0d841f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8a4eb640 1 -- 172.21.15.53:0/1217598793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d841f5700 con 0x7f0d841611e0 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8a4eb640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d841611e0 0x7f0d841f4cb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c001350 tx=0x7f0d7c001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d7a7fc640 1 -- 172.21.15.53:0/1217598793 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c017070 con 0x7f0d841611e0 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d7a7fc640 1 -- 172.21.15.53:0/1217598793 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c012400 con 0x7f0d841611e0 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d841f59e0 con 0x7f0d841611e0 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d7a7fc640 1 -- 172.21.15.53:0/1217598793 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0127d0 con 0x7f0d841611e0 2024-05-10T10:39:49.193 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.189+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841f8720 con 0x7f0d841611e0 2024-05-10T10:39:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.190+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d841f8920 con 0x7f0d841611e0 2024-05-10T10:39:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.191+0000 7f0d7a7fc640 1 -- 172.21.15.53:0/1217598793 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d7c01a070 con 0x7f0d841611e0 2024-05-10T10:39:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.191+0000 7f0d7a7fc640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d680762d0 0x7f0d68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.191+0000 7f0d7a7fc640 1 -- 172.21.15.53:0/1217598793 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f0d7c097900 con 0x7f0d841611e0 2024-05-10T10:39:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.191+0000 7f0d89cea640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d680762d0 0x7f0d68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.192+0000 7f0d89cea640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d680762d0 0x7f0d68078790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0d800011e0 tx=0x7f0d80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.196+0000 7f0d7a7fc640 1 -- 172.21.15.53:0/1217598793 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0d840c6a80 con 0x7f0d841611e0 2024-05-10T10:39:49.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.428+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0d8406a220 con 0x7f0d841611e0 2024-05-10T10:39:49.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.429+0000 7f0d7a7fc640 1 -- 172.21.15.53:0/1217598793 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f0d7c0609c0 con 0x7f0d841611e0 2024-05-10T10:39:49.430 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:39:49.430 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-10T10:39:49.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d680762d0 msgr2=0x7f0d68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d680762d0 0x7f0d68078790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0d800011e0 tx=0x7f0d80009040 comp rx=0 tx=0).stop 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d841611e0 msgr2=0x7f0d841f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d841611e0 0x7f0d841f4cb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c001350 tx=0x7f0d7c001cf0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 shutdown_connections 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d680762d0 0x7f0d68078790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d841611e0 0x7f0d841f4cb0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d8415ce60 0x7f0d840b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 --2- 172.21.15.53:0/1217598793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d840c65f0 0x7f0d841f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 >> 172.21.15.53:0/1217598793 conn(0x7f0d840b2900 msgr2=0x7f0d84158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 shutdown_connections 2024-05-10T10:39:49.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.431+0000 7f0d8bf4e640 1 -- 172.21.15.53:0/1217598793 wait complete. 2024-05-10T10:39:49.443 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T10:39:49.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:49.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-10T10:39:49.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.549+0000 7fddbb111640 1 Processor -- start 2024-05-10T10:39:49.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.549+0000 7fddbb111640 1 -- start start 2024-05-10T10:39:49.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddbb111640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fddb415f1e0 0x7fddb406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddbb111640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fddb406c800 0x7fddb40cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddbb111640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb40d0520 con 0x7fddb40d00f0 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddbb111640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fddb40cc680 con 0x7fddb406c800 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddbb111640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fddb406f580 con 0x7fddb415f1e0 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddb96ae640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fddb40d00f0 0x7fddb40b08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49078/0 (socket says 172.21.15.53:49078) 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddb96ae640 1 -- 172.21.15.53:0/4150620497 learned_addr learned my addr 172.21.15.53:0/4150620497 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddb3fff640 1 --2- 172.21.15.53:0/4150620497 >> v2:172.21.15.53:3300/0 conn(0x7fddb406c800 0x7fddb40cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.550+0000 7fddb8ead640 1 --2- 172.21.15.53:0/4150620497 >> v2:172.21.15.135:3300/0 conn(0x7fddb415f1e0 0x7fddb406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.551+0000 7fddb3fff640 1 -- 172.21.15.53:0/4150620497 >> v2:172.21.15.135:3300/0 conn(0x7fddb415f1e0 msgr2=0x7fddb406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.551+0000 7fddb3fff640 1 --2- 172.21.15.53:0/4150620497 >> v2:172.21.15.135:3300/0 conn(0x7fddb415f1e0 0x7fddb406c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.551+0000 7fddb3fff640 1 -- 172.21.15.53:0/4150620497 >> v1:172.21.15.53:6789/0 conn(0x7fddb40d00f0 legacy=0x7fddb40b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.551+0000 7fddb37fe640 1 -- 172.21.15.53:0/4150620497 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814985386 0 0) 0x7fddb40d0520 con 0x7fddb40d00f0 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.551+0000 7fddb3fff640 1 -- 172.21.15.53:0/4150620497 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40d0790 con 0x7fddb406c800 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.551+0000 7fddb3fff640 1 --2- 172.21.15.53:0/4150620497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb406c800 0x7fddb40cfbb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fddb406e740 tx=0x7fdda802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273ed1c07d82243 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddb37fe640 1 -- 172.21.15.53:0/4150620497 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8037070 con 0x7fddb406c800 2024-05-10T10:39:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddb37fe640 1 -- 172.21.15.53:0/4150620497 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8032440 con 0x7fddb406c800 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddb37fe640 1 -- 172.21.15.53:0/4150620497 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8032810 con 0x7fddb406c800 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 -- 172.21.15.53:0/4150620497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb406c800 msgr2=0x7fddb40cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 --2- 172.21.15.53:0/4150620497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb406c800 0x7fddb40cfbb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fddb406e740 tx=0x7fdda802a1a0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 -- 172.21.15.53:0/4150620497 shutdown_connections 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 --2- 172.21.15.53:0/4150620497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb406c800 0x7fddb40cfbb0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 --2- 172.21.15.53:0/4150620497 >> v2:172.21.15.135:3300/0 conn(0x7fddb415f1e0 0x7fddb406c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 -- 172.21.15.53:0/4150620497 >> 172.21.15.53:0/4150620497 conn(0x7fddb40d1ce0 msgr2=0x7fddb40d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 -- 172.21.15.53:0/4150620497 shutdown_connections 2024-05-10T10:39:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.552+0000 7fddbb111640 1 -- 172.21.15.53:0/4150620497 wait complete. 2024-05-10T10:39:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.553+0000 7fddbb111640 1 Processor -- start 2024-05-10T10:39:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.553+0000 7fddbb111640 1 -- start start 2024-05-10T10:39:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddbb111640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddb406c800 0x7fddb40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddbb111640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb40d00f0 0x7fddb40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddbb111640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb415f1e0 0x7fddb4154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddbb111640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddb415f5c0 con 0x7fddb415f1e0 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddbb111640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fddb406f210 con 0x7fddb40d00f0 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddbb111640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40b0fd0 con 0x7fddb406c800 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddb96ae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb415f1e0 0x7fddb4154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.554+0000 7fddb96ae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb415f1e0 0x7fddb4154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43540/0 (socket says 172.21.15.53:43540) 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb96ae640 1 -- 172.21.15.53:0/1884910996 learned_addr learned my addr 172.21.15.53:0/1884910996 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb3fff640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb40d00f0 0x7fddb40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb8ead640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddb406c800 0x7fddb40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb96ae640 1 -- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddb406c800 msgr2=0x7fddb40c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb96ae640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddb406c800 0x7fddb40c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb96ae640 1 -- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb40d00f0 msgr2=0x7fddb40c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb96ae640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb40d00f0 0x7fddb40c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb96ae640 1 -- 172.21.15.53:0/1884910996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40ca610 con 0x7fddb415f1e0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb3fff640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb40d00f0 0x7fddb40c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb8ead640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddb406c800 0x7fddb40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb96ae640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb415f1e0 0x7fddb4154c80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c7d0 tx=0x7fddac00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb17fa640 1 -- 172.21.15.53:0/1884910996 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac018070 con 0x7fddb415f1e0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb17fa640 1 -- 172.21.15.53:0/1884910996 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac00a640 con 0x7fddb415f1e0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddb17fa640 1 -- 172.21.15.53:0/1884910996 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac00aa70 con 0x7fddb415f1e0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb40ca8f0 con 0x7fddb415f1e0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.555+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb40caeb0 con 0x7fddb415f1e0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.557+0000 7fddb17fa640 1 -- 172.21.15.53:0/1884910996 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fddac01b070 con 0x7fddb415f1e0 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.557+0000 7fddb17fa640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdda00762d0 0x7fdda0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.558+0000 7fddb17fa640 1 -- 172.21.15.53:0/1884910996 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7fddac096e50 con 0x7fddb415f1e0 2024-05-10T10:39:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.558+0000 7fddb8ead640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdda00762d0 0x7fdda0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.558+0000 7fddb8ead640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdda00762d0 0x7fdda0078790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdda4006fd0 tx=0x7fdda4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.570 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.558+0000 7fddbde21740 1 Processor -- start 2024-05-10T10:39:49.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.558+0000 7fddbde21740 1 -- start start 2024-05-10T10:39:49.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddbde21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586dd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586ddc00 0x560a586b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddbde21740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560a58616cf0 con 0x560a586de340 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560a585cfe10 con 0x560a586ddc00 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560a58612cb0 con 0x560a586fb220 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586dd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586dd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43550/0 (socket says 172.21.15.53:43550) 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 -- 172.21.15.53:0/3850750516 learned_addr learned my addr 172.21.15.53:0/3850750516 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb96ae640 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586ddc00 0x560a586b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb8ead640 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 -- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 msgr2=0x560a586adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 -- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586ddc00 msgr2=0x560a586b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586ddc00 0x560a586b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.559+0000 7fddb3fff640 1 -- 172.21.15.53:0/3850750516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a583f07e0 con 0x560a586de340 2024-05-10T10:39:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.560+0000 7fddb3fff640 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586dd6c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009890 tx=0x7fdda802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb000c01943cfd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.560+0000 7fdd9affd640 1 -- 172.21.15.53:0/3850750516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8037040 con 0x560a586de340 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.560+0000 7fdd9affd640 1 -- 172.21.15.53:0/3850750516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8043070 con 0x560a586de340 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.560+0000 7fdd9affd640 1 -- 172.21.15.53:0/3850750516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda803d870 con 0x560a586de340 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.560+0000 7fddbde21740 1 -- 172.21.15.53:0/3850750516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 msgr2=0x560a586dd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.560+0000 7fddbde21740 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586dd6c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009890 tx=0x7fdda802c000 comp rx=0 tx=0).stop 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.560+0000 7fddbde21740 1 -- 172.21.15.53:0/3850750516 shutdown_connections 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586ddc00 0x560a586b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586dd6c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 --2- 172.21.15.53:0/3850750516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 -- 172.21.15.53:0/3850750516 >> 172.21.15.53:0/3850750516 conn(0x560a586afe20 msgr2=0x560a586f86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 -- 172.21.15.53:0/3850750516 shutdown_connections 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 -- 172.21.15.53:0/3850750516 wait complete. 2024-05-10T10:39:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 -- start start 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.561+0000 7fddbde21740 1 Processor -- start 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddbde21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586cb1f0 0x560a586cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddbde21740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560a586b1fd0 con 0x560a586de340 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560a585d3940 con 0x560a586cb1f0 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560a586b13c0 con 0x560a586fb220 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddbde21740 10 client.0 _ll_register_callbacks cb 0x560a585d6440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43556/0 (socket says 172.21.15.53:43556) 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 -- 172.21.15.53:0/526531426 learned_addr learned my addr 172.21.15.53:0/526531426 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb3fff640 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb96ae640 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586cb1f0 0x560a586cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 -- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 msgr2=0x560a586f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 -- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586cb1f0 msgr2=0x560a586cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586cb1f0 0x560a586cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.562+0000 7fddb8ead640 1 -- 172.21.15.53:0/526531426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a58450f70 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddb8ead640 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586ef3e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fdda400ead0 tx=0x7fdda4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d87ea5a3637849c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fdd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fdd937fe640 1 -- 172.21.15.53:0/526531426 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4028060 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fdd937fe640 1 -- 172.21.15.53:0/526531426 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7fdda4023460 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda8043070 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fdd937fe640 1 -- 172.21.15.53:0/526531426 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fdda402e050 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 10 client.7062 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560a582e8170 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fdd937fe640 1 -- 172.21.15.53:0/526531426 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8043070 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fdd937fe640 1 -- 172.21.15.53:0/526531426 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fdda4031020 con 0x560a586de340 2024-05-10T10:39:49.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 10 client.7062 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 10 client.7062 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 20 client.7062 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586cbe60 0x560a586da010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 4 client.7062 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a58624900 con 0x560a586cbe60 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.563+0000 7fddb3fff640 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586cbe60 0x560a586da010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddb3fff640 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586cbe60 0x560a586da010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0249029a059aa5d server_cookie=96f4cb52e6d0a561 in_seq=0 out_seq=0 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fdd937fe640 10 client.7062 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fdd937fe640 1 -- 172.21.15.53:0/526531426 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a58624900 con 0x560a586cbe60 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fdd937fe640 10 client.7062 handle_command_reply: tid=0 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 client.7062 shutdown 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 msgr2=0x560a586ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586ef3e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fdda400ead0 tx=0x7fdda4004730 comp rx=0 tx=0).stop 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 shutdown_connections 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586cb1f0 0x560a586cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586cbe60 0x560a586da010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 --2- 172.21.15.53:0/526531426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586de340 0x560a586ef3e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fdd937fe640 0 client.7062 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.564+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 >> 172.21.15.53:0/526531426 conn(0x560a586ddc00 msgr2=0x560a586f86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.565+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 shutdown_connections 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.565+0000 7fddbde21740 1 -- 172.21.15.53:0/526531426 wait complete. 2024-05-10T10:39:49.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.565+0000 7fddbde21740 20 client.7062 trim_cache size 0 max 16384 2024-05-10T10:39:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 Processor -- start 2024-05-10T10:39:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 -- start start 2024-05-10T10:39:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586cb1f0 0x560a587127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586fb220 0x560a586aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4201e60 con 0x560a586fb220 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4200eb0 con 0x560a586de340 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.583+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40b5860 con 0x560a586cb1f0 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586fb220 0x560a586aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586fb220 0x560a586aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43570/0 (socket says 172.21.15.53:43570) 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 -- 172.21.15.53:0/2646745850 learned_addr learned my addr 172.21.15.53:0/2646745850 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb8ead640 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586cb1f0 0x560a587127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb3fff640 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 -- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586cb1f0 msgr2=0x560a587127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586cb1f0 0x560a587127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 -- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 msgr2=0x560a586f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 -- 172.21.15.53:0/2646745850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a586b0f60 con 0x560a586fb220 2024-05-10T10:39:49.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fddb96ae640 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586fb220 0x560a586aeae0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fddac05bee0 tx=0x7fddac05bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9d6838b1d86c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fdd9a7fc640 1 -- 172.21.15.53:0/2646745850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac068c30 con 0x560a586fb220 2024-05-10T10:39:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fdd9a7fc640 1 -- 172.21.15.53:0/2646745850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac0600c0 con 0x560a586fb220 2024-05-10T10:39:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.584+0000 7fdd9a7fc640 1 -- 172.21.15.53:0/2646745850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0687a0 con 0x560a586fb220 2024-05-10T10:39:49.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 -- 172.21.15.53:0/2646745850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586fb220 msgr2=0x560a586aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586fb220 0x560a586aeae0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fddac05bee0 tx=0x7fddac05bdb0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 -- 172.21.15.53:0/2646745850 shutdown_connections 2024-05-10T10:39:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586fb220 0x560a586aeae0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 --2- 172.21.15.53:0/2646745850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586cb1f0 0x560a587127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 -- 172.21.15.53:0/2646745850 >> 172.21.15.53:0/2646745850 conn(0x560a586ddc00 msgr2=0x560a586cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 -- 172.21.15.53:0/2646745850 shutdown_connections 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 -- 172.21.15.53:0/2646745850 wait complete. 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 -- start start 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.585+0000 7fddbde21740 1 Processor -- start 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddbde21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586cb1f0 0x560a586ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddbde21740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddbde21740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560a586f7c10 con 0x560a586cb1f0 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4200860 con 0x560a586de340 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddbde21740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560a586adaf0 con 0x560a586fb220 2024-05-10T10:39:49.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddbde21740 10 client.0 _ll_register_callbacks cb 0x7fdda4009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddb8ead640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586cb1f0 0x560a586ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddb8ead640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586cb1f0 0x560a586ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43572/0 (socket says 172.21.15.53:43572) 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddb8ead640 1 -- 172.21.15.53:0/909539335 learned_addr learned my addr 172.21.15.53:0/909539335 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddb96ae640 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.586+0000 7fddb3fff640 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddb8ead640 1 -- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 msgr2=0x560a586f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddb8ead640 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddb8ead640 1 -- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 msgr2=0x560a586f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddb8ead640 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddb8ead640 1 -- 172.21.15.53:0/909539335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a58776ef0 con 0x560a586cb1f0 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddb8ead640 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586cb1f0 0x560a586ad0c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdda400efd0 tx=0x7fdda4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e8761795a382bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fdd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fdd937fe640 1 -- 172.21.15.53:0/909539335 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda400f040 con 0x560a586cb1f0 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fdd937fe640 1 -- 172.21.15.53:0/909539335 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7fdda4023a20 con 0x560a586cb1f0 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddac0600c0 con 0x560a586cb1f0 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fdd937fe640 1 -- 172.21.15.53:0/909539335 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fdda4023c20 con 0x560a586cb1f0 2024-05-10T10:39:49.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 10 client.7068 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560a586ad600 con 0x560a586cb1f0 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fdd937fe640 1 -- 172.21.15.53:0/909539335 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac0600c0 con 0x560a586cb1f0 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fdd937fe640 1 -- 172.21.15.53:0/909539335 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fdda4031040 con 0x560a586cb1f0 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 10 client.7068 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 10 client.7068 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 20 client.7068 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586ca440 0x560a5876cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 4 client.7068 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-10T10:39:49.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.587+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x560a5861e770 con 0x560a586ca440 2024-05-10T10:39:49.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fddb3fff640 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586ca440 0x560a5876cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fddb3fff640 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586ca440 0x560a5876cd00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f9d0da1e183cdb server_cookie=2b0430a6941bf30d in_seq=0 out_seq=0 2024-05-10T10:39:49.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fdd937fe640 10 client.7068 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fdd937fe640 1 -- 172.21.15.53:0/909539335 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560a5861e770 con 0x560a586ca440 2024-05-10T10:39:49.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fdd937fe640 10 client.7068 handle_command_reply: tid=0 2024-05-10T10:39:49.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fddbde21740 1 client.7068 shutdown 2024-05-10T10:39:49.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586cb1f0 msgr2=0x560a586ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.588+0000 7fddbde21740 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586cb1f0 0x560a586ad0c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdda400efd0 tx=0x7fdda4012000 comp rx=0 tx=0).stop 2024-05-10T10:39:49.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 shutdown_connections 2024-05-10T10:39:49.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560a586fb220 0x560a586f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560a586de340 0x560a586f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x560a586ca440 0x560a5876cd00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 --2- 172.21.15.53:0/909539335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a586cb1f0 0x560a586ad0c0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 >> 172.21.15.53:0/909539335 conn(0x560a586ddc00 msgr2=0x560a586f1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fdd937fe640 0 client.7068 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 shutdown_connections 2024-05-10T10:39:49.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 1 -- 172.21.15.53:0/909539335 wait complete. 2024-05-10T10:39:49.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbde21740 20 client.7068 trim_cache size 0 max 16384 2024-05-10T10:39:49.613 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:39:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:39:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "1dab3b3f-1ebf-4ab6-a6c1-a79f21d40699", 2024-05-10T10:39:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:39:49.613 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:39:49.615 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:39:49.589+0000 7f54616a2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0)->f(v1 m2024-05-10T10:39:09.698395+0000) n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1)->n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x559c49899b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-10T10:39:49.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdda00762d0 msgr2=0x7fdda0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbb111640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdda00762d0 0x7fdda0078790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdda4006fd0 tx=0x7fdda4008040 comp rx=0 tx=0).stop 2024-05-10T10:39:49.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb415f1e0 msgr2=0x7fddb4154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.589+0000 7fddbb111640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb415f1e0 0x7fddb4154c80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c7d0 tx=0x7fddac00cc90 comp rx=0 tx=0).stop 2024-05-10T10:39:49.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 shutdown_connections 2024-05-10T10:39:49.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdda00762d0 0x7fdda0078790 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddb415f1e0 0x7fddb4154c80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fddb40d00f0 0x7fddb40c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 --2- 172.21.15.53:0/1884910996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fddb406c800 0x7fddb40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 >> 172.21.15.53:0/1884910996 conn(0x7fddb40d1ce0 msgr2=0x7fddb40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 shutdown_connections 2024-05-10T10:39:49.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.590+0000 7fddbb111640 1 -- 172.21.15.53:0/1884910996 wait complete. 2024-05-10T10:39:49.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:39:49.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-10T10:39:49.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.724+0000 7f60be3e4640 1 Processor -- start 2024-05-10T10:39:49.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.724+0000 7f60be3e4640 1 -- start start 2024-05-10T10:39:49.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60be3e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60b8160530 0x7f60b815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60be3e4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f60b815c9e0 0x7f60b80cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60be3e4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b815cfb0 con 0x7f60b806f030 2024-05-10T10:39:49.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60be3e4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f60b80ccb60 con 0x7f60b8160530 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60be3e4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f60b8160990 con 0x7f60b815c9e0 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60b77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60b8160530 0x7f60b815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60b7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f60b806f030 0x7f60b806f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49080/0 (socket says 172.21.15.53:49080) 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.725+0000 7f60b77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60b8160530 0x7f60b815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43584/0 (socket says 172.21.15.53:43584) 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.726+0000 7f60b7fff640 1 -- 172.21.15.53:0/4083434174 learned_addr learned my addr 172.21.15.53:0/4083434174 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.726+0000 7f60bc981640 1 --2- 172.21.15.53:0/4083434174 >> v2:172.21.15.135:3300/0 conn(0x7f60b815c9e0 0x7f60b80cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.726+0000 7f60b77fe640 1 -- 172.21.15.53:0/4083434174 >> v2:172.21.15.135:3300/0 conn(0x7f60b815c9e0 msgr2=0x7f60b80cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.726+0000 7f60b77fe640 1 --2- 172.21.15.53:0/4083434174 >> v2:172.21.15.135:3300/0 conn(0x7f60b815c9e0 0x7f60b80cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.726+0000 7f60b77fe640 1 -- 172.21.15.53:0/4083434174 >> v1:172.21.15.53:6789/0 conn(0x7f60b806f030 legacy=0x7f60b806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.726+0000 7f60b77fe640 1 -- 172.21.15.53:0/4083434174 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b815d290 con 0x7f60b8160530 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.726+0000 7f60b77fe640 1 --2- 172.21.15.53:0/4083434174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8160530 0x7f60b815c4a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f60b80d07e0 tx=0x7f60ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1313196ec9897c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60b6ffd640 1 -- 172.21.15.53:0/4083434174 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac037070 con 0x7f60b8160530 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60b6ffd640 1 -- 172.21.15.53:0/4083434174 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac032440 con 0x7f60b8160530 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60b6ffd640 1 -- 172.21.15.53:0/4083434174 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac032810 con 0x7f60b8160530 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 -- 172.21.15.53:0/4083434174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8160530 msgr2=0x7f60b815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 --2- 172.21.15.53:0/4083434174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8160530 0x7f60b815c4a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f60b80d07e0 tx=0x7f60ac02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 -- 172.21.15.53:0/4083434174 shutdown_connections 2024-05-10T10:39:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 --2- 172.21.15.53:0/4083434174 >> v2:172.21.15.135:3300/0 conn(0x7f60b815c9e0 0x7f60b80cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 --2- 172.21.15.53:0/4083434174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8160530 0x7f60b815c4a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 -- 172.21.15.53:0/4083434174 >> 172.21.15.53:0/4083434174 conn(0x7f60b80d0c90 msgr2=0x7f60b80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 -- 172.21.15.53:0/4083434174 shutdown_connections 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.727+0000 7f60be3e4640 1 -- 172.21.15.53:0/4083434174 wait complete. 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.728+0000 7f60be3e4640 1 Processor -- start 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60be3e4640 1 -- start start 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60be3e4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60b806f030 0x7f60b8158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60be3e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b815c9e0 0x7f60b80b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60be3e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b8160530 0x7f60b80b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60be3e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60b815fa40 con 0x7f60b815c9e0 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60be3e4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60b8160010 con 0x7f60b8160530 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60be3e4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f60b80cb910 con 0x7f60b806f030 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b815c9e0 0x7f60b80b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b815c9e0 0x7f60b80b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43594/0 (socket says 172.21.15.53:43594) 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.729+0000 7f60b77fe640 1 -- 172.21.15.53:0/1662764854 learned_addr learned my addr 172.21.15.53:0/1662764854 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60bc981640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b8160530 0x7f60b80b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b7fff640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60b806f030 0x7f60b8158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b77fe640 1 -- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60b806f030 msgr2=0x7f60b8158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b77fe640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60b806f030 0x7f60b8158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b77fe640 1 -- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b8160530 msgr2=0x7f60b80b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b77fe640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b8160530 0x7f60b80b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b77fe640 1 -- 172.21.15.53:0/1662764854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b80aebd0 con 0x7f60b815c9e0 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60bc981640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b8160530 0x7f60b80b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b7fff640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60b806f030 0x7f60b8158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b77fe640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b815c9e0 0x7f60b80b4240 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b4ff9640 1 -- 172.21.15.53:0/1662764854 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac037040 con 0x7f60b815c9e0 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b4ff9640 1 -- 172.21.15.53:0/1662764854 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac042070 con 0x7f60b815c9e0 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60b4ff9640 1 -- 172.21.15.53:0/1662764854 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac030a50 con 0x7f60b815c9e0 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b80aeeb0 con 0x7f60b815c9e0 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.730+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b80af470 con 0x7f60b815c9e0 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.732+0000 7f60b4ff9640 1 -- 172.21.15.53:0/1662764854 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f60ac030bf0 con 0x7f60b815c9e0 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.732+0000 7f60b4ff9640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f60a4076300 0x7f60a40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.733+0000 7f60b4ff9640 1 -- 172.21.15.53:0/1662764854 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f60ac0b8410 con 0x7f60b815c9e0 2024-05-10T10:39:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.733+0000 7f60b7fff640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f60a4076300 0x7f60a40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.733+0000 7f60c11da740 1 Processor -- start 2024-05-10T10:39:49.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.733+0000 7f60c11da740 1 -- start start 2024-05-10T10:39:49.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.733+0000 7f60b7fff640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f60a4076300 0x7f60a40787c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f60a8000f60 tx=0x7f60a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77e6160 0x55efc7798af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77c8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60c11da740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c8b40 0x55efc779d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60c11da740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55efc7765690 con 0x55efc77c8b40 2024-05-10T10:39:49.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55efc7702f30 con 0x55efc77c9280 2024-05-10T10:39:49.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55efc76bbb20 con 0x55efc77e6160 2024-05-10T10:39:49.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c8b40 0x55efc779d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c8b40 0x55efc779d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43598/0 (socket says 172.21.15.53:43598) 2024-05-10T10:39:49.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 -- 172.21.15.53:0/1637700087 learned_addr learned my addr 172.21.15.53:0/1637700087 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60b7fff640 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77e6160 0x55efc7798af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60b77fe640 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 -- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77e6160 msgr2=0x55efc7798af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77e6160 0x55efc7798af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 -- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 msgr2=0x55efc77c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60bc981640 1 -- 172.21.15.53:0/1637700087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc76b3e60 con 0x55efc77c8b40 2024-05-10T10:39:49.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.734+0000 7f60b77fe640 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:39:49.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60bc981640 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c8b40 0x55efc779d950 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f60b000d380 tx=0x7f60b000d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66cd8b08e5c508b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f609dffb640 1 -- 172.21.15.53:0/1637700087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0003da0 con 0x55efc77c8b40 2024-05-10T10:39:49.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f609dffb640 1 -- 172.21.15.53:0/1637700087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b0004030 con 0x55efc77c8b40 2024-05-10T10:39:49.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f609dffb640 1 -- 172.21.15.53:0/1637700087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0004420 con 0x55efc77c8b40 2024-05-10T10:39:49.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 -- 172.21.15.53:0/1637700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c8b40 msgr2=0x55efc779d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c8b40 0x55efc779d950 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f60b000d380 tx=0x7f60b000d840 comp rx=0 tx=0).stop 2024-05-10T10:39:49.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 -- 172.21.15.53:0/1637700087 shutdown_connections 2024-05-10T10:39:49.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c8b40 0x55efc779d950 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 --2- 172.21.15.53:0/1637700087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77e6160 0x55efc7798af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 -- 172.21.15.53:0/1637700087 >> 172.21.15.53:0/1637700087 conn(0x55efc779ad60 msgr2=0x55efc77e3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 -- 172.21.15.53:0/1637700087 shutdown_connections 2024-05-10T10:39:49.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.735+0000 7f60c11da740 1 -- 172.21.15.53:0/1637700087 wait complete. 2024-05-10T10:39:49.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.736+0000 7f60c11da740 1 -- start start 2024-05-10T10:39:49.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.736+0000 7f60c11da740 1 Processor -- start 2024-05-10T10:39:49.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60c11da740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77e6160 0x55efc77e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc77e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60c11da740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55efc76bf5e0 con 0x55efc77e6160 2024-05-10T10:39:49.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55efc76feef0 con 0x55efc77c9280 2024-05-10T10:39:49.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55efc779cf10 con 0x55efc77b8fe0 2024-05-10T10:39:49.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60c11da740 10 client.0 _ll_register_callbacks cb 0x55efc76ba320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:49.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77e6160 0x55efc77e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77e6160 0x55efc77e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43610/0 (socket says 172.21.15.53:43610) 2024-05-10T10:39:49.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 -- 172.21.15.53:0/3451503790 learned_addr learned my addr 172.21.15.53:0/3451503790 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60bc981640 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc77e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 -- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 msgr2=0x55efc77e4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc77e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 -- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 msgr2=0x55efc77b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b7fff640 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77b8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.737+0000 7f60b77fe640 1 -- 172.21.15.53:0/3451503790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc74de7d0 con 0x55efc77e6160 2024-05-10T10:39:49.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60b77fe640 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77e6160 0x55efc77e4650 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f60ac081310 tx=0x7f60ac081280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a676b30ddbf26257 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f6096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:49.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f6096ffd640 1 -- 172.21.15.53:0/3451503790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac09a030 con 0x55efc77e6160 2024-05-10T10:39:49.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f6096ffd640 1 -- 172.21.15.53:0/3451503790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f60ac0817e0 con 0x55efc77e6160 2024-05-10T10:39:49.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60b0004030 con 0x55efc77e6160 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f6096ffd640 1 -- 172.21.15.53:0/3451503790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f60ac081a80 con 0x55efc77e6160 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 10 client.7083 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55efc74143f0 con 0x55efc77e6160 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f6096ffd640 1 -- 172.21.15.53:0/3451503790 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b0004030 con 0x55efc77e6160 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f6096ffd640 1 -- 172.21.15.53:0/3451503790 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f60ac03a030 con 0x55efc77e6160 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 10 client.7083 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 10 client.7083 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 20 client.7083 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:49.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77fd860 0x55efc77ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 4 client.7083 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc7710b40 con 0x55efc77fd860 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.738+0000 7f60b7fff640 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77fd860 0x55efc77ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f6096ffd640 10 client.7083 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60b7fff640 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77fd860 0x55efc77ffc50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6181a4ed88b293 server_cookie=bf7b2687de538cea in_seq=0 out_seq=0 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f6096ffd640 1 -- 172.21.15.53:0/3451503790 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55efc7710b40 con 0x55efc77fd860 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f6096ffd640 10 client.7083 handle_command_reply: tid=0 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 client.7083 shutdown 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77e6160 msgr2=0x55efc77e4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77e6160 0x55efc77e4650 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f60ac081310 tx=0x7f60ac081280 comp rx=0 tx=0).stop 2024-05-10T10:39:49.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 shutdown_connections 2024-05-10T10:39:49.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc77e4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77fd860 0x55efc77ffc50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77e6160 0x55efc77e4650 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 --2- 172.21.15.53:0/3451503790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77c9280 0x55efc77b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 >> 172.21.15.53:0/3451503790 conn(0x55efc77c8b40 msgr2=0x55efc7799830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.739+0000 7f6096ffd640 0 client.7083 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.740+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 shutdown_connections 2024-05-10T10:39:49.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.740+0000 7f60c11da740 1 -- 172.21.15.53:0/3451503790 wait complete. 2024-05-10T10:39:49.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.740+0000 7f60c11da740 20 client.7083 trim_cache size 0 max 16384 2024-05-10T10:39:49.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 Processor -- start 2024-05-10T10:39:49.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 -- start start 2024-05-10T10:39:49.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc77e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc77da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 0x55efc77e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60b82021a0 con 0x55efc77c9280 2024-05-10T10:39:49.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60b8201200 con 0x55efc77e6160 2024-05-10T10:39:49.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.758+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f60b80b2900 con 0x55efc77b8fe0 2024-05-10T10:39:49.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc77da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc77da7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43624/0 (socket says 172.21.15.53:43624) 2024-05-10T10:39:49.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 -- 172.21.15.53:0/4222486438 learned_addr learned my addr 172.21.15.53:0/4222486438 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 -- 172.21.15.53:0/4222486438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 msgr2=0x55efc77e1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 --2- 172.21.15.53:0/4222486438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc77e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 -- 172.21.15.53:0/4222486438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 msgr2=0x55efc77e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 --2- 172.21.15.53:0/4222486438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 0x55efc77e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 -- 172.21.15.53:0/4222486438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc73d40f0 con 0x55efc77c9280 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f60b77fe640 1 --2- 172.21.15.53:0/4222486438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc77da7e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f60ac081080 tx=0x7f60ac049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cdca651411e9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f609d7fa640 1 -- 172.21.15.53:0/4222486438 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac09a040 con 0x55efc77c9280 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f609d7fa640 1 -- 172.21.15.53:0/4222486438 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac0b6320 con 0x55efc77c9280 2024-05-10T10:39:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.759+0000 7f609d7fa640 1 -- 172.21.15.53:0/4222486438 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac085780 con 0x55efc77c9280 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 -- 172.21.15.53:0/4222486438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 msgr2=0x55efc77da7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 --2- 172.21.15.53:0/4222486438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc77da7e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f60ac081080 tx=0x7f60ac049000 comp rx=0 tx=0).stop 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 -- 172.21.15.53:0/4222486438 shutdown_connections 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 --2- 172.21.15.53:0/4222486438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 0x55efc77e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 --2- 172.21.15.53:0/4222486438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc77da7e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 --2- 172.21.15.53:0/4222486438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc77e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 -- 172.21.15.53:0/4222486438 >> 172.21.15.53:0/4222486438 conn(0x55efc77c8b40 msgr2=0x55efc77e6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 -- 172.21.15.53:0/4222486438 shutdown_connections 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 -- 172.21.15.53:0/4222486438 wait complete. 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 -- start start 2024-05-10T10:39:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.760+0000 7f60c11da740 1 Processor -- start 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc7798710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60c11da740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc785f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60c11da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 0x55efc785fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60c11da740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55efc77c87e0 con 0x55efc77c9280 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60b8200be0 con 0x55efc77e6160 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60c11da740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55efc77e5490 con 0x55efc77b8fe0 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60c11da740 10 client.0 _ll_register_callbacks cb 0x7f60ac030ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc785f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc785f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43636/0 (socket says 172.21.15.53:43636) 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.761+0000 7f60b77fe640 1 -- 172.21.15.53:0/391555488 learned_addr learned my addr 172.21.15.53:0/391555488 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60b77fe640 1 -- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 msgr2=0x55efc7798710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60b7fff640 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc7798710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60bc981640 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 0x55efc785fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60b77fe640 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc7798710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60b77fe640 1 -- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 msgr2=0x55efc785fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60b77fe640 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 0x55efc785fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60b77fe640 1 -- 172.21.15.53:0/391555488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc77d9b40 con 0x55efc77c9280 2024-05-10T10:39:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60b77fe640 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc785f850 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f60ac07e5f0 tx=0x7f60ac0b6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d14f012dee7e072 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f6096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f6096ffd640 1 -- 172.21.15.53:0/391555488 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac09a040 con 0x55efc77c9280 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f6096ffd640 1 -- 172.21.15.53:0/391555488 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f60ac0807f0 con 0x55efc77c9280 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60ac0b6320 con 0x55efc77c9280 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f6096ffd640 1 -- 172.21.15.53:0/391555488 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f60ac090050 con 0x55efc77c9280 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 10 client.7089 fetch_fsmap learned FSMap version 68 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55efc77d9b40 con 0x55efc77c9280 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f6096ffd640 1 -- 172.21.15.53:0/391555488 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac0b6320 con 0x55efc77c9280 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f6096ffd640 1 -- 172.21.15.53:0/391555488 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f60ac093020 con 0x55efc77c9280 2024-05-10T10:39:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 10 client.7089 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 10 client.7089 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 20 client.7089 populate_metadata read hostname 'smithi053' 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77979a0 0x55efc77c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 4 client.7089 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.762+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efc770a9b0 con 0x55efc77979a0 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60b7fff640 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77979a0 0x55efc77c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f6096ffd640 10 client.7089 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60b7fff640 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77979a0 0x55efc77c4570 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f396f6ecaa0231dc server_cookie=3be7733ac0df40cd in_seq=0 out_seq=0 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f6096ffd640 1 -- 172.21.15.53:0/391555488 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55efc770a9b0 con 0x55efc77979a0 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f6096ffd640 10 client.7089 handle_command_reply: tid=0 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 client.7089 shutdown 2024-05-10T10:39:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 msgr2=0x55efc785f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc785f850 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f60ac07e5f0 tx=0x7f60ac0b6520 comp rx=0 tx=0).stop 2024-05-10T10:39:49.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 shutdown_connections 2024-05-10T10:39:49.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55efc77e6160 0x55efc785fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55efc77979a0 0x55efc77c4570 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55efc77c9280 0x55efc785f850 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 --2- 172.21.15.53:0/391555488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55efc77b8fe0 0x55efc7798710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f6096ffd640 0 client.7089 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:39:49.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 >> 172.21.15.53:0/391555488 conn(0x55efc77c8b40 msgr2=0x55efc77da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 shutdown_connections 2024-05-10T10:39:49.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.763+0000 7f60c11da740 1 -- 172.21.15.53:0/391555488 wait complete. 2024-05-10T10:39:49.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60c11da740 20 client.7089 trim_cache size 0 max 16384 2024-05-10T10:39:49.786 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:39:49.786 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout: "1dab3b3f-1ebf-4ab6-a6c1-a79f21d40699": { 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/dir", 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "1dab3b3f-1ebf-4ab6-a6c1-a79f21d40699", 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair" 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:39:49.787 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:39:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f60a4076300 msgr2=0x7f60a40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f60a4076300 0x7f60a40787c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f60a8000f60 tx=0x7f60a8009040 comp rx=0 tx=0).stop 2024-05-10T10:39:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b815c9e0 msgr2=0x7f60b80b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:39:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b815c9e0 0x7f60b80b4240 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac0046c0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 shutdown_connections 2024-05-10T10:39:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f60a4076300 0x7f60a40787c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b8160530 0x7f60b80b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b815c9e0 0x7f60b80b4240 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 --2- 172.21.15.53:0/1662764854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60b806f030 0x7f60b8158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:39:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 >> 172.21.15.53:0/1662764854 conn(0x7f60b80d0c90 msgr2=0x7f60b815b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:39:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 shutdown_connections 2024-05-10T10:39:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:39:49.764+0000 7f60be3e4640 1 -- 172.21.15.53:0/1662764854 wait complete. 2024-05-10T10:39:49.790 INFO:tasks.cephfs.filesystem:scrub status for tag:1dab3b3f-1ebf-4ab6-a6c1-a79f21d40699 - {'path': '/dir', 'tag': '1dab3b3f-1ebf-4ab6-a6c1-a79f21d40699', 'options': 'recursive,repair'} 2024-05-10T10:40:00.001 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:39:59.999+0000 7f39f976c640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-10T10:40:15.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:40:15.014 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:40:19.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:19.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-10T10:40:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.910+0000 7fd221828640 1 Processor -- start 2024-05-10T10:40:19.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.910+0000 7fd221828640 1 -- start start 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.911+0000 7fd221828640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06df20 0x7fd21c0739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.911+0000 7fd221828640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c157a40 con 0x7fd21c073f20 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.911+0000 7fd221828640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c160d80 con 0x7fd21c06c2c0 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.911+0000 7fd221828640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c161990 con 0x7fd21c06df20 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.911+0000 7fd21b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd21c073f20 0x7fd21c15d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37546/0 (socket says 172.21.15.53:37546) 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.911+0000 7fd21b7fe640 1 -- 172.21.15.53:0/891952677 learned_addr learned my addr 172.21.15.53:0/891952677 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/891952677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06df20 0x7fd21c0739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030309424 0 0) 0x7fd20c001110 con 0x7fd21c073f20 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd208003520 con 0x7fd21c073f20 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453601736 0 0) 0x7fd21c160d80 con 0x7fd21c06c2c0 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd20c001110 con 0x7fd21c06c2c0 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd21a7fc640 1 -- 172.21.15.53:0/891952677 >> v1:172.21.15.135:6789/0 conn(0x7fd21c06c2c0 legacy=0x7fd21c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd21a7fc640 1 -- 172.21.15.53:0/891952677 >> v1:172.21.15.53:6789/0 conn(0x7fd21c073f20 legacy=0x7fd21c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 469626477 0 0) 0x7fd208003520 con 0x7fd21c073f20 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.912+0000 7fd21a7fc640 1 -- 172.21.15.53:0/891952677 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c161060 con 0x7fd21c06df20 2024-05-10T10:40:19.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.913+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/891952677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06df20 0x7fd21c0739e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd2040097d0 tx=0x7fd20402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c30d53a233af0fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:19.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.913+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204037070 con 0x7fd21c06df20 2024-05-10T10:40:19.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.913+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20402ac30 con 0x7fd21c06df20 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.913+0000 7fd219ffb640 1 -- 172.21.15.53:0/891952677 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c001840 con 0x7fd21c06df20 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.914+0000 7fd221828640 1 -- 172.21.15.53:0/891952677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06df20 msgr2=0x7fd21c0739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.914+0000 7fd221828640 1 --2- 172.21.15.53:0/891952677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06df20 0x7fd21c0739e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd2040097d0 tx=0x7fd20402a1a0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.914+0000 7fd221828640 1 -- 172.21.15.53:0/891952677 shutdown_connections 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.914+0000 7fd221828640 1 --2- 172.21.15.53:0/891952677 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06df20 0x7fd21c0739e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.914+0000 7fd221828640 1 -- 172.21.15.53:0/891952677 >> 172.21.15.53:0/891952677 conn(0x7fd21c0d0c90 msgr2=0x7fd21c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.914+0000 7fd221828640 1 -- 172.21.15.53:0/891952677 shutdown_connections 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.914+0000 7fd221828640 1 -- 172.21.15.53:0/891952677 wait complete. 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.915+0000 7fd221828640 1 Processor -- start 2024-05-10T10:40:19.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.915+0000 7fd221828640 1 -- start start 2024-05-10T10:40:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd221828640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06c2c0 0x7fd21c146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd221828640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd21c06df20 0x7fd21c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd221828640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c073f20 0x7fd21c15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd221828640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0cd440 con 0x7fd21c06df20 2024-05-10T10:40:19.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd221828640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c161620 con 0x7fd21c073f20 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd221828640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c072180 con 0x7fd21c06c2c0 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd21c06df20 0x7fd21c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd21c06df20 0x7fd21c0c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41128/0 (socket says 172.21.15.53:41128) 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 -- 172.21.15.53:0/138380014 learned_addr learned my addr 172.21.15.53:0/138380014 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21affd640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06c2c0 0x7fd21c146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c073f20 0x7fd21c15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 -- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06c2c0 msgr2=0x7fd21c146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06c2c0 0x7fd21c146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 -- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c073f20 msgr2=0x7fd21c15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c073f20 0x7fd21c15ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.916+0000 7fd21a7fc640 1 -- 172.21.15.53:0/138380014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c157fa0 con 0x7fd21c06df20 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.917+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd21c06df20 0x7fd21c0c5060 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd2040097d0 tx=0x7fd2040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.917+0000 7fd213fff640 1 -- 172.21.15.53:0/138380014 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204041030 con 0x7fd21c06df20 2024-05-10T10:40:19.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.917+0000 7fd213fff640 1 -- 172.21.15.53:0/138380014 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20403d420 con 0x7fd21c06df20 2024-05-10T10:40:19.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.917+0000 7fd213fff640 1 -- 172.21.15.53:0/138380014 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204037040 con 0x7fd21c06df20 2024-05-10T10:40:19.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.917+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c158280 con 0x7fd21c06df20 2024-05-10T10:40:19.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.917+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1473b0 con 0x7fd21c06df20 2024-05-10T10:40:19.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.919+0000 7fd213fff640 1 -- 172.21.15.53:0/138380014 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd204044070 con 0x7fd21c06df20 2024-05-10T10:40:19.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.919+0000 7fd213fff640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1fc076300 0x7fd1fc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.919+0000 7fd213fff640 1 -- 172.21.15.53:0/138380014 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7fd2040b7b10 con 0x7fd21c06df20 2024-05-10T10:40:19.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.920+0000 7fd21affd640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1fc076300 0x7fd1fc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.920+0000 7fd224617740 1 Processor -- start 2024-05-10T10:40:19.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.920+0000 7fd224617740 1 -- start start 2024-05-10T10:40:19.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.920+0000 7fd21affd640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1fc076300 0x7fd1fc0787c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd214001560 tx=0x7fd214009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:19.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a65250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a39160 0x557d25a61830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd224617740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a64600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd224617740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557d25a01690 con 0x557d25a61d70 2024-05-10T10:40:19.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557d2599ef30 con 0x557d25a61450 2024-05-10T10:40:19.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557d25957b20 con 0x557d25a39160 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a64600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a64600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41134/0 (socket says 172.21.15.53:41134) 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1593230260 learned_addr learned my addr 172.21.15.53:0/1593230260 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd21affd640 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a65250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.921+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a39160 0x557d25a61830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a39160 msgr2=0x557d25a61830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a39160 0x557d25a61830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 msgr2=0x557d25a65250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a65250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1593230260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d2594fe60 con 0x557d25a61d70 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd21affd640 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a65250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a64600 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd20c006d70 tx=0x7fd20c001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05b96dad8af74fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:19.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd2117fa640 1 -- 172.21.15.53:0/1593230260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c019070 con 0x557d25a61d70 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd2117fa640 1 -- 172.21.15.53:0/1593230260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c0058d0 con 0x557d25a61d70 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.922+0000 7fd2117fa640 1 -- 172.21.15.53:0/1593230260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c005c80 con 0x557d25a61d70 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 -- 172.21.15.53:0/1593230260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 msgr2=0x557d25a64600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a64600 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd20c006d70 tx=0x7fd20c001d20 comp rx=0 tx=0).stop 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 -- 172.21.15.53:0/1593230260 shutdown_connections 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a64600 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a39160 0x557d25a61830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 --2- 172.21.15.53:0/1593230260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a65250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 -- 172.21.15.53:0/1593230260 >> 172.21.15.53:0/1593230260 conn(0x557d25a36d60 msgr2=0x557d25a7f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 -- 172.21.15.53:0/1593230260 shutdown_connections 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 -- 172.21.15.53:0/1593230260 wait complete. 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 -- start start 2024-05-10T10:40:19.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.923+0000 7fd224617740 1 Processor -- start 2024-05-10T10:40:19.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a67d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a5d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd224617740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a65470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd224617740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557d2595b5e0 con 0x557d25a5d6f0 2024-05-10T10:40:19.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557d2599aef0 con 0x557d25a61d70 2024-05-10T10:40:19.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557d25a350b0 con 0x557d25a61450 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd224617740 10 client.0 _ll_register_callbacks cb 0x557d25956320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a65470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a65470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41138/0 (socket says 172.21.15.53:41138) 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1441735908 learned_addr learned my addr 172.21.15.53:0/1441735908 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a5d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.924+0000 7fd21affd640 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a67d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 msgr2=0x557d25a67d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a67d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 msgr2=0x557d25a5d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a5d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd21b7fe640 1 -- 172.21.15.53:0/1441735908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d2577a7d0 con 0x557d25a5d6f0 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a65470 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd20c006c60 tx=0x7fd20c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ecf4836f9e4c8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:19.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd1fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/1441735908 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c019070 con 0x557d25a5d6f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/1441735908 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7fd20c006580 con 0x557d25a5d6f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c0058d0 con 0x557d25a5d6f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/1441735908 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fd20c02c050 con 0x557d25a5d6f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd224617740 10 client.7101 fetch_fsmap learned FSMap version 68 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x557d256b03f0 con 0x557d25a5d6f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/1441735908 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c0058d0 con 0x557d25a5d6f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.925+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/1441735908 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fd20c02f020 con 0x557d25a5d6f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd224617740 10 client.7101 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd224617740 10 client.7101 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd224617740 20 client.7101 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd224617740 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25a4b3f0 0x557d25a4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd224617740 4 client.7101 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d259acb40 con 0x557d25a4b3f0 2024-05-10T10:40:19.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd21affd640 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25a4b3f0 0x557d25a4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd1fa7fc640 10 client.7101 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.926+0000 7fd21affd640 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25a4b3f0 0x557d25a4b800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4686481699b5a8f1 server_cookie=d5c48a0ddd787dcf in_seq=0 out_seq=0 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/1441735908 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d259acb40 con 0x557d25a4b3f0 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd1fa7fc640 10 client.7101 handle_command_reply: tid=0 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 client.7101 shutdown 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 msgr2=0x557d25a65470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a65470 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd20c006c60 tx=0x7fd20c0057f0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 shutdown_connections 2024-05-10T10:40:19.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25a4b3f0 0x557d25a4b800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a65470 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a5d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd1fa7fc640 0 client.7101 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 --2- 172.21.15.53:0/1441735908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a67d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 >> 172.21.15.53:0/1441735908 conn(0x557d25a39160 msgr2=0x557d25a339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 shutdown_connections 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 1 -- 172.21.15.53:0/1441735908 wait complete. 2024-05-10T10:40:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.927+0000 7fd224617740 20 client.7101 trim_cache size 0 max 16384 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.945+0000 7fd224617740 1 Processor -- start 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.945+0000 7fd224617740 1 -- start start 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a5d6f0 0x557d25a34c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a7f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd224617740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a33c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd224617740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1fdb60 con 0x557d25a61d70 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0d00c0 con 0x557d25a61450 2024-05-10T10:40:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c0b5ea0 con 0x557d25a5d6f0 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a33c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd21b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a33c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41144/0 (socket says 172.21.15.53:41144) 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd21b7fe640 1 -- 172.21.15.53:0/3663637759 learned_addr learned my addr 172.21.15.53:0/3663637759 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd21affd640 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a5d6f0 0x557d25a34c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.946+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a7f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd21b7fe640 1 -- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a5d6f0 msgr2=0x557d25a34c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a5d6f0 0x557d25a34c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd21b7fe640 1 -- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 msgr2=0x557d25a7f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a7f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd21b7fe640 1 -- 172.21.15.53:0/3663637759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d256700f0 con 0x557d25a61d70 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a33c90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a040 tx=0x7fd20c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf2868ceef8e054 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd210ff9640 1 -- 172.21.15.53:0/3663637759 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c035070 con 0x557d25a61d70 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd210ff9640 1 -- 172.21.15.53:0/3663637759 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c00b040 con 0x557d25a61d70 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd210ff9640 1 -- 172.21.15.53:0/3663637759 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c0058d0 con 0x557d25a61d70 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd224617740 1 -- 172.21.15.53:0/3663637759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 msgr2=0x557d25a33c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.947+0000 7fd224617740 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a33c90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a040 tx=0x7fd20c019000 comp rx=0 tx=0).stop 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 -- 172.21.15.53:0/3663637759 shutdown_connections 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a61d70 0x557d25a33c90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61450 0x557d25a7f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 --2- 172.21.15.53:0/3663637759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a5d6f0 0x557d25a34c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 -- 172.21.15.53:0/3663637759 >> 172.21.15.53:0/3663637759 conn(0x557d25a39160 msgr2=0x557d25a48430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:19.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 -- 172.21.15.53:0/3663637759 shutdown_connections 2024-05-10T10:40:19.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 -- 172.21.15.53:0/3663637759 wait complete. 2024-05-10T10:40:19.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 -- start start 2024-05-10T10:40:19.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.948+0000 7fd224617740 1 Processor -- start 2024-05-10T10:40:19.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd224617740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd224617740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd224617740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557d25a48c40 con 0x557d25a5d6f0 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0cfa70 con 0x557d25a61d70 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd224617740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557d2595d990 con 0x557d25a61450 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd224617740 10 client.0 _ll_register_callbacks cb 0x7fd20c017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a33fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41146/0 (socket says 172.21.15.53:41146) 2024-05-10T10:40:19.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 -- 172.21.15.53:0/2684874075 learned_addr learned my addr 172.21.15.53:0/2684874075 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21b7fe640 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 -- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 msgr2=0x557d25a49130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a49130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 -- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 msgr2=0x557d25a49670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a49670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.949+0000 7fd21affd640 1 -- 172.21.15.53:0/2684874075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d25a34510 con 0x557d25a5d6f0 2024-05-10T10:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd21affd640 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a33fd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x557d25a4be00 tx=0x7fd214001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205156bef6e73303 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd1fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/2684874075 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214029590 con 0x557d25a5d6f0 2024-05-10T10:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/2684874075 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7fd214029730 con 0x557d25a5d6f0 2024-05-10T10:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c00b040 con 0x557d25a5d6f0 2024-05-10T10:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/2684874075 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fd21403a050 con 0x557d25a5d6f0 2024-05-10T10:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 10 client.7107 fetch_fsmap learned FSMap version 68 2024-05-10T10:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x557d25aa6f00 con 0x557d25a5d6f0 2024-05-10T10:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/2684874075 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c00b040 con 0x557d25a5d6f0 2024-05-10T10:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/2684874075 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fd214020040 con 0x557d25a5d6f0 2024-05-10T10:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 10 client.7107 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 10 client.7107 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:19.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 20 client.7107 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:19.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25b009b0 0x557d25a92730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:19.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 4 client.7107 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d25a35ed0 con 0x557d25b009b0 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.950+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25b009b0 0x557d25a92730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd1fa7fc640 10 client.7107 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd21a7fc640 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25b009b0 0x557d25a92730 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b692b1cbd78b857 server_cookie=780823b25ca06c77 in_seq=0 out_seq=0 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd1fa7fc640 1 -- 172.21.15.53:0/2684874075 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557d25a35ed0 con 0x557d25b009b0 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd1fa7fc640 10 client.7107 handle_command_reply: tid=0 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 client.7107 shutdown 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 msgr2=0x557d25a33fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a33fd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x557d25a4be00 tx=0x7fd214001250 comp rx=0 tx=0).stop 2024-05-10T10:40:19.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 shutdown_connections 2024-05-10T10:40:19.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d25a61d70 0x557d25a49670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557d25a61450 0x557d25a49130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x557d25b009b0 0x557d25a92730 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 --2- 172.21.15.53:0/2684874075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557d25a5d6f0 0x557d25a33fd0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd1fa7fc640 0 client.7107 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:19.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 >> 172.21.15.53:0/2684874075 conn(0x557d25a39160 msgr2=0x557d25a7f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:19.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 shutdown_connections 2024-05-10T10:40:19.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 1 -- 172.21.15.53:0/2684874075 wait complete. 2024-05-10T10:40:19.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.951+0000 7fd224617740 20 client.7107 trim_cache size 0 max 16384 2024-05-10T10:40:19.967 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:40:19.968 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:40:19.968 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:40:19.968 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:40:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1fc076300 msgr2=0x7fd1fc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1fc076300 0x7fd1fc0787c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd214001560 tx=0x7fd214009040 comp rx=0 tx=0).stop 2024-05-10T10:40:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd21c06df20 msgr2=0x7fd21c0c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd21c06df20 0x7fd21c0c5060 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd2040097d0 tx=0x7fd2040046c0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 shutdown_connections 2024-05-10T10:40:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd1fc076300 0x7fd1fc0787c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c073f20 0x7fd21c15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd21c06df20 0x7fd21c0c5060 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 --2- 172.21.15.53:0/138380014 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd21c06c2c0 0x7fd21c146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 >> 172.21.15.53:0/138380014 conn(0x7fd21c0d0c90 msgr2=0x7fd21c0b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 shutdown_connections 2024-05-10T10:40:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:19.952+0000 7fd221828640 1 -- 172.21.15.53:0/138380014 wait complete. 2024-05-10T10:40:19.972 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:40:19.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:40:20.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.076+0000 7f9266b7f640 1 Processor -- start 2024-05-10T10:40:20.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.076+0000 7f9266b7f640 1 -- start start 2024-05-10T10:40:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.076+0000 7f9266b7f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f926015d290 0x7f9260071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.076+0000 7f9266b7f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9260072330 con 0x7f92600d3510 2024-05-10T10:40:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.076+0000 7f9266b7f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9260071d70 con 0x7f9260071880 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.076+0000 7f9266b7f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9260072050 con 0x7f926015d290 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f926511c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f92600d3510 0x7f92600ce270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37560/0 (socket says 172.21.15.53:37560) 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f926511c640 1 -- 172.21.15.53:0/1255244114 learned_addr learned my addr 172.21.15.53:0/1255244114 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f926491b640 1 --2- 172.21.15.53:0/1255244114 >> v2:172.21.15.135:3300/0 conn(0x7f926015d290 0x7f9260071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 465282150 0 0) 0x7f9260072330 con 0x7f92600d3510 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f924c003520 con 0x7f92600d3510 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3493153603 0 0) 0x7f924c003520 con 0x7f92600d3510 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 >> v2:172.21.15.135:3300/0 conn(0x7f926015d290 msgr2=0x7f9260071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 --2- 172.21.15.53:0/1255244114 >> v2:172.21.15.135:3300/0 conn(0x7f926015d290 0x7f9260071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 >> v1:172.21.15.135:6789/0 conn(0x7f9260071880 legacy=0x7f92600d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92600cda70 con 0x7f92600d3510 2024-05-10T10:40:20.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f9248001670 con 0x7f92600d3510 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 >> v1:172.21.15.53:6789/0 conn(0x7f92600d3510 legacy=0x7f92600ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f924c003a00 0x7f924c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c004800 0x7f924c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f924c0051c0 0x7f924c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9260072050 con 0x7f924c003a00 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f924c00c0c0 con 0x7f924c004800 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f924c00c2c0 con 0x7f924c0051c0 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f926491b640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f924c003a00 0x7f924c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.077+0000 7f925ffff640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c004800 0x7f924c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926511c640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f924c0051c0 0x7f924c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926491b640 1 -- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f924c0051c0 msgr2=0x7f924c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926491b640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f924c0051c0 0x7f924c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926491b640 1 -- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c004800 msgr2=0x7f924c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926491b640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c004800 0x7f924c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926491b640 1 -- 172.21.15.53:0/1255244114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c003520 con 0x7f924c003a00 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926511c640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f924c0051c0 0x7f924c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926511c640 1 -- 172.21.15.53:0/1255244114 reap_dead start 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.078+0000 7f926491b640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f924c003a00 0x7f924c003e20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9258009ac0 tx=0x7f924c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffed80780368a4fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258036070 con 0x7f924c003a00 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9258031400 con 0x7f924c003a00 2024-05-10T10:40:20.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f925f7fe640 1 -- 172.21.15.53:0/1255244114 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258009690 con 0x7f924c003a00 2024-05-10T10:40:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f9266b7f640 1 -- 172.21.15.53:0/1255244114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f924c003a00 msgr2=0x7f924c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f9266b7f640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f924c003a00 0x7f924c003e20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9258009ac0 tx=0x7f924c005830 comp rx=0 tx=0).stop 2024-05-10T10:40:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f9266b7f640 1 -- 172.21.15.53:0/1255244114 shutdown_connections 2024-05-10T10:40:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f9266b7f640 1 --2- 172.21.15.53:0/1255244114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f924c003a00 0x7f924c003e20 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f9266b7f640 1 -- 172.21.15.53:0/1255244114 >> 172.21.15.53:0/1255244114 conn(0x7f926009ba20 msgr2=0x7f926009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f9266b7f640 1 -- 172.21.15.53:0/1255244114 shutdown_connections 2024-05-10T10:40:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.079+0000 7f9266b7f640 1 -- 172.21.15.53:0/1255244114 wait complete. 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.080+0000 7f9266b7f640 1 Processor -- start 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.080+0000 7f9266b7f640 1 -- start start 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f9266b7f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c003a00 0x7f926015ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f9266b7f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f926014cd70 0x7f9260160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f9266b7f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92600cc4b0 0x7f92600cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f9266b7f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f92600d1c20 con 0x7f926014cd70 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f9266b7f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92600d1aa0 con 0x7f924c003a00 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f9266b7f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f92600cc020 con 0x7f92600cc4b0 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f925ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f926014cd70 0x7f9260160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f925ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f926014cd70 0x7f9260160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41166/0 (socket says 172.21.15.53:41166) 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.081+0000 7f925ffff640 1 -- 172.21.15.53:0/937985859 learned_addr learned my addr 172.21.15.53:0/937985859 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925ffff640 1 -- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92600cc4b0 msgr2=0x7f92600cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925ffff640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92600cc4b0 0x7f92600cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925ffff640 1 -- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c003a00 msgr2=0x7f926015ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f926491b640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c003a00 0x7f926015ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925ffff640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c003a00 0x7f926015ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925ffff640 1 -- 172.21.15.53:0/937985859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92600cb6f0 con 0x7f926014cd70 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925ffff640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f926014cd70 0x7f9260160440 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f925000fae0 tx=0x7f92500020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925d7fa640 1 -- 172.21.15.53:0/937985859 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250003b80 con 0x7f926014cd70 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925d7fa640 1 -- 172.21.15.53:0/937985859 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9250018070 con 0x7f926014cd70 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92600cb9d0 con 0x7f926014cd70 2024-05-10T10:40:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f925d7fa640 1 -- 172.21.15.53:0/937985859 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92500050d0 con 0x7f926014cd70 2024-05-10T10:40:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.082+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92600cb3e0 con 0x7f926014cd70 2024-05-10T10:40:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.083+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9260075c70 con 0x7f926014cd70 2024-05-10T10:40:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.084+0000 7f925d7fa640 1 -- 172.21.15.53:0/937985859 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9250005270 con 0x7f926014cd70 2024-05-10T10:40:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.084+0000 7f925d7fa640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9240076300 0x7f92400787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.084+0000 7f925d7fa640 1 -- 172.21.15.53:0/937985859 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f9250097870 con 0x7f926014cd70 2024-05-10T10:40:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.084+0000 7f926491b640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9240076300 0x7f92400787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.085+0000 7f926491b640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9240076300 0x7f92400787c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9258033000 tx=0x7f9258008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.089+0000 7f925d7fa640 1 -- 172.21.15.53:0/937985859 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9260075c70 con 0x7f926014cd70 2024-05-10T10:40:20.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.319+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92600cc1a0 con 0x7f926014cd70 2024-05-10T10:40:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.320+0000 7f925d7fa640 1 -- 172.21.15.53:0/937985859 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5184 (secure 0 0 0) 0x7f9250060f80 con 0x7f926014cd70 2024-05-10T10:40:20.320 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:20.320 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:22.393011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6833/285942496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":285942496},{"type":"v1","addr":"172.21.15.53:6833","nonce":285942496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6888,"qdb_cluster":[6888]},"id":6}]} 2024-05-10T10:40:20.321 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 68 2024-05-10T10:40:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9240076300 msgr2=0x7f92400787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9240076300 0x7f92400787c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9258033000 tx=0x7f9258008030 comp rx=0 tx=0).stop 2024-05-10T10:40:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f926014cd70 msgr2=0x7f9260160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f926014cd70 0x7f9260160440 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f925000fae0 tx=0x7f92500020b0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 shutdown_connections 2024-05-10T10:40:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9240076300 0x7f92400787c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92600cc4b0 0x7f92600cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f926014cd70 0x7f9260160440 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 --2- 172.21.15.53:0/937985859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f924c003a00 0x7f926015ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 >> 172.21.15.53:0/937985859 conn(0x7f926009ba20 msgr2=0x7f92600b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 shutdown_connections 2024-05-10T10:40:20.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.322+0000 7f9266b7f640 1 -- 172.21.15.53:0/937985859 wait complete. 2024-05-10T10:40:20.334 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-10T10:40:20.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.440+0000 7f194e971640 1 Processor -- start 2024-05-10T10:40:20.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.440+0000 7f194e971640 1 -- start start 2024-05-10T10:40:20.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f194e971640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1948162900 0x7f19480d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f194e971640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 0x7f19480b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f194e971640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19480d4190 con 0x7f19480d3950 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f194e971640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f19480cfc40 con 0x7f1948162900 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f194e971640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f19480d3eb0 con 0x7f1948162ce0 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f194cf0e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f19480d3950 0x7f19480d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37574/0 (socket says 172.21.15.53:37574) 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f194cf0e640 1 -- 172.21.15.53:0/1309632696 learned_addr learned my addr 172.21.15.53:0/1309632696 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1947fff640 1 --2- 172.21.15.53:0/1309632696 >> v2:172.21.15.135:3300/0 conn(0x7f1948162900 0x7f19480d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f19477fe640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 0x7f19480b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700197512 0 0) 0x7f1940001490 con 0x7f19480d3950 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1930003520 con 0x7f19480d3950 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 990932912 0 0) 0x7f1930003520 con 0x7f19480d3950 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 msgr2=0x7f19480b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 0x7f19480b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 >> v2:172.21.15.135:3300/0 conn(0x7f1948162900 msgr2=0x7f19480d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 --2- 172.21.15.53:0/1309632696 >> v2:172.21.15.135:3300/0 conn(0x7f1948162900 0x7f19480d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19480cfee0 con 0x7f19480d3950 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f1940001bc0 con 0x7f19480d3950 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 >> v1:172.21.15.53:6789/0 conn(0x7f19480d3950 legacy=0x7f19480d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.441+0000 7f1946ffd640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19300039b0 0x7f1930003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f1946ffd640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f1930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f1946ffd640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 0x7f19300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19480cfc40 con 0x7f1930004800 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f19480d3eb0 con 0x7f19300039b0 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f193000b9c0 con 0x7f1948162ce0 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f19477fe640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f1930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f1947fff640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19300039b0 0x7f1930003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f194cf0e640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 0x7f19300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f19477fe640 1 -- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 msgr2=0x7f19300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f19477fe640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1948162ce0 0x7f19300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f19477fe640 1 -- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19300039b0 msgr2=0x7f1930003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f19477fe640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19300039b0 0x7f1930003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.442+0000 7f19477fe640 1 -- 172.21.15.53:0/1309632696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1930003520 con 0x7f1930004800 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f19477fe640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f1930004c80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f193c009af0 tx=0x7f1930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daabfe62b8e7a269 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c036070 con 0x7f1930004800 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c031400 con 0x7f1930004800 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f1946ffd640 1 -- 172.21.15.53:0/1309632696 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c009690 con 0x7f1930004800 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f194e971640 1 -- 172.21.15.53:0/1309632696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 msgr2=0x7f1930004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f194e971640 1 --2- 172.21.15.53:0/1309632696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f1930004c80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f193c009af0 tx=0x7f1930005830 comp rx=0 tx=0).stop 2024-05-10T10:40:20.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f194cf0e640 1 -- 172.21.15.53:0/1309632696 reap_dead start 2024-05-10T10:40:20.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f194e971640 1 -- 172.21.15.53:0/1309632696 shutdown_connections 2024-05-10T10:40:20.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f194e971640 1 -- 172.21.15.53:0/1309632696 >> 172.21.15.53:0/1309632696 conn(0x7f194809e430 msgr2=0x7f19480d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.443+0000 7f194e971640 1 -- 172.21.15.53:0/1309632696 shutdown_connections 2024-05-10T10:40:20.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.444+0000 7f194e971640 1 -- 172.21.15.53:0/1309632696 wait complete. 2024-05-10T10:40:20.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.445+0000 7f194e971640 1 Processor -- start 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.445+0000 7f194e971640 1 -- start start 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f194815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19480b5780 0x7f19480b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19480cdae0 0x7f19480cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19480b4300 con 0x7f1930004800 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f19480b4180 con 0x7f19480b5780 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f19480d10d0 con 0x7f19480cdae0 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f194815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f194815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41178/0 (socket says 172.21.15.53:41178) 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 -- 172.21.15.53:0/1946847396 learned_addr learned my addr 172.21.15.53:0/1946847396 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194cf0e640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19480cdae0 0x7f19480cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f19477fe640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19480b5780 0x7f19480b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 -- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19480cdae0 msgr2=0x7f19480cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19480cdae0 0x7f19480cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 -- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19480b5780 msgr2=0x7f19480b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19480b5780 0x7f19480b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 -- 172.21.15.53:0/1946847396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19480cc400 con 0x7f1930004800 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f19477fe640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19480b5780 0x7f19480b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194cf0e640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19480cdae0 0x7f19480cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1947fff640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f194815d090 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1938011a70 tx=0x7f1938011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1944ff9640 1 -- 172.21.15.53:0/1946847396 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193800fb50 con 0x7f1930004800 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1944ff9640 1 -- 172.21.15.53:0/1946847396 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193800fcf0 con 0x7f1930004800 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19480cc6e0 con 0x7f1930004800 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f1944ff9640 1 -- 172.21.15.53:0/1946847396 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193800fea0 con 0x7f1930004800 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.446+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194815d5d0 con 0x7f1930004800 2024-05-10T10:40:20.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.448+0000 7f1944ff9640 1 -- 172.21.15.53:0/1946847396 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1938012040 con 0x7f1930004800 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.448+0000 7f1944ff9640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19340762d0 0x7f1934078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.449+0000 7f1944ff9640 1 -- 172.21.15.53:0/1946847396 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f1938097f70 con 0x7f1930004800 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.449+0000 7f19477fe640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19340762d0 0x7f1934078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.449+0000 7f19477fe640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19340762d0 0x7f1934078790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f193c033000 tx=0x7f193c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 Processor -- start 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 -- start start 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa4c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa7bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c0b0 0x55b87fa50ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b87f93c920 con 0x55b87fa7c0b0 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b87f9b1dd0 con 0x55b87fa7c7f0 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b87f9623e0 con 0x55b87fa996d0 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.450+0000 7f194cf0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c0b0 0x55b87fa50ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c0b0 0x55b87fa50ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41192/0 (socket says 172.21.15.53:41192) 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 -- 172.21.15.53:0/1660767176 learned_addr learned my addr 172.21.15.53:0/1660767176 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f1947fff640 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa4c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f19477fe640 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa7bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 -- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 msgr2=0x55b87fa4c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa4c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 -- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 msgr2=0x55b87fa7bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa7bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 -- 172.21.15.53:0/1660767176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87f7937c0 con 0x55b87fa7c0b0 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f19477fe640 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa7bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f194cf0e640 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c0b0 0x55b87fa50ec0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f19480d3de0 tx=0x7f1940007050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01931dd2bcf324b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f192dffb640 1 -- 172.21.15.53:0/1660767176 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940007a60 con 0x55b87fa7c0b0 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f192dffb640 1 -- 172.21.15.53:0/1660767176 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19400133f0 con 0x55b87fa7c0b0 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.451+0000 7f192dffb640 1 -- 172.21.15.53:0/1660767176 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19400135e0 con 0x55b87fa7c0b0 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 -- 172.21.15.53:0/1660767176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c0b0 msgr2=0x55b87fa50ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c0b0 0x55b87fa50ec0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f19480d3de0 tx=0x7f1940007050 comp rx=0 tx=0).stop 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 -- 172.21.15.53:0/1660767176 shutdown_connections 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c0b0 0x55b87fa50ec0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa7bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 --2- 172.21.15.53:0/1660767176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa4c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 -- 172.21.15.53:0/1660767176 >> 172.21.15.53:0/1660767176 conn(0x55b87fa4e560 msgr2=0x55b87fa96b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 -- 172.21.15.53:0/1660767176 shutdown_connections 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 -- 172.21.15.53:0/1660767176 wait complete. 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.452+0000 7f1951763740 1 -- start start 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.453+0000 7f1951763740 1 Processor -- start 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.453+0000 7f1951763740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c7f0 0x55b87fa6c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.453+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa7ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.453+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7d3f0 0x55b87fa78c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.453+0000 7f1951763740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b87f9ba440 con 0x55b87fa7c7f0 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.453+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b87f9b5120 con 0x55b87fa7d3f0 2024-05-10T10:40:20.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.453+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b87f9c3210 con 0x55b87fa996d0 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c7f0 0x55b87fa6c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1951763740 10 client.0 _ll_register_callbacks cb 0x55b87f95a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c7f0 0x55b87fa6c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41200/0 (socket says 172.21.15.53:41200) 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 -- 172.21.15.53:0/3294113101 learned_addr learned my addr 172.21.15.53:0/3294113101 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f19477fe640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa7ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f194cf0e640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7d3f0 0x55b87fa78c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 -- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 msgr2=0x55b87fa7ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa7ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 -- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7d3f0 msgr2=0x55b87fa78c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7d3f0 0x55b87fa78c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 -- 172.21.15.53:0/3294113101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87f7f3f50 con 0x55b87fa7c7f0 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f194cf0e640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7d3f0 0x55b87fa78c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f19477fe640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa7ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:20.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1947fff640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c7f0 0x55b87fa6c4b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1938060d10 tx=0x7f19380615e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f838bd37c93e2af0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1926ffd640 1 -- 172.21.15.53:0/3294113101 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938061010 con 0x55b87fa7c7f0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1926ffd640 1 -- 172.21.15.53:0/3294113101 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f1938060270 con 0x55b87fa7c7f0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19400133f0 con 0x55b87fa7c7f0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1926ffd640 1 -- 172.21.15.53:0/3294113101 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f193807f020 con 0x55b87fa7c7f0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1951763740 10 client.7125 fetch_fsmap learned FSMap version 68 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55b87f6870b0 con 0x55b87fa7c7f0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.454+0000 7f1926ffd640 1 -- 172.21.15.53:0/3294113101 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19400133f0 con 0x55b87fa7c7f0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1926ffd640 1 -- 172.21.15.53:0/3294113101 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f1938025030 con 0x55b87fa7c7f0 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1951763740 10 client.7125 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1951763740 10 client.7125 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1951763740 20 client.7125 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1951763740 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87faaa080 0x55b87fa7ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1951763740 4 client.7125 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b87f9c3a20 con 0x55b87faaa080 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f19477fe640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87faaa080 0x55b87fa7ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f19477fe640 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87faaa080 0x55b87fa7ee30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4b4e2156c2781b server_cookie=77f938eb362cb9e2 in_seq=0 out_seq=0 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.455+0000 7f1926ffd640 10 client.7125 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1926ffd640 1 -- 172.21.15.53:0/3294113101 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b87f9c3a20 con 0x55b87faaa080 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1926ffd640 10 client.7125 handle_command_reply: tid=0 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 client.7125 shutdown 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c7f0 msgr2=0x55b87fa6c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c7f0 0x55b87fa6c4b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1938060d10 tx=0x7f19380615e0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 shutdown_connections 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7d3f0 0x55b87fa78c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa996d0 0x55b87fa7ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87faaa080 0x55b87fa7ee30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 --2- 172.21.15.53:0/3294113101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa7c7f0 0x55b87fa6c4b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 >> 172.21.15.53:0/3294113101 conn(0x55b87fa7c0b0 msgr2=0x55b87fa4ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1926ffd640 0 client.7125 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:20.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 shutdown_connections 2024-05-10T10:40:20.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 1 -- 172.21.15.53:0/3294113101 wait complete. 2024-05-10T10:40:20.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.456+0000 7f1951763740 20 client.7125 trim_cache size 0 max 16384 2024-05-10T10:40:20.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 Processor -- start 2024-05-10T10:40:20.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 -- start start 2024-05-10T10:40:20.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa8f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 0x55b87fa7ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa7f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19482019c0 con 0x55b87fa996d0 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1948200cd0 con 0x55b87fa7c7f0 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.470+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f19482012f0 con 0x55b87fa7d3f0 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa7f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa7f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41212/0 (socket says 172.21.15.53:41212) 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 -- 172.21.15.53:0/3160949496 learned_addr learned my addr 172.21.15.53:0/3160949496 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 -- 172.21.15.53:0/3160949496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 msgr2=0x55b87fa7ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 --2- 172.21.15.53:0/3160949496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 0x55b87fa7ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 -- 172.21.15.53:0/3160949496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 msgr2=0x55b87fa8f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 --2- 172.21.15.53:0/3160949496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa8f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 -- 172.21.15.53:0/3160949496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87fa52b30 con 0x55b87fa996d0 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f194cf0e640 1 --2- 172.21.15.53:0/3160949496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa7f370 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1940015000 tx=0x7f194000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb4cd51d155b565 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f192d7fa640 1 -- 172.21.15.53:0/3160949496 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19400148d0 con 0x55b87fa996d0 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f192d7fa640 1 -- 172.21.15.53:0/3160949496 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940014a70 con 0x55b87fa996d0 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.471+0000 7f192d7fa640 1 -- 172.21.15.53:0/3160949496 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940014e60 con 0x55b87fa996d0 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 -- 172.21.15.53:0/3160949496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 msgr2=0x55b87fa7f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 --2- 172.21.15.53:0/3160949496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa7f370 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1940015000 tx=0x7f194000d000 comp rx=0 tx=0).stop 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 -- 172.21.15.53:0/3160949496 shutdown_connections 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 --2- 172.21.15.53:0/3160949496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa7f370 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 --2- 172.21.15.53:0/3160949496 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 0x55b87fa7ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 --2- 172.21.15.53:0/3160949496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa8f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 -- 172.21.15.53:0/3160949496 >> 172.21.15.53:0/3160949496 conn(0x55b87fa7c0b0 msgr2=0x55b87fa6c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 -- 172.21.15.53:0/3160949496 shutdown_connections 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 -- 172.21.15.53:0/3160949496 wait complete. 2024-05-10T10:40:20.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 -- start start 2024-05-10T10:40:20.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.472+0000 7f1951763740 1 Processor -- start 2024-05-10T10:40:20.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f1951763740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 0x55b87fa95bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f1951763740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f1951763740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b87faa6ac0 con 0x55b87fa996d0 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b87f96ba20 con 0x55b87fa7c7f0 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f1951763740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b87faaa480 con 0x55b87fa7d3f0 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f1951763740 10 client.0 _ll_register_callbacks cb 0x7f193805ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f194cf0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f194cf0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa76b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41214/0 (socket says 172.21.15.53:41214) 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.473+0000 7f194cf0e640 1 -- 172.21.15.53:0/2315139072 learned_addr learned my addr 172.21.15.53:0/2315139072 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1947fff640 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f19477fe640 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 0x55b87fa95bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f194cf0e640 1 -- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 msgr2=0x55b87fa95bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f194cf0e640 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 0x55b87fa95bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f194cf0e640 1 -- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 msgr2=0x55b87fa774b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f194cf0e640 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa774b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f194cf0e640 1 -- 172.21.15.53:0/2315139072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87fa46d40 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f194cf0e640 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa76b80 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1940005e60 tx=0x7f1940001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85e7573216ad832 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1926ffd640 1 -- 172.21.15.53:0/2315139072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19400148d0 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1926ffd640 1 -- 172.21.15.53:0/2315139072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f1940004b60 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1940014a70 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1926ffd640 1 -- 172.21.15.53:0/2315139072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f1940004d60 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 10 client.7131 fetch_fsmap learned FSMap version 68 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55b87fa7fc40 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1926ffd640 1 -- 172.21.15.53:0/2315139072 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940014a70 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1926ffd640 1 -- 172.21.15.53:0/2315139072 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f194001d430 con 0x55b87fa996d0 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 10 client.7131 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T10:40:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 10 client.7131 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 20 client.7131 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87fafa5f0 0x55b87fafa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 4 client.7131 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.474+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 --> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b87f9bd890 con 0x55b87fafa5f0 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1947fff640 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87fafa5f0 0x55b87fafa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1926ffd640 10 client.7131 ms_handle_connect on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1947fff640 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87fafa5f0 0x55b87fafa9d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c2f959b7f891e0 server_cookie=f14a3cbb3c8d4a57 in_seq=0 out_seq=0 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1926ffd640 1 -- 172.21.15.53:0/2315139072 <== mds.0 v2:172.21.15.53:6832/285942496 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x55b87f9bd890 con 0x55b87fafa5f0 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1926ffd640 10 client.7131 handle_command_reply: tid=0 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 client.7131 shutdown 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 msgr2=0x55b87fa76b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa76b80 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1940005e60 tx=0x7f1940001360 comp rx=0 tx=0).stop 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 shutdown_connections 2024-05-10T10:40:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:6832/285942496,v1:172.21.15.53:6833/285942496] conn(0x55b87fafa5f0 0x55b87fafa9d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87fa996d0 0x55b87fa76b80 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b87fa7d3f0 0x55b87fa95bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1926ffd640 0 client.7131 ms_handle_reset on v2:172.21.15.53:6832/285942496 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 --2- 172.21.15.53:0/2315139072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b87fa7c7f0 0x55b87fa774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 >> 172.21.15.53:0/2315139072 conn(0x55b87fa7c0b0 msgr2=0x55b87fa9a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 shutdown_connections 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.475+0000 7f1951763740 1 -- 172.21.15.53:0/2315139072 wait complete. 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f1951763740 20 client.7131 trim_cache size 0 max 16384 2024-05-10T10:40:20.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout: "damage_type": "backtrace", 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 3565970136, 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1099511627777, 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/testdir/hardlink" 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:40:20.483 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19340762d0 msgr2=0x7f1934078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19340762d0 0x7f1934078790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f193c033000 tx=0x7f193c008030 comp rx=0 tx=0).stop 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 msgr2=0x7f194815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f194815d090 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1938011a70 tx=0x7f1938011f30 comp rx=0 tx=0).stop 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 shutdown_connections 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f19340762d0 0x7f1934078790 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19480cdae0 0x7f19480cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19480b5780 0x7f19480b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 --2- 172.21.15.53:0/1946847396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1930004800 0x7f194815d090 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 >> 172.21.15.53:0/1946847396 conn(0x7f194809e430 msgr2=0x7f194815f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 shutdown_connections 2024-05-10T10:40:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.476+0000 7f194e971640 1 -- 172.21.15.53:0/1946847396 wait complete. 2024-05-10T10:40:20.487 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:40:20.487 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:40:20.504 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:40:20.504 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:40:20.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:40:20.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.705+0000 7f0846989640 1 Processor -- start 2024-05-10T10:40:20.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.706+0000 7f0846989640 1 -- start start 2024-05-10T10:40:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.706+0000 7f0846989640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f084015f3b0 0x7f08400d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.706+0000 7f0846989640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f08400d5d30 0x7f08400d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.706+0000 7f0846989640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08400d30b0 con 0x7f084015f790 2024-05-10T10:40:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.706+0000 7f0846989640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f08400d3390 con 0x7f08400d5d30 2024-05-10T10:40:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.706+0000 7f0846989640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f08400c8c80 con 0x7f084015f3b0 2024-05-10T10:40:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f083ffff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f084015f3b0 0x7f08400d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f0844f26640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f08400d5d30 0x7f08400d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f083f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f084015f790 0x7f08400c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:59722/0 (socket says 172.21.15.53:59722) 2024-05-10T10:40:20.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f083f7fe640 1 -- 172.21.15.53:0/944343158 learned_addr learned my addr 172.21.15.53:0/944343158 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:20.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f083effd640 1 -- 172.21.15.53:0/944343158 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511707834 0 0) 0x7f08400d30b0 con 0x7f084015f790 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f083effd640 1 -- 172.21.15.53:0/944343158 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0828003520 con 0x7f084015f790 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f0844f26640 1 -- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f084015f3b0 msgr2=0x7f08400d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f0844f26640 1 --2- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f084015f3b0 0x7f08400d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f0844f26640 1 -- 172.21.15.53:0/944343158 >> v1:172.21.15.135:6789/0 conn(0x7f084015f790 legacy=0x7f08400c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f0844f26640 1 -- 172.21.15.53:0/944343158 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08400c8f60 con 0x7f08400d5d30 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.707+0000 7f083ffff640 1 --2- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f084015f3b0 0x7f08400d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.708+0000 7f0844f26640 1 --2- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08400d5d30 0x7f08400d3c30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0838009d90 tx=0x7f083802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea8518d1e6952e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.708+0000 7f083effd640 1 -- 172.21.15.53:0/944343158 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838007740 con 0x7f08400d5d30 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.708+0000 7f083effd640 1 -- 172.21.15.53:0/944343158 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08380078e0 con 0x7f08400d5d30 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.708+0000 7f083effd640 1 -- 172.21.15.53:0/944343158 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838007cb0 con 0x7f08400d5d30 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 -- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08400d5d30 msgr2=0x7f08400d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 --2- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08400d5d30 0x7f08400d3c30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0838009d90 tx=0x7f083802a660 comp rx=0 tx=0).stop 2024-05-10T10:40:20.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 -- 172.21.15.53:0/944343158 shutdown_connections 2024-05-10T10:40:20.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 --2- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08400d5d30 0x7f08400d3c30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 --2- 172.21.15.53:0/944343158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f084015f3b0 0x7f08400d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 -- 172.21.15.53:0/944343158 >> 172.21.15.53:0/944343158 conn(0x7f08400d5130 msgr2=0x7f08400d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 -- 172.21.15.53:0/944343158 shutdown_connections 2024-05-10T10:40:20.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.709+0000 7f0846989640 1 -- 172.21.15.53:0/944343158 wait complete. 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.710+0000 7f0846989640 1 Processor -- start 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.710+0000 7f0846989640 1 -- start start 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0846989640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08400d5d30 0x7f0840154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0846989640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084015f3b0 0x7f08400c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0846989640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08400cc200 0x7f08400cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0846989640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08400d0e60 con 0x7f08400cc200 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0846989640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08400d0ce0 con 0x7f084015f3b0 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0846989640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f08400d0fe0 con 0x7f08400d5d30 2024-05-10T10:40:20.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08400cc200 0x7f08400cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08400cc200 0x7f08400cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41226/0 (socket says 172.21.15.53:41226) 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 -- 172.21.15.53:0/1279140678 learned_addr learned my addr 172.21.15.53:0/1279140678 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f083ffff640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08400d5d30 0x7f0840154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f083f7fe640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084015f3b0 0x7f08400c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 -- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08400d5d30 msgr2=0x7f0840154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08400d5d30 0x7f0840154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 -- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084015f3b0 msgr2=0x7f08400c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084015f3b0 0x7f08400c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.711+0000 7f0844f26640 1 -- 172.21.15.53:0/1279140678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08400cdcf0 con 0x7f08400cc200 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.712+0000 7f0844f26640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08400cc200 0x7f08400cc6b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0838009eb0 tx=0x7f0838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.712+0000 7f083cff9640 1 -- 172.21.15.53:0/1279140678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838040070 con 0x7f08400cc200 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.712+0000 7f083cff9640 1 -- 172.21.15.53:0/1279140678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0838031bc0 con 0x7f08400cc200 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.712+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08400ccd00 con 0x7f08400cc200 2024-05-10T10:40:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.712+0000 7f083cff9640 1 -- 172.21.15.53:0/1279140678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838009450 con 0x7f08400cc200 2024-05-10T10:40:20.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.712+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08400cdf90 con 0x7f08400cc200 2024-05-10T10:40:20.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.713+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08400ce190 con 0x7f08400cc200 2024-05-10T10:40:20.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.713+0000 7f083cff9640 1 -- 172.21.15.53:0/1279140678 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0838039040 con 0x7f08400cc200 2024-05-10T10:40:20.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.714+0000 7f083cff9640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f082c0762d0 0x7f082c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:20.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.714+0000 7f083cff9640 1 -- 172.21.15.53:0/1279140678 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f08380b8fc0 con 0x7f08400cc200 2024-05-10T10:40:20.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.714+0000 7f083ffff640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f082c0762d0 0x7f082c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:20.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.715+0000 7f083ffff640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f082c0762d0 0x7f082c078790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0830009560 tx=0x7f0830009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:20.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.718+0000 7f083cff9640 1 -- 172.21.15.53:0/1279140678 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f08400ce190 con 0x7f08400cc200 2024-05-10T10:40:20.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.918+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f084006a220 con 0x7f08400cc200 2024-05-10T10:40:20.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.918+0000 7f083cff9640 1 -- 172.21.15.53:0/1279140678 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v90) ==== 80+0+122 (secure 0 0 0) 0x7f0838081f60 con 0x7f08400cc200 2024-05-10T10:40:20.921 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6833/2934725760 2024-05-11T10:39:11.380694+0000 2024-05-10T10:40:20.921 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6832/2934725760 2024-05-11T10:39:11.380694+0000 2024-05-10T10:40:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:listed 2 entries 2024-05-10T10:40:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f082c0762d0 msgr2=0x7f082c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f082c0762d0 0x7f082c078790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0830009560 tx=0x7f0830009190 comp rx=0 tx=0).stop 2024-05-10T10:40:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08400cc200 msgr2=0x7f08400cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08400cc200 0x7f08400cc6b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0838009eb0 tx=0x7f0838004a40 comp rx=0 tx=0).stop 2024-05-10T10:40:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 shutdown_connections 2024-05-10T10:40:20.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f082c0762d0 0x7f082c078790 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08400cc200 0x7f08400cc6b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084015f3b0 0x7f08400c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 --2- 172.21.15.53:0/1279140678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08400d5d30 0x7f0840154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:20.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 >> 172.21.15.53:0/1279140678 conn(0x7f08400d5130 msgr2=0x7f08400c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:20.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 shutdown_connections 2024-05-10T10:40:20.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:20.920+0000 7f0846989640 1 -- 172.21.15.53:0/1279140678 wait complete. 2024-05-10T10:40:20.933 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:40:20.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:20.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:20.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:40:20.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:20.963 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:40:21.019 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:40:21.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.163+0000 7fec6ad11640 1 Processor -- start 2024-05-10T10:40:21.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.163+0000 7fec6ad11640 1 -- start start 2024-05-10T10:40:21.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.164+0000 7fec6ad11640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec6415f790 0x7fec640cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.164+0000 7fec6ad11640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec640d5d30 0x7fec640d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.164+0000 7fec6ad11640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec640d30b0 con 0x7fec6415f3b0 2024-05-10T10:40:21.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.164+0000 7fec6ad11640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fec640d3390 con 0x7fec640d5d30 2024-05-10T10:40:21.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.164+0000 7fec6ad11640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fec640c8c80 con 0x7fec6415f790 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.164+0000 7fec692ae640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec640d5d30 0x7fec640d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec692ae640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec640d5d30 0x7fec640d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41228/0 (socket says 172.21.15.53:41228) 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec692ae640 1 -- 172.21.15.53:0/122670578 learned_addr learned my addr 172.21.15.53:0/122670578 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec63fff640 1 --2- 172.21.15.53:0/122670578 >> v2:172.21.15.135:3300/0 conn(0x7fec6415f790 0x7fec640cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec692ae640 1 -- 172.21.15.53:0/122670578 >> v2:172.21.15.135:3300/0 conn(0x7fec6415f790 msgr2=0x7fec640cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec692ae640 1 --2- 172.21.15.53:0/122670578 >> v2:172.21.15.135:3300/0 conn(0x7fec6415f790 0x7fec640cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec692ae640 1 -- 172.21.15.53:0/122670578 >> v1:172.21.15.135:6789/0 conn(0x7fec6415f3b0 legacy=0x7fec640d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec692ae640 1 -- 172.21.15.53:0/122670578 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec640c8f60 con 0x7fec640d5d30 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.165+0000 7fec692ae640 1 --2- 172.21.15.53:0/122670578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec640d5d30 0x7fec640d0210 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fec640c8c00 tx=0x7fec5c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1640556e5a5095eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec637fe640 1 -- 172.21.15.53:0/122670578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c036070 con 0x7fec640d5d30 2024-05-10T10:40:21.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec637fe640 1 -- 172.21.15.53:0/122670578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5c031600 con 0x7fec640d5d30 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec637fe640 1 -- 172.21.15.53:0/122670578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0319d0 con 0x7fec640d5d30 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 -- 172.21.15.53:0/122670578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec640d5d30 msgr2=0x7fec640d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 --2- 172.21.15.53:0/122670578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec640d5d30 0x7fec640d0210 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fec640c8c00 tx=0x7fec5c02a6f0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 -- 172.21.15.53:0/122670578 shutdown_connections 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 --2- 172.21.15.53:0/122670578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec640d5d30 0x7fec640d0210 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 --2- 172.21.15.53:0/122670578 >> v2:172.21.15.135:3300/0 conn(0x7fec6415f790 0x7fec640cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 -- 172.21.15.53:0/122670578 >> 172.21.15.53:0/122670578 conn(0x7fec640d5130 msgr2=0x7fec640d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 -- 172.21.15.53:0/122670578 shutdown_connections 2024-05-10T10:40:21.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.166+0000 7fec6ad11640 1 -- 172.21.15.53:0/122670578 wait complete. 2024-05-10T10:40:21.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.168+0000 7fec6ad11640 1 Processor -- start 2024-05-10T10:40:21.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.168+0000 7fec6ad11640 1 -- start start 2024-05-10T10:40:21.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.168+0000 7fec6ad11640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec640d5d30 0x7fec641f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.168+0000 7fec6ad11640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec6415f3b0 0x7fec641551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.168+0000 7fec6ad11640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec640cda70 0x7fec6415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec6ad11640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec640d3f80 con 0x7fec6415f3b0 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec6ad11640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fec640d3e00 con 0x7fec640d5d30 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec6ad11640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fec640ca770 con 0x7fec640cda70 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec6415f3b0 0x7fec641551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec6415f3b0 0x7fec641551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41236/0 (socket says 172.21.15.53:41236) 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 -- 172.21.15.53:0/171603331 learned_addr learned my addr 172.21.15.53:0/171603331 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec692ae640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec640cda70 0x7fec6415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec68aad640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec640d5d30 0x7fec641f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 -- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec640cda70 msgr2=0x7fec6415b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec640cda70 0x7fec6415b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 -- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec640d5d30 msgr2=0x7fec641f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec640d5d30 0x7fec641f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 -- 172.21.15.53:0/171603331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec641f5f30 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec63fff640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec6415f3b0 0x7fec641551a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fec5000c7f0 tx=0x7fec5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec617fa640 1 -- 172.21.15.53:0/171603331 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50015070 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec617fa640 1 -- 172.21.15.53:0/171603331 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec50007820 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec617fa640 1 -- 172.21.15.53:0/171603331 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50007bd0 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec640cdfe0 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.169+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec640ce5a0 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.170+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6406a220 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.171+0000 7fec617fa640 1 -- 172.21.15.53:0/171603331 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fec50018070 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.172+0000 7fec617fa640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fec480762d0 0x7fec48078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.172+0000 7fec617fa640 1 -- 172.21.15.53:0/171603331 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7fec500975f0 con 0x7fec6415f3b0 2024-05-10T10:40:21.175 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.172+0000 7fec68aad640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fec480762d0 0x7fec48078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.172+0000 7fec68aad640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fec480762d0 0x7fec48078790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fec58004ea0 tx=0x7fec58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.176+0000 7fec617fa640 1 -- 172.21.15.53:0/171603331 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fec641f5f30 con 0x7fec6415f3b0 2024-05-10T10:40:21.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.406+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec640badd0 con 0x7fec6415f3b0 2024-05-10T10:40:21.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.407+0000 7fec617fa640 1 -- 172.21.15.53:0/171603331 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5184 (secure 0 0 0) 0x7fec50060c60 con 0x7fec6415f3b0 2024-05-10T10:40:21.408 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:21.409 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:39:22.393011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6833/285942496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":285942496},{"type":"v1","addr":"172.21.15.53:6833","nonce":285942496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6888,"qdb_cluster":[6888]},"id":6}]} 2024-05-10T10:40:21.409 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 68 2024-05-10T10:40:21.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fec480762d0 msgr2=0x7fec48078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fec480762d0 0x7fec48078790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fec58004ea0 tx=0x7fec58009040 comp rx=0 tx=0).stop 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec6415f3b0 msgr2=0x7fec641551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec6415f3b0 0x7fec641551a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fec5000c7f0 tx=0x7fec5000ccb0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 shutdown_connections 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fec480762d0 0x7fec48078790 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec640cda70 0x7fec6415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec6415f3b0 0x7fec641551a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 --2- 172.21.15.53:0/171603331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec640d5d30 0x7fec641f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 >> 172.21.15.53:0/171603331 conn(0x7fec640d5130 msgr2=0x7fec640d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 shutdown_connections 2024-05-10T10:40:21.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.409+0000 7fec6ad11640 1 -- 172.21.15.53:0/171603331 wait complete. 2024-05-10T10:40:21.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-10T10:40:21.549 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:40:21.550 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:40:21.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:40:21.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.653+0000 7f6724fb6640 1 Processor -- start 2024-05-10T10:40:21.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.653+0000 7f6724fb6640 1 -- start start 2024-05-10T10:40:21.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f6724fb6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f672015f790 0x7f67200cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f6724fb6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67200d5d30 0x7f67200d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f6724fb6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67200d30b0 con 0x7f672015f3b0 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f6724fb6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f67200d3390 con 0x7f672015f790 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f6724fb6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f67200c8c80 con 0x7f67200d5d30 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f671dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f672015f790 0x7f67200cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f671dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f672015f790 0x7f67200cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41250/0 (socket says 172.21.15.53:41250) 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.654+0000 7f671dd9b640 1 -- 172.21.15.53:0/1134338163 learned_addr learned my addr 172.21.15.53:0/1134338163 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.655+0000 7f671ed9d640 1 --2- 172.21.15.53:0/1134338163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67200d5d30 0x7f67200d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.655+0000 7f671dd9b640 1 -- 172.21.15.53:0/1134338163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67200d5d30 msgr2=0x7f67200d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.655+0000 7f671dd9b640 1 --2- 172.21.15.53:0/1134338163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67200d5d30 0x7f67200d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.655+0000 7f671dd9b640 1 -- 172.21.15.53:0/1134338163 >> v1:172.21.15.135:6789/0 conn(0x7f672015f3b0 legacy=0x7f67200d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.655+0000 7f671dd9b640 1 -- 172.21.15.53:0/1134338163 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67200c8f60 con 0x7f672015f790 2024-05-10T10:40:21.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.655+0000 7f671dd9b640 1 --2- 172.21.15.53:0/1134338163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f672015f790 0x7f67200cfcd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6708009870 tx=0x7f670802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1f362037b90af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f671d59a640 1 -- 172.21.15.53:0/1134338163 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708037070 con 0x7f672015f790 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f671d59a640 1 -- 172.21.15.53:0/1134338163 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6708032440 con 0x7f672015f790 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f671d59a640 1 -- 172.21.15.53:0/1134338163 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708032810 con 0x7f672015f790 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 -- 172.21.15.53:0/1134338163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f672015f790 msgr2=0x7f67200cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1134338163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f672015f790 0x7f67200cfcd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6708009870 tx=0x7f670802a270 comp rx=0 tx=0).stop 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 -- 172.21.15.53:0/1134338163 shutdown_connections 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1134338163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67200d5d30 0x7f67200d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1134338163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f672015f790 0x7f67200cfcd0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 -- 172.21.15.53:0/1134338163 >> 172.21.15.53:0/1134338163 conn(0x7f67200d5130 msgr2=0x7f67200d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 -- 172.21.15.53:0/1134338163 shutdown_connections 2024-05-10T10:40:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.656+0000 7f6724fb6640 1 -- 172.21.15.53:0/1134338163 wait complete. 2024-05-10T10:40:21.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 Processor -- start 2024-05-10T10:40:21.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 -- start start 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67200d5d30 0x7f672014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f672015f3b0 0x7f672014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6720150100 0x7f67200cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67200d3f80 con 0x7f6720150100 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67200d3e00 con 0x7f67200d5d30 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f6724fb6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f67200ca770 con 0x7f672015f3b0 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.658+0000 7f671ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6720150100 0x7f67200cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6720150100 0x7f67200cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41262/0 (socket says 172.21.15.53:41262) 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 -- 172.21.15.53:0/1809254694 learned_addr learned my addr 172.21.15.53:0/1809254694 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671e59c640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67200d5d30 0x7f672014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671dd9b640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f672015f3b0 0x7f672014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 -- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f672015f3b0 msgr2=0x7f672014fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f672015f3b0 0x7f672014fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 -- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67200d5d30 msgr2=0x7f672014d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67200d5d30 0x7f672014d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 -- 172.21.15.53:0/1809254694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672014d860 con 0x7f6720150100 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671dd9b640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f672015f3b0 0x7f672014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671e59c640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67200d5d30 0x7f672014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f671ed9d640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6720150100 0x7f67200cd490 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f671800d800 tx=0x7f671800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f6716ffd640 1 -- 172.21.15.53:0/1809254694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671800b6c0 con 0x7f6720150100 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f6716ffd640 1 -- 172.21.15.53:0/1809254694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6718019070 con 0x7f6720150100 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67200ce260 con 0x7f6720150100 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f6716ffd640 1 -- 172.21.15.53:0/1809254694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671800bac0 con 0x7f6720150100 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.659+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67200ce820 con 0x7f6720150100 2024-05-10T10:40:21.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.660+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672006a220 con 0x7f6720150100 2024-05-10T10:40:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.661+0000 7f6716ffd640 1 -- 172.21.15.53:0/1809254694 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f671800bc60 con 0x7f6720150100 2024-05-10T10:40:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.661+0000 7f6716ffd640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.662+0000 7f6716ffd640 1 -- 172.21.15.53:0/1809254694 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f6718097d90 con 0x7f6720150100 2024-05-10T10:40:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.662+0000 7f671e59c640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.662+0000 7f671e59c640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6710001d70 tx=0x7f6710009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.666+0000 7f6716ffd640 1 -- 172.21.15.53:0/1809254694 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6718061400 con 0x7f6720150100 2024-05-10T10:40:21.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.894+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f67200d0150 con 0x7f6720150100 2024-05-10T10:40:21.896 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:40:21.894+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:40:21.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.895+0000 7f6716ffd640 1 -- 172.21.15.53:0/1809254694 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7f6718060d00 con 0x7f6720150100 2024-05-10T10:40:21.897 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:40:21.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 msgr2=0x7f6700078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6710001d70 tx=0x7f6710009040 comp rx=0 tx=0).stop 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6720150100 msgr2=0x7f67200cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6720150100 0x7f67200cd490 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f671800d800 tx=0x7f671800dcc0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 shutdown_connections 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f67000762d0 0x7f6700078790 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6720150100 0x7f67200cd490 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f672015f3b0 0x7f672014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 --2- 172.21.15.53:0/1809254694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67200d5d30 0x7f672014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 >> 172.21.15.53:0/1809254694 conn(0x7f67200d5130 msgr2=0x7f67200d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 shutdown_connections 2024-05-10T10:40:21.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.898+0000 7f6724fb6640 1 -- 172.21.15.53:0/1809254694 wait complete. 2024-05-10T10:40:21.903 INFO:tasks.ceph.mds.a.smithi053.stderr: -4605> 2024-05-10T10:39:17.390+0000 7f54656aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:40:21.904 INFO:tasks.ceph.mds.a.smithi053.stderr: -4202> 2024-05-10T10:39:18.036+0000 7f54616a2640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-10T10:40:21.904 INFO:tasks.ceph.mds.a.smithi053.stderr: -4040> 2024-05-10T10:39:18.226+0000 7f54616a2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0) n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x559c49899b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-10T10:40:21.906 INFO:tasks.ceph.mds.a.smithi053.stderr: -2171> 2024-05-10T10:39:49.589+0000 7f54616a2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0)->f(v1 m2024-05-10T10:39:09.698395+0000) n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1)->n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x559c49899b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-10T10:39:09.698395+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-10T10:39:09.817392+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-10T10:40:21.911 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-10T10:40:21.932 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:40:21.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.951+0000 7fa790c0f740 1 -- start start 2024-05-10T10:40:21.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.951+0000 7fa790c0f740 1 Processor -- start 2024-05-10T10:40:21.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558025a7ec00 0x55802685c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558025a7f800 0x55802685c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558025a7fc00 0x55802685cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa790c0f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558026870600 con 0x558025a7fc00 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa790c0f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558026870480 con 0x558025a7ec00 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa790c0f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558026870300 con 0x558025a7f800 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa78f02f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558025a7fc00 0x55802685cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa78f02f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558025a7fc00 0x55802685cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41300/0 (socket says 172.21.15.53:41300) 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa78f02f640 1 -- 172.21.15.53:0/288606625 learned_addr learned my addr 172.21.15.53:0/288606625 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa78e02d640 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558025a7f800 0x55802685c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.952+0000 7fa78e82e640 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558025a7ec00 0x55802685c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78f02f640 1 -- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558025a7f800 msgr2=0x55802685c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78f02f640 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558025a7f800 0x55802685c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78f02f640 1 -- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558025a7ec00 msgr2=0x55802685c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78f02f640 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558025a7ec00 0x55802685c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78f02f640 1 -- 172.21.15.53:0/288606625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558025aab680 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78f02f640 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558025a7fc00 0x55802685cb00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x5580268afb60 tx=0x558026824a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ace84492d659ff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78c029640 1 -- 172.21.15.53:0/288606625 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558025aa49c0 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78c029640 1 -- 172.21.15.53:0/288606625 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558025ac6c40 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78c029640 1 -- 172.21.15.53:0/288606625 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558025aa4680 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa790c0f740 1 -- 172.21.15.53:0/288606625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558025aab860 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa790c0f740 1 -- 172.21.15.53:0/288606625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x558025aab2c0 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa790c0f740 4 waiting for MDS map... 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78c029640 1 -- 172.21.15.53:0/288606625 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x558026896200 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa78c029640 1 -- 172.21.15.53:0/288606625 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x558025ad7180 con 0x558025a7fc00 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa790c0f740 4 Got MDS map 69 2024-05-10T10:40:21.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa790c0f740 4 datascan.main: Using metadata pool output 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.953+0000 7fa790c0f740 4 datascan.main: connecting to RADOS... 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.954+0000 7fa790c0f740 1 Processor -- start 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.954+0000 7fa790c0f740 1 -- start start 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.954+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc000 0x55802685d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.954+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc400 0x55802685d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.954+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa790c0f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558026870180 con 0x5580269bc400 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa790c0f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558025ade480 con 0x5580269bc800 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa790c0f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558025ade900 con 0x5580269bc000 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc400 0x55802685d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc400 0x55802685d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41306/0 (socket says 172.21.15.53:41306) 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 -- 172.21.15.53:0/102814966 learned_addr learned my addr 172.21.15.53:0/102814966 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e02d640 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc000 0x55802685d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e02d640 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 -- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc000 msgr2=0x55802685d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc000 0x55802685d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 -- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 msgr2=0x55802685db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 -- 172.21.15.53:0/102814966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558025aab4a0 con 0x5580269bc400 2024-05-10T10:40:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e02d640 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78e82e640 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc400 0x55802685d600 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x55802686f230 tx=0x558026957920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f599e64ebfdcd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78a025640 1 -- 172.21.15.53:0/102814966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558025aa49c0 con 0x5580269bc400 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78a025640 1 -- 172.21.15.53:0/102814966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558025ac7340 con 0x5580269bc400 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.955+0000 7fa78a025640 1 -- 172.21.15.53:0/102814966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558025aa4680 con 0x5580269bc400 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 -- 172.21.15.53:0/102814966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc400 msgr2=0x55802685d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc400 0x55802685d600 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x55802686f230 tx=0x558026957920 comp rx=0 tx=0).stop 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 -- 172.21.15.53:0/102814966 shutdown_connections 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc400 0x55802685d600 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 --2- 172.21.15.53:0/102814966 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc000 0x55802685d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 -- 172.21.15.53:0/102814966 >> 172.21.15.53:0/102814966 conn(0x558026873c00 msgr2=0x5580267fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 -- 172.21.15.53:0/102814966 shutdown_connections 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 -- 172.21.15.53:0/102814966 wait complete. 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 Processor -- start 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.956+0000 7fa790c0f740 1 -- start start 2024-05-10T10:40:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc000 0x55802685d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc400 0x55802685d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa790c0f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa790c0f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558026871380 con 0x5580269bc000 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa790c0f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558026870180 con 0x5580269bc800 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa790c0f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558026871500 con 0x5580269bc400 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc000 0x55802685d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc000 0x55802685d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41318/0 (socket says 172.21.15.53:41318) 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 learned_addr learned my addr 172.21.15.53:0/2002371475 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e82e640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc400 0x55802685d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc400 msgr2=0x55802685d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580269bc400 0x55802685d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 msgr2=0x55802685db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580269bc800 0x55802685db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558025aab0e0 con 0x5580269bc000 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.957+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc000 0x55802685d080 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55802686fe90 tx=0x558026957860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.958+0000 7fa788822640 1 -- 172.21.15.53:0/2002371475 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558025aa49c0 con 0x5580269bc000 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.958+0000 7fa788822640 1 -- 172.21.15.53:0/2002371475 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558025ac61c0 con 0x5580269bc000 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.958+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558025aaba40 con 0x5580269bc000 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.958+0000 7fa788822640 1 -- 172.21.15.53:0/2002371475 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558025aa4680 con 0x5580269bc000 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.958+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558025aaa3c0 con 0x5580269bc000 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.958+0000 7fa790c0f740 4 datascan.init: resolving metadata pool 13 2024-05-10T10:40:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa788822640 1 -- 172.21.15.53:0/2002371475 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x5580269dac00 con 0x5580269bc000 2024-05-10T10:40:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa788822640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x5580269e6c00 0x55802685e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa788822640 1 -- 172.21.15.53:0/2002371475 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x5580269d9000 con 0x5580269bc000 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa790c0f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa790c0f740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa790c0f740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580269bcc00 0x55802685e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.959+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269bd000 con 0x5580269bcc00 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.960+0000 7fa78e82e640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x5580269e6c00 0x55802685e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.960+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580269bcc00 0x55802685e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.960+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580269bcc00 0x55802685e680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.960+0000 7fa78e82e640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x5580269e6c00 0x55802685e100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55802686f230 tx=0x558026824600 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.962+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== osd_op_reply(1 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x5580269bcc00 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.962+0000 7fa789824640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x5580269e7000 0x558025e3a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.962+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269e7400 con 0x5580269e7000 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.962+0000 7fa78f02f640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x5580269e7000 0x558025e3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.962+0000 7fa78f02f640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x5580269e7000 0x558025e3a000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.963+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(2 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.963+0000 7fa78a025640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5580269e7c00 0x558025e3a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.963+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44000 con 0x5580269e7c00 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.963+0000 7fa78e82e640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5580269e7c00 0x558025e3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.963+0000 7fa78e82e640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5580269e7c00 0x558025e3a580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.965+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(3 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.965+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269e7800 con 0x5580269e7000 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.965+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(4 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.965+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44400 con 0x5580269e7000 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.965+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 3 ==== osd_op_reply(5 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.966+0000 7fa789824640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x558025e44800 0x558025e3ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.966+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44c00 con 0x558025e44800 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.966+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x558025e44800 0x558025e3ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.966+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x558025e44800 0x558025e3ab00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== osd_op_reply(6 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e44800 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45400 con 0x5580269e7c00 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== osd_op_reply(7 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45000 con 0x5580269e7c00 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 3 ==== osd_op_reply(8 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa78a025640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x558025e45800 0x558025e3b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45c00 con 0x558025e45800 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.968+0000 7fa78f02f640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x558025e45800 0x558025e3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.969+0000 7fa78f02f640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x558025e45800 0x558025e3b600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.971+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(9 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e45800 2024-05-10T10:40:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.971+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44000 con 0x5580269e7c00 2024-05-10T10:40:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.972+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 4 ==== osd_op_reply(10 [pgnls start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.972+0000 7fa78a025640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x558025e4e000 0x558025e3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.972+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e400 con 0x558025e4e000 2024-05-10T10:40:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.972+0000 7fa78e82e640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x558025e4e000 0x558025e3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.972+0000 7fa78e82e640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x558025e4e000 0x558025e3bb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.974+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(11 [pgnls start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.975+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45400 con 0x5580269bcc00 2024-05-10T10:40:21.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.975+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== osd_op_reply(12 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x5580269bcc00 2024-05-10T10:40:21.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.975+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e800 con 0x5580269e7000 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.975+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 4 ==== osd_op_reply(13 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.975+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45000 con 0x5580269e7c00 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.975+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 5 ==== osd_op_reply(14 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.975+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4ec00 con 0x5580269e7000 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.976+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 5 ==== osd_op_reply(15 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.976+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269bdc00 con 0x5580269e7c00 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.976+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 6 ==== osd_op_reply(16 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.976+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e800 con 0x558025e45800 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.976+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(17 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e45800 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.976+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4f000 con 0x558025e4e000 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.977+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== osd_op_reply(18 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.977+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44400 con 0x5580269bcc00 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.977+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.2 v2:172.21.15.53:6800/2235813461 3 ==== osd_op_reply(19 [pgnls start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x5580269bcc00 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.977+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4f400 con 0x558025e4e000 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.977+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 3 ==== osd_op_reply(20 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.977+0000 7fa78a025640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x558025e45c00 0x558025e3c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.977+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269e7400 con 0x558025e45c00 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.978+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x558025e45c00 0x558025e3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.978+0000 7fa78e02d640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x558025e45c00 0x558025e3c100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.978+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(21 [pgnls start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.978+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4f800 con 0x558025e4e000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.979+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 4 ==== osd_op_reply(22 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.979+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269e7800 con 0x5580269e7000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.979+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 6 ==== osd_op_reply(23 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.979+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4fc00 con 0x5580269e7000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.979+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 7 ==== osd_op_reply(24 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.979+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269bd000 con 0x5580269e7000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 8 ==== osd_op_reply(25 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e400 con 0x558025e4e000 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 5 ==== osd_op_reply(26 [pgnls start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44c00 con 0x558025e44800 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== osd_op_reply(27 [pgnls start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e44800 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4f000 con 0x5580269bcc00 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.2 v2:172.21.15.53:6800/2235813461 4 ==== osd_op_reply(28 [pgnls start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x5580269bcc00 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.980+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45400 con 0x558025e4e000 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.981+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 6 ==== osd_op_reply(29 [pgnls start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.981+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269bdc00 con 0x558025e44800 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.981+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.3 v2:172.21.15.53:6816/2871564334 3 ==== osd_op_reply(30 [pgnls start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e44800 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.981+0000 7fa78a025640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x558025e44400 0x558025e3c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.981+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44000 con 0x558025e44400 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.981+0000 7fa78f02f640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x558025e44400 0x558025e3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.982+0000 7fa78f02f640 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x558025e44400 0x558025e3c680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.982+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(31 [pgnls start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e44400 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.982+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44000 con 0x558025e45c00 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.983+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== osd_op_reply(32 [pgnls start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.983+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45000 con 0x558025e44800 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.983+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.3 v2:172.21.15.53:6816/2871564334 4 ==== osd_op_reply(33 [pgnls start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e44800 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.983+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269bd000 con 0x558025e45c00 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.983+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 3 ==== osd_op_reply(34 [pgnls start_epoch 91 out=49b] v87'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.983+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44c00 con 0x558025e4e000 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.984+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 7 ==== osd_op_reply(35 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.984+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4ec00 con 0x5580269e7c00 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.984+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 7 ==== osd_op_reply(36 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.984+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44000 con 0x558025e45c00 2024-05-10T10:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.984+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 4 ==== osd_op_reply(37 [pgnls start_epoch 91 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.984+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e800 con 0x558025e45800 2024-05-10T10:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.985+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(38 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e45800 2024-05-10T10:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.985+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269bdc00 con 0x558025e45800 2024-05-10T10:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.985+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.6 v2:172.21.15.135:6808/1123855098 4 ==== osd_op_reply(39 [pgnls start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e45800 2024-05-10T10:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.985+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4fc00 con 0x558025e44400 2024-05-10T10:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.985+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== osd_op_reply(40 [pgnls start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e44400 2024-05-10T10:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.985+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4f000 con 0x5580269e7000 2024-05-10T10:40:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.986+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 9 ==== osd_op_reply(41 [pgnls start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.986+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269e7800 con 0x5580269bcc00 2024-05-10T10:40:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.986+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.2 v2:172.21.15.53:6800/2235813461 5 ==== osd_op_reply(42 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x5580269bcc00 2024-05-10T10:40:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.986+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269e7400 con 0x558025e44400 2024-05-10T10:40:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.986+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.7 v2:172.21.15.135:6809/751247261 3 ==== osd_op_reply(43 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e44400 2024-05-10T10:40:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.986+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4f400 con 0x558025e4e000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.987+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 8 ==== osd_op_reply(44 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.987+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e68000 con 0x558025e45800 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.987+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.6 v2:172.21.15.135:6808/1123855098 5 ==== osd_op_reply(45 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e45800 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.987+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4f800 con 0x558025e4e000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.987+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 9 ==== osd_op_reply(46 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.987+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e68400 con 0x5580269e7000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 10 ==== osd_op_reply(47 [pgnls start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e400 con 0x558025e44800 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.3 v2:172.21.15.53:6816/2871564334 5 ==== osd_op_reply(48 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e44800 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e68800 con 0x5580269e7000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 11 ==== osd_op_reply(49 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4ec00 con 0x558025e44400 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.7 v2:172.21.15.135:6809/751247261 4 ==== osd_op_reply(50 [pgnls start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e44400 2024-05-10T10:40:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.988+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e68c00 con 0x5580269bcc00 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.989+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.2 v2:172.21.15.53:6800/2235813461 6 ==== osd_op_reply(51 [pgnls start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x5580269bcc00 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.989+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e45400 con 0x5580269e7c00 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.989+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 8 ==== osd_op_reply(52 [pgnls start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.989+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e69000 con 0x5580269e7000 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.989+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 12 ==== osd_op_reply(53 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.989+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e44c00 con 0x558025e4e000 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.990+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 10 ==== osd_op_reply(54 [pgnls start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.990+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e69400 con 0x5580269e7c00 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.990+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 9 ==== osd_op_reply(55 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.990+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x5580269e7800 con 0x558025e4e000 2024-05-10T10:40:21.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.990+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 11 ==== osd_op_reply(56 [pgnls start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.990+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e69800 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.991+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 5 ==== osd_op_reply(57 [pgnls start_epoch 91 out=81b] v87'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.991+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x558025e68c00 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.991+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 6 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.991+0000 7fa790c0f740 10 datascan.operator(): handling object 1099511627777.0 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.991+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x558025e68000 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.993+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 7 ==== osd_op_reply(59 10000000001.00000000 [call] v91'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.993+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e68400 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.993+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.5 v2:172.21.15.135:6824/3244007917 8 ==== osd_op_reply(60 [pgnls start_epoch 91 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e45c00 2024-05-10T10:40:21.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.993+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e69c00 con 0x5580269e7c00 2024-05-10T10:40:21.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.994+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.1 v2:172.21.15.53:6824/2705510088 10 ==== osd_op_reply(61 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x5580269e7c00 2024-05-10T10:40:21.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.994+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e400 con 0x558025e4e000 2024-05-10T10:40:21.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.994+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 <== osd.4 v2:172.21.15.135:6800/3517077655 12 ==== osd_op_reply(62 [pgnls start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810780 con 0x558025e4e000 2024-05-10T10:40:21.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.994+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e69400 con 0x558025e44400 2024-05-10T10:40:21.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.994+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.7 v2:172.21.15.135:6809/751247261 5 ==== osd_op_reply(63 [pgnls start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x558025e44400 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.994+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4e800 con 0x5580269e7000 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 13 ==== osd_op_reply(64 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78a025640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e69c00 con 0x5580269e7000 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78f02f640 1 -- 172.21.15.53:0/2002371475 <== osd.0 v2:172.21.15.53:6802/1634417257 14 ==== osd_op_reply(65 [pgnls start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810500 con 0x5580269e7000 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa789824640 1 -- 172.21.15.53:0/2002371475 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x558025e4fc00 con 0x558025e44800 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78e02d640 1 -- 172.21.15.53:0/2002371475 <== osd.3 v2:172.21.15.53:6816/2871564334 6 ==== osd_op_reply(66 [pgnls start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558026810000 con 0x558025e44800 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x5580269e7000 msgr2=0x558025e3a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x5580269e7000 0x558025e3a000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5580269e7c00 msgr2=0x558025e3a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5580269e7c00 0x558025e3a580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580269bcc00 msgr2=0x55802685e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580269bcc00 0x55802685e680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x558025e44800 msgr2=0x558025e3ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x558025e44800 0x558025e3ab00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x558025e4e000 msgr2=0x558025e3bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x558025e4e000 0x558025e3bb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x558025e45c00 msgr2=0x558025e3c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x558025e45c00 0x558025e3c100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x558025e45800 msgr2=0x558025e3b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x558025e45800 0x558025e3b600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x558025e44400 msgr2=0x558025e3c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x558025e44400 0x558025e3c680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 reap_dead start 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 reap_dead start 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x5580269e6c00 msgr2=0x55802685e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 reap_dead start 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x5580269e6c00 0x55802685e100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55802686f230 tx=0x558026824600 comp rx=0 tx=0).stop 2024-05-10T10:40:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc000 msgr2=0x55802685d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 --2- 172.21.15.53:0/2002371475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580269bc000 0x55802685d080 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55802686fe90 tx=0x558026957860 comp rx=0 tx=0).stop 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa78e82e640 1 -- 172.21.15.53:0/2002371475 reap_dead start 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 shutdown_connections 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.995+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 >> 172.21.15.53:0/2002371475 conn(0x558026873c00 msgr2=0x5580267fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 shutdown_connections 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 -- 172.21.15.53:0/2002371475 wait complete. 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 -- 172.21.15.53:0/288606625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558025a7fc00 msgr2=0x55802685cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558025a7fc00 0x55802685cb00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x5580268afb60 tx=0x558026824a20 comp rx=0 tx=0).stop 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 -- 172.21.15.53:0/288606625 shutdown_connections 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558025a7fc00 0x55802685cb00 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558025a7f800 0x55802685c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 --2- 172.21.15.53:0/288606625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558025a7ec00 0x55802685c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 -- 172.21.15.53:0/288606625 >> 172.21.15.53:0/288606625 conn(0x558025a7f000 msgr2=0x5580267fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 -- 172.21.15.53:0/288606625 shutdown_connections 2024-05-10T10:40:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:21.996+0000 7fa790c0f740 1 -- 172.21.15.53:0/288606625 wait complete. 2024-05-10T10:40:22.002 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.091324, result: 2024-05-10T10:40:22.003 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.083+0000 7fcac020c740 1 -- start start 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.083+0000 7fcac020c740 1 Processor -- start 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.083+0000 7fcac020c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a651d6000 0x557a651c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.083+0000 7fcac020c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a643ebc00 0x557a651c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.083+0000 7fcac020c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a651d6800 0x557a651c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.083+0000 7fcac020c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557a651ea480 con 0x557a651d6000 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcac020c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a651ea600 con 0x557a651d6800 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcac020c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557a651ea780 con 0x557a643ebc00 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a651d6000 0x557a651c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a651d6000 0x557a651c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41344/0 (socket says 172.21.15.53:41344) 2024-05-10T10:40:22.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 -- 172.21.15.53:0/3608699317 learned_addr learned my addr 172.21.15.53:0/3608699317 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:22.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a651d6800 0x557a651c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a643ebc00 0x557a651c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 -- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a643ebc00 msgr2=0x557a651c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a643ebc00 0x557a651c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 -- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a651d6800 msgr2=0x557a651c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a651d6800 0x557a651c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 -- 172.21.15.53:0/3608699317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a64417680 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a643ebc00 0x557a651c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a651d6000 0x557a651c5080 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x557a652217d0 tx=0x557a65190a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40fb164b4530e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabb649640 1 -- 172.21.15.53:0/3608699317 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a64410b60 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabb649640 1 -- 172.21.15.53:0/3608699317 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a644321c0 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabb649640 1 -- 172.21.15.53:0/3608699317 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a64410d00 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcac020c740 1 -- 172.21.15.53:0/3608699317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a644172c0 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcac020c740 1 -- 172.21.15.53:0/3608699317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x557a64416b40 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcac020c740 4 waiting for MDS map... 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.084+0000 7fcabb649640 1 -- 172.21.15.53:0/3608699317 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x557a651f2400 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.085+0000 7fcabb649640 1 -- 172.21.15.53:0/3608699317 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x557a64443180 con 0x557a651d6000 2024-05-10T10:40:22.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.085+0000 7fcac020c740 4 Got MDS map 69 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.085+0000 7fcac020c740 4 datascan.main: Using metadata pool output 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.085+0000 7fcac020c740 4 datascan.main: connecting to RADOS... 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.085+0000 7fcac020c740 1 Processor -- start 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.085+0000 7fcac020c740 1 -- start start 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcac020c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3400 0x557a651c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcac020c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3800 0x557a651c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcac020c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcac020c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557a6444a180 con 0x557a652e3c00 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcac020c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a651eaf00 con 0x557a652e3400 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcac020c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557a651eb080 con 0x557a652e3800 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41358/0 (socket says 172.21.15.53:41358) 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 -- 172.21.15.53:0/1273778673 learned_addr learned my addr 172.21.15.53:0/1273778673 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabe64f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3400 0x557a651c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabde4e640 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3800 0x557a651c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 -- 172.21.15.53:0/1273778673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3800 msgr2=0x557a651c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3800 0x557a651c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 -- 172.21.15.53:0/1273778673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3400 msgr2=0x557a651c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3400 0x557a651c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 -- 172.21.15.53:0/1273778673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a644163c0 con 0x557a652e3c00 2024-05-10T10:40:22.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.086+0000 7fcabd64d640 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6100 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x557a652e6cc0 tx=0x557a652eb800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54c1be1ba472a6d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcab9645640 1 -- 172.21.15.53:0/1273778673 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a64410b60 con 0x557a652e3c00 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcab9645640 1 -- 172.21.15.53:0/1273778673 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a64432c40 con 0x557a652e3c00 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcab9645640 1 -- 172.21.15.53:0/1273778673 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a64410d00 con 0x557a652e3c00 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 -- 172.21.15.53:0/1273778673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 msgr2=0x557a651c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6100 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x557a652e6cc0 tx=0x557a652eb800 comp rx=0 tx=0).stop 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 -- 172.21.15.53:0/1273778673 shutdown_connections 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6100 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3800 0x557a651c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 --2- 172.21.15.53:0/1273778673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3400 0x557a651c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 -- 172.21.15.53:0/1273778673 >> 172.21.15.53:0/1273778673 conn(0x557a651d6400 msgr2=0x557a65169800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 -- 172.21.15.53:0/1273778673 shutdown_connections 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.087+0000 7fcac020c740 1 -- 172.21.15.53:0/1273778673 wait complete. 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 Processor -- start 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 -- start start 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3400 0x557a651c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3800 0x557a651c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557a651eb380 con 0x557a652e3c00 2024-05-10T10:40:22.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a651eb200 con 0x557a652e3800 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcac020c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557a651eb500 con 0x557a652e3400 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcabd64d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcabd64d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41362/0 (socket says 172.21.15.53:41362) 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 learned_addr learned my addr 172.21.15.53:0/3949867235 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3800 0x557a651c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.088+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3400 0x557a651c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3400 msgr2=0x557a651c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3400 0x557a651c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3800 msgr2=0x557a651c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a652e3800 0x557a651c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a64417a40 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a652e3400 0x557a651c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x557a652e7980 tx=0x557a65191f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcab7e42640 1 -- 172.21.15.53:0/3949867235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a644104e0 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcab7e42640 1 -- 172.21.15.53:0/3949867235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a64432a80 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557a644174a0 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcab7e42640 1 -- 172.21.15.53:0/3949867235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a64410b60 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a64417c20 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.089+0000 7fcac020c740 4 datascan.init: resolving metadata pool 13 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.090+0000 7fcab7e42640 1 -- 172.21.15.53:0/3949867235 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x557a65206c00 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcab7e42640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x557a65344c00 0x557a651c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcab7e42640 1 -- 172.21.15.53:0/3949867235 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x557a65204a00 con 0x557a652e3c00 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcac020c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-10T10:40:22.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcac020c740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcac020c740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x557a65345000 0x557a651c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x557a65345400 con 0x557a65345000 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x557a65344c00 0x557a651c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x557a65345000 0x557a651c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x557a65345000 0x557a651c6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x557a65344c00 0x557a651c7700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x557a65192810 tx=0x557a6471af60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.091+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.092+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x557a65345800 0x557a651c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.092+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x557a65345c00 con 0x557a65345800 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.092+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x557a65345800 0x557a651c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.092+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x557a65345800 0x557a651c4580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.092+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.092+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x557a647a6000 0x557a651c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.092+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6400 con 0x557a647a6000 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.093+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x557a647a6000 0x557a651c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.093+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x557a647a6000 0x557a651c5600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.093+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.093+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7c00 con 0x557a65345000 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.093+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcab9645640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x557a652e2c00 0x557a647b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2800 con 0x557a652e2c00 2024-05-10T10:40:22.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x557a652e2c00 0x557a647b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x557a652e2c00 0x557a647b6000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6400 con 0x557a65345000 2024-05-10T10:40:22.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.094+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2400 con 0x557a65345000 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.095+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.095+0000 7fcab8e44640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x557a65345400 0x557a647b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.095+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7c00 con 0x557a65345400 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.095+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x557a65345400 0x557a647b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.095+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x557a65345400 0x557a647b6580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.095+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.095+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc000 con 0x557a652e2c00 2024-05-10T10:40:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.096+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.096+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bcc00 con 0x557a652e2c00 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.096+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.096+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc400 con 0x557a65345800 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.096+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.096+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a652e2c00 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.097+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.097+0000 7fcab9645640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x557a647bc800 0x557a647b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.097+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc000 con 0x557a647bc800 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.097+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x557a647bc800 0x557a647b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.097+0000 7fcabde4e640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x557a647bc800 0x557a647b6b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd400 con 0x557a647a6000 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7c00 con 0x557a65345000 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd800 con 0x557a652e2c00 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.098+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a65345c00 con 0x557a65345000 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.099+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.099+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bcc00 con 0x557a652e2c00 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.099+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.099+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2800 con 0x557a65345800 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.099+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.099+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a647bc800 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.100+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.100+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc000 con 0x557a647a6000 2024-05-10T10:40:22.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.100+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.100+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc400 con 0x557a647bc800 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.100+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.100+0000 7fcab9645640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x557a65345c00 0x557a647b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.100+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6400 con 0x557a65345c00 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.101+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x557a65345c00 0x557a647b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.101+0000 7fcabd64d640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x557a65345c00 0x557a647b7080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.101+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.101+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bdc00 con 0x557a647bc800 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.102+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.102+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7c00 con 0x557a65345000 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.102+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.102+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd400 con 0x557a65345000 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.102+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.102+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2400 con 0x557a65345000 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.103+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.103+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a647bc800 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.103+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.103+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0000 con 0x557a65345400 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.104+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.104+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1800 con 0x557a647a6000 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.104+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.104+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0400 con 0x557a647bc800 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.104+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.104+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1c00 con 0x557a65345400 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.105+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.105+0000 7fcab9645640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x557a647d0800 0x557a647b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.105+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0c00 con 0x557a647d0800 2024-05-10T10:40:22.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.105+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x557a647d0800 0x557a647b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.105+0000 7fcabe64f640 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x557a647d0800 0x557a647b7600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.105+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.105+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc000 con 0x557a65345c00 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.106+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.106+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1000 con 0x557a65345400 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.106+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.106+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc400 con 0x557a65345c00 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.106+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 91 out=49b] v87'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.106+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1400 con 0x557a647bc800 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.107+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.107+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bdc00 con 0x557a652e2c00 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.107+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.107+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0000 con 0x557a65345c00 2024-05-10T10:40:22.147 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.107+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 91 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.107+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd400 con 0x557a65345800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.108+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.108+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1c00 con 0x557a65345800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.108+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.108+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a647d0800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.108+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.108+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd800 con 0x557a65345000 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.109+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.109+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2800 con 0x557a647a6000 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.109+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.109+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bcc00 con 0x557a647d0800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.109+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.109+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2400 con 0x557a647bc800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.110+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.110+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc000 con 0x557a65345800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.110+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.111+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7c00 con 0x557a647bc800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.111+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.111+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc400 con 0x557a65345000 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.112+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.112+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1000 con 0x557a65345400 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.112+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.112+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6400 con 0x557a65345000 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.112+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.112+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1c00 con 0x557a647d0800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.112+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.113+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647da000 con 0x557a647a6000 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.113+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.113+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0c00 con 0x557a652e2c00 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.113+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.113+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647da400 con 0x557a65345000 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.113+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.113+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1800 con 0x557a647bc800 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.114+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.114+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647da800 con 0x557a652e2c00 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.114+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.114+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd400 con 0x557a647bc800 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.114+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.115+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 91 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647dac00 con 0x557a65345c00 2024-05-10T10:40:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.115+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 91 out=81b] v91'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.115+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6800 con 0x557a647a6000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.115+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.115+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647db000 con 0x557a65345000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.115+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.115+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a652e2c00 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.116+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.116+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647db400 con 0x557a65345000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.116+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.116+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bcc00 con 0x557a65345000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.116+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.116+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647db800 con 0x557a65345400 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bdc00 con 0x557a652e2c00 2024-05-10T10:40:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647dbc00 con 0x557a652e2c00 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 12 ==== osd_op_reply(67 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a65345800 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.117+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647da800 con 0x557a652e2c00 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 13 ==== osd_op_reply(69 [pgnls-filter start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd800 con 0x557a647bc800 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647dac00 con 0x557a647a6000 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc400 con 0x557a65345000 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.118+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647da000 con 0x557a652e2c00 2024-05-10T10:40:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.119+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 14 ==== osd_op_reply(73 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.119+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2800 con 0x557a65345000 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.119+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.119+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7c00 con 0x557a652e2c00 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.119+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.119+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2400 con 0x557a65345800 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.120+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.120+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6400 con 0x557a647bc800 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.120+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.120+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0000 con 0x557a647a6000 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.120+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.120+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647db800 con 0x557a647bc800 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.121+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.121+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0c00 con 0x557a65345c00 2024-05-10T10:40:22.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.121+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 6 ==== osd_op_reply(80 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.121+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647da400 con 0x557a647bc800 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.121+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.121+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0400 con 0x557a65345000 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.122+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.122+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647db000 con 0x557a65345000 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.122+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.122+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1400 con 0x557a65345000 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.122+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.122+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647db400 con 0x557a647bc800 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.123+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.123+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc000 con 0x557a65345400 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.123+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.123+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647dac00 con 0x557a647a6000 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.123+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.123+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a647bc800 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.124+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.124+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6800 con 0x557a65345400 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.124+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.124+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd400 con 0x557a647d0800 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.124+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.124+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1000 con 0x557a65345c00 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.125+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.125+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bcc00 con 0x557a65345400 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.125+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.125+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1c00 con 0x557a65345c00 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.125+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 8 ==== osd_op_reply(93 [pgnls-filter start_epoch 91 out=49b] v87'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.125+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd800 con 0x557a647bc800 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.126+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.126+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1800 con 0x557a652e2c00 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.126+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 16 ==== osd_op_reply(95 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.126+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc400 con 0x557a65345c00 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.126+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 9 ==== osd_op_reply(96 [pgnls-filter start_epoch 91 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.127+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2400 con 0x557a65345800 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.127+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.127+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bdc00 con 0x557a65345800 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.127+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.127+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e4000 con 0x557a647d0800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.127+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.127+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd000 con 0x557a65345000 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.128+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 91 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.128+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e4400 con 0x557a647a6000 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.128+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.128+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0c00 con 0x557a647d0800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.128+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 7 ==== osd_op_reply(102 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.128+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e4800 con 0x557a647bc800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.129+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.129+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1000 con 0x557a65345800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.129+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.129+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e4c00 con 0x557a647bc800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.129+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.129+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0000 con 0x557a65345000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.130+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.130+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e5000 con 0x557a65345400 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.130+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.130+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d0400 con 0x557a65345000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.130+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.130+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e5400 con 0x557a647d0800 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 8 ==== osd_op_reply(109 [pgnls-filter start_epoch 91 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647d1400 con 0x557a647a6000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.2 v2:172.21.15.53:6800/2235813461 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 91 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647a6000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e5800 con 0x557a652e2c00 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 91 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7c00 con 0x557a65345000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.131+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e5c00 con 0x557a647bc800 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.132+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.132+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6400 con 0x557a652e2c00 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.132+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 18 ==== osd_op_reply(114 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.132+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647e4000 con 0x557a647bc800 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 91 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2800 con 0x557a65345c00 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 10 ==== osd_op_reply(116 [pgnls-filter start_epoch 91 out=81b] v91'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcac020c740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcac020c740 10 datascan.operator(): handling object 10000000001.0 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7000 con 0x557a65345c00 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 11 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcac020c740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-05-10T10:40:22.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcac020c740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-05-10T10:40:22.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcac020c740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-05-10T10:40:22.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.133+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7400 con 0x557a652e2c00 2024-05-10T10:40:22.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 19 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-05-10T10:40:22.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7800 con 0x557a65345800 2024-05-10T10:40:22.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.6 v2:172.21.15.135:6808/1123855098 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x557a6517c500 con 0x557a65345800 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 20 datascan.get_frag_of: frag is * 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6400 con 0x557a652e2c00 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 20 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.134+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x557a647a7000 con 0x557a652e2c00 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.135+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 21 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.135+0000 7fcac020c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.135+0000 7fcac020c740 20 datascan.read_dentry: file_head not found in result 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.135+0000 7fcac020c740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.135+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x557a647a6c00 con 0x557a652e2c00 2024-05-10T10:40:22.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.136+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 22 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v91'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.136+0000 7fcac020c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.136+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a652e2800 con 0x557a65345c00 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.136+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.5 v2:172.21.15.135:6824/3244007917 12 ==== osd_op_reply(123 [pgnls-filter start_epoch 91 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a65345c00 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.136+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647f0000 con 0x557a652e2c00 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.136+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 <== osd.1 v2:172.21.15.53:6824/2705510088 23 ==== osd_op_reply(124 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c780 con 0x557a652e2c00 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.136+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bc000 con 0x557a647bc800 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.137+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.4 v2:172.21.15.135:6800/3517077655 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 91 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a647bc800 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.137+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647f0400 con 0x557a647d0800 2024-05-10T10:40:22.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.137+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.7 v2:172.21.15.135:6809/751247261 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 91 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a647d0800 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.137+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bd400 con 0x557a65345000 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcab8e44640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647f0800 con 0x557a65345000 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcabde4e640 1 -- 172.21.15.53:0/3949867235 <== osd.0 v2:172.21.15.53:6802/1634417257 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 91 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c000 con 0x557a65345000 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcab9645640 1 -- 172.21.15.53:0/3949867235 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 91 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x557a647bcc00 con 0x557a65345400 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcabe64f640 1 -- 172.21.15.53:0/3949867235 <== osd.3 v2:172.21.15.53:6816/2871564334 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 91 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a6517c500 con 0x557a65345400 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x557a65345000 msgr2=0x557a651c6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x557a65345000 0x557a651c6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x557a652e2c00 msgr2=0x557a647b6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x557a652e2c00 0x557a647b6000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x557a647a6000 msgr2=0x557a651c5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x557a647a6000 0x557a651c5600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x557a65345400 msgr2=0x557a647b6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x557a65345400 0x557a647b6580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x557a647bc800 msgr2=0x557a647b6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x557a647bc800 0x557a647b6b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 reap_dead start 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x557a65345c00 msgr2=0x557a647b7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x557a65345c00 0x557a647b7080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x557a65345800 msgr2=0x557a651c4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcabd64d640 1 -- 172.21.15.53:0/3949867235 reap_dead start 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x557a65345800 0x557a651c4580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x557a647d0800 msgr2=0x557a647b7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x557a647d0800 0x557a647b7600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x557a65344c00 msgr2=0x557a651c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x557a65344c00 0x557a651c7700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x557a65192810 tx=0x557a6471af60 comp rx=0 tx=0).stop 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 msgr2=0x557a651c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x557a652e7980 tx=0x557a65191f80 comp rx=0 tx=0).stop 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 shutdown_connections 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x557a647d0800 0x557a647b7600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x557a65344c00 0x557a651c7700 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x557a65345800 0x557a651c4580 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 --2- 172.21.15.53:0/3949867235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a652e3c00 0x557a651c6680 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 >> 172.21.15.53:0/3949867235 conn(0x557a651d6400 msgr2=0x557a65169800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 shutdown_connections 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.138+0000 7fcac020c740 1 -- 172.21.15.53:0/3949867235 wait complete. 2024-05-10T10:40:22.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 -- 172.21.15.53:0/3608699317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a651d6000 msgr2=0x557a651c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a651d6000 0x557a651c5080 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x557a652217d0 tx=0x557a65190a80 comp rx=0 tx=0).stop 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 -- 172.21.15.53:0/3608699317 shutdown_connections 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a651d6800 0x557a651c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557a643ebc00 0x557a651c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 --2- 172.21.15.53:0/3608699317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557a651d6000 0x557a651c5080 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 -- 172.21.15.53:0/3608699317 >> 172.21.15.53:0/3608699317 conn(0x557a643eb000 msgr2=0x557a65169000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 -- 172.21.15.53:0/3608699317 shutdown_connections 2024-05-10T10:40:22.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.139+0000 7fcac020c740 1 -- 172.21.15.53:0/3608699317 wait complete. 2024-05-10T10:40:22.168 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.165249, result: 2024-05-10T10:40:22.168 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-10T10:40:22.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.247+0000 7fbd4a430740 1 -- start start 2024-05-10T10:40:22.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.247+0000 7fbd4a430740 1 Processor -- start 2024-05-10T10:40:22.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab613c6c00 0x55ab62196000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab613c7800 0x55ab62196580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab613c7c00 0x55ab62196b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd4a430740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ab621ca180 con 0x55ab613c7800 2024-05-10T10:40:22.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd4a430740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ab61426900 con 0x55ab613c7c00 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd4a430740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ab61426480 con 0x55ab613c6c00 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab613c7800 0x55ab62196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab613c7800 0x55ab62196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41370/0 (socket says 172.21.15.53:41370) 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 -- 172.21.15.53:0/3807686724 learned_addr learned my addr 172.21.15.53:0/3807686724 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd484e4640 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab613c7c00 0x55ab62196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab613c6c00 0x55ab62196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 -- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab613c6c00 msgr2=0x55ab62196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab613c6c00 0x55ab62196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 -- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab613c7c00 msgr2=0x55ab62196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab613c7c00 0x55ab62196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 -- 172.21.15.53:0/3807686724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab613f3680 con 0x55ab613c7800 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd474e2640 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab613c7800 0x55ab62196580 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55ab621ff410 tx=0x55ab6216cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499ed29e0db10dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd454de640 1 -- 172.21.15.53:0/3807686724 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab613ec9c0 con 0x55ab613c7800 2024-05-10T10:40:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.248+0000 7fbd454de640 1 -- 172.21.15.53:0/3807686724 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ab6140e1c0 con 0x55ab613c7800 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd454de640 1 -- 172.21.15.53:0/3807686724 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab613ec680 con 0x55ab613c7800 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 1 -- 172.21.15.53:0/3807686724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab613f3860 con 0x55ab613c7800 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 1 -- 172.21.15.53:0/3807686724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ab613f32c0 con 0x55ab613c7800 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 4 waiting for MDS map... 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd454de640 1 -- 172.21.15.53:0/3807686724 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x55ab621ac200 con 0x55ab613c7800 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd454de640 1 -- 172.21.15.53:0/3807686724 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x55ab6141f180 con 0x55ab613c7800 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 4 Got MDS map 69 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 4 datascan.main: Using metadata pool output 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 4 datascan.main: connecting to RADOS... 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 1 Processor -- start 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.249+0000 7fbd4a430740 1 -- start start 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee400 0x55ab62197600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd4a430740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ab61426180 con 0x55ab622ee800 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd4a430740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ab621cad80 con 0x55ab622ee400 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd4a430740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ab621caf00 con 0x55ab622ee000 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41378/0 (socket says 172.21.15.53:41378) 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 -- 172.21.15.53:0/2744851639 learned_addr learned my addr 172.21.15.53:0/2744851639 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee400 0x55ab62197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd484e4640 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 -- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee000 msgr2=0x55ab62197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 -- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee400 msgr2=0x55ab62197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee400 0x55ab62197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd474e2640 1 -- 172.21.15.53:0/2744851639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab613f34a0 con 0x55ab622ee800 2024-05-10T10:40:22.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.250+0000 7fbd484e4640 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd474e2640 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x55ab621e44b0 tx=0x55ab6216c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f7c56219c4518 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd434da640 1 -- 172.21.15.53:0/2744851639 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab613ecb60 con 0x55ab622ee800 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd434da640 1 -- 172.21.15.53:0/2744851639 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ab6140ee00 con 0x55ab622ee800 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd434da640 1 -- 172.21.15.53:0/2744851639 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab613ec820 con 0x55ab622ee800 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 -- 172.21.15.53:0/2744851639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 msgr2=0x55ab62197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x55ab621e44b0 tx=0x55ab6216c600 comp rx=0 tx=0).stop 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 -- 172.21.15.53:0/2744851639 shutdown_connections 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee400 0x55ab62197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 --2- 172.21.15.53:0/2744851639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 -- 172.21.15.53:0/2744851639 >> 172.21.15.53:0/2744851639 conn(0x55ab621cdc00 msgr2=0x55ab62145000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 -- 172.21.15.53:0/2744851639 shutdown_connections 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.251+0000 7fbd4a430740 1 -- 172.21.15.53:0/2744851639 wait complete. 2024-05-10T10:40:22.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 Processor -- start 2024-05-10T10:40:22.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 -- start start 2024-05-10T10:40:22.266 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee400 0x55ab62197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ab621cb200 con 0x55ab622ee800 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ab621cb080 con 0x55ab622ee000 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd4a430740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ab621cb380 con 0x55ab622ee400 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.252+0000 7fbd474e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41392/0 (socket says 172.21.15.53:41392) 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 learned_addr learned my addr 172.21.15.53:0/1664133994 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd484e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee400 0x55ab62197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee400 msgr2=0x55ab62197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab622ee400 0x55ab62197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee000 msgr2=0x55ab62197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab622ee000 0x55ab62197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab613f30e0 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55ab621e5d10 tx=0x55ab614968a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd41cd7640 1 -- 172.21.15.53:0/1664133994 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab613ecb60 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd41cd7640 1 -- 172.21.15.53:0/1664133994 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ab6140e540 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab613f3a40 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd41cd7640 1 -- 172.21.15.53:0/1664133994 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab613ec820 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab613f23c0 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.253+0000 7fbd4a430740 4 datascan.init: resolving metadata pool 13 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.254+0000 7fbd41cd7640 1 -- 172.21.15.53:0/1664133994 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55ab62310c00 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd41cd7640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ab61762000 0x55ab62198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd41cd7640 1 -- 172.21.15.53:0/1664133994 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x55ab622f4c00 con 0x55ab622ee800 2024-05-10T10:40:22.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd4a430740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd4a430740 4 datascan.main: resolving metadata pool 13 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ab622eec00 0x55ab62198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab622ef000 con 0x55ab622eec00 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ab61762000 0x55ab62198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.255+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ab622eec00 0x55ab62198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.256+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ab61762000 0x55ab62198100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55ab6216f080 tx=0x55ab616e6c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.256+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ab622eec00 0x55ab62198680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.256+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(1 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.256+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ab622ef000 0x55ab61776000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.256+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab61763000 con 0x55ab622ef000 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.256+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ab622ef000 0x55ab61776000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.256+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ab622ef000 0x55ab61776000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.257+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(2 [pgnls start_epoch 91 out=177b] v91'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.257+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ab61763000 0x55ab61776580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.257+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab61762c00 con 0x55ab61763000 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.257+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ab61763000 0x55ab61776580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.257+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ab61763000 0x55ab61776580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.258+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== osd_op_reply(3 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763000 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.258+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab61762c00 con 0x55ab622eec00 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.258+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== osd_op_reply(4 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.258+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ab61762c00 0x55ab61776b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.258+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab61763400 con 0x55ab61762c00 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.258+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ab61762c00 0x55ab61776b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.259+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ab61762c00 0x55ab61776b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.259+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(5 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.259+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ab61763400 0x55ab61777600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.259+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab61763c00 con 0x55ab61763400 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.259+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ab61763400 0x55ab61777600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.259+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ab61763400 0x55ab61777600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(6 [pgnls start_epoch 91 out=149b] v90'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab61763c00 con 0x55ab622ef000 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== osd_op_reply(7 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab61763c00 con 0x55ab61762c00 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(8 [pgnls start_epoch 91 out=73b] v90'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ab61763c00 0x55ab61777b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e000 con 0x55ab61763c00 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.260+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ab61763c00 0x55ab61777b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.261+0000 7fbd47ce3640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ab61763c00 0x55ab61777b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.261+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(9 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.261+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e000 con 0x55ab61763400 2024-05-10T10:40:22.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.261+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(10 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.261+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e000 con 0x55ab61763000 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.262+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== osd_op_reply(11 [pgnls start_epoch 91 out=81b] v90'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763000 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.262+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e000 con 0x55ab622ef000 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.262+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 3 ==== osd_op_reply(12 [pgnls start_epoch 91 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.262+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ab6178e000 0x55ab61778680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.262+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178f800 con 0x55ab6178e000 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.262+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ab6178e000 0x55ab61778680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.263+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ab6178e000 0x55ab61778680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.263+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(13 [pgnls start_epoch 91 out=77b] v90'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.263+0000 7fbd474e2640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ab6178f800 0x55ab61778100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.263+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178f800 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.263+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ab6178f800 0x55ab61778100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.263+0000 7fbd484e4640 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ab6178f800 0x55ab61778100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.264+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== osd_op_reply(14 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab6178f800 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.264+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab622eec00 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.264+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 3 ==== osd_op_reply(15 [pgnls start_epoch 91 out=73b] v90'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.264+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178e000 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.264+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== osd_op_reply(16 [pgnls start_epoch 91 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.264+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab61762c00 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(17 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab622ef000 2024-05-10T10:40:22.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 4 ==== osd_op_reply(18 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab61763c00 2024-05-10T10:40:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== osd_op_reply(19 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178f800 2024-05-10T10:40:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== osd_op_reply(20 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab6178f800 2024-05-10T10:40:22.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.265+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178e000 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.266+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 3 ==== osd_op_reply(21 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.266+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178e000 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.266+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 4 ==== osd_op_reply(22 [pgnls start_epoch 91 out=97b] v90'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.266+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178e000 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.266+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 5 ==== osd_op_reply(23 [pgnls start_epoch 91 out=74b] v86'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.266+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab622eec00 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.267+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 4 ==== osd_op_reply(24 [pgnls start_epoch 91 out=73b] v90'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.267+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab61763400 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.267+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 3 ==== osd_op_reply(25 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.267+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab622eec00 2024-05-10T10:40:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.267+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 5 ==== osd_op_reply(26 [pgnls start_epoch 91 out=98b] v90'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.267+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab61763c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.268+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 3 ==== osd_op_reply(27 [pgnls start_epoch 91 out=76b] v90'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.268+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178e000 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.268+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 6 ==== osd_op_reply(28 [pgnls start_epoch 91 out=73b] v90'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.268+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab622eec00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.269+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 6 ==== osd_op_reply(29 [pgnls start_epoch 91 out=73b] v90'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.269+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab61763c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.269+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 4 ==== osd_op_reply(30 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.269+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab61762c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.269+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 4 ==== osd_op_reply(31 [pgnls start_epoch 91 out=73b] v86'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab61762c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 5 ==== osd_op_reply(32 [pgnls start_epoch 91 out=101b] v90'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c000 con 0x55ab622ef000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c400 con 0x55ab622ef000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.270+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622ef000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 20 datascan.scan_links: injected ino detected 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-10T10:40:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179cc00 con 0x55ab622ef000 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d000 con 0x55ab622ef000 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d400 con 0x55ab61763400 2024-05-10T10:40:22.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.271+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c000 con 0x55ab61762c00 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c400 con 0x55ab61763000 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.2 v2:172.21.15.53:6800/2235813461 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763000 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.272+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622eec00 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-10T10:40:22.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179cc00 con 0x55ab6178e000 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d800 con 0x55ab622eec00 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.273+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-10T10:40:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179dc00 con 0x55ab622eec00 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178fc00 con 0x55ab622eec00 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-10T10:40:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.274+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c400 con 0x55ab61762c00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.275+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.275+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622eec00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.275+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 11 ==== osd_op_reply(47 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.275+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622ef000 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.275+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 10 ==== osd_op_reply(48 [pgnls start_epoch 91 out=177b] v91'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.275+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763000 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.276+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.2 v2:172.21.15.53:6800/2235813461 4 ==== osd_op_reply(49 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763000 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.276+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622eec00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.276+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 12 ==== osd_op_reply(50 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.276+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61762c00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.276+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 8 ==== osd_op_reply(51 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.276+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763400 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.277+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 5 ==== osd_op_reply(52 [pgnls start_epoch 91 out=149b] v90'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.277+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622ef000 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.277+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 11 ==== osd_op_reply(53 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.277+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61762c00 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.277+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 9 ==== osd_op_reply(54 [pgnls start_epoch 91 out=73b] v90'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.277+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763c00 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.277+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 5 ==== osd_op_reply(55 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.278+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763400 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.278+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 6 ==== osd_op_reply(56 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.278+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763000 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.278+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.2 v2:172.21.15.53:6800/2235813461 5 ==== osd_op_reply(57 [pgnls start_epoch 91 out=81b] v90'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763000 2024-05-10T10:40:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.278+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622ef000 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.278+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 12 ==== osd_op_reply(58 [pgnls start_epoch 91 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.278+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178e000 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.279+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 8 ==== osd_op_reply(59 [pgnls start_epoch 91 out=77b] v90'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.279+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178f800 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.279+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.3 v2:172.21.15.53:6816/2871564334 3 ==== osd_op_reply(60 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab6178f800 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.279+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622eec00 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.279+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 13 ==== osd_op_reply(61 [pgnls start_epoch 91 out=73b] v90'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.279+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178e000 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.280+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 9 ==== osd_op_reply(62 [pgnls start_epoch 91 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.280+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61762c00 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.280+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 10 ==== osd_op_reply(63 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.280+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622ef000 2024-05-10T10:40:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.280+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 13 ==== osd_op_reply(64 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.280+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763c00 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.281+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 6 ==== osd_op_reply(65 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.281+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178f800 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.281+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.3 v2:172.21.15.53:6816/2871564334 4 ==== osd_op_reply(66 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab6178f800 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.281+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178e000 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.281+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 10 ==== osd_op_reply(67 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.281+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178e000 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.282+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 11 ==== osd_op_reply(68 [pgnls start_epoch 91 out=97b] v90'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.282+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178e000 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.282+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 12 ==== osd_op_reply(69 [pgnls start_epoch 91 out=74b] v86'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.282+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622eec00 2024-05-10T10:40:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.282+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 14 ==== osd_op_reply(70 [pgnls start_epoch 91 out=73b] v90'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.282+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763400 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.283+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 7 ==== osd_op_reply(71 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.283+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622eec00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.283+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 15 ==== osd_op_reply(72 [pgnls start_epoch 91 out=98b] v90'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.283+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763c00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.284+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 7 ==== osd_op_reply(73 [pgnls start_epoch 91 out=76b] v90'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.284+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab6178e000 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.284+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 13 ==== osd_op_reply(74 [pgnls start_epoch 91 out=73b] v90'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.284+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab622eec00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.284+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 16 ==== osd_op_reply(75 [pgnls start_epoch 91 out=73b] v90'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.284+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61763c00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.285+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.5 v2:172.21.15.135:6824/3244007917 8 ==== osd_op_reply(76 [pgnls start_epoch 91 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763c00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.285+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61762c00 2024-05-10T10:40:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.285+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 11 ==== osd_op_reply(77 [pgnls start_epoch 91 out=73b] v86'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.285+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 91 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c800 con 0x55ab61762c00 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.285+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 12 ==== osd_op_reply(78 [pgnls start_epoch 91 out=101b] v90'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.285+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.285+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d800 con 0x55ab622ef000 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab61763800 con 0x55ab622ef000 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-10T10:40:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c000 con 0x55ab622ef000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 20 datascan.scan_links: injected ino detected 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179cc00 con 0x55ab622ef000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.286+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c400 con 0x55ab622ef000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178f400 con 0x55ab61763400 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.0 v2:172.21.15.53:6802/1634417257 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158500 con 0x55ab61763400 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178f000 con 0x55ab61762c00 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.287+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-05-10T10:40:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178ec00 con 0x55ab61763000 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.2 v2:172.21.15.53:6800/2235813461 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55ab62158000 con 0x55ab61763000 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e800 con 0x55ab622eec00 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.288+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e400 con 0x55ab6178e000 2024-05-10T10:40:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.289+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.289+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.289+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.289+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.289+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.289+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.289+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d000 con 0x55ab622eec00 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d400 con 0x55ab622eec00 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-10T10:40:22.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-10T10:40:22.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-05-10T10:40:22.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-10T10:40:22.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-10T10:40:22.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d800 con 0x55ab622eec00 2024-05-10T10:40:22.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.290+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179c000 con 0x55ab61762c00 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.6 v2:172.21.15.135:6808/1123855098 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x55ab62158780 con 0x55ab61762c00 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd4a430740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd4a430740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd4a430740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd4a430740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179cc00 con 0x55ab622ef000 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd4a430740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-10T10:40:22.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.291+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55ab61763800 con 0x55ab622ef000 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v91'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 10 datascan.scan_links: processing 1 injected_inos 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179dc00 con 0x55ab622ef000 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 20 datascan.scan_links: first is now 2 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.292+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55ab6179d000 con 0x55ab622ef000 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.293+0000 7fbd484e4640 1 -- 172.21.15.53:0/1664133994 <== osd.1 v2:172.21.15.53:6824/2705510088 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v91'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ab62158500 con 0x55ab622ef000 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.293+0000 7fbd4a430740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.293+0000 7fbd4a430740 10 datascan.scan_links: updating inotable 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.293+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178fc00 con 0x55ab622eec00 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.294+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.294+0000 7fbd4a430740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.294+0000 7fbd4a430740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.294+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178ec00 con 0x55ab622eec00 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.295+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 <== osd.7 v2:172.21.15.135:6809/751247261 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v91'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55ab62158000 con 0x55ab622eec00 2024-05-10T10:40:22.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.295+0000 7fbd4a430740 10 datascan.scan_links: updating snaptable 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.295+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55ab6178e800 con 0x55ab6178e000 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd474e2640 1 -- 172.21.15.53:0/1664133994 <== osd.4 v2:172.21.15.135:6800/3517077655 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55ab62158780 con 0x55ab6178e000 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ab61763400 msgr2=0x55ab61777600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ab61763400 0x55ab61777600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ab622ef000 msgr2=0x55ab61776000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ab622ef000 0x55ab61776000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ab61763000 msgr2=0x55ab61776580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ab61763000 0x55ab61776580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ab6178f800 msgr2=0x55ab61778100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ab6178f800 0x55ab61778100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ab6178e000 msgr2=0x55ab61778680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ab6178e000 0x55ab61778680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 reap_dead start 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ab61763c00 msgr2=0x55ab61777b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 reap_dead start 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ab61763c00 0x55ab61777b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 reap_dead start 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ab61762c00 msgr2=0x55ab61776b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ab61762c00 0x55ab61776b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ab622eec00 msgr2=0x55ab62198680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ab622eec00 0x55ab62198680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ab61762000 msgr2=0x55ab62198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ab61762000 0x55ab62198100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55ab6216f080 tx=0x55ab616e6c00 comp rx=0 tx=0).stop 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 msgr2=0x55ab62197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/1664133994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab622ee800 0x55ab62197b80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55ab621e5d10 tx=0x55ab614968a0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd47ce3640 1 -- 172.21.15.53:0/1664133994 reap_dead start 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 shutdown_connections 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 >> 172.21.15.53:0/1664133994 conn(0x55ab621cdc00 msgr2=0x55ab62145000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 shutdown_connections 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/1664133994 wait complete. 2024-05-10T10:40:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/3807686724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab613c7800 msgr2=0x55ab62196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab613c7800 0x55ab62196580 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55ab621ff410 tx=0x55ab6216cc60 comp rx=0 tx=0).stop 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/3807686724 shutdown_connections 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab613c7c00 0x55ab62196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab613c7800 0x55ab62196580 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 --2- 172.21.15.53:0/3807686724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ab613c6c00 0x55ab62196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.296+0000 7fbd4a430740 1 -- 172.21.15.53:0/3807686724 >> 172.21.15.53:0/3807686724 conn(0x55ab613c7000 msgr2=0x55ab62144800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.297+0000 7fbd4a430740 1 -- 172.21.15.53:0/3807686724 shutdown_connections 2024-05-10T10:40:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.297+0000 7fbd4a430740 1 -- 172.21.15.53:0/3807686724 wait complete. 2024-05-10T10:40:22.326 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.157380, result: 2024-05-10T10:40:22.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-10T10:40:22.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.446+0000 7fa9581f0640 1 Processor -- start 2024-05-10T10:40:22.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.447+0000 7fa9581f0640 1 -- start start 2024-05-10T10:40:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.447+0000 7fa9581f0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 0x7fa95006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.447+0000 7fa9581f0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa95006e920 0x7fa95006d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.447+0000 7fa9581f0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95006d920 con 0x7fa95015f190 2024-05-10T10:40:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.447+0000 7fa9581f0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa95006dc00 con 0x7fa95006e920 2024-05-10T10:40:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.447+0000 7fa9581f0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa9500c6b80 con 0x7fa9500d0080 2024-05-10T10:40:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa955f8c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa95015f190 0x7fa9500cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37598/0 (socket says 172.21.15.53:37598) 2024-05-10T10:40:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa955f8c640 1 -- 172.21.15.53:0/3240712501 learned_addr learned my addr 172.21.15.53:0/3240712501 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735445408 0 0) 0x7fa95006d920 con 0x7fa95015f190 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa938003520 con 0x7fa95015f190 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa95678d640 1 --2- 172.21.15.53:0/3240712501 >> v2:172.21.15.135:3300/0 conn(0x7fa95006e920 0x7fa95006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa95578b640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 0x7fa95006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1496053859 0 0) 0x7fa938003520 con 0x7fa95015f190 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 msgr2=0x7fa95006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 0x7fa95006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 >> v2:172.21.15.135:3300/0 conn(0x7fa95006e920 msgr2=0x7fa95006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 --2- 172.21.15.53:0/3240712501 >> v2:172.21.15.135:3300/0 conn(0x7fa95006e920 0x7fa95006d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9500c6e60 con 0x7fa95015f190 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fa940001a10 con 0x7fa95015f190 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 >> v1:172.21.15.53:6789/0 conn(0x7fa95015f190 legacy=0x7fa9500cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9380039b0 0x7fa938003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa938004800 0x7fa938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 0x7fa9380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa95006dc00 con 0x7fa9380039b0 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9500c6b80 con 0x7fa938004800 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa93800b9c0 con 0x7fa9500d0080 2024-05-10T10:40:22.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa955f8c640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9380039b0 0x7fa938003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.448+0000 7fa95578b640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa938004800 0x7fa938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.449+0000 7fa95678d640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 0x7fa9380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.449+0000 7fa955f8c640 1 -- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 msgr2=0x7fa9380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.449+0000 7fa955f8c640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9500d0080 0x7fa9380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.449+0000 7fa955f8c640 1 -- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa938004800 msgr2=0x7fa938004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.449+0000 7fa955f8c640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa938004800 0x7fa938004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.449+0000 7fa955f8c640 1 -- 172.21.15.53:0/3240712501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa938003520 con 0x7fa9380039b0 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.449+0000 7fa955f8c640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9380039b0 0x7fa938003dd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa94000a870 tx=0x7fa938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7923a89a439790aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94000a4c0 con 0x7fa9380039b0 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94002f070 con 0x7fa9380039b0 2024-05-10T10:40:22.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa954f8a640 1 -- 172.21.15.53:0/3240712501 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94002bcc0 con 0x7fa9380039b0 2024-05-10T10:40:22.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa9581f0640 1 -- 172.21.15.53:0/3240712501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9380039b0 msgr2=0x7fa938003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa9581f0640 1 --2- 172.21.15.53:0/3240712501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9380039b0 0x7fa938003dd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa94000a870 tx=0x7fa938005830 comp rx=0 tx=0).stop 2024-05-10T10:40:22.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa95678d640 1 -- 172.21.15.53:0/3240712501 reap_dead start 2024-05-10T10:40:22.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa9581f0640 1 -- 172.21.15.53:0/3240712501 shutdown_connections 2024-05-10T10:40:22.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa9581f0640 1 -- 172.21.15.53:0/3240712501 >> 172.21.15.53:0/3240712501 conn(0x7fa9500ce0a0 msgr2=0x7fa9500ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:22.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa9581f0640 1 -- 172.21.15.53:0/3240712501 shutdown_connections 2024-05-10T10:40:22.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.450+0000 7fa9581f0640 1 -- 172.21.15.53:0/3240712501 wait complete. 2024-05-10T10:40:22.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.451+0000 7fa9581f0640 1 Processor -- start 2024-05-10T10:40:22.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.451+0000 7fa9581f0640 1 -- start start 2024-05-10T10:40:22.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa9581f0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9380039b0 0x7fa9500b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa9581f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9500b2660 0x7fa95014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa9581f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa950150120 0x7fa95014ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa9581f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa95006ccd0 con 0x7fa950150120 2024-05-10T10:40:22.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa9581f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa95015f570 con 0x7fa9500b2660 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa9581f0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa95006ce50 con 0x7fa9380039b0 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa95678d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa950150120 0x7fa95014ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa95678d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa950150120 0x7fa95014ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41416/0 (socket says 172.21.15.53:41416) 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa95678d640 1 -- 172.21.15.53:0/842485915 learned_addr learned my addr 172.21.15.53:0/842485915 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.452+0000 7fa955f8c640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9380039b0 0x7fa9500b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa95578b640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9500b2660 0x7fa95014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa95678d640 1 -- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9380039b0 msgr2=0x7fa9500b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa95678d640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9380039b0 0x7fa9500b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa95678d640 1 -- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9500b2660 msgr2=0x7fa95014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa95678d640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9500b2660 0x7fa95014fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa95678d640 1 -- 172.21.15.53:0/842485915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95014dcf0 con 0x7fa950150120 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa95678d640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa950150120 0x7fa95014ed90 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa9500ce070 tx=0x7fa948011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa94e7fc640 1 -- 172.21.15.53:0/842485915 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948004240 con 0x7fa950150120 2024-05-10T10:40:22.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa94e7fc640 1 -- 172.21.15.53:0/842485915 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94801b070 con 0x7fa950150120 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95014dfd0 con 0x7fa950150120 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa94e7fc640 1 -- 172.21.15.53:0/842485915 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94800f740 con 0x7fa950150120 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.453+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95014e590 con 0x7fa950150120 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.454+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95006a220 con 0x7fa950150120 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.455+0000 7fa94e7fc640 1 -- 172.21.15.53:0/842485915 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa948012040 con 0x7fa950150120 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.455+0000 7fa94e7fc640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa93c076300 0x7fa93c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.455+0000 7fa94e7fc640 1 -- 172.21.15.53:0/842485915 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fa948097650 con 0x7fa950150120 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.455+0000 7fa955f8c640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa93c076300 0x7fa93c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:22.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.456+0000 7fa955f8c640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa93c076300 0x7fa93c0787c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa940033000 tx=0x7fa940007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:22.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.459+0000 7fa94e7fc640 1 -- 172.21.15.53:0/842485915 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa948060c70 con 0x7fa950150120 2024-05-10T10:40:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.689+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa95015f930 con 0x7fa950150120 2024-05-10T10:40:23.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.901+0000 7fa94e7fc640 1 -- 172.21.15.53:0/842485915 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v70) ==== 162+0+0 (secure 0 0 0) 0x7fa948060570 con 0x7fa950150120 2024-05-10T10:40:23.000 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-10T10:40:23.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa93c076300 msgr2=0x7fa93c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa93c076300 0x7fa93c0787c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa940033000 tx=0x7fa940007000 comp rx=0 tx=0).stop 2024-05-10T10:40:23.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa950150120 msgr2=0x7fa95014ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa950150120 0x7fa95014ed90 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa9500ce070 tx=0x7fa948011cf0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 shutdown_connections 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa93c076300 0x7fa93c0787c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa950150120 0x7fa95014ed90 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9500b2660 0x7fa95014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 --2- 172.21.15.53:0/842485915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9380039b0 0x7fa9500b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 >> 172.21.15.53:0/842485915 conn(0x7fa9500ce0a0 msgr2=0x7fa9500cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.903+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 shutdown_connections 2024-05-10T10:40:23.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:22.904+0000 7fa9581f0640 1 -- 172.21.15.53:0/842485915 wait complete. 2024-05-10T10:40:23.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:40:23.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.107+0000 7fe356322640 1 Processor -- start 2024-05-10T10:40:23.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.107+0000 7fe356322640 1 -- start start 2024-05-10T10:40:23.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe356322640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe35015b6d0 0x7fe3500cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe356322640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3500c6db0 0x7fe3500cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe356322640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe3500cf6b0 0x7fe35015f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe356322640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe3500c5d10 con 0x7fe3500cf6b0 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe356322640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe3500c5ff0 con 0x7fe35015b6d0 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe356322640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe3500c62d0 con 0x7fe3500c6db0 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe3500cf6b0 0x7fe35015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe3500cf6b0 0x7fe35015f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47256/0 (socket says 172.21.15.53:47256) 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 -- 172.21.15.53:0/1206146521 learned_addr learned my addr 172.21.15.53:0/1206146521 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe34f7fe640 1 --2- 172.21.15.53:0/1206146521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3500c6db0 0x7fe3500cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe34ffff640 1 --2- 172.21.15.53:0/1206146521 >> v2:172.21.15.135:3300/0 conn(0x7fe35015b6d0 0x7fe3500cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 -- 172.21.15.53:0/1206146521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3500c6db0 msgr2=0x7fe3500cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 --2- 172.21.15.53:0/1206146521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3500c6db0 0x7fe3500cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 -- 172.21.15.53:0/1206146521 >> v2:172.21.15.135:3300/0 conn(0x7fe35015b6d0 msgr2=0x7fe3500cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 --2- 172.21.15.53:0/1206146521 >> v2:172.21.15.135:3300/0 conn(0x7fe35015b6d0 0x7fe3500cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe3548bf640 1 -- 172.21.15.53:0/1206146521 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3500c65b0 con 0x7fe3500cf6b0 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe34f7fe640 1 --2- 172.21.15.53:0/1206146521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3500c6db0 0x7fe3500cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.108+0000 7fe34ffff640 1 --2- 172.21.15.53:0/1206146521 >> v2:172.21.15.135:3300/0 conn(0x7fe35015b6d0 0x7fe3500cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.109+0000 7fe3548bf640 1 --2- 172.21.15.53:0/1206146521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35015f120 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe3500cb2b0 tx=0x7fe34802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aa2967e46082cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.109+0000 7fe34effd640 1 -- 172.21.15.53:0/1206146521 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348036070 con 0x7fe3500cf6b0 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.109+0000 7fe34effd640 1 -- 172.21.15.53:0/1206146521 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe348031600 con 0x7fe3500cf6b0 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.109+0000 7fe34effd640 1 -- 172.21.15.53:0/1206146521 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3480319d0 con 0x7fe3500cf6b0 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 -- 172.21.15.53:0/1206146521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 msgr2=0x7fe35015f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 --2- 172.21.15.53:0/1206146521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35015f120 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe3500cb2b0 tx=0x7fe34802a6a0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 -- 172.21.15.53:0/1206146521 shutdown_connections 2024-05-10T10:40:23.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 --2- 172.21.15.53:0/1206146521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35015f120 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 --2- 172.21.15.53:0/1206146521 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3500c6db0 0x7fe3500cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 --2- 172.21.15.53:0/1206146521 >> v2:172.21.15.135:3300/0 conn(0x7fe35015b6d0 0x7fe3500cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 -- 172.21.15.53:0/1206146521 >> 172.21.15.53:0/1206146521 conn(0x7fe3500b2900 msgr2=0x7fe3500b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 -- 172.21.15.53:0/1206146521 shutdown_connections 2024-05-10T10:40:23.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.110+0000 7fe356322640 1 -- 172.21.15.53:0/1206146521 wait complete. 2024-05-10T10:40:23.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.111+0000 7fe356322640 1 Processor -- start 2024-05-10T10:40:23.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.111+0000 7fe356322640 1 -- start start 2024-05-10T10:40:23.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe356322640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3500c6db0 0x7fe3501f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe356322640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35009bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe356322640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe35015b6d0 0x7fe3501efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe356322640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe3500cd2f0 con 0x7fe3500cf6b0 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe356322640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3500cd170 con 0x7fe3500c6db0 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe356322640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe3500cdc10 con 0x7fe35015b6d0 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35009bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35009bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47258/0 (socket says 172.21.15.53:47258) 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 -- 172.21.15.53:0/2894394455 learned_addr learned my addr 172.21.15.53:0/2894394455 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34ffff640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3500c6db0 0x7fe3501f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe3548bf640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe35015b6d0 0x7fe3501efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 -- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe35015b6d0 msgr2=0x7fe3501efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe35015b6d0 0x7fe3501efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 -- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3500c6db0 msgr2=0x7fe3501f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3500c6db0 0x7fe3501f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34f7fe640 1 -- 172.21.15.53:0/2894394455 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3500ade80 con 0x7fe3500cf6b0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.112+0000 7fe34ffff640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3500c6db0 0x7fe3501f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.113+0000 7fe34f7fe640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35009bc90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c7c0 tx=0x7fe34400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.113+0000 7fe34cff9640 1 -- 172.21.15.53:0/2894394455 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344015070 con 0x7fe3500cf6b0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.113+0000 7fe34cff9640 1 -- 172.21.15.53:0/2894394455 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe344007a30 con 0x7fe3500cf6b0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.113+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3500ae160 con 0x7fe3500cf6b0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.113+0000 7fe34cff9640 1 -- 172.21.15.53:0/2894394455 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344007e00 con 0x7fe3500cf6b0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.113+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3500ae720 con 0x7fe3500cf6b0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.114+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3500b2ce0 con 0x7fe3500cf6b0 2024-05-10T10:40:23.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.114+0000 7fe34cff9640 1 -- 172.21.15.53:0/2894394455 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe344018070 con 0x7fe3500cf6b0 2024-05-10T10:40:23.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.115+0000 7fe34cff9640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe33c0762d0 0x7fe33c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:23.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.115+0000 7fe34cff9640 1 -- 172.21.15.53:0/2894394455 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fe344097790 con 0x7fe3500cf6b0 2024-05-10T10:40:23.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.115+0000 7fe34ffff640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe33c0762d0 0x7fe33c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:23.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.116+0000 7fe34ffff640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe33c0762d0 0x7fe33c078790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe34000a990 tx=0x7fe340008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:23.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.120+0000 7fe34cff9640 1 -- 172.21.15.53:0/2894394455 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe3500b2ce0 con 0x7fe3500cf6b0 2024-05-10T10:40:23.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.349+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3500ccc70 con 0x7fe3500cf6b0 2024-05-10T10:40:23.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.350+0000 7fe34cff9640 1 -- 172.21.15.53:0/2894394455 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5180 (secure 0 0 0) 0x7fe344060ed0 con 0x7fe3500cf6b0 2024-05-10T10:40:23.352 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:23.352 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:22.904604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-10T10:40:23.352 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 71 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe33c0762d0 msgr2=0x7fe33c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe33c0762d0 0x7fe33c078790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe34000a990 tx=0x7fe340008040 comp rx=0 tx=0).stop 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 msgr2=0x7fe35009bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35009bc90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c7c0 tx=0x7fe34400cc80 comp rx=0 tx=0).stop 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 shutdown_connections 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe33c0762d0 0x7fe33c078790 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe35015b6d0 0x7fe3501efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3500cf6b0 0x7fe35009bc90 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 --2- 172.21.15.53:0/2894394455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3500c6db0 0x7fe3501f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 >> 172.21.15.53:0/2894394455 conn(0x7fe3500b2900 msgr2=0x7fe3500cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:23.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 shutdown_connections 2024-05-10T10:40:23.355 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:23.352+0000 7fe356322640 1 -- 172.21.15.53:0/2894394455 wait complete. 2024-05-10T10:40:23.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:38:57.607866+0000', 'modified': '2024-05-10T10:40:22.904604+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7152}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7152': {'gid': 7152, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/1098976714', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1098976714}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1098976714}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:40:23.768 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7152:up:replay 2024-05-10T10:40:24.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:40:24.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.885+0000 7fde0a245640 1 Processor -- start 2024-05-10T10:40:24.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.885+0000 7fde0a245640 1 -- start start 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde0a245640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162750 0x7fde040d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde0a245640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde040b45f0 con 0x7fde040d5cb0 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde0a245640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde040b48d0 con 0x7fde04162b30 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde0a245640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fde040b4f20 con 0x7fde04162750 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde02ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fde040d5cb0 0x7fde040d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34010/0 (socket says 172.21.15.53:34010) 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde02ffd640 1 -- 172.21.15.53:0/3487273650 learned_addr learned my addr 172.21.15.53:0/3487273650 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde037fe640 1 --2- 172.21.15.53:0/3487273650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162750 0x7fde040d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800293120 0 0) 0x7fde040b45f0 con 0x7fde040d5cb0 2024-05-10T10:40:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddec003520 con 0x7fde040d5cb0 2024-05-10T10:40:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 89750996 0 0) 0x7fde040b48d0 con 0x7fde04162b30 2024-05-10T10:40:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde040b45f0 con 0x7fde04162b30 2024-05-10T10:40:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde037fe640 1 -- 172.21.15.53:0/3487273650 >> v1:172.21.15.135:6789/0 conn(0x7fde04162b30 legacy=0x7fde040b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2616476139 0 0) 0x7fddec003520 con 0x7fde040d5cb0 2024-05-10T10:40:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde037fe640 1 -- 172.21.15.53:0/3487273650 >> v1:172.21.15.53:6789/0 conn(0x7fde040d5cb0 legacy=0x7fde040d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.886+0000 7fde037fe640 1 -- 172.21.15.53:0/3487273650 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde040b4bb0 con 0x7fde04162750 2024-05-10T10:40:24.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.887+0000 7fde037fe640 1 --2- 172.21.15.53:0/3487273650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162750 0x7fde040d0670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddf40097d0 tx=0x7fddf402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa5b2763e3d96f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:24.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.887+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4037070 con 0x7fde04162750 2024-05-10T10:40:24.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.887+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf402ab90 con 0x7fde04162750 2024-05-10T10:40:24.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde027fc640 1 -- 172.21.15.53:0/3487273650 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8001670 con 0x7fde04162750 2024-05-10T10:40:24.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde0a245640 1 -- 172.21.15.53:0/3487273650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162750 msgr2=0x7fde040d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde0a245640 1 --2- 172.21.15.53:0/3487273650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162750 0x7fde040d0670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddf40097d0 tx=0x7fddf402a100 comp rx=0 tx=0).stop 2024-05-10T10:40:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde0a245640 1 -- 172.21.15.53:0/3487273650 shutdown_connections 2024-05-10T10:40:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde0a245640 1 --2- 172.21.15.53:0/3487273650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162750 0x7fde040d0670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde0a245640 1 -- 172.21.15.53:0/3487273650 >> 172.21.15.53:0/3487273650 conn(0x7fde0409e3b0 msgr2=0x7fde040d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde0a245640 1 -- 172.21.15.53:0/3487273650 shutdown_connections 2024-05-10T10:40:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.888+0000 7fde0a245640 1 -- 172.21.15.53:0/3487273650 wait complete. 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.889+0000 7fde0a245640 1 Processor -- start 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde0a245640 1 -- start start 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde0a245640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde04162750 0x7fde04157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde0a245640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162b30 0x7fde040c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde0a245640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde040cdff0 0x7fde040b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde0a245640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fde040d3d40 con 0x7fde04162750 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde0a245640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fde040d0fe0 con 0x7fde040cdff0 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde0a245640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fde040d3ec0 con 0x7fde04162b30 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde04162750 0x7fde04157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:24.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde04162750 0x7fde04157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47262/0 (socket says 172.21.15.53:47262) 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.890+0000 7fde037fe640 1 -- 172.21.15.53:0/2865701636 learned_addr learned my addr 172.21.15.53:0/2865701636 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde02ffd640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162b30 0x7fde040c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde03fff640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde040cdff0 0x7fde040b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde037fe640 1 -- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162b30 msgr2=0x7fde040c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde037fe640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162b30 0x7fde040c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde037fe640 1 -- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde040cdff0 msgr2=0x7fde040b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde037fe640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde040cdff0 0x7fde040b9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde037fe640 1 -- 172.21.15.53:0/2865701636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde040c90c0 con 0x7fde04162750 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde03fff640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde040cdff0 0x7fde040b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde02ffd640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162b30 0x7fde040c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde037fe640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde04162750 0x7fde04157930 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fddf40097d0 tx=0x7fddf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fddebfff640 1 -- 172.21.15.53:0/2865701636 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4037050 con 0x7fde04162750 2024-05-10T10:40:24.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fddebfff640 1 -- 172.21.15.53:0/2865701636 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf4038040 con 0x7fde04162750 2024-05-10T10:40:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fddebfff640 1 -- 172.21.15.53:0/2865701636 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4006e10 con 0x7fde04162750 2024-05-10T10:40:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde040ce840 con 0x7fde04162750 2024-05-10T10:40:24.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.891+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde040b9410 con 0x7fde04162750 2024-05-10T10:40:24.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.892+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0406a220 con 0x7fde04162750 2024-05-10T10:40:24.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.893+0000 7fddebfff640 1 -- 172.21.15.53:0/2865701636 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fddf4045020 con 0x7fde04162750 2024-05-10T10:40:24.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.893+0000 7fddebfff640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddf00762d0 0x7fddf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:24.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.893+0000 7fddebfff640 1 -- 172.21.15.53:0/2865701636 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fddf40b78e0 con 0x7fde04162750 2024-05-10T10:40:24.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.894+0000 7fde02ffd640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddf00762d0 0x7fddf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:24.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.894+0000 7fde02ffd640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddf00762d0 0x7fddf0078790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fddf80017e0 tx=0x7fddf8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:24.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:24.898+0000 7fddebfff640 1 -- 172.21.15.53:0/2865701636 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fddf404c020 con 0x7fde04162750 2024-05-10T10:40:25.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.134+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde040b5d00 con 0x7fde04162750 2024-05-10T10:40:25.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.134+0000 7fddebfff640 1 -- 172.21.15.53:0/2865701636 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5180 (secure 0 0 0) 0x7fddf4080f90 con 0x7fde04162750 2024-05-10T10:40:25.136 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:25.137 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:23.911301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-10T10:40:25.137 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 73 2024-05-10T10:40:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddf00762d0 msgr2=0x7fddf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddf00762d0 0x7fddf0078790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fddf80017e0 tx=0x7fddf8009040 comp rx=0 tx=0).stop 2024-05-10T10:40:25.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde04162750 msgr2=0x7fde04157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde04162750 0x7fde04157930 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fddf40097d0 tx=0x7fddf40046c0 comp rx=0 tx=0).stop 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 shutdown_connections 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fddf00762d0 0x7fddf0078790 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde040cdff0 0x7fde040b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde04162b30 0x7fde040c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 --2- 172.21.15.53:0/2865701636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde04162750 0x7fde04157930 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.136+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 >> 172.21.15.53:0/2865701636 conn(0x7fde0409e3b0 msgr2=0x7fde040d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.137+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 shutdown_connections 2024-05-10T10:40:25.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:25.137+0000 7fde0a245640 1 -- 172.21.15.53:0/2865701636 wait complete. 2024-05-10T10:40:25.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:38:57.607866+0000', 'modified': '2024-05-10T10:40:23.911301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7152}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7152': {'gid': 7152, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.53:6833/1098976714', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1098976714}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1098976714}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:40:25.553 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7152:up:rejoin 2024-05-10T10:40:26.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:40:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.670+0000 7f90a25a4640 1 Processor -- start 2024-05-10T10:40:26.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.670+0000 7f90a25a4640 1 -- start start 2024-05-10T10:40:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f90a25a4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f909c0b8860 0x7f909c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f90a25a4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f909c15fc90 con 0x7f909c0d0230 2024-05-10T10:40:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f90a25a4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f909c0b8c40 con 0x7f909c0cfd80 2024-05-10T10:40:26.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f90a25a4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f909c06ee50 con 0x7f909c0b8860 2024-05-10T10:40:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f909bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f909c0b8860 0x7f909c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f909b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f909c0d0230 0x7f909c0cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34026/0 (socket says 172.21.15.53:34026) 2024-05-10T10:40:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f909b7fe640 1 -- 172.21.15.53:0/255752969 learned_addr learned my addr 172.21.15.53:0/255752969 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f909bfff640 1 -- 172.21.15.53:0/255752969 >> v1:172.21.15.135:6789/0 conn(0x7f909c0cfd80 legacy=0x7f909c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.671+0000 7f909bfff640 1 -- 172.21.15.53:0/255752969 >> v1:172.21.15.53:6789/0 conn(0x7f909c0d0230 legacy=0x7f909c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.672+0000 7f909bfff640 1 -- 172.21.15.53:0/255752969 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c0cd570 con 0x7f909c0b8860 2024-05-10T10:40:26.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.672+0000 7f909affd640 1 -- 172.21.15.53:0/255752969 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030319362 0 0) 0x7f909c15fc90 con 0x7f909c0d0230 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.672+0000 7f909bfff640 1 --2- 172.21.15.53:0/255752969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c06e520 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f908c0097a0 tx=0x7f908c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf35ae391de83cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.672+0000 7f909affd640 1 -- 172.21.15.53:0/255752969 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c037070 con 0x7f909c0b8860 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.672+0000 7f909affd640 1 -- 172.21.15.53:0/255752969 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908c02ae50 con 0x7f909c0b8860 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.672+0000 7f909affd640 1 -- 172.21.15.53:0/255752969 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c032660 con 0x7f909c0b8860 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.673+0000 7f90a25a4640 1 -- 172.21.15.53:0/255752969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 msgr2=0x7f909c06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.673+0000 7f90a25a4640 1 --2- 172.21.15.53:0/255752969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c06e520 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f908c0097a0 tx=0x7f908c02a140 comp rx=0 tx=0).stop 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.673+0000 7f90a25a4640 1 -- 172.21.15.53:0/255752969 shutdown_connections 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.673+0000 7f90a25a4640 1 --2- 172.21.15.53:0/255752969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c06e520 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.673+0000 7f90a25a4640 1 -- 172.21.15.53:0/255752969 >> 172.21.15.53:0/255752969 conn(0x7f909c0d2df0 msgr2=0x7f909c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.673+0000 7f90a25a4640 1 -- 172.21.15.53:0/255752969 shutdown_connections 2024-05-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.673+0000 7f90a25a4640 1 -- 172.21.15.53:0/255752969 wait complete. 2024-05-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.674+0000 7f90a25a4640 1 Processor -- start 2024-05-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f90a25a4640 1 -- start start 2024-05-10T10:40:26.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f90a25a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:26.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f90a25a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c0cfd80 0x7f909c158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f90a25a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f909c0d0230 0x7f909c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f90a25a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f909c0cde10 con 0x7f909c0b8860 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f90a25a4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f909c0cdb30 con 0x7f909c0cfd80 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f90a25a4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f909c06ea60 con 0x7f909c0d0230 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f909bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.675+0000 7f909bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47278/0 (socket says 172.21.15.53:47278) 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909bfff640 1 -- 172.21.15.53:0/2168849848 learned_addr learned my addr 172.21.15.53:0/2168849848 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909b7fe640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c0cfd80 0x7f909c158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f90a0b41640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f909c0d0230 0x7f909c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909bfff640 1 -- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f909c0d0230 msgr2=0x7f909c0ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:26.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909bfff640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f909c0d0230 0x7f909c0ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909bfff640 1 -- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c0cfd80 msgr2=0x7f909c158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:26.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909bfff640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c0cfd80 0x7f909c158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909bfff640 1 -- 172.21.15.53:0/2168849848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c0caec0 con 0x7f909c0b8860 2024-05-10T10:40:26.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f909bfff640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c0cb300 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f908c0098c0 tx=0x7f908c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:26.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f9098ff9640 1 -- 172.21.15.53:0/2168849848 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c042070 con 0x7f909c0b8860 2024-05-10T10:40:26.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f9098ff9640 1 -- 172.21.15.53:0/2168849848 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908c03e670 con 0x7f909c0b8860 2024-05-10T10:40:26.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c0cb920 con 0x7f909c0b8860 2024-05-10T10:40:26.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f9098ff9640 1 -- 172.21.15.53:0/2168849848 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c037040 con 0x7f909c0b8860 2024-05-10T10:40:26.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.676+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c0cbee0 con 0x7f909c0b8860 2024-05-10T10:40:26.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.677+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909c06a220 con 0x7f909c0b8860 2024-05-10T10:40:26.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.678+0000 7f9098ff9640 1 -- 172.21.15.53:0/2168849848 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f908c045070 con 0x7f909c0b8860 2024-05-10T10:40:26.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.678+0000 7f9098ff9640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f90880762d0 0x7f9088078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:26.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.679+0000 7f9098ff9640 1 -- 172.21.15.53:0/2168849848 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f908c0b80e0 con 0x7f909c0b8860 2024-05-10T10:40:26.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.679+0000 7f909b7fe640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f90880762d0 0x7f9088078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:26.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.679+0000 7f909b7fe640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f90880762d0 0x7f9088078790 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f909c15f0a0 tx=0x7f9090007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:26.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.683+0000 7f9098ff9640 1 -- 172.21.15.53:0/2168849848 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f908c081770 con 0x7f909c0b8860 2024-05-10T10:40:26.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.915+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f909c0d1e20 con 0x7f909c0b8860 2024-05-10T10:40:26.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.916+0000 7f9098ff9640 1 -- 172.21.15.53:0/2168849848 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5187 (secure 0 0 0) 0x7f908c081070 con 0x7f909c0b8860 2024-05-10T10:40:26.920 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:26.920 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:25.917116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7152,"qdb_cluster":[7152]},"id":6}]} 2024-05-10T10:40:26.920 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 74 2024-05-10T10:40:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f90880762d0 msgr2=0x7f9088078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f90880762d0 0x7f9088078790 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f909c15f0a0 tx=0x7f9090007c00 comp rx=0 tx=0).stop 2024-05-10T10:40:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 msgr2=0x7f909c0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c0cb300 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f908c0098c0 tx=0x7f908c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 shutdown_connections 2024-05-10T10:40:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f90880762d0 0x7f9088078790 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f909c0d0230 0x7f909c0ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c0cfd80 0x7f909c158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 --2- 172.21.15.53:0/2168849848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f909c0b8860 0x7f909c0cb300 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 >> 172.21.15.53:0/2168849848 conn(0x7f909c0d2df0 msgr2=0x7f909c0d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 shutdown_connections 2024-05-10T10:40:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:26.918+0000 7f90a25a4640 1 -- 172.21.15.53:0/2168849848 wait complete. 2024-05-10T10:40:27.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:38:57.607866+0000', 'modified': '2024-05-10T10:40:25.917116+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7152}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7152': {'gid': 7152, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6833/1098976714', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1098976714}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1098976714}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7152, 'qdb_cluster': [7152]} 2024-05-10T10:40:27.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:40:27.348 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T10:40:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.461+0000 7f8325bef640 1 Processor -- start 2024-05-10T10:40:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.461+0000 7f8325bef640 1 -- start start 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f8325bef640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83201598c0 0x7f832006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f8325bef640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f832006f1c0 con 0x7f832006ec40 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f8325bef640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f832006f4a0 con 0x7f83200c7390 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f8325bef640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f832006f780 con 0x7f83201598c0 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f831f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83201598c0 0x7f832006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f831effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f832006ec40 0x7f83200c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34028/0 (socket says 172.21.15.53:34028) 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f831effd640 1 -- 172.21.15.53:0/2373961399 learned_addr learned my addr 172.21.15.53:0/2373961399 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f831f7fe640 1 -- 172.21.15.53:0/2373961399 >> v1:172.21.15.135:6789/0 conn(0x7f83200c7390 legacy=0x7f83200cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f831f7fe640 1 -- 172.21.15.53:0/2373961399 >> v1:172.21.15.53:6789/0 conn(0x7f832006ec40 legacy=0x7f83200c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f831f7fe640 1 -- 172.21.15.53:0/2373961399 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832006fa60 con 0x7f83201598c0 2024-05-10T10:40:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.462+0000 7f831e7fc640 1 -- 172.21.15.53:0/2373961399 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152708691 0 0) 0x7f832006f1c0 con 0x7f832006ec40 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.463+0000 7f831f7fe640 1 --2- 172.21.15.53:0/2373961399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f832006e700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f83100097a0 tx=0x7f831002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad62663b78c4bebd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.463+0000 7f831e7fc640 1 -- 172.21.15.53:0/2373961399 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310037070 con 0x7f83201598c0 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.463+0000 7f831e7fc640 1 -- 172.21.15.53:0/2373961399 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310032440 con 0x7f83201598c0 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.463+0000 7f831e7fc640 1 -- 172.21.15.53:0/2373961399 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310032810 con 0x7f83201598c0 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.464+0000 7f8325bef640 1 -- 172.21.15.53:0/2373961399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 msgr2=0x7f832006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.464+0000 7f8325bef640 1 --2- 172.21.15.53:0/2373961399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f832006e700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f83100097a0 tx=0x7f831002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.464+0000 7f8325bef640 1 -- 172.21.15.53:0/2373961399 shutdown_connections 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.464+0000 7f8325bef640 1 --2- 172.21.15.53:0/2373961399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f832006e700 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.464+0000 7f8325bef640 1 -- 172.21.15.53:0/2373961399 >> 172.21.15.53:0/2373961399 conn(0x7f83200b2900 msgr2=0x7f83200b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.464+0000 7f8325bef640 1 -- 172.21.15.53:0/2373961399 shutdown_connections 2024-05-10T10:40:27.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.464+0000 7f8325bef640 1 -- 172.21.15.53:0/2373961399 wait complete. 2024-05-10T10:40:27.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.465+0000 7f8325bef640 1 Processor -- start 2024-05-10T10:40:27.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.465+0000 7f8325bef640 1 -- start start 2024-05-10T10:40:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f8325bef640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f832006ec40 0x7f83200cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f8325bef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83200c7390 0x7f8320072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f8325bef640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f83201566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f8325bef640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8320159ca0 con 0x7f83201598c0 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f8325bef640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f832015c6e0 con 0x7f83200c7390 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f8325bef640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8320159e20 con 0x7f832006ec40 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f831ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f83201566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f831ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f83201566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47302/0 (socket says 172.21.15.53:47302) 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.466+0000 7f831ffff640 1 -- 172.21.15.53:0/2583044674 learned_addr learned my addr 172.21.15.53:0/2583044674 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831effd640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83200c7390 0x7f8320072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831f7fe640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f832006ec40 0x7f83200cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831ffff640 1 -- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f832006ec40 msgr2=0x7f83200cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831ffff640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f832006ec40 0x7f83200cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831ffff640 1 -- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83200c7390 msgr2=0x7f8320072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831ffff640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83200c7390 0x7f8320072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831ffff640 1 -- 172.21.15.53:0/2583044674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8320073ee0 con 0x7f83201598c0 2024-05-10T10:40:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f831ffff640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f83201566d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f831800c700 tx=0x7f831800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f8307fff640 1 -- 172.21.15.53:0/2583044674 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318016020 con 0x7f83201598c0 2024-05-10T10:40:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f8307fff640 1 -- 172.21.15.53:0/2583044674 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f831800aa50 con 0x7f83201598c0 2024-05-10T10:40:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8320073920 con 0x7f83201598c0 2024-05-10T10:40:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f8307fff640 1 -- 172.21.15.53:0/2583044674 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831800ce10 con 0x7f83201598c0 2024-05-10T10:40:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.467+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8320071f20 con 0x7f83201598c0 2024-05-10T10:40:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.469+0000 7f8307fff640 1 -- 172.21.15.53:0/2583044674 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8318019020 con 0x7f83201598c0 2024-05-10T10:40:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.469+0000 7f8307fff640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f830c0762d0 0x7f830c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.469+0000 7f8307fff640 1 -- 172.21.15.53:0/2583044674 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f8318097a70 con 0x7f83201598c0 2024-05-10T10:40:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.470+0000 7f831f7fe640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f830c0762d0 0x7f830c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.470+0000 7f831f7fe640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f830c0762d0 0x7f830c078790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8310034000 tx=0x7f831002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.470+0000 7f83289e0740 1 Processor -- start 2024-05-10T10:40:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.470+0000 7f83289e0740 1 -- start start 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f83289e0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ab8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ab8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2a88bb0 0x5572f2a8c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f83289e0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5572f29f28d0 con 0x5572f2ab5f50 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5572f29abc10 con 0x5572f2a88bb0 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5572f29ee890 con 0x5572f2ab6330 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ab8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ab8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47314/0 (socket says 172.21.15.53:47314) 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.471+0000 7f831f7fe640 1 -- 172.21.15.53:0/888281998 learned_addr learned my addr 172.21.15.53:0/888281998 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831effd640 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ab8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831f7fe640 1 -- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 msgr2=0x5572f2ab8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831f7fe640 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ab8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831ffff640 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2a88bb0 0x5572f2a8c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831f7fe640 1 -- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2a88bb0 msgr2=0x5572f2a8c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831f7fe640 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2a88bb0 0x5572f2a8c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831f7fe640 1 -- 172.21.15.53:0/888281998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f29b35d0 con 0x5572f2ab5f50 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831effd640 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ab8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f831f7fe640 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ab8760 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8310009560 tx=0x7f831002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd65c3e9d2af35b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f83057fa640 1 -- 172.21.15.53:0/888281998 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310037040 con 0x5572f2ab5f50 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f83057fa640 1 -- 172.21.15.53:0/888281998 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310032bb0 con 0x5572f2ab5f50 2024-05-10T10:40:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.472+0000 7f83057fa640 1 -- 172.21.15.53:0/888281998 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831003c440 con 0x5572f2ab5f50 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 -- 172.21.15.53:0/888281998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 msgr2=0x5572f2ab8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ab8760 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8310009560 tx=0x7f831002c000 comp rx=0 tx=0).stop 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 -- 172.21.15.53:0/888281998 shutdown_connections 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2a88bb0 0x5572f2a8c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ab8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 --2- 172.21.15.53:0/888281998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ab8760 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 -- 172.21.15.53:0/888281998 >> 172.21.15.53:0/888281998 conn(0x5572f2a8ae20 msgr2=0x5572f2ad3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 -- 172.21.15.53:0/888281998 shutdown_connections 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 -- 172.21.15.53:0/888281998 wait complete. 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 -- start start 2024-05-10T10:40:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.473+0000 7f83289e0740 1 Processor -- start 2024-05-10T10:40:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f83289e0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ad2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 0x5572f2ab9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f83289e0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5572f2a8dfd0 con 0x5572f2ab5f50 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5572f29af760 con 0x5572f2aac0b0 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5572f2a8e2b0 con 0x5572f2ab6330 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f83289e0740 10 client.0 _ll_register_callbacks cb 0x5572f29e8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47316/0 (socket says 172.21.15.53:47316) 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f831f7fe640 1 -- 172.21.15.53:0/1691416568 learned_addr learned my addr 172.21.15.53:0/1691416568 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f831ffff640 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 0x5572f2ab9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.474+0000 7f831effd640 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ad2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f831f7fe640 1 -- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 msgr2=0x5572f2ad2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f831f7fe640 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ad2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f831f7fe640 1 -- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 msgr2=0x5572f2ab9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f831f7fe640 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 0x5572f2ab9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f831f7fe640 1 -- 172.21.15.53:0/1691416568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f27042d0 con 0x5572f2ab5f50 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f831f7fe640 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad2440 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f83100098c0 tx=0x7f8310004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9572fc160a3e8533 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f82fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f82fe7fc640 1 -- 172.21.15.53:0/1691416568 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310057070 con 0x5572f2ab5f50 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f82fe7fc640 1 -- 172.21.15.53:0/1691416568 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f83100042d0 con 0x5572f2ab5f50 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8310032bb0 con 0x5572f2ab5f50 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f82fe7fc640 1 -- 172.21.15.53:0/1691416568 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f831005f050 con 0x5572f2ab5f50 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f83289e0740 10 client.7230 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5572f27ce6b0 con 0x5572f2ab5f50 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f82fe7fc640 1 -- 172.21.15.53:0/1691416568 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310032bb0 con 0x5572f2ab5f50 2024-05-10T10:40:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f82fe7fc640 1 -- 172.21.15.53:0/1691416568 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f8310062020 con 0x5572f2ab5f50 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f83289e0740 10 client.7230 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f83289e0740 10 client.7230 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.475+0000 7f83289e0740 20 client.7230 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.476+0000 7f83289e0740 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2ad62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.476+0000 7f83289e0740 4 client.7230 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.476+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572f2976760 con 0x5572f2ad5ee0 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.476+0000 7f831effd640 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2ad62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.476+0000 7f831effd640 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2ad62c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a657fd081becf421 server_cookie=bfce07325f3fbe08 in_seq=0 out_seq=0 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.476+0000 7f82fe7fc640 10 client.7230 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f82fe7fc640 1 -- 172.21.15.53:0/1691416568 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5572f2976760 con 0x5572f2ad5ee0 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f82fe7fc640 10 client.7230 handle_command_reply: tid=0 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 client.7230 shutdown 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 msgr2=0x5572f2ad2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad2440 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f83100098c0 tx=0x7f8310004730 comp rx=0 tx=0).stop 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 shutdown_connections 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2ad62c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 0x5572f2ab9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f82fe7fc640 0 client.7230 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2ad2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 --2- 172.21.15.53:0/1691416568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad2440 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 >> 172.21.15.53:0/1691416568 conn(0x5572f2a88bb0 msgr2=0x5572f2a87a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 shutdown_connections 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 1 -- 172.21.15.53:0/1691416568 wait complete. 2024-05-10T10:40:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.477+0000 7f83289e0740 20 client.7230 trim_cache size 0 max 16384 2024-05-10T10:40:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 Processor -- start 2024-05-10T10:40:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 -- start start 2024-05-10T10:40:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 0x5572f2ad0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2a89650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83201fee90 con 0x5572f2ab5f50 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83201f3910 con 0x5572f2aac0b0 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.496+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83201fe840 con 0x5572f2ab6330 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47326/0 (socket says 172.21.15.53:47326) 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 -- 172.21.15.53:0/2306509128 learned_addr learned my addr 172.21.15.53:0/2306509128 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 -- 172.21.15.53:0/2306509128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 msgr2=0x5572f2a89650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 --2- 172.21.15.53:0/2306509128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2a89650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 -- 172.21.15.53:0/2306509128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 msgr2=0x5572f2ad0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 --2- 172.21.15.53:0/2306509128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 0x5572f2ad0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 -- 172.21.15.53:0/2306509128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f2abab10 con 0x5572f2ab5f50 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f831effd640 1 --2- 172.21.15.53:0/2306509128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad62c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f83140011e0 tx=0x7f8314000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ee112a3f9ac171 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f8304ff9640 1 -- 172.21.15.53:0/2306509128 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010040 con 0x5572f2ab5f50 2024-05-10T10:40:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f8304ff9640 1 -- 172.21.15.53:0/2306509128 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f831401f070 con 0x5572f2ab5f50 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.497+0000 7f8304ff9640 1 -- 172.21.15.53:0/2306509128 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314017d30 con 0x5572f2ab5f50 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 -- 172.21.15.53:0/2306509128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 msgr2=0x5572f2ad62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 --2- 172.21.15.53:0/2306509128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad62c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f83140011e0 tx=0x7f8314000ed0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 -- 172.21.15.53:0/2306509128 shutdown_connections 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 --2- 172.21.15.53:0/2306509128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2ab6330 0x5572f2a89650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 --2- 172.21.15.53:0/2306509128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2ad62c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 --2- 172.21.15.53:0/2306509128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2aac0b0 0x5572f2ad0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 -- 172.21.15.53:0/2306509128 >> 172.21.15.53:0/2306509128 conn(0x5572f2a88bb0 msgr2=0x5572f2ab5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 -- 172.21.15.53:0/2306509128 shutdown_connections 2024-05-10T10:40:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 -- 172.21.15.53:0/2306509128 wait complete. 2024-05-10T10:40:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 -- start start 2024-05-10T10:40:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.498+0000 7f83289e0740 1 Processor -- start 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2aac0b0 0x5572f2af6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f83289e0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2a87e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f83289e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2ab6330 0x5572f2aed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f83289e0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5572f2ad86a0 con 0x5572f2ab5f50 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5572f2ad7760 con 0x5572f2ab6330 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f83289e0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5572f2aba000 con 0x5572f2aac0b0 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f83289e0740 10 client.0 _ll_register_callbacks cb 0x7f8310009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f831effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2a87e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f831effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2a87e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47334/0 (socket says 172.21.15.53:47334) 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f831effd640 1 -- 172.21.15.53:0/2689018712 learned_addr learned my addr 172.21.15.53:0/2689018712 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.499+0000 7f831f7fe640 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2aac0b0 0x5572f2af6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f831ffff640 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2ab6330 0x5572f2aed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f831effd640 1 -- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2aac0b0 msgr2=0x5572f2af6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f831effd640 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2aac0b0 0x5572f2af6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f831effd640 1 -- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2ab6330 msgr2=0x5572f2aed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f831effd640 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2ab6330 0x5572f2aed3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f831effd640 1 -- 172.21.15.53:0/2689018712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f2aad260 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f831effd640 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2a87e40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f831401c040 tx=0x7f831003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287d8188ed91396f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f82fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f82fe7fc640 1 -- 172.21.15.53:0/2689018712 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010040 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f82fe7fc640 1 -- 172.21.15.53:0/2689018712 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f831401a0e0 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831401f070 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f82fe7fc640 1 -- 172.21.15.53:0/2689018712 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f8314034050 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 10 client.7236 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5572f2abc210 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f82fe7fc640 1 -- 172.21.15.53:0/2689018712 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f831401f070 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f82fe7fc640 1 -- 172.21.15.53:0/2689018712 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f8314037020 con 0x5572f2ab5f50 2024-05-10T10:40:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 10 client.7236 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 10 client.7236 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 20 client.7236 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2acec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 4 client.7236 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.500+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "status"}) -- 0x5572f2a8e620 con 0x5572f2ad5ee0 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f831f7fe640 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2acec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f82fe7fc640 10 client.7236 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f831f7fe640 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2acec20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e70004dec88302e6 server_cookie=2b5ffc2ba7c9d6dd in_seq=0 out_seq=0 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f82fe7fc640 1 -- 172.21.15.53:0/2689018712 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5572f2a8e620 con 0x5572f2ad5ee0 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f82fe7fc640 10 client.7236 handle_command_reply: tid=0 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 client.7236 shutdown 2024-05-10T10:40:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 msgr2=0x5572f2a87e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2a87e40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f831401c040 tx=0x7f831003c000 comp rx=0 tx=0).stop 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 shutdown_connections 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5572f2ad5ee0 0x5572f2acec20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572f2ab6330 0x5572f2aed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f82fe7fc640 0 client.7236 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5572f2ab5f50 0x5572f2a87e40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 --2- 172.21.15.53:0/2689018712 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5572f2aac0b0 0x5572f2af6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 >> 172.21.15.53:0/2689018712 conn(0x5572f2a88bb0 msgr2=0x5572f2ad0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 shutdown_connections 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 1 -- 172.21.15.53:0/2689018712 wait complete. 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.501+0000 7f83289e0740 20 client.7236 trim_cache size 0 max 16384 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 7152, 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:40:27.511 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 4.5941285719999998, 2024-05-10T10:40:27.512 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 74, 2024-05-10T10:40:27.512 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 91, 2024-05-10T10:40:27.512 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 91, 2024-05-10T10:40:27.512 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 5.5685402660000003 2024-05-10T10:40:27.512 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f830c0762d0 msgr2=0x7f830c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f830c0762d0 0x7f830c078790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8310034000 tx=0x7f831002e000 comp rx=0 tx=0).stop 2024-05-10T10:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 msgr2=0x7f83201566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f83201566d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f831800c700 tx=0x7f831800cbc0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 shutdown_connections 2024-05-10T10:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f830c0762d0 0x7f830c078790 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83201598c0 0x7f83201566d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83200c7390 0x7f8320072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 --2- 172.21.15.53:0/2583044674 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f832006ec40 0x7f83200cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 >> 172.21.15.53:0/2583044674 conn(0x7f83200b2900 msgr2=0x7f8320157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 shutdown_connections 2024-05-10T10:40:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.502+0000 7f8325bef640 1 -- 172.21.15.53:0/2583044674 wait complete. 2024-05-10T10:40:27.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:40:27.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.624+0000 7f7e09be4640 1 Processor -- start 2024-05-10T10:40:27.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.624+0000 7f7e09be4640 1 -- start start 2024-05-10T10:40:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e09be4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e09be4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e040cfbb0 0x7f7e040ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e09be4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e0415fc90 con 0x7f7e040b8860 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e09be4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7e0406ec30 con 0x7f7e040cfbb0 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e09be4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0406ee50 con 0x7f7e040d0230 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e03fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e040cfbb0 0x7f7e040ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e03fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e040cfbb0 0x7f7e040ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47340/0 (socket says 172.21.15.53:47340) 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e03fff640 1 -- 172.21.15.53:0/3215734110 learned_addr learned my addr 172.21.15.53:0/3215734110 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e02ffd640 1 --2- 172.21.15.53:0/3215734110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e03fff640 1 -- 172.21.15.53:0/3215734110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 msgr2=0x7f7e040cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e03fff640 1 --2- 172.21.15.53:0/3215734110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e03fff640 1 -- 172.21.15.53:0/3215734110 >> v1:172.21.15.135:6789/0 conn(0x7f7e040b8860 legacy=0x7f7e0406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.625+0000 7f7e03fff640 1 -- 172.21.15.53:0/3215734110 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e040b8c40 con 0x7f7e040cfbb0 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.626+0000 7f7e03fff640 1 --2- 172.21.15.53:0/3215734110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040ce330 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009e80 tx=0x7f7dfc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91da26cccb1d7a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.626+0000 7f7e027fc640 1 -- 172.21.15.53:0/3215734110 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc036070 con 0x7f7e040cfbb0 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.626+0000 7f7e027fc640 1 -- 172.21.15.53:0/3215734110 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dfc031600 con 0x7f7e040cfbb0 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.626+0000 7f7e027fc640 1 -- 172.21.15.53:0/3215734110 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0319d0 con 0x7f7e040cfbb0 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 -- 172.21.15.53:0/3215734110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 msgr2=0x7f7e040ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 --2- 172.21.15.53:0/3215734110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040ce330 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009e80 tx=0x7f7dfc02a660 comp rx=0 tx=0).stop 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 -- 172.21.15.53:0/3215734110 shutdown_connections 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 --2- 172.21.15.53:0/3215734110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040ce330 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 --2- 172.21.15.53:0/3215734110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 -- 172.21.15.53:0/3215734110 >> 172.21.15.53:0/3215734110 conn(0x7f7e040d2df0 msgr2=0x7f7e040d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 -- 172.21.15.53:0/3215734110 shutdown_connections 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.627+0000 7f7e09be4640 1 -- 172.21.15.53:0/3215734110 wait complete. 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.628+0000 7f7e09be4640 1 Processor -- start 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.628+0000 7f7e09be4640 1 -- start start 2024-05-10T10:40:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e09be4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e040b8860 0x7f7e04153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e09be4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e09be4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e09be4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e040cdb30 con 0x7f7e040cfbb0 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e09be4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7e040cd850 con 0x7f7e040b8860 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e09be4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7e040cde10 con 0x7f7e040d0230 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47346/0 (socket says 172.21.15.53:47346) 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 -- 172.21.15.53:0/4275363287 learned_addr learned my addr 172.21.15.53:0/4275363287 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e037fe640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e040b8860 0x7f7e04153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e03fff640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 -- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 msgr2=0x7f7e040b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 -- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e040b8860 msgr2=0x7f7e04153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e040b8860 0x7f7e04153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.629+0000 7f7e02ffd640 1 -- 172.21.15.53:0/4275363287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e040c7a30 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.630+0000 7f7e02ffd640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040c9120 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d620 tx=0x7f7df800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.630+0000 7f7debfff640 1 -- 172.21.15.53:0/4275363287 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df800b620 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.630+0000 7f7debfff640 1 -- 172.21.15.53:0/4275363287 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df8019070 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.630+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e040c7d10 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.630+0000 7f7debfff640 1 -- 172.21.15.53:0/4275363287 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df800ba00 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.630+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e040c82d0 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.631+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e041614d0 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.632+0000 7f7debfff640 1 -- 172.21.15.53:0/4275363287 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7df800bba0 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.632+0000 7f7debfff640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7df00762d0 0x7f7df0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.632+0000 7f7debfff640 1 -- 172.21.15.53:0/4275363287 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f7df8097b70 con 0x7f7e040cfbb0 2024-05-10T10:40:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.632+0000 7f7e037fe640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7df00762d0 0x7f7df0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.633+0000 7f7e037fe640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7df00762d0 0x7f7df0078790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7df4004f60 tx=0x7f7df4007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.637+0000 7f7debfff640 1 -- 172.21.15.53:0/4275363287 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7df8060bd0 con 0x7f7e040cfbb0 2024-05-10T10:40:27.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.865+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7e0406a220 con 0x7f7e040cfbb0 2024-05-10T10:40:27.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.865+0000 7f7debfff640 1 -- 172.21.15.53:0/4275363287 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f7df8064ed0 con 0x7f7e040cfbb0 2024-05-10T10:40:27.869 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:40:27.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7df00762d0 msgr2=0x7f7df0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7df00762d0 0x7f7df0078790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7df4004f60 tx=0x7f7df4007e30 comp rx=0 tx=0).stop 2024-05-10T10:40:27.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 msgr2=0x7f7e040c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:27.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040c9120 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d620 tx=0x7f7df800dae0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 shutdown_connections 2024-05-10T10:40:27.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7df00762d0 0x7f7df0078790 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e040d0230 0x7f7e040b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e040cfbb0 0x7f7e040c9120 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 --2- 172.21.15.53:0/4275363287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e040b8860 0x7f7e04153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:27.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 >> 172.21.15.53:0/4275363287 conn(0x7f7e040d2df0 msgr2=0x7f7e040757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:27.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 shutdown_connections 2024-05-10T10:40:27.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:27.867+0000 7f7e09be4640 1 -- 172.21.15.53:0/4275363287 wait complete. 2024-05-10T10:40:27.880 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:40:27.880 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:40:27.880 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:40:27.880 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:40:27.881 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:27.881 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:40:27.881 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:40:27.881 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:40:27.881 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:40:27.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:27.881 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:40:27.906 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:40:27.907 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42341sec preferred_lft 42341sec 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:40:27.908 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:40:27.909 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:40:27.909 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:27.909 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:40:27.909 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:40:27.909 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:40:27.909 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:40:27.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:27.909 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:40:27.975 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:40:27.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:27.977 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:40:28.038 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:40:28.038 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:40:28.039 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:40:28.039 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:40:28.039 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:40:28.039 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:40:28.039 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:40:28.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:40:28.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:40:28.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:40:28.039 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:40:28.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:40:28.165 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:28.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:28.172 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:40:28.189 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:40:28.189 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:40:28.189 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:40:28.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:28.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:40:28.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:40:28.257 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:28.316 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:40:28.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:40:28.317 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:28.378 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:40:28.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:28.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:40:28.465 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:40:28.465 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:40:28.465 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:40:28.473 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:40:28.474 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:40:28.474 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:40:28.474 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:40:28.474 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:40:28.479 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:40:28.479 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:40:28.480 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:40:28.480 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:40:28.480 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:40:28.480 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:40:28.480 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:40:28.480 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:40:28.481 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:40:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:40:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:40:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:40:28.481 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:40:28.482 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:40:28.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:28.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:40:28.551 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:40:28.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:40:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:40:28.651 INFO:teuthology.orchestra.run.smithi053.stdout:client6613 2024-05-10T10:40:28.651 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:40:28.651 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:40:28.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:28.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-10T10:40:28.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.816+0000 7fa8df693640 1 Processor -- start 2024-05-10T10:40:28.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.816+0000 7fa8df693640 1 -- start start 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8df693640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa8d8162990 0x7fa8d80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8df693640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d80d0830 con 0x7fa8d80d1aa0 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8df693640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d80d0b10 con 0x7fa8d80b5f30 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8df693640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa8d80d10d0 con 0x7fa8d8162990 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8dd42f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa8d8162990 0x7fa8d80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8ddc30640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa8d80d1aa0 0x7fa8d80d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34038/0 (socket says 172.21.15.53:34038) 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8ddc30640 1 -- 172.21.15.53:0/3086167607 learned_addr learned my addr 172.21.15.53:0/3086167607 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8dd42f640 1 -- 172.21.15.53:0/3086167607 >> v1:172.21.15.135:6789/0 conn(0x7fa8d80b5f30 legacy=0x7fa8d80d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8dd42f640 1 -- 172.21.15.53:0/3086167607 >> v1:172.21.15.53:6789/0 conn(0x7fa8d80d1aa0 legacy=0x7fa8d80d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8cffff640 1 -- 172.21.15.53:0/3086167607 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944826403 0 0) 0x7fa8d80d0830 con 0x7fa8d80d1aa0 2024-05-10T10:40:28.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.817+0000 7fa8dd42f640 1 -- 172.21.15.53:0/3086167607 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d80d0df0 con 0x7fa8d8162990 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.818+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/3086167607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d8162990 0x7fa8d80b48a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097a0 tx=0x7fa8d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc2ff0ceee524ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.818+0000 7fa8cffff640 1 -- 172.21.15.53:0/3086167607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037070 con 0x7fa8d8162990 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.818+0000 7fa8cffff640 1 -- 172.21.15.53:0/3086167607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4032440 con 0x7fa8d8162990 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.818+0000 7fa8cffff640 1 -- 172.21.15.53:0/3086167607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4032810 con 0x7fa8d8162990 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.819+0000 7fa8df693640 1 -- 172.21.15.53:0/3086167607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d8162990 msgr2=0x7fa8d80b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.819+0000 7fa8df693640 1 --2- 172.21.15.53:0/3086167607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d8162990 0x7fa8d80b48a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40097a0 tx=0x7fa8d402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.819+0000 7fa8df693640 1 -- 172.21.15.53:0/3086167607 shutdown_connections 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.819+0000 7fa8df693640 1 --2- 172.21.15.53:0/3086167607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d8162990 0x7fa8d80b48a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.819+0000 7fa8df693640 1 -- 172.21.15.53:0/3086167607 >> 172.21.15.53:0/3086167607 conn(0x7fa8d809e3f0 msgr2=0x7fa8d80d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.819+0000 7fa8df693640 1 -- 172.21.15.53:0/3086167607 shutdown_connections 2024-05-10T10:40:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.819+0000 7fa8df693640 1 -- 172.21.15.53:0/3086167607 wait complete. 2024-05-10T10:40:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.820+0000 7fa8df693640 1 Processor -- start 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.820+0000 7fa8df693640 1 -- start start 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8df693640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d80b5f30 0x7fa8d814eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8df693640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8d80d1aa0 0x7fa8d814bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8df693640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8d8162990 0x7fa8d80b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8df693640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d80d1f30 con 0x7fa8d80b5f30 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8df693640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8162d70 con 0x7fa8d80d1aa0 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8df693640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d80d20b0 con 0x7fa8d8162990 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d80b5f30 0x7fa8d814eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d80b5f30 0x7fa8d814eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47352/0 (socket says 172.21.15.53:47352) 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 -- 172.21.15.53:0/4177767373 learned_addr learned my addr 172.21.15.53:0/4177767373 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:28.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8d80d1aa0 0x7fa8d814bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8d8162990 0x7fa8d80b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 -- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8d8162990 msgr2=0x7fa8d80b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8d8162990 0x7fa8d80b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 -- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8d80d1aa0 msgr2=0x7fa8d814bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8d80d1aa0 0x7fa8d814bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 -- 172.21.15.53:0/4177767373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d814f430 con 0x7fa8d80b5f30 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d80b5f30 0x7fa8d814eef0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4000c00 tx=0x7fa8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.821+0000 7fa8cdffb640 1 -- 172.21.15.53:0/4177767373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037040 con 0x7fa8d80b5f30 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.822+0000 7fa8cdffb640 1 -- 172.21.15.53:0/4177767373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4042070 con 0x7fa8d80b5f30 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.822+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d80cafa0 con 0x7fa8d80b5f30 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.822+0000 7fa8cdffb640 1 -- 172.21.15.53:0/4177767373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4030a70 con 0x7fa8d80b5f30 2024-05-10T10:40:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.822+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d80cb560 con 0x7fa8d80b5f30 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.823+0000 7fa8cdffb640 1 -- 172.21.15.53:0/4177767373 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa8d4030c10 con 0x7fa8d80b5f30 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.824+0000 7fa8cdffb640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8bc0762d0 0x7fa8bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.824+0000 7fa8cdffb640 1 -- 172.21.15.53:0/4177767373 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fa8d40b8330 con 0x7fa8d80b5f30 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.824+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8bc0762d0 0x7fa8bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 Processor -- start 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 -- start start 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8bc0762d0 0x7fa8bc078790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80ccb30 tx=0x7fa8c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc54c00 0x55f4bdc29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f4bdb8dcf0 con 0x55f4bdc55340 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f4bdb46e10 con 0x55f4bdc54c00 2024-05-10T10:40:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.825+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f4bdb89cb0 con 0x55f4bdc72220 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47362/0 (socket says 172.21.15.53:47362) 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 -- 172.21.15.53:0/570570965 learned_addr learned my addr 172.21.15.53:0/570570965 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc54c00 0x55f4bdc29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 -- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 msgr2=0x55f4bdc24bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc24bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 -- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc54c00 msgr2=0x55f4bdc29a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc54c00 0x55f4bdc29a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 -- 172.21.15.53:0/570570965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bd9677e0 con 0x55f4bdc55340 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc54c00 0x55f4bdc29a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc546c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800e7e0 tx=0x7fa8c8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1d51a75546174d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8c2ffd640 1 -- 172.21.15.53:0/570570965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c80159b0 con 0x55f4bdc55340 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8c2ffd640 1 -- 172.21.15.53:0/570570965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c8015b50 con 0x55f4bdc55340 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.826+0000 7fa8c2ffd640 1 -- 172.21.15.53:0/570570965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8007020 con 0x55f4bdc55340 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 -- 172.21.15.53:0/570570965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 msgr2=0x55f4bdc546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc546c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800e7e0 tx=0x7fa8c8011000 comp rx=0 tx=0).stop 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 -- 172.21.15.53:0/570570965 shutdown_connections 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc54c00 0x55f4bdc29a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc546c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/570570965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc24bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 -- 172.21.15.53:0/570570965 >> 172.21.15.53:0/570570965 conn(0x55f4bdc26e20 msgr2=0x55f4bdc6f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 -- 172.21.15.53:0/570570965 shutdown_connections 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 -- 172.21.15.53:0/570570965 wait complete. 2024-05-10T10:40:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.827+0000 7fa8e23ae740 1 -- start start 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.828+0000 7fa8e23ae740 1 Processor -- start 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.828+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55340 0x55f4bdc45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc55830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55d70 0x55f4bdc29ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f4bdc28fd0 con 0x55f4bdc55d70 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f4bdb4a940 con 0x55f4bdc55340 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f4bdc283c0 con 0x55f4bdc72220 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8e23ae740 10 client.0 _ll_register_callbacks cb 0x55f4bdb4d440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55d70 0x55f4bdc29ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55d70 0x55f4bdc29ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47378/0 (socket says 172.21.15.53:47378) 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 -- 172.21.15.53:0/2240916644 learned_addr learned my addr 172.21.15.53:0/2240916644 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55340 0x55f4bdc45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc55830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 -- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 msgr2=0x55f4bdc55830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc55830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 -- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55340 msgr2=0x55f4bdc45020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55340 0x55f4bdc45020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 -- 172.21.15.53:0/2240916644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bd9c7f70 con 0x55f4bdc55d70 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55d70 0x55f4bdc29ec0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0007b40 tx=0x7fa8d0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8416a241c3bd42f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8b7fff640 1 -- 172.21.15.53:0/2240916644 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d001a060 con 0x55f4bdc55d70 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8b7fff640 1 -- 172.21.15.53:0/2240916644 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fa8d0004d60 con 0x55f4bdc55d70 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.829+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c8015b50 con 0x55f4bdc55d70 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8b7fff640 1 -- 172.21.15.53:0/2240916644 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa8d000d070 con 0x55f4bdc55d70 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 10 client.7257 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f4bd85f170 con 0x55f4bdc55d70 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8b7fff640 1 -- 172.21.15.53:0/2240916644 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c8015b50 con 0x55f4bdc55d70 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8b7fff640 1 -- 172.21.15.53:0/2240916644 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fa8d0013030 con 0x55f4bdc55d70 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 10 client.7257 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 10 client.7257 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 20 client.7257 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc82b00 0x55f4bdc82f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 4 client.7257 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4bdb9b900 con 0x55f4bdc82b00 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc82b00 0x55f4bdc82f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8b7fff640 10 client.7257 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.830+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc82b00 0x55f4bdc82f50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369395f4cc66731a server_cookie=fd58c0e6a661cdc8 in_seq=0 out_seq=0 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8b7fff640 1 -- 172.21.15.53:0/2240916644 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4bdb9b900 con 0x55f4bdc82b00 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8b7fff640 10 client.7257 handle_command_reply: tid=0 2024-05-10T10:40:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 client.7257 shutdown 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55d70 msgr2=0x55f4bdc29ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55d70 0x55f4bdc29ec0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0007b40 tx=0x7fa8d0001a00 comp rx=0 tx=0).stop 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 shutdown_connections 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc82b00 0x55f4bdc82f50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55d70 0x55f4bdc29ec0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc55830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8b7fff640 0 client.7257 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/2240916644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55340 0x55f4bdc45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 >> 172.21.15.53:0/2240916644 conn(0x55f4bdc54c00 msgr2=0x55f4bdc6f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 shutdown_connections 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 1 -- 172.21.15.53:0/2240916644 wait complete. 2024-05-10T10:40:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.831+0000 7fa8e23ae740 20 client.7257 trim_cache size 0 max 16384 2024-05-10T10:40:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 Processor -- start 2024-05-10T10:40:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 -- start start 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc51d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc24eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc29a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8202000 con 0x55f4bdc55340 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8201060 con 0x55f4bdc55d70 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d814d260 con 0x55f4bdc72220 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8dd42f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc51d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8dd42f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc51d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47380/0 (socket says 172.21.15.53:47380) 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.850+0000 7fa8dd42f640 1 -- 172.21.15.53:0/575956494 learned_addr learned my addr 172.21.15.53:0/575956494 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc24eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc29a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8dd42f640 1 -- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 msgr2=0x55f4bdc29a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc29a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8dd42f640 1 -- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 msgr2=0x55f4bdc24eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc24eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8dd42f640 1 -- 172.21.15.53:0/575956494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bdc55ba0 con 0x55f4bdc55340 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc51d60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa8d407d360 tx=0x7fa8d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b7d92d194925ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8c27fc640 1 -- 172.21.15.53:0/575956494 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40819e0 con 0x55f4bdc55340 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8c27fc640 1 -- 172.21.15.53:0/575956494 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d40812e0 con 0x55f4bdc55340 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.851+0000 7fa8c27fc640 1 -- 172.21.15.53:0/575956494 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4085690 con 0x55f4bdc55340 2024-05-10T10:40:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 -- 172.21.15.53:0/575956494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 msgr2=0x55f4bdc51d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc51d60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa8d407d360 tx=0x7fa8d4008040 comp rx=0 tx=0).stop 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 -- 172.21.15.53:0/575956494 shutdown_connections 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc29a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc24eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/575956494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc51d60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 -- 172.21.15.53:0/575956494 >> 172.21.15.53:0/575956494 conn(0x55f4bdc54c00 msgr2=0x55f4bdc82ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 -- 172.21.15.53:0/575956494 shutdown_connections 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 -- 172.21.15.53:0/575956494 wait complete. 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 -- start start 2024-05-10T10:40:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.852+0000 7fa8e23ae740 1 Processor -- start 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc6ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc4f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8e23ae740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc4f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f4bdc54830 con 0x55f4bdc55340 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8200a60 con 0x55f4bdc55d70 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8e23ae740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f4bdceec20 con 0x55f4bdc72220 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8e23ae740 10 client.0 _ll_register_callbacks cb 0x7fa8d0005bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8dd42f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc6ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8dd42f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc6ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47394/0 (socket says 172.21.15.53:47394) 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8dd42f640 1 -- 172.21.15.53:0/3619712096 learned_addr learned my addr 172.21.15.53:0/3619712096 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc4f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.853+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc4f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8dd42f640 1 -- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 msgr2=0x55f4bdc4f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc4f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8dd42f640 1 -- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 msgr2=0x55f4bdc4f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc4f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8dd42f640 1 -- 172.21.15.53:0/3619712096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bdc89470 con 0x55f4bdc55340 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8ddc30640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc4f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc4f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8dd42f640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc6ddf0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa8d407d8d0 tx=0x7fa8d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a784b0f35458ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8b7fff640 1 -- 172.21.15.53:0/3619712096 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40b4060 con 0x55f4bdc55340 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8b7fff640 1 -- 172.21.15.53:0/3619712096 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fa8d4081540 con 0x55f4bdc55340 2024-05-10T10:40:28.862 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d40812e0 con 0x55f4bdc55340 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8b7fff640 1 -- 172.21.15.53:0/3619712096 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa8d409f050 con 0x55f4bdc55340 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 10 client.7263 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f4bdc258f0 con 0x55f4bdc55340 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8b7fff640 1 -- 172.21.15.53:0/3619712096 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d40812e0 con 0x55f4bdc55340 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8b7fff640 1 -- 172.21.15.53:0/3619712096 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fa8d403a030 con 0x55f4bdc55340 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 10 client.7263 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 10 client.7263 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 20 client.7263 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc2ba10 0x55f4bdced9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 4 client.7263 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.854+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55f4bdc26270 con 0x55f4bdc2ba10 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc2ba10 0x55f4bdced9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8b7fff640 10 client.7263 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8dcc2e640 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc2ba10 0x55f4bdced9d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985ce48d93d0e13b server_cookie=3a580a2a6aa76eee in_seq=0 out_seq=0 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8b7fff640 1 -- 172.21.15.53:0/3619712096 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f4bdc26270 con 0x55f4bdc2ba10 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8b7fff640 10 client.7263 handle_command_reply: tid=0 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 client.7263 shutdown 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 msgr2=0x55f4bdc6ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc6ddf0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa8d407d8d0 tx=0x7fa8d0005f90 comp rx=0 tx=0).stop 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 shutdown_connections 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55f4bdc2ba10 0x55f4bdced9d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f4bdc72220 0x55f4bdc4f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4bdc55d70 0x55f4bdc4f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 --2- 172.21.15.53:0/3619712096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4bdc55340 0x55f4bdc6ddf0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 >> 172.21.15.53:0/3619712096 conn(0x55f4bdc54c00 msgr2=0x55f4bdc24960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:28.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.855+0000 7fa8b7fff640 0 client.7263 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:28.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 shutdown_connections 2024-05-10T10:40:28.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8e23ae740 1 -- 172.21.15.53:0/3619712096 wait complete. 2024-05-10T10:40:28.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8e23ae740 20 client.7263 trim_cache size 0 max 16384 2024-05-10T10:40:28.865 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:40:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:40:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "658990c8-3f12-4666-afe5-57255f150647", 2024-05-10T10:40:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:40:28.866 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8bc0762d0 msgr2=0x7fa8bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8bc0762d0 0x7fa8bc078790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80ccb30 tx=0x7fa8c8009040 comp rx=0 tx=0).stop 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d80b5f30 msgr2=0x7fa8d814eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d80b5f30 0x7fa8d814eef0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4000c00 tx=0x7fa8d40046c0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 shutdown_connections 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa8bc0762d0 0x7fa8bc078790 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8d8162990 0x7fa8d80b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8d80d1aa0 0x7fa8d814bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 --2- 172.21.15.53:0/4177767373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d80b5f30 0x7fa8d814eef0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 >> 172.21.15.53:0/4177767373 conn(0x7fa8d809e3f0 msgr2=0x7fa8d80b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.856+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 shutdown_connections 2024-05-10T10:40:28.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.857+0000 7fa8df693640 1 -- 172.21.15.53:0/4177767373 wait complete. 2024-05-10T10:40:28.868 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:40:28.857+0000 7fc6708d3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x564d95482100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:40:28.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:28.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-10T10:40:28.911 INFO:tasks.ceph.mds.a.smithi053.stderr:2024-05-10T10:40:28.907+0000 7fc6748db640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:40:28.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.975+0000 7fb043724640 1 Processor -- start 2024-05-10T10:40:28.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.975+0000 7fb043724640 1 -- start start 2024-05-10T10:40:28.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb043724640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb03c162900 0x7fb03c0d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb043724640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb03c0d3950 0x7fb03c0d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb043724640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb03c0d4190 con 0x7fb03c162ce0 2024-05-10T10:40:28.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb043724640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb03c0cfc40 con 0x7fb03c0d3950 2024-05-10T10:40:28.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb043724640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb03c0d3eb0 con 0x7fb03c162900 2024-05-10T10:40:28.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb041cc1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb03c0d3950 0x7fb03c0d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb040cbf640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb03c162ce0 0x7fb03c0b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34042/0 (socket says 172.21.15.53:34042) 2024-05-10T10:40:28.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb040cbf640 1 -- 172.21.15.53:0/3614903555 learned_addr learned my addr 172.21.15.53:0/3614903555 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:28.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb0414c0640 1 --2- 172.21.15.53:0/3614903555 >> v2:172.21.15.135:3300/0 conn(0x7fb03c162900 0x7fb03c0d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb041cc1640 1 -- 172.21.15.53:0/3614903555 >> v2:172.21.15.135:3300/0 conn(0x7fb03c162900 msgr2=0x7fb03c0d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb041cc1640 1 --2- 172.21.15.53:0/3614903555 >> v2:172.21.15.135:3300/0 conn(0x7fb03c162900 0x7fb03c0d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb041cc1640 1 -- 172.21.15.53:0/3614903555 >> v1:172.21.15.53:6789/0 conn(0x7fb03c162ce0 legacy=0x7fb03c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb041cc1640 1 -- 172.21.15.53:0/3614903555 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c0cfee0 con 0x7fb03c0d3950 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb03bfff640 1 -- 172.21.15.53:0/3614903555 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256038724 0 0) 0x7fb030001110 con 0x7fb03c162ce0 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.976+0000 7fb0414c0640 1 --2- 172.21.15.53:0/3614903555 >> v2:172.21.15.135:3300/0 conn(0x7fb03c162900 0x7fb03c0d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.977+0000 7fb041cc1640 1 --2- 172.21.15.53:0/3614903555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c0d3950 0x7fb03c0d09c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb03c06c520 tx=0x7fb03402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0fb3c6718ae40a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.977+0000 7fb03bfff640 1 -- 172.21.15.53:0/3614903555 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034037070 con 0x7fb03c0d3950 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.977+0000 7fb03bfff640 1 -- 172.21.15.53:0/3614903555 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb034032440 con 0x7fb03c0d3950 2024-05-10T10:40:28.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.977+0000 7fb03bfff640 1 -- 172.21.15.53:0/3614903555 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03402ae60 con 0x7fb03c0d3950 2024-05-10T10:40:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 -- 172.21.15.53:0/3614903555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c0d3950 msgr2=0x7fb03c0d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 --2- 172.21.15.53:0/3614903555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c0d3950 0x7fb03c0d09c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb03c06c520 tx=0x7fb03402a110 comp rx=0 tx=0).stop 2024-05-10T10:40:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 -- 172.21.15.53:0/3614903555 shutdown_connections 2024-05-10T10:40:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 --2- 172.21.15.53:0/3614903555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c0d3950 0x7fb03c0d09c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 --2- 172.21.15.53:0/3614903555 >> v2:172.21.15.135:3300/0 conn(0x7fb03c162900 0x7fb03c0d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 -- 172.21.15.53:0/3614903555 >> 172.21.15.53:0/3614903555 conn(0x7fb03c09e430 msgr2=0x7fb03c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 -- 172.21.15.53:0/3614903555 shutdown_connections 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.978+0000 7fb043724640 1 -- 172.21.15.53:0/3614903555 wait complete. 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.979+0000 7fb043724640 1 Processor -- start 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.979+0000 7fb043724640 1 -- start start 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb043724640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb03c0d3950 0x7fb03c1f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb043724640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c162900 0x7fb03c1f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb043724640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb03c162ce0 0x7fb03c1613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb043724640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0b4300 con 0x7fb03c162900 2024-05-10T10:40:28.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb043724640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0b4180 con 0x7fb03c0d3950 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb043724640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb03c0b4ff0 con 0x7fb03c162ce0 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c162900 0x7fb03c1f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c162900 0x7fb03c1f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47408/0 (socket says 172.21.15.53:47408) 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 -- 172.21.15.53:0/2341849738 learned_addr learned my addr 172.21.15.53:0/2341849738 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 -- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb03c162ce0 msgr2=0x7fb03c1613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb03c162ce0 0x7fb03c1613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 -- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb03c0d3950 msgr2=0x7fb03c1f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:28.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb03c0d3950 0x7fb03c1f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 -- 172.21.15.53:0/2341849738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c1f6150 con 0x7fb03c162900 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.980+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c162900 0x7fb03c1f2690 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c780 tx=0x7fb03000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.981+0000 7fb039ffb640 1 -- 172.21.15.53:0/2341849738 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030018070 con 0x7fb03c162900 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.981+0000 7fb039ffb640 1 -- 172.21.15.53:0/2341849738 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0300072d0 con 0x7fb03c162900 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.981+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c1f2bd0 con 0x7fb03c162900 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.981+0000 7fb039ffb640 1 -- 172.21.15.53:0/2341849738 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0300047e0 con 0x7fb03c162900 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.981+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c0b8440 con 0x7fb03c162900 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.983+0000 7fb039ffb640 1 -- 172.21.15.53:0/2341849738 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb03001b070 con 0x7fb03c162900 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.983+0000 7fb039ffb640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb028076300 0x7fb0280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.983+0000 7fb039ffb640 1 -- 172.21.15.53:0/2341849738 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fb030096bb0 con 0x7fb03c162900 2024-05-10T10:40:28.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.984+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb028076300 0x7fb0280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.984+0000 7fb04643f740 1 Processor -- start 2024-05-10T10:40:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.984+0000 7fb04643f740 1 -- start start 2024-05-10T10:40:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.984+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb028076300 0x7fb0280787c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009f50 tx=0x7fb02c0094a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a863aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df280 0x562a863de600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb04643f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863deb40 0x562a863b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb04643f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562a8637b690 con 0x562a863deb40 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562a86318f30 con 0x562a863fc160 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x562a862d1b20 con 0x562a863df280 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863deb40 0x562a863b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863deb40 0x562a863b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47422/0 (socket says 172.21.15.53:47422) 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 -- 172.21.15.53:0/2736211334 learned_addr learned my addr 172.21.15.53:0/2736211334 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a863aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df280 0x562a863de600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 -- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df280 msgr2=0x562a863de600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df280 0x562a863de600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 -- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 msgr2=0x562a863aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a863aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 -- 172.21.15.53:0/2736211334 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a862c9e60 con 0x562a863deb40 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a863aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb041cc1640 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863deb40 0x562a863b3950 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb03402a700 tx=0x7fb03402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8554dd44f7fbbbb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb022ffd640 1 -- 172.21.15.53:0/2736211334 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034037040 con 0x562a863deb40 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb022ffd640 1 -- 172.21.15.53:0/2736211334 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb034042070 con 0x562a863deb40 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.985+0000 7fb022ffd640 1 -- 172.21.15.53:0/2736211334 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034006e10 con 0x562a863deb40 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 -- 172.21.15.53:0/2736211334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863deb40 msgr2=0x562a863b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863deb40 0x562a863b3950 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb03402a700 tx=0x7fb03402c070 comp rx=0 tx=0).stop 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 -- 172.21.15.53:0/2736211334 shutdown_connections 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863deb40 0x562a863b3950 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df280 0x562a863de600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 --2- 172.21.15.53:0/2736211334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a863aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 -- 172.21.15.53:0/2736211334 >> 172.21.15.53:0/2736211334 conn(0x562a863b0d60 msgr2=0x562a863f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 -- 172.21.15.53:0/2736211334 shutdown_connections 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.986+0000 7fb04643f740 1 -- 172.21.15.53:0/2736211334 wait complete. 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.987+0000 7fb04643f740 1 -- start start 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.987+0000 7fb04643f740 1 Processor -- start 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863e1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863fc160 0x562a863ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a863fca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562a862d55e0 con 0x562a863fc160 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562a86314ef0 con 0x562a863df280 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x562a863b2f10 con 0x562a863df770 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 10 client.0 _ll_register_callbacks cb 0x562a862d0320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863fc160 0x562a863ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863fc160 0x562a863ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47430/0 (socket says 172.21.15.53:47430) 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 -- 172.21.15.53:0/2859448402 learned_addr learned my addr 172.21.15.53:0/2859448402 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb041cc1640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a863fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 -- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 msgr2=0x562a863fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a863fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 -- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 msgr2=0x562a863e1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863e1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 -- 172.21.15.53:0/2859448402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a860f47d0 con 0x562a863fc160 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb041cc1640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a863fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863fc160 0x562a863ceda0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c750 tx=0x7fb030060260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b738db990ee4886 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb01bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb01bfff640 1 -- 172.21.15.53:0/2859448402 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030068930 con 0x562a863fc160 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb01bfff640 1 -- 172.21.15.53:0/2859448402 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fb030066110 con 0x562a863fc160 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.988+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb034042070 con 0x562a863fc160 2024-05-10T10:40:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb01bfff640 1 -- 172.21.15.53:0/2859448402 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fb030066310 con 0x562a863fc160 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 10 client.7278 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x562a8602a3f0 con 0x562a863fc160 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb01bfff640 1 -- 172.21.15.53:0/2859448402 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb034042070 con 0x562a863fc160 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb01bfff640 1 -- 172.21.15.53:0/2859448402 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fb03005b4d0 con 0x562a863fc160 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 10 client.7278 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 10 client.7278 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 20 client.7278 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863dfbf0 0x562a86409270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 4 client.7278 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a86326b40 con 0x562a863dfbf0 2024-05-10T10:40:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863dfbf0 0x562a86409270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb01bfff640 10 client.7278 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.989+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863dfbf0 0x562a86409270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6195f7aaf3178320 server_cookie=9ca41923404edafb in_seq=0 out_seq=0 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb01bfff640 1 -- 172.21.15.53:0/2859448402 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a86326b40 con 0x562a863dfbf0 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb01bfff640 10 client.7278 handle_command_reply: tid=0 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 client.7278 shutdown 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863fc160 msgr2=0x562a863ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863fc160 0x562a863ceda0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c750 tx=0x7fb030060260 comp rx=0 tx=0).stop 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 shutdown_connections 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863dfbf0 0x562a86409270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a863fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb01bfff640 0 client.7278 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863fc160 0x562a863ceda0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 --2- 172.21.15.53:0/2859448402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863e1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 >> 172.21.15.53:0/2859448402 conn(0x562a863deb40 msgr2=0x562a863ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 shutdown_connections 2024-05-10T10:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 1 -- 172.21.15.53:0/2859448402 wait complete. 2024-05-10T10:40:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:28.990+0000 7fb04643f740 20 client.7278 trim_cache size 0 max 16384 2024-05-10T10:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.008+0000 7fb04643f740 1 Processor -- start 2024-05-10T10:40:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.008+0000 7fb04643f740 1 -- start start 2024-05-10T10:40:29.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb04643f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df280 0x562a863e2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a864091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb04643f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c202130 con 0x562a863df280 2024-05-10T10:40:29.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c201170 con 0x562a863fc160 2024-05-10T10:40:29.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb03c0cddf0 con 0x562a863df770 2024-05-10T10:40:29.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb0414c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df280 0x562a863e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb0414c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df280 0x562a863e2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47440/0 (socket says 172.21.15.53:47440) 2024-05-10T10:40:29.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb0414c0640 1 -- 172.21.15.53:0/1708843540 learned_addr learned my addr 172.21.15.53:0/1708843540 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:29.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb0414c0640 1 -- 172.21.15.53:0/1708843540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 msgr2=0x562a864091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:29.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.009+0000 7fb0414c0640 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a864091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb0414c0640 1 -- 172.21.15.53:0/1708843540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 msgr2=0x562a86409700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:29.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb041cc1640 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb0414c0640 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb0414c0640 1 -- 172.21.15.53:0/1708843540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a863b1ea0 con 0x562a863df280 2024-05-10T10:40:29.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb0414c0640 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df280 0x562a863e2750 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb02c003ca0 tx=0x7fb02c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68132f37e6eb0fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:29.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb0227fc640 1 -- 172.21.15.53:0/1708843540 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c02b070 con 0x562a863df280 2024-05-10T10:40:29.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb0227fc640 1 -- 172.21.15.53:0/1708843540 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb02c025450 con 0x562a863df280 2024-05-10T10:40:29.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb0227fc640 1 -- 172.21.15.53:0/1708843540 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c025800 con 0x562a863df280 2024-05-10T10:40:29.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 -- 172.21.15.53:0/1708843540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df280 msgr2=0x562a863e2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:29.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df280 0x562a863e2750 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb02c003ca0 tx=0x7fb02c013000 comp rx=0 tx=0).stop 2024-05-10T10:40:29.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 -- 172.21.15.53:0/1708843540 shutdown_connections 2024-05-10T10:40:29.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863df770 0x562a864091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 --2- 172.21.15.53:0/1708843540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df280 0x562a863e2750 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 -- 172.21.15.53:0/1708843540 >> 172.21.15.53:0/1708843540 conn(0x562a863deb40 msgr2=0x562a863f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:29.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.010+0000 7fb04643f740 1 -- 172.21.15.53:0/1708843540 shutdown_connections 2024-05-10T10:40:29.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.011+0000 7fb04643f740 1 -- 172.21.15.53:0/1708843540 wait complete. 2024-05-10T10:40:29.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.011+0000 7fb04643f740 1 -- start start 2024-05-10T10:40:29.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.011+0000 7fb04643f740 1 Processor -- start 2024-05-10T10:40:29.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb04643f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df770 0x562a864091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb04643f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb04643f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562a863e1340 con 0x562a863df770 2024-05-10T10:40:29.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0ce440 con 0x562a863df280 2024-05-10T10:40:29.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb04643f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x562a863f6030 con 0x562a863fc160 2024-05-10T10:40:29.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb04643f740 10 client.0 _ll_register_callbacks cb 0x7fb03005fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:29.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df770 0x562a864091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df770 0x562a864091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47448/0 (socket says 172.21.15.53:47448) 2024-05-10T10:40:29.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 -- 172.21.15.53:0/2911526513 learned_addr learned my addr 172.21.15.53:0/2911526513 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:29.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb041cc1640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 -- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863fc160 msgr2=0x562a86409700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:29.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 -- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 msgr2=0x562a863f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:29.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb040cbf640 1 -- 172.21.15.53:0/2911526513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a85fea0f0 con 0x562a863df770 2024-05-10T10:40:29.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.012+0000 7fb041cc1640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:29.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb040cbf640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df770 0x562a864091c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb03005c100 tx=0x7fb03005bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16392315a9aac05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:29.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb01bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:29.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb01bfff640 1 -- 172.21.15.53:0/2911526513 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030068930 con 0x562a863df770 2024-05-10T10:40:29.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb01bfff640 1 -- 172.21.15.53:0/2911526513 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fb030081210 con 0x562a863df770 2024-05-10T10:40:29.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb02c025450 con 0x562a863df770 2024-05-10T10:40:29.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb01bfff640 1 -- 172.21.15.53:0/2911526513 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fb03005ea40 con 0x562a863df770 2024-05-10T10:40:29.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 10 client.7284 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:29.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x562a863b40a0 con 0x562a863df770 2024-05-10T10:40:29.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb01bfff640 1 -- 172.21.15.53:0/2911526513 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb02c025450 con 0x562a863df770 2024-05-10T10:40:29.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb01bfff640 1 -- 172.21.15.53:0/2911526513 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fb030068b80 con 0x562a863df770 2024-05-10T10:40:29.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 10 client.7284 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:29.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 10 client.7284 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:29.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 20 client.7284 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:29.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863eef90 0x562a863b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:29.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 4 client.7284 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:40:29.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562a863209b0 con 0x562a863eef90 2024-05-10T10:40:29.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.013+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863eef90 0x562a863b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:29.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb01bfff640 10 client.7284 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:29.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb0414c0640 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863eef90 0x562a863b3130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a1539fee6ebcbe server_cookie=d93c5f0830184147 in_seq=0 out_seq=0 2024-05-10T10:40:29.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb01bfff640 1 -- 172.21.15.53:0/2911526513 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x562a863209b0 con 0x562a863eef90 2024-05-10T10:40:29.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb01bfff640 10 client.7284 handle_command_reply: tid=0 2024-05-10T10:40:29.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 client.7284 shutdown 2024-05-10T10:40:29.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df770 msgr2=0x562a864091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df770 0x562a864091c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb03005c100 tx=0x7fb03005bb50 comp rx=0 tx=0).stop 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 shutdown_connections 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x562a863eef90 0x562a863b3130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x562a863fc160 0x562a86409700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562a863df770 0x562a864091c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 --2- 172.21.15.53:0/2911526513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a863df280 0x562a863f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb01bfff640 0 client.7284 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:29.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 >> 172.21.15.53:0/2911526513 conn(0x562a863deb40 msgr2=0x562a863f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:29.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 shutdown_connections 2024-05-10T10:40:29.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 1 -- 172.21.15.53:0/2911526513 wait complete. 2024-05-10T10:40:29.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.014+0000 7fb04643f740 20 client.7284 trim_cache size 0 max 16384 2024-05-10T10:40:29.035 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:40:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:40:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:40:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "658990c8-3f12-4666-afe5-57255f150647": { 2024-05-10T10:40:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/dir", 2024-05-10T10:40:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "658990c8-3f12-4666-afe5-57255f150647", 2024-05-10T10:40:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair" 2024-05-10T10:40:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:40:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:40:29.036 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb028076300 msgr2=0x7fb0280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:29.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb028076300 0x7fb0280787c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009f50 tx=0x7fb02c0094a0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c162900 msgr2=0x7fb03c1f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c162900 0x7fb03c1f2690 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c780 tx=0x7fb03000cc40 comp rx=0 tx=0).stop 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 shutdown_connections 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb028076300 0x7fb0280787c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb03c162ce0 0x7fb03c1613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb03c162900 0x7fb03c1f2690 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 --2- 172.21.15.53:0/2341849738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb03c0d3950 0x7fb03c1f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 >> 172.21.15.53:0/2341849738 conn(0x7fb03c09e430 msgr2=0x7fb03c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 shutdown_connections 2024-05-10T10:40:29.040 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:29.015+0000 7fb043724640 1 -- 172.21.15.53:0/2341849738 wait complete. 2024-05-10T10:40:29.040 INFO:tasks.cephfs.filesystem:scrub status for tag:658990c8-3f12-4666-afe5-57255f150647 - {'path': '/dir', 'tag': '658990c8-3f12-4666-afe5-57255f150647', 'options': 'recursive,repair'} 2024-05-10T10:40:45.362 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:40:45.366 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:40:59.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:40:59.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-10T10:40:59.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.176+0000 7f7d1a5da640 1 Processor -- start 2024-05-10T10:40:59.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.176+0000 7f7d1a5da640 1 -- start start 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.177+0000 7f7d1a5da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d140cb9e0 0x7f7d1406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.177+0000 7f7d1a5da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d140cf3d0 con 0x7f7d140cb580 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.177+0000 7f7d1a5da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d140cf6b0 con 0x7f7d14159340 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.177+0000 7f7d1a5da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7d14159720 con 0x7f7d140cb9e0 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.177+0000 7f7d18b77640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d140cb9e0 0x7f7d1406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.177+0000 7f7d137fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d140cb580 0x7f7d1406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48140/0 (socket says 172.21.15.53:48140) 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.177+0000 7f7d137fe640 1 -- 172.21.15.53:0/720475312 learned_addr learned my addr 172.21.15.53:0/720475312 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:59.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.178+0000 7f7d18b77640 1 -- 172.21.15.53:0/720475312 >> v1:172.21.15.135:6789/0 conn(0x7f7d14159340 legacy=0x7f7d140ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.178+0000 7f7d18b77640 1 -- 172.21.15.53:0/720475312 >> v1:172.21.15.53:6789/0 conn(0x7f7d140cb580 legacy=0x7f7d1406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.178+0000 7f7d18b77640 1 -- 172.21.15.53:0/720475312 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d140cf990 con 0x7f7d140cb9e0 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.178+0000 7f7d18b77640 1 --2- 172.21.15.53:0/720475312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d1406eac0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009d90 tx=0x7f7d0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9459781d2a55f318 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.178+0000 7f7d12ffd640 1 -- 172.21.15.53:0/720475312 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c007740 con 0x7f7d140cb9e0 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.178+0000 7f7d12ffd640 1 -- 172.21.15.53:0/720475312 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c0078e0 con 0x7f7d140cb9e0 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.178+0000 7f7d12ffd640 1 -- 172.21.15.53:0/720475312 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c007cb0 con 0x7f7d140cb9e0 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.179+0000 7f7d1a5da640 1 -- 172.21.15.53:0/720475312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 msgr2=0x7f7d1406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.179+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/720475312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d1406eac0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009d90 tx=0x7f7d0c02a6a0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.179+0000 7f7d1a5da640 1 -- 172.21.15.53:0/720475312 shutdown_connections 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.179+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/720475312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d1406eac0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.179+0000 7f7d1a5da640 1 -- 172.21.15.53:0/720475312 >> 172.21.15.53:0/720475312 conn(0x7f7d140d0c90 msgr2=0x7f7d140d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.179+0000 7f7d1a5da640 1 -- 172.21.15.53:0/720475312 shutdown_connections 2024-05-10T10:40:59.180 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.179+0000 7f7d1a5da640 1 -- 172.21.15.53:0/720475312 wait complete. 2024-05-10T10:40:59.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.181+0000 7f7d1a5da640 1 Processor -- start 2024-05-10T10:40:59.181 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.181+0000 7f7d1a5da640 1 -- start start 2024-05-10T10:40:59.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d140cb580 0x7f7d14149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d140c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d14159340 0x7f7d140b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1415bfd0 con 0x7f7d140cb9e0 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1406df20 con 0x7f7d140cb580 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7d140cbdf0 con 0x7f7d14159340 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d140c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d140c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59072/0 (socket says 172.21.15.53:59072) 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 -- 172.21.15.53:0/1554123515 learned_addr learned my addr 172.21.15.53:0/1554123515 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d13fff640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d140cb580 0x7f7d14149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d18b77640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d14159340 0x7f7d140b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 -- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d14159340 msgr2=0x7f7d140b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d14159340 0x7f7d140b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 -- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d140cb580 msgr2=0x7f7d14149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d140cb580 0x7f7d14149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 -- 172.21.15.53:0/1554123515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d140c7160 con 0x7f7d140cb9e0 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d13fff640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d140cb580 0x7f7d14149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d18b77640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d14159340 0x7f7d140b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d137fe640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d140c7580 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800c3c0 tx=0x7f7d0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d10ff9640 1 -- 172.21.15.53:0/1554123515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08017070 con 0x7f7d140cb9e0 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d10ff9640 1 -- 172.21.15.53:0/1554123515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08005ce0 con 0x7f7d140cb9e0 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d140b5750 con 0x7f7d140cb9e0 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d10ff9640 1 -- 172.21.15.53:0/1554123515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080072f0 con 0x7f7d140cb9e0 2024-05-10T10:40:59.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.182+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d140c6680 con 0x7f7d140cb9e0 2024-05-10T10:40:59.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.184+0000 7f7d10ff9640 1 -- 172.21.15.53:0/1554123515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d0801a070 con 0x7f7d140cb9e0 2024-05-10T10:40:59.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.185+0000 7f7d10ff9640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d000762d0 0x7f7d00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.185+0000 7f7d10ff9640 1 -- 172.21.15.53:0/1554123515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f7d08096c20 con 0x7f7d140cb9e0 2024-05-10T10:40:59.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.185+0000 7f7d13fff640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d000762d0 0x7f7d00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.185+0000 7f7d1d3ca740 1 Processor -- start 2024-05-10T10:40:59.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.185+0000 7f7d1d3ca740 1 -- start start 2024-05-10T10:40:59.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d13fff640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d000762d0 0x7f7d00078790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d04007fd0 tx=0x7f7d04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfc3160 0x5610bcfeb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcfee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610bcf8b690 con 0x5610bcfeb450 2024-05-10T10:40:59.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610bcf28f30 con 0x5610bcfebd70 2024-05-10T10:40:59.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5610bcee1b20 con 0x5610bcfc3160 2024-05-10T10:40:59.189 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfef250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59078/0 (socket says 172.21.15.53:59078) 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d13fff640 1 -- 172.21.15.53:0/3398427061 learned_addr learned my addr 172.21.15.53:0/3398427061 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d137fe640 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfc3160 0x5610bcfeb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.186+0000 7f7d18b77640 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcfee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7d13fff640 1 -- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfc3160 msgr2=0x5610bcfeb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7d13fff640 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfc3160 0x5610bcfeb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7d13fff640 1 -- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 msgr2=0x5610bcfee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7d13fff640 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcfee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7d13fff640 1 -- 172.21.15.53:0/3398427061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610bced9e60 con 0x5610bcfeb450 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7d18b77640 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcfee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7d13fff640 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfef250 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7d040053c0 tx=0x7f7d04010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e2246b1a04aeae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7cf9ffb640 1 -- 172.21.15.53:0/3398427061 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04003a00 con 0x5610bcfeb450 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7cf9ffb640 1 -- 172.21.15.53:0/3398427061 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d04003ba0 con 0x5610bcfeb450 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.187+0000 7f7cf9ffb640 1 -- 172.21.15.53:0/3398427061 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04020450 con 0x5610bcfeb450 2024-05-10T10:40:59.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3398427061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 msgr2=0x5610bcfef250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfef250 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7d040053c0 tx=0x7f7d04010000 comp rx=0 tx=0).stop 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3398427061 shutdown_connections 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcfee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfc3160 0x5610bcfeb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3398427061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfef250 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3398427061 >> 172.21.15.53:0/3398427061 conn(0x5610bcfc0d60 msgr2=0x5610bd009600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3398427061 shutdown_connections 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3398427061 wait complete. 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.188+0000 7f7d1d3ca740 1 -- start start 2024-05-10T10:40:59.191 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 1 Processor -- start 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 0x5610bcfdf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 0x5610bd01e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610bcee55e0 con 0x5610bcfe1890 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610bcf24ef0 con 0x5610bcfeb450 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5610bcfbf0b0 con 0x5610bcfebd70 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d1d3ca740 10 client.0 _ll_register_callbacks cb 0x5610bcee0320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d18b77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d18b77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59084/0 (socket says 172.21.15.53:59084) 2024-05-10T10:40:59.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d18b77640 1 -- 172.21.15.53:0/105538099 learned_addr learned my addr 172.21.15.53:0/105538099 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.189+0000 7f7d13fff640 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 0x5610bcfdf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d137fe640 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 0x5610bd01e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d18b77640 1 -- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 msgr2=0x5610bd01e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d18b77640 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 0x5610bd01e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d18b77640 1 -- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 msgr2=0x5610bcfdf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d18b77640 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 0x5610bcfdf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d18b77640 1 -- 172.21.15.53:0/105538099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610bcd047d0 con 0x5610bcfe1890 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d18b77640 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfea210 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009eb0 tx=0x7f7d0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac6651443162653 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/105538099 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c009450 con 0x5610bcfe1890 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/105538099 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f7d0c046d50 con 0x5610bcfe1890 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d04003ba0 con 0x5610bcfe1890 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/105538099 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f7d0c03c620 con 0x5610bcfe1890 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d1d3ca740 10 client.7299 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:59.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5610bcc3a3f0 con 0x5610bcfe1890 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.190+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/105538099 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d04003ba0 con 0x5610bcfe1890 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/105538099 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f7d0c04d050 con 0x5610bcfe1890 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d1d3ca740 10 client.7299 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d1d3ca740 10 client.7299 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d1d3ca740 20 client.7299 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfea750 0x5610bd00bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d1d3ca740 4 client.7299 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610bcf36b40 con 0x5610bcfea750 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d13fff640 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfea750 0x5610bd00bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7d13fff640 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfea750 0x5610bd00bb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c107f156dcf17ee server_cookie=5ad5256353c01d08 in_seq=0 out_seq=0 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.191+0000 7f7cf2ffd640 10 client.7299 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/105538099 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5610bcf36b40 con 0x5610bcfea750 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7cf2ffd640 10 client.7299 handle_command_reply: tid=0 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 client.7299 shutdown 2024-05-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 msgr2=0x5610bcfea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfea210 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009eb0 tx=0x7f7d0c004ab0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 shutdown_connections 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfea750 0x5610bd00bb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfea210 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 0x5610bd01e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7cf2ffd640 0 client.7299 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/105538099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 0x5610bcfdf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 >> 172.21.15.53:0/105538099 conn(0x5610bcfc3160 msgr2=0x5610bcfc4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 shutdown_connections 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/105538099 wait complete. 2024-05-10T10:40:59.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.192+0000 7f7d1d3ca740 20 client.7299 trim_cache size 0 max 16384 2024-05-10T10:40:59.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 Processor -- start 2024-05-10T10:40:59.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 -- start start 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfbe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 0x5610bd009760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 0x5610bd009ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141fdec0 con 0x5610bcfe1890 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1415aab0 con 0x5610bcfeb450 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7d140b40a0 con 0x5610bcfebd70 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfbe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfbe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59100/0 (socket says 172.21.15.53:59100) 2024-05-10T10:40:59.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.211+0000 7f7d13fff640 1 -- 172.21.15.53:0/1691219986 learned_addr learned my addr 172.21.15.53:0/1691219986 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d13fff640 1 -- 172.21.15.53:0/1691219986 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 msgr2=0x5610bd009ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d13fff640 1 --2- 172.21.15.53:0/1691219986 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 0x5610bd009ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d13fff640 1 -- 172.21.15.53:0/1691219986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 msgr2=0x5610bd009760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d13fff640 1 --2- 172.21.15.53:0/1691219986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 0x5610bd009760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d13fff640 1 -- 172.21.15.53:0/1691219986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610bcbfa0f0 con 0x5610bcfe1890 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d13fff640 1 --2- 172.21.15.53:0/1691219986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfbe0e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400d630 tx=0x7f7d04005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c348b19fa5eb4f68 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7cf97fa640 1 -- 172.21.15.53:0/1691219986 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04003a00 con 0x5610bcfe1890 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7cf97fa640 1 -- 172.21.15.53:0/1691219986 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d04003ba0 con 0x5610bcfe1890 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7cf97fa640 1 -- 172.21.15.53:0/1691219986 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04020670 con 0x5610bcfe1890 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/1691219986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 msgr2=0x5610bcfbe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/1691219986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfbe0e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400d630 tx=0x7f7d04005000 comp rx=0 tx=0).stop 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/1691219986 shutdown_connections 2024-05-10T10:40:59.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/1691219986 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfebd70 0x5610bd009ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/1691219986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfeb450 0x5610bd009760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/1691219986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfe1890 0x5610bcfbe0e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.212+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/1691219986 >> 172.21.15.53:0/1691219986 conn(0x5610bcfc3160 msgr2=0x5610bcfc2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.213+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/1691219986 shutdown_connections 2024-05-10T10:40:59.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.213+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/1691219986 wait complete. 2024-05-10T10:40:59.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.213+0000 7f7d1d3ca740 1 -- start start 2024-05-10T10:40:59.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.213+0000 7f7d1d3ca740 1 Processor -- start 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfe1890 0x5610bd08f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfc2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d1d3ca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcff1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610bd020f00 con 0x5610bcfeb450 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d140b46f0 con 0x5610bcfebd70 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d1d3ca740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5610bd021390 con 0x5610bcfe1890 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d1d3ca740 10 client.0 _ll_register_callbacks cb 0x7f7d140718f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfc2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfc2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59116/0 (socket says 172.21.15.53:59116) 2024-05-10T10:40:59.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 -- 172.21.15.53:0/3963685890 learned_addr learned my addr 172.21.15.53:0/3963685890 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d13fff640 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfe1890 0x5610bd08f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d18b77640 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcff1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 -- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfe1890 msgr2=0x5610bd08f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfe1890 0x5610bd08f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 -- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 msgr2=0x5610bcff1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcff1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.214+0000 7f7d137fe640 1 -- 172.21.15.53:0/3963685890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610bcfc2c40 con 0x5610bcfeb450 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d137fe640 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfc2700 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800cd80 tx=0x7f7d0805f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a1a5521a56283f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/3963685890 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080602c0 con 0x5610bcfeb450 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/3963685890 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f7d08066170 con 0x5610bcfeb450 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d04003ba0 con 0x5610bcfeb450 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/3963685890 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f7d0805fbc0 con 0x5610bcfeb450 2024-05-10T10:40:59.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 10 client.7305 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5610bcfff6c0 con 0x5610bcfeb450 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/3963685890 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d04003ba0 con 0x5610bcfeb450 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/3963685890 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f7d0805b530 con 0x5610bcfeb450 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 10 client.7305 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 10 client.7305 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 20 client.7305 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfeca20 0x5610bcff24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 4 client.7305 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610bcf309b0 con 0x5610bcfeca20 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.215+0000 7f7d13fff640 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfeca20 0x5610bcff24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7cf2ffd640 10 client.7305 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d13fff640 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfeca20 0x5610bcff24b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0dd151641da0a6 server_cookie=3ed74a145524ab2f in_seq=0 out_seq=0 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7cf2ffd640 1 -- 172.21.15.53:0/3963685890 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610bcf309b0 con 0x5610bcfeca20 2024-05-10T10:40:59.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7cf2ffd640 10 client.7305 handle_command_reply: tid=0 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 client.7305 shutdown 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 msgr2=0x5610bcfc2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfc2700 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800cd80 tx=0x7f7d0805f450 comp rx=0 tx=0).stop 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 shutdown_connections 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x5610bcfeca20 0x5610bcff24b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7cf2ffd640 0 client.7305 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610bcfebd70 0x5610bcff1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610bcfeb450 0x5610bcfc2700 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 --2- 172.21.15.53:0/3963685890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5610bcfe1890 0x5610bd08f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 >> 172.21.15.53:0/3963685890 conn(0x5610bcfc3160 msgr2=0x5610bd030ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 shutdown_connections 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 1 -- 172.21.15.53:0/3963685890 wait complete. 2024-05-10T10:40:59.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.216+0000 7f7d1d3ca740 20 client.7305 trim_cache size 0 max 16384 2024-05-10T10:40:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:40:59.223 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:40:59.223 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:40:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d000762d0 msgr2=0x7f7d00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d000762d0 0x7f7d00078790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d04007fd0 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 msgr2=0x7f7d140c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d140c7580 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800c3c0 tx=0x7f7d0800c880 comp rx=0 tx=0).stop 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 shutdown_connections 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d000762d0 0x7f7d00078790 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d14159340 0x7f7d140b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d140cb9e0 0x7f7d140c7580 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 --2- 172.21.15.53:0/1554123515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d140cb580 0x7f7d14149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 >> 172.21.15.53:0/1554123515 conn(0x7f7d140d0c90 msgr2=0x7f7d14150230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 shutdown_connections 2024-05-10T10:40:59.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.217+0000 7f7d1a5da640 1 -- 172.21.15.53:0/1554123515 wait complete. 2024-05-10T10:40:59.229 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:40:59.229 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:40:59.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.333+0000 7f1c21eda640 1 Processor -- start 2024-05-10T10:40:59.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.333+0000 7f1c21eda640 1 -- start start 2024-05-10T10:40:59.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.334+0000 7f1c21eda640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1c1c1611e0 0x7f1c1c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.334+0000 7f1c21eda640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1c0c6a80 con 0x7f1c1c15ce60 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.334+0000 7f1c21eda640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1c0cd090 con 0x7f1c1c0c65f0 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.334+0000 7f1c21eda640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1c1c0cd370 con 0x7f1c1c1611e0 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.334+0000 7f1c1b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1c1c15ce60 0x7f1c1c160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48142/0 (socket says 172.21.15.53:48142) 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.334+0000 7f1c1b7fe640 1 -- 172.21.15.53:0/4241008686 learned_addr learned my addr 172.21.15.53:0/4241008686 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.334+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4234126714 0 0) 0x7f1c0c001110 con 0x7f1c1c15ce60 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c04003520 con 0x7f1c1c15ce60 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1affd640 1 --2- 172.21.15.53:0/4241008686 >> v2:172.21.15.135:3300/0 conn(0x7f1c1c1611e0 0x7f1c1c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 346987351 0 0) 0x7f1c04003520 con 0x7f1c1c15ce60 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 >> v2:172.21.15.135:3300/0 conn(0x7f1c1c1611e0 msgr2=0x7f1c1c0cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 --2- 172.21.15.53:0/4241008686 >> v2:172.21.15.135:3300/0 conn(0x7f1c1c1611e0 0x7f1c1c0cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 >> v1:172.21.15.135:6789/0 conn(0x7f1c1c0c65f0 legacy=0x7f1c1c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c0cd650 con 0x7f1c1c15ce60 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f1c0c001840 con 0x7f1c1c15ce60 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 >> v1:172.21.15.53:6789/0 conn(0x7f1c1c15ce60 legacy=0x7f1c1c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040039b0 0x7f1c04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c04004800 0x7f1c04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c040051c0 0x7f1c0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c0cd370 con 0x7f1c04004800 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0400c160 con 0x7f1c040039b0 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0400c360 con 0x7f1c040051c0 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1affd640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c04004800 0x7f1c04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1b7fe640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040039b0 0x7f1c04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1bfff640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c040051c0 0x7f1c0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1affd640 1 -- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c040051c0 msgr2=0x7f1c0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1affd640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c040051c0 0x7f1c0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1affd640 1 -- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040039b0 msgr2=0x7f1c04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1affd640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c040039b0 0x7f1c04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1affd640 1 -- 172.21.15.53:0/4241008686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c04003520 con 0x7f1c04004800 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.335+0000 7f1c1bfff640 1 -- 172.21.15.53:0/4241008686 reap_dead start 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.336+0000 7f1c1affd640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c04004800 0x7f1c04004c80 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ac0 tx=0x7f1c04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d954a6f5240ef81 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.336+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10036070 con 0x7f1c04004800 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.336+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c10031400 con 0x7f1c04004800 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.336+0000 7f1c1a7fc640 1 -- 172.21.15.53:0/4241008686 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10009690 con 0x7f1c04004800 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.337+0000 7f1c21eda640 1 -- 172.21.15.53:0/4241008686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c04004800 msgr2=0x7f1c04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.337+0000 7f1c21eda640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c04004800 0x7f1c04004c80 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ac0 tx=0x7f1c04005830 comp rx=0 tx=0).stop 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.337+0000 7f1c21eda640 1 -- 172.21.15.53:0/4241008686 shutdown_connections 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.337+0000 7f1c21eda640 1 --2- 172.21.15.53:0/4241008686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c04004800 0x7f1c04004c80 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.337+0000 7f1c21eda640 1 -- 172.21.15.53:0/4241008686 >> 172.21.15.53:0/4241008686 conn(0x7f1c1c0b2900 msgr2=0x7f1c1c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.337+0000 7f1c21eda640 1 -- 172.21.15.53:0/4241008686 shutdown_connections 2024-05-10T10:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.337+0000 7f1c21eda640 1 -- 172.21.15.53:0/4241008686 wait complete. 2024-05-10T10:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.338+0000 7f1c21eda640 1 Processor -- start 2024-05-10T10:40:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.338+0000 7f1c21eda640 1 -- start start 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c21eda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c04004800 0x7f1c1c0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c21eda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c1c151750 0x7f1c1c151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c21eda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c1c0afb30 0x7f1c1c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c21eda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c0cde50 con 0x7f1c1c0afb30 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c21eda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c0ccd60 con 0x7f1c04004800 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c21eda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c0cdfd0 con 0x7f1c1c151750 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c1c0afb30 0x7f1c1c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c1c0afb30 0x7f1c1c0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59142/0 (socket says 172.21.15.53:59142) 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 -- 172.21.15.53:0/1228622492 learned_addr learned my addr 172.21.15.53:0/1228622492 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1b7fe640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c04004800 0x7f1c1c0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1affd640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c1c151750 0x7f1c1c151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 -- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c1c151750 msgr2=0x7f1c1c151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c1c151750 0x7f1c1c151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 -- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c04004800 msgr2=0x7f1c1c0aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c04004800 0x7f1c1c0aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 -- 172.21.15.53:0/1228622492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c0addf0 con 0x7f1c1c0afb30 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.339+0000 7f1c1bfff640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c1c0afb30 0x7f1c1c0c7e60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1c14007fd0 tx=0x7f1c1400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.340+0000 7f1c03fff640 1 -- 172.21.15.53:0/1228622492 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14019070 con 0x7f1c1c0afb30 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.340+0000 7f1c03fff640 1 -- 172.21.15.53:0/1228622492 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c14003b80 con 0x7f1c1c0afb30 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.340+0000 7f1c03fff640 1 -- 172.21.15.53:0/1228622492 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140050d0 con 0x7f1c1c0afb30 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.340+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c0ae0d0 con 0x7f1c1c0afb30 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.340+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c0ae690 con 0x7f1c1c0afb30 2024-05-10T10:40:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.341+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c06a220 con 0x7f1c1c0afb30 2024-05-10T10:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.341+0000 7f1c03fff640 1 -- 172.21.15.53:0/1228622492 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1c1401c070 con 0x7f1c1c0afb30 2024-05-10T10:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.342+0000 7f1c03fff640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1c080762d0 0x7f1c08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.342+0000 7f1c03fff640 1 -- 172.21.15.53:0/1228622492 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f1c14096d70 con 0x7f1c1c0afb30 2024-05-10T10:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.342+0000 7f1c1b7fe640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1c080762d0 0x7f1c08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.342+0000 7f1c1b7fe640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1c080762d0 0x7f1c08078790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c001820 tx=0x7f1c0c001110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.346+0000 7f1c03fff640 1 -- 172.21.15.53:0/1228622492 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1c1405fe60 con 0x7f1c1c0afb30 2024-05-10T10:40:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.580+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c1c09ec60 con 0x7f1c1c0afb30 2024-05-10T10:40:59.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.580+0000 7f1c03fff640 1 -- 172.21.15.53:0/1228622492 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5187 (secure 0 0 0) 0x7f1c14064160 con 0x7f1c1c0afb30 2024-05-10T10:40:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:59.581 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:25.917116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7152,"qdb_cluster":[7152]},"id":6}]} 2024-05-10T10:40:59.581 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 74 2024-05-10T10:40:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1c080762d0 msgr2=0x7f1c08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1c080762d0 0x7f1c08078790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c001820 tx=0x7f1c0c001110 comp rx=0 tx=0).stop 2024-05-10T10:40:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c1c0afb30 msgr2=0x7f1c1c0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c1c0afb30 0x7f1c1c0c7e60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1c14007fd0 tx=0x7f1c1400ead0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 shutdown_connections 2024-05-10T10:40:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1c080762d0 0x7f1c08078790 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c1c0afb30 0x7f1c1c0c7e60 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c1c151750 0x7f1c1c151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 --2- 172.21.15.53:0/1228622492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c04004800 0x7f1c1c0aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.582+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 >> 172.21.15.53:0/1228622492 conn(0x7f1c1c0b2900 msgr2=0x7f1c1c072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.583+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 shutdown_connections 2024-05-10T10:40:59.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.583+0000 7f1c21eda640 1 -- 172.21.15.53:0/1228622492 wait complete. 2024-05-10T10:40:59.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-10T10:40:59.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.700+0000 7fe5438a7640 1 Processor -- start 2024-05-10T10:40:59.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.700+0000 7fe5438a7640 1 -- start start 2024-05-10T10:40:59.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe5438a7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe53c0cf3e0 0x7fe53c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe5438a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0d2190 0x7fe53c0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe5438a7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c06e720 con 0x7fe53c0cd500 2024-05-10T10:40:59.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe5438a7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe53c06de80 con 0x7fe53c0cf3e0 2024-05-10T10:40:59.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe5438a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c06e160 con 0x7fe53c0d2190 2024-05-10T10:40:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe541643640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe53c0cf3e0 0x7fe53c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe540e42640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe53c0cd500 0x7fe53c0b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:60326/0 (socket says 172.21.15.53:60326) 2024-05-10T10:40:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe541e44640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0d2190 0x7fe53c0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.701+0000 7fe540e42640 1 -- 172.21.15.53:0/340398733 learned_addr learned my addr 172.21.15.53:0/340398733 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.702+0000 7fe533fff640 1 -- 172.21.15.53:0/340398733 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741936928 0 0) 0x7fe53c06e720 con 0x7fe53c0cd500 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.702+0000 7fe533fff640 1 -- 172.21.15.53:0/340398733 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe528003520 con 0x7fe53c0cd500 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.702+0000 7fe541e44640 1 -- 172.21.15.53:0/340398733 >> v2:172.21.15.135:3300/0 conn(0x7fe53c0cf3e0 msgr2=0x7fe53c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.702+0000 7fe541e44640 1 --2- 172.21.15.53:0/340398733 >> v2:172.21.15.135:3300/0 conn(0x7fe53c0cf3e0 0x7fe53c0cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.702+0000 7fe541e44640 1 -- 172.21.15.53:0/340398733 >> v1:172.21.15.135:6789/0 conn(0x7fe53c0cd500 legacy=0x7fe53c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.702+0000 7fe541e44640 1 -- 172.21.15.53:0/340398733 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c06e440 con 0x7fe53c0d2190 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.702+0000 7fe541643640 1 --2- 172.21.15.53:0/340398733 >> v2:172.21.15.135:3300/0 conn(0x7fe53c0cf3e0 0x7fe53c0cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe541e44640 1 --2- 172.21.15.53:0/340398733 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0d2190 0x7fe53c0cfe00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe534009d90 tx=0x7fe53402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c736dac0a7cf19fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe533fff640 1 -- 172.21.15.53:0/340398733 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534007740 con 0x7fe53c0d2190 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe533fff640 1 -- 172.21.15.53:0/340398733 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5340078e0 con 0x7fe53c0d2190 2024-05-10T10:40:59.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe533fff640 1 -- 172.21.15.53:0/340398733 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534007cb0 con 0x7fe53c0d2190 2024-05-10T10:40:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 -- 172.21.15.53:0/340398733 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0d2190 msgr2=0x7fe53c0cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 --2- 172.21.15.53:0/340398733 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0d2190 0x7fe53c0cfe00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe534009d90 tx=0x7fe53402a660 comp rx=0 tx=0).stop 2024-05-10T10:40:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 -- 172.21.15.53:0/340398733 shutdown_connections 2024-05-10T10:40:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 --2- 172.21.15.53:0/340398733 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0d2190 0x7fe53c0cfe00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 --2- 172.21.15.53:0/340398733 >> v2:172.21.15.135:3300/0 conn(0x7fe53c0cf3e0 0x7fe53c0cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 -- 172.21.15.53:0/340398733 >> 172.21.15.53:0/340398733 conn(0x7fe53c0d2eb0 msgr2=0x7fe53c0d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 -- 172.21.15.53:0/340398733 shutdown_connections 2024-05-10T10:40:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.703+0000 7fe5438a7640 1 -- 172.21.15.53:0/340398733 wait complete. 2024-05-10T10:40:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 Processor -- start 2024-05-10T10:40:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 -- start start 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe53c0cd500 0x7fe53c0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0cf3e0 0x7fe53c1f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe53c0d2190 0x7fe53c1f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0b34e0 con 0x7fe53c0d2190 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0b4510 con 0x7fe53c0cd500 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe5438a7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c0b3660 con 0x7fe53c0cf3e0 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.705+0000 7fe541e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe53c0d2190 0x7fe53c1f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe53c0d2190 0x7fe53c1f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59150/0 (socket says 172.21.15.53:59150) 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 -- 172.21.15.53:0/1647534152 learned_addr learned my addr 172.21.15.53:0/1647534152 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541643640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe53c0cd500 0x7fe53c0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe540e42640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0cf3e0 0x7fe53c1f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 -- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0cf3e0 msgr2=0x7fe53c1f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0cf3e0 0x7fe53c1f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 -- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe53c0cd500 msgr2=0x7fe53c0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe53c0cd500 0x7fe53c0cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 -- 172.21.15.53:0/1647534152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c15f580 con 0x7fe53c0d2190 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe540e42640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0cf3e0 0x7fe53c1f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541643640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe53c0cd500 0x7fe53c0cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe541e44640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe53c0d2190 0x7fe53c1f8860 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe534009eb0 tx=0x7fe534004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe531ffb640 1 -- 172.21.15.53:0/1647534152 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53403f070 con 0x7fe53c0d2190 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe531ffb640 1 -- 172.21.15.53:0/1647534152 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe534031bc0 con 0x7fe53c0d2190 2024-05-10T10:40:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c15f860 con 0x7fe53c0d2190 2024-05-10T10:40:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.706+0000 7fe531ffb640 1 -- 172.21.15.53:0/1647534152 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534009450 con 0x7fe53c0d2190 2024-05-10T10:40:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.707+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c0c7a30 con 0x7fe53c0d2190 2024-05-10T10:40:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.708+0000 7fe531ffb640 1 -- 172.21.15.53:0/1647534152 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe53403b410 con 0x7fe53c0d2190 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.709+0000 7fe531ffb640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe5200762d0 0x7fe520078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.709+0000 7fe531ffb640 1 -- 172.21.15.53:0/1647534152 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fe5340b90f0 con 0x7fe53c0d2190 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.709+0000 7fe541643640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe5200762d0 0x7fe520078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.709+0000 7fe5465c9740 1 Processor -- start 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.709+0000 7fe5465c9740 1 -- start start 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.709+0000 7fe541643640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe5200762d0 0x7fe520078790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0b8da0 tx=0x7fe538008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecb82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ece7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece80b0 0x55886ecbcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe5465c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55886eba8920 con 0x55886ece80b0 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55886ec1ddd0 con 0x55886ece87f0 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55886ebce3e0 con 0x55886ed056d0 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe541e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece80b0 0x55886ecbcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe541e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece80b0 0x55886ecbcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59162/0 (socket says 172.21.15.53:59162) 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe541e44640 1 -- 172.21.15.53:0/1406136799 learned_addr learned my addr 172.21.15.53:0/1406136799 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe540e42640 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ece7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.710+0000 7fe541643640 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecb82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe541e44640 1 -- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 msgr2=0x55886ecb82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe541e44640 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecb82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe541e44640 1 -- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 msgr2=0x55886ece7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe541e44640 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ece7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe541e44640 1 -- 172.21.15.53:0/1406136799 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886e9ff7c0 con 0x55886ece80b0 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe541e44640 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece80b0 0x55886ecbcec0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe53407e080 tx=0x7fe53407e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff515ad86f04e41c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe526ffd640 1 -- 172.21.15.53:0/1406136799 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534082000 con 0x55886ece80b0 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe526ffd640 1 -- 172.21.15.53:0/1406136799 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5340863b0 con 0x55886ece80b0 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe526ffd640 1 -- 172.21.15.53:0/1406136799 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53408add0 con 0x55886ece80b0 2024-05-10T10:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe5465c9740 1 -- 172.21.15.53:0/1406136799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece80b0 msgr2=0x55886ecbcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.711+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece80b0 0x55886ecbcec0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe53407e080 tx=0x7fe53407e650 comp rx=0 tx=0).stop 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 -- 172.21.15.53:0/1406136799 shutdown_connections 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece80b0 0x55886ecbcec0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ece7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1406136799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecb82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 -- 172.21.15.53:0/1406136799 >> 172.21.15.53:0/1406136799 conn(0x55886ecba560 msgr2=0x55886ed02b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 -- 172.21.15.53:0/1406136799 shutdown_connections 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 -- 172.21.15.53:0/1406136799 wait complete. 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 -- start start 2024-05-10T10:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.712+0000 7fe5465c9740 1 Processor -- start 2024-05-10T10:40:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ecf9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecd8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe5465c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55886ec26440 con 0x55886ece3d70 2024-05-10T10:40:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55886ec21120 con 0x55886ece87f0 2024-05-10T10:40:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55886ec2f210 con 0x55886ed056d0 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe5465c9740 10 client.0 _ll_register_callbacks cb 0x55886ebc6690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59170/0 (socket says 172.21.15.53:59170) 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 -- 172.21.15.53:0/1700531548 learned_addr learned my addr 172.21.15.53:0/1700531548 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541643640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ecf9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe540e42640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecd8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 -- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 msgr2=0x55886ecd8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecd8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 -- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 msgr2=0x55886ecf9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ecf9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe541e44640 1 -- 172.21.15.53:0/1700531548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886ea5ff50 con 0x55886ece3d70 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.713+0000 7fe540e42640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecd8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe541e44640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbc910 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe53407f270 tx=0x7fe534009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d37ac9a4f8a093 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe51bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe51bfff640 1 -- 172.21.15.53:0/1700531548 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534082000 con 0x55886ece3d70 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe51bfff640 1 -- 172.21.15.53:0/1700531548 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fe534082470 con 0x55886ece3d70 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5340863b0 con 0x55886ece3d70 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe51bfff640 1 -- 172.21.15.53:0/1700531548 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fe53408a940 con 0x55886ece3d70 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 10 client.7320 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55886e8f30b0 con 0x55886ece3d70 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe51bfff640 1 -- 172.21.15.53:0/1700531548 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5340863b0 con 0x55886ece3d70 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe51bfff640 1 -- 172.21.15.53:0/1700531548 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fe534081260 con 0x55886ece3d70 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 10 client.7320 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 10 client.7320 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 20 client.7320 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed1d970 0x55886ed1fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 4 client.7320 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886ec2fa20 con 0x55886ed1d970 2024-05-10T10:40:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.714+0000 7fe541643640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed1d970 0x55886ed1fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe541643640 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed1d970 0x55886ed1fd60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36675d411e99f2e4 server_cookie=14e14547cf9235e2 in_seq=0 out_seq=0 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe51bfff640 10 client.7320 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe51bfff640 1 -- 172.21.15.53:0/1700531548 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55886ec2fa20 con 0x55886ed1d970 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe51bfff640 10 client.7320 handle_command_reply: tid=0 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 client.7320 shutdown 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 msgr2=0x55886ecbc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbc910 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe53407f270 tx=0x7fe534009750 comp rx=0 tx=0).stop 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 shutdown_connections 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed1d970 0x55886ed1fd60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbc910 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ecd8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1700531548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ecf9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe51bfff640 0 client.7320 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.715+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 >> 172.21.15.53:0/1700531548 conn(0x55886ece80b0 msgr2=0x55886ecb9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.716+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 shutdown_connections 2024-05-10T10:40:59.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.716+0000 7fe5465c9740 1 -- 172.21.15.53:0/1700531548 wait complete. 2024-05-10T10:40:59.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.716+0000 7fe5465c9740 20 client.7320 trim_cache size 0 max 16384 2024-05-10T10:40:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 Processor -- start 2024-05-10T10:40:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 -- start start 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ecb7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ed11270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c201140 con 0x55886ece3d70 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1f5c30 con 0x55886ece87f0 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c0c9010 con 0x55886ed056d0 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe541643640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe541643640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59182/0 (socket says 172.21.15.53:59182) 2024-05-10T10:40:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.730+0000 7fe541643640 1 -- 172.21.15.53:0/1542211018 learned_addr learned my addr 172.21.15.53:0/1542211018 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe541643640 1 -- 172.21.15.53:0/1542211018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 msgr2=0x55886ed11270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe541643640 1 --2- 172.21.15.53:0/1542211018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ed11270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe541643640 1 -- 172.21.15.53:0/1542211018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 msgr2=0x55886ecb7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe541643640 1 --2- 172.21.15.53:0/1542211018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ecb7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe541643640 1 -- 172.21.15.53:0/1542211018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886ece1df0 con 0x55886ece3d70 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe541643640 1 --2- 172.21.15.53:0/1542211018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbbb60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x55886ec2f270 tx=0x7fe538013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cea54bb4541548 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5267fc640 1 -- 172.21.15.53:0/1542211018 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53801c040 con 0x55886ece3d70 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5267fc640 1 -- 172.21.15.53:0/1542211018 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53801d040 con 0x55886ece3d70 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5267fc640 1 -- 172.21.15.53:0/1542211018 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53800f040 con 0x55886ece3d70 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5465c9740 1 -- 172.21.15.53:0/1542211018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 msgr2=0x55886ecbbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1542211018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbbb60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x55886ec2f270 tx=0x7fe538013000 comp rx=0 tx=0).stop 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5465c9740 1 -- 172.21.15.53:0/1542211018 shutdown_connections 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1542211018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ed11270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1542211018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ecb7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.731+0000 7fe5465c9740 1 --2- 172.21.15.53:0/1542211018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ecbbb60 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.732+0000 7fe5465c9740 1 -- 172.21.15.53:0/1542211018 >> 172.21.15.53:0/1542211018 conn(0x55886ece80b0 msgr2=0x55886ed029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.732+0000 7fe5465c9740 1 -- 172.21.15.53:0/1542211018 shutdown_connections 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.732+0000 7fe5465c9740 1 -- 172.21.15.53:0/1542211018 wait complete. 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.732+0000 7fe5465c9740 1 -- start start 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.732+0000 7fe5465c9740 1 Processor -- start 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ed83290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ed02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe5465c9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ed11270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe5465c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55886ece91d0 con 0x55886ece3d70 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55886ebd7a20 con 0x55886ece87f0 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe5465c9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55886ed15690 con 0x55886ed056d0 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe5465c9740 10 client.0 _ll_register_callbacks cb 0x7fe52c007cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:40:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ed83290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ed83290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59196/0 (socket says 172.21.15.53:59196) 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 -- 172.21.15.53:0/3516310230 learned_addr learned my addr 172.21.15.53:0/3516310230 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541e44640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ed11270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe540e42640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ed02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 -- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 msgr2=0x55886ed11270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ed11270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 -- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 msgr2=0x55886ed02760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ed02760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe541643640 1 -- 172.21.15.53:0/3516310230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886ed02ca0 con 0x55886ece3d70 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.733+0000 7fe540e42640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ed02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe541643640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ed83290 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe5380107e0 tx=0x7fe5380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c149d2b2a9bbbe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe51bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:40:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe51bfff640 1 -- 172.21.15.53:0/3516310230 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538039060 con 0x55886ece3d70 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe51bfff640 1 -- 172.21.15.53:0/3516310230 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fe53802b850 con 0x55886ece3d70 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe53801d040 con 0x55886ece3d70 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe51bfff640 1 -- 172.21.15.53:0/3516310230 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fe53801c040 con 0x55886ece3d70 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 10 client.7326 fetch_fsmap learned FSMap version 74 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55886ecb8d30 con 0x55886ece3d70 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe51bfff640 1 -- 172.21.15.53:0/3516310230 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53801d040 con 0x55886ece3d70 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe51bfff640 1 -- 172.21.15.53:0/3516310230 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fe53802bce0 con 0x55886ece3d70 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 10 client.7326 fetch_fsmap finished waiting for FSMap version 74 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 10 client.7326 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 20 client.7326 populate_metadata read hostname 'smithi053' 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed15100 0x55886ed05f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 4 client.7326 mds_command: new command op to 7152 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 --> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55886ec29890 con 0x55886ed15100 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.734+0000 7fe540e42640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed15100 0x55886ed05f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe51bfff640 10 client.7326 ms_handle_connect on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe540e42640 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed15100 0x55886ed05f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef28010c9a27bc8 server_cookie=d41adaa78daa54f9 in_seq=0 out_seq=0 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe51bfff640 1 -- 172.21.15.53:0/3516310230 <== mds.0 v2:172.21.15.53:6832/1098976714 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55886ec29890 con 0x55886ed15100 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe51bfff640 10 client.7326 handle_command_reply: tid=0 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 client.7326 shutdown 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 msgr2=0x55886ed83290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ed83290 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe5380107e0 tx=0x7fe5380046c0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 shutdown_connections 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:6832/1098976714,v1:172.21.15.53:6833/1098976714] conn(0x55886ed15100 0x55886ed05f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55886ed056d0 0x55886ed11270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886ece87f0 0x55886ed02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe51bfff640 0 client.7326 ms_handle_reset on v2:172.21.15.53:6832/1098976714 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 --2- 172.21.15.53:0/3516310230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55886ece3d70 0x55886ed83290 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 >> 172.21.15.53:0/3516310230 conn(0x55886ece80b0 msgr2=0x55886ed01c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 shutdown_connections 2024-05-10T10:40:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 1 -- 172.21.15.53:0/3516310230 wait complete. 2024-05-10T10:40:59.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.735+0000 7fe5465c9740 20 client.7326 trim_cache size 0 max 16384 2024-05-10T10:40:59.745 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:40:59.745 INFO:teuthology.orchestra.run.smithi053.stdout:[] 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe5200762d0 msgr2=0x7fe520078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe5200762d0 0x7fe520078790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0b8da0 tx=0x7fe538008040 comp rx=0 tx=0).stop 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe53c0d2190 msgr2=0x7fe53c1f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe53c0d2190 0x7fe53c1f8860 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe534009eb0 tx=0x7fe534004a40 comp rx=0 tx=0).stop 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 shutdown_connections 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe5200762d0 0x7fe520078790 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe53c0d2190 0x7fe53c1f8860 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe53c0cf3e0 0x7fe53c1f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 --2- 172.21.15.53:0/1647534152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe53c0cd500 0x7fe53c0cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 >> 172.21.15.53:0/1647534152 conn(0x7fe53c0d2eb0 msgr2=0x7fe53c1623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 shutdown_connections 2024-05-10T10:40:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.736+0000 7fe5438a7640 1 -- 172.21.15.53:0/1647534152 wait complete. 2024-05-10T10:40:59.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T10:40:59.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.850+0000 7fba45608640 1 Processor -- start 2024-05-10T10:40:59.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba45608640 1 -- start start 2024-05-10T10:40:59.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba45608640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fba400b5520 0x7fba400d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba45608640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b9180 0x7fba400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba45608640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba400d41f0 con 0x7fba400b5900 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba45608640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fba400d5c60 con 0x7fba400b5520 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba45608640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fba400d5eb0 con 0x7fba400b9180 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba3effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fba400b5520 0x7fba400d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba3e7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fba400b5900 0x7fba400d2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48146/0 (socket says 172.21.15.53:48146) 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.851+0000 7fba3e7fc640 1 -- 172.21.15.53:0/951884230 learned_addr learned my addr 172.21.15.53:0/951884230 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.852+0000 7fba3f7fe640 1 --2- 172.21.15.53:0/951884230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b9180 0x7fba400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.852+0000 7fba3effd640 1 -- 172.21.15.53:0/951884230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b9180 msgr2=0x7fba400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.852+0000 7fba3effd640 1 --2- 172.21.15.53:0/951884230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b9180 0x7fba400d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.852+0000 7fba3effd640 1 -- 172.21.15.53:0/951884230 >> v1:172.21.15.53:6789/0 conn(0x7fba400b5900 legacy=0x7fba400d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.852+0000 7fba3effd640 1 -- 172.21.15.53:0/951884230 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba400d0a30 con 0x7fba400b5520 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.852+0000 7fba3effd640 1 --2- 172.21.15.53:0/951884230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400d18a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fba30009d90 tx=0x7fba3002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4d516f3d07117f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba3dffb640 1 -- 172.21.15.53:0/951884230 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30031600 con 0x7fba400b5520 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba3dffb640 1 -- 172.21.15.53:0/951884230 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba300317a0 con 0x7fba400b5520 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba3dffb640 1 -- 172.21.15.53:0/951884230 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30031b70 con 0x7fba400b5520 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 -- 172.21.15.53:0/951884230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 msgr2=0x7fba400d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 --2- 172.21.15.53:0/951884230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400d18a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fba30009d90 tx=0x7fba3002a5d0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 -- 172.21.15.53:0/951884230 shutdown_connections 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 --2- 172.21.15.53:0/951884230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b9180 0x7fba400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 --2- 172.21.15.53:0/951884230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400d18a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 -- 172.21.15.53:0/951884230 >> 172.21.15.53:0/951884230 conn(0x7fba400d5170 msgr2=0x7fba400d5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 -- 172.21.15.53:0/951884230 shutdown_connections 2024-05-10T10:40:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.853+0000 7fba45608640 1 -- 172.21.15.53:0/951884230 wait complete. 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 Processor -- start 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 -- start start 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b5900 0x7fba401f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba400b9180 0x7fba400b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba400b6460 con 0x7fba400b5520 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba400b62e0 con 0x7fba400b9180 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.855+0000 7fba45608640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fba400b65e0 con 0x7fba400b5900 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59214/0 (socket says 172.21.15.53:59214) 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 -- 172.21.15.53:0/1649029467 learned_addr learned my addr 172.21.15.53:0/1649029467 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:40:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3f7fe640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba400b9180 0x7fba400b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3e7fc640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b5900 0x7fba401f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 -- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b5900 msgr2=0x7fba401f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b5900 0x7fba401f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 -- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba400b9180 msgr2=0x7fba400b9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba400b9180 0x7fba400b9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 -- 172.21.15.53:0/1649029467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba400cc780 con 0x7fba400b5520 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3f7fe640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba400b9180 0x7fba400b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3e7fc640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b5900 0x7fba401f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba3effd640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400cc240 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fba30009eb0 tx=0x7fba30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba277fe640 1 -- 172.21.15.53:0/1649029467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30040070 con 0x7fba400b5520 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba277fe640 1 -- 172.21.15.53:0/1649029467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba30030be0 con 0x7fba400b5520 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4014f200 con 0x7fba400b5520 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba277fe640 1 -- 172.21.15.53:0/1649029467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30009450 con 0x7fba400b5520 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.856+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4014f7c0 con 0x7fba400b5520 2024-05-10T10:40:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.857+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba400693f0 con 0x7fba400b5520 2024-05-10T10:40:59.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.858+0000 7fba277fe640 1 -- 172.21.15.53:0/1649029467 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fba30039040 con 0x7fba400b5520 2024-05-10T10:40:59.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.858+0000 7fba277fe640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fba2c076300 0x7fba2c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:40:59.860 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.858+0000 7fba277fe640 1 -- 172.21.15.53:0/1649029467 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fba300b8f70 con 0x7fba400b5520 2024-05-10T10:40:59.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.858+0000 7fba3e7fc640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fba2c076300 0x7fba2c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:40:59.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.859+0000 7fba3e7fc640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fba2c076300 0x7fba2c0787c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba401f8300 tx=0x7fba34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:40:59.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:40:59.863+0000 7fba277fe640 1 -- 172.21.15.53:0/1649029467 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fba30082620 con 0x7fba400b5520 2024-05-10T10:41:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.092+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fba400d3fb0 con 0x7fba400b5520 2024-05-10T10:41:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.093+0000 7fba277fe640 1 -- 172.21.15.53:0/1649029467 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fba30081f20 con 0x7fba400b5520 2024-05-10T10:41:00.093 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:00.094 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T10:41:00.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fba2c076300 msgr2=0x7fba2c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fba2c076300 0x7fba2c0787c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba401f8300 tx=0x7fba34009040 comp rx=0 tx=0).stop 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 msgr2=0x7fba400cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400cc240 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fba30009eb0 tx=0x7fba30004a40 comp rx=0 tx=0).stop 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 shutdown_connections 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fba2c076300 0x7fba2c0787c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba400b9180 0x7fba400b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba400b5900 0x7fba401f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 --2- 172.21.15.53:0/1649029467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba400b5520 0x7fba400cc240 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 >> 172.21.15.53:0/1649029467 conn(0x7fba400d5170 msgr2=0x7fba400b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 shutdown_connections 2024-05-10T10:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.095+0000 7fba45608640 1 -- 172.21.15.53:0/1649029467 wait complete. 2024-05-10T10:41:00.106 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T10:41:00.107 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:41:00.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:41:00.150 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:41:00.201 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:41:00.201 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:41:00.201 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:00.201 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:00.217 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:00.218 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:00.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:41:00.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.419+0000 7f88c8e08640 1 Processor -- start 2024-05-10T10:41:00.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.419+0000 7f88c8e08640 1 -- start start 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c8e08640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f88c40cfd80 0x7f88c416a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c8e08640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c415fc90 con 0x7f88c40d0230 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c8e08640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c40b8c40 con 0x7f88c40b8860 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c8e08640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f88c406ec30 con 0x7f88c40cfd80 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88bbfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f88c40d0230 0x7f88c40cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48160/0 (socket says 172.21.15.53:48160) 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88bbfff640 1 -- 172.21.15.53:0/1120288570 learned_addr learned my addr 172.21.15.53:0/1120288570 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c2d9d640 1 --2- 172.21.15.53:0/1120288570 >> v2:172.21.15.53:3300/0 conn(0x7f88c40cfd80 0x7f88c416a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c1d9b640 1 -- 172.21.15.53:0/1120288570 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094822951 0 0) 0x7f88c415fc90 con 0x7f88c40d0230 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c1d9b640 1 -- 172.21.15.53:0/1120288570 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88b0003520 con 0x7f88c40d0230 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c2d9d640 1 -- 172.21.15.53:0/1120288570 >> v1:172.21.15.135:6789/0 conn(0x7f88c40b8860 legacy=0x7f88c406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c2d9d640 1 -- 172.21.15.53:0/1120288570 >> v1:172.21.15.53:6789/0 conn(0x7f88c40d0230 legacy=0x7f88c40cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.420+0000 7f88c2d9d640 1 -- 172.21.15.53:0/1120288570 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c40cd570 con 0x7f88c40cfd80 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.421+0000 7f88c2d9d640 1 --2- 172.21.15.53:0/1120288570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40cfd80 0x7f88c416a3a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009d90 tx=0x7f88bc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da83eb952bb30a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.421+0000 7f88c1d9b640 1 -- 172.21.15.53:0/1120288570 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc031600 con 0x7f88c40cfd80 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.421+0000 7f88c1d9b640 1 -- 172.21.15.53:0/1120288570 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc0317a0 con 0x7f88c40cfd80 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.421+0000 7f88c1d9b640 1 -- 172.21.15.53:0/1120288570 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc031b70 con 0x7f88c40cfd80 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.421+0000 7f88c8e08640 1 -- 172.21.15.53:0/1120288570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40cfd80 msgr2=0x7f88c416a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.421+0000 7f88c8e08640 1 --2- 172.21.15.53:0/1120288570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40cfd80 0x7f88c416a3a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009d90 tx=0x7f88bc02a610 comp rx=0 tx=0).stop 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.422+0000 7f88c8e08640 1 -- 172.21.15.53:0/1120288570 shutdown_connections 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.422+0000 7f88c8e08640 1 --2- 172.21.15.53:0/1120288570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40cfd80 0x7f88c416a3a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.422+0000 7f88c8e08640 1 -- 172.21.15.53:0/1120288570 >> 172.21.15.53:0/1120288570 conn(0x7f88c40d2df0 msgr2=0x7f88c40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:00.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.422+0000 7f88c8e08640 1 -- 172.21.15.53:0/1120288570 shutdown_connections 2024-05-10T10:41:00.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.422+0000 7f88c8e08640 1 -- 172.21.15.53:0/1120288570 wait complete. 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.423+0000 7f88c8e08640 1 Processor -- start 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.423+0000 7f88c8e08640 1 -- start start 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c8e08640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88c40b8860 0x7f88c40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c8e08640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88c40cfd80 0x7f88c4162890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c8e08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40d0230 0x7f88c40a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c8e08640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f88c40cdb30 con 0x7f88c40d0230 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c8e08640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88c406ee50 con 0x7f88c40cfd80 2024-05-10T10:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c8e08640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f88c40cde10 con 0x7f88c40b8860 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40d0230 0x7f88c40a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40d0230 0x7f88c40a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59232/0 (socket says 172.21.15.53:59232) 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 -- 172.21.15.53:0/709120184 learned_addr learned my addr 172.21.15.53:0/709120184 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88bbfff640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88c40cfd80 0x7f88c4162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 -- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88c40b8860 msgr2=0x7f88c40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c259c640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88c40b8860 0x7f88c40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88c40b8860 0x7f88c40cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 -- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88c40cfd80 msgr2=0x7f88c4162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88c40cfd80 0x7f88c4162890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 -- 172.21.15.53:0/709120184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c41f8610 con 0x7f88c40d0230 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88bbfff640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88c40cfd80 0x7f88c4162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c259c640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88c40b8860 0x7f88c40cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c2d9d640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40d0230 0x7f88c40a0010 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009d90 tx=0x7f88bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88baffd640 1 -- 172.21.15.53:0/709120184 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc040070 con 0x7f88c40d0230 2024-05-10T10:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88baffd640 1 -- 172.21.15.53:0/709120184 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc030b60 con 0x7f88c40d0230 2024-05-10T10:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c41f88f0 con 0x7f88c40d0230 2024-05-10T10:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.424+0000 7f88baffd640 1 -- 172.21.15.53:0/709120184 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc009450 con 0x7f88c40d0230 2024-05-10T10:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.425+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c41f8eb0 con 0x7f88c40d0230 2024-05-10T10:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.426+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c406a220 con 0x7f88c40d0230 2024-05-10T10:41:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.426+0000 7f88baffd640 1 -- 172.21.15.53:0/709120184 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f88bc039040 con 0x7f88c40d0230 2024-05-10T10:41:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.427+0000 7f88baffd640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f88a40762d0 0x7f88a4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.427+0000 7f88baffd640 1 -- 172.21.15.53:0/709120184 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f88bc0b8850 con 0x7f88c40d0230 2024-05-10T10:41:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.427+0000 7f88c259c640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f88a40762d0 0x7f88a4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.428+0000 7f88c259c640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f88a40762d0 0x7f88a4078790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003eb0 tx=0x7f88b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:00.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.431+0000 7f88baffd640 1 -- 172.21.15.53:0/709120184 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f88bc081830 con 0x7f88c40d0230 2024-05-10T10:41:00.633 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.633+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88c4075140 con 0x7f88c40d0230 2024-05-10T10:41:00.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.633+0000 7f88baffd640 1 -- 172.21.15.53:0/709120184 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v91) ==== 80+0+242 (secure 0 0 0) 0x7f88bc088e20 con 0x7f88c40d0230 2024-05-10T10:41:00.634 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6833/285942496 2024-05-11T10:40:21.895821+0000 2024-05-10T10:41:00.634 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6832/285942496 2024-05-11T10:40:21.895821+0000 2024-05-10T10:41:00.634 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6833/2934725760 2024-05-11T10:39:11.380694+0000 2024-05-10T10:41:00.634 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6832/2934725760 2024-05-11T10:39:11.380694+0000 2024-05-10T10:41:00.635 INFO:teuthology.orchestra.run.smithi053.stderr:listed 4 entries 2024-05-10T10:41:00.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.635+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f88a40762d0 msgr2=0x7f88a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.635+0000 7f88c8e08640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f88a40762d0 0x7f88a4078790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003eb0 tx=0x7f88b4009040 comp rx=0 tx=0).stop 2024-05-10T10:41:00.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.635+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40d0230 msgr2=0x7f88c40a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.635+0000 7f88c8e08640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40d0230 0x7f88c40a0010 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009d90 tx=0x7f88bc004a40 comp rx=0 tx=0).stop 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 shutdown_connections 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f88a40762d0 0x7f88a4078790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88c40d0230 0x7f88c40a0010 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88c40cfd80 0x7f88c4162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 --2- 172.21.15.53:0/709120184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88c40b8860 0x7f88c40cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 >> 172.21.15.53:0/709120184 conn(0x7f88c40d2df0 msgr2=0x7f88c40d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 shutdown_connections 2024-05-10T10:41:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.636+0000 7f88c8e08640 1 -- 172.21.15.53:0/709120184 wait complete. 2024-05-10T10:41:00.647 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:41:00.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:00.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:00.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:41:00.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:00.676 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:41:00.732 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:41:00.733 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:41:00.733 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:00.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:41:00.752 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:41:00.752 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:00.752 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:00.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:41:00.788 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:41:00.789 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:41:00.789 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:00.812 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:41:00.812 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.932+0000 7f3c32cc8640 1 Processor -- start 2024-05-10T10:41:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.932+0000 7f3c32cc8640 1 -- start start 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.932+0000 7f3c32cc8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c2c0d5cb0 0x7f3c2c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.932+0000 7f3c32cc8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c0b45f0 con 0x7f3c2c162750 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.932+0000 7f3c32cc8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c0b48d0 con 0x7f3c2c162b30 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.932+0000 7f3c32cc8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c0b4f20 con 0x7f3c2c0d5cb0 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c30a64640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3c2c162750 0x7f3c2c0d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48172/0 (socket says 172.21.15.53:48172) 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c30a64640 1 -- 172.21.15.53:0/3163429357 learned_addr learned my addr 172.21.15.53:0/3163429357 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3662393981 0 0) 0x7f3c2c0b45f0 con 0x7f3c2c162750 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c18003520 con 0x7f3c2c162750 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2bfff640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c2c0d5cb0 0x7f3c2c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2027428195 0 0) 0x7f3c18003520 con 0x7f3c2c162750 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c2c0d5cb0 msgr2=0x7f3c2c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c2c0d5cb0 0x7f3c2c0d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 >> v1:172.21.15.135:6789/0 conn(0x7f3c2c162b30 legacy=0x7f3c2c0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c0b4bb0 con 0x7f3c2c162750 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f3c1c0019f0 con 0x7f3c2c162750 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 >> v1:172.21.15.53:6789/0 conn(0x7f3c2c162750 legacy=0x7f3c2c0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c180039e0 0x7f3c18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c18004800 0x7f3c18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180051c0 0x7f3c1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0b4f20 con 0x7f3c180039e0 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1800bf00 con 0x7f3c18004800 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1800c0d0 con 0x7f3c180051c0 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c30a64640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c180039e0 0x7f3c18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c2bfff640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c18004800 0x7f3c18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.933+0000 7f3c31265640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180051c0 0x7f3c1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.934+0000 7f3c30a64640 1 -- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180051c0 msgr2=0x7f3c1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.934+0000 7f3c30a64640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180051c0 0x7f3c1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.934+0000 7f3c30a64640 1 -- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c18004800 msgr2=0x7f3c18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.934+0000 7f3c30a64640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c18004800 0x7f3c18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.934+0000 7f3c30a64640 1 -- 172.21.15.53:0/3163429357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c18003520 con 0x7f3c180039e0 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.934+0000 7f3c30a64640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c180039e0 0x7f3c18003e00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00a850 tx=0x7f3c18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2c958eeb6e3585 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.934+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00a4a0 con 0x7f3c180039e0 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c02f070 con 0x7f3c180039e0 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c2b7fe640 1 -- 172.21.15.53:0/3163429357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c02bce0 con 0x7f3c180039e0 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c32cc8640 1 -- 172.21.15.53:0/3163429357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c180039e0 msgr2=0x7f3c18003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c32cc8640 1 --2- 172.21.15.53:0/3163429357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c180039e0 0x7f3c18003e00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00a850 tx=0x7f3c18005830 comp rx=0 tx=0).stop 2024-05-10T10:41:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c31265640 1 -- 172.21.15.53:0/3163429357 reap_dead start 2024-05-10T10:41:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c32cc8640 1 -- 172.21.15.53:0/3163429357 shutdown_connections 2024-05-10T10:41:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c32cc8640 1 -- 172.21.15.53:0/3163429357 >> 172.21.15.53:0/3163429357 conn(0x7f3c2c09e3b0 msgr2=0x7f3c2c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c32cc8640 1 -- 172.21.15.53:0/3163429357 shutdown_connections 2024-05-10T10:41:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.935+0000 7f3c32cc8640 1 -- 172.21.15.53:0/3163429357 wait complete. 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.936+0000 7f3c32cc8640 1 Processor -- start 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.936+0000 7f3c32cc8640 1 -- start start 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c32cc8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180039e0 0x7f3c2c0d5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c32cc8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c2c0a20b0 0x7f3c2c09fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c32cc8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c2c15f1d0 0x7f3c2c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c32cc8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0d3a40 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c32cc8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0d38c0 con 0x7f3c2c15f1d0 2024-05-10T10:41:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c32cc8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c0d3bc0 con 0x7f3c180039e0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c2c0a20b0 0x7f3c2c09fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c2c0a20b0 0x7f3c2c09fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59244/0 (socket says 172.21.15.53:59244) 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.937+0000 7f3c2bfff640 1 -- 172.21.15.53:0/1758152376 learned_addr learned my addr 172.21.15.53:0/1758152376 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c2bfff640 1 -- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180039e0 msgr2=0x7f3c2c0d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c2bfff640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180039e0 0x7f3c2c0d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c2bfff640 1 -- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c2c15f1d0 msgr2=0x7f3c2c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c2bfff640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c2c15f1d0 0x7f3c2c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c2bfff640 1 -- 172.21.15.53:0/1758152376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c15fb90 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c2bfff640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c2c0a20b0 0x7f3c2c09fa40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3c14011600 tx=0x7f3c14011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c297fa640 1 -- 172.21.15.53:0/1758152376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400f920 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c297fa640 1 -- 172.21.15.53:0/1758152376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1400fac0 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c0a25e0 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c297fa640 1 -- 172.21.15.53:0/1758152376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400fe70 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.938+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c0a2ba0 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.939+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c06a220 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.940+0000 7f3c297fa640 1 -- 172.21.15.53:0/1758152376 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3c14015460 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.940+0000 7f3c297fa640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c0c076300 0x7f3c0c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.940+0000 7f3c297fa640 1 -- 172.21.15.53:0/1758152376 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f3c140981c0 con 0x7f3c2c0a20b0 2024-05-10T10:41:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.940+0000 7f3c30a64640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c0c076300 0x7f3c0c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.941+0000 7f3c30a64640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c0c076300 0x7f3c0c0787c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c033000 tx=0x7f3c1c038040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:00.944+0000 7f3c297fa640 1 -- 172.21.15.53:0/1758152376 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3c2c15fb90 con 0x7f3c2c0a20b0 2024-05-10T10:41:01.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.173+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c2c06def0 con 0x7f3c2c0a20b0 2024-05-10T10:41:01.174 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.174+0000 7f3c297fa640 1 -- 172.21.15.53:0/1758152376 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5187 (secure 0 0 0) 0x7f3c14019030 con 0x7f3c2c0a20b0 2024-05-10T10:41:01.175 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:01.175 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:25.917116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7152,"qdb_cluster":[7152]},"id":6}]} 2024-05-10T10:41:01.175 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 74 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c0c076300 msgr2=0x7f3c0c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c0c076300 0x7f3c0c0787c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c033000 tx=0x7f3c1c038040 comp rx=0 tx=0).stop 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c2c0a20b0 msgr2=0x7f3c2c09fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c2c0a20b0 0x7f3c2c09fa40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3c14011600 tx=0x7f3c14011ac0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 shutdown_connections 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3c0c076300 0x7f3c0c0787c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c2c15f1d0 0x7f3c2c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c2c0a20b0 0x7f3c2c09fa40 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 --2- 172.21.15.53:0/1758152376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3c180039e0 0x7f3c2c0d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 >> 172.21.15.53:0/1758152376 conn(0x7f3c2c09e3b0 msgr2=0x7f3c2c160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 shutdown_connections 2024-05-10T10:41:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.176+0000 7f3c32cc8640 1 -- 172.21.15.53:0/1758152376 wait complete. 2024-05-10T10:41:01.188 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:01.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.290+0000 7fdd432a0640 1 Processor -- start 2024-05-10T10:41:01.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.291+0000 7fdd432a0640 1 -- start start 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.291+0000 7fdd432a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c15f3b0 0x7fdd3c0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.291+0000 7fdd432a0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdd3c15f790 0x7fdd3c0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.291+0000 7fdd432a0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd3c0d5d30 0x7fdd3c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.291+0000 7fdd432a0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fdd3c0d30b0 con 0x7fdd3c0d5d30 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.291+0000 7fdd432a0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdd3c0d3390 con 0x7fdd3c15f790 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.291+0000 7fdd432a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c0c8c80 con 0x7fdd3c15f3b0 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd3c0d5d30 0x7fdd3c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd3c0d5d30 0x7fdd3c0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59246/0 (socket says 172.21.15.53:59246) 2024-05-10T10:41:01.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 -- 172.21.15.53:0/3951309827 learned_addr learned my addr 172.21.15.53:0/3951309827 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4103c640 1 --2- 172.21.15.53:0/3951309827 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c15f3b0 0x7fdd3c0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4083b640 1 --2- 172.21.15.53:0/3951309827 >> v2:172.21.15.135:3300/0 conn(0x7fdd3c15f790 0x7fdd3c0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 -- 172.21.15.53:0/3951309827 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c15f3b0 msgr2=0x7fdd3c0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 --2- 172.21.15.53:0/3951309827 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c15f3b0 0x7fdd3c0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 -- 172.21.15.53:0/3951309827 >> v2:172.21.15.135:3300/0 conn(0x7fdd3c15f790 msgr2=0x7fdd3c0d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 --2- 172.21.15.53:0/3951309827 >> v2:172.21.15.135:3300/0 conn(0x7fdd3c15f790 0x7fdd3c0d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.292+0000 7fdd4183d640 1 -- 172.21.15.53:0/3951309827 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c0c8f60 con 0x7fdd3c0d5d30 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd4183d640 1 --2- 172.21.15.53:0/3951309827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c0d5d30 0x7fdd3c0c9760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c06c640 tx=0x7fdd3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ac20a1e277a0c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd3bfff640 1 -- 172.21.15.53:0/3951309827 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34036070 con 0x7fdd3c0d5d30 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd3bfff640 1 -- 172.21.15.53:0/3951309827 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd34031600 con 0x7fdd3c0d5d30 2024-05-10T10:41:01.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd3bfff640 1 -- 172.21.15.53:0/3951309827 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340319d0 con 0x7fdd3c0d5d30 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 -- 172.21.15.53:0/3951309827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c0d5d30 msgr2=0x7fdd3c0c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 --2- 172.21.15.53:0/3951309827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c0d5d30 0x7fdd3c0c9760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c06c640 tx=0x7fdd3402a6a0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 -- 172.21.15.53:0/3951309827 shutdown_connections 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 --2- 172.21.15.53:0/3951309827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c0d5d30 0x7fdd3c0c9760 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 --2- 172.21.15.53:0/3951309827 >> v2:172.21.15.135:3300/0 conn(0x7fdd3c15f790 0x7fdd3c0d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 --2- 172.21.15.53:0/3951309827 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c15f3b0 0x7fdd3c0d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 -- 172.21.15.53:0/3951309827 >> 172.21.15.53:0/3951309827 conn(0x7fdd3c0d5130 msgr2=0x7fdd3c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:01.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 -- 172.21.15.53:0/3951309827 shutdown_connections 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.293+0000 7fdd432a0640 1 -- 172.21.15.53:0/3951309827 wait complete. 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 Processor -- start 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 -- start start 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c0d5d30 0x7fdd3c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c15f3b0 0x7fdd3c1f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c1f55b0 0x7fdd3c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c0c9e20 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c0c9ca0 con 0x7fdd3c15f3b0 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd432a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c0c9fa0 con 0x7fdd3c0d5d30 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd4183d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c1f55b0 0x7fdd3c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd4183d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c1f55b0 0x7fdd3c0cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59248/0 (socket says 172.21.15.53:59248) 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.295+0000 7fdd4183d640 1 -- 172.21.15.53:0/1933411132 learned_addr learned my addr 172.21.15.53:0/1933411132 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4083b640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c15f3b0 0x7fdd3c1f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4103c640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c0d5d30 0x7fdd3c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4183d640 1 -- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c0d5d30 msgr2=0x7fdd3c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4183d640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c0d5d30 0x7fdd3c15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4183d640 1 -- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c15f3b0 msgr2=0x7fdd3c1f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4183d640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c15f3b0 0x7fdd3c1f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4183d640 1 -- 172.21.15.53:0/1933411132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c0b9500 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4083b640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c15f3b0 0x7fdd3c1f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd4183d640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c1f55b0 0x7fdd3c0cd750 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009f40 tx=0x7fdd34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd39ffb640 1 -- 172.21.15.53:0/1933411132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34036050 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd39ffb640 1 -- 172.21.15.53:0/1933411132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd34042070 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c0b8f40 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd39ffb640 1 -- 172.21.15.53:0/1933411132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34009510 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.296+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c0cd230 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.298+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c0cd040 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.298+0000 7fdd39ffb640 1 -- 172.21.15.53:0/1933411132 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdd34007bd0 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.299+0000 7fdd39ffb640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd280762d0 0x7fdd28078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.299+0000 7fdd39ffb640 1 -- 172.21.15.53:0/1933411132 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fdd340b8810 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.299+0000 7fdd4103c640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd280762d0 0x7fdd28078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.299+0000 7fdd4103c640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd280762d0 0x7fdd28078790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c006fd0 tx=0x7fdd2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.303+0000 7fdd39ffb640 1 -- 172.21.15.53:0/1933411132 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fdd3c0b9500 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.533+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd3c0c8a60 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.534+0000 7fdd39ffb640 1 -- 172.21.15.53:0/1933411132 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5187 (secure 0 0 0) 0x7fdd34081e30 con 0x7fdd3c1f55b0 2024-05-10T10:41:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:01.535 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:25.917116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7152,"qdb_cluster":[7152]},"id":6}]} 2024-05-10T10:41:01.535 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 74 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd280762d0 msgr2=0x7fdd28078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd280762d0 0x7fdd28078790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c006fd0 tx=0x7fdd2c008040 comp rx=0 tx=0).stop 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c1f55b0 msgr2=0x7fdd3c0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c1f55b0 0x7fdd3c0cd750 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009f40 tx=0x7fdd34004a40 comp rx=0 tx=0).stop 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 shutdown_connections 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fdd280762d0 0x7fdd28078790 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd3c1f55b0 0x7fdd3c0cd750 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c15f3b0 0x7fdd3c1f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 --2- 172.21.15.53:0/1933411132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd3c0d5d30 0x7fdd3c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 >> 172.21.15.53:0/1933411132 conn(0x7fdd3c0d5130 msgr2=0x7fdd3c0cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 shutdown_connections 2024-05-10T10:41:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.536+0000 7fdd432a0640 1 -- 172.21.15.53:0/1933411132 wait complete. 2024-05-10T10:41:01.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:41:01.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.650+0000 7f687d32b640 1 Processor -- start 2024-05-10T10:41:01.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.650+0000 7f687d32b640 1 -- start start 2024-05-10T10:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f687d32b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68780c6c80 0x7f68780c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f687d32b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68780cee40 0x7f68780cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f687d32b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687806f1c0 con 0x7f68781598c0 2024-05-10T10:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f687d32b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f687806f4a0 con 0x7f68780cee40 2024-05-10T10:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f687d32b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f687806f780 con 0x7f68780c6c80 2024-05-10T10:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f68777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68780cee40 0x7f68780cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f6876ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f68781598c0 0x7f687806e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48176/0 (socket says 172.21.15.53:48176) 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f6876ffd640 1 -- 172.21.15.53:0/4125203379 learned_addr learned my addr 172.21.15.53:0/4125203379 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f68767fc640 1 --2- 172.21.15.53:0/4125203379 >> v2:172.21.15.135:3300/0 conn(0x7f68780c6c80 0x7f68780c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f68777fe640 1 -- 172.21.15.53:0/4125203379 >> v2:172.21.15.135:3300/0 conn(0x7f68780c6c80 msgr2=0x7f68780c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f68777fe640 1 --2- 172.21.15.53:0/4125203379 >> v2:172.21.15.135:3300/0 conn(0x7f68780c6c80 0x7f68780c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f68777fe640 1 -- 172.21.15.53:0/4125203379 >> v1:172.21.15.53:6789/0 conn(0x7f68781598c0 legacy=0x7f687806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f68777fe640 1 -- 172.21.15.53:0/4125203379 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687806fa60 con 0x7f68780cee40 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f6875ffb640 1 -- 172.21.15.53:0/4125203379 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250531952 0 0) 0x7f687806f1c0 con 0x7f68781598c0 2024-05-10T10:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.651+0000 7f68767fc640 1 --2- 172.21.15.53:0/4125203379 >> v2:172.21.15.135:3300/0 conn(0x7f68780c6c80 0x7f68780c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.652+0000 7f68777fe640 1 --2- 172.21.15.53:0/4125203379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f68780cf290 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6878070400 tx=0x7f687002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab81c7387c783cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.652+0000 7f6875ffb640 1 -- 172.21.15.53:0/4125203379 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870007740 con 0x7f68780cee40 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.652+0000 7f6875ffb640 1 -- 172.21.15.53:0/4125203379 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68700078e0 con 0x7f68780cee40 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.652+0000 7f6875ffb640 1 -- 172.21.15.53:0/4125203379 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870007cb0 con 0x7f68780cee40 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 -- 172.21.15.53:0/4125203379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 msgr2=0x7f68780cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 --2- 172.21.15.53:0/4125203379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f68780cf290 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6878070400 tx=0x7f687002a680 comp rx=0 tx=0).stop 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 -- 172.21.15.53:0/4125203379 shutdown_connections 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 --2- 172.21.15.53:0/4125203379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f68780cf290 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 --2- 172.21.15.53:0/4125203379 >> v2:172.21.15.135:3300/0 conn(0x7f68780c6c80 0x7f68780c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 -- 172.21.15.53:0/4125203379 >> 172.21.15.53:0/4125203379 conn(0x7f68780b2900 msgr2=0x7f68780b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 -- 172.21.15.53:0/4125203379 shutdown_connections 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.653+0000 7f687d32b640 1 -- 172.21.15.53:0/4125203379 wait complete. 2024-05-10T10:41:01.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.654+0000 7f687d32b640 1 Processor -- start 2024-05-10T10:41:01.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.654+0000 7f687d32b640 1 -- start start 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f687d32b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68780c6c80 0x7f687815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f687d32b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f687814a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f687d32b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68781598c0 0x7f68780cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f687d32b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f687806ee10 con 0x7f68780cee40 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f687d32b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f687815c6e0 con 0x7f68781598c0 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f687d32b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f687815c240 con 0x7f68780c6c80 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f687814a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f687814a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59266/0 (socket says 172.21.15.53:59266) 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 -- 172.21.15.53:0/1288423278 learned_addr learned my addr 172.21.15.53:0/1288423278 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:01.658 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68777fe640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68781598c0 0x7f68780cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68780c6c80 0x7f687815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 -- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68780c6c80 msgr2=0x7f687815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68780c6c80 0x7f687815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 -- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68781598c0 msgr2=0x7f68780cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68781598c0 0x7f68780cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68767fc640 1 -- 172.21.15.53:0/1288423278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6878155f50 con 0x7f68780cee40 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.655+0000 7f68777fe640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68781598c0 0x7f68780cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.656+0000 7f68767fc640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f687814a7f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f686c00c820 tx=0x7f686c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.656+0000 7f685f7fe640 1 -- 172.21.15.53:0/1288423278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c015070 con 0x7f68780cee40 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.656+0000 7f685f7fe640 1 -- 172.21.15.53:0/1288423278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c007a30 con 0x7f68780cee40 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.656+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687815bf60 con 0x7f68780cee40 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.656+0000 7f685f7fe640 1 -- 172.21.15.53:0/1288423278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c007e00 con 0x7f68780cee40 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.656+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68780ae160 con 0x7f68780cee40 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.657+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68781583f0 con 0x7f68780cee40 2024-05-10T10:41:01.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.658+0000 7f685f7fe640 1 -- 172.21.15.53:0/1288423278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f686c018070 con 0x7f68780cee40 2024-05-10T10:41:01.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.658+0000 7f685f7fe640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6864076300 0x7f68640787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.658+0000 7f685f7fe640 1 -- 172.21.15.53:0/1288423278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f686c097740 con 0x7f68780cee40 2024-05-10T10:41:01.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.659+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6864076300 0x7f68640787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.659+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6864076300 0x7f68640787c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6868000f60 tx=0x7f6868007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.663+0000 7f685f7fe640 1 -- 172.21.15.53:0/1288423278 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f68781583f0 con 0x7f68780cee40 2024-05-10T10:41:01.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.853+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f687806a220 con 0x7f68780cee40 2024-05-10T10:41:01.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.854+0000 7f685f7fe640 1 -- 172.21.15.53:0/1288423278 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+17336 (secure 0 0 0) 0x7f686c067e20 con 0x7f68780cee40 2024-05-10T10:41:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:01.855 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":91,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:40:21.895844+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:38:55.295910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T10:38:56.564675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6833/285942496":"2024-05-11T10:40:21.895821+0000","172.21.15.53:6832/285942496":"2024-05-11T10:40:21.895821+0000","172.21.15.53:6833/2934725760":"2024-05-11T10:39:11.380694+0000","172.21.15.53:6832/2934725760":"2024-05-11T10:39:11.380694+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6864076300 msgr2=0x7f68640787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6864076300 0x7f68640787c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6868000f60 tx=0x7f6868007c00 comp rx=0 tx=0).stop 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 msgr2=0x7f687814a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f687814a7f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f686c00c820 tx=0x7f686c00cc20 comp rx=0 tx=0).stop 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 shutdown_connections 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6864076300 0x7f68640787c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68781598c0 0x7f68780cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68780cee40 0x7f687814a7f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 --2- 172.21.15.53:0/1288423278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68780c6c80 0x7f687815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 >> 172.21.15.53:0/1288423278 conn(0x7f68780b2900 msgr2=0x7f6878156960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 shutdown_connections 2024-05-10T10:41:01.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.856+0000 7f687d32b640 1 -- 172.21.15.53:0/1288423278 wait complete. 2024-05-10T10:41:01.868 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:41:01.868 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:01.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.971+0000 7f51f8d91640 1 Processor -- start 2024-05-10T10:41:01.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.971+0000 7f51f8d91640 1 -- start start 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f8d91640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51f41598c0 0x7f51f406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f8d91640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f406ec40 0x7f51f40c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f8d91640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f51f40c71c0 0x7f51f415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f8d91640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f51f406f1c0 con 0x7f51f41598c0 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f8d91640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f51f406f4a0 con 0x7f51f40c71c0 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f8d91640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f51f406f780 con 0x7f51f406ec40 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51f41598c0 0x7f51f406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51f41598c0 0x7f51f406e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59282/0 (socket says 172.21.15.53:59282) 2024-05-10T10:41:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 -- 172.21.15.53:0/264601573 learned_addr learned my addr 172.21.15.53:0/264601573 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f2d9d640 1 --2- 172.21.15.53:0/264601573 >> v2:172.21.15.135:3300/0 conn(0x7f51f40c71c0 0x7f51f415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 -- 172.21.15.53:0/264601573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f406ec40 msgr2=0x7f51f40c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 --2- 172.21.15.53:0/264601573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f406ec40 0x7f51f40c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 -- 172.21.15.53:0/264601573 >> v2:172.21.15.135:3300/0 conn(0x7f51f40c71c0 msgr2=0x7f51f415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 --2- 172.21.15.53:0/264601573 >> v2:172.21.15.135:3300/0 conn(0x7f51f40c71c0 0x7f51f415fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.972+0000 7f51f259c640 1 -- 172.21.15.53:0/264601573 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f406fa60 con 0x7f51f41598c0 2024-05-10T10:41:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.973+0000 7f51f259c640 1 --2- 172.21.15.53:0/264601573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f41598c0 0x7f51f406e700 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f51f415d5b0 tx=0x7f51e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dbfd244c24ac7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.973+0000 7f51f159a640 1 -- 172.21.15.53:0/264601573 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4037070 con 0x7f51f41598c0 2024-05-10T10:41:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.973+0000 7f51f159a640 1 -- 172.21.15.53:0/264601573 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e4032440 con 0x7f51f41598c0 2024-05-10T10:41:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.973+0000 7f51f159a640 1 -- 172.21.15.53:0/264601573 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4032810 con 0x7f51f41598c0 2024-05-10T10:41:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 -- 172.21.15.53:0/264601573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f41598c0 msgr2=0x7f51f406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 --2- 172.21.15.53:0/264601573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f41598c0 0x7f51f406e700 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f51f415d5b0 tx=0x7f51e402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 -- 172.21.15.53:0/264601573 shutdown_connections 2024-05-10T10:41:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 --2- 172.21.15.53:0/264601573 >> v2:172.21.15.135:3300/0 conn(0x7f51f40c71c0 0x7f51f415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 --2- 172.21.15.53:0/264601573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f406ec40 0x7f51f40c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 --2- 172.21.15.53:0/264601573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f41598c0 0x7f51f406e700 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 -- 172.21.15.53:0/264601573 >> 172.21.15.53:0/264601573 conn(0x7f51f40b2900 msgr2=0x7f51f40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 -- 172.21.15.53:0/264601573 shutdown_connections 2024-05-10T10:41:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.974+0000 7f51f8d91640 1 -- 172.21.15.53:0/264601573 wait complete. 2024-05-10T10:41:01.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.975+0000 7f51f8d91640 1 Processor -- start 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.975+0000 7f51f8d91640 1 -- start start 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f8d91640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f406ec40 0x7f51f4148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f8d91640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f40c71c0 0x7f51f40afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f8d91640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f41598c0 0x7f51f414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f8d91640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4159ca0 con 0x7f51f406ec40 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f8d91640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51f415c6e0 con 0x7f51f41598c0 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f8d91640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f51f4159e20 con 0x7f51f40c71c0 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f406ec40 0x7f51f4148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f406ec40 0x7f51f4148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59294/0 (socket says 172.21.15.53:59294) 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 -- 172.21.15.53:0/2721332707 learned_addr learned my addr 172.21.15.53:0/2721332707 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 -- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f40c71c0 msgr2=0x7f51f40afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f40c71c0 0x7f51f40afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 -- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f41598c0 msgr2=0x7f51f414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f41598c0 0x7f51f414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.976+0000 7f51f259c640 1 -- 172.21.15.53:0/2721332707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f40aede0 con 0x7f51f406ec40 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.977+0000 7f51f259c640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f406ec40 0x7f51f4148490 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f51e4000c00 tx=0x7f51e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.977+0000 7f51e2ffd640 1 -- 172.21.15.53:0/2721332707 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4041020 con 0x7f51f406ec40 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.977+0000 7f51e2ffd640 1 -- 172.21.15.53:0/2721332707 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e4030970 con 0x7f51f406ec40 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.977+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f40b0360 con 0x7f51f406ec40 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.977+0000 7f51e2ffd640 1 -- 172.21.15.53:0/2721332707 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4037040 con 0x7f51f406ec40 2024-05-10T10:41:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.977+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f40af340 con 0x7f51f406ec40 2024-05-10T10:41:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.978+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f406a220 con 0x7f51f406ec40 2024-05-10T10:41:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.978+0000 7f51e2ffd640 1 -- 172.21.15.53:0/2721332707 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f51e4044020 con 0x7f51f406ec40 2024-05-10T10:41:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.979+0000 7f51e2ffd640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51d40762d0 0x7f51d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.979+0000 7f51e2ffd640 1 -- 172.21.15.53:0/2721332707 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f51e40b82d0 con 0x7f51f406ec40 2024-05-10T10:41:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.979+0000 7f51f1d9b640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51d40762d0 0x7f51d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.980+0000 7f51f1d9b640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51d40762d0 0x7f51d4078790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f51f40cfc30 tx=0x7f51e8006bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:01.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:01.983+0000 7f51e2ffd640 1 -- 172.21.15.53:0/2721332707 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f51e404b030 con 0x7f51f406ec40 2024-05-10T10:41:02.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.212+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51f4156d20 con 0x7f51f406ec40 2024-05-10T10:41:02.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.213+0000 7f51e2ffd640 1 -- 172.21.15.53:0/2721332707 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5187 (secure 0 0 0) 0x7f51f40aede0 con 0x7f51f406ec40 2024-05-10T10:41:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:02.214 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:25.917116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7152,"qdb_cluster":[7152]},"id":6}]} 2024-05-10T10:41:02.214 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 74 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51d40762d0 msgr2=0x7f51d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51d40762d0 0x7f51d4078790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f51f40cfc30 tx=0x7f51e8006bf0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f406ec40 msgr2=0x7f51f4148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f406ec40 0x7f51f4148490 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f51e4000c00 tx=0x7f51e40046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 shutdown_connections 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f51d40762d0 0x7f51d4078790 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f41598c0 0x7f51f414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51f40c71c0 0x7f51f40afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 --2- 172.21.15.53:0/2721332707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f406ec40 0x7f51f4148490 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 >> 172.21.15.53:0/2721332707 conn(0x7f51f40b2900 msgr2=0x7f51f40c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 shutdown_connections 2024-05-10T10:41:02.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.215+0000 7f51f8d91640 1 -- 172.21.15.53:0/2721332707 wait complete. 2024-05-10T10:41:02.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:02.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.331+0000 7fd50e4e8640 1 Processor -- start 2024-05-10T10:41:02.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.331+0000 7fd50e4e8640 1 -- start start 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.331+0000 7fd50e4e8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd50815f3b0 0x7fd5080d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.331+0000 7fd50e4e8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd5080d5d30 0x7fd5080d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.331+0000 7fd50e4e8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5080d30b0 con 0x7fd50815f790 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.331+0000 7fd50e4e8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd5080d3390 con 0x7fd50815f3b0 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.331+0000 7fd50e4e8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd5080c8c80 con 0x7fd5080d5d30 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd507fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd50815f3b0 0x7fd5080d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd4fffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd50815f790 0x7fd5080c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48180/0 (socket says 172.21.15.53:48180) 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd4fffff640 1 -- 172.21.15.53:0/3118716283 learned_addr learned my addr 172.21.15.53:0/3118716283 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd50ca85640 1 --2- 172.21.15.53:0/3118716283 >> v2:172.21.15.135:3300/0 conn(0x7fd5080d5d30 0x7fd5080d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd507fff640 1 -- 172.21.15.53:0/3118716283 >> v2:172.21.15.135:3300/0 conn(0x7fd5080d5d30 msgr2=0x7fd5080d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd507fff640 1 --2- 172.21.15.53:0/3118716283 >> v2:172.21.15.135:3300/0 conn(0x7fd5080d5d30 0x7fd5080d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd507fff640 1 -- 172.21.15.53:0/3118716283 >> v1:172.21.15.53:6789/0 conn(0x7fd50815f790 legacy=0x7fd5080c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd5077fe640 1 -- 172.21.15.53:0/3118716283 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542544269 0 0) 0x7fd5080d30b0 con 0x7fd50815f790 2024-05-10T10:41:02.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.332+0000 7fd507fff640 1 -- 172.21.15.53:0/3118716283 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5080c8f60 con 0x7fd50815f3b0 2024-05-10T10:41:02.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.333+0000 7fd507fff640 1 --2- 172.21.15.53:0/3118716283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd50815f3b0 0x7fd5080d36f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd50808d220 tx=0x7fd4f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd80f25d434fbdb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.333+0000 7fd5077fe640 1 -- 172.21.15.53:0/3118716283 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8037070 con 0x7fd50815f3b0 2024-05-10T10:41:02.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.333+0000 7fd5077fe640 1 -- 172.21.15.53:0/3118716283 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f8032440 con 0x7fd50815f3b0 2024-05-10T10:41:02.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.333+0000 7fd5077fe640 1 -- 172.21.15.53:0/3118716283 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8032810 con 0x7fd50815f3b0 2024-05-10T10:41:02.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 -- 172.21.15.53:0/3118716283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd50815f3b0 msgr2=0x7fd5080d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/3118716283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd50815f3b0 0x7fd5080d36f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd50808d220 tx=0x7fd4f802a1a0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 -- 172.21.15.53:0/3118716283 shutdown_connections 2024-05-10T10:41:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/3118716283 >> v2:172.21.15.135:3300/0 conn(0x7fd5080d5d30 0x7fd5080d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/3118716283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd50815f3b0 0x7fd5080d36f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 -- 172.21.15.53:0/3118716283 >> 172.21.15.53:0/3118716283 conn(0x7fd5080d5130 msgr2=0x7fd5080d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 -- 172.21.15.53:0/3118716283 shutdown_connections 2024-05-10T10:41:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.334+0000 7fd50e4e8640 1 -- 172.21.15.53:0/3118716283 wait complete. 2024-05-10T10:41:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.335+0000 7fd50e4e8640 1 Processor -- start 2024-05-10T10:41:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.335+0000 7fd50e4e8640 1 -- start start 2024-05-10T10:41:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.335+0000 7fd50e4e8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5080d5d30 0x7fd5080ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.335+0000 7fd50e4e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd50815f3b0 0x7fd5081529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50e4e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5080cd250 0x7fd508161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50e4e8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd5080d0e60 con 0x7fd5080cd250 2024-05-10T10:41:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50e4e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5080d0ce0 con 0x7fd50815f3b0 2024-05-10T10:41:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50e4e8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd5080d0fe0 con 0x7fd5080d5d30 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5080cd250 0x7fd508161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5080cd250 0x7fd508161b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59306/0 (socket says 172.21.15.53:59306) 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 -- 172.21.15.53:0/1254764957 learned_addr learned my addr 172.21.15.53:0/1254764957 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd507fff640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5080d5d30 0x7fd5080ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd4fffff640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd50815f3b0 0x7fd5081529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 -- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5080d5d30 msgr2=0x7fd5080ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5080d5d30 0x7fd5080ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 -- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd50815f3b0 msgr2=0x7fd5081529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd50815f3b0 0x7fd5081529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 -- 172.21.15.53:0/1254764957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5080cd730 con 0x7fd5080cd250 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd507fff640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5080d5d30 0x7fd5080ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd4fffff640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd50815f3b0 0x7fd5081529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50ca85640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5080cd250 0x7fd508161b90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd500007e40 tx=0x7fd50000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd5057fa640 1 -- 172.21.15.53:0/1254764957 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500014620 con 0x7fd5080cd250 2024-05-10T10:41:02.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd5057fa640 1 -- 172.21.15.53:0/1254764957 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd500019070 con 0x7fd5080cd250 2024-05-10T10:41:02.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5080cda10 con 0x7fd5080cd250 2024-05-10T10:41:02.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd5057fa640 1 -- 172.21.15.53:0/1254764957 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500014a20 con 0x7fd5080cd250 2024-05-10T10:41:02.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.336+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5080cdfd0 con 0x7fd5080cd250 2024-05-10T10:41:02.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.337+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5080ce1d0 con 0x7fd5080cd250 2024-05-10T10:41:02.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.338+0000 7fd5057fa640 1 -- 172.21.15.53:0/1254764957 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd500014bc0 con 0x7fd5080cd250 2024-05-10T10:41:02.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.338+0000 7fd5057fa640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd4e8076300 0x7fd4e80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.338+0000 7fd5057fa640 1 -- 172.21.15.53:0/1254764957 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fd5000981f0 con 0x7fd5080cd250 2024-05-10T10:41:02.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.339+0000 7fd507fff640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd4e8076300 0x7fd4e80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.339+0000 7fd507fff640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd4e8076300 0x7fd4e80787c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8034000 tx=0x7fd4f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.343+0000 7fd5057fa640 1 -- 172.21.15.53:0/1254764957 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd5080ce1d0 con 0x7fd5080cd250 2024-05-10T10:41:02.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.575+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd50806a220 con 0x7fd5080cd250 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.575+0000 7fd5057fa640 1 -- 172.21.15.53:0/1254764957 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5187 (secure 0 0 0) 0x7fd500061800 con 0x7fd5080cd250 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:02.576 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:38:57.607866+0000","modified":"2024-05-10T10:40:25.917116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7152":{"gid":7152,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1098976714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1098976714},{"type":"v1","addr":"172.21.15.53:6833","nonce":1098976714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7152,"qdb_cluster":[7152]},"id":6}]} 2024-05-10T10:41:02.577 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 74 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd4e8076300 msgr2=0x7fd4e80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd4e8076300 0x7fd4e80787c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8034000 tx=0x7fd4f802e000 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5080cd250 msgr2=0x7fd508161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5080cd250 0x7fd508161b90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd500007e40 tx=0x7fd50000e4b0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 shutdown_connections 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd4e8076300 0x7fd4e80787c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5080cd250 0x7fd508161b90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd50815f3b0 0x7fd5081529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 --2- 172.21.15.53:0/1254764957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd5080d5d30 0x7fd5080ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 >> 172.21.15.53:0/1254764957 conn(0x7fd5080d5130 msgr2=0x7fd5080cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 shutdown_connections 2024-05-10T10:41:02.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.578+0000 7fd50e4e8640 1 -- 172.21.15.53:0/1254764957 wait complete. 2024-05-10T10:41:02.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.693+0000 7f13a701d640 1 Processor -- start 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.693+0000 7f13a701d640 1 -- start start 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a701d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a006d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a701d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13a015f8c0 0x7f13a006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a701d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f13a006e450 0x7f13a00afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a701d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f13a00b23f0 con 0x7f13a006e450 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a701d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f13a00b4150 con 0x7f13a015f8c0 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a701d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f13a006e8c0 con 0x7f13a015f4e0 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f13a006e450 0x7f13a00afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f13a006e450 0x7f13a00afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39392/0 (socket says 172.21.15.53:39392) 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 -- 172.21.15.53:0/2606395824 learned_addr learned my addr 172.21.15.53:0/2606395824 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a4db9640 1 --2- 172.21.15.53:0/2606395824 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a006d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f139ffff640 1 --2- 172.21.15.53:0/2606395824 >> v2:172.21.15.135:3300/0 conn(0x7f13a015f8c0 0x7f13a006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 -- 172.21.15.53:0/2606395824 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 msgr2=0x7f13a006d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 --2- 172.21.15.53:0/2606395824 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a006d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 -- 172.21.15.53:0/2606395824 >> v2:172.21.15.135:3300/0 conn(0x7f13a015f8c0 msgr2=0x7f13a006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 --2- 172.21.15.53:0/2606395824 >> v2:172.21.15.135:3300/0 conn(0x7f13a015f8c0 0x7f13a006df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.694+0000 7f13a55ba640 1 -- 172.21.15.53:0/2606395824 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a00b3540 con 0x7f13a006e450 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.695+0000 7f13a55ba640 1 --2- 172.21.15.53:0/2606395824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a006e450 0x7f13a00afa80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f13a00730a0 tx=0x7f139002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77df2161d4c997a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.695+0000 7f139f7fe640 1 -- 172.21.15.53:0/2606395824 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390036070 con 0x7f13a006e450 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.695+0000 7f139f7fe640 1 -- 172.21.15.53:0/2606395824 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1390031600 con 0x7f13a006e450 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.695+0000 7f139f7fe640 1 -- 172.21.15.53:0/2606395824 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13900319d0 con 0x7f13a006e450 2024-05-10T10:41:02.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 -- 172.21.15.53:0/2606395824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a006e450 msgr2=0x7f13a00afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 --2- 172.21.15.53:0/2606395824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a006e450 0x7f13a00afa80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f13a00730a0 tx=0x7f139002a610 comp rx=0 tx=0).stop 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 -- 172.21.15.53:0/2606395824 shutdown_connections 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 --2- 172.21.15.53:0/2606395824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a006e450 0x7f13a00afa80 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 --2- 172.21.15.53:0/2606395824 >> v2:172.21.15.135:3300/0 conn(0x7f13a015f8c0 0x7f13a006df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 --2- 172.21.15.53:0/2606395824 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a006d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 -- 172.21.15.53:0/2606395824 >> 172.21.15.53:0/2606395824 conn(0x7f13a00d2df0 msgr2=0x7f13a00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 -- 172.21.15.53:0/2606395824 shutdown_connections 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.696+0000 7f13a701d640 1 -- 172.21.15.53:0/2606395824 wait complete. 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.697+0000 7f13a701d640 1 Processor -- start 2024-05-10T10:41:02.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.697+0000 7f13a701d640 1 -- start start 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a701d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a006e450 0x7f13a0150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a701d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a701d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a015f8c0 0x7f13a0160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a701d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00b3de0 con 0x7f13a015f8c0 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a701d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00b3b00 con 0x7f13a006e450 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a701d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f13a00b45e0 con 0x7f13a015f4e0 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a015f8c0 0x7f13a0160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a015f8c0 0x7f13a0160520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39396/0 (socket says 172.21.15.53:39396) 2024-05-10T10:41:02.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 -- 172.21.15.53:0/583308029 learned_addr learned my addr 172.21.15.53:0/583308029 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a4db9640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a006e450 0x7f13a0150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f139ffff640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 -- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 msgr2=0x7f13a015ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a015ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 -- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a006e450 msgr2=0x7f13a0150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a006e450 0x7f13a0150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 -- 172.21.15.53:0/583308029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a0160a60 con 0x7f13a015f8c0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.698+0000 7f13a55ba640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a015f8c0 0x7f13a0160520 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1390034040 tx=0x7f1390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.699+0000 7f139d7fa640 1 -- 172.21.15.53:0/583308029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390036050 con 0x7f13a015f8c0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.699+0000 7f139d7fa640 1 -- 172.21.15.53:0/583308029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1390042070 con 0x7f13a015f8c0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.699+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a0160d10 con 0x7f13a015f8c0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.699+0000 7f139d7fa640 1 -- 172.21.15.53:0/583308029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390009510 con 0x7f13a015f8c0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.699+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01612d0 con 0x7f13a015f8c0 2024-05-10T10:41:02.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.700+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a006a220 con 0x7f13a015f8c0 2024-05-10T10:41:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.700+0000 7f139d7fa640 1 -- 172.21.15.53:0/583308029 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1390007bd0 con 0x7f13a015f8c0 2024-05-10T10:41:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.701+0000 7f139d7fa640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.701+0000 7f139d7fa640 1 -- 172.21.15.53:0/583308029 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7f13900b85e0 con 0x7f13a015f8c0 2024-05-10T10:41:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.701+0000 7f13a4db9640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:02.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.702+0000 7f13a4db9640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1398003610 tx=0x7f1398008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:02.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.705+0000 7f139d7fa640 1 -- 172.21.15.53:0/583308029 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1390081640 con 0x7f13a015f8c0 2024-05-10T10:41:02.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.899+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13a0070f70 con 0x7f13a015f8c0 2024-05-10T10:41:02.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.900+0000 7f139d7fa640 1 -- 172.21.15.53:0/583308029 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+17336 (secure 0 0 0) 0x7f1390085940 con 0x7f13a015f8c0 2024-05-10T10:41:02.901 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:02.901 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":91,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:40:21.895844+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:38:55.295910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T10:38:56.564675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6833/285942496":"2024-05-11T10:40:21.895821+0000","172.21.15.53:6832/285942496":"2024-05-11T10:40:21.895821+0000","172.21.15.53:6833/2934725760":"2024-05-11T10:39:11.380694+0000","172.21.15.53:6832/2934725760":"2024-05-11T10:39:11.380694+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 msgr2=0x7f1380078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1398003610 tx=0x7f1398008040 comp rx=0 tx=0).stop 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a015f8c0 msgr2=0x7f13a0160520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a015f8c0 0x7f13a0160520 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1390034040 tx=0x7f1390004a40 comp rx=0 tx=0).stop 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 shutdown_connections 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f13800762d0 0x7f1380078790 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a015f8c0 0x7f13a0160520 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13a015f4e0 0x7f13a015ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 --2- 172.21.15.53:0/583308029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a006e450 0x7f13a0150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 >> 172.21.15.53:0/583308029 conn(0x7f13a00d2df0 msgr2=0x7f13a00b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 shutdown_connections 2024-05-10T10:41:02.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:02.902+0000 7f13a701d640 1 -- 172.21.15.53:0/583308029 wait complete. 2024-05-10T10:41:02.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:41:03.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.019+0000 7fb9e43da640 1 Processor -- start 2024-05-10T10:41:03.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.019+0000 7fb9e43da640 1 -- start start 2024-05-10T10:41:03.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e43da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9dc162940 0x7fb9dc0b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e43da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb9dc0d3d80 0x7fb9dc0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e43da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9dc0d2d20 con 0x7fb9dc0d59d0 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e43da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb9dc162d20 con 0x7fb9dc162940 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e43da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb9dc0d3000 con 0x7fb9dc0d3d80 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e2176640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9dc162940 0x7fb9dc0b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e1975640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb9dc0d59d0 0x7fb9dc0d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55690/0 (socket says 172.21.15.53:55690) 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e1975640 1 -- 172.21.15.53:0/572643 learned_addr learned my addr 172.21.15.53:0/572643 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e2977640 1 --2- 172.21.15.53:0/572643 >> v2:172.21.15.135:3300/0 conn(0x7fb9dc0d3d80 0x7fb9dc0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e2176640 1 -- 172.21.15.53:0/572643 >> v2:172.21.15.135:3300/0 conn(0x7fb9dc0d3d80 msgr2=0x7fb9dc0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e2176640 1 --2- 172.21.15.53:0/572643 >> v2:172.21.15.135:3300/0 conn(0x7fb9dc0d3d80 0x7fb9dc0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e2176640 1 -- 172.21.15.53:0/572643 >> v1:172.21.15.53:6789/0 conn(0x7fb9dc0d59d0 legacy=0x7fb9dc0d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e2176640 1 -- 172.21.15.53:0/572643 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc0d32e0 con 0x7fb9dc162940 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e1174640 1 -- 172.21.15.53:0/572643 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009863959 0 0) 0x7fb9dc0d2d20 con 0x7fb9dc0d59d0 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.020+0000 7fb9e2977640 1 --2- 172.21.15.53:0/572643 >> v2:172.21.15.135:3300/0 conn(0x7fb9dc0d3d80 0x7fb9dc0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.021+0000 7fb9e2176640 1 --2- 172.21.15.53:0/572643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b5e20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0d5e50 tx=0x7fb9cc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63f00f6ebba051 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.021+0000 7fb9e1174640 1 -- 172.21.15.53:0/572643 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc037070 con 0x7fb9dc162940 2024-05-10T10:41:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.021+0000 7fb9e1174640 1 -- 172.21.15.53:0/572643 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9cc032440 con 0x7fb9dc162940 2024-05-10T10:41:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.021+0000 7fb9e1174640 1 -- 172.21.15.53:0/572643 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc02ae60 con 0x7fb9dc162940 2024-05-10T10:41:03.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.021+0000 7fb9e43da640 1 -- 172.21.15.53:0/572643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 msgr2=0x7fb9dc0b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.021+0000 7fb9e43da640 1 --2- 172.21.15.53:0/572643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b5e20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0d5e50 tx=0x7fb9cc02a110 comp rx=0 tx=0).stop 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.022+0000 7fb9e43da640 1 -- 172.21.15.53:0/572643 shutdown_connections 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.022+0000 7fb9e43da640 1 --2- 172.21.15.53:0/572643 >> v2:172.21.15.135:3300/0 conn(0x7fb9dc0d3d80 0x7fb9dc0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.022+0000 7fb9e43da640 1 --2- 172.21.15.53:0/572643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b5e20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.022+0000 7fb9e43da640 1 -- 172.21.15.53:0/572643 >> 172.21.15.53:0/572643 conn(0x7fb9dc09e380 msgr2=0x7fb9dc0d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.022+0000 7fb9e43da640 1 -- 172.21.15.53:0/572643 shutdown_connections 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.022+0000 7fb9e43da640 1 -- 172.21.15.53:0/572643 wait complete. 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.023+0000 7fb9e43da640 1 Processor -- start 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.023+0000 7fb9e43da640 1 -- start start 2024-05-10T10:41:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9dc0d3d80 0x7fb9dc0b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9dc0d59d0 0x7fb9dc153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0d2390 con 0x7fb9dc162940 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0cfc50 con 0x7fb9dc0d59d0 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb9dc0d2510 con 0x7fb9dc0d3d80 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39426/0 (socket says 172.21.15.53:39426) 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 -- 172.21.15.53:0/2835733101 learned_addr learned my addr 172.21.15.53:0/2835733101 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e1975640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9dc0d59d0 0x7fb9dc153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2176640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9dc0d3d80 0x7fb9dc0b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 -- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9dc0d3d80 msgr2=0x7fb9dc0b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9dc0d3d80 0x7fb9dc0b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 -- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9dc0d59d0 msgr2=0x7fb9dc153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9dc0d59d0 0x7fb9dc153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 -- 172.21.15.53:0/2835733101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1f6580 con 0x7fb9dc162940 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2176640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9dc0d3d80 0x7fb9dc0b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e1975640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9dc0d59d0 0x7fb9dc153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e2977640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b8fc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400cd80 tx=0x7fb9d400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9daffd640 1 -- 172.21.15.53:0/2835733101 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400f680 con 0x7fb9dc162940 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9daffd640 1 -- 172.21.15.53:0/2835733101 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d400f840 con 0x7fb9dc162940 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc15e920 con 0x7fb9dc162940 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9daffd640 1 -- 172.21.15.53:0/2835733101 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400fc30 con 0x7fb9dc162940 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.024+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc15eee0 con 0x7fb9dc162940 2024-05-10T10:41:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.025+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc06a220 con 0x7fb9dc162940 2024-05-10T10:41:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.026+0000 7fb9daffd640 1 -- 172.21.15.53:0/2835733101 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb9d40213f0 con 0x7fb9dc162940 2024-05-10T10:41:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.026+0000 7fb9daffd640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb9c80762d0 0x7fb9c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.027+0000 7fb9daffd640 1 -- 172.21.15.53:0/2835733101 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6389+0+0 (secure 0 0 0) 0x7fb9d4098580 con 0x7fb9dc162940 2024-05-10T10:41:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.027+0000 7fb9e2176640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb9c80762d0 0x7fb9c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.027+0000 7fb9e2176640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb9c80762d0 0x7fb9c8078790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc034000 tx=0x7fb9cc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:03.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.031+0000 7fb9daffd640 1 -- 172.21.15.53:0/2835733101 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb9d4014030 con 0x7fb9dc162940 2024-05-10T10:41:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.266+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb9dc0d3b30 con 0x7fb9dc162940 2024-05-10T10:41:03.267 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:41:03.267+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:41:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.268+0000 7fb9daffd640 1 -- 172.21.15.53:0/2835733101 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7fb9d4061cc0 con 0x7fb9dc162940 2024-05-10T10:41:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb9c80762d0 msgr2=0x7fb9c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb9c80762d0 0x7fb9c8078790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc034000 tx=0x7fb9cc02e000 comp rx=0 tx=0).stop 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 msgr2=0x7fb9dc0b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b8fc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400cd80 tx=0x7fb9d400a650 comp rx=0 tx=0).stop 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 shutdown_connections 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb9c80762d0 0x7fb9c8078790 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9dc162940 0x7fb9dc0b8fc0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9dc0d59d0 0x7fb9dc153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 --2- 172.21.15.53:0/2835733101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9dc0d3d80 0x7fb9dc0b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 >> 172.21.15.53:0/2835733101 conn(0x7fb9dc09e380 msgr2=0x7fb9dc06cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 shutdown_connections 2024-05-10T10:41:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.271+0000 7fb9e43da640 1 -- 172.21.15.53:0/2835733101 wait complete. 2024-05-10T10:41:03.274 INFO:tasks.ceph.mds.a.smithi053.stderr: -1845> 2024-05-10T10:40:28.857+0000 7fc6708d3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x564d95482100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-10T10:41:03.274 INFO:tasks.ceph.mds.a.smithi053.stderr: -1836> 2024-05-10T10:40:28.907+0000 7fc6748db640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:41:03.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:41:03.302 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:41:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.391+0000 7f8d81200640 1 Processor -- start 2024-05-10T10:41:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.391+0000 7f8d81200640 1 -- start start 2024-05-10T10:41:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d81200640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c073540 0x7f8d7c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d81200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8d7c0cfd00 0x7f8d7c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d81200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d7c0cd170 con 0x7f8d7c15be10 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d81200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8d7c0cd450 con 0x7f8d7c0cfd00 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d81200640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c0cd730 con 0x7f8d7c073540 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d7b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8d7c0cfd00 0x7f8d7c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d7ad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8d7c15be10 0x7f8d7c072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55696/0 (socket says 172.21.15.53:55696) 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.392+0000 7f8d7ad9d640 1 -- 172.21.15.53:0/2132781163 learned_addr learned my addr 172.21.15.53:0/2132781163 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d7a59c640 1 --2- 172.21.15.53:0/2132781163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c073540 0x7f8d7c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d7b59e640 1 -- 172.21.15.53:0/2132781163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c073540 msgr2=0x7f8d7c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d79d9b640 1 -- 172.21.15.53:0/2132781163 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3810419031 0 0) 0x7f8d7c0cd170 con 0x7f8d7c15be10 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d7b59e640 1 --2- 172.21.15.53:0/2132781163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c073540 0x7f8d7c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d7b59e640 1 -- 172.21.15.53:0/2132781163 >> v1:172.21.15.53:6789/0 conn(0x7f8d7c15be10 legacy=0x7f8d7c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d7b59e640 1 -- 172.21.15.53:0/2132781163 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c0cda10 con 0x7f8d7c0cfd00 2024-05-10T10:41:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d7a59c640 1 --2- 172.21.15.53:0/2132781163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c073540 0x7f8d7c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.393+0000 7f8d7b59e640 1 --2- 172.21.15.53:0/2132781163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0cfd00 0x7f8d7c06ebb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8d740097a0 tx=0x7f8d7402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a00fd2fc5d97eac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d79d9b640 1 -- 172.21.15.53:0/2132781163 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74037070 con 0x7f8d7c0cfd00 2024-05-10T10:41:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d79d9b640 1 -- 172.21.15.53:0/2132781163 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d74032440 con 0x7f8d7c0cfd00 2024-05-10T10:41:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d79d9b640 1 -- 172.21.15.53:0/2132781163 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74032810 con 0x7f8d7c0cfd00 2024-05-10T10:41:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 -- 172.21.15.53:0/2132781163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0cfd00 msgr2=0x7f8d7c06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 --2- 172.21.15.53:0/2132781163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0cfd00 0x7f8d7c06ebb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8d740097a0 tx=0x7f8d7402a180 comp rx=0 tx=0).stop 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 -- 172.21.15.53:0/2132781163 shutdown_connections 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 --2- 172.21.15.53:0/2132781163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0cfd00 0x7f8d7c06ebb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 --2- 172.21.15.53:0/2132781163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c073540 0x7f8d7c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 -- 172.21.15.53:0/2132781163 >> 172.21.15.53:0/2132781163 conn(0x7f8d7c0d1c90 msgr2=0x7f8d7c0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 -- 172.21.15.53:0/2132781163 shutdown_connections 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.394+0000 7f8d81200640 1 -- 172.21.15.53:0/2132781163 wait complete. 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 Processor -- start 2024-05-10T10:41:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 -- start start 2024-05-10T10:41:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c073540 0x7f8d7c1e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c0cfd00 0x7f8d7c0b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7c15be10 0x7f8d7c14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c0ce210 con 0x7f8d7c073540 2024-05-10T10:41:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c074000 con 0x7f8d7c15be10 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.396+0000 7f8d81200640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c15c1f0 con 0x7f8d7c0cfd00 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c073540 0x7f8d7c1e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c073540 0x7f8d7c1e91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39470/0 (socket says 172.21.15.53:39470) 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 -- 172.21.15.53:0/1182995970 learned_addr learned my addr 172.21.15.53:0/1182995970 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7a59c640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c0cfd00 0x7f8d7c0b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7b59e640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7c15be10 0x7f8d7c14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 -- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c0cfd00 msgr2=0x7f8d7c0b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c0cfd00 0x7f8d7c0b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 -- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7c15be10 msgr2=0x7f8d7c14a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7c15be10 0x7f8d7c14a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 -- 172.21.15.53:0/1182995970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c0ca050 con 0x7f8d7c073540 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d7ad9d640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c073540 0x7f8d7c1e91d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c7c0 tx=0x7f8d6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d737fe640 1 -- 172.21.15.53:0/1182995970 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c018070 con 0x7f8d7c073540 2024-05-10T10:41:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d737fe640 1 -- 172.21.15.53:0/1182995970 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00a640 con 0x7f8d7c073540 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d737fe640 1 -- 172.21.15.53:0/1182995970 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00aa30 con 0x7f8d7c073540 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c1e86f0 con 0x7f8d7c073540 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.397+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c1e8cb0 con 0x7f8d7c073540 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.398+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7c0693f0 con 0x7f8d7c073540 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.399+0000 7f8d737fe640 1 -- 172.21.15.53:0/1182995970 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8d6c01b070 con 0x7f8d7c073540 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.399+0000 7f8d737fe640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8d5c0762d0 0x7f8d5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.400+0000 7f8d737fe640 1 -- 172.21.15.53:0/1182995970 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6475+0+0 (secure 0 0 0) 0x7f8d6c096fe0 con 0x7f8d7c073540 2024-05-10T10:41:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.400+0000 7f8d7a59c640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8d5c0762d0 0x7f8d5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:03.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.400+0000 7f8d7a59c640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8d5c0762d0 0x7f8d5c078790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c14ddb0 tx=0x7f8d64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:03.404 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.404+0000 7f8d737fe640 1 -- 172.21.15.53:0/1182995970 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8d6c0605a0 con 0x7f8d7c073540 2024-05-10T10:41:03.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:03.639+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8d7c0cc020 con 0x7f8d7c073540 2024-05-10T10:41:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.269+0000 7f8d737fe640 1 -- 172.21.15.53:0/1182995970 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7f8d6c05fea0 con 0x7f8d7c073540 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8d5c0762d0 msgr2=0x7f8d5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8d5c0762d0 0x7f8d5c078790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c14ddb0 tx=0x7f8d64008040 comp rx=0 tx=0).stop 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c073540 msgr2=0x7f8d7c1e91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c073540 0x7f8d7c1e91d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c7c0 tx=0x7f8d6c00cc80 comp rx=0 tx=0).stop 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 shutdown_connections 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8d5c0762d0 0x7f8d5c078790 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7c15be10 0x7f8d7c14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8d7c0cfd00 0x7f8d7c0b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 --2- 172.21.15.53:0/1182995970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c073540 0x7f8d7c1e91d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 >> 172.21.15.53:0/1182995970 conn(0x7f8d7c0d1c90 msgr2=0x7f8d7c1e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:04.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 shutdown_connections 2024-05-10T10:41:04.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.272+0000 7f8d81200640 1 -- 172.21.15.53:0/1182995970 wait complete. 2024-05-10T10:41:04.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.444+0000 7fb73dd9e640 1 Processor -- start 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.444+0000 7fb73dd9e640 1 -- start start 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.445+0000 7fb73dd9e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7380cdab0 0x7fb7380cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.445+0000 7fb73dd9e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb73806c9d0 0x7fb73815e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.445+0000 7fb73dd9e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb738163d00 con 0x7fb7381637d0 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.445+0000 7fb73dd9e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb7380d1a70 con 0x7fb73806c9d0 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.445+0000 7fb73dd9e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb7380d15e0 con 0x7fb7380cdab0 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.445+0000 7fb737fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb7381637d0 0x7fb73806c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55702/0 (socket says 172.21.15.53:55702) 2024-05-10T10:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.445+0000 7fb737fff640 1 -- 172.21.15.53:0/3693090133 learned_addr learned my addr 172.21.15.53:0/3693090133 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb73cd9c640 1 --2- 172.21.15.53:0/3693090133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7380cdab0 0x7fb7380cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb73d59d640 1 --2- 172.21.15.53:0/3693090133 >> v2:172.21.15.135:3300/0 conn(0x7fb73806c9d0 0x7fb73815e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb7377fe640 1 -- 172.21.15.53:0/3693090133 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095937037 0 0) 0x7fb738163d00 con 0x7fb7381637d0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb7377fe640 1 -- 172.21.15.53:0/3693090133 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb720003520 con 0x7fb7381637d0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb73cd9c640 1 -- 172.21.15.53:0/3693090133 >> v2:172.21.15.135:3300/0 conn(0x7fb73806c9d0 msgr2=0x7fb73815e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb73cd9c640 1 --2- 172.21.15.53:0/3693090133 >> v2:172.21.15.135:3300/0 conn(0x7fb73806c9d0 0x7fb73815e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb73cd9c640 1 -- 172.21.15.53:0/3693090133 >> v1:172.21.15.53:6789/0 conn(0x7fb7381637d0 legacy=0x7fb73806c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb7377fe640 1 -- 172.21.15.53:0/3693090133 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4126469311 0 0) 0x7fb720003520 con 0x7fb7381637d0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb73cd9c640 1 -- 172.21.15.53:0/3693090133 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb738070b50 con 0x7fb7380cdab0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.446+0000 7fb73d59d640 1 --2- 172.21.15.53:0/3693090133 >> v2:172.21.15.135:3300/0 conn(0x7fb73806c9d0 0x7fb73815e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.447+0000 7fb73cd9c640 1 --2- 172.21.15.53:0/3693090133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7380cdab0 0x7fb7380cf830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb7280097a0 tx=0x7fb72802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ab37edc91fc070 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.447+0000 7fb7377fe640 1 -- 172.21.15.53:0/3693090133 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728037070 con 0x7fb7380cdab0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.447+0000 7fb7377fe640 1 -- 172.21.15.53:0/3693090133 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72802aba0 con 0x7fb7380cdab0 2024-05-10T10:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.447+0000 7fb7377fe640 1 -- 172.21.15.53:0/3693090133 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c001670 con 0x7fb7380cdab0 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 -- 172.21.15.53:0/3693090133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7380cdab0 msgr2=0x7fb7380cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/3693090133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7380cdab0 0x7fb7380cf830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb7280097a0 tx=0x7fb72802a110 comp rx=0 tx=0).stop 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 -- 172.21.15.53:0/3693090133 shutdown_connections 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/3693090133 >> v2:172.21.15.135:3300/0 conn(0x7fb73806c9d0 0x7fb73815e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/3693090133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7380cdab0 0x7fb7380cf830 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 -- 172.21.15.53:0/3693090133 >> 172.21.15.53:0/3693090133 conn(0x7fb7380d1c60 msgr2=0x7fb7380d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 -- 172.21.15.53:0/3693090133 shutdown_connections 2024-05-10T10:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.448+0000 7fb73dd9e640 1 -- 172.21.15.53:0/3693090133 wait complete. 2024-05-10T10:41:04.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.449+0000 7fb73dd9e640 1 Processor -- start 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.449+0000 7fb73dd9e640 1 -- start start 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73dd9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73806c9d0 0x7fb738150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73dd9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7380cdab0 0x7fb7380b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73dd9e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7381637d0 0x7fb7380af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73dd9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb738070450 con 0x7fb73806c9d0 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73dd9e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb738070170 con 0x7fb7380cdab0 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73dd9e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb7380d1200 con 0x7fb7381637d0 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73cd9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73806c9d0 0x7fb738150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73cd9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73806c9d0 0x7fb738150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39480/0 (socket says 172.21.15.53:39480) 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73cd9c640 1 -- 172.21.15.53:0/1162822996 learned_addr learned my addr 172.21.15.53:0/1162822996 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb737fff640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7380cdab0 0x7fb7380b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:04.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73d59d640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7381637d0 0x7fb7380af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73cd9c640 1 -- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7381637d0 msgr2=0x7fb7380af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.450+0000 7fb73cd9c640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7381637d0 0x7fb7380af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb73cd9c640 1 -- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7380cdab0 msgr2=0x7fb7380b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb73cd9c640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7380cdab0 0x7fb7380b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb73cd9c640 1 -- 172.21.15.53:0/1162822996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7380b5590 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb73cd9c640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73806c9d0 0x7fb738150300 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098c0 tx=0x7fb7280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb7357fa640 1 -- 172.21.15.53:0/1162822996 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728037050 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb7357fa640 1 -- 172.21.15.53:0/1162822996 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb728043070 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7380b5870 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb7357fa640 1 -- 172.21.15.53:0/1162822996 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728006e10 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.451+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7380cb360 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.452+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7380cae70 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.453+0000 7fb7357fa640 1 -- 172.21.15.53:0/1162822996 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb72803ca50 con 0x7fb73806c9d0 2024-05-10T10:41:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.453+0000 7fb7357fa640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7240762d0 0x7fb724078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.453+0000 7fb7357fa640 1 -- 172.21.15.53:0/1162822996 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6475+0+0 (secure 0 0 0) 0x7fb7280b81c0 con 0x7fb73806c9d0 2024-05-10T10:41:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.453+0000 7fb737fff640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7240762d0 0x7fb724078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.454+0000 7fb737fff640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7240762d0 0x7fb724078790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb738161650 tx=0x7fb72c001de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.458+0000 7fb7357fa640 1 -- 172.21.15.53:0/1162822996 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb728081820 con 0x7fb73806c9d0 2024-05-10T10:41:04.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:04.647+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb7380cb5a0 con 0x7fb73806c9d0 2024-05-10T10:41:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.378+0000 7fb7357fa640 1 -- 172.21.15.53:0/1162822996 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7fb728081120 con 0x7fb73806c9d0 2024-05-10T10:41:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:41:05.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.382+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7240762d0 msgr2=0x7fb724078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.382+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7240762d0 0x7fb724078790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb738161650 tx=0x7fb72c001de0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.382+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73806c9d0 msgr2=0x7fb738150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.382+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73806c9d0 0x7fb738150300 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098c0 tx=0x7fb7280046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 shutdown_connections 2024-05-10T10:41:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7240762d0 0x7fb724078790 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7381637d0 0x7fb7380af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7380cdab0 0x7fb7380b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 --2- 172.21.15.53:0/1162822996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb73806c9d0 0x7fb738150300 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 >> 172.21.15.53:0/1162822996 conn(0x7fb7380d1c60 msgr2=0x7fb7380b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 shutdown_connections 2024-05-10T10:41:05.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.383+0000 7fb73dd9e640 1 -- 172.21.15.53:0/1162822996 wait complete. 2024-05-10T10:41:05.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:41:05.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.513+0000 7f3d3a437640 1 Processor -- start 2024-05-10T10:41:05.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.513+0000 7f3d3a437640 1 -- start start 2024-05-10T10:41:05.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d3a437640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d3415f370 0x7f3d340d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d3a437640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d341639a0 0x7f3d34070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d3a437640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d340d2510 con 0x7f3d34071520 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d3a437640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3d34070220 con 0x7f3d3415f370 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d3a437640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3d34070500 con 0x7f3d341639a0 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d33fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d3415f370 0x7f3d340d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d33fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d3415f370 0x7f3d340d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39496/0 (socket says 172.21.15.53:39496) 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.514+0000 7f3d33fff640 1 -- 172.21.15.53:0/1080829707 learned_addr learned my addr 172.21.15.53:0/1080829707 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.515+0000 7f3d337fe640 1 --2- 172.21.15.53:0/1080829707 >> v2:172.21.15.135:3300/0 conn(0x7f3d341639a0 0x7f3d34070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.515+0000 7f3d33fff640 1 -- 172.21.15.53:0/1080829707 >> v2:172.21.15.135:3300/0 conn(0x7f3d341639a0 msgr2=0x7f3d34070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.515+0000 7f3d33fff640 1 --2- 172.21.15.53:0/1080829707 >> v2:172.21.15.135:3300/0 conn(0x7f3d341639a0 0x7f3d34070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.515+0000 7f3d33fff640 1 -- 172.21.15.53:0/1080829707 >> v1:172.21.15.135:6789/0 conn(0x7f3d34071520 legacy=0x7f3d34162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.515+0000 7f3d33fff640 1 -- 172.21.15.53:0/1080829707 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d340707e0 con 0x7f3d3415f370 2024-05-10T10:41:05.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.515+0000 7f3d33fff640 1 --2- 172.21.15.53:0/1080829707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340d1cc0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3d340c6910 tx=0x7f3d2402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd411ecba7986c5a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d32ffd640 1 -- 172.21.15.53:0/1080829707 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24037070 con 0x7f3d3415f370 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d32ffd640 1 -- 172.21.15.53:0/1080829707 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d24032440 con 0x7f3d3415f370 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d32ffd640 1 -- 172.21.15.53:0/1080829707 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24032810 con 0x7f3d3415f370 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 -- 172.21.15.53:0/1080829707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 msgr2=0x7f3d340d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 --2- 172.21.15.53:0/1080829707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340d1cc0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3d340c6910 tx=0x7f3d2402a2c0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 -- 172.21.15.53:0/1080829707 shutdown_connections 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 --2- 172.21.15.53:0/1080829707 >> v2:172.21.15.135:3300/0 conn(0x7f3d341639a0 0x7f3d34070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 --2- 172.21.15.53:0/1080829707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340d1cc0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 -- 172.21.15.53:0/1080829707 >> 172.21.15.53:0/1080829707 conn(0x7f3d340d2e10 msgr2=0x7f3d340d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 -- 172.21.15.53:0/1080829707 shutdown_connections 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.516+0000 7f3d3a437640 1 -- 172.21.15.53:0/1080829707 wait complete. 2024-05-10T10:41:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.517+0000 7f3d3a437640 1 Processor -- start 2024-05-10T10:41:05.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.518+0000 7f3d3a437640 1 -- start start 2024-05-10T10:41:05.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.518+0000 7f3d3a437640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.518+0000 7f3d3a437640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d341639a0 0x7f3d3409cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.518+0000 7f3d3a437640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d34161ef0 0x7f3d3414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.518+0000 7f3d3a437640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34163510 con 0x7f3d3415f370 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.518+0000 7f3d3a437640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3d340d2200 con 0x7f3d341639a0 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.518+0000 7f3d3a437640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3d34163690 con 0x7f3d34161ef0 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39506/0 (socket says 172.21.15.53:39506) 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 -- 172.21.15.53:0/371147833 learned_addr learned my addr 172.21.15.53:0/371147833 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d337fe640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d341639a0 0x7f3d3409cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d389d4640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d34161ef0 0x7f3d3414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 -- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d34161ef0 msgr2=0x7f3d3414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d34161ef0 0x7f3d3414ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 -- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d341639a0 msgr2=0x7f3d3409cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d341639a0 0x7f3d3409cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 -- 172.21.15.53:0/371147833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3409d1d0 con 0x7f3d3415f370 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d389d4640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d34161ef0 0x7f3d3414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d33fff640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340a0730 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d30ff9640 1 -- 172.21.15.53:0/371147833 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24041020 con 0x7f3d3415f370 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d30ff9640 1 -- 172.21.15.53:0/371147833 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d240306c0 con 0x7f3d3415f370 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3409d470 con 0x7f3d3415f370 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d30ff9640 1 -- 172.21.15.53:0/371147833 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24037040 con 0x7f3d3415f370 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.519+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3409da30 con 0x7f3d3415f370 2024-05-10T10:41:05.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.520+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3406a220 con 0x7f3d3415f370 2024-05-10T10:41:05.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.521+0000 7f3d30ff9640 1 -- 172.21.15.53:0/371147833 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d24044020 con 0x7f3d3415f370 2024-05-10T10:41:05.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.522+0000 7f3d30ff9640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3d200762d0 0x7f3d20078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:05.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.522+0000 7f3d30ff9640 1 -- 172.21.15.53:0/371147833 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6037+0+0 (secure 0 0 0) 0x7f3d240b7fb0 con 0x7f3d3415f370 2024-05-10T10:41:05.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.522+0000 7f3d337fe640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3d200762d0 0x7f3d20078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:05.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.522+0000 7f3d337fe640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3d200762d0 0x7f3d20078790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d28006fd0 tx=0x7f3d28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:05.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.526+0000 7f3d30ff9640 1 -- 172.21.15.53:0/371147833 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3d2404b030 con 0x7f3d3415f370 2024-05-10T10:41:05.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:05.714+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3d340b8d90 con 0x7f3d3415f370 2024-05-10T10:41:06.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.383+0000 7f3d30ff9640 1 -- 172.21.15.53:0/371147833 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f3d240817c0 con 0x7f3d3415f370 2024-05-10T10:41:06.384 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:41:06.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3d200762d0 msgr2=0x7f3d20078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3d200762d0 0x7f3d20078790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d28006fd0 tx=0x7f3d28008040 comp rx=0 tx=0).stop 2024-05-10T10:41:06.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 msgr2=0x7f3d340a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340a0730 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d240046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 shutdown_connections 2024-05-10T10:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3d200762d0 0x7f3d20078790 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d34161ef0 0x7f3d3414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d341639a0 0x7f3d3409cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 --2- 172.21.15.53:0/371147833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3415f370 0x7f3d340a0730 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.388+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 >> 172.21.15.53:0/371147833 conn(0x7f3d340d2e10 msgr2=0x7f3d3406e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.389+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 shutdown_connections 2024-05-10T10:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.389+0000 7f3d3a437640 1 -- 172.21.15.53:0/371147833 wait complete. 2024-05-10T10:41:06.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T10:41:06.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.516+0000 7f1021faf640 1 Processor -- start 2024-05-10T10:41:06.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.516+0000 7f1021faf640 1 -- start start 2024-05-10T10:41:06.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f1021faf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f101c15f3b0 0x7f101c163930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f1021faf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f101c0d0e90 0x7f101c06ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f1021faf640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f101c163e70 con 0x7f101c0cfdd0 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f1021faf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f101c0cf160 con 0x7f101c0d0e90 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f1021faf640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f101c15f790 con 0x7f101c15f3b0 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f101c0d0e90 0x7f101c06ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f101c0d0e90 0x7f101c06ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39522/0 (socket says 172.21.15.53:39522) 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101bfff640 1 -- 172.21.15.53:0/4028240850 learned_addr learned my addr 172.21.15.53:0/4028240850 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101b7fe640 1 --2- 172.21.15.53:0/4028240850 >> v2:172.21.15.135:3300/0 conn(0x7f101c15f3b0 0x7f101c163930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101bfff640 1 -- 172.21.15.53:0/4028240850 >> v2:172.21.15.135:3300/0 conn(0x7f101c15f3b0 msgr2=0x7f101c163930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101bfff640 1 --2- 172.21.15.53:0/4028240850 >> v2:172.21.15.135:3300/0 conn(0x7f101c15f3b0 0x7f101c163930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101bfff640 1 -- 172.21.15.53:0/4028240850 >> v1:172.21.15.135:6789/0 conn(0x7f101c0cfdd0 legacy=0x7f101c0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:06.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101bfff640 1 -- 172.21.15.53:0/4028240850 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c0ce550 con 0x7f101c0d0e90 2024-05-10T10:41:06.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.517+0000 7f101b7fe640 1 --2- 172.21.15.53:0/4028240850 >> v2:172.21.15.135:3300/0 conn(0x7f101c15f3b0 0x7f101c163930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:06.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.518+0000 7f101bfff640 1 --2- 172.21.15.53:0/4028240850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0d0e90 0x7f101c06ce30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f101c0743c0 tx=0x7f101402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b61e6b2c384526 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:06.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.518+0000 7f101a7fc640 1 -- 172.21.15.53:0/4028240850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014036070 con 0x7f101c0d0e90 2024-05-10T10:41:06.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.518+0000 7f101a7fc640 1 -- 172.21.15.53:0/4028240850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1014031600 con 0x7f101c0d0e90 2024-05-10T10:41:06.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.518+0000 7f101a7fc640 1 -- 172.21.15.53:0/4028240850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140319d0 con 0x7f101c0d0e90 2024-05-10T10:41:06.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 -- 172.21.15.53:0/4028240850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0d0e90 msgr2=0x7f101c06ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 --2- 172.21.15.53:0/4028240850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0d0e90 0x7f101c06ce30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f101c0743c0 tx=0x7f101402a660 comp rx=0 tx=0).stop 2024-05-10T10:41:06.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 -- 172.21.15.53:0/4028240850 shutdown_connections 2024-05-10T10:41:06.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 --2- 172.21.15.53:0/4028240850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0d0e90 0x7f101c06ce30 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 --2- 172.21.15.53:0/4028240850 >> v2:172.21.15.135:3300/0 conn(0x7f101c15f3b0 0x7f101c163930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 -- 172.21.15.53:0/4028240850 >> 172.21.15.53:0/4028240850 conn(0x7f101c0b3c60 msgr2=0x7f101c0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 -- 172.21.15.53:0/4028240850 shutdown_connections 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.519+0000 7f1021faf640 1 -- 172.21.15.53:0/4028240850 wait complete. 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.520+0000 7f1021faf640 1 Processor -- start 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.520+0000 7f1021faf640 1 -- start start 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f1021faf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0cfdd0 0x7f101c1f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f1021faf640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f101c0d0e90 0x7f101c160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f1021faf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101c15f3b0 0x7f101c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f1021faf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f101c0cedf0 con 0x7f101c0cfdd0 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f1021faf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f101c0ceb10 con 0x7f101c15f3b0 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f1021faf640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f101c06d370 con 0x7f101c0d0e90 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0cfdd0 0x7f101c1f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0cfdd0 0x7f101c1f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39526/0 (socket says 172.21.15.53:39526) 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 -- 172.21.15.53:0/3886542926 learned_addr learned my addr 172.21.15.53:0/3886542926 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:06.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101bfff640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101c15f3b0 0x7f101c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101affd640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f101c0d0e90 0x7f101c160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 -- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f101c0d0e90 msgr2=0x7f101c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f101c0d0e90 0x7f101c160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 -- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101c15f3b0 msgr2=0x7f101c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101c15f3b0 0x7f101c158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 -- 172.21.15.53:0/3886542926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c1f4bf0 con 0x7f101c0cfdd0 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101bfff640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101c15f3b0 0x7f101c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.521+0000 7f101b7fe640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0cfdd0 0x7f101c1f7ff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c7f0 tx=0x7f100c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.522+0000 7f1003fff640 1 -- 172.21.15.53:0/3886542926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c015070 con 0x7f101c0cfdd0 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.522+0000 7f1003fff640 1 -- 172.21.15.53:0/3886542926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c007820 con 0x7f101c0cfdd0 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.522+0000 7f1003fff640 1 -- 172.21.15.53:0/3886542926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c007bd0 con 0x7f101c0cfdd0 2024-05-10T10:41:06.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.522+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c1f86d0 con 0x7f101c0cfdd0 2024-05-10T10:41:06.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.522+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c1f8c90 con 0x7f101c0cfdd0 2024-05-10T10:41:06.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.523+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101c1f4a00 con 0x7f101c0cfdd0 2024-05-10T10:41:06.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.523+0000 7f1003fff640 1 -- 172.21.15.53:0/3886542926 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f100c018070 con 0x7f101c0cfdd0 2024-05-10T10:41:06.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.524+0000 7f1003fff640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1008076300 0x7f10080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.524+0000 7f1003fff640 1 -- 172.21.15.53:0/3886542926 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7f100c097590 con 0x7f101c0cfdd0 2024-05-10T10:41:06.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.524+0000 7f101affd640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1008076300 0x7f10080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.524+0000 7f101affd640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1008076300 0x7f10080787c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f101c0b6bb0 tx=0x7f1010009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:06.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.528+0000 7f1003fff640 1 -- 172.21.15.53:0/3886542926 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f100c061040 con 0x7f101c0cfdd0 2024-05-10T10:41:06.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.721+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f101c06a220 con 0x7f101c0cfdd0 2024-05-10T10:41:06.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.724+0000 7f1003fff640 1 -- 172.21.15.53:0/3886542926 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f100c060940 con 0x7f101c0cfdd0 2024-05-10T10:41:06.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.728+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1008076300 msgr2=0x7f10080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.728+0000 7f1021faf640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1008076300 0x7f10080787c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f101c0b6bb0 tx=0x7f1010009040 comp rx=0 tx=0).stop 2024-05-10T10:41:06.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.728+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0cfdd0 msgr2=0x7f101c1f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.728+0000 7f1021faf640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0cfdd0 0x7f101c1f7ff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c7f0 tx=0x7f100c00ccb0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.729+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 shutdown_connections 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.729+0000 7f1021faf640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1008076300 0x7f10080787c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.729+0000 7f1021faf640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101c15f3b0 0x7f101c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.729+0000 7f1021faf640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f101c0d0e90 0x7f101c160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.729+0000 7f1021faf640 1 --2- 172.21.15.53:0/3886542926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c0cfdd0 0x7f101c1f7ff0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.729+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 >> 172.21.15.53:0/3886542926 conn(0x7f101c0b3c60 msgr2=0x7f101c0d1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.729+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 shutdown_connections 2024-05-10T10:41:06.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.730+0000 7f1021faf640 1 -- 172.21.15.53:0/3886542926 wait complete. 2024-05-10T10:41:06.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-10T10:41:06.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.848+0000 7f2a2aca0640 1 Processor -- start 2024-05-10T10:41:06.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.848+0000 7f2a2aca0640 1 -- start start 2024-05-10T10:41:06.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2aca0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2a24162940 0x7f2a240b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2aca0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a240d3d80 0x7f2a240d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2aca0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a240d2d20 con 0x7f2a240d59d0 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2aca0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2a24162d20 con 0x7f2a240d3d80 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2aca0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2a240d3000 con 0x7f2a24162940 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2923d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a240d3d80 0x7f2a240d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2923d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a240d3d80 0x7f2a240d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39540/0 (socket says 172.21.15.53:39540) 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2923d640 1 -- 172.21.15.53:0/3350511122 learned_addr learned my addr 172.21.15.53:0/3350511122 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a28a3c640 1 --2- 172.21.15.53:0/3350511122 >> v2:172.21.15.135:3300/0 conn(0x7f2a24162940 0x7f2a240b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2923d640 1 -- 172.21.15.53:0/3350511122 >> v2:172.21.15.135:3300/0 conn(0x7f2a24162940 msgr2=0x7f2a240b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2923d640 1 --2- 172.21.15.53:0/3350511122 >> v2:172.21.15.135:3300/0 conn(0x7f2a24162940 0x7f2a240b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2923d640 1 -- 172.21.15.53:0/3350511122 >> v1:172.21.15.135:6789/0 conn(0x7f2a240d59d0 legacy=0x7f2a240d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.849+0000 7f2a2923d640 1 -- 172.21.15.53:0/3350511122 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a240d32e0 con 0x7f2a240d3d80 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.850+0000 7f2a2923d640 1 --2- 172.21.15.53:0/3350511122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a240d3d80 0x7f2a240d01e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2a2408d970 tx=0x7f2a1c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb00727fe8e47e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.850+0000 7f2a237fe640 1 -- 172.21.15.53:0/3350511122 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c036070 con 0x7f2a240d3d80 2024-05-10T10:41:06.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.850+0000 7f2a237fe640 1 -- 172.21.15.53:0/3350511122 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c031600 con 0x7f2a240d3d80 2024-05-10T10:41:06.852 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.850+0000 7f2a237fe640 1 -- 172.21.15.53:0/3350511122 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c0319d0 con 0x7f2a240d3d80 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 -- 172.21.15.53:0/3350511122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a240d3d80 msgr2=0x7f2a240d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/3350511122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a240d3d80 0x7f2a240d01e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2a2408d970 tx=0x7f2a1c02a660 comp rx=0 tx=0).stop 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 -- 172.21.15.53:0/3350511122 shutdown_connections 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/3350511122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a240d3d80 0x7f2a240d01e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/3350511122 >> v2:172.21.15.135:3300/0 conn(0x7f2a24162940 0x7f2a240b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 -- 172.21.15.53:0/3350511122 >> 172.21.15.53:0/3350511122 conn(0x7f2a2409e380 msgr2=0x7f2a240d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 -- 172.21.15.53:0/3350511122 shutdown_connections 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.851+0000 7f2a2aca0640 1 -- 172.21.15.53:0/3350511122 wait complete. 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.852+0000 7f2a2aca0640 1 Processor -- start 2024-05-10T10:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.852+0000 7f2a2aca0640 1 -- start start 2024-05-10T10:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2aca0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2a240d3d80 0x7f2a240b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2aca0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a240d59d0 0x7f2a240b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2aca0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a24162940 0x7f2a240b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2aca0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a240d2390 con 0x7f2a24162940 2024-05-10T10:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2aca0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a240cfc50 con 0x7f2a240d59d0 2024-05-10T10:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2aca0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2a240d2510 con 0x7f2a240d3d80 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a24162940 0x7f2a240b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a24162940 0x7f2a240b8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39554/0 (socket says 172.21.15.53:39554) 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 -- 172.21.15.53:0/1537004428 learned_addr learned my addr 172.21.15.53:0/1537004428 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a23fff640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a240d59d0 0x7f2a240b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a28a3c640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2a240d3d80 0x7f2a240b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 -- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2a240d3d80 msgr2=0x7f2a240b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2a240d3d80 0x7f2a240b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 -- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a240d59d0 msgr2=0x7f2a240b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a240d59d0 0x7f2a240b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.853+0000 7f2a2923d640 1 -- 172.21.15.53:0/1537004428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a240b8d20 con 0x7f2a24162940 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.854+0000 7f2a2923d640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a24162940 0x7f2a240b8f10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009f40 tx=0x7f2a1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.854+0000 7f2a217fa640 1 -- 172.21.15.53:0/1537004428 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c036050 con 0x7f2a24162940 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.854+0000 7f2a217fa640 1 -- 172.21.15.53:0/1537004428 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c042070 con 0x7f2a24162940 2024-05-10T10:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.854+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a240b3510 con 0x7f2a24162940 2024-05-10T10:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.854+0000 7f2a217fa640 1 -- 172.21.15.53:0/1537004428 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c009510 con 0x7f2a24162940 2024-05-10T10:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.854+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a240b3ad0 con 0x7f2a24162940 2024-05-10T10:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.855+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2406a220 con 0x7f2a24162940 2024-05-10T10:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.855+0000 7f2a217fa640 1 -- 172.21.15.53:0/1537004428 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2a1c007bd0 con 0x7f2a24162940 2024-05-10T10:41:06.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.856+0000 7f2a217fa640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2a080762d0 0x7f2a08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:06.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.856+0000 7f2a217fa640 1 -- 172.21.15.53:0/1537004428 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7f2a1c0b8460 con 0x7f2a24162940 2024-05-10T10:41:06.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.856+0000 7f2a28a3c640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2a080762d0 0x7f2a08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:06.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.857+0000 7f2a28a3c640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2a080762d0 0x7f2a08078790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2a18006fd0 tx=0x7f2a18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:06.861 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:06.860+0000 7f2a217fa640 1 -- 172.21.15.53:0/1537004428 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f2a1c081de0 con 0x7f2a24162940 2024-05-10T10:41:07.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.125+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f2a240cf570 con 0x7f2a24162940 2024-05-10T10:41:07.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.385+0000 7f2a217fa640 1 -- 172.21.15.53:0/1537004428 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v349) ==== 167+0+0 (secure 0 0 0) 0x7f2a1c0816e0 con 0x7f2a24162940 2024-05-10T10:41:07.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2a080762d0 msgr2=0x7f2a08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2a080762d0 0x7f2a08078790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2a18006fd0 tx=0x7f2a18008040 comp rx=0 tx=0).stop 2024-05-10T10:41:07.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a24162940 msgr2=0x7f2a240b8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a24162940 0x7f2a240b8f10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009f40 tx=0x7f2a1c004a40 comp rx=0 tx=0).stop 2024-05-10T10:41:07.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 shutdown_connections 2024-05-10T10:41:07.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2a080762d0 0x7f2a08078790 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a24162940 0x7f2a240b8f10 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a240d59d0 0x7f2a240b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 --2- 172.21.15.53:0/1537004428 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2a240d3d80 0x7f2a240b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 >> 172.21.15.53:0/1537004428 conn(0x7f2a2409e380 msgr2=0x7f2a24162eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:07.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 shutdown_connections 2024-05-10T10:41:07.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.387+0000 7f2a2aca0640 1 -- 172.21.15.53:0/1537004428 wait complete. 2024-05-10T10:41:07.400 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-10T10:41:07.400 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-10T10:41:07.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:41:07.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.505+0000 7f58ecff0640 1 Processor -- start 2024-05-10T10:41:07.505 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.505+0000 7f58ecff0640 1 -- start start 2024-05-10T10:41:07.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ecff0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ecff0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58e80703c0 0x7f58e8070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ecff0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e806cbe0 con 0x7f58e806edc0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ecff0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f58e8070d60 con 0x7f58e80703c0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ecff0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f58e80ce8e0 con 0x7f58e806c2c0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ddd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58e80703c0 0x7f58e8070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ddd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58e80703c0 0x7f58e8070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39570/0 (socket says 172.21.15.53:39570) 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ddd9b640 1 -- 172.21.15.53:0/3946725213 learned_addr learned my addr 172.21.15.53:0/3946725213 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58e659c640 1 --2- 172.21.15.53:0/3946725213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ddd9b640 1 -- 172.21.15.53:0/3946725213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 msgr2=0x7f58e806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ddd9b640 1 --2- 172.21.15.53:0/3946725213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ddd9b640 1 -- 172.21.15.53:0/3946725213 >> v1:172.21.15.135:6789/0 conn(0x7f58e806edc0 legacy=0x7f58e806f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58ddd9b640 1 -- 172.21.15.53:0/3946725213 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e80cebc0 con 0x7f58e80703c0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.506+0000 7f58e659c640 1 --2- 172.21.15.53:0/3946725213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.507+0000 7f58ddd9b640 1 --2- 172.21.15.53:0/3946725213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e80703c0 0x7f58e8070820 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009870 tx=0x7f58d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94e592348460c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.507+0000 7f58e5d9b640 1 -- 172.21.15.53:0/3946725213 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037070 con 0x7f58e80703c0 2024-05-10T10:41:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.507+0000 7f58e5d9b640 1 -- 172.21.15.53:0/3946725213 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0032440 con 0x7f58e80703c0 2024-05-10T10:41:07.509 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.507+0000 7f58e5d9b640 1 -- 172.21.15.53:0/3946725213 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0032810 con 0x7f58e80703c0 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 -- 172.21.15.53:0/3946725213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e80703c0 msgr2=0x7f58e8070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 --2- 172.21.15.53:0/3946725213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e80703c0 0x7f58e8070820 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009870 tx=0x7f58d002a210 comp rx=0 tx=0).stop 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 -- 172.21.15.53:0/3946725213 shutdown_connections 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 --2- 172.21.15.53:0/3946725213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e80703c0 0x7f58e8070820 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 --2- 172.21.15.53:0/3946725213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 -- 172.21.15.53:0/3946725213 >> 172.21.15.53:0/3946725213 conn(0x7f58e80b3980 msgr2=0x7f58e80b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 -- 172.21.15.53:0/3946725213 shutdown_connections 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.508+0000 7f58ecff0640 1 -- 172.21.15.53:0/3946725213 wait complete. 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.509+0000 7f58ecff0640 1 Processor -- start 2024-05-10T10:41:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.509+0000 7f58ecff0640 1 -- start start 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ecff0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ecff0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e806edc0 0x7f58e80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ecff0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58e80703c0 0x7f58e80b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ecff0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58e80cfc70 con 0x7f58e806edc0 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ecff0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58e80cf180 con 0x7f58e80703c0 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ecff0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f58e815d2a0 con 0x7f58e806c2c0 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e806edc0 0x7f58e80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e806edc0 0x7f58e80afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39584/0 (socket says 172.21.15.53:39584) 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 -- 172.21.15.53:0/4021093550 learned_addr learned my addr 172.21.15.53:0/4021093550 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58e659c640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58e6d9d640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58e80703c0 0x7f58e80b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 -- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 msgr2=0x7f58e814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e814d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 -- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58e80703c0 msgr2=0x7f58e80b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58e80703c0 0x7f58e80b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58ddd9b640 1 -- 172.21.15.53:0/4021093550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e81f5640 con 0x7f58e806edc0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.510+0000 7f58e6d9d640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58e80703c0 0x7f58e80b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.511+0000 7f58ddd9b640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e806edc0 0x7f58e80afdb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097c0 tx=0x7f58d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.511+0000 7f58df7fe640 1 -- 172.21.15.53:0/4021093550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0041020 con 0x7f58e806edc0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.511+0000 7f58df7fe640 1 -- 172.21.15.53:0/4021093550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0030970 con 0x7f58e806edc0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.511+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e81f58e0 con 0x7f58e806edc0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.511+0000 7f58df7fe640 1 -- 172.21.15.53:0/4021093550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037040 con 0x7f58e806edc0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.511+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e81f5ea0 con 0x7f58e806edc0 2024-05-10T10:41:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.512+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e80cfdf0 con 0x7f58e806edc0 2024-05-10T10:41:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.512+0000 7f58df7fe640 1 -- 172.21.15.53:0/4021093550 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58d0044020 con 0x7f58e806edc0 2024-05-10T10:41:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.513+0000 7f58df7fe640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58c80762d0 0x7f58c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.513+0000 7f58df7fe640 1 -- 172.21.15.53:0/4021093550 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7f58d00b7e20 con 0x7f58e806edc0 2024-05-10T10:41:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.513+0000 7f58e659c640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58c80762d0 0x7f58c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.514+0000 7f58e659c640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58c80762d0 0x7f58c8078790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58e80b3400 tx=0x7f58d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.517+0000 7f58df7fe640 1 -- 172.21.15.53:0/4021093550 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f58e80cfdf0 con 0x7f58e806edc0 2024-05-10T10:41:07.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.708+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58e806a220 con 0x7f58e806edc0 2024-05-10T10:41:07.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.709+0000 7f58df7fe640 1 -- 172.21.15.53:0/4021093550 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13552 (secure 0 0 0) 0x7f58d004b030 con 0x7f58e806edc0 2024-05-10T10:41:07.710 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:07.710 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":94,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:41:06.379768+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6833/1098976714":"2024-05-11T10:41:03.267623+0000","172.21.15.53:6833/285942496":"2024-05-11T10:40:21.895821+0000","172.21.15.53:6832/285942496":"2024-05-11T10:40:21.895821+0000","172.21.15.53:6833/2934725760":"2024-05-11T10:39:11.380694+0000","172.21.15.53:6832/1098976714":"2024-05-11T10:41:03.267623+0000","172.21.15.53:6832/2934725760":"2024-05-11T10:39:11.380694+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:41:07.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58c80762d0 msgr2=0x7f58c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58c80762d0 0x7f58c8078790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58e80b3400 tx=0x7f58d8008040 comp rx=0 tx=0).stop 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e806edc0 msgr2=0x7f58e80afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e806edc0 0x7f58e80afdb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097c0 tx=0x7f58d00046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 shutdown_connections 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f58c80762d0 0x7f58c8078790 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58e80703c0 0x7f58e80b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58e806edc0 0x7f58e80afdb0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 --2- 172.21.15.53:0/4021093550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f58e806c2c0 0x7f58e814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 >> 172.21.15.53:0/4021093550 conn(0x7f58e80b3980 msgr2=0x7f58e806f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 shutdown_connections 2024-05-10T10:41:07.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.711+0000 7f58ecff0640 1 -- 172.21.15.53:0/4021093550 wait complete. 2024-05-10T10:41:07.724 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:33:28.171977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T10:41:07.724 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:41:07.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.830+0000 7f05b6dda640 1 Processor -- start 2024-05-10T10:41:07.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.830+0000 7f05b6dda640 1 -- start start 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.830+0000 7f05b6dda640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f05b00cffd0 0x7f05b00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.830+0000 7f05b6dda640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b006e720 con 0x7f05b00cd500 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.830+0000 7f05b6dda640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b006de80 con 0x7f05b00cf3e0 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.830+0000 7f05b6dda640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f05b006e160 con 0x7f05b00cffd0 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05b5377640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f05b00cd500 0x7f05b00d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55714/0 (socket says 172.21.15.53:55714) 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05b5377640 1 -- 172.21.15.53:0/4259506268 learned_addr learned my addr 172.21.15.53:0/4259506268 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 442922555 0 0) 0x7f05b006e720 con 0x7f05b00cd500 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05affff640 1 --2- 172.21.15.53:0/4259506268 >> v2:172.21.15.135:3300/0 conn(0x7f05b00cffd0 0x7f05b00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f059c003520 con 0x7f05b00cd500 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1566659932 0 0) 0x7f059c003520 con 0x7f05b00cd500 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 >> v2:172.21.15.135:3300/0 conn(0x7f05b00cffd0 msgr2=0x7f05b00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 --2- 172.21.15.53:0/4259506268 >> v2:172.21.15.135:3300/0 conn(0x7f05b00cffd0 0x7f05b00b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 >> v1:172.21.15.135:6789/0 conn(0x7f05b00cf3e0 legacy=0x7f05b00cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b006e440 con 0x7f05b00cd500 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f05a00019f0 con 0x7f05b00cd500 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 >> v1:172.21.15.53:6789/0 conn(0x7f05b00cd500 legacy=0x7f05b00d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f059c0039b0 0x7f059c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f059c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059c0051c0 0x7f059c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05b006e160 con 0x7f059c0051c0 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f059c00c060 con 0x7f059c004800 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f059c00c260 con 0x7f059c0039b0 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05b5377640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059c0051c0 0x7f059c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05affff640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f059c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.831+0000 7f05b4b76640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f059c0039b0 0x7f059c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05affff640 1 -- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f059c0039b0 msgr2=0x7f059c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05affff640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f059c0039b0 0x7f059c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05affff640 1 -- 172.21.15.53:0/4259506268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059c0051c0 msgr2=0x7f059c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05affff640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059c0051c0 0x7f059c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05affff640 1 -- 172.21.15.53:0/4259506268 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c003520 con 0x7f059c004800 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05b5377640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059c0051c0 0x7f059c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05b5377640 1 -- 172.21.15.53:0/4259506268 reap_dead start 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.832+0000 7f05affff640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f059c004c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0598009ac0 tx=0x7f059c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54a5c5334c0d5e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598036070 con 0x7f059c004800 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0598031400 con 0x7f059c004800 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05af7fe640 1 -- 172.21.15.53:0/4259506268 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598009690 con 0x7f059c004800 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05b6dda640 1 -- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 msgr2=0x7f059c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05b6dda640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f059c004c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0598009ac0 tx=0x7f059c005830 comp rx=0 tx=0).stop 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05b6dda640 1 -- 172.21.15.53:0/4259506268 shutdown_connections 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05b6dda640 1 --2- 172.21.15.53:0/4259506268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f059c004c80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05b6dda640 1 -- 172.21.15.53:0/4259506268 >> 172.21.15.53:0/4259506268 conn(0x7f05b00d2eb0 msgr2=0x7f05b00d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:07.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05b6dda640 1 -- 172.21.15.53:0/4259506268 shutdown_connections 2024-05-10T10:41:07.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.833+0000 7f05b6dda640 1 -- 172.21.15.53:0/4259506268 wait complete. 2024-05-10T10:41:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.834+0000 7f05b6dda640 1 Processor -- start 2024-05-10T10:41:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.835+0000 7f05b6dda640 1 -- start start 2024-05-10T10:41:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.835+0000 7f05b6dda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f05b015da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.835+0000 7f05b6dda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b015c5f0 0x7f05b01532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.835+0000 7f05b6dda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05b00c8bb0 0x7f05b00c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.835+0000 7f05b6dda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05b00cf8c0 con 0x7f05b015c5f0 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.835+0000 7f05b6dda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05b00b4510 con 0x7f059c004800 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.835+0000 7f05b6dda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f05b00cfa40 con 0x7f05b00c8bb0 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b015c5f0 0x7f05b01532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b015c5f0 0x7f05b01532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39590/0 (socket says 172.21.15.53:39590) 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 -- 172.21.15.53:0/2150666793 learned_addr learned my addr 172.21.15.53:0/2150666793 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05b4b76640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f05b015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05b5377640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05b00c8bb0 0x7f05b00c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 -- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05b00c8bb0 msgr2=0x7f05b00c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05b00c8bb0 0x7f05b00c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 -- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 msgr2=0x7f05b015da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f05b015da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 -- 172.21.15.53:0/2150666793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b00ca2e0 con 0x7f05b015c5f0 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05affff640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b015c5f0 0x7f05b01532e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0598034040 tx=0x7f0598004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05ad7fa640 1 -- 172.21.15.53:0/2150666793 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059803f020 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05ad7fa640 1 -- 172.21.15.53:0/2150666793 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0598037040 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b00ca5c0 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05ad7fa640 1 -- 172.21.15.53:0/2150666793 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598036070 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.836+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b00cab80 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.837+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b006a220 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.838+0000 7f05ad7fa640 1 -- 172.21.15.53:0/2150666793 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0598042020 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.838+0000 7f05ad7fa640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0590076300 0x7f05900787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.839+0000 7f05ad7fa640 1 -- 172.21.15.53:0/2150666793 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7f05980b74f0 con 0x7f05b015c5f0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.839+0000 7f05b4b76640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0590076300 0x7f05900787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.839+0000 7f05b4b76640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0590076300 0x7f05900787c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:07.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:07.843+0000 7f05ad7fa640 1 -- 172.21.15.53:0/2150666793 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0598049030 con 0x7f05b015c5f0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.033+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05b006e720 con 0x7f05b015c5f0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.034+0000 7f05ad7fa640 1 -- 172.21.15.53:0/2150666793 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f0598080980 con 0x7f05b015c5f0 2024-05-10T10:41:08.034 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-05-10T10:41:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.035+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0590076300 msgr2=0x7f05900787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.035+0000 7f05b6dda640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0590076300 0x7f05900787c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a8009040 comp rx=0 tx=0).stop 2024-05-10T10:41:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.035+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b015c5f0 msgr2=0x7f05b01532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.035+0000 7f05b6dda640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b015c5f0 0x7f05b01532e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0598034040 tx=0x7f0598004730 comp rx=0 tx=0).stop 2024-05-10T10:41:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 shutdown_connections 2024-05-10T10:41:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0590076300 0x7f05900787c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05b00c8bb0 0x7f05b00c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05b015c5f0 0x7f05b01532e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 --2- 172.21.15.53:0/2150666793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f059c004800 0x7f05b015da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 >> 172.21.15.53:0/2150666793 conn(0x7f05b00d2eb0 msgr2=0x7f05b00d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 shutdown_connections 2024-05-10T10:41:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.036+0000 7f05b6dda640 1 -- 172.21.15.53:0/2150666793 wait complete. 2024-05-10T10:41:08.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-10T10:41:08.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.151+0000 7f95d3cb0640 1 Processor -- start 2024-05-10T10:41:08.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.151+0000 7f95d3cb0640 1 -- start start 2024-05-10T10:41:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d3cb0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f95cc0d5cb0 0x7f95cc0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d3cb0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95cc0b45f0 con 0x7f95cc162750 2024-05-10T10:41:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d3cb0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95cc0b48d0 con 0x7f95cc162b30 2024-05-10T10:41:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d3cb0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f95cc0b4f20 con 0x7f95cc0d5cb0 2024-05-10T10:41:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d1a4c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f95cc162750 0x7f95cc0d0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55726/0 (socket says 172.21.15.53:55726) 2024-05-10T10:41:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d1a4c640 1 -- 172.21.15.53:0/1737382980 learned_addr learned my addr 172.21.15.53:0/1737382980 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:08.153 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d124b640 1 --2- 172.21.15.53:0/1737382980 >> v2:172.21.15.135:3300/0 conn(0x7f95cc0d5cb0 0x7f95cc0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749531888 0 0) 0x7f95cc0b45f0 con 0x7f95cc162750 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95b4003520 con 0x7f95cc162750 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2304306940 0 0) 0x7f95cc0b48d0 con 0x7f95cc162b30 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95cc0b45f0 con 0x7f95cc162b30 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 516234938 0 0) 0x7f95b4003520 con 0x7f95cc162750 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d124b640 1 -- 172.21.15.53:0/1737382980 >> v1:172.21.15.135:6789/0 conn(0x7f95cc162b30 legacy=0x7f95cc0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d124b640 1 -- 172.21.15.53:0/1737382980 >> v1:172.21.15.53:6789/0 conn(0x7f95cc162750 legacy=0x7f95cc0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.152+0000 7f95d124b640 1 -- 172.21.15.53:0/1737382980 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc0b4bb0 con 0x7f95cc0d5cb0 2024-05-10T10:41:08.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.153+0000 7f95d124b640 1 --2- 172.21.15.53:0/1737382980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0d5cb0 0x7f95cc0d3630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009d90 tx=0x7f95c002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9cd5fb80d5c4316 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.153+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4001670 con 0x7f95cc0d5cb0 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.153+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c0007540 con 0x7f95cc0d5cb0 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.153+0000 7f95d0a4a640 1 -- 172.21.15.53:0/1737382980 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c00078f0 con 0x7f95cc0d5cb0 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.154+0000 7f95d3cb0640 1 -- 172.21.15.53:0/1737382980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0d5cb0 msgr2=0x7f95cc0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.154+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/1737382980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0d5cb0 0x7f95cc0d3630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009d90 tx=0x7f95c002a5f0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.154+0000 7f95d3cb0640 1 -- 172.21.15.53:0/1737382980 shutdown_connections 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.154+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/1737382980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0d5cb0 0x7f95cc0d3630 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.154+0000 7f95d3cb0640 1 -- 172.21.15.53:0/1737382980 >> 172.21.15.53:0/1737382980 conn(0x7f95cc09e3b0 msgr2=0x7f95cc0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.154+0000 7f95d3cb0640 1 -- 172.21.15.53:0/1737382980 shutdown_connections 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.154+0000 7f95d3cb0640 1 -- 172.21.15.53:0/1737382980 wait complete. 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.155+0000 7f95d3cb0640 1 Processor -- start 2024-05-10T10:41:08.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.155+0000 7f95d3cb0640 1 -- start start 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d3cb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc162750 0x7f95cc157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d3cb0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95cc162b30 0x7f95cc15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d3cb0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0b8650 0x7f95cc1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d3cb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc0d3f40 con 0x7f95cc162750 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d3cb0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc0d3dc0 con 0x7f95cc0b8650 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d3cb0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc0d0fe0 con 0x7f95cc162b30 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc162750 0x7f95cc157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc162750 0x7f95cc157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39600/0 (socket says 172.21.15.53:39600) 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 -- 172.21.15.53:0/2340391491 learned_addr learned my addr 172.21.15.53:0/2340391491 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d124b640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95cc162b30 0x7f95cc15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d224d640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0b8650 0x7f95cc1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 -- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95cc162b30 msgr2=0x7f95cc15fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95cc162b30 0x7f95cc15fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 -- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0b8650 msgr2=0x7f95cc1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0b8650 0x7f95cc1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.156+0000 7f95d1a4c640 1 -- 172.21.15.53:0/2340391491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc1f94d0 con 0x7f95cc162750 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.157+0000 7f95d1a4c640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc162750 0x7f95cc157130 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c5a0 tx=0x7f95c400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.157+0000 7f95ca7fc640 1 -- 172.21.15.53:0/2340391491 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4017070 con 0x7f95cc162750 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.157+0000 7f95ca7fc640 1 -- 172.21.15.53:0/2340391491 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4005ce0 con 0x7f95cc162750 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.157+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc157670 con 0x7f95cc162750 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.157+0000 7f95ca7fc640 1 -- 172.21.15.53:0/2340391491 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c40072b0 con 0x7f95cc162750 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.157+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc0cd560 con 0x7f95cc162750 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.158+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc06a220 con 0x7f95cc162750 2024-05-10T10:41:08.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.158+0000 7f95ca7fc640 1 -- 172.21.15.53:0/2340391491 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f95c401a070 con 0x7f95cc162750 2024-05-10T10:41:08.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.159+0000 7f95ca7fc640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95b80762d0 0x7f95b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:08.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.159+0000 7f95ca7fc640 1 -- 172.21.15.53:0/2340391491 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7f95c4096a20 con 0x7f95cc162750 2024-05-10T10:41:08.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.159+0000 7f95d124b640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95b80762d0 0x7f95b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:08.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.160+0000 7f95d124b640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95b80762d0 0x7f95b8078790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f95c0034000 tx=0x7f95c002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:08.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.163+0000 7f95ca7fc640 1 -- 172.21.15.53:0/2340391491 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f95c4060440 con 0x7f95cc162750 2024-05-10T10:41:08.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:08.432+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f95cc0d58f0 con 0x7f95cc162750 2024-05-10T10:41:09.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.390+0000 7f95ca7fc640 1 -- 172.21.15.53:0/2340391491 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v351) ==== 149+0+0 (secure 0 0 0) 0x7f95c405fd40 con 0x7f95cc162750 2024-05-10T10:41:09.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95b80762d0 msgr2=0x7f95b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95b80762d0 0x7f95b8078790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f95c0034000 tx=0x7f95c002f000 comp rx=0 tx=0).stop 2024-05-10T10:41:09.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc162750 msgr2=0x7f95cc157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc162750 0x7f95cc157130 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c5a0 tx=0x7f95c400ca60 comp rx=0 tx=0).stop 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 shutdown_connections 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95b80762d0 0x7f95b8078790 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95cc0b8650 0x7f95cc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95cc162b30 0x7f95cc15fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.393+0000 7f95d3cb0640 1 --2- 172.21.15.53:0/2340391491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc162750 0x7f95cc157130 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.394+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 >> 172.21.15.53:0/2340391491 conn(0x7f95cc09e3b0 msgr2=0x7f95cc0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.394+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 shutdown_connections 2024-05-10T10:41:09.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.394+0000 7f95d3cb0640 1 -- 172.21.15.53:0/2340391491 wait complete. 2024-05-10T10:41:09.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T10:41:09.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.522+0000 7fb422609640 1 Processor -- start 2024-05-10T10:41:09.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.522+0000 7fb422609640 1 -- start start 2024-05-10T10:41:09.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.522+0000 7fb422609640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb41c0b4f90 0x7fb41c0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.522+0000 7fb422609640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c0d2800 con 0x7fb41c15f410 2024-05-10T10:41:09.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.522+0000 7fb422609640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c0d2ae0 con 0x7fb41c0d36d0 2024-05-10T10:41:09.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.522+0000 7fb422609640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb41c0d01d0 con 0x7fb41c0b4f90 2024-05-10T10:41:09.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb41c15f410 0x7fb41c0d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55734/0 (socket says 172.21.15.53:55734) 2024-05-10T10:41:09.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41bfff640 1 -- 172.21.15.53:0/583026574 learned_addr learned my addr 172.21.15.53:0/583026574 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb413fff640 1 --2- 172.21.15.53:0/583026574 >> v2:172.21.15.135:3300/0 conn(0x7fb41c0b4f90 0x7fb41c0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152162345 0 0) 0x7fb41c0d2800 con 0x7fb41c15f410 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb408003520 con 0x7fb41c15f410 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2746609804 0 0) 0x7fb408003520 con 0x7fb41c15f410 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 >> v2:172.21.15.135:3300/0 conn(0x7fb41c0b4f90 msgr2=0x7fb41c0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 --2- 172.21.15.53:0/583026574 >> v2:172.21.15.135:3300/0 conn(0x7fb41c0b4f90 0x7fb41c0b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 >> v1:172.21.15.135:6789/0 conn(0x7fb41c0d36d0 legacy=0x7fb41c0d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c0d04b0 con 0x7fb41c15f410 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fb40c001670 con 0x7fb41c15f410 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 >> v1:172.21.15.53:6789/0 conn(0x7fb41c15f410 legacy=0x7fb41c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4080039b0 0x7fb408003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb408004800 0x7fb408004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4080051c0 0x7fb40800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0d01d0 con 0x7fb408004800 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb40800c160 con 0x7fb4080039b0 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb40800c360 con 0x7fb4080051c0 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb413fff640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb408004800 0x7fb408004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.523+0000 7fb41bfff640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4080039b0 0x7fb408003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb420ba6640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4080051c0 0x7fb40800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb413fff640 1 -- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4080051c0 msgr2=0x7fb40800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb413fff640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4080051c0 0x7fb40800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb413fff640 1 -- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4080039b0 msgr2=0x7fb408003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:09.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb413fff640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4080039b0 0x7fb408003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb420ba6640 1 -- 172.21.15.53:0/583026574 reap_dead start 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb413fff640 1 -- 172.21.15.53:0/583026574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb408003520 con 0x7fb408004800 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.524+0000 7fb413fff640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb408004800 0x7fb408004c80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb404009ac0 tx=0x7fb408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64df34fde0a4b97d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404036070 con 0x7fb408004800 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb404031400 con 0x7fb408004800 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb41b7fe640 1 -- 172.21.15.53:0/583026574 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404009690 con 0x7fb408004800 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb422609640 1 -- 172.21.15.53:0/583026574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb408004800 msgr2=0x7fb408004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb422609640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb408004800 0x7fb408004c80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb404009ac0 tx=0x7fb408005830 comp rx=0 tx=0).stop 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb422609640 1 -- 172.21.15.53:0/583026574 shutdown_connections 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb422609640 1 --2- 172.21.15.53:0/583026574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb408004800 0x7fb408004c80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb422609640 1 -- 172.21.15.53:0/583026574 >> 172.21.15.53:0/583026574 conn(0x7fb41c08ed30 msgr2=0x7fb41c09e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb422609640 1 -- 172.21.15.53:0/583026574 shutdown_connections 2024-05-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.525+0000 7fb422609640 1 -- 172.21.15.53:0/583026574 wait complete. 2024-05-10T10:41:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.526+0000 7fb422609640 1 Processor -- start 2024-05-10T10:41:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb422609640 1 -- start start 2024-05-10T10:41:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb422609640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb408004800 0x7fb41c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb422609640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb41c0b7f40 0x7fb41c161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb422609640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb41c0cc3f0 0x7fb41c0cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb422609640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0cffb0 con 0x7fb41c0b7f40 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb422609640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0cfe30 con 0x7fb408004800 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb422609640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb41c0b5410 con 0x7fb41c0cc3f0 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb413fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb41c0b7f40 0x7fb41c161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb413fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb41c0b7f40 0x7fb41c161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39618/0 (socket says 172.21.15.53:39618) 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.527+0000 7fb413fff640 1 -- 172.21.15.53:0/2666933234 learned_addr learned my addr 172.21.15.53:0/2666933234 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb420ba6640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb41c0cc3f0 0x7fb41c0cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb41bfff640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb408004800 0x7fb41c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb413fff640 1 -- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb41c0cc3f0 msgr2=0x7fb41c0cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb413fff640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb41c0cc3f0 0x7fb41c0cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb413fff640 1 -- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb408004800 msgr2=0x7fb41c15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb413fff640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb408004800 0x7fb41c15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb413fff640 1 -- 172.21.15.53:0/2666933234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c15ab90 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb413fff640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb41c0b7f40 0x7fb41c161980 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb404009be0 tx=0x7fb404004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb4197fa640 1 -- 172.21.15.53:0/2666933234 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40403f020 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb4197fa640 1 -- 172.21.15.53:0/2666933234 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb404037040 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c0b8400 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb4197fa640 1 -- 172.21.15.53:0/2666933234 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404036070 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.528+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c0b89c0 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.529+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c06a220 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.530+0000 7fb4197fa640 1 -- 172.21.15.53:0/2666933234 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb404042020 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.530+0000 7fb4197fa640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3fc0762d0 0x7fb3fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.530+0000 7fb4197fa640 1 -- 172.21.15.53:0/2666933234 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7fb4040b72e0 con 0x7fb41c0b7f40 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.531+0000 7fb41bfff640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3fc0762d0 0x7fb3fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:09.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.531+0000 7fb41bfff640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3fc0762d0 0x7fb3fc078790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb40c007c10 tx=0x7fb40c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:09.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.535+0000 7fb4197fa640 1 -- 172.21.15.53:0/2666933234 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb404049070 con 0x7fb41c0b7f40 2024-05-10T10:41:09.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.730+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb41c15d530 con 0x7fb41c0b7f40 2024-05-10T10:41:09.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.733+0000 7fb4197fa640 1 -- 172.21.15.53:0/2666933234 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fb404080d00 con 0x7fb41c0b7f40 2024-05-10T10:41:09.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3fc0762d0 msgr2=0x7fb3fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3fc0762d0 0x7fb3fc078790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb40c007c10 tx=0x7fb40c009040 comp rx=0 tx=0).stop 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb41c0b7f40 msgr2=0x7fb41c161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb41c0b7f40 0x7fb41c161980 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb404009be0 tx=0x7fb404004730 comp rx=0 tx=0).stop 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 shutdown_connections 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3fc0762d0 0x7fb3fc078790 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb41c0cc3f0 0x7fb41c0cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb41c0b7f40 0x7fb41c161980 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 --2- 172.21.15.53:0/2666933234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb408004800 0x7fb41c15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 >> 172.21.15.53:0/2666933234 conn(0x7fb41c08ed30 msgr2=0x7fb41c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 shutdown_connections 2024-05-10T10:41:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:09.736+0000 7fb422609640 1 -- 172.21.15.53:0/2666933234 wait complete. 2024-05-10T10:41:09.747 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:41:09.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:41:09.790 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:41:09.843 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:41:09.843 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:09.844 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:09.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:41:09.860 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:41:09.860 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:09.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:41:09.879 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:09.879 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:09.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:41:09.916 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:41:09.917 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:09.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:41:09.935 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T10:41:09.936 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:10.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.059+0000 7f56e55f2640 1 Processor -- start 2024-05-10T10:41:10.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.060+0000 7f56e55f2640 1 -- start start 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.060+0000 7f56e55f2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56e015f290 0x7f56e00d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.060+0000 7f56e55f2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e006f3c0 con 0x7f56e00d3450 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.060+0000 7f56e55f2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e00d16f0 con 0x7f56e00702e0 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.060+0000 7f56e55f2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f56e00d14c0 con 0x7f56e015f290 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.060+0000 7f56df7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f56e00d3450 0x7f56e006d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55744/0 (socket says 172.21.15.53:55744) 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56df7fe640 1 -- 172.21.15.53:0/4038899716 learned_addr learned my addr 172.21.15.53:0/4038899716 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536100856 0 0) 0x7f56e006f3c0 con 0x7f56e00d3450 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56cc003520 con 0x7f56e00d3450 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56deffd640 1 --2- 172.21.15.53:0/4038899716 >> v2:172.21.15.135:3300/0 conn(0x7f56e015f290 0x7f56e00d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2963719733 0 0) 0x7f56cc003520 con 0x7f56e00d3450 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 >> v2:172.21.15.135:3300/0 conn(0x7f56e015f290 msgr2=0x7f56e00d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 --2- 172.21.15.53:0/4038899716 >> v2:172.21.15.135:3300/0 conn(0x7f56e015f290 0x7f56e00d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 >> v1:172.21.15.135:6789/0 conn(0x7f56e00702e0 legacy=0x7f56e00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e00d19d0 con 0x7f56e00d3450 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f56d00019f0 con 0x7f56e00d3450 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 >> v1:172.21.15.53:6789/0 conn(0x7f56e00d3450 legacy=0x7f56e006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56cc0039b0 0x7f56cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56cc004800 0x7f56cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc0051c0 0x7f56cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc00c040 con 0x7f56cc004800 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc00c1f0 con 0x7f56cc0039b0 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc00c3f0 con 0x7f56cc0051c0 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56de7fc640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56cc004800 0x7f56cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56deffd640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56cc0039b0 0x7f56cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.061+0000 7f56df7fe640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc0051c0 0x7f56cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56de7fc640 1 -- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc0051c0 msgr2=0x7f56cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56de7fc640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc0051c0 0x7f56cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56de7fc640 1 -- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56cc0039b0 msgr2=0x7f56cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56de7fc640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56cc0039b0 0x7f56cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56de7fc640 1 -- 172.21.15.53:0/4038899716 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc003520 con 0x7f56cc004800 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56df7fe640 1 -- 172.21.15.53:0/4038899716 reap_dead start 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56deffd640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56cc0039b0 0x7f56cc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56de7fc640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56cc004800 0x7f56cc004c80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f56c8001750 tx=0x7f56cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400bdfa896e32880 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8001370 con 0x7f56cc004800 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c80052d0 con 0x7f56cc004800 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.062+0000 7f56ddffb640 1 -- 172.21.15.53:0/4038899716 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c802abc0 con 0x7f56cc004800 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.063+0000 7f56e55f2640 1 -- 172.21.15.53:0/4038899716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56cc004800 msgr2=0x7f56cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.063+0000 7f56e55f2640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56cc004800 0x7f56cc004c80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f56c8001750 tx=0x7f56cc005830 comp rx=0 tx=0).stop 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.063+0000 7f56e55f2640 1 -- 172.21.15.53:0/4038899716 shutdown_connections 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.063+0000 7f56e55f2640 1 --2- 172.21.15.53:0/4038899716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56cc004800 0x7f56cc004c80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.063+0000 7f56e55f2640 1 -- 172.21.15.53:0/4038899716 >> 172.21.15.53:0/4038899716 conn(0x7f56e009ba20 msgr2=0x7f56e009be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.063+0000 7f56e55f2640 1 -- 172.21.15.53:0/4038899716 shutdown_connections 2024-05-10T10:41:10.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.063+0000 7f56e55f2640 1 -- 172.21.15.53:0/4038899716 wait complete. 2024-05-10T10:41:10.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.064+0000 7f56e55f2640 1 Processor -- start 2024-05-10T10:41:10.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.064+0000 7f56e55f2640 1 -- start start 2024-05-10T10:41:10.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56e55f2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc004800 0x7f56e01f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56e55f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e00c85c0 0x7f56e00c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56e55f2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e01f1b30 0x7f56e01f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56e55f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56e00d1f90 con 0x7f56e00c85c0 2024-05-10T10:41:10.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56e55f2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56e00d2300 con 0x7f56e01f1b30 2024-05-10T10:41:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56e55f2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f56e015f670 con 0x7f56cc004800 2024-05-10T10:41:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56de7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e00c85c0 0x7f56e00c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56de7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e00c85c0 0x7f56e00c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39634/0 (socket says 172.21.15.53:39634) 2024-05-10T10:41:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56de7fc640 1 -- 172.21.15.53:0/37957266 learned_addr learned my addr 172.21.15.53:0/37957266 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56de7fc640 1 -- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc004800 msgr2=0x7f56e01f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56df7fe640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e01f1b30 0x7f56e01f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.065+0000 7f56deffd640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc004800 0x7f56e01f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56de7fc640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc004800 0x7f56e01f8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56de7fc640 1 -- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e01f1b30 msgr2=0x7f56e01f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56de7fc640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e01f1b30 0x7f56e01f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56de7fc640 1 -- 172.21.15.53:0/37957266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e01f90d0 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56de7fc640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e00c85c0 0x7f56e00c8a40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f56c8000c00 tx=0x7f56c8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56d77fe640 1 -- 172.21.15.53:0/37957266 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8040030 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56d77fe640 1 -- 172.21.15.53:0/37957266 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c802be50 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e01f4fa0 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56d77fe640 1 -- 172.21.15.53:0/37957266 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8001370 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.066+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e01f5560 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.067+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e00693f0 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.067+0000 7f56d77fe640 1 -- 172.21.15.53:0/37957266 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f56c8043070 con 0x7f56e00c85c0 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.068+0000 7f56d77fe640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56c0076300 0x7f56c00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.068+0000 7f56d77fe640 1 -- 172.21.15.53:0/37957266 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7f56c80b7070 con 0x7f56e00c85c0 2024-05-10T10:41:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.068+0000 7f56deffd640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56c0076300 0x7f56c00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.069+0000 7f56deffd640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56c0076300 0x7f56c00787c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f56e015bfa0 tx=0x7f56d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.072+0000 7f56d77fe640 1 -- 172.21.15.53:0/37957266 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f56c804a070 con 0x7f56e00c85c0 2024-05-10T10:41:10.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.302+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56e006e890 con 0x7f56e00c85c0 2024-05-10T10:41:10.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.303+0000 7f56d77fe640 1 -- 172.21.15.53:0/37957266 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3872 (secure 0 0 0) 0x7f56e01f90d0 con 0x7f56e00c85c0 2024-05-10T10:41:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:10.304 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3466509794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3466509794},{"type":"v1","addr":"172.21.15.135:6836","nonce":3466509794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3995509937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3995509937},{"type":"v1","addr":"172.21.15.53:6835","nonce":3995509937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6184,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/623774165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":623774165},{"type":"v1","addr":"172.21.15.135:6837","nonce":623774165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7401,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/4195598823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":4195598823},{"type":"v1","addr":"172.21.15.53:6833","nonce":4195598823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-05-10T10:41:10.304 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 76 2024-05-10T10:41:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56c0076300 msgr2=0x7f56c00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56c0076300 0x7f56c00787c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f56e015bfa0 tx=0x7f56d8008040 comp rx=0 tx=0).stop 2024-05-10T10:41:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e00c85c0 msgr2=0x7f56e00c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e00c85c0 0x7f56e00c8a40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f56c8000c00 tx=0x7f56c8001120 comp rx=0 tx=0).stop 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 shutdown_connections 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f56c0076300 0x7f56c00787c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e01f1b30 0x7f56e01f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e00c85c0 0x7f56e00c8a40 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 --2- 172.21.15.53:0/37957266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f56cc004800 0x7f56e01f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 >> 172.21.15.53:0/37957266 conn(0x7f56e009ba20 msgr2=0x7f56e0161b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 shutdown_connections 2024-05-10T10:41:10.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.305+0000 7f56e55f2640 1 -- 172.21.15.53:0/37957266 wait complete. 2024-05-10T10:41:10.317 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:41:10.318 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T10:41:10.318 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:41:10.318 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:41:10.318 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:41:10.318 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T10:41:10.318 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:41:10.318 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:41:10.319 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:41:10.319 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T10:41:10.319 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:41:10.319 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:41:10.319 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:41:10.319 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T10:41:10.319 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:41:10.319 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:41:10.327 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:41:10.327 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:41:10.328 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:41:10.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:41:10.330 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:41:10.331 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:41:10.331 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:41:10.331 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:41:10.370 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:41:10.370 DEBUG:teuthology.parallel:result is None 2024-05-10T10:41:10.371 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:41:10.371 DEBUG:teuthology.parallel:result is None 2024-05-10T10:41:10.373 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:41:10.373 DEBUG:teuthology.parallel:result is None 2024-05-10T10:41:10.374 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:41:10.374 DEBUG:teuthology.parallel:result is None 2024-05-10T10:41:10.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T10:41:10.448 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:41:10.449 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:41:10.458 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:41:10.458 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:41:10.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.483+0000 7fbf5b2b0640 1 Processor -- start 2024-05-10T10:41:10.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.483+0000 7fbf5b2b0640 1 -- start start 2024-05-10T10:41:10.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.484+0000 7fbf5b2b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf541626d0 0x7fbf5406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.484+0000 7fbf5b2b0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf540af680 0x7fbf540b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.484+0000 7fbf5b2b0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf54163bd0 con 0x7fbf5406c800 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.484+0000 7fbf5b2b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbf5406cc60 con 0x7fbf541626d0 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.484+0000 7fbf5b2b0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf54162ab0 con 0x7fbf540af680 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.484+0000 7fbf5904c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf541626d0 0x7fbf5406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5904c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf541626d0 0x7fbf5406c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39672/0 (socket says 172.21.15.53:39672) 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5904c640 1 -- 172.21.15.53:0/684621678 learned_addr learned my addr 172.21.15.53:0/684621678 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5984d640 1 --2- 172.21.15.53:0/684621678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf540af680 0x7fbf540b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5904c640 1 -- 172.21.15.53:0/684621678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf540af680 msgr2=0x7fbf540b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5904c640 1 --2- 172.21.15.53:0/684621678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf540af680 0x7fbf540b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5904c640 1 -- 172.21.15.53:0/684621678 >> v1:172.21.15.135:6789/0 conn(0x7fbf5406c800 legacy=0x7fbf540aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:10.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5904c640 1 -- 172.21.15.53:0/684621678 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf54162e10 con 0x7fbf541626d0 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.485+0000 7fbf5904c640 1 --2- 172.21.15.53:0/684621678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf541626d0 0x7fbf5406c2c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbf500098a0 tx=0x7fbf5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daad880302c64aaa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf4bfff640 1 -- 172.21.15.53:0/684621678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50037070 con 0x7fbf541626d0 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf4bfff640 1 -- 172.21.15.53:0/684621678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf50032440 con 0x7fbf541626d0 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf4bfff640 1 -- 172.21.15.53:0/684621678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50032810 con 0x7fbf541626d0 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/684621678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf541626d0 msgr2=0x7fbf5406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/684621678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf541626d0 0x7fbf5406c2c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbf500098a0 tx=0x7fbf5002a1a0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/684621678 shutdown_connections 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/684621678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf540af680 0x7fbf540b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/684621678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf541626d0 0x7fbf5406c2c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/684621678 >> 172.21.15.53:0/684621678 conn(0x7fbf540d1d70 msgr2=0x7fbf540d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.486+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/684621678 shutdown_connections 2024-05-10T10:41:10.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.487+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/684621678 wait complete. 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.488+0000 7fbf5b2b0640 1 Processor -- start 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.488+0000 7fbf5b2b0640 1 -- start start 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5b2b0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf5406c800 0x7fbf5414d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5b2b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf540af680 0x7fbf540c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5b2b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf541626d0 0x7fbf540c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5b2b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf541636b0 con 0x7fbf540af680 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5b2b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf541633d0 con 0x7fbf541626d0 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf540af680 0x7fbf540c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5b2b0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbf540afc10 con 0x7fbf5406c800 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf540af680 0x7fbf540c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39686/0 (socket says 172.21.15.53:39686) 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 -- 172.21.15.53:0/2738547927 learned_addr learned my addr 172.21.15.53:0/2738547927 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5904c640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf5406c800 0x7fbf5414d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5984d640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf541626d0 0x7fbf540c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 -- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf5406c800 msgr2=0x7fbf5414d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf5406c800 0x7fbf5414d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 -- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf541626d0 msgr2=0x7fbf540c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf541626d0 0x7fbf540c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5884b640 1 -- 172.21.15.53:0/2738547927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf540c6eb0 con 0x7fbf540af680 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.489+0000 7fbf5984d640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf541626d0 0x7fbf540c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.490+0000 7fbf5884b640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf540af680 0x7fbf540c9620 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400c430 tx=0x7fbf4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.490+0000 7fbf49ffb640 1 -- 172.21.15.53:0/2738547927 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44011790 con 0x7fbf540af680 2024-05-10T10:41:10.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.490+0000 7fbf49ffb640 1 -- 172.21.15.53:0/2738547927 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf44017070 con 0x7fbf540af680 2024-05-10T10:41:10.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.490+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf540c7190 con 0x7fbf540af680 2024-05-10T10:41:10.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.490+0000 7fbf49ffb640 1 -- 172.21.15.53:0/2738547927 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44011b70 con 0x7fbf540af680 2024-05-10T10:41:10.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.490+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf540c7750 con 0x7fbf540af680 2024-05-10T10:41:10.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.491+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf540693f0 con 0x7fbf540af680 2024-05-10T10:41:10.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.491+0000 7fbf49ffb640 1 -- 172.21.15.53:0/2738547927 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf44011d10 con 0x7fbf540af680 2024-05-10T10:41:10.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.492+0000 7fbf49ffb640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf380762d0 0x7fbf38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:10.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.492+0000 7fbf49ffb640 1 -- 172.21.15.53:0/2738547927 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5507+0+0 (secure 0 0 0) 0x7fbf440973c0 con 0x7fbf540af680 2024-05-10T10:41:10.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.492+0000 7fbf5904c640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf380762d0 0x7fbf38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:10.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.493+0000 7fbf5904c640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf380762d0 0x7fbf38078790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf50034000 tx=0x7fbf5002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:10.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.496+0000 7fbf49ffb640 1 -- 172.21.15.53:0/2738547927 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbf44060d40 con 0x7fbf540af680 2024-05-10T10:41:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:10.700+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbf540b56b0 con 0x7fbf540af680 2024-05-10T10:41:11.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.745+0000 7fbf49ffb640 1 -- 172.21.15.53:0/2738547927 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7fbf44060640 con 0x7fbf540af680 2024-05-10T10:41:11.747 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2024-05-10T10:41:11.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf380762d0 msgr2=0x7fbf38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf380762d0 0x7fbf38078790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf50034000 tx=0x7fbf5002e000 comp rx=0 tx=0).stop 2024-05-10T10:41:11.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf540af680 msgr2=0x7fbf540c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf540af680 0x7fbf540c9620 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400c430 tx=0x7fbf4400c8f0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 shutdown_connections 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbf380762d0 0x7fbf38078790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf541626d0 0x7fbf540c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf540af680 0x7fbf540c9620 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.748+0000 7fbf5b2b0640 1 --2- 172.21.15.53:0/2738547927 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf5406c800 0x7fbf5414d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.749+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 >> 172.21.15.53:0/2738547927 conn(0x7fbf540d1d70 msgr2=0x7fbf5406ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.749+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 shutdown_connections 2024-05-10T10:41:11.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.749+0000 7fbf5b2b0640 1 -- 172.21.15.53:0/2738547927 wait complete. 2024-05-10T10:41:11.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T10:41:11.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.872+0000 7f64fc6a9640 1 Processor -- start 2024-05-10T10:41:11.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.872+0000 7f64fc6a9640 1 -- start start 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64fc6a9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64f4070bd0 0x7f64f415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64fc6a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f4160190 con 0x7f64f406c990 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64fc6a9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f4160470 con 0x7f64f415f620 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64fc6a9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f64f406cd70 con 0x7f64f4070bd0 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64fa445640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f64f406c990 0x7f64f4163660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55762/0 (socket says 172.21.15.53:55762) 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64fa445640 1 -- 172.21.15.53:0/1943620269 learned_addr learned my addr 172.21.15.53:0/1943620269 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64fac46640 1 --2- 172.21.15.53:0/1943620269 >> v2:172.21.15.135:3300/0 conn(0x7f64f4070bd0 0x7f64f415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156193486 0 0) 0x7f64f4160190 con 0x7f64f406c990 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.873+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e0003520 con 0x7f64f406c990 2024-05-10T10:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2906129804 0 0) 0x7f64e0003520 con 0x7f64f406c990 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 >> v2:172.21.15.135:3300/0 conn(0x7f64f4070bd0 msgr2=0x7f64f415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 --2- 172.21.15.53:0/1943620269 >> v2:172.21.15.135:3300/0 conn(0x7f64f4070bd0 0x7f64f415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 >> v1:172.21.15.135:6789/0 conn(0x7f64f415f620 legacy=0x7f64f40704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f4162e60 con 0x7f64f406c990 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f64f0001670 con 0x7f64f406c990 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 >> v1:172.21.15.53:6789/0 conn(0x7f64f406c990 legacy=0x7f64f4163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e00039b0 0x7f64e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00051c0 0x7f64e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64f406cd70 con 0x7f64e0004800 2024-05-10T10:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64e000c060 con 0x7f64e00051c0 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f64e000c260 con 0x7f64e00039b0 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9c44640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64fa445640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e00039b0 0x7f64e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64fac46640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00051c0 0x7f64e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9c44640 1 -- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e00039b0 msgr2=0x7f64e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9c44640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e00039b0 0x7f64e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9c44640 1 -- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00051c0 msgr2=0x7f64e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9c44640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00051c0 0x7f64e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64f9c44640 1 -- 172.21.15.53:0/1943620269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e0003520 con 0x7f64e0004800 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64fac46640 1 -- 172.21.15.53:0/1943620269 reap_dead start 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.874+0000 7f64fa445640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e00039b0 0x7f64e0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.875+0000 7f64f9c44640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f64e4001750 tx=0x7f64e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23f09c5283b558f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.875+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4001370 con 0x7f64e0004800 2024-05-10T10:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.875+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e40052d0 con 0x7f64e0004800 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.875+0000 7f64f9443640 1 -- 172.21.15.53:0/1943620269 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e402abc0 con 0x7f64e0004800 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.876+0000 7f64fc6a9640 1 -- 172.21.15.53:0/1943620269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e0004800 msgr2=0x7f64e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.876+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f64e4001750 tx=0x7f64e0005830 comp rx=0 tx=0).stop 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.876+0000 7f64fc6a9640 1 -- 172.21.15.53:0/1943620269 shutdown_connections 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.876+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/1943620269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.876+0000 7f64fc6a9640 1 -- 172.21.15.53:0/1943620269 >> 172.21.15.53:0/1943620269 conn(0x7f64f40b3980 msgr2=0x7f64f40b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.876+0000 7f64fc6a9640 1 -- 172.21.15.53:0/1943620269 shutdown_connections 2024-05-10T10:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.876+0000 7f64fc6a9640 1 -- 172.21.15.53:0/1943620269 wait complete. 2024-05-10T10:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.877+0000 7f64fc6a9640 1 Processor -- start 2024-05-10T10:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.877+0000 7f64fc6a9640 1 -- start start 2024-05-10T10:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fc6a9640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e0004800 0x7f64f414c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fc6a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f415c2d0 0x7f64f40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fc6a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f415e7b0 0x7f64f415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fc6a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64f415fa80 con 0x7f64f415e7b0 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fc6a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4073e70 con 0x7f64f415c2d0 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fc6a9640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f64f415fc00 con 0x7f64e0004800 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f415e7b0 0x7f64f415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f415e7b0 0x7f64f415ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39702/0 (socket says 172.21.15.53:39702) 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 -- 172.21.15.53:0/3939957937 learned_addr learned my addr 172.21.15.53:0/3939957937 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fa445640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e0004800 0x7f64f414c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64f9c44640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f415c2d0 0x7f64f40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 -- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e0004800 msgr2=0x7f64f414c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e0004800 0x7f64f414c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 -- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f415c2d0 msgr2=0x7f64f40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f415c2d0 0x7f64f40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 -- 172.21.15.53:0/3939957937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40b5d70 con 0x7f64f415e7b0 2024-05-10T10:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.878+0000 7f64fac46640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f415e7b0 0x7f64f415ec30 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f64f406fd50 tx=0x7f64ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.879+0000 7f64eaffd640 1 -- 172.21.15.53:0/3939957937 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011e10 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.879+0000 7f64eaffd640 1 -- 172.21.15.53:0/3939957937 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec01b070 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.879+0000 7f64eaffd640 1 -- 172.21.15.53:0/3939957937 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec003990 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.879+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f40af6d0 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.879+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f40afc90 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.880+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f415f170 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.880+0000 7f64eaffd640 1 -- 172.21.15.53:0/3939957937 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f64ec005d10 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.881+0000 7f64eaffd640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64d8076300 0x7f64d80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.881+0000 7f64eaffd640 1 -- 172.21.15.53:0/3939957937 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5249+0+0 (secure 0 0 0) 0x7f64ec097850 con 0x7f64f415e7b0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.881+0000 7f64fa445640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64d8076300 0x7f64d80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:11.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.881+0000 7f64fa445640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64d8076300 0x7f64d80787c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001eb0 tx=0x7f64f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:11.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:11.885+0000 7f64eaffd640 1 -- 172.21.15.53:0/3939957937 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f64ec061370 con 0x7f64f415e7b0 2024-05-10T10:41:12.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.150+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f64f40b5f40 con 0x7f64f415e7b0 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.150+0000 7f64eaffd640 1 -- 172.21.15.53:0/3939957937 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f64ec067020 con 0x7f64f415e7b0 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg==", 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.152 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ==", 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:41:12.153 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA==", 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew==", 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2024-05-10T10:41:12.155 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA==", 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA==", 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.156 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw==", 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2024-05-10T10:41:12.157 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA==", 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g==", 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.158 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A==", 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2024-05-10T10:41:12.159 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA==", 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg==", 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg==", 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g==", 2024-05-10T10:41:12.161 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw==", 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:41:12.162 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw==", 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1mKY2tIhAAzPSpOQLOFy1GEKdl5bPBEw==", 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw==", 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.164 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ==", 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:41:12.166 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:41:12.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64d8076300 msgr2=0x7f64d80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64d8076300 0x7f64d80787c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001eb0 tx=0x7f64f0009040 comp rx=0 tx=0).stop 2024-05-10T10:41:12.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f415e7b0 msgr2=0x7f64f415ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f415e7b0 0x7f64f415ec30 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f64f406fd50 tx=0x7f64ec0115a0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 shutdown_connections 2024-05-10T10:41:12.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f64d8076300 0x7f64d80787c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f415e7b0 0x7f64f415ec30 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f415c2d0 0x7f64f40aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 --2- 172.21.15.53:0/3939957937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64e0004800 0x7f64f414c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.152+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 >> 172.21.15.53:0/3939957937 conn(0x7f64f40b3980 msgr2=0x7f64f40cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:12.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.153+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 shutdown_connections 2024-05-10T10:41:12.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.153+0000 7f64fc6a9640 1 -- 172.21.15.53:0/3939957937 wait complete. 2024-05-10T10:41:12.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:41:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.270+0000 7fd84ffda640 1 Processor -- start 2024-05-10T10:41:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.271+0000 7fd84ffda640 1 -- start start 2024-05-10T10:41:12.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.271+0000 7fd84ffda640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd84815f790 0x7fd8480cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.271+0000 7fd84ffda640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8480d30b0 con 0x7fd84815f3b0 2024-05-10T10:41:12.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.271+0000 7fd84ffda640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8480d3390 con 0x7fd8480d5d30 2024-05-10T10:41:12.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.271+0000 7fd84ffda640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd8480c8c80 con 0x7fd84815f790 2024-05-10T10:41:12.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84dd76640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd84815f3b0 0x7fd8480d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55774/0 (socket says 172.21.15.53:55774) 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84dd76640 1 -- 172.21.15.53:0/4078816263 learned_addr learned my addr 172.21.15.53:0/4078816263 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84d575640 1 --2- 172.21.15.53:0/4078816263 >> v2:172.21.15.135:3300/0 conn(0x7fd84815f790 0x7fd8480cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019651069 0 0) 0x7fd8480d30b0 con 0x7fd84815f3b0 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd834003520 con 0x7fd84815f3b0 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3438153688 0 0) 0x7fd834003520 con 0x7fd84815f3b0 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 >> v2:172.21.15.135:3300/0 conn(0x7fd84815f790 msgr2=0x7fd8480cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 --2- 172.21.15.53:0/4078816263 >> v2:172.21.15.135:3300/0 conn(0x7fd84815f790 0x7fd8480cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 >> v1:172.21.15.135:6789/0 conn(0x7fd8480d5d30 legacy=0x7fd8480d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8480c8f60 con 0x7fd84815f3b0 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fd8440019f0 con 0x7fd84815f3b0 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 >> v1:172.21.15.53:6789/0 conn(0x7fd84815f3b0 legacy=0x7fd8480d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd834003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd834004800 0x7fd834004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8340051c0 0x7fd83400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480c8c80 con 0x7fd8340039b0 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd83400c060 con 0x7fd834004800 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd83400c260 con 0x7fd8340051c0 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84dd76640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd834003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84d575640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd834004800 0x7fd834004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.272+0000 7fd84e577640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8340051c0 0x7fd83400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84dd76640 1 -- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8340051c0 msgr2=0x7fd83400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84dd76640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8340051c0 0x7fd83400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84dd76640 1 -- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd834004800 msgr2=0x7fd834004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84dd76640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd834004800 0x7fd834004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84dd76640 1 -- 172.21.15.53:0/4078816263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd834003520 con 0x7fd8340039b0 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84e577640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8340051c0 0x7fd83400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84e577640 1 -- 172.21.15.53:0/4078816263 reap_dead start 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84dd76640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd834003dd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd84400a840 tx=0x7fd834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6157422fca432ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.273+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84400a460 con 0x7fd8340039b0 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84402f070 con 0x7fd8340039b0 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84cd74640 1 -- 172.21.15.53:0/4078816263 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84402bcf0 con 0x7fd8340039b0 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84ffda640 1 -- 172.21.15.53:0/4078816263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 msgr2=0x7fd834003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84ffda640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd834003dd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd84400a840 tx=0x7fd834005830 comp rx=0 tx=0).stop 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84ffda640 1 -- 172.21.15.53:0/4078816263 shutdown_connections 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84ffda640 1 --2- 172.21.15.53:0/4078816263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd834003dd0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84ffda640 1 -- 172.21.15.53:0/4078816263 >> 172.21.15.53:0/4078816263 conn(0x7fd8480d5130 msgr2=0x7fd8480d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84ffda640 1 -- 172.21.15.53:0/4078816263 shutdown_connections 2024-05-10T10:41:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.274+0000 7fd84ffda640 1 -- 172.21.15.53:0/4078816263 wait complete. 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.275+0000 7fd84ffda640 1 Processor -- start 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.275+0000 7fd84ffda640 1 -- start start 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84ffda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd8481624c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84ffda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd848160bd0 0x7fd8480ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84ffda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8480bac00 0x7fd8480b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84ffda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480d3870 con 0x7fd8340039b0 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84ffda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480d36f0 con 0x7fd8480bac00 2024-05-10T10:41:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84ffda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd8480d39f0 con 0x7fd848160bd0 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84dd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd8481624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84dd76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd8481624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39724/0 (socket says 172.21.15.53:39724) 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84dd76640 1 -- 172.21.15.53:0/3293410578 learned_addr learned my addr 172.21.15.53:0/3293410578 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84d575640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd848160bd0 0x7fd8480ba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.276+0000 7fd84e577640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8480bac00 0x7fd8480b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84dd76640 1 -- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd848160bd0 msgr2=0x7fd8480ba6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84dd76640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd848160bd0 0x7fd8480ba6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84dd76640 1 -- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8480bac00 msgr2=0x7fd8480b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84dd76640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8480bac00 0x7fd8480b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84dd76640 1 -- 172.21.15.53:0/3293410578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8481f4dc0 con 0x7fd8340039b0 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84e577640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8480bac00 0x7fd8480b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84dd76640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd8481624c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd84400a840 tx=0x7fd8480d3280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd83e7fc640 1 -- 172.21.15.53:0/3293410578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844001fb0 con 0x7fd8340039b0 2024-05-10T10:41:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd83e7fc640 1 -- 172.21.15.53:0/3293410578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844001250 con 0x7fd8340039b0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd848161420 con 0x7fd8340039b0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd83e7fc640 1 -- 172.21.15.53:0/3293410578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844034870 con 0x7fd8340039b0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.277+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8481619e0 con 0x7fd8340039b0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.278+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84806a220 con 0x7fd8340039b0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.278+0000 7fd83e7fc640 1 -- 172.21.15.53:0/3293410578 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd844034a10 con 0x7fd8340039b0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.279+0000 7fd83e7fc640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd82c0762d0 0x7fd82c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.279+0000 7fd83e7fc640 1 -- 172.21.15.53:0/3293410578 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5249+0+0 (secure 0 0 0) 0x7fd8440b7b30 con 0x7fd8340039b0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.279+0000 7fd84d575640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd82c0762d0 0x7fd82c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.280+0000 7fd84d575640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd82c0762d0 0x7fd82c078790 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd848155800 tx=0x7fd838008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:12.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.283+0000 7fd83e7fc640 1 -- 172.21.15.53:0/3293410578 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd8440815d0 con 0x7fd8340039b0 2024-05-10T10:41:12.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.476+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd84806a220 con 0x7fd8340039b0 2024-05-10T10:41:12.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.476+0000 7fd83e7fc640 1 -- 172.21.15.53:0/3293410578 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7fd844080ed0 con 0x7fd8340039b0 2024-05-10T10:41:12.479 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:12.479 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:41:12.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:41:12.479 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:41:12.479 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:41:12.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:12.480 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:41:12.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd82c0762d0 msgr2=0x7fd82c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd82c0762d0 0x7fd82c078790 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd848155800 tx=0x7fd838008040 comp rx=0 tx=0).stop 2024-05-10T10:41:12.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 msgr2=0x7fd8481624c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd8481624c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd84400a840 tx=0x7fd8480d3280 comp rx=0 tx=0).stop 2024-05-10T10:41:12.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 shutdown_connections 2024-05-10T10:41:12.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd82c0762d0 0x7fd82c078790 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8480bac00 0x7fd8480b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd848160bd0 0x7fd8480ba6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 --2- 172.21.15.53:0/3293410578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8340039b0 0x7fd8481624c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 >> 172.21.15.53:0/3293410578 conn(0x7fd8480d5130 msgr2=0x7fd84806cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:12.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.478+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 shutdown_connections 2024-05-10T10:41:12.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.479+0000 7fd84ffda640 1 -- 172.21.15.53:0/3293410578 wait complete. 2024-05-10T10:41:12.491 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:41:12.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:41:12.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.597+0000 7f5b5cec4640 1 Processor -- start 2024-05-10T10:41:12.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.597+0000 7f5b5cec4640 1 -- start start 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5cec4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5b5815f600 0x7f5b580d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5cec4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5cec4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b580b46b0 con 0x7f5b580d0b90 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5cec4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5b580b4990 con 0x7f5b5815f600 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5cec4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5b580b4c70 con 0x7f5b580d3d20 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5b5815f600 0x7f5b580d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b55d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5b580d0b90 0x7f5b580d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55784/0 (socket says 172.21.15.53:55784) 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b55d9b640 1 -- 172.21.15.53:0/3620830118 learned_addr learned my addr 172.21.15.53:0/3620830118 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b56d9d640 1 --2- 172.21.15.53:0/3620830118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5659c640 1 -- 172.21.15.53:0/3620830118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 msgr2=0x7f5b580b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5659c640 1 --2- 172.21.15.53:0/3620830118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5659c640 1 -- 172.21.15.53:0/3620830118 >> v1:172.21.15.53:6789/0 conn(0x7f5b580d0b90 legacy=0x7f5b580d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5559a640 1 -- 172.21.15.53:0/3620830118 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 179572591 0 0) 0x7f5b580b46b0 con 0x7f5b580d0b90 2024-05-10T10:41:12.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b5659c640 1 -- 172.21.15.53:0/3620830118 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b580b4f50 con 0x7f5b5815f600 2024-05-10T10:41:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.598+0000 7f5b56d9d640 1 --2- 172.21.15.53:0/3620830118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.599+0000 7f5b5659c640 1 --2- 172.21.15.53:0/3620830118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5815f600 0x7f5b580d0650 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f5b480097a0 tx=0x7f5b4802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a216c36a00966e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.599+0000 7f5b5559a640 1 -- 172.21.15.53:0/3620830118 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48037070 con 0x7f5b5815f600 2024-05-10T10:41:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.599+0000 7f5b5559a640 1 -- 172.21.15.53:0/3620830118 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4802ae50 con 0x7f5b5815f600 2024-05-10T10:41:12.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.599+0000 7f5b5559a640 1 -- 172.21.15.53:0/3620830118 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48032660 con 0x7f5b5815f600 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3620830118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5815f600 msgr2=0x7f5b580d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3620830118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5815f600 0x7f5b580d0650 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f5b480097a0 tx=0x7f5b4802a100 comp rx=0 tx=0).stop 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3620830118 shutdown_connections 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3620830118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3620830118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b5815f600 0x7f5b580d0650 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3620830118 >> 172.21.15.53:0/3620830118 conn(0x7f5b5809e390 msgr2=0x7f5b580d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3620830118 shutdown_connections 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.600+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3620830118 wait complete. 2024-05-10T10:41:12.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.601+0000 7f5b5cec4640 1 Processor -- start 2024-05-10T10:41:12.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.601+0000 7f5b5cec4640 1 -- start start 2024-05-10T10:41:12.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5cec4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b580d0b90 0x7f5b580cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5cec4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5cec4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b5815f600 0x7f5b580b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5cec4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580b5e10 con 0x7f5b580d0b90 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5cec4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580b5c90 con 0x7f5b5815f600 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5cec4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5b580b5f90 con 0x7f5b580d3d20 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b580d0b90 0x7f5b580cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b580d0b90 0x7f5b580cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39740/0 (socket says 172.21.15.53:39740) 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 -- 172.21.15.53:0/3416196269 learned_addr learned my addr 172.21.15.53:0/3416196269 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b56d9d640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b5815f600 0x7f5b580b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b55d9b640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 -- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 msgr2=0x7f5b580b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 -- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b5815f600 msgr2=0x7f5b580b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b5815f600 0x7f5b580b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.602+0000 7f5b5659c640 1 -- 172.21.15.53:0/3416196269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5815bbc0 con 0x7f5b580d0b90 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.603+0000 7f5b5659c640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b580d0b90 0x7f5b580cb390 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f5b480097a0 tx=0x7f5b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.603+0000 7f5b4effd640 1 -- 172.21.15.53:0/3416196269 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48037040 con 0x7f5b580d0b90 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.603+0000 7f5b4effd640 1 -- 172.21.15.53:0/3416196269 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b48004030 con 0x7f5b580d0b90 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.603+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b580ba790 con 0x7f5b580d0b90 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.603+0000 7f5b4effd640 1 -- 172.21.15.53:0/3416196269 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48004420 con 0x7f5b580d0b90 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.603+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b580bad50 con 0x7f5b580d0b90 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.604+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b580b46b0 con 0x7f5b580d0b90 2024-05-10T10:41:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.604+0000 7f5b4effd640 1 -- 172.21.15.53:0/3416196269 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b48003610 con 0x7f5b580d0b90 2024-05-10T10:41:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.605+0000 7f5b4effd640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b380762d0 0x7f5b38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.605+0000 7f5b4effd640 1 -- 172.21.15.53:0/3416196269 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5249+0+0 (secure 0 0 0) 0x7f5b480b80b0 con 0x7f5b580d0b90 2024-05-10T10:41:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.605+0000 7f5b55d9b640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b380762d0 0x7f5b38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:12.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.606+0000 7f5b55d9b640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b380762d0 0x7f5b38078790 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5b5814c210 tx=0x7f5b40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:12.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.609+0000 7f5b4effd640 1 -- 172.21.15.53:0/3416196269 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5b580b46b0 con 0x7f5b580d0b90 2024-05-10T10:41:12.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:12.800+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5b580cd150 con 0x7f5b580d0b90 2024-05-10T10:41:13.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.748+0000 7f5b4effd640 1 -- 172.21.15.53:0/3416196269 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7f5b4804c070 con 0x7f5b580d0b90 2024-05-10T10:41:13.750 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:41:13.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b380762d0 msgr2=0x7f5b38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b380762d0 0x7f5b38078790 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5b5814c210 tx=0x7f5b40009040 comp rx=0 tx=0).stop 2024-05-10T10:41:13.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b580d0b90 msgr2=0x7f5b580cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b580d0b90 0x7f5b580cb390 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f5b480097a0 tx=0x7f5b480046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 shutdown_connections 2024-05-10T10:41:13.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5b380762d0 0x7f5b38078790 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b5815f600 0x7f5b580b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b580d3d20 0x7f5b580b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 --2- 172.21.15.53:0/3416196269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b580d0b90 0x7f5b580cb390 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 >> 172.21.15.53:0/3416196269 conn(0x7f5b5809e390 msgr2=0x7f5b580b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 shutdown_connections 2024-05-10T10:41:13.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.751+0000 7f5b5cec4640 1 -- 172.21.15.53:0/3416196269 wait complete. 2024-05-10T10:41:13.770 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:41:13.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.886+0000 7f535f76a640 1 Processor -- start 2024-05-10T10:41:13.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.886+0000 7f535f76a640 1 -- start start 2024-05-10T10:41:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f535f76a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5358160530 0x7f535815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f535f76a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f535815cfb0 con 0x7f535806f030 2024-05-10T10:41:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f535f76a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f535815d290 con 0x7f535815c9e0 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f535f76a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f53580ccb60 con 0x7f5358160530 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f535d506640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f535806f030 0x7f535806f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46328/0 (socket says 172.21.15.53:46328) 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f535d506640 1 -- 172.21.15.53:0/515856630 learned_addr learned my addr 172.21.15.53:0/515856630 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f535cd05640 1 --2- 172.21.15.53:0/515856630 >> v2:172.21.15.135:3300/0 conn(0x7f5358160530 0x7f535815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035721872 0 0) 0x7f535815cfb0 con 0x7f535806f030 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.887+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5344003520 con 0x7f535806f030 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4183622738 0 0) 0x7f5344003520 con 0x7f535806f030 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 >> v2:172.21.15.135:3300/0 conn(0x7f5358160530 msgr2=0x7f535815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 --2- 172.21.15.53:0/515856630 >> v2:172.21.15.135:3300/0 conn(0x7f5358160530 0x7f535815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 >> v1:172.21.15.135:6789/0 conn(0x7f535815c9e0 legacy=0x7f53580cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535815fd30 con 0x7f535806f030 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f5354001670 con 0x7f535806f030 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 >> v1:172.21.15.53:6789/0 conn(0x7f535806f030 legacy=0x7f535806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53440039b0 0x7f5344003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5344004800 0x7f5344004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53440051c0 0x7f534400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f534400c030 con 0x7f5344004800 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f534400c1e0 con 0x7f53440039b0 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f534400c3e0 con 0x7f53440051c0 2024-05-10T10:41:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535cd05640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5344004800 0x7f5344004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535d506640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53440039b0 0x7f5344003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535dd07640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53440051c0 0x7f534400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535cd05640 1 -- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53440051c0 msgr2=0x7f534400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535cd05640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53440051c0 0x7f534400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535cd05640 1 -- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53440039b0 msgr2=0x7f5344003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535cd05640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53440039b0 0x7f5344003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535cd05640 1 -- 172.21.15.53:0/515856630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5344003520 con 0x7f5344004800 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.888+0000 7f535dd07640 1 -- 172.21.15.53:0/515856630 reap_dead start 2024-05-10T10:41:13.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.889+0000 7f535cd05640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5344004800 0x7f5344004c80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5348009ac0 tx=0x7f5344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f656443559fe4a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.889+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348036070 con 0x7f5344004800 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.889+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5348031400 con 0x7f5344004800 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.889+0000 7f534ffff640 1 -- 172.21.15.53:0/515856630 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348009690 con 0x7f5344004800 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.890+0000 7f535f76a640 1 -- 172.21.15.53:0/515856630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5344004800 msgr2=0x7f5344004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.890+0000 7f535f76a640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5344004800 0x7f5344004c80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5348009ac0 tx=0x7f5344005830 comp rx=0 tx=0).stop 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.890+0000 7f535f76a640 1 -- 172.21.15.53:0/515856630 shutdown_connections 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.890+0000 7f535f76a640 1 --2- 172.21.15.53:0/515856630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5344004800 0x7f5344004c80 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.890+0000 7f535f76a640 1 -- 172.21.15.53:0/515856630 >> 172.21.15.53:0/515856630 conn(0x7f53580d0c90 msgr2=0x7f53580d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:13.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.890+0000 7f535f76a640 1 -- 172.21.15.53:0/515856630 shutdown_connections 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.890+0000 7f535f76a640 1 -- 172.21.15.53:0/515856630 wait complete. 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.891+0000 7f535f76a640 1 Processor -- start 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.891+0000 7f535f76a640 1 -- start start 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535f76a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5344004800 0x7f5358155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535f76a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53580b0b60 0x7f53580b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535f76a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f535806df20 0x7f535806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535f76a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f535815fa40 con 0x7f53580b0b60 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535f76a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5358160990 con 0x7f5344004800 2024-05-10T10:41:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535f76a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f53580cb910 con 0x7f535806df20 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53580b0b60 0x7f53580b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53580b0b60 0x7f53580b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50874/0 (socket says 172.21.15.53:50874) 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 -- 172.21.15.53:0/1302198435 learned_addr learned my addr 172.21.15.53:0/1302198435 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535d506640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5344004800 0x7f5358155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535dd07640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f535806df20 0x7f535806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 -- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f535806df20 msgr2=0x7f535806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f535806df20 0x7f535806e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 -- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5344004800 msgr2=0x7f5358155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5344004800 0x7f5358155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 -- 172.21.15.53:0/1302198435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53580b2720 con 0x7f53580b0b60 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.892+0000 7f535cd05640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53580b0b60 0x7f53580b1fb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f534802abb0 tx=0x7f5348004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.893+0000 7f534dffb640 1 -- 172.21.15.53:0/1302198435 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534803f020 con 0x7f53580b0b60 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.893+0000 7f534dffb640 1 -- 172.21.15.53:0/1302198435 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5348037040 con 0x7f53580b0b60 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.893+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53580b2a00 con 0x7f53580b0b60 2024-05-10T10:41:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.893+0000 7f534dffb640 1 -- 172.21.15.53:0/1302198435 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348036070 con 0x7f53580b0b60 2024-05-10T10:41:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.893+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53580b2fc0 con 0x7f53580b0b60 2024-05-10T10:41:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.894+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f535806e8e0 con 0x7f53580b0b60 2024-05-10T10:41:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.895+0000 7f534dffb640 1 -- 172.21.15.53:0/1302198435 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5348042020 con 0x7f53580b0b60 2024-05-10T10:41:13.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.895+0000 7f534dffb640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f533c0762d0 0x7f533c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.895+0000 7f534dffb640 1 -- 172.21.15.53:0/1302198435 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5619+0+0 (secure 0 0 0) 0x7f53480b79e0 con 0x7f53580b0b60 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.896+0000 7f535d506640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f533c0762d0 0x7f533c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.896+0000 7f535d506640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f533c0762d0 0x7f533c078790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f53580b6780 tx=0x7f5354000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:13.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:13.899+0000 7f534dffb640 1 -- 172.21.15.53:0/1302198435 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5348049030 con 0x7f53580b0b60 2024-05-10T10:41:14.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.090+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f535806a220 con 0x7f53580b0b60 2024-05-10T10:41:14.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.754+0000 7f534dffb640 1 -- 172.21.15.53:0/1302198435 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7f5348089380 con 0x7f53580b0b60 2024-05-10T10:41:14.756 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:41:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f533c0762d0 msgr2=0x7f533c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f533c0762d0 0x7f533c078790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f53580b6780 tx=0x7f5354000ed0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53580b0b60 msgr2=0x7f53580b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:14.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53580b0b60 0x7f53580b1fb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f534802abb0 tx=0x7f5348004730 comp rx=0 tx=0).stop 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 shutdown_connections 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f533c0762d0 0x7f533c078790 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f535806df20 0x7f535806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53580b0b60 0x7f53580b1fb0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 --2- 172.21.15.53:0/1302198435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5344004800 0x7f5358155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.757+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 >> 172.21.15.53:0/1302198435 conn(0x7f53580d0c90 msgr2=0x7f53580733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.758+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 shutdown_connections 2024-05-10T10:41:14.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.758+0000 7f535f76a640 1 -- 172.21.15.53:0/1302198435 wait complete. 2024-05-10T10:41:14.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:41:14.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.895+0000 7fa162628640 1 Processor -- start 2024-05-10T10:41:14.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.895+0000 7fa162628640 1 -- start start 2024-05-10T10:41:14.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.895+0000 7fa162628640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:14.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.895+0000 7fa162628640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa15c0d17f0 0x7fa15c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:14.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.895+0000 7fa162628640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c0d1ed0 con 0x7fa15c06e290 2024-05-10T10:41:14.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.895+0000 7fa162628640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa15c06e6f0 con 0x7fa15c0d17f0 2024-05-10T10:41:14.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.895+0000 7fa162628640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c0d1c80 con 0x7fa15c15f400 2024-05-10T10:41:14.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa160bc5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa15c0d17f0 0x7fa15c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:14.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15bfff640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa160bc5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa15c0d17f0 0x7fa15c0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:46608/0 (socket says 172.21.15.53:46608) 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa160bc5640 1 -- 172.21.15.53:0/813904099 learned_addr learned my addr 172.21.15.53:0/813904099 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15b7fe640 1 --1- 172.21.15.53:0/813904099 >> v1:172.21.15.135:6789/0 conn(0x7fa15c06e290 0x7fa15c0d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:34868/0 (socket says 172.21.15.53:34868) 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15affd640 1 -- 172.21.15.53:0/813904099 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125293774 0 0) 0x7fa15c0d1ed0 con 0x7fa15c06e290 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15affd640 1 -- 172.21.15.53:0/813904099 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa144003520 con 0x7fa15c06e290 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15bfff640 1 -- 172.21.15.53:0/813904099 >> v2:172.21.15.135:3300/0 conn(0x7fa15c0d17f0 msgr2=0x7fa15c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15bfff640 1 --2- 172.21.15.53:0/813904099 >> v2:172.21.15.135:3300/0 conn(0x7fa15c0d17f0 0x7fa15c0b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15bfff640 1 -- 172.21.15.53:0/813904099 >> v1:172.21.15.135:6789/0 conn(0x7fa15c06e290 legacy=0x7fa15c0d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa15bfff640 1 -- 172.21.15.53:0/813904099 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c0d21b0 con 0x7fa15c15f400 2024-05-10T10:41:14.899 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.896+0000 7fa160bc5640 1 --2- 172.21.15.53:0/813904099 >> v2:172.21.15.135:3300/0 conn(0x7fa15c0d17f0 0x7fa15c0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.897+0000 7fa15bfff640 1 --2- 172.21.15.53:0/813904099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c06dd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009790 tx=0x7fa14c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80181abbdca7fba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.897+0000 7fa15affd640 1 -- 172.21.15.53:0/813904099 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c037070 con 0x7fa15c15f400 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.897+0000 7fa15affd640 1 -- 172.21.15.53:0/813904099 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14c032440 con 0x7fa15c15f400 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.897+0000 7fa15affd640 1 -- 172.21.15.53:0/813904099 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0327f0 con 0x7fa15c15f400 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 -- 172.21.15.53:0/813904099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 msgr2=0x7fa15c06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 --2- 172.21.15.53:0/813904099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c06dd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009790 tx=0x7fa14c02a180 comp rx=0 tx=0).stop 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 -- 172.21.15.53:0/813904099 shutdown_connections 2024-05-10T10:41:14.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 --2- 172.21.15.53:0/813904099 >> v2:172.21.15.135:3300/0 conn(0x7fa15c0d17f0 0x7fa15c0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 --2- 172.21.15.53:0/813904099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c06dd50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 -- 172.21.15.53:0/813904099 >> 172.21.15.53:0/813904099 conn(0x7fa15c09ba20 msgr2=0x7fa15c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:14.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 -- 172.21.15.53:0/813904099 shutdown_connections 2024-05-10T10:41:14.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.898+0000 7fa162628640 1 -- 172.21.15.53:0/813904099 wait complete. 2024-05-10T10:41:14.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.899+0000 7fa162628640 1 Processor -- start 2024-05-10T10:41:14.902 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.899+0000 7fa162628640 1 -- start start 2024-05-10T10:41:14.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa162628640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa15c06e290 0x7fa15c0b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:14.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa162628640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c0d17f0 0x7fa15c156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa162628640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa162628640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c15f7e0 con 0x7fa15c06e290 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa162628640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c0d2770 con 0x7fa15c0d17f0 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa162628640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c0b30a0 con 0x7fa15c15f400 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa15c06e290 0x7fa15c0b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa15c06e290 0x7fa15c0b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50880/0 (socket says 172.21.15.53:50880) 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 -- 172.21.15.53:0/3064719635 learned_addr learned my addr 172.21.15.53:0/3064719635 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa160bc5640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15b7fe640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c0d17f0 0x7fa15c156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 -- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 msgr2=0x7fa15c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 -- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c0d17f0 msgr2=0x7fa15c156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c0d17f0 0x7fa15c156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:14.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.900+0000 7fa15bfff640 1 -- 172.21.15.53:0/3064719635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c157ba0 con 0x7fa15c06e290 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.901+0000 7fa15bfff640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa15c06e290 0x7fa15c0b8490 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa14c02a1c0 tx=0x7fa14c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.901+0000 7fa158ff9640 1 -- 172.21.15.53:0/3064719635 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c041070 con 0x7fa15c06e290 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.901+0000 7fa158ff9640 1 -- 172.21.15.53:0/3064719635 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa14c03c870 con 0x7fa15c06e290 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.901+0000 7fa158ff9640 1 -- 172.21.15.53:0/3064719635 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c037040 con 0x7fa15c06e290 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.901+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c157e80 con 0x7fa15c06e290 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.901+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c158440 con 0x7fa15c06e290 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.902+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c0b3220 con 0x7fa15c06e290 2024-05-10T10:41:14.906 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.903+0000 7fa158ff9640 1 -- 172.21.15.53:0/3064719635 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa14c044070 con 0x7fa15c06e290 2024-05-10T10:41:14.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.903+0000 7fa158ff9640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa1480762d0 0x7fa148078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:14.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.903+0000 7fa158ff9640 1 -- 172.21.15.53:0/3064719635 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6001+0+0 (secure 0 0 0) 0x7fa14c0b8020 con 0x7fa15c06e290 2024-05-10T10:41:14.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.904+0000 7fa15b7fe640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa1480762d0 0x7fa148078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:14.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.904+0000 7fa15b7fe640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa1480762d0 0x7fa148078790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa15c145150 tx=0x7fa150007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:14.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:14.908+0000 7fa158ff9640 1 -- 172.21.15.53:0/3064719635 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa14c081210 con 0x7fa15c06e290 2024-05-10T10:41:15.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.138+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa15c14b080 con 0x7fa15c06e290 2024-05-10T10:41:15.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:41:15.512 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:41:15.757 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.755+0000 7fa158ff9640 1 -- 172.21.15.53:0/3064719635 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v78) ==== 174+0+0 (secure 0 0 0) 0x7fa14c085510 con 0x7fa15c06e290 2024-05-10T10:41:15.757 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 15 and data pool 16 2024-05-10T10:41:15.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.759+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa1480762d0 msgr2=0x7fa148078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:15.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.759+0000 7fa162628640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa1480762d0 0x7fa148078790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa15c145150 tx=0x7fa150007cf0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.759+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa15c06e290 msgr2=0x7fa15c0b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:15.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.759+0000 7fa162628640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa15c06e290 0x7fa15c0b8490 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa14c02a1c0 tx=0x7fa14c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.760+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 shutdown_connections 2024-05-10T10:41:15.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.760+0000 7fa162628640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa1480762d0 0x7fa148078790 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.760+0000 7fa162628640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa15c15f400 0x7fa15c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.760+0000 7fa162628640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c0d17f0 0x7fa15c156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.760+0000 7fa162628640 1 --2- 172.21.15.53:0/3064719635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa15c06e290 0x7fa15c0b8490 secure :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa14c02a1c0 tx=0x7fa14c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.760+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 >> 172.21.15.53:0/3064719635 conn(0x7fa15c09ba20 msgr2=0x7fa15c15bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.761+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 shutdown_connections 2024-05-10T10:41:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.761+0000 7fa162628640 1 -- 172.21.15.53:0/3064719635 wait complete. 2024-05-10T10:41:15.796 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:41:15.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.899+0000 7efd9ba1d640 1 Processor -- start 2024-05-10T10:41:15.901 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.899+0000 7efd9ba1d640 1 -- start start 2024-05-10T10:41:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd9ba1d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd9ba1d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd9415f790 0x7efd940d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd9ba1d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efd940d5d30 0x7efd940c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd9ba1d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7efd940d30b0 con 0x7efd9415f790 2024-05-10T10:41:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd9ba1d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7efd940d3390 con 0x7efd940d5d30 2024-05-10T10:41:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd9ba1d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efd940c8c80 con 0x7efd9415f3b0 2024-05-10T10:41:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd98fb8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd9415f790 0x7efd940d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd98fb8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd9415f790 0x7efd940d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50888/0 (socket says 172.21.15.53:50888) 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.900+0000 7efd98fb8640 1 -- 172.21.15.53:0/695685804 learned_addr learned my addr 172.21.15.53:0/695685804 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd997b9640 1 --2- 172.21.15.53:0/695685804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd99fba640 1 --2- 172.21.15.53:0/695685804 >> v2:172.21.15.135:3300/0 conn(0x7efd940d5d30 0x7efd940c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd98fb8640 1 -- 172.21.15.53:0/695685804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 msgr2=0x7efd940d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd98fb8640 1 --2- 172.21.15.53:0/695685804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd98fb8640 1 -- 172.21.15.53:0/695685804 >> v2:172.21.15.135:3300/0 conn(0x7efd940d5d30 msgr2=0x7efd940c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd98fb8640 1 --2- 172.21.15.53:0/695685804 >> v2:172.21.15.135:3300/0 conn(0x7efd940d5d30 0x7efd940c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd98fb8640 1 -- 172.21.15.53:0/695685804 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd940c8f60 con 0x7efd9415f790 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd99fba640 1 --2- 172.21.15.53:0/695685804 >> v2:172.21.15.135:3300/0 conn(0x7efd940d5d30 0x7efd940c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.901+0000 7efd98fb8640 1 --2- 172.21.15.53:0/695685804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd9415f790 0x7efd940d3c30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7efd940d36a0 tx=0x7efd9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b4d8bd11332bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd87fff640 1 -- 172.21.15.53:0/695685804 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90037070 con 0x7efd9415f790 2024-05-10T10:41:15.904 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd87fff640 1 -- 172.21.15.53:0/695685804 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90032440 con 0x7efd9415f790 2024-05-10T10:41:15.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd87fff640 1 -- 172.21.15.53:0/695685804 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90032810 con 0x7efd9415f790 2024-05-10T10:41:15.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 -- 172.21.15.53:0/695685804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd9415f790 msgr2=0x7efd940d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:15.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/695685804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd9415f790 0x7efd940d3c30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7efd940d36a0 tx=0x7efd9002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 -- 172.21.15.53:0/695685804 shutdown_connections 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/695685804 >> v2:172.21.15.135:3300/0 conn(0x7efd940d5d30 0x7efd940c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/695685804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd9415f790 0x7efd940d3c30 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/695685804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 -- 172.21.15.53:0/695685804 >> 172.21.15.53:0/695685804 conn(0x7efd940d5130 msgr2=0x7efd940d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 -- 172.21.15.53:0/695685804 shutdown_connections 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.902+0000 7efd9ba1d640 1 -- 172.21.15.53:0/695685804 wait complete. 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.903+0000 7efd9ba1d640 1 Processor -- start 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd9ba1d640 1 -- start start 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd9ba1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940d5d30 0x7efd940b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd9ba1d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd9ba1d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd940b1c90 0x7efd941534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd9ba1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efd940c9e20 con 0x7efd940d5d30 2024-05-10T10:41:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd9ba1d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd940c9ca0 con 0x7efd940b1c90 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd9ba1d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efd940c9fa0 con 0x7efd9415f3b0 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd997b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940d5d30 0x7efd940b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd997b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940d5d30 0x7efd940b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50904/0 (socket says 172.21.15.53:50904) 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd997b9640 1 -- 172.21.15.53:0/3220057724 learned_addr learned my addr 172.21.15.53:0/3220057724 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd99fba640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd940b1c90 0x7efd941534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.904+0000 7efd98fb8640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd997b9640 1 -- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 msgr2=0x7efd940cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd997b9640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd997b9640 1 -- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd940b1c90 msgr2=0x7efd941534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd997b9640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd940b1c90 0x7efd941534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd997b9640 1 -- 172.21.15.53:0/3220057724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd940cf3a0 con 0x7efd940d5d30 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd997b9640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940d5d30 0x7efd940b32a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d800 tx=0x7efd8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd85ffb640 1 -- 172.21.15.53:0/3220057724 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8800b6c0 con 0x7efd940d5d30 2024-05-10T10:41:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd85ffb640 1 -- 172.21.15.53:0/3220057724 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd88019070 con 0x7efd940d5d30 2024-05-10T10:41:15.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd940b2200 con 0x7efd940d5d30 2024-05-10T10:41:15.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd85ffb640 1 -- 172.21.15.53:0/3220057724 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8800ba80 con 0x7efd940d5d30 2024-05-10T10:41:15.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.905+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd940b27c0 con 0x7efd940d5d30 2024-05-10T10:41:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.907+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9406a220 con 0x7efd940d5d30 2024-05-10T10:41:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.907+0000 7efd85ffb640 1 -- 172.21.15.53:0/3220057724 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efd8800bc20 con 0x7efd940d5d30 2024-05-10T10:41:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.907+0000 7efd85ffb640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd780762d0 0x7efd78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.907+0000 7efd85ffb640 1 -- 172.21.15.53:0/3220057724 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6101+0+0 (secure 0 0 0) 0x7efd88097b00 con 0x7efd940d5d30 2024-05-10T10:41:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.907+0000 7efd98fb8640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd780762d0 0x7efd78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.908+0000 7efd98fb8640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd780762d0 0x7efd78078790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efd90034000 tx=0x7efd9002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:15.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:15.912+0000 7efd85ffb640 1 -- 172.21.15.53:0/3220057724 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7efd88060c80 con 0x7efd940d5d30 2024-05-10T10:41:16.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.123+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd9406d4a0 con 0x7efd940d5d30 2024-05-10T10:41:16.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.124+0000 7efd85ffb640 1 -- 172.21.15.53:0/3220057724 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16897 (secure 0 0 0) 0x7efd88064f80 con 0x7efd940d5d30 2024-05-10T10:41:16.126 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:16.126 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":102,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:41:15.140839+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:41:12.805545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T10:41:14.096305+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd780762d0 msgr2=0x7efd78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd780762d0 0x7efd78078790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efd90034000 tx=0x7efd9002e000 comp rx=0 tx=0).stop 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940d5d30 msgr2=0x7efd940b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940d5d30 0x7efd940b32a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d800 tx=0x7efd8800dcc0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 shutdown_connections 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efd780762d0 0x7efd78078790 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd940b1c90 0x7efd941534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd9415f3b0 0x7efd940cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 --2- 172.21.15.53:0/3220057724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd940d5d30 0x7efd940b32a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 >> 172.21.15.53:0/3220057724 conn(0x7efd940d5130 msgr2=0x7efd9406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:16.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 shutdown_connections 2024-05-10T10:41:16.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.126+0000 7efd9ba1d640 1 -- 172.21.15.53:0/3220057724 wait complete. 2024-05-10T10:41:16.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:41:16.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.242+0000 7f41ebf0a640 1 Processor -- start 2024-05-10T10:41:16.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.242+0000 7f41ebf0a640 1 -- start start 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41ebf0a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e40d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41ebf0a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f41e415f790 0x7f41e40d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41ebf0a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e40d30b0 con 0x7f41e40d5d30 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41ebf0a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f41e40d3390 con 0x7f41e415f790 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41ebf0a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41e40c8c80 con 0x7f41e415f3b0 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41ea4a7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f41e40d5d30 0x7f41e40c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46340/0 (socket says 172.21.15.53:46340) 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41ea4a7640 1 -- 172.21.15.53:0/1421090357 learned_addr learned my addr 172.21.15.53:0/1421090357 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:16.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41e94a5640 1 --2- 172.21.15.53:0/1421090357 >> v2:172.21.15.135:3300/0 conn(0x7f41e415f790 0x7f41e40d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41e9ca6640 1 --2- 172.21.15.53:0/1421090357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e40d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41e8ca4640 1 -- 172.21.15.53:0/1421090357 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386963018 0 0) 0x7f41e40d30b0 con 0x7f41e40d5d30 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41e8ca4640 1 -- 172.21.15.53:0/1421090357 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41d0003520 con 0x7f41e40d5d30 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41e9ca6640 1 -- 172.21.15.53:0/1421090357 >> v2:172.21.15.135:3300/0 conn(0x7f41e415f790 msgr2=0x7f41e40d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.243+0000 7f41e9ca6640 1 --2- 172.21.15.53:0/1421090357 >> v2:172.21.15.135:3300/0 conn(0x7f41e415f790 0x7f41e40d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.244+0000 7f41e9ca6640 1 -- 172.21.15.53:0/1421090357 >> v1:172.21.15.53:6789/0 conn(0x7f41e40d5d30 legacy=0x7f41e40c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.244+0000 7f41e9ca6640 1 -- 172.21.15.53:0/1421090357 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e40c8f60 con 0x7f41e415f3b0 2024-05-10T10:41:16.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.244+0000 7f41e94a5640 1 --2- 172.21.15.53:0/1421090357 >> v2:172.21.15.135:3300/0 conn(0x7f41e415f790 0x7f41e40d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:16.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.244+0000 7f41e9ca6640 1 --2- 172.21.15.53:0/1421090357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e40d36f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41e00097a0 tx=0x7f41e002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de76eba92901362 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:16.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41e8ca4640 1 -- 172.21.15.53:0/1421090357 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0037070 con 0x7f41e415f3b0 2024-05-10T10:41:16.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41e8ca4640 1 -- 172.21.15.53:0/1421090357 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e0032440 con 0x7f41e415f3b0 2024-05-10T10:41:16.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41e8ca4640 1 -- 172.21.15.53:0/1421090357 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0032810 con 0x7f41e415f3b0 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 -- 172.21.15.53:0/1421090357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 msgr2=0x7f41e40d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/1421090357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e40d36f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41e00097a0 tx=0x7f41e002a180 comp rx=0 tx=0).stop 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 -- 172.21.15.53:0/1421090357 shutdown_connections 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/1421090357 >> v2:172.21.15.135:3300/0 conn(0x7f41e415f790 0x7f41e40d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/1421090357 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e40d36f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 -- 172.21.15.53:0/1421090357 >> 172.21.15.53:0/1421090357 conn(0x7f41e40d5130 msgr2=0x7f41e40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 -- 172.21.15.53:0/1421090357 shutdown_connections 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.245+0000 7f41ebf0a640 1 -- 172.21.15.53:0/1421090357 wait complete. 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.246+0000 7f41ebf0a640 1 Processor -- start 2024-05-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.246+0000 7f41ebf0a640 1 -- start start 2024-05-10T10:41:16.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ebf0a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e40d5d30 0x7f41e40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ebf0a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e4157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ebf0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e41545a0 0x7f41e4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ebf0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40d0e60 con 0x7f41e41545a0 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ebf0a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40d0ce0 con 0x7f41e40d5d30 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ebf0a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41e40d0fe0 con 0x7f41e415f3b0 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ea4a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e41545a0 0x7f41e4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ea4a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e41545a0 0x7f41e4161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50914/0 (socket says 172.21.15.53:50914) 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41ea4a7640 1 -- 172.21.15.53:0/3345842120 learned_addr learned my addr 172.21.15.53:0/3345842120 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41e9ca6640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e40d5d30 0x7f41e40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.247+0000 7f41e94a5640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e4157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ea4a7640 1 -- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 msgr2=0x7f41e4157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ea4a7640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e4157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ea4a7640 1 -- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e40d5d30 msgr2=0x7f41e40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ea4a7640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e40d5d30 0x7f41e40cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ea4a7640 1 -- 172.21.15.53:0/3345842120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e40cc890 con 0x7f41e41545a0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ea4a7640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e41545a0 0x7f41e4161d50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00cbb0 tx=0x7f41dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41da7fc640 1 -- 172.21.15.53:0/3345842120 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc017070 con 0x7f41e41545a0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e41f4eb0 con 0x7f41e41545a0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41da7fc640 1 -- 172.21.15.53:0/3345842120 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc00ad40 con 0x7f41e41545a0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41da7fc640 1 -- 172.21.15.53:0/3345842120 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc012600 con 0x7f41e41545a0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.248+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e41f5470 con 0x7f41e41545a0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.249+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e406a220 con 0x7f41e41545a0 2024-05-10T10:41:16.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.250+0000 7f41da7fc640 1 -- 172.21.15.53:0/3345842120 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41dc01a070 con 0x7f41e41545a0 2024-05-10T10:41:16.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.251+0000 7f41da7fc640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41c80762d0 0x7f41c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.251+0000 7f41da7fc640 1 -- 172.21.15.53:0/3345842120 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6101+0+0 (secure 0 0 0) 0x7f41dc097480 con 0x7f41e41545a0 2024-05-10T10:41:16.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.251+0000 7f41e9ca6640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41c80762d0 0x7f41c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.251+0000 7f41e9ca6640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41c80762d0 0x7f41c8078790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f41e0034000 tx=0x7f41e002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:16.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.255+0000 7f41da7fc640 1 -- 172.21.15.53:0/3345842120 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f41dc060570 con 0x7f41e41545a0 2024-05-10T10:41:16.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.447+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41e40c8a10 con 0x7f41e41545a0 2024-05-10T10:41:16.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.448+0000 7f41da7fc640 1 -- 172.21.15.53:0/3345842120 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16897 (secure 0 0 0) 0x7f41dc064870 con 0x7f41e41545a0 2024-05-10T10:41:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:16.450 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":102,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:41:15.140839+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:41:12.805545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T10:41:14.096305+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:41:16.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.450+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41c80762d0 msgr2=0x7f41c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.450+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41c80762d0 0x7f41c8078790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f41e0034000 tx=0x7f41e002e000 comp rx=0 tx=0).stop 2024-05-10T10:41:16.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.450+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e41545a0 msgr2=0x7f41e4161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.450+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e41545a0 0x7f41e4161d50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00cbb0 tx=0x7f41dc00a430 comp rx=0 tx=0).stop 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 shutdown_connections 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41c80762d0 0x7f41c8078790 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41e41545a0 0x7f41e4161d50 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41e415f3b0 0x7f41e4157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 --2- 172.21.15.53:0/3345842120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41e40d5d30 0x7f41e40cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 >> 172.21.15.53:0/3345842120 conn(0x7f41e40d5130 msgr2=0x7f41e406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 shutdown_connections 2024-05-10T10:41:16.453 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.451+0000 7f41ebf0a640 1 -- 172.21.15.53:0/3345842120 wait complete. 2024-05-10T10:41:16.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:41:16.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.570+0000 7fa489620640 1 Processor -- start 2024-05-10T10:41:16.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.570+0000 7fa489620640 1 -- start start 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.570+0000 7fa489620640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4840d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.570+0000 7fa489620640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa484162ce0 0x7fa4840b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.570+0000 7fa489620640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4840d4190 con 0x7fa4840d3950 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.570+0000 7fa489620640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa4840cfc40 con 0x7fa484162ce0 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.570+0000 7fa489620640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa4840d3eb0 con 0x7fa484162900 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa4827fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa484162ce0 0x7fa4840b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4840d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa4837fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa4840d3950 0x7fa4840d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:34882/0 (socket says 172.21.15.53:34882) 2024-05-10T10:41:16.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa4837fe640 1 -- 172.21.15.53:0/1960319604 learned_addr learned my addr 172.21.15.53:0/1960319604 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:16.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa4827fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa484162ce0 0x7fa4840b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:46660/0 (socket says 172.21.15.53:46660) 2024-05-10T10:41:16.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4840d3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:48612/0 (socket says 172.21.15.53:48612) 2024-05-10T10:41:16.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa482ffd640 1 -- 172.21.15.53:0/1960319604 >> v2:172.21.15.135:3300/0 conn(0x7fa484162ce0 msgr2=0x7fa4840b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:16.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa482ffd640 1 --2- 172.21.15.53:0/1960319604 >> v2:172.21.15.135:3300/0 conn(0x7fa484162ce0 0x7fa4840b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa482ffd640 1 -- 172.21.15.53:0/1960319604 >> v1:172.21.15.135:6789/0 conn(0x7fa4840d3950 legacy=0x7fa4840d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa481ffb640 1 -- 172.21.15.53:0/1960319604 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 202978976 0 0) 0x7fa474001110 con 0x7fa4840d3950 2024-05-10T10:41:16.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa482ffd640 1 -- 172.21.15.53:0/1960319604 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840cfee0 con 0x7fa484162900 2024-05-10T10:41:16.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.571+0000 7fa4827fc640 1 --2- 172.21.15.53:0/1960319604 >> v2:172.21.15.135:3300/0 conn(0x7fa484162ce0 0x7fa4840b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:16.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.572+0000 7fa482ffd640 1 --2- 172.21.15.53:0/1960319604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4840d3410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009d90 tx=0x7fa47c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281591039f7161ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.572+0000 7fa481ffb640 1 -- 172.21.15.53:0/1960319604 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c031600 con 0x7fa484162900 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.572+0000 7fa481ffb640 1 -- 172.21.15.53:0/1960319604 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c0317a0 con 0x7fa484162900 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.572+0000 7fa481ffb640 1 -- 172.21.15.53:0/1960319604 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c031b70 con 0x7fa484162900 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 -- 172.21.15.53:0/1960319604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 msgr2=0x7fa4840d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 --2- 172.21.15.53:0/1960319604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4840d3410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009d90 tx=0x7fa47c02a5d0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 -- 172.21.15.53:0/1960319604 shutdown_connections 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 --2- 172.21.15.53:0/1960319604 >> v2:172.21.15.135:3300/0 conn(0x7fa484162ce0 0x7fa4840b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 --2- 172.21.15.53:0/1960319604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4840d3410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 -- 172.21.15.53:0/1960319604 >> 172.21.15.53:0/1960319604 conn(0x7fa48409e430 msgr2=0x7fa4840d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 -- 172.21.15.53:0/1960319604 shutdown_connections 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.573+0000 7fa489620640 1 -- 172.21.15.53:0/1960319604 wait complete. 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.574+0000 7fa489620640 1 Processor -- start 2024-05-10T10:41:16.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.574+0000 7fa489620640 1 -- start start 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa489620640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4840d3950 0x7fa48415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa489620640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4841f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa489620640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa484162ce0 0x7fa4840cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa489620640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840b4300 con 0x7fa4840d3950 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa489620640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840b4180 con 0x7fa484162ce0 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa489620640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa4840d10d0 con 0x7fa484162900 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4840d3950 0x7fa48415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4840d3950 0x7fa48415d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50924/0 (socket says 172.21.15.53:50924) 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa482ffd640 1 -- 172.21.15.53:0/3021907726 learned_addr learned my addr 172.21.15.53:0/3021907726 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa4827fc640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4841f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa4837fe640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa484162ce0 0x7fa4840cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa4827fc640 1 -- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa484162ce0 msgr2=0x7fa4840cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa4827fc640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa484162ce0 0x7fa4840cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa4827fc640 1 -- 172.21.15.53:0/3021907726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4840d3950 msgr2=0x7fa48415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa4827fc640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4840d3950 0x7fa48415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.575+0000 7fa4827fc640 1 -- 172.21.15.53:0/3021907726 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840b87a0 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.576+0000 7fa4827fc640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4841f5100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e930 tx=0x7fa46c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.576+0000 7fa47b7fe640 1 -- 172.21.15.53:0/3021907726 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c017070 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.576+0000 7fa47b7fe640 1 -- 172.21.15.53:0/3021907726 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c00ca00 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.576+0000 7fa47b7fe640 1 -- 172.21.15.53:0/3021907726 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00cdd0 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.576+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4840b8a80 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.576+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4840ba270 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.577+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48406a220 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.577+0000 7fa47b7fe640 1 -- 172.21.15.53:0/3021907726 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa46c01a070 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.578+0000 7fa47b7fe640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa4640762d0 0x7fa464078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.578+0000 7fa47b7fe640 1 -- 172.21.15.53:0/3021907726 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 6101+0+0 (secure 0 0 0) 0x7fa46c0975c0 con 0x7fa484162900 2024-05-10T10:41:16.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.578+0000 7fa482ffd640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa4640762d0 0x7fa464078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:16.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.579+0000 7fa482ffd640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa4640762d0 0x7fa464078790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009bd0 tx=0x7fa47c02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:16.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.583+0000 7fa47b7fe640 1 -- 172.21.15.53:0/3021907726 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa46c060730 con 0x7fa484162900 2024-05-10T10:41:16.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:16.813+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa48409e840 con 0x7fa484162900 2024-05-10T10:41:17.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.785+0000 7fa47b7fe640 1 -- 172.21.15.53:0/3021907726 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7fa46c064a30 con 0x7fa484162900 2024-05-10T10:41:17.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa4640762d0 msgr2=0x7fa464078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa4640762d0 0x7fa464078790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009bd0 tx=0x7fa47c02f000 comp rx=0 tx=0).stop 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 msgr2=0x7fa4841f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4841f5100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e930 tx=0x7fa46c00edf0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 shutdown_connections 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa4640762d0 0x7fa464078790 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa484162ce0 0x7fa4840cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa484162900 0x7fa4841f5100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 --2- 172.21.15.53:0/3021907726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4840d3950 0x7fa48415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 >> 172.21.15.53:0/3021907726 conn(0x7fa48409e430 msgr2=0x7fa4840b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 shutdown_connections 2024-05-10T10:41:17.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.788+0000 7fa489620640 1 -- 172.21.15.53:0/3021907726 wait complete. 2024-05-10T10:41:17.803 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T10:41:17.803 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:41:17.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.915+0000 7f4ecd5dd640 1 Processor -- start 2024-05-10T10:41:17.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.915+0000 7f4ecd5dd640 1 -- start start 2024-05-10T10:41:17.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ecd5dd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4ec8160870 0x7f4ec80cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ecd5dd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec8160c50 con 0x7f4ec80cfcc0 2024-05-10T10:41:17.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ecd5dd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec80d39d0 con 0x7f4ec80b3970 2024-05-10T10:41:17.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ecd5dd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec80d3cb0 con 0x7f4ec8160870 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ec67fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4ec80cfcc0 0x7f4ec80cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46356/0 (socket says 172.21.15.53:46356) 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ec67fc640 1 -- 172.21.15.53:0/4101121574 learned_addr learned my addr 172.21.15.53:0/4101121574 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ec6ffd640 1 --2- 172.21.15.53:0/4101121574 >> v2:172.21.15.135:3300/0 conn(0x7f4ec8160870 0x7f4ec80cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ec5ffb640 1 -- 172.21.15.53:0/4101121574 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586665996 0 0) 0x7f4ec8160c50 con 0x7f4ec80cfcc0 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.916+0000 7f4ec5ffb640 1 -- 172.21.15.53:0/4101121574 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb0003520 con 0x7f4ec80cfcc0 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.917+0000 7f4ec5ffb640 1 -- 172.21.15.53:0/4101121574 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2081680865 0 0) 0x7f4eb0003520 con 0x7f4ec80cfcc0 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.917+0000 7f4ec6ffd640 1 -- 172.21.15.53:0/4101121574 >> v1:172.21.15.135:6789/0 conn(0x7f4ec80b3970 legacy=0x7f4ec80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.917+0000 7f4ec6ffd640 1 -- 172.21.15.53:0/4101121574 >> v1:172.21.15.53:6789/0 conn(0x7f4ec80cfcc0 legacy=0x7f4ec80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.917+0000 7f4ec6ffd640 1 -- 172.21.15.53:0/4101121574 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec80d1770 con 0x7f4ec8160870 2024-05-10T10:41:17.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.917+0000 7f4ec6ffd640 1 --2- 172.21.15.53:0/4101121574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec8160870 0x7f4ec80cf780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80097a0 tx=0x7f4eb802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc976a9d3f56dcca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ec5ffb640 1 -- 172.21.15.53:0/4101121574 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8037070 con 0x7f4ec8160870 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ec5ffb640 1 -- 172.21.15.53:0/4101121574 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb802aad0 con 0x7f4ec8160870 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ec5ffb640 1 -- 172.21.15.53:0/4101121574 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc001670 con 0x7f4ec8160870 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/4101121574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec8160870 msgr2=0x7f4ec80cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/4101121574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec8160870 0x7f4ec80cf780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80097a0 tx=0x7f4eb802a040 comp rx=0 tx=0).stop 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/4101121574 shutdown_connections 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/4101121574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec8160870 0x7f4ec80cf780 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/4101121574 >> 172.21.15.53:0/4101121574 conn(0x7f4ec80d2eb0 msgr2=0x7f4ec80d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/4101121574 shutdown_connections 2024-05-10T10:41:17.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.918+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/4101121574 wait complete. 2024-05-10T10:41:17.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 Processor -- start 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 -- start start 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ec80b3970 0x7f4ec815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec80cfcc0 0x7f4ec81542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8160870 0x7f4ec80c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80d1460 con 0x7f4ec8160870 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80d1d30 con 0x7f4ec80cfcc0 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.920+0000 7f4ecd5dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec80b3d80 con 0x7f4ec80b3970 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8160870 0x7f4ec80c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8160870 0x7f4ec80c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50938/0 (socket says 172.21.15.53:50938) 2024-05-10T10:41:17.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 -- 172.21.15.53:0/2038374238 learned_addr learned my addr 172.21.15.53:0/2038374238 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec6ffd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ec80b3970 0x7f4ec815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec67fc640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec80cfcc0 0x7f4ec81542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 -- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ec80b3970 msgr2=0x7f4ec815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ec80b3970 0x7f4ec815d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 -- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec80cfcc0 msgr2=0x7f4ec81542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec80cfcc0 0x7f4ec81542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 -- 172.21.15.53:0/2038374238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec80c8f40 con 0x7f4ec8160870 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec6ffd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ec80b3970 0x7f4ec815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ec77fe640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8160870 0x7f4ec80c6730 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000c8c0 tx=0x7f4ec000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4eaf7fe640 1 -- 172.21.15.53:0/2038374238 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0003da0 con 0x7f4ec8160870 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4eaf7fe640 1 -- 172.21.15.53:0/2038374238 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0005070 con 0x7f4ec8160870 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4eaf7fe640 1 -- 172.21.15.53:0/2038374238 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0005460 con 0x7f4ec8160870 2024-05-10T10:41:17.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec80c9d60 con 0x7f4ec8160870 2024-05-10T10:41:17.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.921+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec80c94c0 con 0x7f4ec8160870 2024-05-10T10:41:17.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.922+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec806a220 con 0x7f4ec8160870 2024-05-10T10:41:17.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.923+0000 7f4eaf7fe640 1 -- 172.21.15.53:0/2038374238 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4ec0005600 con 0x7f4ec8160870 2024-05-10T10:41:17.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.924+0000 7f4eaf7fe640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4eb40762d0 0x7f4eb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:17.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.924+0000 7f4eaf7fe640 1 -- 172.21.15.53:0/2038374238 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f4ec00977a0 con 0x7f4ec8160870 2024-05-10T10:41:17.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.924+0000 7f4ec6ffd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4eb40762d0 0x7f4eb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:17.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.924+0000 7f4ec6ffd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4eb40762d0 0x7f4eb4078790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8034000 tx=0x7f4eb802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:17.928+0000 7f4eaf7fe640 1 -- 172.21.15.53:0/2038374238 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4ec0060ed0 con 0x7f4ec8160870 2024-05-10T10:41:18.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.162+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4ec80cf350 con 0x7f4ec8160870 2024-05-10T10:41:18.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.788+0000 7f4eaf7fe640 1 -- 172.21.15.53:0/2038374238 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f4ec00607d0 con 0x7f4ec8160870 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4eb40762d0 msgr2=0x7f4eb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4eb40762d0 0x7f4eb4078790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8034000 tx=0x7f4eb802e000 comp rx=0 tx=0).stop 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8160870 msgr2=0x7f4ec80c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8160870 0x7f4ec80c6730 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000c8c0 tx=0x7f4ec000cd80 comp rx=0 tx=0).stop 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 shutdown_connections 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4eb40762d0 0x7f4eb4078790 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec8160870 0x7f4ec80c6730 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec80cfcc0 0x7f4ec81542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 --2- 172.21.15.53:0/2038374238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4ec80b3970 0x7f4ec815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 >> 172.21.15.53:0/2038374238 conn(0x7f4ec80d2eb0 msgr2=0x7f4ec8071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 shutdown_connections 2024-05-10T10:41:18.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.791+0000 7f4ecd5dd640 1 -- 172.21.15.53:0/2038374238 wait complete. 2024-05-10T10:41:18.806 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:41:18.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:18.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.915+0000 7f45d4919640 1 Processor -- start 2024-05-10T10:41:18.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.915+0000 7f45d4919640 1 -- start start 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.916+0000 7f45d4919640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015f790 0x7f45d00cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.916+0000 7f45d4919640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d00d30b0 con 0x7f45d00d5d30 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.916+0000 7f45d4919640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d00d3390 con 0x7f45d015f3b0 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.916+0000 7f45d4919640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f45d00c8c80 con 0x7f45d015f790 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.916+0000 7f45ced9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f45d00d5d30 0x7f45d00d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46362/0 (socket says 172.21.15.53:46362) 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.916+0000 7f45ced9d640 1 -- 172.21.15.53:0/655848951 learned_addr learned my addr 172.21.15.53:0/655848951 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cdd9b640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015f790 0x7f45d00cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3961462066 0 0) 0x7f45d00d30b0 con 0x7f45d00d5d30 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45bc003520 con 0x7f45d00d5d30 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2669485518 0 0) 0x7f45bc003520 con 0x7f45d00d5d30 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015f790 msgr2=0x7f45d00cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015f790 0x7f45d00cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 >> v1:172.21.15.135:6789/0 conn(0x7f45d015f3b0 legacy=0x7f45d00d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d00c8f60 con 0x7f45d00d5d30 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f45c0001670 con 0x7f45d00d5d30 2024-05-10T10:41:18.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 >> v1:172.21.15.53:6789/0 conn(0x7f45d00d5d30 legacy=0x7f45d00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45bc0039e0 0x7f45bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc004340 0x7f45bc004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45bc005020 0x7f45bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00c8c80 con 0x7f45bc005020 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc00bf00 con 0x7f45bc004340 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc00c0b0 con 0x7f45bc0039e0 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45ced9d640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45bc005020 0x7f45bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45ce59c640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45bc0039e0 0x7f45bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45cdd9b640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc004340 0x7f45bc004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.917+0000 7f45ced9d640 1 -- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45bc0039e0 msgr2=0x7f45bc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:18.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45ced9d640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45bc0039e0 0x7f45bc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45ced9d640 1 -- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc004340 msgr2=0x7f45bc004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45ced9d640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc004340 0x7f45bc004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45ced9d640 1 -- 172.21.15.53:0/655848951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc003520 con 0x7f45bc005020 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45ced9d640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45bc005020 0x7f45bc00b9c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f45c000c210 tx=0x7f45bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501dfc8300e35fa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0007cf0 con 0x7f45bc005020 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c0005af0 con 0x7f45bc005020 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.918+0000 7f45cd59a640 1 -- 172.21.15.53:0/655848951 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0007070 con 0x7f45bc005020 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.919+0000 7f45d4919640 1 -- 172.21.15.53:0/655848951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45bc005020 msgr2=0x7f45bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.919+0000 7f45d4919640 1 --2- 172.21.15.53:0/655848951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45bc005020 0x7f45bc00b9c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f45c000c210 tx=0x7f45bc005830 comp rx=0 tx=0).stop 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.919+0000 7f45ced9d640 1 -- 172.21.15.53:0/655848951 reap_dead start 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.919+0000 7f45d4919640 1 -- 172.21.15.53:0/655848951 shutdown_connections 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.919+0000 7f45d4919640 1 -- 172.21.15.53:0/655848951 >> 172.21.15.53:0/655848951 conn(0x7f45d00d5130 msgr2=0x7f45d00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.919+0000 7f45d4919640 1 -- 172.21.15.53:0/655848951 shutdown_connections 2024-05-10T10:41:18.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.919+0000 7f45d4919640 1 -- 172.21.15.53:0/655848951 wait complete. 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 Processor -- start 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 -- start start 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc005020 0x7f45d0153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d015c960 0x7f45d015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015fa40 0x7f45d015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00d3f80 con 0x7f45d015c960 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00d3e00 con 0x7f45bc005020 2024-05-10T10:41:18.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45d4919640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f45d00ca770 con 0x7f45d015fa40 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.921+0000 7f45cdd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d015c960 0x7f45d015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d015c960 0x7f45d015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50946/0 (socket says 172.21.15.53:50946) 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 -- 172.21.15.53:0/2954027705 learned_addr learned my addr 172.21.15.53:0/2954027705 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45ce59c640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc005020 0x7f45d0153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45ced9d640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015fa40 0x7f45d015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 -- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015fa40 msgr2=0x7f45d015fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015fa40 0x7f45d015fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 -- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc005020 msgr2=0x7f45d0153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc005020 0x7f45d0153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 -- 172.21.15.53:0/2954027705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d0153ea0 con 0x7f45d015c960 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45ce59c640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc005020 0x7f45d0153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:18.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45ced9d640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015fa40 0x7f45d015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45cdd9b640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d015c960 0x7f45d015cde0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f45b8011990 tx=0x7f45b8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45c6ffd640 1 -- 172.21.15.53:0/2954027705 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800fa10 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45c6ffd640 1 -- 172.21.15.53:0/2954027705 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b800fbb0 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d01f86d0 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45c6ffd640 1 -- 172.21.15.53:0/2954027705 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8014440 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.922+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01f8c90 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.923+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d015d320 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.924+0000 7f45c6ffd640 1 -- 172.21.15.53:0/2954027705 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f45b8012040 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.924+0000 7f45c6ffd640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f45b00762d0 0x7f45b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.924+0000 7f45c6ffd640 1 -- 172.21.15.53:0/2954027705 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f45b8097de0 con 0x7f45d015c960 2024-05-10T10:41:18.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.925+0000 7f45ce59c640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f45b00762d0 0x7f45b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:18.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.925+0000 7f45ce59c640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f45b00762d0 0x7f45b0078790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f45c80019f0 tx=0x7f45c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:18.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:18.929+0000 7f45c6ffd640 1 -- 172.21.15.53:0/2954027705 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f45b8019030 con 0x7f45d015c960 2024-05-10T10:41:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.159+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45d006a220 con 0x7f45d015c960 2024-05-10T10:41:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.159+0000 7f45c6ffd640 1 -- 172.21.15.53:0/2954027705 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5189 (secure 0 0 0) 0x7f45b80615a0 con 0x7f45d015c960 2024-05-10T10:41:19.161 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:19.161 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:41:18.164463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4122299129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":4122299129},{"type":"v1","addr":"172.21.15.53:6835","nonce":4122299129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7479,"qdb_cluster":[7479]},"id":7}]} 2024-05-10T10:41:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 82 2024-05-10T10:41:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f45b00762d0 msgr2=0x7f45b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f45b00762d0 0x7f45b0078790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f45c80019f0 tx=0x7f45c8009040 comp rx=0 tx=0).stop 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d015c960 msgr2=0x7f45d015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d015c960 0x7f45d015cde0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f45b8011990 tx=0x7f45b8011e50 comp rx=0 tx=0).stop 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 shutdown_connections 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f45b00762d0 0x7f45b0078790 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45d015fa40 0x7f45d015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45d015c960 0x7f45d015cde0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 --2- 172.21.15.53:0/2954027705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45bc005020 0x7f45d0153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 >> 172.21.15.53:0/2954027705 conn(0x7f45d00d5130 msgr2=0x7f45d00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 shutdown_connections 2024-05-10T10:41:19.164 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.162+0000 7f45d4919640 1 -- 172.21.15.53:0/2954027705 wait complete. 2024-05-10T10:41:19.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:41:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.277+0000 7f3ad741c640 1 Processor -- start 2024-05-10T10:41:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.277+0000 7f3ad741c640 1 -- start start 2024-05-10T10:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ad741c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad0163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ad741c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ad0163ba0 con 0x7f3ad015f620 2024-05-10T10:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ad741c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ad0160190 con 0x7f3ad0070bd0 2024-05-10T10:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ad741c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad006cd70 con 0x7f3ad006c990 2024-05-10T10:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ad49b7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3ad015f620 0x7f3ad00704c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46378/0 (socket says 172.21.15.53:46378) 2024-05-10T10:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ad49b7640 1 -- 172.21.15.53:0/319034919 learned_addr learned my addr 172.21.15.53:0/319034919 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ad51b8640 1 --2- 172.21.15.53:0/319034919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad0163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ac7fff640 1 -- 172.21.15.53:0/319034919 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307629441 0 0) 0x7f3ad0163ba0 con 0x7f3ad015f620 2024-05-10T10:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.278+0000 7f3ac7fff640 1 -- 172.21.15.53:0/319034919 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3abc003520 con 0x7f3ad015f620 2024-05-10T10:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.279+0000 7f3ad51b8640 1 -- 172.21.15.53:0/319034919 >> v1:172.21.15.135:6789/0 conn(0x7f3ad0070bd0 legacy=0x7f3ad00cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.279+0000 7f3ac7fff640 1 -- 172.21.15.53:0/319034919 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 246571393 0 0) 0x7f3abc003520 con 0x7f3ad015f620 2024-05-10T10:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.279+0000 7f3ad51b8640 1 -- 172.21.15.53:0/319034919 >> v1:172.21.15.53:6789/0 conn(0x7f3ad015f620 legacy=0x7f3ad00704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.279+0000 7f3ad51b8640 1 -- 172.21.15.53:0/319034919 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad0160470 con 0x7f3ad006c990 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.279+0000 7f3ad51b8640 1 --2- 172.21.15.53:0/319034919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad0163660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0097d0 tx=0x7f3acc02a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8f8570eec599871 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ac7fff640 1 -- 172.21.15.53:0/319034919 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc037070 con 0x7f3ad006c990 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ac7fff640 1 -- 172.21.15.53:0/319034919 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3acc02ab40 con 0x7f3ad006c990 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ac7fff640 1 -- 172.21.15.53:0/319034919 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0001690 con 0x7f3ad006c990 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ad741c640 1 -- 172.21.15.53:0/319034919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 msgr2=0x7f3ad0163660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ad741c640 1 --2- 172.21.15.53:0/319034919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad0163660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0097d0 tx=0x7f3acc02a0b0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ad741c640 1 -- 172.21.15.53:0/319034919 shutdown_connections 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ad741c640 1 --2- 172.21.15.53:0/319034919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad0163660 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ad741c640 1 -- 172.21.15.53:0/319034919 >> 172.21.15.53:0/319034919 conn(0x7f3ad00b3980 msgr2=0x7f3ad00b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ad741c640 1 -- 172.21.15.53:0/319034919 shutdown_connections 2024-05-10T10:41:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.280+0000 7f3ad741c640 1 -- 172.21.15.53:0/319034919 wait complete. 2024-05-10T10:41:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.281+0000 7f3ad741c640 1 Processor -- start 2024-05-10T10:41:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad741c640 1 -- start start 2024-05-10T10:41:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad741c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad741c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad0070bd0 0x7f3ad00cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad741c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ad015f620 0x7f3ad00b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad741c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0073e70 con 0x7f3ad0070bd0 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad741c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0163140 con 0x7f3ad015f620 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad741c640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad015fa80 con 0x7f3ad006c990 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad49b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad0070bd0 0x7f3ad00cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad49b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad0070bd0 0x7f3ad00cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50952/0 (socket says 172.21.15.53:50952) 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.282+0000 7f3ad49b7640 1 -- 172.21.15.53:0/14723174 learned_addr learned my addr 172.21.15.53:0/14723174 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad51b8640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad59b9640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ad015f620 0x7f3ad00b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad49b7640 1 -- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 msgr2=0x7f3ad015a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad49b7640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad015a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad49b7640 1 -- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ad015f620 msgr2=0x7f3ad00b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad49b7640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ad015f620 0x7f3ad00b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad49b7640 1 -- 172.21.15.53:0/14723174 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad00b4830 con 0x7f3ad0070bd0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad51b8640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad015a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad59b9640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ad015f620 0x7f3ad00b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad49b7640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad0070bd0 0x7f3ad00cfab0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0001800 tx=0x7f3ac000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ac5ffb640 1 -- 172.21.15.53:0/14723174 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0018070 con 0x7f3ad0070bd0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ac5ffb640 1 -- 172.21.15.53:0/14723174 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac00074e0 con 0x7f3ad0070bd0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad00b4b10 con 0x7f3ad0070bd0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ac5ffb640 1 -- 172.21.15.53:0/14723174 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0004a20 con 0x7f3ad0070bd0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.283+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad00b67c0 con 0x7f3ad0070bd0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.284+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad006a220 con 0x7f3ad0070bd0 2024-05-10T10:41:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.285+0000 7f3ac5ffb640 1 -- 172.21.15.53:0/14723174 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3ac001b070 con 0x7f3ad0070bd0 2024-05-10T10:41:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.286+0000 7f3ac5ffb640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3ab40762d0 0x7f3ab4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.286+0000 7f3ac5ffb640 1 -- 172.21.15.53:0/14723174 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f3ac0096bc0 con 0x7f3ad0070bd0 2024-05-10T10:41:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.286+0000 7f3ad51b8640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3ab40762d0 0x7f3ab4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.286+0000 7f3ad51b8640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3ab40762d0 0x7f3ab4078790 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3acc034000 tx=0x7f3acc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.290+0000 7f3ac5ffb640 1 -- 172.21.15.53:0/14723174 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3ac00602f0 con 0x7f3ad0070bd0 2024-05-10T10:41:19.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.483+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ad0073540 con 0x7f3ad0070bd0 2024-05-10T10:41:19.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.484+0000 7f3ac5ffb640 1 -- 172.21.15.53:0/14723174 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16884 (secure 0 0 0) 0x7f3ac005fbf0 con 0x7f3ad0070bd0 2024-05-10T10:41:19.486 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:19.486 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:41:16.768369+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:41:12.805545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T10:41:14.096305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:41:19.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3ab40762d0 msgr2=0x7f3ab4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3ab40762d0 0x7f3ab4078790 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3acc034000 tx=0x7f3acc02e000 comp rx=0 tx=0).stop 2024-05-10T10:41:19.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad0070bd0 msgr2=0x7f3ad00cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad0070bd0 0x7f3ad00cfab0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0001800 tx=0x7f3ac000c9b0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 shutdown_connections 2024-05-10T10:41:19.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3ab40762d0 0x7f3ab4078790 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ad015f620 0x7f3ad00b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad0070bd0 0x7f3ad00cfab0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 --2- 172.21.15.53:0/14723174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ad006c990 0x7f3ad015a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.486+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 >> 172.21.15.53:0/14723174 conn(0x7f3ad00b3980 msgr2=0x7f3ad00d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:19.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.487+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 shutdown_connections 2024-05-10T10:41:19.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.487+0000 7f3ad741c640 1 -- 172.21.15.53:0/14723174 wait complete. 2024-05-10T10:41:19.500 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:41:19.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:41:19.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.602+0000 7f39b6938640 1 Processor -- start 2024-05-10T10:41:19.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.602+0000 7f39b6938640 1 -- start start 2024-05-10T10:41:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b6938640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f39b0162750 0x7f39b00d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b6938640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b00d5cb0 0x7f39b00d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b6938640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39b0162b30 0x7f39b00d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b6938640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f39b00b4f20 con 0x7f39b0162b30 2024-05-10T10:41:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b6938640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f39b00d0fe0 con 0x7f39b0162750 2024-05-10T10:41:19.605 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b6938640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f39b00b45f0 con 0x7f39b00d5cb0 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39b0162b30 0x7f39b00d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39b0162b30 0x7f39b00d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50960/0 (socket says 172.21.15.53:50960) 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 -- 172.21.15.53:0/1051587045 learned_addr learned my addr 172.21.15.53:0/1051587045 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39af7fe640 1 --2- 172.21.15.53:0/1051587045 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b00d5cb0 0x7f39b00d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39affff640 1 --2- 172.21.15.53:0/1051587045 >> v2:172.21.15.135:3300/0 conn(0x7f39b0162750 0x7f39b00d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 -- 172.21.15.53:0/1051587045 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b00d5cb0 msgr2=0x7f39b00d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 --2- 172.21.15.53:0/1051587045 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b00d5cb0 0x7f39b00d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 -- 172.21.15.53:0/1051587045 >> v2:172.21.15.135:3300/0 conn(0x7f39b0162750 msgr2=0x7f39b00d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 --2- 172.21.15.53:0/1051587045 >> v2:172.21.15.135:3300/0 conn(0x7f39b0162750 0x7f39b00d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39b4ed5640 1 -- 172.21.15.53:0/1051587045 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b00b48d0 con 0x7f39b0162b30 2024-05-10T10:41:19.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.603+0000 7f39af7fe640 1 --2- 172.21.15.53:0/1051587045 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b00d5cb0 0x7f39b00d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.604+0000 7f39b4ed5640 1 --2- 172.21.15.53:0/1051587045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b0162b30 0x7f39b00d3b70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f39b00b7c40 tx=0x7f39a802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6074bef41f77b9b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.604+0000 7f39aeffd640 1 -- 172.21.15.53:0/1051587045 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8036070 con 0x7f39b0162b30 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.604+0000 7f39aeffd640 1 -- 172.21.15.53:0/1051587045 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a8031600 con 0x7f39b0162b30 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.604+0000 7f39aeffd640 1 -- 172.21.15.53:0/1051587045 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a80319d0 con 0x7f39b0162b30 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 -- 172.21.15.53:0/1051587045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b0162b30 msgr2=0x7f39b00d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 --2- 172.21.15.53:0/1051587045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b0162b30 0x7f39b00d3b70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f39b00b7c40 tx=0x7f39a802a610 comp rx=0 tx=0).stop 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 -- 172.21.15.53:0/1051587045 shutdown_connections 2024-05-10T10:41:19.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 --2- 172.21.15.53:0/1051587045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b0162b30 0x7f39b00d3b70 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 --2- 172.21.15.53:0/1051587045 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b00d5cb0 0x7f39b00d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 --2- 172.21.15.53:0/1051587045 >> v2:172.21.15.135:3300/0 conn(0x7f39b0162750 0x7f39b00d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 -- 172.21.15.53:0/1051587045 >> 172.21.15.53:0/1051587045 conn(0x7f39b009e3b0 msgr2=0x7f39b00d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 -- 172.21.15.53:0/1051587045 shutdown_connections 2024-05-10T10:41:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.605+0000 7f39b6938640 1 -- 172.21.15.53:0/1051587045 wait complete. 2024-05-10T10:41:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.606+0000 7f39b6938640 1 Processor -- start 2024-05-10T10:41:19.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.606+0000 7f39b6938640 1 -- start start 2024-05-10T10:41:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b6938640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b0162750 0x7f39b01f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.611 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b6938640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b0162b30 0x7f39b00ba260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b6938640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b00cdcd0 0x7f39b00ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b6938640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f39b00d0d30 con 0x7f39b00cdcd0 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b6938640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39b00d0bb0 con 0x7f39b0162750 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b6938640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f39b00d1660 con 0x7f39b0162b30 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b4ed5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b00cdcd0 0x7f39b00ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b4ed5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b00cdcd0 0x7f39b00ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50966/0 (socket says 172.21.15.53:50966) 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39b4ed5640 1 -- 172.21.15.53:0/2523983266 learned_addr learned my addr 172.21.15.53:0/2523983266 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39af7fe640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b0162b30 0x7f39b00ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.607+0000 7f39affff640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b0162750 0x7f39b01f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b4ed5640 1 -- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b0162b30 msgr2=0x7f39b00ba260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b4ed5640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b0162b30 0x7f39b00ba260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b4ed5640 1 -- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b0162750 msgr2=0x7f39b01f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b4ed5640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b0162750 0x7f39b01f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b4ed5640 1 -- 172.21.15.53:0/2523983266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b00ccb00 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39affff640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b0162750 0x7f39b01f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b4ed5640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b00cdcd0 0x7f39b00ce180 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f39a8034040 tx=0x7f39a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39acff9640 1 -- 172.21.15.53:0/2523983266 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8036050 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39acff9640 1 -- 172.21.15.53:0/2523983266 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a8042070 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39acff9640 1 -- 172.21.15.53:0/2523983266 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8009510 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b00ccd80 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.608+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b00cd340 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.609+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b00cd510 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.610+0000 7f39acff9640 1 -- 172.21.15.53:0/2523983266 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f39a8007bd0 con 0x7f39b00cdcd0 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.610+0000 7f39acff9640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f399c076300 0x7f399c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.610+0000 7f39acff9640 1 -- 172.21.15.53:0/2523983266 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f39a80b8500 con 0x7f39b00cdcd0 2024-05-10T10:41:19.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.610+0000 7f39affff640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f399c076300 0x7f399c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.611+0000 7f39affff640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f399c076300 0x7f399c0787c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006fd0 tx=0x7f39a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.614+0000 7f39acff9640 1 -- 172.21.15.53:0/2523983266 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f39b00ccb00 con 0x7f39b00cdcd0 2024-05-10T10:41:19.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.801+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39b006a220 con 0x7f399c076300 2024-05-10T10:41:19.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.808+0000 7f39acff9640 1 -- 172.21.15.53:0/2523983266 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f39b006a220 con 0x7f399c076300 2024-05-10T10:41:19.812 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:19.814 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:41:19.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f399c076300 msgr2=0x7f399c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f399c076300 0x7f399c0787c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006fd0 tx=0x7f39a0008040 comp rx=0 tx=0).stop 2024-05-10T10:41:19.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b00cdcd0 msgr2=0x7f39b00ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b00cdcd0 0x7f39b00ce180 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f39a8034040 tx=0x7f39a8004a40 comp rx=0 tx=0).stop 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 shutdown_connections 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f399c076300 0x7f399c0787c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39b00cdcd0 0x7f39b00ce180 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39b0162b30 0x7f39b00ba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 --2- 172.21.15.53:0/2523983266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b0162750 0x7f39b01f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 >> 172.21.15.53:0/2523983266 conn(0x7f39b009e3b0 msgr2=0x7f39b00d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 shutdown_connections 2024-05-10T10:41:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.811+0000 7f39b6938640 1 -- 172.21.15.53:0/2523983266 wait complete. 2024-05-10T10:41:19.824 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-05-10T10:41:18.270798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260632,"kb_used_data":11172,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714088,"statfs":{"total":773094113280,"available":772827226112,"internally_reserved":0,"allocated":11440128,"data_stored":5896936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.519135"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774193+0000","last_change":"2024-05-10T10:41:15.778935+0000","last_active":"2024-05-10T10:41:16.774193+0000","last_peered":"2024-05-10T10:41:16.774193+0000","last_clean":"2024-05-10T10:41:16.774193+0000","last_became_active":"2024-05-10T10:41:15.778493+0000","last_became_peered":"2024-05-10T10:41:15.778493+0000","last_unstale":"2024-05-10T10:41:16.774193+0000","last_undegraded":"2024-05-10T10:41:16.774193+0000","last_fullsized":"2024-05-10T10:41:16.774193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:24.554910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628783+0000","last_change":"2024-05-10T10:41:15.783290+0000","last_active":"2024-05-10T10:41:17.628783+0000","last_peered":"2024-05-10T10:41:17.628783+0000","last_clean":"2024-05-10T10:41:17.628783+0000","last_became_active":"2024-05-10T10:41:15.782785+0000","last_became_peered":"2024-05-10T10:41:15.782785+0000","last_unstale":"2024-05-10T10:41:17.628783+0000","last_undegraded":"2024-05-10T10:41:17.628783+0000","last_fullsized":"2024-05-10T10:41:17.628783+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:24.931549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773996+0000","last_change":"2024-05-10T10:41:15.781495+0000","last_active":"2024-05-10T10:41:16.773996+0000","last_peered":"2024-05-10T10:41:16.773996+0000","last_clean":"2024-05-10T10:41:16.773996+0000","last_became_active":"2024-05-10T10:41:15.780697+0000","last_became_peered":"2024-05-10T10:41:15.780697+0000","last_unstale":"2024-05-10T10:41:16.773996+0000","last_undegraded":"2024-05-10T10:41:16.773996+0000","last_fullsized":"2024-05-10T10:41:16.773996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:15.016802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628610+0000","last_change":"2024-05-10T10:41:15.780359+0000","last_active":"2024-05-10T10:41:17.628610+0000","last_peered":"2024-05-10T10:41:17.628610+0000","last_clean":"2024-05-10T10:41:17.628610+0000","last_became_active":"2024-05-10T10:41:15.780115+0000","last_became_peered":"2024-05-10T10:41:15.780115+0000","last_unstale":"2024-05-10T10:41:17.628610+0000","last_undegraded":"2024-05-10T10:41:17.628610+0000","last_fullsized":"2024-05-10T10:41:17.628610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:50.245204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443392+0000","last_change":"2024-05-10T10:41:15.779027+0000","last_active":"2024-05-10T10:41:17.443392+0000","last_peered":"2024-05-10T10:41:17.443392+0000","last_clean":"2024-05-10T10:41:17.443392+0000","last_became_active":"2024-05-10T10:41:15.778778+0000","last_became_peered":"2024-05-10T10:41:15.778778+0000","last_unstale":"2024-05-10T10:41:17.443392+0000","last_undegraded":"2024-05-10T10:41:17.443392+0000","last_fullsized":"2024-05-10T10:41:17.443392+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:24.931549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628232+0000","last_change":"2024-05-10T10:41:15.783523+0000","last_active":"2024-05-10T10:41:17.628232+0000","last_peered":"2024-05-10T10:41:17.628232+0000","last_clean":"2024-05-10T10:41:17.628232+0000","last_became_active":"2024-05-10T10:41:15.783255+0000","last_became_peered":"2024-05-10T10:41:15.783255+0000","last_unstale":"2024-05-10T10:41:17.628232+0000","last_undegraded":"2024-05-10T10:41:17.628232+0000","last_fullsized":"2024-05-10T10:41:17.628232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:43.737629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628223+0000","last_change":"2024-05-10T10:41:15.782753+0000","last_active":"2024-05-10T10:41:17.628223+0000","last_peered":"2024-05-10T10:41:17.628223+0000","last_clean":"2024-05-10T10:41:17.628223+0000","last_became_active":"2024-05-10T10:41:15.782129+0000","last_became_peered":"2024-05-10T10:41:15.782129+0000","last_unstale":"2024-05-10T10:41:17.628223+0000","last_undegraded":"2024-05-10T10:41:17.628223+0000","last_fullsized":"2024-05-10T10:41:17.628223+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:52:11.478952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935747+0000","last_change":"2024-05-10T10:41:15.775862+0000","last_active":"2024-05-10T10:41:16.935747+0000","last_peered":"2024-05-10T10:41:16.935747+0000","last_clean":"2024-05-10T10:41:16.935747+0000","last_became_active":"2024-05-10T10:41:15.775593+0000","last_became_peered":"2024-05-10T10:41:15.775593+0000","last_unstale":"2024-05-10T10:41:16.935747+0000","last_undegraded":"2024-05-10T10:41:16.935747+0000","last_fullsized":"2024-05-10T10:41:16.935747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:54.459998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628728+0000","last_change":"2024-05-10T10:41:15.783349+0000","last_active":"2024-05-10T10:41:17.628728+0000","last_peered":"2024-05-10T10:41:17.628728+0000","last_clean":"2024-05-10T10:41:17.628728+0000","last_became_active":"2024-05-10T10:41:15.783036+0000","last_became_peered":"2024-05-10T10:41:15.783036+0000","last_unstale":"2024-05-10T10:41:17.628728+0000","last_undegraded":"2024-05-10T10:41:17.628728+0000","last_fullsized":"2024-05-10T10:41:17.628728+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:35.161552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443668+0000","last_change":"2024-05-10T10:41:15.784171+0000","last_active":"2024-05-10T10:41:17.443668+0000","last_peered":"2024-05-10T10:41:17.443668+0000","last_clean":"2024-05-10T10:41:17.443668+0000","last_became_active":"2024-05-10T10:41:15.783892+0000","last_became_peered":"2024-05-10T10:41:15.783892+0000","last_unstale":"2024-05-10T10:41:17.443668+0000","last_undegraded":"2024-05-10T10:41:17.443668+0000","last_fullsized":"2024-05-10T10:41:17.443668+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776950+0000","last_change":"2024-05-10T10:41:15.784291+0000","last_active":"2024-05-10T10:41:16.776950+0000","last_peered":"2024-05-10T10:41:16.776950+0000","last_clean":"2024-05-10T10:41:16.776950+0000","last_became_active":"2024-05-10T10:41:15.784012+0000","last_became_peered":"2024-05-10T10:41:15.784012+0000","last_unstale":"2024-05-10T10:41:16.776950+0000","last_undegraded":"2024-05-10T10:41:16.776950+0000","last_fullsized":"2024-05-10T10:41:16.776950+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:02.498815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628324+0000","last_change":"2024-05-10T10:41:15.774082+0000","last_active":"2024-05-10T10:41:17.628324+0000","last_peered":"2024-05-10T10:41:17.628324+0000","last_clean":"2024-05-10T10:41:17.628324+0000","last_became_active":"2024-05-10T10:41:15.773775+0000","last_became_peered":"2024-05-10T10:41:15.773775+0000","last_unstale":"2024-05-10T10:41:17.628324+0000","last_undegraded":"2024-05-10T10:41:17.628324+0000","last_fullsized":"2024-05-10T10:41:17.628324+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:35.161552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628471+0000","last_change":"2024-05-10T10:41:15.782607+0000","last_active":"2024-05-10T10:41:17.628471+0000","last_peered":"2024-05-10T10:41:17.628471+0000","last_clean":"2024-05-10T10:41:17.628471+0000","last_became_active":"2024-05-10T10:41:15.782348+0000","last_became_peered":"2024-05-10T10:41:15.782348+0000","last_unstale":"2024-05-10T10:41:17.628471+0000","last_undegraded":"2024-05-10T10:41:17.628471+0000","last_fullsized":"2024-05-10T10:41:17.628471+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:02.498815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776986+0000","last_change":"2024-05-10T10:41:15.775645+0000","last_active":"2024-05-10T10:41:16.776986+0000","last_peered":"2024-05-10T10:41:16.776986+0000","last_clean":"2024-05-10T10:41:16.776986+0000","last_became_active":"2024-05-10T10:41:15.775351+0000","last_became_peered":"2024-05-10T10:41:15.775351+0000","last_unstale":"2024-05-10T10:41:16.776986+0000","last_undegraded":"2024-05-10T10:41:16.776986+0000","last_fullsized":"2024-05-10T10:41:16.776986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:49.090489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776807+0000","last_change":"2024-05-10T10:41:15.765127+0000","last_active":"2024-05-10T10:41:16.776807+0000","last_peered":"2024-05-10T10:41:16.776807+0000","last_clean":"2024-05-10T10:41:16.776807+0000","last_became_active":"2024-05-10T10:41:15.764804+0000","last_became_peered":"2024-05-10T10:41:15.764804+0000","last_unstale":"2024-05-10T10:41:16.776807+0000","last_undegraded":"2024-05-10T10:41:16.776807+0000","last_fullsized":"2024-05-10T10:41:16.776807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:09.552841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776637+0000","last_change":"2024-05-10T10:41:15.769947+0000","last_active":"2024-05-10T10:41:16.776637+0000","last_peered":"2024-05-10T10:41:16.776637+0000","last_clean":"2024-05-10T10:41:16.776637+0000","last_became_active":"2024-05-10T10:41:15.769634+0000","last_became_peered":"2024-05-10T10:41:15.769634+0000","last_unstale":"2024-05-10T10:41:16.776637+0000","last_undegraded":"2024-05-10T10:41:16.776637+0000","last_fullsized":"2024-05-10T10:41:16.776637+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:50.245204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774202+0000","last_change":"2024-05-10T10:41:15.779643+0000","last_active":"2024-05-10T10:41:16.774202+0000","last_peered":"2024-05-10T10:41:16.774202+0000","last_clean":"2024-05-10T10:41:16.774202+0000","last_became_active":"2024-05-10T10:41:15.779407+0000","last_became_peered":"2024-05-10T10:41:15.779407+0000","last_unstale":"2024-05-10T10:41:16.774202+0000","last_undegraded":"2024-05-10T10:41:16.774202+0000","last_fullsized":"2024-05-10T10:41:16.774202+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:23.327486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628865+0000","last_change":"2024-05-10T10:41:15.782386+0000","last_active":"2024-05-10T10:41:17.628865+0000","last_peered":"2024-05-10T10:41:17.628865+0000","last_clean":"2024-05-10T10:41:17.628865+0000","last_became_active":"2024-05-10T10:41:15.781984+0000","last_became_peered":"2024-05-10T10:41:15.781984+0000","last_unstale":"2024-05-10T10:41:17.628865+0000","last_undegraded":"2024-05-10T10:41:17.628865+0000","last_fullsized":"2024-05-10T10:41:17.628865+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774173+0000","last_change":"2024-05-10T10:41:15.788308+0000","last_active":"2024-05-10T10:41:16.774173+0000","last_peered":"2024-05-10T10:41:16.774173+0000","last_clean":"2024-05-10T10:41:16.774173+0000","last_became_active":"2024-05-10T10:41:15.788020+0000","last_became_peered":"2024-05-10T10:41:15.788020+0000","last_unstale":"2024-05-10T10:41:16.774173+0000","last_undegraded":"2024-05-10T10:41:16.774173+0000","last_fullsized":"2024-05-10T10:41:16.774173+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:59:18.078967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628344+0000","last_change":"2024-05-10T10:41:15.774087+0000","last_active":"2024-05-10T10:41:17.628344+0000","last_peered":"2024-05-10T10:41:17.628344+0000","last_clean":"2024-05-10T10:41:17.628344+0000","last_became_active":"2024-05-10T10:41:15.773769+0000","last_became_peered":"2024-05-10T10:41:15.773769+0000","last_unstale":"2024-05-10T10:41:17.628344+0000","last_undegraded":"2024-05-10T10:41:17.628344+0000","last_fullsized":"2024-05-10T10:41:17.628344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:24.931549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936020+0000","last_change":"2024-05-10T10:41:15.773651+0000","last_active":"2024-05-10T10:41:16.936020+0000","last_peered":"2024-05-10T10:41:16.936020+0000","last_clean":"2024-05-10T10:41:16.936020+0000","last_became_active":"2024-05-10T10:41:15.773375+0000","last_became_peered":"2024-05-10T10:41:15.773375+0000","last_unstale":"2024-05-10T10:41:16.936020+0000","last_undegraded":"2024-05-10T10:41:16.936020+0000","last_fullsized":"2024-05-10T10:41:16.936020+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:21.812285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443367+0000","last_change":"2024-05-10T10:41:15.775102+0000","last_active":"2024-05-10T10:41:17.443367+0000","last_peered":"2024-05-10T10:41:17.443367+0000","last_clean":"2024-05-10T10:41:17.443367+0000","last_became_active":"2024-05-10T10:41:15.774802+0000","last_became_peered":"2024-05-10T10:41:15.774802+0000","last_unstale":"2024-05-10T10:41:17.443367+0000","last_undegraded":"2024-05-10T10:41:17.443367+0000","last_fullsized":"2024-05-10T10:41:17.443367+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:35.161552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773832+0000","last_change":"2024-05-10T10:41:15.774238+0000","last_active":"2024-05-10T10:41:16.773832+0000","last_peered":"2024-05-10T10:41:16.773832+0000","last_clean":"2024-05-10T10:41:16.773832+0000","last_became_active":"2024-05-10T10:41:15.773993+0000","last_became_peered":"2024-05-10T10:41:15.773993+0000","last_unstale":"2024-05-10T10:41:16.773832+0000","last_undegraded":"2024-05-10T10:41:16.773832+0000","last_fullsized":"2024-05-10T10:41:16.773832+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:37.552832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443190+0000","last_change":"2024-05-10T10:41:15.790978+0000","last_active":"2024-05-10T10:41:17.443190+0000","last_peered":"2024-05-10T10:41:17.443190+0000","last_clean":"2024-05-10T10:41:17.443190+0000","last_became_active":"2024-05-10T10:41:15.789961+0000","last_became_peered":"2024-05-10T10:41:15.789961+0000","last_unstale":"2024-05-10T10:41:17.443190+0000","last_undegraded":"2024-05-10T10:41:17.443190+0000","last_fullsized":"2024-05-10T10:41:17.443190+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:38:43.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628685+0000","last_change":"2024-05-10T10:41:15.778105+0000","last_active":"2024-05-10T10:41:17.628685+0000","last_peered":"2024-05-10T10:41:17.628685+0000","last_clean":"2024-05-10T10:41:17.628685+0000","last_became_active":"2024-05-10T10:41:15.777822+0000","last_became_peered":"2024-05-10T10:41:15.777822+0000","last_unstale":"2024-05-10T10:41:17.628685+0000","last_undegraded":"2024-05-10T10:41:17.628685+0000","last_fullsized":"2024-05-10T10:41:17.628685+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:26:44.754221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628624+0000","last_change":"2024-05-10T10:41:15.785593+0000","last_active":"2024-05-10T10:41:17.628624+0000","last_peered":"2024-05-10T10:41:17.628624+0000","last_clean":"2024-05-10T10:41:17.628624+0000","last_became_active":"2024-05-10T10:41:15.785287+0000","last_became_peered":"2024-05-10T10:41:15.785287+0000","last_unstale":"2024-05-10T10:41:17.628624+0000","last_undegraded":"2024-05-10T10:41:17.628624+0000","last_fullsized":"2024-05-10T10:41:17.628624+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776943+0000","last_change":"2024-05-10T10:41:15.764267+0000","last_active":"2024-05-10T10:41:16.776943+0000","last_peered":"2024-05-10T10:41:16.776943+0000","last_clean":"2024-05-10T10:41:16.776943+0000","last_became_active":"2024-05-10T10:41:15.763940+0000","last_became_peered":"2024-05-10T10:41:15.763940+0000","last_unstale":"2024-05-10T10:41:16.776943+0000","last_undegraded":"2024-05-10T10:41:16.776943+0000","last_fullsized":"2024-05-10T10:41:16.776943+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:26.540192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628817+0000","last_change":"2024-05-10T10:41:15.782858+0000","last_active":"2024-05-10T10:41:17.628817+0000","last_peered":"2024-05-10T10:41:17.628817+0000","last_clean":"2024-05-10T10:41:17.628817+0000","last_became_active":"2024-05-10T10:41:15.782403+0000","last_became_peered":"2024-05-10T10:41:15.782403+0000","last_unstale":"2024-05-10T10:41:17.628817+0000","last_undegraded":"2024-05-10T10:41:17.628817+0000","last_fullsized":"2024-05-10T10:41:17.628817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936055+0000","last_change":"2024-05-10T10:41:15.767203+0000","last_active":"2024-05-10T10:41:16.936055+0000","last_peered":"2024-05-10T10:41:16.936055+0000","last_clean":"2024-05-10T10:41:16.936055+0000","last_became_active":"2024-05-10T10:41:15.766951+0000","last_became_peered":"2024-05-10T10:41:15.766951+0000","last_unstale":"2024-05-10T10:41:16.936055+0000","last_undegraded":"2024-05-10T10:41:16.936055+0000","last_fullsized":"2024-05-10T10:41:16.936055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:18:21.298157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628344+0000","last_change":"2024-05-10T10:41:15.773963+0000","last_active":"2024-05-10T10:41:17.628344+0000","last_peered":"2024-05-10T10:41:17.628344+0000","last_clean":"2024-05-10T10:41:17.628344+0000","last_became_active":"2024-05-10T10:41:15.773687+0000","last_became_peered":"2024-05-10T10:41:15.773687+0000","last_unstale":"2024-05-10T10:41:17.628344+0000","last_undegraded":"2024-05-10T10:41:17.628344+0000","last_fullsized":"2024-05-10T10:41:17.628344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628384+0000","last_change":"2024-05-10T10:41:15.783053+0000","last_active":"2024-05-10T10:41:17.628384+0000","last_peered":"2024-05-10T10:41:17.628384+0000","last_clean":"2024-05-10T10:41:17.628384+0000","last_became_active":"2024-05-10T10:41:15.782238+0000","last_became_peered":"2024-05-10T10:41:15.782238+0000","last_unstale":"2024-05-10T10:41:17.628384+0000","last_undegraded":"2024-05-10T10:41:17.628384+0000","last_fullsized":"2024-05-10T10:41:17.628384+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:38:43.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935857+0000","last_change":"2024-05-10T10:41:15.770987+0000","last_active":"2024-05-10T10:41:16.935857+0000","last_peered":"2024-05-10T10:41:16.935857+0000","last_clean":"2024-05-10T10:41:16.935857+0000","last_became_active":"2024-05-10T10:41:15.770716+0000","last_became_peered":"2024-05-10T10:41:15.770716+0000","last_unstale":"2024-05-10T10:41:16.935857+0000","last_undegraded":"2024-05-10T10:41:16.935857+0000","last_fullsized":"2024-05-10T10:41:16.935857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:25:24.901684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773965+0000","last_change":"2024-05-10T10:41:15.784690+0000","last_active":"2024-05-10T10:41:16.773965+0000","last_peered":"2024-05-10T10:41:16.773965+0000","last_clean":"2024-05-10T10:41:16.773965+0000","last_became_active":"2024-05-10T10:41:15.784422+0000","last_became_peered":"2024-05-10T10:41:15.784422+0000","last_unstale":"2024-05-10T10:41:16.773965+0000","last_undegraded":"2024-05-10T10:41:16.773965+0000","last_fullsized":"2024-05-10T10:41:16.773965+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:16:36.103269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628824+0000","last_change":"2024-05-10T10:41:14.775066+0000","last_active":"2024-05-10T10:41:17.628824+0000","last_peered":"2024-05-10T10:41:17.628824+0000","last_clean":"2024-05-10T10:41:17.628824+0000","last_became_active":"2024-05-10T10:41:14.774728+0000","last_became_peered":"2024-05-10T10:41:14.774728+0000","last_unstale":"2024-05-10T10:41:17.628824+0000","last_undegraded":"2024-05-10T10:41:17.628824+0000","last_fullsized":"2024-05-10T10:41:17.628824+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:38:42.293587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935821+0000","last_change":"2024-05-10T10:41:15.777808+0000","last_active":"2024-05-10T10:41:16.935821+0000","last_peered":"2024-05-10T10:41:16.935821+0000","last_clean":"2024-05-10T10:41:16.935821+0000","last_became_active":"2024-05-10T10:41:15.777563+0000","last_became_peered":"2024-05-10T10:41:15.777563+0000","last_unstale":"2024-05-10T10:41:16.935821+0000","last_undegraded":"2024-05-10T10:41:16.935821+0000","last_fullsized":"2024-05-10T10:41:16.935821+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:34:55.286901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936164+0000","last_change":"2024-05-10T10:41:14.773000+0000","last_active":"2024-05-10T10:41:16.936164+0000","last_peered":"2024-05-10T10:41:16.936164+0000","last_clean":"2024-05-10T10:41:16.936164+0000","last_became_active":"2024-05-10T10:41:14.772714+0000","last_became_peered":"2024-05-10T10:41:14.772714+0000","last_unstale":"2024-05-10T10:41:16.936164+0000","last_undegraded":"2024-05-10T10:41:16.936164+0000","last_fullsized":"2024-05-10T10:41:16.936164+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:25:21.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628438+0000","last_change":"2024-05-10T10:41:15.783915+0000","last_active":"2024-05-10T10:41:17.628438+0000","last_peered":"2024-05-10T10:41:17.628438+0000","last_clean":"2024-05-10T10:41:17.628438+0000","last_became_active":"2024-05-10T10:41:15.783648+0000","last_became_peered":"2024-05-10T10:41:15.783648+0000","last_unstale":"2024-05-10T10:41:17.628438+0000","last_undegraded":"2024-05-10T10:41:17.628438+0000","last_fullsized":"2024-05-10T10:41:17.628438+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628500+0000","last_change":"2024-05-10T10:41:14.771387+0000","last_active":"2024-05-10T10:41:17.628500+0000","last_peered":"2024-05-10T10:41:17.628500+0000","last_clean":"2024-05-10T10:41:17.628500+0000","last_became_active":"2024-05-10T10:41:14.771063+0000","last_became_peered":"2024-05-10T10:41:14.771063+0000","last_unstale":"2024-05-10T10:41:17.628500+0000","last_undegraded":"2024-05-10T10:41:17.628500+0000","last_fullsized":"2024-05-10T10:41:17.628500+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:37:53.826208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776308+0000","last_change":"2024-05-10T10:41:15.771738+0000","last_active":"2024-05-10T10:41:16.776308+0000","last_peered":"2024-05-10T10:41:16.776308+0000","last_clean":"2024-05-10T10:41:16.776308+0000","last_became_active":"2024-05-10T10:41:15.771490+0000","last_became_peered":"2024-05-10T10:41:15.771490+0000","last_unstale":"2024-05-10T10:41:16.776308+0000","last_undegraded":"2024-05-10T10:41:16.776308+0000","last_fullsized":"2024-05-10T10:41:16.776308+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:26:44.754221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443577+0000","last_change":"2024-05-10T10:41:14.769973+0000","last_active":"2024-05-10T10:41:17.443577+0000","last_peered":"2024-05-10T10:41:17.443577+0000","last_clean":"2024-05-10T10:41:17.443577+0000","last_became_active":"2024-05-10T10:41:14.769707+0000","last_became_peered":"2024-05-10T10:41:14.769707+0000","last_unstale":"2024-05-10T10:41:17.443577+0000","last_undegraded":"2024-05-10T10:41:17.443577+0000","last_fullsized":"2024-05-10T10:41:17.443577+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:33:31.189870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628684+0000","last_change":"2024-05-10T10:41:15.787899+0000","last_active":"2024-05-10T10:41:17.628684+0000","last_peered":"2024-05-10T10:41:17.628684+0000","last_clean":"2024-05-10T10:41:17.628684+0000","last_became_active":"2024-05-10T10:41:15.787621+0000","last_became_peered":"2024-05-10T10:41:15.787621+0000","last_unstale":"2024-05-10T10:41:17.628684+0000","last_undegraded":"2024-05-10T10:41:17.628684+0000","last_fullsized":"2024-05-10T10:41:17.628684+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:09.552841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777192+0000","last_change":"2024-05-10T10:41:14.772365+0000","last_active":"2024-05-10T10:41:16.777192+0000","last_peered":"2024-05-10T10:41:16.777192+0000","last_clean":"2024-05-10T10:41:16.777192+0000","last_became_active":"2024-05-10T10:41:14.772085+0000","last_became_peered":"2024-05-10T10:41:14.772085+0000","last_unstale":"2024-05-10T10:41:16.777192+0000","last_undegraded":"2024-05-10T10:41:16.777192+0000","last_fullsized":"2024-05-10T10:41:16.777192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:56.141822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774317+0000","last_change":"2024-05-10T10:41:15.786425+0000","last_active":"2024-05-10T10:41:16.774317+0000","last_peered":"2024-05-10T10:41:16.774317+0000","last_clean":"2024-05-10T10:41:16.774317+0000","last_became_active":"2024-05-10T10:41:15.786108+0000","last_became_peered":"2024-05-10T10:41:15.786108+0000","last_unstale":"2024-05-10T10:41:16.774317+0000","last_undegraded":"2024-05-10T10:41:16.774317+0000","last_fullsized":"2024-05-10T10:41:16.774317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:00.049649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776127+0000","last_change":"2024-05-10T10:41:14.774933+0000","last_active":"2024-05-10T10:41:16.776127+0000","last_peered":"2024-05-10T10:41:16.776127+0000","last_clean":"2024-05-10T10:41:16.776127+0000","last_became_active":"2024-05-10T10:41:14.774679+0000","last_became_peered":"2024-05-10T10:41:14.774679+0000","last_unstale":"2024-05-10T10:41:16.776127+0000","last_undegraded":"2024-05-10T10:41:16.776127+0000","last_fullsized":"2024-05-10T10:41:16.776127+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:40:35.673531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628952+0000","last_change":"2024-05-10T10:41:15.782664+0000","last_active":"2024-05-10T10:41:17.628952+0000","last_peered":"2024-05-10T10:41:17.628952+0000","last_clean":"2024-05-10T10:41:17.628952+0000","last_became_active":"2024-05-10T10:41:15.781956+0000","last_became_peered":"2024-05-10T10:41:15.781956+0000","last_unstale":"2024-05-10T10:41:17.628952+0000","last_undegraded":"2024-05-10T10:41:17.628952+0000","last_fullsized":"2024-05-10T10:41:17.628952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628020+0000","last_change":"2024-05-10T10:41:14.771441+0000","last_active":"2024-05-10T10:41:17.628020+0000","last_peered":"2024-05-10T10:41:17.628020+0000","last_clean":"2024-05-10T10:41:17.628020+0000","last_became_active":"2024-05-10T10:41:14.771125+0000","last_became_peered":"2024-05-10T10:41:14.771125+0000","last_unstale":"2024-05-10T10:41:17.628020+0000","last_undegraded":"2024-05-10T10:41:17.628020+0000","last_fullsized":"2024-05-10T10:41:17.628020+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:33:31.189870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628791+0000","last_change":"2024-05-10T10:41:15.790041+0000","last_active":"2024-05-10T10:41:17.628791+0000","last_peered":"2024-05-10T10:41:17.628791+0000","last_clean":"2024-05-10T10:41:17.628791+0000","last_became_active":"2024-05-10T10:41:15.789790+0000","last_became_peered":"2024-05-10T10:41:15.789790+0000","last_unstale":"2024-05-10T10:41:17.628791+0000","last_undegraded":"2024-05-10T10:41:17.628791+0000","last_fullsized":"2024-05-10T10:41:17.628791+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:49.090489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628328+0000","last_change":"2024-05-10T10:41:14.768068+0000","last_active":"2024-05-10T10:41:17.628328+0000","last_peered":"2024-05-10T10:41:17.628328+0000","last_clean":"2024-05-10T10:41:17.628328+0000","last_became_active":"2024-05-10T10:41:14.767741+0000","last_became_peered":"2024-05-10T10:41:14.767741+0000","last_unstale":"2024-05-10T10:41:17.628328+0000","last_undegraded":"2024-05-10T10:41:17.628328+0000","last_fullsized":"2024-05-10T10:41:17.628328+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:06:57.899386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773813+0000","last_change":"2024-05-10T10:41:15.785589+0000","last_active":"2024-05-10T10:41:16.773813+0000","last_peered":"2024-05-10T10:41:16.773813+0000","last_clean":"2024-05-10T10:41:16.773813+0000","last_became_active":"2024-05-10T10:41:15.785312+0000","last_became_peered":"2024-05-10T10:41:15.785312+0000","last_unstale":"2024-05-10T10:41:16.773813+0000","last_undegraded":"2024-05-10T10:41:16.773813+0000","last_fullsized":"2024-05-10T10:41:16.773813+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:25.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628707+0000","last_change":"2024-05-10T10:41:14.771832+0000","last_active":"2024-05-10T10:41:17.628707+0000","last_peered":"2024-05-10T10:41:17.628707+0000","last_clean":"2024-05-10T10:41:17.628707+0000","last_became_active":"2024-05-10T10:41:14.771517+0000","last_became_peered":"2024-05-10T10:41:14.771517+0000","last_unstale":"2024-05-10T10:41:17.628707+0000","last_undegraded":"2024-05-10T10:41:17.628707+0000","last_fullsized":"2024-05-10T10:41:17.628707+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:25.522332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443242+0000","last_change":"2024-05-10T10:41:15.776876+0000","last_active":"2024-05-10T10:41:17.443242+0000","last_peered":"2024-05-10T10:41:17.443242+0000","last_clean":"2024-05-10T10:41:17.443242+0000","last_became_active":"2024-05-10T10:41:15.776634+0000","last_became_peered":"2024-05-10T10:41:15.776634+0000","last_unstale":"2024-05-10T10:41:17.443242+0000","last_undegraded":"2024-05-10T10:41:17.443242+0000","last_fullsized":"2024-05-10T10:41:17.443242+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:13.788111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773942+0000","last_change":"2024-05-10T10:41:14.775097+0000","last_active":"2024-05-10T10:41:16.773942+0000","last_peered":"2024-05-10T10:41:16.773942+0000","last_clean":"2024-05-10T10:41:16.773942+0000","last_became_active":"2024-05-10T10:41:14.774435+0000","last_became_peered":"2024-05-10T10:41:14.774435+0000","last_unstale":"2024-05-10T10:41:16.773942+0000","last_undegraded":"2024-05-10T10:41:16.773942+0000","last_fullsized":"2024-05-10T10:41:16.773942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.781752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935875+0000","last_change":"2024-05-10T10:41:15.778402+0000","last_active":"2024-05-10T10:41:16.935875+0000","last_peered":"2024-05-10T10:41:16.935875+0000","last_clean":"2024-05-10T10:41:16.935875+0000","last_became_active":"2024-05-10T10:41:15.778157+0000","last_became_peered":"2024-05-10T10:41:15.778157+0000","last_unstale":"2024-05-10T10:41:16.935875+0000","last_undegraded":"2024-05-10T10:41:16.935875+0000","last_fullsized":"2024-05-10T10:41:16.935875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:12.275951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777005+0000","last_change":"2024-05-10T10:41:14.769181+0000","last_active":"2024-05-10T10:41:16.777005+0000","last_peered":"2024-05-10T10:41:16.777005+0000","last_clean":"2024-05-10T10:41:16.777005+0000","last_became_active":"2024-05-10T10:41:14.768894+0000","last_became_peered":"2024-05-10T10:41:14.768894+0000","last_unstale":"2024-05-10T10:41:16.777005+0000","last_undegraded":"2024-05-10T10:41:16.777005+0000","last_fullsized":"2024-05-10T10:41:16.777005+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:23.929251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443655+0000","last_change":"2024-05-10T10:41:15.775989+0000","last_active":"2024-05-10T10:41:17.443655+0000","last_peered":"2024-05-10T10:41:17.443655+0000","last_clean":"2024-05-10T10:41:17.443655+0000","last_became_active":"2024-05-10T10:41:15.775422+0000","last_became_peered":"2024-05-10T10:41:15.775422+0000","last_unstale":"2024-05-10T10:41:17.443655+0000","last_undegraded":"2024-05-10T10:41:17.443655+0000","last_fullsized":"2024-05-10T10:41:17.443655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628929+0000","last_change":"2024-05-10T10:41:14.770676+0000","last_active":"2024-05-10T10:41:17.628929+0000","last_peered":"2024-05-10T10:41:17.628929+0000","last_clean":"2024-05-10T10:41:17.628929+0000","last_became_active":"2024-05-10T10:41:14.770348+0000","last_became_peered":"2024-05-10T10:41:14.770348+0000","last_unstale":"2024-05-10T10:41:17.628929+0000","last_undegraded":"2024-05-10T10:41:17.628929+0000","last_fullsized":"2024-05-10T10:41:17.628929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:12.593453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776179+0000","last_change":"2024-05-10T10:41:15.777201+0000","last_active":"2024-05-10T10:41:16.776179+0000","last_peered":"2024-05-10T10:41:16.776179+0000","last_clean":"2024-05-10T10:41:16.776179+0000","last_became_active":"2024-05-10T10:41:15.776849+0000","last_became_peered":"2024-05-10T10:41:15.776849+0000","last_unstale":"2024-05-10T10:41:16.776179+0000","last_undegraded":"2024-05-10T10:41:16.776179+0000","last_fullsized":"2024-05-10T10:41:16.776179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443695+0000","last_change":"2024-05-10T10:41:14.768756+0000","last_active":"2024-05-10T10:41:17.443695+0000","last_peered":"2024-05-10T10:41:17.443695+0000","last_clean":"2024-05-10T10:41:17.443695+0000","last_became_active":"2024-05-10T10:41:14.768454+0000","last_became_peered":"2024-05-10T10:41:14.768454+0000","last_unstale":"2024-05-10T10:41:17.443695+0000","last_undegraded":"2024-05-10T10:41:17.443695+0000","last_fullsized":"2024-05-10T10:41:17.443695+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:38:41.192594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628584+0000","last_change":"2024-05-10T10:41:15.782260+0000","last_active":"2024-05-10T10:41:17.628584+0000","last_peered":"2024-05-10T10:41:17.628584+0000","last_clean":"2024-05-10T10:41:17.628584+0000","last_became_active":"2024-05-10T10:41:15.781671+0000","last_became_peered":"2024-05-10T10:41:15.781671+0000","last_unstale":"2024-05-10T10:41:17.628584+0000","last_undegraded":"2024-05-10T10:41:17.628584+0000","last_fullsized":"2024-05-10T10:41:17.628584+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:13.788111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628425+0000","last_change":"2024-05-10T10:41:14.773906+0000","last_active":"2024-05-10T10:41:17.628425+0000","last_peered":"2024-05-10T10:41:17.628425+0000","last_clean":"2024-05-10T10:41:17.628425+0000","last_became_active":"2024-05-10T10:41:14.773612+0000","last_became_peered":"2024-05-10T10:41:14.773612+0000","last_unstale":"2024-05-10T10:41:17.628425+0000","last_undegraded":"2024-05-10T10:41:17.628425+0000","last_fullsized":"2024-05-10T10:41:17.628425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:12.785813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936110+0000","last_change":"2024-05-10T10:41:15.772932+0000","last_active":"2024-05-10T10:41:16.936110+0000","last_peered":"2024-05-10T10:41:16.936110+0000","last_clean":"2024-05-10T10:41:16.936110+0000","last_became_active":"2024-05-10T10:41:15.772531+0000","last_became_peered":"2024-05-10T10:41:15.772531+0000","last_unstale":"2024-05-10T10:41:16.936110+0000","last_undegraded":"2024-05-10T10:41:16.936110+0000","last_fullsized":"2024-05-10T10:41:16.936110+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:18:44.800947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935829+0000","last_change":"2024-05-10T10:41:14.778067+0000","last_active":"2024-05-10T10:41:16.935829+0000","last_peered":"2024-05-10T10:41:16.935829+0000","last_clean":"2024-05-10T10:41:16.935829+0000","last_became_active":"2024-05-10T10:41:14.777823+0000","last_became_peered":"2024-05-10T10:41:14.777823+0000","last_unstale":"2024-05-10T10:41:16.935829+0000","last_undegraded":"2024-05-10T10:41:16.935829+0000","last_fullsized":"2024-05-10T10:41:16.935829+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:59.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774310+0000","last_change":"2024-05-10T10:41:15.778732+0000","last_active":"2024-05-10T10:41:16.774310+0000","last_peered":"2024-05-10T10:41:16.774310+0000","last_clean":"2024-05-10T10:41:16.774310+0000","last_became_active":"2024-05-10T10:41:15.778168+0000","last_became_peered":"2024-05-10T10:41:15.778168+0000","last_unstale":"2024-05-10T10:41:16.774310+0000","last_undegraded":"2024-05-10T10:41:16.774310+0000","last_fullsized":"2024-05-10T10:41:16.774310+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:30:05.827338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628015+0000","last_change":"2024-05-10T10:41:14.771181+0000","last_active":"2024-05-10T10:41:17.628015+0000","last_peered":"2024-05-10T10:41:17.628015+0000","last_clean":"2024-05-10T10:41:17.628015+0000","last_became_active":"2024-05-10T10:41:14.770896+0000","last_became_peered":"2024-05-10T10:41:14.770896+0000","last_unstale":"2024-05-10T10:41:17.628015+0000","last_undegraded":"2024-05-10T10:41:17.628015+0000","last_fullsized":"2024-05-10T10:41:17.628015+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:36:14.116695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935901+0000","last_change":"2024-05-10T10:41:15.771784+0000","last_active":"2024-05-10T10:41:16.935901+0000","last_peered":"2024-05-10T10:41:16.935901+0000","last_clean":"2024-05-10T10:41:16.935901+0000","last_became_active":"2024-05-10T10:41:15.771539+0000","last_became_peered":"2024-05-10T10:41:15.771539+0000","last_unstale":"2024-05-10T10:41:16.935901+0000","last_undegraded":"2024-05-10T10:41:16.935901+0000","last_fullsized":"2024-05-10T10:41:16.935901+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:06:13.500814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628768+0000","last_change":"2024-05-10T10:41:14.776190+0000","last_active":"2024-05-10T10:41:17.628768+0000","last_peered":"2024-05-10T10:41:17.628768+0000","last_clean":"2024-05-10T10:41:17.628768+0000","last_became_active":"2024-05-10T10:41:14.775878+0000","last_became_peered":"2024-05-10T10:41:14.775878+0000","last_unstale":"2024-05-10T10:41:17.628768+0000","last_undegraded":"2024-05-10T10:41:17.628768+0000","last_fullsized":"2024-05-10T10:41:17.628768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:23.662520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628380+0000","last_change":"2024-05-10T10:41:15.782482+0000","last_active":"2024-05-10T10:41:17.628380+0000","last_peered":"2024-05-10T10:41:17.628380+0000","last_clean":"2024-05-10T10:41:17.628380+0000","last_became_active":"2024-05-10T10:41:15.781922+0000","last_became_peered":"2024-05-10T10:41:15.781922+0000","last_unstale":"2024-05-10T10:41:17.628380+0000","last_undegraded":"2024-05-10T10:41:17.628380+0000","last_fullsized":"2024-05-10T10:41:17.628380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:06:58.901684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777094+0000","last_change":"2024-05-10T10:41:14.766018+0000","last_active":"2024-05-10T10:41:16.777094+0000","last_peered":"2024-05-10T10:41:16.777094+0000","last_clean":"2024-05-10T10:41:16.777094+0000","last_became_active":"2024-05-10T10:41:14.765716+0000","last_became_peered":"2024-05-10T10:41:14.765716+0000","last_unstale":"2024-05-10T10:41:16.777094+0000","last_undegraded":"2024-05-10T10:41:16.777094+0000","last_fullsized":"2024-05-10T10:41:16.777094+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:24.270566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443550+0000","last_change":"2024-05-10T10:41:15.776187+0000","last_active":"2024-05-10T10:41:17.443550+0000","last_peered":"2024-05-10T10:41:17.443550+0000","last_clean":"2024-05-10T10:41:17.443550+0000","last_became_active":"2024-05-10T10:41:15.775748+0000","last_became_peered":"2024-05-10T10:41:15.775748+0000","last_unstale":"2024-05-10T10:41:17.443550+0000","last_undegraded":"2024-05-10T10:41:17.443550+0000","last_fullsized":"2024-05-10T10:41:17.443550+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776513+0000","last_change":"2024-05-10T10:41:14.769976+0000","last_active":"2024-05-10T10:41:16.776513+0000","last_peered":"2024-05-10T10:41:16.776513+0000","last_clean":"2024-05-10T10:41:16.776513+0000","last_became_active":"2024-05-10T10:41:14.769684+0000","last_became_peered":"2024-05-10T10:41:14.769684+0000","last_unstale":"2024-05-10T10:41:16.776513+0000","last_undegraded":"2024-05-10T10:41:16.776513+0000","last_fullsized":"2024-05-10T10:41:16.776513+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:51.351714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773938+0000","last_change":"2024-05-10T10:41:15.777885+0000","last_active":"2024-05-10T10:41:16.773938+0000","last_peered":"2024-05-10T10:41:16.773938+0000","last_clean":"2024-05-10T10:41:16.773938+0000","last_became_active":"2024-05-10T10:41:15.777644+0000","last_became_peered":"2024-05-10T10:41:15.777644+0000","last_unstale":"2024-05-10T10:41:16.773938+0000","last_undegraded":"2024-05-10T10:41:16.773938+0000","last_fullsized":"2024-05-10T10:41:16.773938+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:32.623167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936177+0000","last_change":"2024-05-10T10:41:14.780386+0000","last_active":"2024-05-10T10:41:16.936177+0000","last_peered":"2024-05-10T10:41:16.936177+0000","last_clean":"2024-05-10T10:41:16.936177+0000","last_became_active":"2024-05-10T10:41:14.778333+0000","last_became_peered":"2024-05-10T10:41:14.778333+0000","last_unstale":"2024-05-10T10:41:16.936177+0000","last_undegraded":"2024-05-10T10:41:16.936177+0000","last_fullsized":"2024-05-10T10:41:16.936177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:48:12.560346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774166+0000","last_change":"2024-05-10T10:41:15.781182+0000","last_active":"2024-05-10T10:41:16.774166+0000","last_peered":"2024-05-10T10:41:16.774166+0000","last_clean":"2024-05-10T10:41:16.774166+0000","last_became_active":"2024-05-10T10:41:15.780911+0000","last_became_peered":"2024-05-10T10:41:15.780911+0000","last_unstale":"2024-05-10T10:41:16.774166+0000","last_undegraded":"2024-05-10T10:41:16.774166+0000","last_fullsized":"2024-05-10T10:41:16.774166+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:11.537759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935802+0000","last_change":"2024-05-10T10:41:14.779340+0000","last_active":"2024-05-10T10:41:16.935802+0000","last_peered":"2024-05-10T10:41:16.935802+0000","last_clean":"2024-05-10T10:41:16.935802+0000","last_became_active":"2024-05-10T10:41:14.778986+0000","last_became_peered":"2024-05-10T10:41:14.778986+0000","last_unstale":"2024-05-10T10:41:16.935802+0000","last_undegraded":"2024-05-10T10:41:16.935802+0000","last_fullsized":"2024-05-10T10:41:16.935802+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:55.182479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628767+0000","last_change":"2024-05-10T10:41:15.786444+0000","last_active":"2024-05-10T10:41:17.628767+0000","last_peered":"2024-05-10T10:41:17.628767+0000","last_clean":"2024-05-10T10:41:17.628767+0000","last_became_active":"2024-05-10T10:41:15.786163+0000","last_became_peered":"2024-05-10T10:41:15.786163+0000","last_unstale":"2024-05-10T10:41:17.628767+0000","last_undegraded":"2024-05-10T10:41:17.628767+0000","last_fullsized":"2024-05-10T10:41:17.628767+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:12.413148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628649+0000","last_change":"2024-05-10T10:41:14.774675+0000","last_active":"2024-05-10T10:41:17.628649+0000","last_peered":"2024-05-10T10:41:17.628649+0000","last_clean":"2024-05-10T10:41:17.628649+0000","last_became_active":"2024-05-10T10:41:14.774374+0000","last_became_peered":"2024-05-10T10:41:14.774374+0000","last_unstale":"2024-05-10T10:41:17.628649+0000","last_undegraded":"2024-05-10T10:41:17.628649+0000","last_fullsized":"2024-05-10T10:41:17.628649+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:37:04.797470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936091+0000","last_change":"2024-05-10T10:41:15.777186+0000","last_active":"2024-05-10T10:41:16.936091+0000","last_peered":"2024-05-10T10:41:16.936091+0000","last_clean":"2024-05-10T10:41:16.936091+0000","last_became_active":"2024-05-10T10:41:15.776896+0000","last_became_peered":"2024-05-10T10:41:15.776896+0000","last_unstale":"2024-05-10T10:41:16.936091+0000","last_undegraded":"2024-05-10T10:41:16.936091+0000","last_fullsized":"2024-05-10T10:41:16.936091+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:46.567398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935938+0000","last_change":"2024-05-10T10:41:14.779920+0000","last_active":"2024-05-10T10:41:16.935938+0000","last_peered":"2024-05-10T10:41:16.935938+0000","last_clean":"2024-05-10T10:41:16.935938+0000","last_became_active":"2024-05-10T10:41:14.779652+0000","last_became_peered":"2024-05-10T10:41:14.779652+0000","last_unstale":"2024-05-10T10:41:16.935938+0000","last_undegraded":"2024-05-10T10:41:16.935938+0000","last_fullsized":"2024-05-10T10:41:16.935938+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:53:18.910071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773986+0000","last_change":"2024-05-10T10:41:15.777834+0000","last_active":"2024-05-10T10:41:16.773986+0000","last_peered":"2024-05-10T10:41:16.773986+0000","last_clean":"2024-05-10T10:41:16.773986+0000","last_became_active":"2024-05-10T10:41:15.777551+0000","last_became_peered":"2024-05-10T10:41:15.777551+0000","last_unstale":"2024-05-10T10:41:16.773986+0000","last_undegraded":"2024-05-10T10:41:16.773986+0000","last_fullsized":"2024-05-10T10:41:16.773986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:13:48.075231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443641+0000","last_change":"2024-05-10T10:41:14.770637+0000","last_active":"2024-05-10T10:41:17.443641+0000","last_peered":"2024-05-10T10:41:17.443641+0000","last_clean":"2024-05-10T10:41:17.443641+0000","last_became_active":"2024-05-10T10:41:14.770317+0000","last_became_peered":"2024-05-10T10:41:14.770317+0000","last_unstale":"2024-05-10T10:41:17.443641+0000","last_undegraded":"2024-05-10T10:41:17.443641+0000","last_fullsized":"2024-05-10T10:41:17.443641+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:10.949088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628212+0000","last_change":"2024-05-10T10:41:15.780646+0000","last_active":"2024-05-10T10:41:17.628212+0000","last_peered":"2024-05-10T10:41:17.628212+0000","last_clean":"2024-05-10T10:41:17.628212+0000","last_became_active":"2024-05-10T10:41:15.780390+0000","last_became_peered":"2024-05-10T10:41:15.780390+0000","last_unstale":"2024-05-10T10:41:17.628212+0000","last_undegraded":"2024-05-10T10:41:17.628212+0000","last_fullsized":"2024-05-10T10:41:17.628212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:13.788111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.765518+0000","last_change":"2024-05-10T10:41:14.775314+0000","last_active":"2024-05-10T10:41:17.765518+0000","last_peered":"2024-05-10T10:41:17.765518+0000","last_clean":"2024-05-10T10:41:17.765518+0000","last_became_active":"2024-05-10T10:41:14.774832+0000","last_became_peered":"2024-05-10T10:41:14.774832+0000","last_unstale":"2024-05-10T10:41:17.765518+0000","last_undegraded":"2024-05-10T10:41:17.765518+0000","last_fullsized":"2024-05-10T10:41:17.765518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:42:37.625354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628391+0000","last_change":"2024-05-10T10:41:15.785689+0000","last_active":"2024-05-10T10:41:17.628391+0000","last_peered":"2024-05-10T10:41:17.628391+0000","last_clean":"2024-05-10T10:41:17.628391+0000","last_became_active":"2024-05-10T10:41:15.785422+0000","last_became_peered":"2024-05-10T10:41:15.785422+0000","last_unstale":"2024-05-10T10:41:17.628391+0000","last_undegraded":"2024-05-10T10:41:17.628391+0000","last_fullsized":"2024-05-10T10:41:17.628391+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:26.540192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628662+0000","last_change":"2024-05-10T10:41:14.775234+0000","last_active":"2024-05-10T10:41:17.628662+0000","last_peered":"2024-05-10T10:41:17.628662+0000","last_clean":"2024-05-10T10:41:17.628662+0000","last_became_active":"2024-05-10T10:41:14.774959+0000","last_became_peered":"2024-05-10T10:41:14.774959+0000","last_unstale":"2024-05-10T10:41:17.628662+0000","last_undegraded":"2024-05-10T10:41:17.628662+0000","last_fullsized":"2024-05-10T10:41:17.628662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:27:56.760836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776256+0000","last_change":"2024-05-10T10:41:15.777117+0000","last_active":"2024-05-10T10:41:16.776256+0000","last_peered":"2024-05-10T10:41:16.776256+0000","last_clean":"2024-05-10T10:41:16.776256+0000","last_became_active":"2024-05-10T10:41:15.776833+0000","last_became_peered":"2024-05-10T10:41:15.776833+0000","last_unstale":"2024-05-10T10:41:16.776256+0000","last_undegraded":"2024-05-10T10:41:16.776256+0000","last_fullsized":"2024-05-10T10:41:16.776256+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443611+0000","last_change":"2024-05-10T10:41:14.775323+0000","last_active":"2024-05-10T10:41:17.443611+0000","last_peered":"2024-05-10T10:41:17.443611+0000","last_clean":"2024-05-10T10:41:17.443611+0000","last_became_active":"2024-05-10T10:41:14.775045+0000","last_became_peered":"2024-05-10T10:41:14.775045+0000","last_unstale":"2024-05-10T10:41:17.443611+0000","last_undegraded":"2024-05-10T10:41:17.443611+0000","last_fullsized":"2024-05-10T10:41:17.443611+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:36:14.116695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936158+0000","last_change":"2024-05-10T10:41:15.775892+0000","last_active":"2024-05-10T10:41:16.936158+0000","last_peered":"2024-05-10T10:41:16.936158+0000","last_clean":"2024-05-10T10:41:16.936158+0000","last_became_active":"2024-05-10T10:41:15.775631+0000","last_became_peered":"2024-05-10T10:41:15.775631+0000","last_unstale":"2024-05-10T10:41:16.936158+0000","last_undegraded":"2024-05-10T10:41:16.936158+0000","last_fullsized":"2024-05-10T10:41:16.936158+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:00:56.386129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628117+0000","last_change":"2024-05-10T10:41:14.774153+0000","last_active":"2024-05-10T10:41:17.628117+0000","last_peered":"2024-05-10T10:41:17.628117+0000","last_clean":"2024-05-10T10:41:17.628117+0000","last_became_active":"2024-05-10T10:41:14.773869+0000","last_became_peered":"2024-05-10T10:41:14.773869+0000","last_unstale":"2024-05-10T10:41:17.628117+0000","last_undegraded":"2024-05-10T10:41:17.628117+0000","last_fullsized":"2024-05-10T10:41:17.628117+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:10.949088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936145+0000","last_change":"2024-05-10T10:41:15.776526+0000","last_active":"2024-05-10T10:41:16.936145+0000","last_peered":"2024-05-10T10:41:16.936145+0000","last_clean":"2024-05-10T10:41:16.936145+0000","last_became_active":"2024-05-10T10:41:15.776272+0000","last_became_peered":"2024-05-10T10:41:15.776272+0000","last_unstale":"2024-05-10T10:41:16.936145+0000","last_undegraded":"2024-05-10T10:41:16.936145+0000","last_fullsized":"2024-05-10T10:41:16.936145+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:36:51.010333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936116+0000","last_change":"2024-05-10T10:41:14.777081+0000","last_active":"2024-05-10T10:41:16.936116+0000","last_peered":"2024-05-10T10:41:16.936116+0000","last_clean":"2024-05-10T10:41:16.936116+0000","last_became_active":"2024-05-10T10:41:14.776821+0000","last_became_peered":"2024-05-10T10:41:14.776821+0000","last_unstale":"2024-05-10T10:41:16.936116+0000","last_undegraded":"2024-05-10T10:41:16.936116+0000","last_fullsized":"2024-05-10T10:41:16.936116+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:57.365909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776834+0000","last_change":"2024-05-10T10:41:15.766728+0000","last_active":"2024-05-10T10:41:16.776834+0000","last_peered":"2024-05-10T10:41:16.776834+0000","last_clean":"2024-05-10T10:41:16.776834+0000","last_became_active":"2024-05-10T10:41:15.765213+0000","last_became_peered":"2024-05-10T10:41:15.765213+0000","last_unstale":"2024-05-10T10:41:16.776834+0000","last_undegraded":"2024-05-10T10:41:16.776834+0000","last_fullsized":"2024-05-10T10:41:16.776834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:12.413148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628656+0000","last_change":"2024-05-10T10:41:14.775821+0000","last_active":"2024-05-10T10:41:17.628656+0000","last_peered":"2024-05-10T10:41:17.628656+0000","last_clean":"2024-05-10T10:41:17.628656+0000","last_became_active":"2024-05-10T10:41:14.775438+0000","last_became_peered":"2024-05-10T10:41:14.775438+0000","last_unstale":"2024-05-10T10:41:17.628656+0000","last_undegraded":"2024-05-10T10:41:17.628656+0000","last_fullsized":"2024-05-10T10:41:17.628656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:19.119069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776665+0000","last_change":"2024-05-10T10:41:15.765624+0000","last_active":"2024-05-10T10:41:16.776665+0000","last_peered":"2024-05-10T10:41:16.776665+0000","last_clean":"2024-05-10T10:41:16.776665+0000","last_became_active":"2024-05-10T10:41:15.765326+0000","last_became_peered":"2024-05-10T10:41:15.765326+0000","last_unstale":"2024-05-10T10:41:16.776665+0000","last_undegraded":"2024-05-10T10:41:16.776665+0000","last_fullsized":"2024-05-10T10:41:16.776665+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:09:24.288572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936011+0000","last_change":"2024-05-10T10:41:14.772638+0000","last_active":"2024-05-10T10:41:16.936011+0000","last_peered":"2024-05-10T10:41:16.936011+0000","last_clean":"2024-05-10T10:41:16.936011+0000","last_became_active":"2024-05-10T10:41:14.772333+0000","last_became_peered":"2024-05-10T10:41:14.772333+0000","last_unstale":"2024-05-10T10:41:16.936011+0000","last_undegraded":"2024-05-10T10:41:16.936011+0000","last_fullsized":"2024-05-10T10:41:16.936011+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:24.849979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.775959+0000","last_change":"2024-05-10T10:41:15.771722+0000","last_active":"2024-05-10T10:41:16.775959+0000","last_peered":"2024-05-10T10:41:16.775959+0000","last_clean":"2024-05-10T10:41:16.775959+0000","last_became_active":"2024-05-10T10:41:15.771453+0000","last_became_peered":"2024-05-10T10:41:15.771453+0000","last_unstale":"2024-05-10T10:41:16.775959+0000","last_undegraded":"2024-05-10T10:41:16.775959+0000","last_fullsized":"2024-05-10T10:41:16.775959+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936209+0000","last_change":"2024-05-10T10:41:14.779327+0000","last_active":"2024-05-10T10:41:16.936209+0000","last_peered":"2024-05-10T10:41:16.936209+0000","last_clean":"2024-05-10T10:41:16.936209+0000","last_became_active":"2024-05-10T10:41:14.779018+0000","last_became_peered":"2024-05-10T10:41:14.779018+0000","last_unstale":"2024-05-10T10:41:16.936209+0000","last_undegraded":"2024-05-10T10:41:16.936209+0000","last_fullsized":"2024-05-10T10:41:16.936209+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:36:12.011464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":279,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777008+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:41:16.777008+0000","last_peered":"2024-05-10T10:41:16.777008+0000","last_clean":"2024-05-10T10:41:16.777008+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:41:16.777008+0000","last_undegraded":"2024-05-10T10:41:16.777008+0000","last_fullsized":"2024-05-10T10:41:16.777008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1181960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":586956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":591432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":584832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":592014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":591466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":584774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":1183502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:41:19.829 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:41:19.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.966+0000 7fb67b511640 1 Processor -- start 2024-05-10T10:41:19.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.966+0000 7fb67b511640 1 -- start start 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb67b511640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb6740b8860 0x7fb67406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb67b511640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb6740cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb67b511640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6740cfbb0 0x7fb6740d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb67b511640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb67406ee50 con 0x7fb6740cfbb0 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb67b511640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb67415fc90 con 0x7fb6740b8860 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb67b511640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb6740b8c40 con 0x7fb6740d0230 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb679aae640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6740cfbb0 0x7fb6740d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb679aae640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6740cfbb0 0x7fb6740d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50968/0 (socket says 172.21.15.53:50968) 2024-05-10T10:41:19.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb679aae640 1 -- 172.21.15.53:0/1687968910 learned_addr learned my addr 172.21.15.53:0/1687968910 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb678aac640 1 --2- 172.21.15.53:0/1687968910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb6740cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.967+0000 7fb6792ad640 1 --2- 172.21.15.53:0/1687968910 >> v2:172.21.15.135:3300/0 conn(0x7fb6740b8860 0x7fb67406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb679aae640 1 -- 172.21.15.53:0/1687968910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 msgr2=0x7fb6740cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb679aae640 1 --2- 172.21.15.53:0/1687968910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb6740cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb679aae640 1 -- 172.21.15.53:0/1687968910 >> v2:172.21.15.135:3300/0 conn(0x7fb6740b8860 msgr2=0x7fb67406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb679aae640 1 --2- 172.21.15.53:0/1687968910 >> v2:172.21.15.135:3300/0 conn(0x7fb6740b8860 0x7fb67406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb679aae640 1 -- 172.21.15.53:0/1687968910 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6740cd570 con 0x7fb6740cfbb0 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb6792ad640 1 --2- 172.21.15.53:0/1687968910 >> v2:172.21.15.135:3300/0 conn(0x7fb6740b8860 0x7fb67406e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb678aac640 1 --2- 172.21.15.53:0/1687968910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb6740cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb679aae640 1 --2- 172.21.15.53:0/1687968910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb6740d0cf0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb6740730a0 tx=0x7fb66c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edb8559c0b301ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.968+0000 7fb66bfff640 1 -- 172.21.15.53:0/1687968910 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c036070 con 0x7fb6740cfbb0 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb66bfff640 1 -- 172.21.15.53:0/1687968910 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66c031600 con 0x7fb6740cfbb0 2024-05-10T10:41:19.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb66bfff640 1 -- 172.21.15.53:0/1687968910 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0319d0 con 0x7fb6740cfbb0 2024-05-10T10:41:19.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 -- 172.21.15.53:0/1687968910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 msgr2=0x7fb6740d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 --2- 172.21.15.53:0/1687968910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb6740d0cf0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb6740730a0 tx=0x7fb66c02a630 comp rx=0 tx=0).stop 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 -- 172.21.15.53:0/1687968910 shutdown_connections 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 --2- 172.21.15.53:0/1687968910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb6740d0cf0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 --2- 172.21.15.53:0/1687968910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb6740cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 --2- 172.21.15.53:0/1687968910 >> v2:172.21.15.135:3300/0 conn(0x7fb6740b8860 0x7fb67406e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 -- 172.21.15.53:0/1687968910 >> 172.21.15.53:0/1687968910 conn(0x7fb6740d2df0 msgr2=0x7fb6740d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 -- 172.21.15.53:0/1687968910 shutdown_connections 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.969+0000 7fb67b511640 1 -- 172.21.15.53:0/1687968910 wait complete. 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.970+0000 7fb67b511640 1 Processor -- start 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.970+0000 7fb67b511640 1 -- start start 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb67b511640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6740b8860 0x7fb674154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb67b511640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb674160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb67b511640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb674160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb67b511640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6740cde10 con 0x7fb6740cfbb0 2024-05-10T10:41:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb67b511640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6740cdb30 con 0x7fb6740b8860 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb67b511640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb67406ea60 con 0x7fb6740d0230 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb678aac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb674160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb678aac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb674160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50984/0 (socket says 172.21.15.53:50984) 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.971+0000 7fb678aac640 1 -- 172.21.15.53:0/1207137309 learned_addr learned my addr 172.21.15.53:0/1207137309 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb678aac640 1 -- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 msgr2=0x7fb674160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb678aac640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb674160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb678aac640 1 -- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6740b8860 msgr2=0x7fb674154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb6792ad640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6740b8860 0x7fb674154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb678aac640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6740b8860 0x7fb674154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb678aac640 1 -- 172.21.15.53:0/1207137309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6740cb8f0 con 0x7fb6740cfbb0 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb678aac640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb674160650 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb66400e7e0 tx=0x7fb66400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb669ffb640 1 -- 172.21.15.53:0/1207137309 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66400f040 con 0x7fb6740cfbb0 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb669ffb640 1 -- 172.21.15.53:0/1207137309 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb664004030 con 0x7fb6740cfbb0 2024-05-10T10:41:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6740cbbd0 con 0x7fb6740cfbb0 2024-05-10T10:41:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb669ffb640 1 -- 172.21.15.53:0/1207137309 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664004400 con 0x7fb6740cfbb0 2024-05-10T10:41:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.972+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6740cc190 con 0x7fb6740cfbb0 2024-05-10T10:41:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.973+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67406a220 con 0x7fb6740cfbb0 2024-05-10T10:41:19.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.974+0000 7fb669ffb640 1 -- 172.21.15.53:0/1207137309 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb66400c670 con 0x7fb6740cfbb0 2024-05-10T10:41:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.974+0000 7fb669ffb640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6580762d0 0x7fb658078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.974+0000 7fb669ffb640 1 -- 172.21.15.53:0/1207137309 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7fb6640975c0 con 0x7fb6740cfbb0 2024-05-10T10:41:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.975+0000 7fb6792ad640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6580762d0 0x7fb658078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.975+0000 7fb6792ad640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6580762d0 0x7fb658078790 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb670006fd0 tx=0x7fb670008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:19.979+0000 7fb669ffb640 1 -- 172.21.15.53:0/1207137309 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb664017020 con 0x7fb6740cfbb0 2024-05-10T10:41:20.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.164+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb67406e7f0 con 0x7fb6580762d0 2024-05-10T10:41:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.171+0000 7fb669ffb640 1 -- 172.21.15.53:0/1207137309 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7fb67406e7f0 con 0x7fb6580762d0 2024-05-10T10:41:20.175 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:41:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6580762d0 msgr2=0x7fb658078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6580762d0 0x7fb658078790 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb670006fd0 tx=0x7fb670008040 comp rx=0 tx=0).stop 2024-05-10T10:41:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 msgr2=0x7fb674160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb674160650 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb66400e7e0 tx=0x7fb66400eca0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 shutdown_connections 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb6580762d0 0x7fb658078790 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6740d0230 0x7fb674160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6740cfbb0 0x7fb674160650 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 --2- 172.21.15.53:0/1207137309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6740b8860 0x7fb674154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 >> 172.21.15.53:0/1207137309 conn(0x7fb6740d2df0 msgr2=0x7fb6740d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 shutdown_connections 2024-05-10T10:41:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.174+0000 7fb67b511640 1 -- 172.21.15.53:0/1207137309 wait complete. 2024-05-10T10:41:20.187 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-05-10T10:41:18.270798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260632,"kb_used_data":11172,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714088,"statfs":{"total":773094113280,"available":772827226112,"internally_reserved":0,"allocated":11440128,"data_stored":5896936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.519135"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774193+0000","last_change":"2024-05-10T10:41:15.778935+0000","last_active":"2024-05-10T10:41:16.774193+0000","last_peered":"2024-05-10T10:41:16.774193+0000","last_clean":"2024-05-10T10:41:16.774193+0000","last_became_active":"2024-05-10T10:41:15.778493+0000","last_became_peered":"2024-05-10T10:41:15.778493+0000","last_unstale":"2024-05-10T10:41:16.774193+0000","last_undegraded":"2024-05-10T10:41:16.774193+0000","last_fullsized":"2024-05-10T10:41:16.774193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:24.554910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628783+0000","last_change":"2024-05-10T10:41:15.783290+0000","last_active":"2024-05-10T10:41:17.628783+0000","last_peered":"2024-05-10T10:41:17.628783+0000","last_clean":"2024-05-10T10:41:17.628783+0000","last_became_active":"2024-05-10T10:41:15.782785+0000","last_became_peered":"2024-05-10T10:41:15.782785+0000","last_unstale":"2024-05-10T10:41:17.628783+0000","last_undegraded":"2024-05-10T10:41:17.628783+0000","last_fullsized":"2024-05-10T10:41:17.628783+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:24.931549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773996+0000","last_change":"2024-05-10T10:41:15.781495+0000","last_active":"2024-05-10T10:41:16.773996+0000","last_peered":"2024-05-10T10:41:16.773996+0000","last_clean":"2024-05-10T10:41:16.773996+0000","last_became_active":"2024-05-10T10:41:15.780697+0000","last_became_peered":"2024-05-10T10:41:15.780697+0000","last_unstale":"2024-05-10T10:41:16.773996+0000","last_undegraded":"2024-05-10T10:41:16.773996+0000","last_fullsized":"2024-05-10T10:41:16.773996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:11:15.016802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628610+0000","last_change":"2024-05-10T10:41:15.780359+0000","last_active":"2024-05-10T10:41:17.628610+0000","last_peered":"2024-05-10T10:41:17.628610+0000","last_clean":"2024-05-10T10:41:17.628610+0000","last_became_active":"2024-05-10T10:41:15.780115+0000","last_became_peered":"2024-05-10T10:41:15.780115+0000","last_unstale":"2024-05-10T10:41:17.628610+0000","last_undegraded":"2024-05-10T10:41:17.628610+0000","last_fullsized":"2024-05-10T10:41:17.628610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:50.245204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443392+0000","last_change":"2024-05-10T10:41:15.779027+0000","last_active":"2024-05-10T10:41:17.443392+0000","last_peered":"2024-05-10T10:41:17.443392+0000","last_clean":"2024-05-10T10:41:17.443392+0000","last_became_active":"2024-05-10T10:41:15.778778+0000","last_became_peered":"2024-05-10T10:41:15.778778+0000","last_unstale":"2024-05-10T10:41:17.443392+0000","last_undegraded":"2024-05-10T10:41:17.443392+0000","last_fullsized":"2024-05-10T10:41:17.443392+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:24.931549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628232+0000","last_change":"2024-05-10T10:41:15.783523+0000","last_active":"2024-05-10T10:41:17.628232+0000","last_peered":"2024-05-10T10:41:17.628232+0000","last_clean":"2024-05-10T10:41:17.628232+0000","last_became_active":"2024-05-10T10:41:15.783255+0000","last_became_peered":"2024-05-10T10:41:15.783255+0000","last_unstale":"2024-05-10T10:41:17.628232+0000","last_undegraded":"2024-05-10T10:41:17.628232+0000","last_fullsized":"2024-05-10T10:41:17.628232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:52:43.737629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628223+0000","last_change":"2024-05-10T10:41:15.782753+0000","last_active":"2024-05-10T10:41:17.628223+0000","last_peered":"2024-05-10T10:41:17.628223+0000","last_clean":"2024-05-10T10:41:17.628223+0000","last_became_active":"2024-05-10T10:41:15.782129+0000","last_became_peered":"2024-05-10T10:41:15.782129+0000","last_unstale":"2024-05-10T10:41:17.628223+0000","last_undegraded":"2024-05-10T10:41:17.628223+0000","last_fullsized":"2024-05-10T10:41:17.628223+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:52:11.478952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935747+0000","last_change":"2024-05-10T10:41:15.775862+0000","last_active":"2024-05-10T10:41:16.935747+0000","last_peered":"2024-05-10T10:41:16.935747+0000","last_clean":"2024-05-10T10:41:16.935747+0000","last_became_active":"2024-05-10T10:41:15.775593+0000","last_became_peered":"2024-05-10T10:41:15.775593+0000","last_unstale":"2024-05-10T10:41:16.935747+0000","last_undegraded":"2024-05-10T10:41:16.935747+0000","last_fullsized":"2024-05-10T10:41:16.935747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:42:54.459998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628728+0000","last_change":"2024-05-10T10:41:15.783349+0000","last_active":"2024-05-10T10:41:17.628728+0000","last_peered":"2024-05-10T10:41:17.628728+0000","last_clean":"2024-05-10T10:41:17.628728+0000","last_became_active":"2024-05-10T10:41:15.783036+0000","last_became_peered":"2024-05-10T10:41:15.783036+0000","last_unstale":"2024-05-10T10:41:17.628728+0000","last_undegraded":"2024-05-10T10:41:17.628728+0000","last_fullsized":"2024-05-10T10:41:17.628728+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:35.161552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443668+0000","last_change":"2024-05-10T10:41:15.784171+0000","last_active":"2024-05-10T10:41:17.443668+0000","last_peered":"2024-05-10T10:41:17.443668+0000","last_clean":"2024-05-10T10:41:17.443668+0000","last_became_active":"2024-05-10T10:41:15.783892+0000","last_became_peered":"2024-05-10T10:41:15.783892+0000","last_unstale":"2024-05-10T10:41:17.443668+0000","last_undegraded":"2024-05-10T10:41:17.443668+0000","last_fullsized":"2024-05-10T10:41:17.443668+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776950+0000","last_change":"2024-05-10T10:41:15.784291+0000","last_active":"2024-05-10T10:41:16.776950+0000","last_peered":"2024-05-10T10:41:16.776950+0000","last_clean":"2024-05-10T10:41:16.776950+0000","last_became_active":"2024-05-10T10:41:15.784012+0000","last_became_peered":"2024-05-10T10:41:15.784012+0000","last_unstale":"2024-05-10T10:41:16.776950+0000","last_undegraded":"2024-05-10T10:41:16.776950+0000","last_fullsized":"2024-05-10T10:41:16.776950+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:02.498815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628324+0000","last_change":"2024-05-10T10:41:15.774082+0000","last_active":"2024-05-10T10:41:17.628324+0000","last_peered":"2024-05-10T10:41:17.628324+0000","last_clean":"2024-05-10T10:41:17.628324+0000","last_became_active":"2024-05-10T10:41:15.773775+0000","last_became_peered":"2024-05-10T10:41:15.773775+0000","last_unstale":"2024-05-10T10:41:17.628324+0000","last_undegraded":"2024-05-10T10:41:17.628324+0000","last_fullsized":"2024-05-10T10:41:17.628324+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:35.161552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628471+0000","last_change":"2024-05-10T10:41:15.782607+0000","last_active":"2024-05-10T10:41:17.628471+0000","last_peered":"2024-05-10T10:41:17.628471+0000","last_clean":"2024-05-10T10:41:17.628471+0000","last_became_active":"2024-05-10T10:41:15.782348+0000","last_became_peered":"2024-05-10T10:41:15.782348+0000","last_unstale":"2024-05-10T10:41:17.628471+0000","last_undegraded":"2024-05-10T10:41:17.628471+0000","last_fullsized":"2024-05-10T10:41:17.628471+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:02.498815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776986+0000","last_change":"2024-05-10T10:41:15.775645+0000","last_active":"2024-05-10T10:41:16.776986+0000","last_peered":"2024-05-10T10:41:16.776986+0000","last_clean":"2024-05-10T10:41:16.776986+0000","last_became_active":"2024-05-10T10:41:15.775351+0000","last_became_peered":"2024-05-10T10:41:15.775351+0000","last_unstale":"2024-05-10T10:41:16.776986+0000","last_undegraded":"2024-05-10T10:41:16.776986+0000","last_fullsized":"2024-05-10T10:41:16.776986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:49.090489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776807+0000","last_change":"2024-05-10T10:41:15.765127+0000","last_active":"2024-05-10T10:41:16.776807+0000","last_peered":"2024-05-10T10:41:16.776807+0000","last_clean":"2024-05-10T10:41:16.776807+0000","last_became_active":"2024-05-10T10:41:15.764804+0000","last_became_peered":"2024-05-10T10:41:15.764804+0000","last_unstale":"2024-05-10T10:41:16.776807+0000","last_undegraded":"2024-05-10T10:41:16.776807+0000","last_fullsized":"2024-05-10T10:41:16.776807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:09.552841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776637+0000","last_change":"2024-05-10T10:41:15.769947+0000","last_active":"2024-05-10T10:41:16.776637+0000","last_peered":"2024-05-10T10:41:16.776637+0000","last_clean":"2024-05-10T10:41:16.776637+0000","last_became_active":"2024-05-10T10:41:15.769634+0000","last_became_peered":"2024-05-10T10:41:15.769634+0000","last_unstale":"2024-05-10T10:41:16.776637+0000","last_undegraded":"2024-05-10T10:41:16.776637+0000","last_fullsized":"2024-05-10T10:41:16.776637+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:55:50.245204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774202+0000","last_change":"2024-05-10T10:41:15.779643+0000","last_active":"2024-05-10T10:41:16.774202+0000","last_peered":"2024-05-10T10:41:16.774202+0000","last_clean":"2024-05-10T10:41:16.774202+0000","last_became_active":"2024-05-10T10:41:15.779407+0000","last_became_peered":"2024-05-10T10:41:15.779407+0000","last_unstale":"2024-05-10T10:41:16.774202+0000","last_undegraded":"2024-05-10T10:41:16.774202+0000","last_fullsized":"2024-05-10T10:41:16.774202+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:23.327486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628865+0000","last_change":"2024-05-10T10:41:15.782386+0000","last_active":"2024-05-10T10:41:17.628865+0000","last_peered":"2024-05-10T10:41:17.628865+0000","last_clean":"2024-05-10T10:41:17.628865+0000","last_became_active":"2024-05-10T10:41:15.781984+0000","last_became_peered":"2024-05-10T10:41:15.781984+0000","last_unstale":"2024-05-10T10:41:17.628865+0000","last_undegraded":"2024-05-10T10:41:17.628865+0000","last_fullsized":"2024-05-10T10:41:17.628865+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774173+0000","last_change":"2024-05-10T10:41:15.788308+0000","last_active":"2024-05-10T10:41:16.774173+0000","last_peered":"2024-05-10T10:41:16.774173+0000","last_clean":"2024-05-10T10:41:16.774173+0000","last_became_active":"2024-05-10T10:41:15.788020+0000","last_became_peered":"2024-05-10T10:41:15.788020+0000","last_unstale":"2024-05-10T10:41:16.774173+0000","last_undegraded":"2024-05-10T10:41:16.774173+0000","last_fullsized":"2024-05-10T10:41:16.774173+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:59:18.078967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628344+0000","last_change":"2024-05-10T10:41:15.774087+0000","last_active":"2024-05-10T10:41:17.628344+0000","last_peered":"2024-05-10T10:41:17.628344+0000","last_clean":"2024-05-10T10:41:17.628344+0000","last_became_active":"2024-05-10T10:41:15.773769+0000","last_became_peered":"2024-05-10T10:41:15.773769+0000","last_unstale":"2024-05-10T10:41:17.628344+0000","last_undegraded":"2024-05-10T10:41:17.628344+0000","last_fullsized":"2024-05-10T10:41:17.628344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:24.931549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936020+0000","last_change":"2024-05-10T10:41:15.773651+0000","last_active":"2024-05-10T10:41:16.936020+0000","last_peered":"2024-05-10T10:41:16.936020+0000","last_clean":"2024-05-10T10:41:16.936020+0000","last_became_active":"2024-05-10T10:41:15.773375+0000","last_became_peered":"2024-05-10T10:41:15.773375+0000","last_unstale":"2024-05-10T10:41:16.936020+0000","last_undegraded":"2024-05-10T10:41:16.936020+0000","last_fullsized":"2024-05-10T10:41:16.936020+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:21.812285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443367+0000","last_change":"2024-05-10T10:41:15.775102+0000","last_active":"2024-05-10T10:41:17.443367+0000","last_peered":"2024-05-10T10:41:17.443367+0000","last_clean":"2024-05-10T10:41:17.443367+0000","last_became_active":"2024-05-10T10:41:15.774802+0000","last_became_peered":"2024-05-10T10:41:15.774802+0000","last_unstale":"2024-05-10T10:41:17.443367+0000","last_undegraded":"2024-05-10T10:41:17.443367+0000","last_fullsized":"2024-05-10T10:41:17.443367+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:02:35.161552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773832+0000","last_change":"2024-05-10T10:41:15.774238+0000","last_active":"2024-05-10T10:41:16.773832+0000","last_peered":"2024-05-10T10:41:16.773832+0000","last_clean":"2024-05-10T10:41:16.773832+0000","last_became_active":"2024-05-10T10:41:15.773993+0000","last_became_peered":"2024-05-10T10:41:15.773993+0000","last_unstale":"2024-05-10T10:41:16.773832+0000","last_undegraded":"2024-05-10T10:41:16.773832+0000","last_fullsized":"2024-05-10T10:41:16.773832+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:18:37.552832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443190+0000","last_change":"2024-05-10T10:41:15.790978+0000","last_active":"2024-05-10T10:41:17.443190+0000","last_peered":"2024-05-10T10:41:17.443190+0000","last_clean":"2024-05-10T10:41:17.443190+0000","last_became_active":"2024-05-10T10:41:15.789961+0000","last_became_peered":"2024-05-10T10:41:15.789961+0000","last_unstale":"2024-05-10T10:41:17.443190+0000","last_undegraded":"2024-05-10T10:41:17.443190+0000","last_fullsized":"2024-05-10T10:41:17.443190+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:38:43.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628685+0000","last_change":"2024-05-10T10:41:15.778105+0000","last_active":"2024-05-10T10:41:17.628685+0000","last_peered":"2024-05-10T10:41:17.628685+0000","last_clean":"2024-05-10T10:41:17.628685+0000","last_became_active":"2024-05-10T10:41:15.777822+0000","last_became_peered":"2024-05-10T10:41:15.777822+0000","last_unstale":"2024-05-10T10:41:17.628685+0000","last_undegraded":"2024-05-10T10:41:17.628685+0000","last_fullsized":"2024-05-10T10:41:17.628685+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:26:44.754221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628624+0000","last_change":"2024-05-10T10:41:15.785593+0000","last_active":"2024-05-10T10:41:17.628624+0000","last_peered":"2024-05-10T10:41:17.628624+0000","last_clean":"2024-05-10T10:41:17.628624+0000","last_became_active":"2024-05-10T10:41:15.785287+0000","last_became_peered":"2024-05-10T10:41:15.785287+0000","last_unstale":"2024-05-10T10:41:17.628624+0000","last_undegraded":"2024-05-10T10:41:17.628624+0000","last_fullsized":"2024-05-10T10:41:17.628624+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776943+0000","last_change":"2024-05-10T10:41:15.764267+0000","last_active":"2024-05-10T10:41:16.776943+0000","last_peered":"2024-05-10T10:41:16.776943+0000","last_clean":"2024-05-10T10:41:16.776943+0000","last_became_active":"2024-05-10T10:41:15.763940+0000","last_became_peered":"2024-05-10T10:41:15.763940+0000","last_unstale":"2024-05-10T10:41:16.776943+0000","last_undegraded":"2024-05-10T10:41:16.776943+0000","last_fullsized":"2024-05-10T10:41:16.776943+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:26.540192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628817+0000","last_change":"2024-05-10T10:41:15.782858+0000","last_active":"2024-05-10T10:41:17.628817+0000","last_peered":"2024-05-10T10:41:17.628817+0000","last_clean":"2024-05-10T10:41:17.628817+0000","last_became_active":"2024-05-10T10:41:15.782403+0000","last_became_peered":"2024-05-10T10:41:15.782403+0000","last_unstale":"2024-05-10T10:41:17.628817+0000","last_undegraded":"2024-05-10T10:41:17.628817+0000","last_fullsized":"2024-05-10T10:41:17.628817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936055+0000","last_change":"2024-05-10T10:41:15.767203+0000","last_active":"2024-05-10T10:41:16.936055+0000","last_peered":"2024-05-10T10:41:16.936055+0000","last_clean":"2024-05-10T10:41:16.936055+0000","last_became_active":"2024-05-10T10:41:15.766951+0000","last_became_peered":"2024-05-10T10:41:15.766951+0000","last_unstale":"2024-05-10T10:41:16.936055+0000","last_undegraded":"2024-05-10T10:41:16.936055+0000","last_fullsized":"2024-05-10T10:41:16.936055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:18:21.298157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628344+0000","last_change":"2024-05-10T10:41:15.773963+0000","last_active":"2024-05-10T10:41:17.628344+0000","last_peered":"2024-05-10T10:41:17.628344+0000","last_clean":"2024-05-10T10:41:17.628344+0000","last_became_active":"2024-05-10T10:41:15.773687+0000","last_became_peered":"2024-05-10T10:41:15.773687+0000","last_unstale":"2024-05-10T10:41:17.628344+0000","last_undegraded":"2024-05-10T10:41:17.628344+0000","last_fullsized":"2024-05-10T10:41:17.628344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628384+0000","last_change":"2024-05-10T10:41:15.783053+0000","last_active":"2024-05-10T10:41:17.628384+0000","last_peered":"2024-05-10T10:41:17.628384+0000","last_clean":"2024-05-10T10:41:17.628384+0000","last_became_active":"2024-05-10T10:41:15.782238+0000","last_became_peered":"2024-05-10T10:41:15.782238+0000","last_unstale":"2024-05-10T10:41:17.628384+0000","last_undegraded":"2024-05-10T10:41:17.628384+0000","last_fullsized":"2024-05-10T10:41:17.628384+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:38:43.295885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935857+0000","last_change":"2024-05-10T10:41:15.770987+0000","last_active":"2024-05-10T10:41:16.935857+0000","last_peered":"2024-05-10T10:41:16.935857+0000","last_clean":"2024-05-10T10:41:16.935857+0000","last_became_active":"2024-05-10T10:41:15.770716+0000","last_became_peered":"2024-05-10T10:41:15.770716+0000","last_unstale":"2024-05-10T10:41:16.935857+0000","last_undegraded":"2024-05-10T10:41:16.935857+0000","last_fullsized":"2024-05-10T10:41:16.935857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:25:24.901684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773965+0000","last_change":"2024-05-10T10:41:15.784690+0000","last_active":"2024-05-10T10:41:16.773965+0000","last_peered":"2024-05-10T10:41:16.773965+0000","last_clean":"2024-05-10T10:41:16.773965+0000","last_became_active":"2024-05-10T10:41:15.784422+0000","last_became_peered":"2024-05-10T10:41:15.784422+0000","last_unstale":"2024-05-10T10:41:16.773965+0000","last_undegraded":"2024-05-10T10:41:16.773965+0000","last_fullsized":"2024-05-10T10:41:16.773965+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:16:36.103269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628824+0000","last_change":"2024-05-10T10:41:14.775066+0000","last_active":"2024-05-10T10:41:17.628824+0000","last_peered":"2024-05-10T10:41:17.628824+0000","last_clean":"2024-05-10T10:41:17.628824+0000","last_became_active":"2024-05-10T10:41:14.774728+0000","last_became_peered":"2024-05-10T10:41:14.774728+0000","last_unstale":"2024-05-10T10:41:17.628824+0000","last_undegraded":"2024-05-10T10:41:17.628824+0000","last_fullsized":"2024-05-10T10:41:17.628824+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:38:42.293587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935821+0000","last_change":"2024-05-10T10:41:15.777808+0000","last_active":"2024-05-10T10:41:16.935821+0000","last_peered":"2024-05-10T10:41:16.935821+0000","last_clean":"2024-05-10T10:41:16.935821+0000","last_became_active":"2024-05-10T10:41:15.777563+0000","last_became_peered":"2024-05-10T10:41:15.777563+0000","last_unstale":"2024-05-10T10:41:16.935821+0000","last_undegraded":"2024-05-10T10:41:16.935821+0000","last_fullsized":"2024-05-10T10:41:16.935821+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:34:55.286901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936164+0000","last_change":"2024-05-10T10:41:14.773000+0000","last_active":"2024-05-10T10:41:16.936164+0000","last_peered":"2024-05-10T10:41:16.936164+0000","last_clean":"2024-05-10T10:41:16.936164+0000","last_became_active":"2024-05-10T10:41:14.772714+0000","last_became_peered":"2024-05-10T10:41:14.772714+0000","last_unstale":"2024-05-10T10:41:16.936164+0000","last_undegraded":"2024-05-10T10:41:16.936164+0000","last_fullsized":"2024-05-10T10:41:16.936164+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:25:21.744730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628438+0000","last_change":"2024-05-10T10:41:15.783915+0000","last_active":"2024-05-10T10:41:17.628438+0000","last_peered":"2024-05-10T10:41:17.628438+0000","last_clean":"2024-05-10T10:41:17.628438+0000","last_became_active":"2024-05-10T10:41:15.783648+0000","last_became_peered":"2024-05-10T10:41:15.783648+0000","last_unstale":"2024-05-10T10:41:17.628438+0000","last_undegraded":"2024-05-10T10:41:17.628438+0000","last_fullsized":"2024-05-10T10:41:17.628438+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628500+0000","last_change":"2024-05-10T10:41:14.771387+0000","last_active":"2024-05-10T10:41:17.628500+0000","last_peered":"2024-05-10T10:41:17.628500+0000","last_clean":"2024-05-10T10:41:17.628500+0000","last_became_active":"2024-05-10T10:41:14.771063+0000","last_became_peered":"2024-05-10T10:41:14.771063+0000","last_unstale":"2024-05-10T10:41:17.628500+0000","last_undegraded":"2024-05-10T10:41:17.628500+0000","last_fullsized":"2024-05-10T10:41:17.628500+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:37:53.826208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776308+0000","last_change":"2024-05-10T10:41:15.771738+0000","last_active":"2024-05-10T10:41:16.776308+0000","last_peered":"2024-05-10T10:41:16.776308+0000","last_clean":"2024-05-10T10:41:16.776308+0000","last_became_active":"2024-05-10T10:41:15.771490+0000","last_became_peered":"2024-05-10T10:41:15.771490+0000","last_unstale":"2024-05-10T10:41:16.776308+0000","last_undegraded":"2024-05-10T10:41:16.776308+0000","last_fullsized":"2024-05-10T10:41:16.776308+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:26:44.754221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443577+0000","last_change":"2024-05-10T10:41:14.769973+0000","last_active":"2024-05-10T10:41:17.443577+0000","last_peered":"2024-05-10T10:41:17.443577+0000","last_clean":"2024-05-10T10:41:17.443577+0000","last_became_active":"2024-05-10T10:41:14.769707+0000","last_became_peered":"2024-05-10T10:41:14.769707+0000","last_unstale":"2024-05-10T10:41:17.443577+0000","last_undegraded":"2024-05-10T10:41:17.443577+0000","last_fullsized":"2024-05-10T10:41:17.443577+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:33:31.189870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628684+0000","last_change":"2024-05-10T10:41:15.787899+0000","last_active":"2024-05-10T10:41:17.628684+0000","last_peered":"2024-05-10T10:41:17.628684+0000","last_clean":"2024-05-10T10:41:17.628684+0000","last_became_active":"2024-05-10T10:41:15.787621+0000","last_became_peered":"2024-05-10T10:41:15.787621+0000","last_unstale":"2024-05-10T10:41:17.628684+0000","last_undegraded":"2024-05-10T10:41:17.628684+0000","last_fullsized":"2024-05-10T10:41:17.628684+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:09.552841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777192+0000","last_change":"2024-05-10T10:41:14.772365+0000","last_active":"2024-05-10T10:41:16.777192+0000","last_peered":"2024-05-10T10:41:16.777192+0000","last_clean":"2024-05-10T10:41:16.777192+0000","last_became_active":"2024-05-10T10:41:14.772085+0000","last_became_peered":"2024-05-10T10:41:14.772085+0000","last_unstale":"2024-05-10T10:41:16.777192+0000","last_undegraded":"2024-05-10T10:41:16.777192+0000","last_fullsized":"2024-05-10T10:41:16.777192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:56.141822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774317+0000","last_change":"2024-05-10T10:41:15.786425+0000","last_active":"2024-05-10T10:41:16.774317+0000","last_peered":"2024-05-10T10:41:16.774317+0000","last_clean":"2024-05-10T10:41:16.774317+0000","last_became_active":"2024-05-10T10:41:15.786108+0000","last_became_peered":"2024-05-10T10:41:15.786108+0000","last_unstale":"2024-05-10T10:41:16.774317+0000","last_undegraded":"2024-05-10T10:41:16.774317+0000","last_fullsized":"2024-05-10T10:41:16.774317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:54:00.049649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776127+0000","last_change":"2024-05-10T10:41:14.774933+0000","last_active":"2024-05-10T10:41:16.776127+0000","last_peered":"2024-05-10T10:41:16.776127+0000","last_clean":"2024-05-10T10:41:16.776127+0000","last_became_active":"2024-05-10T10:41:14.774679+0000","last_became_peered":"2024-05-10T10:41:14.774679+0000","last_unstale":"2024-05-10T10:41:16.776127+0000","last_undegraded":"2024-05-10T10:41:16.776127+0000","last_fullsized":"2024-05-10T10:41:16.776127+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:40:35.673531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628952+0000","last_change":"2024-05-10T10:41:15.782664+0000","last_active":"2024-05-10T10:41:17.628952+0000","last_peered":"2024-05-10T10:41:17.628952+0000","last_clean":"2024-05-10T10:41:17.628952+0000","last_became_active":"2024-05-10T10:41:15.781956+0000","last_became_peered":"2024-05-10T10:41:15.781956+0000","last_unstale":"2024-05-10T10:41:17.628952+0000","last_undegraded":"2024-05-10T10:41:17.628952+0000","last_fullsized":"2024-05-10T10:41:17.628952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628020+0000","last_change":"2024-05-10T10:41:14.771441+0000","last_active":"2024-05-10T10:41:17.628020+0000","last_peered":"2024-05-10T10:41:17.628020+0000","last_clean":"2024-05-10T10:41:17.628020+0000","last_became_active":"2024-05-10T10:41:14.771125+0000","last_became_peered":"2024-05-10T10:41:14.771125+0000","last_unstale":"2024-05-10T10:41:17.628020+0000","last_undegraded":"2024-05-10T10:41:17.628020+0000","last_fullsized":"2024-05-10T10:41:17.628020+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:33:31.189870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628791+0000","last_change":"2024-05-10T10:41:15.790041+0000","last_active":"2024-05-10T10:41:17.628791+0000","last_peered":"2024-05-10T10:41:17.628791+0000","last_clean":"2024-05-10T10:41:17.628791+0000","last_became_active":"2024-05-10T10:41:15.789790+0000","last_became_peered":"2024-05-10T10:41:15.789790+0000","last_unstale":"2024-05-10T10:41:17.628791+0000","last_undegraded":"2024-05-10T10:41:17.628791+0000","last_fullsized":"2024-05-10T10:41:17.628791+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:20:49.090489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628328+0000","last_change":"2024-05-10T10:41:14.768068+0000","last_active":"2024-05-10T10:41:17.628328+0000","last_peered":"2024-05-10T10:41:17.628328+0000","last_clean":"2024-05-10T10:41:17.628328+0000","last_became_active":"2024-05-10T10:41:14.767741+0000","last_became_peered":"2024-05-10T10:41:14.767741+0000","last_unstale":"2024-05-10T10:41:17.628328+0000","last_undegraded":"2024-05-10T10:41:17.628328+0000","last_fullsized":"2024-05-10T10:41:17.628328+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:06:57.899386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773813+0000","last_change":"2024-05-10T10:41:15.785589+0000","last_active":"2024-05-10T10:41:16.773813+0000","last_peered":"2024-05-10T10:41:16.773813+0000","last_clean":"2024-05-10T10:41:16.773813+0000","last_became_active":"2024-05-10T10:41:15.785312+0000","last_became_peered":"2024-05-10T10:41:15.785312+0000","last_unstale":"2024-05-10T10:41:16.773813+0000","last_undegraded":"2024-05-10T10:41:16.773813+0000","last_fullsized":"2024-05-10T10:41:16.773813+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:59:25.922185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628707+0000","last_change":"2024-05-10T10:41:14.771832+0000","last_active":"2024-05-10T10:41:17.628707+0000","last_peered":"2024-05-10T10:41:17.628707+0000","last_clean":"2024-05-10T10:41:17.628707+0000","last_became_active":"2024-05-10T10:41:14.771517+0000","last_became_peered":"2024-05-10T10:41:14.771517+0000","last_unstale":"2024-05-10T10:41:17.628707+0000","last_undegraded":"2024-05-10T10:41:17.628707+0000","last_fullsized":"2024-05-10T10:41:17.628707+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:56:25.522332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443242+0000","last_change":"2024-05-10T10:41:15.776876+0000","last_active":"2024-05-10T10:41:17.443242+0000","last_peered":"2024-05-10T10:41:17.443242+0000","last_clean":"2024-05-10T10:41:17.443242+0000","last_became_active":"2024-05-10T10:41:15.776634+0000","last_became_peered":"2024-05-10T10:41:15.776634+0000","last_unstale":"2024-05-10T10:41:17.443242+0000","last_undegraded":"2024-05-10T10:41:17.443242+0000","last_fullsized":"2024-05-10T10:41:17.443242+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:13.788111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773942+0000","last_change":"2024-05-10T10:41:14.775097+0000","last_active":"2024-05-10T10:41:16.773942+0000","last_peered":"2024-05-10T10:41:16.773942+0000","last_clean":"2024-05-10T10:41:16.773942+0000","last_became_active":"2024-05-10T10:41:14.774435+0000","last_became_peered":"2024-05-10T10:41:14.774435+0000","last_unstale":"2024-05-10T10:41:16.773942+0000","last_undegraded":"2024-05-10T10:41:16.773942+0000","last_fullsized":"2024-05-10T10:41:16.773942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:15:16.781752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935875+0000","last_change":"2024-05-10T10:41:15.778402+0000","last_active":"2024-05-10T10:41:16.935875+0000","last_peered":"2024-05-10T10:41:16.935875+0000","last_clean":"2024-05-10T10:41:16.935875+0000","last_became_active":"2024-05-10T10:41:15.778157+0000","last_became_peered":"2024-05-10T10:41:15.778157+0000","last_unstale":"2024-05-10T10:41:16.935875+0000","last_undegraded":"2024-05-10T10:41:16.935875+0000","last_fullsized":"2024-05-10T10:41:16.935875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:12.275951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777005+0000","last_change":"2024-05-10T10:41:14.769181+0000","last_active":"2024-05-10T10:41:16.777005+0000","last_peered":"2024-05-10T10:41:16.777005+0000","last_clean":"2024-05-10T10:41:16.777005+0000","last_became_active":"2024-05-10T10:41:14.768894+0000","last_became_peered":"2024-05-10T10:41:14.768894+0000","last_unstale":"2024-05-10T10:41:16.777005+0000","last_undegraded":"2024-05-10T10:41:16.777005+0000","last_fullsized":"2024-05-10T10:41:16.777005+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:58:23.929251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443655+0000","last_change":"2024-05-10T10:41:15.775989+0000","last_active":"2024-05-10T10:41:17.443655+0000","last_peered":"2024-05-10T10:41:17.443655+0000","last_clean":"2024-05-10T10:41:17.443655+0000","last_became_active":"2024-05-10T10:41:15.775422+0000","last_became_peered":"2024-05-10T10:41:15.775422+0000","last_unstale":"2024-05-10T10:41:17.443655+0000","last_undegraded":"2024-05-10T10:41:17.443655+0000","last_fullsized":"2024-05-10T10:41:17.443655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628929+0000","last_change":"2024-05-10T10:41:14.770676+0000","last_active":"2024-05-10T10:41:17.628929+0000","last_peered":"2024-05-10T10:41:17.628929+0000","last_clean":"2024-05-10T10:41:17.628929+0000","last_became_active":"2024-05-10T10:41:14.770348+0000","last_became_peered":"2024-05-10T10:41:14.770348+0000","last_unstale":"2024-05-10T10:41:17.628929+0000","last_undegraded":"2024-05-10T10:41:17.628929+0000","last_fullsized":"2024-05-10T10:41:17.628929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:12.593453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776179+0000","last_change":"2024-05-10T10:41:15.777201+0000","last_active":"2024-05-10T10:41:16.776179+0000","last_peered":"2024-05-10T10:41:16.776179+0000","last_clean":"2024-05-10T10:41:16.776179+0000","last_became_active":"2024-05-10T10:41:15.776849+0000","last_became_peered":"2024-05-10T10:41:15.776849+0000","last_unstale":"2024-05-10T10:41:16.776179+0000","last_undegraded":"2024-05-10T10:41:16.776179+0000","last_fullsized":"2024-05-10T10:41:16.776179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:57.144120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443695+0000","last_change":"2024-05-10T10:41:14.768756+0000","last_active":"2024-05-10T10:41:17.443695+0000","last_peered":"2024-05-10T10:41:17.443695+0000","last_clean":"2024-05-10T10:41:17.443695+0000","last_became_active":"2024-05-10T10:41:14.768454+0000","last_became_peered":"2024-05-10T10:41:14.768454+0000","last_unstale":"2024-05-10T10:41:17.443695+0000","last_undegraded":"2024-05-10T10:41:17.443695+0000","last_fullsized":"2024-05-10T10:41:17.443695+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:38:41.192594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628584+0000","last_change":"2024-05-10T10:41:15.782260+0000","last_active":"2024-05-10T10:41:17.628584+0000","last_peered":"2024-05-10T10:41:17.628584+0000","last_clean":"2024-05-10T10:41:17.628584+0000","last_became_active":"2024-05-10T10:41:15.781671+0000","last_became_peered":"2024-05-10T10:41:15.781671+0000","last_unstale":"2024-05-10T10:41:17.628584+0000","last_undegraded":"2024-05-10T10:41:17.628584+0000","last_fullsized":"2024-05-10T10:41:17.628584+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:13.788111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628425+0000","last_change":"2024-05-10T10:41:14.773906+0000","last_active":"2024-05-10T10:41:17.628425+0000","last_peered":"2024-05-10T10:41:17.628425+0000","last_clean":"2024-05-10T10:41:17.628425+0000","last_became_active":"2024-05-10T10:41:14.773612+0000","last_became_peered":"2024-05-10T10:41:14.773612+0000","last_unstale":"2024-05-10T10:41:17.628425+0000","last_undegraded":"2024-05-10T10:41:17.628425+0000","last_fullsized":"2024-05-10T10:41:17.628425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:12.785813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936110+0000","last_change":"2024-05-10T10:41:15.772932+0000","last_active":"2024-05-10T10:41:16.936110+0000","last_peered":"2024-05-10T10:41:16.936110+0000","last_clean":"2024-05-10T10:41:16.936110+0000","last_became_active":"2024-05-10T10:41:15.772531+0000","last_became_peered":"2024-05-10T10:41:15.772531+0000","last_unstale":"2024-05-10T10:41:16.936110+0000","last_undegraded":"2024-05-10T10:41:16.936110+0000","last_fullsized":"2024-05-10T10:41:16.936110+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:18:44.800947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935829+0000","last_change":"2024-05-10T10:41:14.778067+0000","last_active":"2024-05-10T10:41:16.935829+0000","last_peered":"2024-05-10T10:41:16.935829+0000","last_clean":"2024-05-10T10:41:16.935829+0000","last_became_active":"2024-05-10T10:41:14.777823+0000","last_became_peered":"2024-05-10T10:41:14.777823+0000","last_unstale":"2024-05-10T10:41:16.935829+0000","last_undegraded":"2024-05-10T10:41:16.935829+0000","last_fullsized":"2024-05-10T10:41:16.935829+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:59.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774310+0000","last_change":"2024-05-10T10:41:15.778732+0000","last_active":"2024-05-10T10:41:16.774310+0000","last_peered":"2024-05-10T10:41:16.774310+0000","last_clean":"2024-05-10T10:41:16.774310+0000","last_became_active":"2024-05-10T10:41:15.778168+0000","last_became_peered":"2024-05-10T10:41:15.778168+0000","last_unstale":"2024-05-10T10:41:16.774310+0000","last_undegraded":"2024-05-10T10:41:16.774310+0000","last_fullsized":"2024-05-10T10:41:16.774310+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:30:05.827338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628015+0000","last_change":"2024-05-10T10:41:14.771181+0000","last_active":"2024-05-10T10:41:17.628015+0000","last_peered":"2024-05-10T10:41:17.628015+0000","last_clean":"2024-05-10T10:41:17.628015+0000","last_became_active":"2024-05-10T10:41:14.770896+0000","last_became_peered":"2024-05-10T10:41:14.770896+0000","last_unstale":"2024-05-10T10:41:17.628015+0000","last_undegraded":"2024-05-10T10:41:17.628015+0000","last_fullsized":"2024-05-10T10:41:17.628015+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:36:14.116695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935901+0000","last_change":"2024-05-10T10:41:15.771784+0000","last_active":"2024-05-10T10:41:16.935901+0000","last_peered":"2024-05-10T10:41:16.935901+0000","last_clean":"2024-05-10T10:41:16.935901+0000","last_became_active":"2024-05-10T10:41:15.771539+0000","last_became_peered":"2024-05-10T10:41:15.771539+0000","last_unstale":"2024-05-10T10:41:16.935901+0000","last_undegraded":"2024-05-10T10:41:16.935901+0000","last_fullsized":"2024-05-10T10:41:16.935901+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:06:13.500814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628768+0000","last_change":"2024-05-10T10:41:14.776190+0000","last_active":"2024-05-10T10:41:17.628768+0000","last_peered":"2024-05-10T10:41:17.628768+0000","last_clean":"2024-05-10T10:41:17.628768+0000","last_became_active":"2024-05-10T10:41:14.775878+0000","last_became_peered":"2024-05-10T10:41:14.775878+0000","last_unstale":"2024-05-10T10:41:17.628768+0000","last_undegraded":"2024-05-10T10:41:17.628768+0000","last_fullsized":"2024-05-10T10:41:17.628768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:07:23.662520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628380+0000","last_change":"2024-05-10T10:41:15.782482+0000","last_active":"2024-05-10T10:41:17.628380+0000","last_peered":"2024-05-10T10:41:17.628380+0000","last_clean":"2024-05-10T10:41:17.628380+0000","last_became_active":"2024-05-10T10:41:15.781922+0000","last_became_peered":"2024-05-10T10:41:15.781922+0000","last_unstale":"2024-05-10T10:41:17.628380+0000","last_undegraded":"2024-05-10T10:41:17.628380+0000","last_fullsized":"2024-05-10T10:41:17.628380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:06:58.901684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777094+0000","last_change":"2024-05-10T10:41:14.766018+0000","last_active":"2024-05-10T10:41:16.777094+0000","last_peered":"2024-05-10T10:41:16.777094+0000","last_clean":"2024-05-10T10:41:16.777094+0000","last_became_active":"2024-05-10T10:41:14.765716+0000","last_became_peered":"2024-05-10T10:41:14.765716+0000","last_unstale":"2024-05-10T10:41:16.777094+0000","last_undegraded":"2024-05-10T10:41:16.777094+0000","last_fullsized":"2024-05-10T10:41:16.777094+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:24.270566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443550+0000","last_change":"2024-05-10T10:41:15.776187+0000","last_active":"2024-05-10T10:41:17.443550+0000","last_peered":"2024-05-10T10:41:17.443550+0000","last_clean":"2024-05-10T10:41:17.443550+0000","last_became_active":"2024-05-10T10:41:15.775748+0000","last_became_peered":"2024-05-10T10:41:15.775748+0000","last_unstale":"2024-05-10T10:41:17.443550+0000","last_undegraded":"2024-05-10T10:41:17.443550+0000","last_fullsized":"2024-05-10T10:41:17.443550+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776513+0000","last_change":"2024-05-10T10:41:14.769976+0000","last_active":"2024-05-10T10:41:16.776513+0000","last_peered":"2024-05-10T10:41:16.776513+0000","last_clean":"2024-05-10T10:41:16.776513+0000","last_became_active":"2024-05-10T10:41:14.769684+0000","last_became_peered":"2024-05-10T10:41:14.769684+0000","last_unstale":"2024-05-10T10:41:16.776513+0000","last_undegraded":"2024-05-10T10:41:16.776513+0000","last_fullsized":"2024-05-10T10:41:16.776513+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:29:51.351714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773938+0000","last_change":"2024-05-10T10:41:15.777885+0000","last_active":"2024-05-10T10:41:16.773938+0000","last_peered":"2024-05-10T10:41:16.773938+0000","last_clean":"2024-05-10T10:41:16.773938+0000","last_became_active":"2024-05-10T10:41:15.777644+0000","last_became_peered":"2024-05-10T10:41:15.777644+0000","last_unstale":"2024-05-10T10:41:16.773938+0000","last_undegraded":"2024-05-10T10:41:16.773938+0000","last_fullsized":"2024-05-10T10:41:16.773938+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:22:32.623167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936177+0000","last_change":"2024-05-10T10:41:14.780386+0000","last_active":"2024-05-10T10:41:16.936177+0000","last_peered":"2024-05-10T10:41:16.936177+0000","last_clean":"2024-05-10T10:41:16.936177+0000","last_became_active":"2024-05-10T10:41:14.778333+0000","last_became_peered":"2024-05-10T10:41:14.778333+0000","last_unstale":"2024-05-10T10:41:16.936177+0000","last_undegraded":"2024-05-10T10:41:16.936177+0000","last_fullsized":"2024-05-10T10:41:16.936177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:48:12.560346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.774166+0000","last_change":"2024-05-10T10:41:15.781182+0000","last_active":"2024-05-10T10:41:16.774166+0000","last_peered":"2024-05-10T10:41:16.774166+0000","last_clean":"2024-05-10T10:41:16.774166+0000","last_became_active":"2024-05-10T10:41:15.780911+0000","last_became_peered":"2024-05-10T10:41:15.780911+0000","last_unstale":"2024-05-10T10:41:16.774166+0000","last_undegraded":"2024-05-10T10:41:16.774166+0000","last_fullsized":"2024-05-10T10:41:16.774166+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:41:11.537759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935802+0000","last_change":"2024-05-10T10:41:14.779340+0000","last_active":"2024-05-10T10:41:16.935802+0000","last_peered":"2024-05-10T10:41:16.935802+0000","last_clean":"2024-05-10T10:41:16.935802+0000","last_became_active":"2024-05-10T10:41:14.778986+0000","last_became_peered":"2024-05-10T10:41:14.778986+0000","last_unstale":"2024-05-10T10:41:16.935802+0000","last_undegraded":"2024-05-10T10:41:16.935802+0000","last_fullsized":"2024-05-10T10:41:16.935802+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:09:55.182479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628767+0000","last_change":"2024-05-10T10:41:15.786444+0000","last_active":"2024-05-10T10:41:17.628767+0000","last_peered":"2024-05-10T10:41:17.628767+0000","last_clean":"2024-05-10T10:41:17.628767+0000","last_became_active":"2024-05-10T10:41:15.786163+0000","last_became_peered":"2024-05-10T10:41:15.786163+0000","last_unstale":"2024-05-10T10:41:17.628767+0000","last_undegraded":"2024-05-10T10:41:17.628767+0000","last_fullsized":"2024-05-10T10:41:17.628767+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:12.413148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628649+0000","last_change":"2024-05-10T10:41:14.774675+0000","last_active":"2024-05-10T10:41:17.628649+0000","last_peered":"2024-05-10T10:41:17.628649+0000","last_clean":"2024-05-10T10:41:17.628649+0000","last_became_active":"2024-05-10T10:41:14.774374+0000","last_became_peered":"2024-05-10T10:41:14.774374+0000","last_unstale":"2024-05-10T10:41:17.628649+0000","last_undegraded":"2024-05-10T10:41:17.628649+0000","last_fullsized":"2024-05-10T10:41:17.628649+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:37:04.797470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936091+0000","last_change":"2024-05-10T10:41:15.777186+0000","last_active":"2024-05-10T10:41:16.936091+0000","last_peered":"2024-05-10T10:41:16.936091+0000","last_clean":"2024-05-10T10:41:16.936091+0000","last_became_active":"2024-05-10T10:41:15.776896+0000","last_became_peered":"2024-05-10T10:41:15.776896+0000","last_unstale":"2024-05-10T10:41:16.936091+0000","last_undegraded":"2024-05-10T10:41:16.936091+0000","last_fullsized":"2024-05-10T10:41:16.936091+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:46.567398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.935938+0000","last_change":"2024-05-10T10:41:14.779920+0000","last_active":"2024-05-10T10:41:16.935938+0000","last_peered":"2024-05-10T10:41:16.935938+0000","last_clean":"2024-05-10T10:41:16.935938+0000","last_became_active":"2024-05-10T10:41:14.779652+0000","last_became_peered":"2024-05-10T10:41:14.779652+0000","last_unstale":"2024-05-10T10:41:16.935938+0000","last_undegraded":"2024-05-10T10:41:16.935938+0000","last_fullsized":"2024-05-10T10:41:16.935938+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:53:18.910071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.773986+0000","last_change":"2024-05-10T10:41:15.777834+0000","last_active":"2024-05-10T10:41:16.773986+0000","last_peered":"2024-05-10T10:41:16.773986+0000","last_clean":"2024-05-10T10:41:16.773986+0000","last_became_active":"2024-05-10T10:41:15.777551+0000","last_became_peered":"2024-05-10T10:41:15.777551+0000","last_unstale":"2024-05-10T10:41:16.773986+0000","last_undegraded":"2024-05-10T10:41:16.773986+0000","last_fullsized":"2024-05-10T10:41:16.773986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:13:48.075231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443641+0000","last_change":"2024-05-10T10:41:14.770637+0000","last_active":"2024-05-10T10:41:17.443641+0000","last_peered":"2024-05-10T10:41:17.443641+0000","last_clean":"2024-05-10T10:41:17.443641+0000","last_became_active":"2024-05-10T10:41:14.770317+0000","last_became_peered":"2024-05-10T10:41:14.770317+0000","last_unstale":"2024-05-10T10:41:17.443641+0000","last_undegraded":"2024-05-10T10:41:17.443641+0000","last_fullsized":"2024-05-10T10:41:17.443641+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:10.949088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628212+0000","last_change":"2024-05-10T10:41:15.780646+0000","last_active":"2024-05-10T10:41:17.628212+0000","last_peered":"2024-05-10T10:41:17.628212+0000","last_clean":"2024-05-10T10:41:17.628212+0000","last_became_active":"2024-05-10T10:41:15.780390+0000","last_became_peered":"2024-05-10T10:41:15.780390+0000","last_unstale":"2024-05-10T10:41:17.628212+0000","last_undegraded":"2024-05-10T10:41:17.628212+0000","last_fullsized":"2024-05-10T10:41:17.628212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:30:13.788111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.765518+0000","last_change":"2024-05-10T10:41:14.775314+0000","last_active":"2024-05-10T10:41:17.765518+0000","last_peered":"2024-05-10T10:41:17.765518+0000","last_clean":"2024-05-10T10:41:17.765518+0000","last_became_active":"2024-05-10T10:41:14.774832+0000","last_became_peered":"2024-05-10T10:41:14.774832+0000","last_unstale":"2024-05-10T10:41:17.765518+0000","last_undegraded":"2024-05-10T10:41:17.765518+0000","last_fullsized":"2024-05-10T10:41:17.765518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:42:37.625354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628391+0000","last_change":"2024-05-10T10:41:15.785689+0000","last_active":"2024-05-10T10:41:17.628391+0000","last_peered":"2024-05-10T10:41:17.628391+0000","last_clean":"2024-05-10T10:41:17.628391+0000","last_became_active":"2024-05-10T10:41:15.785422+0000","last_became_peered":"2024-05-10T10:41:15.785422+0000","last_unstale":"2024-05-10T10:41:17.628391+0000","last_undegraded":"2024-05-10T10:41:17.628391+0000","last_fullsized":"2024-05-10T10:41:17.628391+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:22:26.540192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628662+0000","last_change":"2024-05-10T10:41:14.775234+0000","last_active":"2024-05-10T10:41:17.628662+0000","last_peered":"2024-05-10T10:41:17.628662+0000","last_clean":"2024-05-10T10:41:17.628662+0000","last_became_active":"2024-05-10T10:41:14.774959+0000","last_became_peered":"2024-05-10T10:41:14.774959+0000","last_unstale":"2024-05-10T10:41:17.628662+0000","last_undegraded":"2024-05-10T10:41:17.628662+0000","last_fullsized":"2024-05-10T10:41:17.628662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:27:56.760836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776256+0000","last_change":"2024-05-10T10:41:15.777117+0000","last_active":"2024-05-10T10:41:16.776256+0000","last_peered":"2024-05-10T10:41:16.776256+0000","last_clean":"2024-05-10T10:41:16.776256+0000","last_became_active":"2024-05-10T10:41:15.776833+0000","last_became_peered":"2024-05-10T10:41:15.776833+0000","last_unstale":"2024-05-10T10:41:16.776256+0000","last_undegraded":"2024-05-10T10:41:16.776256+0000","last_fullsized":"2024-05-10T10:41:16.776256+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:05:50.184954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.443611+0000","last_change":"2024-05-10T10:41:14.775323+0000","last_active":"2024-05-10T10:41:17.443611+0000","last_peered":"2024-05-10T10:41:17.443611+0000","last_clean":"2024-05-10T10:41:17.443611+0000","last_became_active":"2024-05-10T10:41:14.775045+0000","last_became_peered":"2024-05-10T10:41:14.775045+0000","last_unstale":"2024-05-10T10:41:17.443611+0000","last_undegraded":"2024-05-10T10:41:17.443611+0000","last_fullsized":"2024-05-10T10:41:17.443611+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:36:14.116695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936158+0000","last_change":"2024-05-10T10:41:15.775892+0000","last_active":"2024-05-10T10:41:16.936158+0000","last_peered":"2024-05-10T10:41:16.936158+0000","last_clean":"2024-05-10T10:41:16.936158+0000","last_became_active":"2024-05-10T10:41:15.775631+0000","last_became_peered":"2024-05-10T10:41:15.775631+0000","last_unstale":"2024-05-10T10:41:16.936158+0000","last_undegraded":"2024-05-10T10:41:16.936158+0000","last_fullsized":"2024-05-10T10:41:16.936158+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:00:56.386129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628117+0000","last_change":"2024-05-10T10:41:14.774153+0000","last_active":"2024-05-10T10:41:17.628117+0000","last_peered":"2024-05-10T10:41:17.628117+0000","last_clean":"2024-05-10T10:41:17.628117+0000","last_became_active":"2024-05-10T10:41:14.773869+0000","last_became_peered":"2024-05-10T10:41:14.773869+0000","last_unstale":"2024-05-10T10:41:17.628117+0000","last_undegraded":"2024-05-10T10:41:17.628117+0000","last_fullsized":"2024-05-10T10:41:17.628117+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:02:10.949088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936145+0000","last_change":"2024-05-10T10:41:15.776526+0000","last_active":"2024-05-10T10:41:16.936145+0000","last_peered":"2024-05-10T10:41:16.936145+0000","last_clean":"2024-05-10T10:41:16.936145+0000","last_became_active":"2024-05-10T10:41:15.776272+0000","last_became_peered":"2024-05-10T10:41:15.776272+0000","last_unstale":"2024-05-10T10:41:16.936145+0000","last_undegraded":"2024-05-10T10:41:16.936145+0000","last_fullsized":"2024-05-10T10:41:16.936145+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:36:51.010333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936116+0000","last_change":"2024-05-10T10:41:14.777081+0000","last_active":"2024-05-10T10:41:16.936116+0000","last_peered":"2024-05-10T10:41:16.936116+0000","last_clean":"2024-05-10T10:41:16.936116+0000","last_became_active":"2024-05-10T10:41:14.776821+0000","last_became_peered":"2024-05-10T10:41:14.776821+0000","last_unstale":"2024-05-10T10:41:16.936116+0000","last_undegraded":"2024-05-10T10:41:16.936116+0000","last_fullsized":"2024-05-10T10:41:16.936116+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:57.365909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776834+0000","last_change":"2024-05-10T10:41:15.766728+0000","last_active":"2024-05-10T10:41:16.776834+0000","last_peered":"2024-05-10T10:41:16.776834+0000","last_clean":"2024-05-10T10:41:16.776834+0000","last_became_active":"2024-05-10T10:41:15.765213+0000","last_became_peered":"2024-05-10T10:41:15.765213+0000","last_unstale":"2024-05-10T10:41:16.776834+0000","last_undegraded":"2024-05-10T10:41:16.776834+0000","last_fullsized":"2024-05-10T10:41:16.776834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:38:12.413148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:17.628656+0000","last_change":"2024-05-10T10:41:14.775821+0000","last_active":"2024-05-10T10:41:17.628656+0000","last_peered":"2024-05-10T10:41:17.628656+0000","last_clean":"2024-05-10T10:41:17.628656+0000","last_became_active":"2024-05-10T10:41:14.775438+0000","last_became_peered":"2024-05-10T10:41:14.775438+0000","last_unstale":"2024-05-10T10:41:17.628656+0000","last_undegraded":"2024-05-10T10:41:17.628656+0000","last_fullsized":"2024-05-10T10:41:17.628656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:18:19.119069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.776665+0000","last_change":"2024-05-10T10:41:15.765624+0000","last_active":"2024-05-10T10:41:16.776665+0000","last_peered":"2024-05-10T10:41:16.776665+0000","last_clean":"2024-05-10T10:41:16.776665+0000","last_became_active":"2024-05-10T10:41:15.765326+0000","last_became_peered":"2024-05-10T10:41:15.765326+0000","last_unstale":"2024-05-10T10:41:16.776665+0000","last_undegraded":"2024-05-10T10:41:16.776665+0000","last_fullsized":"2024-05-10T10:41:16.776665+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:09:24.288572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936011+0000","last_change":"2024-05-10T10:41:14.772638+0000","last_active":"2024-05-10T10:41:16.936011+0000","last_peered":"2024-05-10T10:41:16.936011+0000","last_clean":"2024-05-10T10:41:16.936011+0000","last_became_active":"2024-05-10T10:41:14.772333+0000","last_became_peered":"2024-05-10T10:41:14.772333+0000","last_unstale":"2024-05-10T10:41:16.936011+0000","last_undegraded":"2024-05-10T10:41:16.936011+0000","last_fullsized":"2024-05-10T10:41:16.936011+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:24.849979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.775959+0000","last_change":"2024-05-10T10:41:15.771722+0000","last_active":"2024-05-10T10:41:16.775959+0000","last_peered":"2024-05-10T10:41:16.775959+0000","last_clean":"2024-05-10T10:41:16.775959+0000","last_became_active":"2024-05-10T10:41:15.771453+0000","last_became_peered":"2024-05-10T10:41:15.771453+0000","last_unstale":"2024-05-10T10:41:16.775959+0000","last_undegraded":"2024-05-10T10:41:16.775959+0000","last_fullsized":"2024-05-10T10:41:16.775959+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:14.750465+0000","last_clean_scrub_stamp":"2024-05-10T10:41:14.750465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:36:25.272864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.936209+0000","last_change":"2024-05-10T10:41:14.779327+0000","last_active":"2024-05-10T10:41:16.936209+0000","last_peered":"2024-05-10T10:41:16.936209+0000","last_clean":"2024-05-10T10:41:16.936209+0000","last_became_active":"2024-05-10T10:41:14.779018+0000","last_became_peered":"2024-05-10T10:41:14.779018+0000","last_unstale":"2024-05-10T10:41:16.936209+0000","last_undegraded":"2024-05-10T10:41:16.936209+0000","last_fullsized":"2024-05-10T10:41:16.936209+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:41:13.748167+0000","last_clean_scrub_stamp":"2024-05-10T10:41:13.748167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:36:12.011464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":279,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T10:41:16.777008+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:41:16.777008+0000","last_peered":"2024-05-10T10:41:16.777008+0000","last_clean":"2024-05-10T10:41:16.777008+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:41:16.777008+0000","last_undegraded":"2024-05-10T10:41:16.777008+0000","last_fullsized":"2024-05-10T10:41:16.777008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1181960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":586956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":591432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":584832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":592014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1282048,"data_stored":591466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":584774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":1183502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:41:20.191 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:41:20.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T10:41:20.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.337+0000 7f9ab72e3640 1 Processor -- start 2024-05-10T10:41:20.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.337+0000 7f9ab72e3640 1 -- start start 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab72e3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ab00d3de0 0x7f9ab00b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab72e3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab00d2800 con 0x7f9ab00b4f90 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab72e3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab00d2ae0 con 0x7f9ab015f410 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab72e3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab00d01d0 con 0x7f9ab00d3de0 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab5880640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ab00d3de0 0x7f9ab00b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab5880640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ab00d3de0 0x7f9ab00b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50988/0 (socket says 172.21.15.53:50988) 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab5880640 1 -- 172.21.15.53:0/510482231 learned_addr learned my addr 172.21.15.53:0/510482231 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:20.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab487e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9ab00b4f90 0x7f9ab00d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46390/0 (socket says 172.21.15.53:46390) 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab5880640 1 -- 172.21.15.53:0/510482231 >> v1:172.21.15.135:6789/0 conn(0x7f9ab015f410 legacy=0x7f9ab00d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab5880640 1 -- 172.21.15.53:0/510482231 >> v1:172.21.15.53:6789/0 conn(0x7f9ab00b4f90 legacy=0x7f9ab00d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9ab5880640 1 -- 172.21.15.53:0/510482231 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab00d04b0 con 0x7f9ab00d3de0 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.338+0000 7f9aa7fff640 1 -- 172.21.15.53:0/510482231 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2619449928 0 0) 0x7f9ab00d2800 con 0x7f9ab00b4f90 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.339+0000 7f9ab5880640 1 --2- 172.21.15.53:0/510482231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00d3de0 0x7f9ab00b5410 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009e50 tx=0x7f9aa802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c093f6ffe22d7a61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.339+0000 7f9aa7fff640 1 -- 172.21.15.53:0/510482231 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8036070 con 0x7f9ab00d3de0 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.339+0000 7f9aa7fff640 1 -- 172.21.15.53:0/510482231 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8031600 con 0x7f9ab00d3de0 2024-05-10T10:41:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.339+0000 7f9aa7fff640 1 -- 172.21.15.53:0/510482231 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80319d0 con 0x7f9ab00d3de0 2024-05-10T10:41:20.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.340+0000 7f9ab72e3640 1 -- 172.21.15.53:0/510482231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00d3de0 msgr2=0x7f9ab00b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.340+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/510482231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00d3de0 0x7f9ab00b5410 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009e50 tx=0x7f9aa802a6d0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.340+0000 7f9ab72e3640 1 -- 172.21.15.53:0/510482231 shutdown_connections 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.340+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/510482231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00d3de0 0x7f9ab00b5410 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.340+0000 7f9ab72e3640 1 -- 172.21.15.53:0/510482231 >> 172.21.15.53:0/510482231 conn(0x7f9ab008ed30 msgr2=0x7f9ab009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.340+0000 7f9ab72e3640 1 -- 172.21.15.53:0/510482231 shutdown_connections 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.340+0000 7f9ab72e3640 1 -- 172.21.15.53:0/510482231 wait complete. 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.341+0000 7f9ab72e3640 1 Processor -- start 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.341+0000 7f9ab72e3640 1 -- start start 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab72e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00b4f90 0x7f9ab014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab72e3640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ab00d3de0 0x7f9ab00cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab72e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ab015f410 0x7f9ab006db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab72e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab00cffb0 con 0x7f9ab00b4f90 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab72e3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab00cfe30 con 0x7f9ab015f410 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab72e3640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab00d10e0 con 0x7f9ab00d3de0 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00b4f90 0x7f9ab014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00b4f90 0x7f9ab014ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51000/0 (socket says 172.21.15.53:51000) 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 -- 172.21.15.53:0/2401710796 learned_addr learned my addr 172.21.15.53:0/2401710796 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab5880640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ab015f410 0x7f9ab006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab487e640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ab00d3de0 0x7f9ab00cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 -- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ab00d3de0 msgr2=0x7f9ab00cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ab00d3de0 0x7f9ab00cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 -- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ab015f410 msgr2=0x7f9ab006db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ab015f410 0x7f9ab006db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab507f640 1 -- 172.21.15.53:0/2401710796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab00cad30 con 0x7f9ab00b4f90 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.342+0000 7f9ab487e640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ab00d3de0 0x7f9ab00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.343+0000 7f9ab507f640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00b4f90 0x7f9ab014ae30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00c430 tx=0x7f9aac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.343+0000 7f9aa5ffb640 1 -- 172.21.15.53:0/2401710796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac011790 con 0x7f9ab00b4f90 2024-05-10T10:41:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.343+0000 7f9aa5ffb640 1 -- 172.21.15.53:0/2401710796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aac017070 con 0x7f9ab00b4f90 2024-05-10T10:41:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.343+0000 7f9aa5ffb640 1 -- 172.21.15.53:0/2401710796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac011bb0 con 0x7f9ab00b4f90 2024-05-10T10:41:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.343+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab014e490 con 0x7f9ab00b4f90 2024-05-10T10:41:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.343+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab00cb2b0 con 0x7f9ab00b4f90 2024-05-10T10:41:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.344+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab006a220 con 0x7f9ab00b4f90 2024-05-10T10:41:20.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.345+0000 7f9aa5ffb640 1 -- 172.21.15.53:0/2401710796 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9aac01c460 con 0x7f9ab00b4f90 2024-05-10T10:41:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.345+0000 7f9aa5ffb640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9a94076300 0x7f9a940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.345+0000 7f9aa5ffb640 1 -- 172.21.15.53:0/2401710796 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f9aac0974c0 con 0x7f9ab00b4f90 2024-05-10T10:41:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.345+0000 7f9ab487e640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9a94076300 0x7f9a940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.346+0000 7f9ab487e640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9a94076300 0x7f9a940787c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0151e00 tx=0x7f9aa0007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.350+0000 7f9aa5ffb640 1 -- 172.21.15.53:0/2401710796 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9aac060d20 con 0x7f9ab00b4f90 2024-05-10T10:41:20.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.614+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9ab00d6090 con 0x7f9ab00b4f90 2024-05-10T10:41:20.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.616+0000 7f9aa5ffb640 1 -- 172.21.15.53:0/2401710796 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f9aac060620 con 0x7f9ab00b4f90 2024-05-10T10:41:20.618 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:41:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:41:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2024-05-10T10:41:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2024-05-10T10:41:20.618 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-10T10:41:20.618 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2024-05-10T10:41:20.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9a94076300 msgr2=0x7f9a940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9a94076300 0x7f9a940787c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0151e00 tx=0x7f9aa0007ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00b4f90 msgr2=0x7f9ab014ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00b4f90 0x7f9ab014ae30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00c430 tx=0x7f9aac00c8f0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 shutdown_connections 2024-05-10T10:41:20.620 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9a94076300 0x7f9a940787c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ab015f410 0x7f9ab006db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ab00d3de0 0x7f9ab00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 --2- 172.21.15.53:0/2401710796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ab00b4f90 0x7f9ab014ae30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 >> 172.21.15.53:0/2401710796 conn(0x7f9ab008ed30 msgr2=0x7f9ab00b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:20.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 shutdown_connections 2024-05-10T10:41:20.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.618+0000 7f9ab72e3640 1 -- 172.21.15.53:0/2401710796 wait complete. 2024-05-10T10:41:20.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:20.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.734+0000 7f924e6b4640 1 Processor -- start 2024-05-10T10:41:20.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.734+0000 7f924e6b4640 1 -- start start 2024-05-10T10:41:20.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.734+0000 7f924e6b4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f92481598c0 0x7f924806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f924e6b4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924806f1c0 con 0x7f924806ec40 2024-05-10T10:41:20.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f924e6b4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924806f4a0 con 0x7f92480c7390 2024-05-10T10:41:20.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f924e6b4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f924806f780 con 0x7f92481598c0 2024-05-10T10:41:20.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f92477fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f924806ec40 0x7f92480c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46394/0 (socket says 172.21.15.53:46394) 2024-05-10T10:41:20.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f92477fe640 1 -- 172.21.15.53:0/3737166780 learned_addr learned my addr 172.21.15.53:0/3737166780 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:20.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370559332 0 0) 0x7f924806f1c0 con 0x7f924806ec40 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9230003520 con 0x7f924806ec40 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9247fff640 1 --2- 172.21.15.53:0/3737166780 >> v2:172.21.15.135:3300/0 conn(0x7f92481598c0 0x7f924806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 375313402 0 0) 0x7f9230003520 con 0x7f924806ec40 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 >> v2:172.21.15.135:3300/0 conn(0x7f92481598c0 msgr2=0x7f924806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 --2- 172.21.15.53:0/3737166780 >> v2:172.21.15.135:3300/0 conn(0x7f92481598c0 0x7f924806e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 >> v1:172.21.15.135:6789/0 conn(0x7f92480c7390 legacy=0x7f92480cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924806fa60 con 0x7f924806ec40 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f923c001670 con 0x7f924806ec40 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 >> v1:172.21.15.53:6789/0 conn(0x7f924806ec40 legacy=0x7f92480c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92300039b0 0x7f9230003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9230004800 0x7f9230004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f924806f780 con 0x7f9230004800 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f923000c060 con 0x7f92300051c0 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f923000c260 con 0x7f92300039b0 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.735+0000 7f92477fe640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9230004800 0x7f9230004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f9247fff640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92300039b0 0x7f9230003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f924cc51640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f92477fe640 1 -- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92300039b0 msgr2=0x7f9230003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:20.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f92477fe640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f92300039b0 0x7f9230003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f92477fe640 1 -- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92300051c0 msgr2=0x7f923000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:20.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f92477fe640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f924cc51640 1 -- 172.21.15.53:0/3737166780 reap_dead start 2024-05-10T10:41:20.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.736+0000 7f92477fe640 1 -- 172.21.15.53:0/3737166780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9230003520 con 0x7f9230004800 2024-05-10T10:41:20.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f92477fe640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9230004800 0x7f9230004c80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c240 tx=0x7f9230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da725bdc2a13d3cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:20.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c007d10 con 0x7f9230004800 2024-05-10T10:41:20.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923c005ad0 con 0x7f9230004800 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f9246ffd640 1 -- 172.21.15.53:0/3737166780 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c007070 con 0x7f9230004800 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f924e6b4640 1 -- 172.21.15.53:0/3737166780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9230004800 msgr2=0x7f9230004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f924e6b4640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9230004800 0x7f9230004c80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c240 tx=0x7f9230005830 comp rx=0 tx=0).stop 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f924e6b4640 1 -- 172.21.15.53:0/3737166780 shutdown_connections 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f924e6b4640 1 --2- 172.21.15.53:0/3737166780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9230004800 0x7f9230004c80 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f924e6b4640 1 -- 172.21.15.53:0/3737166780 >> 172.21.15.53:0/3737166780 conn(0x7f92480b2900 msgr2=0x7f92480b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f924e6b4640 1 -- 172.21.15.53:0/3737166780 shutdown_connections 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.737+0000 7f924e6b4640 1 -- 172.21.15.53:0/3737166780 wait complete. 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 Processor -- start 2024-05-10T10:41:20.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 -- start start 2024-05-10T10:41:20.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9230004800 0x7f924815acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9248072f20 0x7f92480733a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92480cd5b0 0x7f92480cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9248159ca0 con 0x7f9248072f20 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f924815c6e0 con 0x7f92480cd5b0 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.739+0000 7f924e6b4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9248159e20 con 0x7f9230004800 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9248072f20 0x7f92480733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9248072f20 0x7f92480733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51032/0 (socket says 172.21.15.53:51032) 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 -- 172.21.15.53:0/1833161372 learned_addr learned my addr 172.21.15.53:0/1833161372 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f9247fff640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9230004800 0x7f924815acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 -- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9230004800 msgr2=0x7f924815acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f924cc51640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92480cd5b0 0x7f92480cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9230004800 0x7f924815acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 -- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92480cd5b0 msgr2=0x7f92480cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92480cd5b0 0x7f92480cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 -- 172.21.15.53:0/1833161372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92480738e0 con 0x7f9248072f20 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f924cc51640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92480cd5b0 0x7f92480cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f92477fe640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9248072f20 0x7f92480733a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c240 tx=0x7f924806f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f9244ff9640 1 -- 172.21.15.53:0/1833161372 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c041070 con 0x7f9248072f20 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f9244ff9640 1 -- 172.21.15.53:0/1833161372 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923c000ed0 con 0x7f9248072f20 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f9244ff9640 1 -- 172.21.15.53:0/1833161372 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c007d10 con 0x7f9248072f20 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924815b940 con 0x7f9248072f20 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.740+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924815b660 con 0x7f9248072f20 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.741+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924806fce0 con 0x7f9248072f20 2024-05-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.742+0000 7f9244ff9640 1 -- 172.21.15.53:0/1833161372 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f923c044070 con 0x7f9248072f20 2024-05-10T10:41:20.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.743+0000 7f9244ff9640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f92340762d0 0x7f9234078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:20.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.743+0000 7f9244ff9640 1 -- 172.21.15.53:0/1833161372 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f923c0b7e20 con 0x7f9248072f20 2024-05-10T10:41:20.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.743+0000 7f9247fff640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f92340762d0 0x7f9234078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:20.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.744+0000 7f9247fff640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f92340762d0 0x7f9234078790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f924815f290 tx=0x7f9238008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:20.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.747+0000 7f9244ff9640 1 -- 172.21.15.53:0/1833161372 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f924806fce0 con 0x7f9248072f20 2024-05-10T10:41:20.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.980+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f924806a220 con 0x7f9248072f20 2024-05-10T10:41:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.980+0000 7f9244ff9640 1 -- 172.21.15.53:0/1833161372 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5189 (secure 0 0 0) 0x7f923c081680 con 0x7f9248072f20 2024-05-10T10:41:20.982 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:20.982 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:41:18.164463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4122299129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":4122299129},{"type":"v1","addr":"172.21.15.53:6835","nonce":4122299129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7479,"qdb_cluster":[7479]},"id":7}]} 2024-05-10T10:41:20.983 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 82 2024-05-10T10:41:20.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f92340762d0 msgr2=0x7f9234078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f92340762d0 0x7f9234078790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f924815f290 tx=0x7f9238008040 comp rx=0 tx=0).stop 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9248072f20 msgr2=0x7f92480733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9248072f20 0x7f92480733a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c240 tx=0x7f924806f670 comp rx=0 tx=0).stop 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 shutdown_connections 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f92340762d0 0x7f9234078790 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92480cd5b0 0x7f92480cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9248072f20 0x7f92480733a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 --2- 172.21.15.53:0/1833161372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9230004800 0x7f924815acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.982+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 >> 172.21.15.53:0/1833161372 conn(0x7f92480b2900 msgr2=0x7f924815f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.983+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 shutdown_connections 2024-05-10T10:41:20.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:20.983+0000 7f924e6b4640 1 -- 172.21.15.53:0/1833161372 wait complete. 2024-05-10T10:41:21.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:41:15.140801+0000', 'modified': '2024-05-10T10:41:18.164463+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7479': {'gid': 7479, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/4122299129', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 4122299129}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 4122299129}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7479, 'qdb_cluster': [7479]} 2024-05-10T10:41:21.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:41:21.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T10:41:21.404 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:41:21.403+0000 7f17a2b40640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:41:21.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.508+0000 7f28507c8640 1 Processor -- start 2024-05-10T10:41:21.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.508+0000 7f28507c8640 1 -- start start 2024-05-10T10:41:21.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f28507c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2848168220 0x7f284816a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f28507c8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28480b0a90 con 0x7f284815f8c0 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f28507c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28480b3540 con 0x7f284815f4e0 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f28507c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f28480b23f0 con 0x7f2848168220 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f284dd63640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f284815f8c0 0x7f284806e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46402/0 (socket says 172.21.15.53:46402) 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f284dd63640 1 -- 172.21.15.53:0/2755148193 learned_addr learned my addr 172.21.15.53:0/2755148193 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f284ed65640 1 --2- 172.21.15.53:0/2755148193 >> v2:172.21.15.135:3300/0 conn(0x7f2848168220 0x7f284816a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1708535474 0 0) 0x7f28480b0a90 con 0x7f284815f8c0 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.509+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2830003520 con 0x7f284815f8c0 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2156546289 0 0) 0x7f2830003520 con 0x7f284815f8c0 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 >> v2:172.21.15.135:3300/0 conn(0x7f2848168220 msgr2=0x7f284816a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 --2- 172.21.15.53:0/2755148193 >> v2:172.21.15.135:3300/0 conn(0x7f2848168220 0x7f284816a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 >> v1:172.21.15.135:6789/0 conn(0x7f284815f4e0 legacy=0x7f284806d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28480b3820 con 0x7f284815f8c0 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f283c001670 con 0x7f284815f8c0 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 >> v1:172.21.15.53:6789/0 conn(0x7f284815f8c0 legacy=0x7f284806e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28300039b0 0x7f2830003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830004800 0x7f2830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28300051c0 0x7f283000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28480b23f0 con 0x7f28300051c0 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f283000c160 con 0x7f28300039b0 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f283000c360 con 0x7f2830004800 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284ed65640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28300051c0 0x7f283000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284e564640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28300039b0 0x7f2830003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284dd63640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830004800 0x7f2830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284ed65640 1 -- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830004800 msgr2=0x7f2830004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284ed65640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830004800 0x7f2830004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284ed65640 1 -- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28300039b0 msgr2=0x7f2830003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284ed65640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28300039b0 0x7f2830003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.510+0000 7f284ed65640 1 -- 172.21.15.53:0/2755148193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2830003520 con 0x7f28300051c0 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.511+0000 7f284ed65640 1 -- 172.21.15.53:0/2755148193 reap_dead start 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.511+0000 7f284ed65640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28300051c0 0x7f283000b9c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2840009a90 tx=0x7f2830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec6617f693edbc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.511+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840036070 con 0x7f28300051c0 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.511+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2840031400 con 0x7f28300051c0 2024-05-10T10:41:21.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.511+0000 7f284d562640 1 -- 172.21.15.53:0/2755148193 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840009690 con 0x7f28300051c0 2024-05-10T10:41:21.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.512+0000 7f28507c8640 1 -- 172.21.15.53:0/2755148193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28300051c0 msgr2=0x7f283000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.512+0000 7f28507c8640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28300051c0 0x7f283000b9c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2840009a90 tx=0x7f2830005830 comp rx=0 tx=0).stop 2024-05-10T10:41:21.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.512+0000 7f28507c8640 1 -- 172.21.15.53:0/2755148193 shutdown_connections 2024-05-10T10:41:21.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.512+0000 7f28507c8640 1 --2- 172.21.15.53:0/2755148193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28300051c0 0x7f283000b9c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.512+0000 7f28507c8640 1 -- 172.21.15.53:0/2755148193 >> 172.21.15.53:0/2755148193 conn(0x7f28480d2df0 msgr2=0x7f28480d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.512+0000 7f28507c8640 1 -- 172.21.15.53:0/2755148193 shutdown_connections 2024-05-10T10:41:21.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.512+0000 7f28507c8640 1 -- 172.21.15.53:0/2755148193 wait complete. 2024-05-10T10:41:21.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.513+0000 7f28507c8640 1 Processor -- start 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.513+0000 7f28507c8640 1 -- start start 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f28507c8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28300051c0 0x7f2848156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f28507c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28481f8210 0x7f28481f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f28507c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f284815ddd0 0x7f284815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f28507c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28480b3de0 con 0x7f284815ddd0 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f28507c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28480b4150 con 0x7f28481f8210 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f28507c8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f28480b45e0 con 0x7f28300051c0 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f284815ddd0 0x7f284815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f284815ddd0 0x7f284815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51050/0 (socket says 172.21.15.53:51050) 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 -- 172.21.15.53:0/3244365406 learned_addr learned my addr 172.21.15.53:0/3244365406 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284dd63640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28481f8210 0x7f28481f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284e564640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28300051c0 0x7f2848156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 -- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28300051c0 msgr2=0x7f2848156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28300051c0 0x7f2848156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 -- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28481f8210 msgr2=0x7f28481f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28481f8210 0x7f28481f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.514+0000 7f284ed65640 1 -- 172.21.15.53:0/3244365406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28481f9020 con 0x7f284815ddd0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.515+0000 7f284ed65640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f284815ddd0 0x7f284815e250 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bb0 tx=0x7f28400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.515+0000 7f2846ffd640 1 -- 172.21.15.53:0/3244365406 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840036050 con 0x7f284815ddd0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.515+0000 7f2846ffd640 1 -- 172.21.15.53:0/3244365406 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2840037040 con 0x7f284815ddd0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.515+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28481614d0 con 0x7f284815ddd0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.515+0000 7f2846ffd640 1 -- 172.21.15.53:0/3244365406 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840009690 con 0x7f284815ddd0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.515+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284815d2f0 con 0x7f284815ddd0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.516+0000 7f2846ffd640 1 -- 172.21.15.53:0/3244365406 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2840044020 con 0x7f284815ddd0 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.517+0000 7f2846ffd640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2834076300 0x7f28340787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.517+0000 7f2846ffd640 1 -- 172.21.15.53:0/3244365406 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f28400b7750 con 0x7f284815ddd0 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.517+0000 7f284e564640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2834076300 0x7f28340787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.518+0000 7f28535bf740 1 Processor -- start 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.518+0000 7f284e564640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2834076300 0x7f28340787c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2838000f80 tx=0x7f2838009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.518+0000 7f28535bf740 1 -- start start 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f28535bf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427c9bb0 0x55e9427cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f28535bf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e9427338d0 con 0x55e9427f6f50 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e9426ecc10 con 0x55e9427f7330 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e94272f890 con 0x55e9427c9bb0 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51052/0 (socket says 172.21.15.53:51052) 2024-05-10T10:41:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 -- 172.21.15.53:0/911178994 learned_addr learned my addr 172.21.15.53:0/911178994 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284ed65640 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427c9bb0 0x55e9427cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284dd63640 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 -- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427c9bb0 msgr2=0x55e9427cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427c9bb0 0x55e9427cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 -- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 msgr2=0x55e9427f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 -- 172.21.15.53:0/911178994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9426f45d0 con 0x55e9427f6f50 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f284e564640 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427f9760 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2838007fd0 tx=0x7f2838015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9455b90ebea65bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f282ffff640 1 -- 172.21.15.53:0/911178994 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838019070 con 0x55e9427f6f50 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f282ffff640 1 -- 172.21.15.53:0/911178994 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283801a040 con 0x55e9427f6f50 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.519+0000 7f282ffff640 1 -- 172.21.15.53:0/911178994 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838015c90 con 0x55e9427f6f50 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 -- 172.21.15.53:0/911178994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 msgr2=0x55e9427f9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427f9760 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2838007fd0 tx=0x7f2838015000 comp rx=0 tx=0).stop 2024-05-10T10:41:21.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 -- 172.21.15.53:0/911178994 shutdown_connections 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427c9bb0 0x55e9427cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 --2- 172.21.15.53:0/911178994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427f9760 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 -- 172.21.15.53:0/911178994 >> 172.21.15.53:0/911178994 conn(0x55e9427cbe20 msgr2=0x55e942814760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 -- 172.21.15.53:0/911178994 shutdown_connections 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.520+0000 7f28535bf740 1 -- 172.21.15.53:0/911178994 wait complete. 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.521+0000 7f28535bf740 1 -- start start 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.521+0000 7f28535bf740 1 Processor -- start 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f6f50 0x55e9428176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f28535bf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f28535bf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e9427cefd0 con 0x55e942817c10 2024-05-10T10:41:21.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e9426f0760 con 0x55e9427f7330 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e9427cf2b0 con 0x55e9427f6f50 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f28535bf740 10 client.0 _ll_register_callbacks cb 0x55e942729cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427f97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51064/0 (socket says 172.21.15.53:51064) 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 -- 172.21.15.53:0/1155922482 learned_addr learned my addr 172.21.15.53:0/1155922482 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284e564640 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f6f50 0x55e9428176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284dd63640 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 -- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f6f50 msgr2=0x55e9428176d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f6f50 0x55e9428176d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 -- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 msgr2=0x55e9427f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.522+0000 7f284ed65640 1 -- 172.21.15.53:0/1155922482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9424452d0 con 0x55e942817c10 2024-05-10T10:41:21.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f284ed65640 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427f97d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2840009a90 tx=0x7f28400802e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a67d11f459a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f282d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f282d7fa640 1 -- 172.21.15.53:0/1155922482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840084bd0 con 0x55e942817c10 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f282d7fa640 1 -- 172.21.15.53:0/1155922482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f2840095be0 con 0x55e942817c10 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283801a040 con 0x55e942817c10 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f282d7fa640 1 -- 172.21.15.53:0/1155922482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f28400b5050 con 0x55e942817c10 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 10 client.7599 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e94250f6b0 con 0x55e942817c10 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f282d7fa640 1 -- 172.21.15.53:0/1155922482 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283801a040 con 0x55e942817c10 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f282d7fa640 1 -- 172.21.15.53:0/1155922482 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f2840081b50 con 0x55e942817c10 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 10 client.7599 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 10 client.7599 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 20 client.7599 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e942815d60 0x55e942816140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 4 client.7599 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9426b7760 con 0x55e942815d60 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.523+0000 7f284e564640 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e942815d60 0x55e942816140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.524+0000 7f282d7fa640 10 client.7599 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.524+0000 7f284e564640 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e942815d60 0x55e942816140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbca7b9785d194f server_cookie=34c0a8cd5612ad50 in_seq=0 out_seq=0 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.524+0000 7f282d7fa640 1 -- 172.21.15.53:0/1155922482 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9426b7760 con 0x55e942815d60 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.524+0000 7f282d7fa640 10 client.7599 handle_command_reply: tid=0 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.524+0000 7f28535bf740 1 client.7599 shutdown 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.524+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 msgr2=0x55e9427f97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.524+0000 7f28535bf740 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427f97d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2840009a90 tx=0x7f28400802e0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 shutdown_connections 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e942815d60 0x55e942816140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427f97d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f282d7fa640 0 client.7599 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:21.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 --2- 172.21.15.53:0/1155922482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f6f50 0x55e9428176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 >> 172.21.15.53:0/1155922482 conn(0x55e9427c9bb0 msgr2=0x55e9427cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 shutdown_connections 2024-05-10T10:41:21.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 1 -- 172.21.15.53:0/1155922482 wait complete. 2024-05-10T10:41:21.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.525+0000 7f28535bf740 20 client.7599 trim_cache size 0 max 16384 2024-05-10T10:41:21.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.544+0000 7f28535bf740 1 Processor -- start 2024-05-10T10:41:21.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.544+0000 7f28535bf740 1 -- start start 2024-05-10T10:41:21.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.544+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f6f50 0x55e9427ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.544+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f7330 0x55e9427cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.544+0000 7f28535bf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f28535bf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2848201360 con 0x55e942817c10 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28482006c0 con 0x55e9427f6f50 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2848200d10 con 0x55e9427f7330 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51076/0 (socket says 172.21.15.53:51076) 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 -- 172.21.15.53:0/332549102 learned_addr learned my addr 172.21.15.53:0/332549102 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284dd63640 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f7330 0x55e9427cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284e564640 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f6f50 0x55e9427ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 -- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f7330 msgr2=0x55e9427cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f7330 0x55e9427cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 -- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f6f50 msgr2=0x55e9427ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f6f50 0x55e9427ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 -- 172.21.15.53:0/332549102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9427f9e80 con 0x55e942817c10 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f284ed65640 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427cd900 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f284007db40 tx=0x7f2840047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaa24268db1ed93 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f282f7fe640 1 -- 172.21.15.53:0/332549102 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840089640 con 0x55e942817c10 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f282f7fe640 1 -- 172.21.15.53:0/332549102 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2840084bd0 con 0x55e942817c10 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.545+0000 7f282f7fe640 1 -- 172.21.15.53:0/332549102 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840095be0 con 0x55e942817c10 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 -- 172.21.15.53:0/332549102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 msgr2=0x55e9427cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427cd900 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f284007db40 tx=0x7f2840047000 comp rx=0 tx=0).stop 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 -- 172.21.15.53:0/332549102 shutdown_connections 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e942817c10 0x55e9427cd900 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e9427f7330 0x55e9427cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 --2- 172.21.15.53:0/332549102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f6f50 0x55e9427ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 -- 172.21.15.53:0/332549102 >> 172.21.15.53:0/332549102 conn(0x55e9427c9bb0 msgr2=0x55e9428243e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 -- 172.21.15.53:0/332549102 shutdown_connections 2024-05-10T10:41:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 -- 172.21.15.53:0/332549102 wait complete. 2024-05-10T10:41:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.546+0000 7f28535bf740 1 -- start start 2024-05-10T10:41:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.547+0000 7f28535bf740 1 Processor -- start 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.547+0000 7f28535bf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.547+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.547+0000 7f28535bf740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e942817c10 0x55e9427c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.547+0000 7f28535bf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e9427cefd0 con 0x55e9427f6f50 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.547+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e9427cf2b0 con 0x55e9427f7330 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.547+0000 7f28535bf740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e9426f0760 con 0x55e942817c10 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f28535bf740 10 client.0 _ll_register_callbacks cb 0x7f2840009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51082/0 (socket says 172.21.15.53:51082) 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 -- 172.21.15.53:0/1376191110 learned_addr learned my addr 172.21.15.53:0/1376191110 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284dd63640 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284ed65640 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e942817c10 0x55e9427c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 -- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e942817c10 msgr2=0x55e9427c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e942817c10 0x55e9427c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 -- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 msgr2=0x55e9427c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 -- 172.21.15.53:0/1376191110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9427cdc10 con 0x55e9427f6f50 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f284e564640 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427cd3c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2838007fa0 tx=0x7f28380014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1447473dd60fa860 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f282d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f282d7fa640 1 -- 172.21.15.53:0/1376191110 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838019070 con 0x55e9427f6f50 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f282d7fa640 1 -- 172.21.15.53:0/1376191110 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f2838026040 con 0x55e9427f6f50 2024-05-10T10:41:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2840084bd0 con 0x55e9427f6f50 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f282d7fa640 1 -- 172.21.15.53:0/1376191110 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2838043050 con 0x55e9427f6f50 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f28535bf740 10 client.7605 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.548+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e942814d80 con 0x55e9427f6f50 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f282d7fa640 1 -- 172.21.15.53:0/1376191110 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2840084bd0 con 0x55e9427f6f50 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f282d7fa640 1 -- 172.21.15.53:0/1376191110 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f2838046020 con 0x55e9427f6f50 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 10 client.7605 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 10 client.7605 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 20 client.7605 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e94280d400 0x55e94280d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 4 client.7605 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:41:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "status"}) -- 0x55e9427414e0 con 0x55e94280d400 2024-05-10T10:41:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f284dd63640 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e94280d400 0x55e94280d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f282d7fa640 10 client.7605 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f284dd63640 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e94280d400 0x55e94280d7e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3bfc22195face3 server_cookie=ac33864e24a17a5f in_seq=0 out_seq=0 2024-05-10T10:41:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f282d7fa640 1 -- 172.21.15.53:0/1376191110 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55e9427414e0 con 0x55e94280d400 2024-05-10T10:41:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f282d7fa640 10 client.7605 handle_command_reply: tid=0 2024-05-10T10:41:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 1 client.7605 shutdown 2024-05-10T10:41:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 msgr2=0x55e9427cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.549+0000 7f28535bf740 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427cd3c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2838007fa0 tx=0x7f28380014a0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 shutdown_connections 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55e94280d400 0x55e94280d7e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e942817c10 0x55e9427c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9427f7330 0x55e9427c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f282d7fa640 0 client.7605 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 --2- 172.21.15.53:0/1376191110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e9427f6f50 0x55e9427cd3c0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 >> 172.21.15.53:0/1376191110 conn(0x55e9427c9bb0 msgr2=0x55e9427ed200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 shutdown_connections 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 1 -- 172.21.15.53:0/1376191110 wait complete. 2024-05-10T10:41:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28535bf740 20 client.7605 trim_cache size 0 max 16384 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 7479, 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 5.7933082550000004, 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 82, 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 103, 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 103, 2024-05-10T10:41:21.561 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 11.099455085000001 2024-05-10T10:41:21.562 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:41:21.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2834076300 msgr2=0x7f28340787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2834076300 0x7f28340787c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2838000f80 tx=0x7f2838009040 comp rx=0 tx=0).stop 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f284815ddd0 msgr2=0x7f284815e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f284815ddd0 0x7f284815e250 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bb0 tx=0x7f28400046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 shutdown_connections 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2834076300 0x7f28340787c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f284815ddd0 0x7f284815e250 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28481f8210 0x7f28481f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 --2- 172.21.15.53:0/3244365406 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28300051c0 0x7f2848156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 >> 172.21.15.53:0/3244365406 conn(0x7f28480d2df0 msgr2=0x7f2848158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 shutdown_connections 2024-05-10T10:41:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.550+0000 7f28507c8640 1 -- 172.21.15.53:0/3244365406 wait complete. 2024-05-10T10:41:21.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:41:21.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.706+0000 7f0ec0990640 1 Processor -- start 2024-05-10T10:41:21.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.706+0000 7f0ec0990640 1 -- start start 2024-05-10T10:41:21.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0ec0990640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 0x7f0ebc16a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0ec0990640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ebc0b0a90 con 0x7f0ebc15f4e0 2024-05-10T10:41:21.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0ec0990640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ebc0b3540 con 0x7f0ebc15f8c0 2024-05-10T10:41:21.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0ec0990640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0ebc0b23f0 con 0x7f0ebc168220 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0eba59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0ebc15f4e0 0x7f0ebc06d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46414/0 (socket says 172.21.15.53:46414) 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0eba59c640 1 -- 172.21.15.53:0/840426882 learned_addr learned my addr 172.21.15.53:0/840426882 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0ebad9d640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 0x7f0ebc16a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538842923 0 0) 0x7f0ebc0b0a90 con 0x7f0ebc15f4e0 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.707+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea4003520 con 0x7f0ebc15f4e0 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2415805783 0 0) 0x7f0ea4003520 con 0x7f0ebc15f4e0 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 msgr2=0x7f0ebc16a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 0x7f0ebc16a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 >> v1:172.21.15.135:6789/0 conn(0x7f0ebc15f8c0 legacy=0x7f0ebc06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc0b3820 con 0x7f0ebc15f4e0 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f0eac001670 con 0x7f0ebc15f4e0 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 >> v1:172.21.15.53:6789/0 conn(0x7f0ebc15f4e0 legacy=0x7f0ebc06d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ea40039e0 0x7f0ea4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 0x7f0ea4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0b23f0 con 0x7f0ea40039e0 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea400baf0 con 0x7f0ea4004d40 2024-05-10T10:41:21.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea400bcf0 con 0x7f0ebc168220 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eba59c640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ea40039e0 0x7f0ea4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eb9d9b640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 0x7f0ea4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0ebad9d640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eba59c640 1 -- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 msgr2=0x7f0ea4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eba59c640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc168220 0x7f0ea4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eba59c640 1 -- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea4004d40 msgr2=0x7f0ea40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eba59c640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0eba59c640 1 -- 172.21.15.53:0/840426882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea4003520 con 0x7f0ea40039e0 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.708+0000 7f0ebad9d640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea4004d40 0x7f0ea40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.709+0000 7f0eba59c640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ea40039e0 0x7f0ea4003e00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c210 tx=0x7f0ea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613354275ce7f438 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.709+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac007d30 con 0x7f0ea40039e0 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.709+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac005af0 con 0x7f0ea40039e0 2024-05-10T10:41:21.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.709+0000 7f0eb959a640 1 -- 172.21.15.53:0/840426882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac007070 con 0x7f0ea40039e0 2024-05-10T10:41:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.709+0000 7f0ec0990640 1 -- 172.21.15.53:0/840426882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ea40039e0 msgr2=0x7f0ea4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.709+0000 7f0ec0990640 1 --2- 172.21.15.53:0/840426882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ea40039e0 0x7f0ea4003e00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c210 tx=0x7f0ea4005830 comp rx=0 tx=0).stop 2024-05-10T10:41:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.709+0000 7f0ebad9d640 1 -- 172.21.15.53:0/840426882 reap_dead start 2024-05-10T10:41:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.710+0000 7f0ec0990640 1 -- 172.21.15.53:0/840426882 shutdown_connections 2024-05-10T10:41:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.710+0000 7f0ec0990640 1 -- 172.21.15.53:0/840426882 >> 172.21.15.53:0/840426882 conn(0x7f0ebc0d2df0 msgr2=0x7f0ebc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.710+0000 7f0ec0990640 1 -- 172.21.15.53:0/840426882 shutdown_connections 2024-05-10T10:41:21.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.710+0000 7f0ec0990640 1 -- 172.21.15.53:0/840426882 wait complete. 2024-05-10T10:41:21.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.711+0000 7f0ec0990640 1 Processor -- start 2024-05-10T10:41:21.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.711+0000 7f0ec0990640 1 -- start start 2024-05-10T10:41:21.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea40039e0 0x7f0ebc0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ebc157c90 0x7f0ebc1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc0c8b50 0x7f0ebc0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0b3de0 con 0x7f0ebc157c90 2024-05-10T10:41:21.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0b4150 con 0x7f0ea40039e0 2024-05-10T10:41:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0ebc06e7f0 con 0x7f0ebc0c8b50 2024-05-10T10:41:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ebc157c90 0x7f0ebc1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ebc157c90 0x7f0ebc1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51094/0 (socket says 172.21.15.53:51094) 2024-05-10T10:41:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 -- 172.21.15.53:0/808533316 learned_addr learned my addr 172.21.15.53:0/808533316 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ebad9d640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc0c8b50 0x7f0ebc0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eba59c640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea40039e0 0x7f0ebc0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 -- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc0c8b50 msgr2=0x7f0ebc0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc0c8b50 0x7f0ebc0c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 -- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea40039e0 msgr2=0x7f0ebc0c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea40039e0 0x7f0ebc0c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 -- 172.21.15.53:0/808533316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc154bb0 con 0x7f0ebc157c90 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ebad9d640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc0c8b50 0x7f0ebc0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eba59c640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea40039e0 0x7f0ebc0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eb9d9b640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ebc157c90 0x7f0ebc1536b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e580 tx=0x7f0eb000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eaaffd640 1 -- 172.21.15.53:0/808533316 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00052e0 con 0x7f0ebc157c90 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eaaffd640 1 -- 172.21.15.53:0/808533316 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb0018070 con 0x7f0ebc157c90 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc154e90 con 0x7f0ebc157c90 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0eaaffd640 1 -- 172.21.15.53:0/808533316 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0003810 con 0x7f0ebc157c90 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.712+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc155450 con 0x7f0ebc157c90 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.714+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc06a220 con 0x7f0ebc157c90 2024-05-10T10:41:21.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.714+0000 7f0eaaffd640 1 -- 172.21.15.53:0/808533316 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0eb00039b0 con 0x7f0ebc157c90 2024-05-10T10:41:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.715+0000 7f0eaaffd640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0e9c076300 0x7f0e9c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.715+0000 7f0eaaffd640 1 -- 172.21.15.53:0/808533316 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f0eb0097510 con 0x7f0ebc157c90 2024-05-10T10:41:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.715+0000 7f0eba59c640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0e9c076300 0x7f0e9c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.715+0000 7f0eba59c640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0e9c076300 0x7f0e9c0787c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0eac032000 tx=0x7f0eac009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:21.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.719+0000 7f0eaaffd640 1 -- 172.21.15.53:0/808533316 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0eb0060cf0 con 0x7f0ebc157c90 2024-05-10T10:41:21.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.949+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0ebc069640 con 0x7f0ebc157c90 2024-05-10T10:41:21.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.950+0000 7f0eaaffd640 1 -- 172.21.15.53:0/808533316 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f0eb00605f0 con 0x7f0ebc157c90 2024-05-10T10:41:21.952 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0e9c076300 msgr2=0x7f0e9c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0e9c076300 0x7f0e9c0787c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0eac032000 tx=0x7f0eac009030 comp rx=0 tx=0).stop 2024-05-10T10:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ebc157c90 msgr2=0x7f0ebc1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ebc157c90 0x7f0ebc1536b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e580 tx=0x7f0eb000ea40 comp rx=0 tx=0).stop 2024-05-10T10:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 shutdown_connections 2024-05-10T10:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0e9c076300 0x7f0e9c0787c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ebc0c8b50 0x7f0ebc0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ebc157c90 0x7f0ebc1536b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 --2- 172.21.15.53:0/808533316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ea40039e0 0x7f0ebc0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 >> 172.21.15.53:0/808533316 conn(0x7f0ebc0d2df0 msgr2=0x7f0ebc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 shutdown_connections 2024-05-10T10:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:21.952+0000 7f0ec0990640 1 -- 172.21.15.53:0/808533316 wait complete. 2024-05-10T10:41:21.965 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:41:21.965 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:41:21.965 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:41:21.966 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:41:21.966 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:21.966 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:41:21.966 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:41:21.966 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:41:21.966 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:41:21.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:21.967 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:41:21.985 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:41:21.985 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:41:21.985 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:41:21.985 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:21.985 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:41:21.986 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:21.986 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:41:21.986 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:21.986 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:41:21.986 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:41:21.987 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:21.987 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:41:21.987 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:41:21.987 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:21.987 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:41:21.987 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:41:21.987 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42287sec preferred_lft 42287sec 2024-05-10T10:41:21.988 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:41:21.988 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:21.988 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:41:21.988 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:21.988 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:41:21.988 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:41:21.989 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:21.989 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:41:21.989 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:21.989 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:41:21.989 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:21.989 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:41:21.990 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:41:21.990 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:41:21.990 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:21.990 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:41:21.991 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:41:21.991 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:41:21.991 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:21.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:21.992 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:41:22.050 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:41:22.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:22.052 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:41:22.108 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:41:22.109 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:41:22.109 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:41:22.109 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:41:22.109 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:41:22.109 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:41:22.109 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:41:22.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:41:22.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:41:22.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:41:22.110 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:41:22.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:41:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:22.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:22.240 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:41:22.256 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:41:22.256 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:41:22.257 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:41:22.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:22.257 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:41:22.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:41:22.323 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:22.379 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:41:22.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:41:22.379 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:22.435 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:41:22.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:22.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:22.521 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:22.521 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:41:22.521 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:41:22.530 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:22.530 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:41:22.530 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:41:22.530 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:41:22.530 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:41:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:41:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:41:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:41:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:41:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:41:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:41:22.536 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:41:22.536 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:41:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:41:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:41:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:41:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:41:22.538 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:41:22.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:22.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:22.564 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:41:22.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:41:22.660 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:22.667 INFO:teuthology.orchestra.run.smithi053.stdout:client6233 2024-05-10T10:41:22.667 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:22.667 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:22.684 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:22.684 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:22.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:22.741 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-10T10:41:22.840 INFO:teuthology.orchestra.run.smithi053.stderr:6+0 records in 2024-05-10T10:41:22.841 INFO:teuthology.orchestra.run.smithi053.stderr:6+0 records out 2024-05-10T10:41:22.841 INFO:teuthology.orchestra.run.smithi053.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0360369 s, 175 MB/s 2024-05-10T10:41:22.841 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:22.841 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:22.858 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:22.858 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:22.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:41:23.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.058+0000 7f25fd919640 1 Processor -- start 2024-05-10T10:41:23.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.058+0000 7f25fd919640 1 -- start start 2024-05-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25fd919640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f25f80c6c80 0x7f25f80c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25fd919640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f25f80cee40 0x7f25f80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25fd919640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f806f1c0 con 0x7f25f81598c0 2024-05-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25fd919640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f25f806f4a0 con 0x7f25f80cee40 2024-05-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25fd919640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f25f806f780 con 0x7f25f80c6c80 2024-05-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25f77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f25f80cee40 0x7f25f80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25f77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f25f80cee40 0x7f25f80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40916/0 (socket says 172.21.15.53:40916) 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25f77fe640 1 -- 172.21.15.53:0/1627824405 learned_addr learned my addr 172.21.15.53:0/1627824405 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25f6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f25f81598c0 0x7f25f806e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49476/0 (socket says 172.21.15.53:49476) 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.059+0000 7f25f67fc640 1 --2- 172.21.15.53:0/1627824405 >> v2:172.21.15.135:3300/0 conn(0x7f25f80c6c80 0x7f25f80c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f77fe640 1 -- 172.21.15.53:0/1627824405 >> v2:172.21.15.135:3300/0 conn(0x7f25f80c6c80 msgr2=0x7f25f80c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f77fe640 1 --2- 172.21.15.53:0/1627824405 >> v2:172.21.15.135:3300/0 conn(0x7f25f80c6c80 0x7f25f80c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f77fe640 1 -- 172.21.15.53:0/1627824405 >> v1:172.21.15.53:6789/0 conn(0x7f25f81598c0 legacy=0x7f25f806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f77fe640 1 -- 172.21.15.53:0/1627824405 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f806fa60 con 0x7f25f80cee40 2024-05-10T10:41:23.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1627824405 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 100877194 0 0) 0x7f25f806f1c0 con 0x7f25f81598c0 2024-05-10T10:41:23.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f77fe640 1 --2- 172.21.15.53:0/1627824405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f80cee40 0x7f25f80cf290 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f25f8070400 tx=0x7f25f002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91db13094413be67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:23.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1627824405 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000b2e0 con 0x7f25f80cee40 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1627824405 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f0037070 con 0x7f25f80cee40 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.060+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1627824405 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000b480 con 0x7f25f80cee40 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 -- 172.21.15.53:0/1627824405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f80cee40 msgr2=0x7f25f80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 --2- 172.21.15.53:0/1627824405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f80cee40 0x7f25f80cf290 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f25f8070400 tx=0x7f25f002c940 comp rx=0 tx=0).stop 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 -- 172.21.15.53:0/1627824405 shutdown_connections 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 --2- 172.21.15.53:0/1627824405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f80cee40 0x7f25f80cf290 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 --2- 172.21.15.53:0/1627824405 >> v2:172.21.15.135:3300/0 conn(0x7f25f80c6c80 0x7f25f80c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 -- 172.21.15.53:0/1627824405 >> 172.21.15.53:0/1627824405 conn(0x7f25f80b2900 msgr2=0x7f25f80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 -- 172.21.15.53:0/1627824405 shutdown_connections 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.061+0000 7f25fd919640 1 -- 172.21.15.53:0/1627824405 wait complete. 2024-05-10T10:41:23.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.062+0000 7f25fd919640 1 Processor -- start 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25fd919640 1 -- start start 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25fd919640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25f80c6c80 0x7f25f8152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25fd919640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25f80cee40 0x7f25f815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25fd919640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f81598c0 0x7f25f8153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25fd919640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f25f806ee10 con 0x7f25f81598c0 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25fd919640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25f815c6e0 con 0x7f25f80cee40 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25fd919640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f25f815ef60 con 0x7f25f80c6c80 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25f77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f81598c0 0x7f25f8153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25f77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f81598c0 0x7f25f8153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40920/0 (socket says 172.21.15.53:40920) 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.063+0000 7f25f77fe640 1 -- 172.21.15.53:0/561261901 learned_addr learned my addr 172.21.15.53:0/561261901 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f67fc640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25f80cee40 0x7f25f815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f6ffd640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25f80c6c80 0x7f25f8152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f77fe640 1 -- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25f80c6c80 msgr2=0x7f25f8152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f77fe640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25f80c6c80 0x7f25f8152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f77fe640 1 -- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25f80cee40 msgr2=0x7f25f815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f77fe640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25f80cee40 0x7f25f815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f77fe640 1 -- 172.21.15.53:0/561261901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f8152d90 con 0x7f25f81598c0 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f6ffd640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25f80c6c80 0x7f25f8152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f67fc640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25f80cee40 0x7f25f815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25f77fe640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f81598c0 0x7f25f8153bd0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f25f0034040 tx=0x7f25f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25dffff640 1 -- 172.21.15.53:0/561261901 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000b720 con 0x7f25f81598c0 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25dffff640 1 -- 172.21.15.53:0/561261901 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f0032840 con 0x7f25f81598c0 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f80ae170 con 0x7f25f81598c0 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25dffff640 1 -- 172.21.15.53:0/561261901 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0032c30 con 0x7f25f81598c0 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.064+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f80ae990 con 0x7f25f81598c0 2024-05-10T10:41:23.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.065+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f806a220 con 0x7f25f81598c0 2024-05-10T10:41:23.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.066+0000 7f25dffff640 1 -- 172.21.15.53:0/561261901 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f25f003b400 con 0x7f25f81598c0 2024-05-10T10:41:23.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.066+0000 7f25dffff640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25e4076300 0x7f25e40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.066+0000 7f25dffff640 1 -- 172.21.15.53:0/561261901 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f25f00baed0 con 0x7f25f81598c0 2024-05-10T10:41:23.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.067+0000 7f25f6ffd640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25e4076300 0x7f25e40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.067+0000 7f25f6ffd640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25e4076300 0x7f25e40787c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000f60 tx=0x7f25e8007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:23.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.071+0000 7f25dffff640 1 -- 172.21.15.53:0/561261901 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f25f8152d90 con 0x7f25f81598c0 2024-05-10T10:41:23.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.275+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f25f8156db0 con 0x7f25f81598c0 2024-05-10T10:41:23.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.276+0000 7f25dffff640 1 -- 172.21.15.53:0/561261901 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) ==== 80+0+0 (secure 0 0 0) 0x7f25f0084610 con 0x7f25f81598c0 2024-05-10T10:41:23.277 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25e4076300 msgr2=0x7f25e40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25e4076300 0x7f25e40787c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25e8000f60 tx=0x7f25e8007ca0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f81598c0 msgr2=0x7f25f8153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f81598c0 0x7f25f8153bd0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f25f0034040 tx=0x7f25f0004a40 comp rx=0 tx=0).stop 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 shutdown_connections 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25e4076300 0x7f25e40787c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f81598c0 0x7f25f8153bd0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25f80cee40 0x7f25f815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 --2- 172.21.15.53:0/561261901 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25f80c6c80 0x7f25f8152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 >> 172.21.15.53:0/561261901 conn(0x7f25f80b2900 msgr2=0x7f25f80cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 shutdown_connections 2024-05-10T10:41:23.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.278+0000 7f25fd919640 1 -- 172.21.15.53:0/561261901 wait complete. 2024-05-10T10:41:23.290 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:41:23.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:23.291 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:23.322 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:41:23.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:23.322 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:41:23.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:23.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.520+0000 7fa6570d1640 1 Processor -- start 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa6570d1640 1 -- start start 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa6570d1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa65015f790 0x7fa6500cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa6570d1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6500d5d30 0x7fa6500d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa6570d1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6500d30b0 con 0x7fa65015f3b0 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa6570d1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa6500d3390 con 0x7fa6500d5d30 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa6570d1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa6500c8c80 con 0x7fa65015f790 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa65566e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6500d5d30 0x7fa6500d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa65566e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6500d5d30 0x7fa6500d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40922/0 (socket says 172.21.15.53:40922) 2024-05-10T10:41:23.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.521+0000 7fa65566e640 1 -- 172.21.15.53:0/3248273966 learned_addr learned my addr 172.21.15.53:0/3248273966 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:23.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.522+0000 7fa65566e640 1 -- 172.21.15.53:0/3248273966 >> v2:172.21.15.135:3300/0 conn(0x7fa65015f790 msgr2=0x7fa6500cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:23.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.522+0000 7fa65566e640 1 --2- 172.21.15.53:0/3248273966 >> v2:172.21.15.135:3300/0 conn(0x7fa65015f790 0x7fa6500cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.522+0000 7fa65566e640 1 -- 172.21.15.53:0/3248273966 >> v1:172.21.15.135:6789/0 conn(0x7fa65015f3b0 legacy=0x7fa6500d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:23.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.522+0000 7fa65566e640 1 -- 172.21.15.53:0/3248273966 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6500c8f60 con 0x7fa6500d5d30 2024-05-10T10:41:23.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.522+0000 7fa65566e640 1 --2- 172.21.15.53:0/3248273966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6500d5d30 0x7fa6500d0210 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa6500c8c00 tx=0x7fa64002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95d1548f6cff18d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:23.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa64f7fe640 1 -- 172.21.15.53:0/3248273966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640037070 con 0x7fa6500d5d30 2024-05-10T10:41:23.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa64f7fe640 1 -- 172.21.15.53:0/3248273966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa640032440 con 0x7fa6500d5d30 2024-05-10T10:41:23.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa64f7fe640 1 -- 172.21.15.53:0/3248273966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640032810 con 0x7fa6500d5d30 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 -- 172.21.15.53:0/3248273966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6500d5d30 msgr2=0x7fa6500d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 --2- 172.21.15.53:0/3248273966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6500d5d30 0x7fa6500d0210 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa6500c8c00 tx=0x7fa64002a230 comp rx=0 tx=0).stop 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 -- 172.21.15.53:0/3248273966 shutdown_connections 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 --2- 172.21.15.53:0/3248273966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6500d5d30 0x7fa6500d0210 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 --2- 172.21.15.53:0/3248273966 >> v2:172.21.15.135:3300/0 conn(0x7fa65015f790 0x7fa6500cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 -- 172.21.15.53:0/3248273966 >> 172.21.15.53:0/3248273966 conn(0x7fa6500d5130 msgr2=0x7fa6500d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 -- 172.21.15.53:0/3248273966 shutdown_connections 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.523+0000 7fa6570d1640 1 -- 172.21.15.53:0/3248273966 wait complete. 2024-05-10T10:41:23.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.524+0000 7fa6570d1640 1 Processor -- start 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.524+0000 7fa6570d1640 1 -- start start 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.525+0000 7fa6570d1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6500d5d30 0x7fa6500cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.525+0000 7fa6570d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa65015f3b0 0x7fa6500cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.525+0000 7fa6570d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6501f8400 0x7fa6501f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.525+0000 7fa6570d1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa6500d3f80 con 0x7fa65015f3b0 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.525+0000 7fa6570d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6500d3e00 con 0x7fa6501f8400 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.525+0000 7fa6570d1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa6500ca770 con 0x7fa6500d5d30 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.525+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa65015f3b0 0x7fa6500cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa65015f3b0 0x7fa6500cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40938/0 (socket says 172.21.15.53:40938) 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 -- 172.21.15.53:0/2937609784 learned_addr learned my addr 172.21.15.53:0/2937609784 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa65566e640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6501f8400 0x7fa6501f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa654e6d640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6500d5d30 0x7fa6500cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 -- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6500d5d30 msgr2=0x7fa6500cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6500d5d30 0x7fa6500cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 -- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6501f8400 msgr2=0x7fa6501f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6501f8400 0x7fa6501f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 -- 172.21.15.53:0/2937609784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6500cf520 con 0x7fa65015f3b0 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa654e6d640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6500d5d30 0x7fa6500cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64ffff640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa65015f3b0 0x7fa6500cefe0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa63800b410 tx=0x7fa63800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:23.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64d7fa640 1 -- 172.21.15.53:0/2937609784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638015020 con 0x7fa65015f3b0 2024-05-10T10:41:23.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64d7fa640 1 -- 172.21.15.53:0/2937609784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa638011850 con 0x7fa65015f3b0 2024-05-10T10:41:23.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa64d7fa640 1 -- 172.21.15.53:0/2937609784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638011c40 con 0x7fa65015f3b0 2024-05-10T10:41:23.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6500cc8e0 con 0x7fa65015f3b0 2024-05-10T10:41:23.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.526+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6500ccea0 con 0x7fa65015f3b0 2024-05-10T10:41:23.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.527+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65006a220 con 0x7fa65015f3b0 2024-05-10T10:41:23.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.528+0000 7fa64d7fa640 1 -- 172.21.15.53:0/2937609784 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa638018020 con 0x7fa65015f3b0 2024-05-10T10:41:23.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.528+0000 7fa64d7fa640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6300762d0 0x7fa630078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:23.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.528+0000 7fa64d7fa640 1 -- 172.21.15.53:0/2937609784 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7fa6380975e0 con 0x7fa65015f3b0 2024-05-10T10:41:23.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.528+0000 7fa654e6d640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6300762d0 0x7fa630078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:23.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.529+0000 7fa654e6d640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6300762d0 0x7fa630078790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa648001dc0 tx=0x7fa648009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:23.537 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.532+0000 7fa64d7fa640 1 -- 172.21.15.53:0/2937609784 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa63801f020 con 0x7fa65015f3b0 2024-05-10T10:41:23.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.762+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa65015e780 con 0x7fa65015f3b0 2024-05-10T10:41:23.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.762+0000 7fa64d7fa640 1 -- 172.21.15.53:0/2937609784 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5189 (secure 0 0 0) 0x7fa638060e40 con 0x7fa65015f3b0 2024-05-10T10:41:23.764 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:23.764 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:41:18.164463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4122299129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":4122299129},{"type":"v1","addr":"172.21.15.53:6835","nonce":4122299129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7479,"qdb_cluster":[7479]},"id":7}]} 2024-05-10T10:41:23.764 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 82 2024-05-10T10:41:23.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.764+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6300762d0 msgr2=0x7fa630078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.764+0000 7fa6570d1640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6300762d0 0x7fa630078790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa648001dc0 tx=0x7fa648009040 comp rx=0 tx=0).stop 2024-05-10T10:41:23.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.764+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa65015f3b0 msgr2=0x7fa6500cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:23.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.764+0000 7fa6570d1640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa65015f3b0 0x7fa6500cefe0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa63800b410 tx=0x7fa63800b8d0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 shutdown_connections 2024-05-10T10:41:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6300762d0 0x7fa630078790 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6501f8400 0x7fa6501f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa65015f3b0 0x7fa6500cefe0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 --2- 172.21.15.53:0/2937609784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6500d5d30 0x7fa6500cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 >> 172.21.15.53:0/2937609784 conn(0x7fa6500d5130 msgr2=0x7fa6500cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 shutdown_connections 2024-05-10T10:41:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:23.765+0000 7fa6570d1640 1 -- 172.21.15.53:0/2937609784 wait complete. 2024-05-10T10:41:23.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-10T10:41:23.911 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:41:23.912 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:41:23.912 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:24.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.014+0000 7f0746847640 1 Processor -- start 2024-05-10T10:41:24.015 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.014+0000 7f0746847640 1 -- start start 2024-05-10T10:41:24.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0746847640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0740071360 0x7f07400b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0746847640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07400aed60 0x7f07400af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0746847640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07400b5cc0 con 0x7f074015be70 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0746847640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f074006fb70 con 0x7f07400aed60 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0746847640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f07400703f0 con 0x7f0740071360 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0744de4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07400aed60 0x7f07400af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0744de4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07400aed60 0x7f07400af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40946/0 (socket says 172.21.15.53:40946) 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0744de4640 1 -- 172.21.15.53:0/34910127 learned_addr learned my addr 172.21.15.53:0/34910127 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f073ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f074015be70 0x7f0740070880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49480/0 (socket says 172.21.15.53:49480) 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0737fff640 1 --2- 172.21.15.53:0/34910127 >> v2:172.21.15.135:3300/0 conn(0x7f0740071360 0x7f07400b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0744de4640 1 -- 172.21.15.53:0/34910127 >> v2:172.21.15.135:3300/0 conn(0x7f0740071360 msgr2=0x7f07400b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0744de4640 1 --2- 172.21.15.53:0/34910127 >> v2:172.21.15.135:3300/0 conn(0x7f0740071360 0x7f07400b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0744de4640 1 -- 172.21.15.53:0/34910127 >> v1:172.21.15.53:6789/0 conn(0x7f074015be70 legacy=0x7f0740070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0744de4640 1 -- 172.21.15.53:0/34910127 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074015c250 con 0x7f07400aed60 2024-05-10T10:41:24.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f073f7fe640 1 -- 172.21.15.53:0/34910127 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3433197274 0 0) 0x7f07400b5cc0 con 0x7f074015be70 2024-05-10T10:41:24.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.015+0000 7f0737fff640 1 --2- 172.21.15.53:0/34910127 >> v2:172.21.15.135:3300/0 conn(0x7f0740071360 0x7f07400b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:24.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.016+0000 7f0744de4640 1 --2- 172.21.15.53:0/34910127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07400aed60 0x7f07400af1b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f07400d2c10 tx=0x7f073802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf73842694074be9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.016+0000 7f073f7fe640 1 -- 172.21.15.53:0/34910127 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738036070 con 0x7f07400aed60 2024-05-10T10:41:24.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.016+0000 7f073f7fe640 1 -- 172.21.15.53:0/34910127 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0738031600 con 0x7f07400aed60 2024-05-10T10:41:24.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.016+0000 7f073f7fe640 1 -- 172.21.15.53:0/34910127 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07380319d0 con 0x7f07400aed60 2024-05-10T10:41:24.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 -- 172.21.15.53:0/34910127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07400aed60 msgr2=0x7f07400af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 --2- 172.21.15.53:0/34910127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07400aed60 0x7f07400af1b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f07400d2c10 tx=0x7f073802a730 comp rx=0 tx=0).stop 2024-05-10T10:41:24.022 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 -- 172.21.15.53:0/34910127 shutdown_connections 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 --2- 172.21.15.53:0/34910127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07400aed60 0x7f07400af1b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 --2- 172.21.15.53:0/34910127 >> v2:172.21.15.135:3300/0 conn(0x7f0740071360 0x7f07400b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 -- 172.21.15.53:0/34910127 >> 172.21.15.53:0/34910127 conn(0x7f07400d1d70 msgr2=0x7f07400d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 -- 172.21.15.53:0/34910127 shutdown_connections 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.017+0000 7f0746847640 1 -- 172.21.15.53:0/34910127 wait complete. 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.018+0000 7f0746847640 1 Processor -- start 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.018+0000 7f0746847640 1 -- start start 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0746847640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0740071360 0x7f074015eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0746847640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07400aed60 0x7f07401590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0746847640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f074015be70 0x7f0740143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0746847640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0740070080 con 0x7f0740071360 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0746847640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f074006fda0 con 0x7f07400aed60 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0746847640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f07400b5690 con 0x7f074015be70 2024-05-10T10:41:24.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0740071360 0x7f074015eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0740071360 0x7f074015eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40956/0 (socket says 172.21.15.53:40956) 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 -- 172.21.15.53:0/2415044753 learned_addr learned my addr 172.21.15.53:0/2415044753 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0737fff640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07400aed60 0x7f07401590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0744de4640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f074015be70 0x7f0740143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 -- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f074015be70 msgr2=0x7f0740143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f074015be70 0x7f0740143390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 -- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07400aed60 msgr2=0x7f07401590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07400aed60 0x7f07401590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 -- 172.21.15.53:0/2415044753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074015c5f0 con 0x7f0740071360 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0737fff640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07400aed60 0x7f07401590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073ffff640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0740071360 0x7f074015eae0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f073000e780 tx=0x7f073000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073d7fa640 1 -- 172.21.15.53:0/2415044753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073001a070 con 0x7f0740071360 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073d7fa640 1 -- 172.21.15.53:0/2415044753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0730006bf0 con 0x7f0740071360 2024-05-10T10:41:24.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074015c8d0 con 0x7f0740071360 2024-05-10T10:41:24.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.019+0000 7f073d7fa640 1 -- 172.21.15.53:0/2415044753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07300090d0 con 0x7f0740071360 2024-05-10T10:41:24.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.020+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074015ce90 con 0x7f0740071360 2024-05-10T10:41:24.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.021+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07400693f0 con 0x7f0740071360 2024-05-10T10:41:24.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.021+0000 7f073d7fa640 1 -- 172.21.15.53:0/2415044753 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f073001d070 con 0x7f0740071360 2024-05-10T10:41:24.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.022+0000 7f073d7fa640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f07200762d0 0x7f0720078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.022+0000 7f073d7fa640 1 -- 172.21.15.53:0/2415044753 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6159+0+0 (secure 0 0 0) 0x7f0730098cc0 con 0x7f0740071360 2024-05-10T10:41:24.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.022+0000 7f0737fff640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f07200762d0 0x7f0720078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.022+0000 7f0737fff640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f07200762d0 0x7f0720078790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f074006d8d0 tx=0x7f0728008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.026+0000 7f073d7fa640 1 -- 172.21.15.53:0/2415044753 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f07300623d0 con 0x7f0740071360 2024-05-10T10:41:24.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.256+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07401559e0 con 0x7f0740071360 2024-05-10T10:41:24.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.256+0000 7f073d7fa640 1 -- 172.21.15.53:0/2415044753 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5189 (secure 0 0 0) 0x7f0730061cd0 con 0x7f0740071360 2024-05-10T10:41:24.258 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:24.258 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:41:18.164463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4122299129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":4122299129},{"type":"v1","addr":"172.21.15.53:6835","nonce":4122299129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7479,"qdb_cluster":[7479]},"id":7}]} 2024-05-10T10:41:24.258 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 82 2024-05-10T10:41:24.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f07200762d0 msgr2=0x7f0720078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f07200762d0 0x7f0720078790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f074006d8d0 tx=0x7f0728008040 comp rx=0 tx=0).stop 2024-05-10T10:41:24.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0740071360 msgr2=0x7f074015eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0740071360 0x7f074015eae0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f073000e780 tx=0x7f073000ec40 comp rx=0 tx=0).stop 2024-05-10T10:41:24.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 shutdown_connections 2024-05-10T10:41:24.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f07200762d0 0x7f0720078790 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.260 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f074015be70 0x7f0740143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07400aed60 0x7f07401590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 --2- 172.21.15.53:0/2415044753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0740071360 0x7f074015eae0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 >> 172.21.15.53:0/2415044753 conn(0x7f07400d1d70 msgr2=0x7f07400af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.258+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 shutdown_connections 2024-05-10T10:41:24.261 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.259+0000 7f0746847640 1 -- 172.21.15.53:0/2415044753 wait complete. 2024-05-10T10:41:24.271 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-05-10T10:41:24.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 Processor -- start 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 -- start start 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef3985e720 0x55ef39861780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef399068e0 0x55ef39904f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef39905ad0 0x55ef399054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef397daf50 con 0x55ef39905ad0 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef397def90 con 0x55ef399068e0 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.315+0000 7f8cb491cf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ef39863a60 con 0x55ef3985e720 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef39905ad0 0x55ef399054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef39905ad0 0x55ef399054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40966/0 (socket says 172.21.15.53:40966) 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 -- 172.21.15.53:0/1825279502 learned_addr learned my addr 172.21.15.53:0/1825279502 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb194a640 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef3985e720 0x55ef39861780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb1149640 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef399068e0 0x55ef39904f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 -- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef3985e720 msgr2=0x55ef39861780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef3985e720 0x55ef39861780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 -- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef399068e0 msgr2=0x55ef39904f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef399068e0 0x55ef39904f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 -- 172.21.15.53:0/1825279502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef39835290 con 0x55ef39905ad0 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8cb214b640 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef39905ad0 0x55ef399054c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x55ef39836970 tx=0x7f8cac00d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33c4f974c2c9f66 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8ca3fff640 1 -- 172.21.15.53:0/1825279502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00f650 con 0x55ef39905ad0 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8ca3fff640 1 -- 172.21.15.53:0/1825279502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cac00f7f0 con 0x55ef39905ad0 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.316+0000 7f8ca3fff640 1 -- 172.21.15.53:0/1825279502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00fbc0 con 0x55ef39905ad0 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 -- 172.21.15.53:0/1825279502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef39905ad0 msgr2=0x55ef399054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef39905ad0 0x55ef399054c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x55ef39836970 tx=0x7f8cac00d7f0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 -- 172.21.15.53:0/1825279502 shutdown_connections 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef39905ad0 0x55ef399054c0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef399068e0 0x55ef39904f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/1825279502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef3985e720 0x55ef39861780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 -- 172.21.15.53:0/1825279502 >> 172.21.15.53:0/1825279502 conn(0x55ef3985d800 msgr2=0x55ef398357f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 -- 172.21.15.53:0/1825279502 shutdown_connections 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 -- 172.21.15.53:0/1825279502 wait complete. 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 Processor -- start 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.317+0000 7f8cb491cf40 1 -- start start 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb491cf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef3985e720 0x55ef39901960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb491cf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef39905ad0 0x55ef3985a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb491cf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef399068e0 0x55ef39904b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb491cf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef39906510 con 0x55ef399068e0 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb491cf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef39906230 con 0x55ef3985e720 2024-05-10T10:41:24.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb491cf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ef39861260 con 0x55ef39905ad0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef399068e0 0x55ef39904b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef399068e0 0x55ef39904b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40982/0 (socket says 172.21.15.53:40982) 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 -- 172.21.15.53:0/3492292210 learned_addr learned my addr 172.21.15.53:0/3492292210 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb1149640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef39905ad0 0x55ef3985a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb194a640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef3985e720 0x55ef39901960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 -- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef39905ad0 msgr2=0x55ef3985a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef39905ad0 0x55ef3985a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 -- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef3985e720 msgr2=0x55ef39901960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef3985e720 0x55ef39901960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.318+0000 7f8cb214b640 1 -- 172.21.15.53:0/3492292210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3985abe0 con 0x55ef399068e0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.319+0000 7f8cb214b640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef399068e0 0x55ef39904b10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8cac01d660 tx=0x55ef39863670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.319+0000 7f8ca2ffd640 1 -- 172.21.15.53:0/3492292210 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac01dc50 con 0x55ef399068e0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.319+0000 7f8ca2ffd640 1 -- 172.21.15.53:0/3492292210 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cac01ddf0 con 0x55ef399068e0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.319+0000 7f8ca2ffd640 1 -- 172.21.15.53:0/3492292210 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac026680 con 0x55ef399068e0 2024-05-10T10:41:24.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.319+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ef3985aec0 con 0x55ef399068e0 2024-05-10T10:41:24.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.319+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3985b480 con 0x55ef399068e0 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.320+0000 7f8ca2ffd640 1 -- 172.21.15.53:0/3492292210 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8cac01a3f0 con 0x55ef399068e0 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.321+0000 7f8ca2ffd640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c94074ce0 0x7f8c940771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.321+0000 7f8ca2ffd640 1 -- 172.21.15.53:0/3492292210 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6159+0+0 (secure 0 0 0) 0x7f8cac098af0 con 0x55ef399068e0 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.321+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef3985a710 0x55ef39859540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.321+0000 7f8cb194a640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c94074ce0 0x7f8c940771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.321+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x55ef39901ea0 con 0x55ef3985a710 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.321+0000 7f8cb1149640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef3985a710 0x55ef39859540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb1149640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef3985a710 0x55ef39859540 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb194a640 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c94074ce0 0x7f8c940771a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55ef39861b90 tx=0x7f8ca4007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb1149640 1 -- 172.21.15.53:0/3492292210 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f8ca80547a0 con 0x55ef3985a710 2024-05-10T10:41:24.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef3985a710 msgr2=0x55ef39859540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef3985a710 0x55ef39859540 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c94074ce0 msgr2=0x7f8c940771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8c94074ce0 0x7f8c940771a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55ef39861b90 tx=0x7f8ca4007040 comp rx=0 tx=0).stop 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef399068e0 msgr2=0x55ef39904b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 --2- 172.21.15.53:0/3492292210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef399068e0 0x55ef39904b10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8cac01d660 tx=0x55ef39863670 comp rx=0 tx=0).stop 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb214b640 1 -- 172.21.15.53:0/3492292210 reap_dead start 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 shutdown_connections 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 >> 172.21.15.53:0/3492292210 conn(0x55ef3985d800 msgr2=0x55ef398356d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 shutdown_connections 2024-05-10T10:41:24.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.322+0000 7f8cb491cf40 1 -- 172.21.15.53:0/3492292210 wait complete. 2024-05-10T10:41:24.331 INFO:teuthology.orchestra.run.smithi053.stdout:����� 2024-05-10T10:41:24.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:41:24.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.433+0000 7f392c445640 1 Processor -- start 2024-05-10T10:41:24.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.433+0000 7f392c445640 1 -- start start 2024-05-10T10:41:24.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.434+0000 7f392c445640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f392415f790 0x7f39240cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.434+0000 7f392c445640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39240d30b0 con 0x7f392415f3b0 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.434+0000 7f392c445640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39240d3390 con 0x7f39240d5d30 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.434+0000 7f392c445640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f39240c8c80 con 0x7f392415f790 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.434+0000 7f392a1e1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f392415f3b0 0x7f39240d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49506/0 (socket says 172.21.15.53:49506) 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.434+0000 7f392a1e1640 1 -- 172.21.15.53:0/3967457356 learned_addr learned my addr 172.21.15.53:0/3967457356 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39299e0640 1 --2- 172.21.15.53:0/3967457356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f392415f790 0x7f39240cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39291df640 1 -- 172.21.15.53:0/3967457356 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077833229 0 0) 0x7f39240d30b0 con 0x7f392415f3b0 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39291df640 1 -- 172.21.15.53:0/3967457356 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f390c003520 con 0x7f392415f3b0 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39291df640 1 -- 172.21.15.53:0/3967457356 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354072033 0 0) 0x7f39240d3390 con 0x7f39240d5d30 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39291df640 1 -- 172.21.15.53:0/3967457356 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39240d30b0 con 0x7f39240d5d30 2024-05-10T10:41:24.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39299e0640 1 -- 172.21.15.53:0/3967457356 >> v1:172.21.15.135:6789/0 conn(0x7f39240d5d30 legacy=0x7f39240d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39299e0640 1 -- 172.21.15.53:0/3967457356 >> v1:172.21.15.53:6789/0 conn(0x7f392415f3b0 legacy=0x7f39240d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.435+0000 7f39299e0640 1 -- 172.21.15.53:0/3967457356 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240c8f60 con 0x7f392415f790 2024-05-10T10:41:24.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.436+0000 7f39299e0640 1 --2- 172.21.15.53:0/3967457356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f392415f790 0x7f39240cfcd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39180097d0 tx=0x7f391802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71bf8493e262ac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.436+0000 7f39291df640 1 -- 172.21.15.53:0/3967457356 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918037070 con 0x7f392415f790 2024-05-10T10:41:24.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.436+0000 7f39291df640 1 -- 172.21.15.53:0/3967457356 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3918032440 con 0x7f392415f790 2024-05-10T10:41:24.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.436+0000 7f39291df640 1 -- 172.21.15.53:0/3967457356 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39180327f0 con 0x7f392415f790 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.437+0000 7f392c445640 1 -- 172.21.15.53:0/3967457356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f392415f790 msgr2=0x7f39240cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.437+0000 7f392c445640 1 --2- 172.21.15.53:0/3967457356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f392415f790 0x7f39240cfcd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39180097d0 tx=0x7f391802a210 comp rx=0 tx=0).stop 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.437+0000 7f392c445640 1 -- 172.21.15.53:0/3967457356 shutdown_connections 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.437+0000 7f392c445640 1 --2- 172.21.15.53:0/3967457356 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f392415f790 0x7f39240cfcd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.437+0000 7f392c445640 1 -- 172.21.15.53:0/3967457356 >> 172.21.15.53:0/3967457356 conn(0x7f39240d5130 msgr2=0x7f39240d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.437+0000 7f392c445640 1 -- 172.21.15.53:0/3967457356 shutdown_connections 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.437+0000 7f392c445640 1 -- 172.21.15.53:0/3967457356 wait complete. 2024-05-10T10:41:24.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.438+0000 7f392c445640 1 Processor -- start 2024-05-10T10:41:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.438+0000 7f392c445640 1 -- start start 2024-05-10T10:41:24.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392c445640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39240d5d30 0x7f39240ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392c445640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392415f3b0 0x7f3924161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392c445640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240ce290 0x7f3924152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392c445640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f39240d3f80 con 0x7f392415f3b0 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392c445640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39240d3e00 con 0x7f39240ce290 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392c445640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f39240ca770 con 0x7f39240d5d30 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392415f3b0 0x7f3924161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392415f3b0 0x7f3924161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40994/0 (socket says 172.21.15.53:40994) 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 -- 172.21.15.53:0/2676343525 learned_addr learned my addr 172.21.15.53:0/2676343525 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 -- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39240d5d30 msgr2=0x7f39240ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392a1e1640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39240d5d30 0x7f39240ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392a9e2640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240ce290 0x7f3924152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39240d5d30 0x7f39240ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 -- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240ce290 msgr2=0x7f3924152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240ce290 0x7f3924152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 -- 172.21.15.53:0/2676343525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3924162900 con 0x7f392415f3b0 2024-05-10T10:41:24.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392a1e1640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39240d5d30 0x7f39240ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:24.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392a9e2640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240ce290 0x7f3924152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:24.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f39299e0640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392415f3b0 0x7f3924161f70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f391802a250 tx=0x7f39180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f3922ffd640 1 -- 172.21.15.53:0/2676343525 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918041020 con 0x7f392415f3b0 2024-05-10T10:41:24.446 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f3922ffd640 1 -- 172.21.15.53:0/2676343525 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391803d6e0 con 0x7f392415f3b0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f3922ffd640 1 -- 172.21.15.53:0/2676343525 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918037040 con 0x7f392415f3b0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.439+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3924152e90 con 0x7f392415f3b0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.440+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39240cd7b0 con 0x7f392415f3b0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.440+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392406a220 con 0x7f392415f3b0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.441+0000 7f3922ffd640 1 -- 172.21.15.53:0/2676343525 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3918044020 con 0x7f392415f3b0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.442+0000 7f3922ffd640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39100762d0 0x7f3910078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.442+0000 7f3922ffd640 1 -- 172.21.15.53:0/2676343525 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6159+0+0 (secure 0 0 0) 0x7f39180b7ba0 con 0x7f392415f3b0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.442+0000 7f392a1e1640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39100762d0 0x7f3910078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.442+0000 7f392a1e1640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39100762d0 0x7f3910078790 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f391c000f60 tx=0x7f391c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:24.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.446+0000 7f3922ffd640 1 -- 172.21.15.53:0/2676343525 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f391804b070 con 0x7f392415f3b0 2024-05-10T10:41:24.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.676+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39240d09d0 con 0x7f392415f3b0 2024-05-10T10:41:24.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.676+0000 7f3922ffd640 1 -- 172.21.15.53:0/2676343525 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f3918081350 con 0x7f392415f3b0 2024-05-10T10:41:24.678 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39100762d0 msgr2=0x7f3910078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39100762d0 0x7f3910078790 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f391c000f60 tx=0x7f391c009040 comp rx=0 tx=0).stop 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392415f3b0 msgr2=0x7f3924161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392415f3b0 0x7f3924161f70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f391802a250 tx=0x7f39180046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 shutdown_connections 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f39100762d0 0x7f3910078790 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39240ce290 0x7f3924152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392415f3b0 0x7f3924161f70 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 --2- 172.21.15.53:0/2676343525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39240d5d30 0x7f39240ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 >> 172.21.15.53:0/2676343525 conn(0x7f39240d5130 msgr2=0x7f39240d2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 shutdown_connections 2024-05-10T10:41:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24.678+0000 7f392c445640 1 -- 172.21.15.53:0/2676343525 wait complete. 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:41:24.691 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:41:24.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:24.692 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:41:24.710 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:41:24.710 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:41:24.710 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:41:24.710 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:24.710 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:41:24.710 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:24.711 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42285sec preferred_lft 42285sec 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:24.712 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:41:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:41:24.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:24.714 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:41:24.776 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:41:24.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:24.778 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:41:24.836 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:41:24.836 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:41:24.837 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:41:24.837 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:41:24.837 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:41:24.837 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:41:24.838 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:41:24.838 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:41:24.838 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:41:24.838 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:41:24.838 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:41:24.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:41:24.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:24.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:24.974 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:41:24.994 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:41:24.994 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:41:24.994 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:41:24.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:24.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:41:25.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:41:25.061 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:25.117 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:41:25.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:41:25.118 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:25.174 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:41:25.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:25.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:25.259 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:25.260 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:41:25.260 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:41:25.268 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:25.269 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:41:25.269 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:41:25.269 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:41:25.269 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:41:25.278 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:41:25.278 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:41:25.278 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:41:25.279 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:41:25.279 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:41:25.279 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:41:25.279 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:41:25.279 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:41:25.279 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:41:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:41:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:41:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:41:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:41:25.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:41:25.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:25.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:25.307 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:41:25.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:41:25.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:25.406 INFO:teuthology.orchestra.run.smithi053.stdout:client6913 2024-05-10T10:41:25.407 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:25.407 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:25.424 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:25.424 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:25.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:25.480 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-10T10:41:25.568 INFO:teuthology.orchestra.run.smithi053.stderr:6+0 records in 2024-05-10T10:41:25.568 INFO:teuthology.orchestra.run.smithi053.stderr:6+0 records out 2024-05-10T10:41:25.568 INFO:teuthology.orchestra.run.smithi053.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0287755 s, 219 MB/s 2024-05-10T10:41:25.568 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:25.569 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:25.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:25.585 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-10T10:41:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:6+0 records in 2024-05-10T10:41:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:6+0 records out 2024-05-10T10:41:25.677 INFO:teuthology.orchestra.run.smithi053.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0340705 s, 185 MB/s 2024-05-10T10:41:25.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:25.677 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-10T10:41:25.695 INFO:teuthology.orchestra.run.smithi053.stdout:./ 2024-05-10T10:41:25.695 INFO:teuthology.orchestra.run.smithi053.stdout:./file2_sixmegs 2024-05-10T10:41:25.695 INFO:teuthology.orchestra.run.smithi053.stdout:./file3_sixmegs 2024-05-10T10:41:25.695 INFO:teuthology.orchestra.run.smithi053.stdout:./file1_sixmegs 2024-05-10T10:41:25.696 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:41:25.696 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:41:25.696 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:41:25.696 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:41:25.696 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-10T10:41:25.696 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:41:25.804 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:41:25.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:25.814 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:41:25.814 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:41:25.814 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:41:25.814 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:41:25.814 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-05-10T10:41:25.814 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:41:25.921 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2024-05-10T10:41:25.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:25.930 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:41:25.930 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:41:25.930 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:41:25.931 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:41:25.931 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-05-10T10:41:25.931 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:41:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627778 2024-05-10T10:41:26.041 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:26.048 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:41:26.048 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:41:26.048 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:41:26.048 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:41:26.048 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-05-10T10:41:26.048 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:41:26.156 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2024-05-10T10:41:26.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:41:26.166 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:26.166 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:26.224 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:26.224 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:26.282 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:41:26.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.428+0000 7fd4554da640 1 Processor -- start 2024-05-10T10:41:26.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.428+0000 7fd4554da640 1 -- start start 2024-05-10T10:41:26.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd4554da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4501598c0 0x7fd45006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd4554da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4500c7390 0x7fd4500cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd4554da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd45006f1c0 con 0x7fd45006ec40 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd4554da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd45006f4a0 con 0x7fd4500c7390 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd4554da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd45006f780 con 0x7fd4501598c0 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd44f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4500c7390 0x7fd4500cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd44e7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd45006ec40 0x7fd4500c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49522/0 (socket says 172.21.15.53:49522) 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd44f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4500c7390 0x7fd4500cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41000/0 (socket says 172.21.15.53:41000) 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.429+0000 7fd44e7fc640 1 -- 172.21.15.53:0/2089233837 learned_addr learned my addr 172.21.15.53:0/2089233837 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.430+0000 7fd44effd640 1 --2- 172.21.15.53:0/2089233837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4501598c0 0x7fd45006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.430+0000 7fd44f7fe640 1 -- 172.21.15.53:0/2089233837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4501598c0 msgr2=0x7fd45006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.430+0000 7fd44f7fe640 1 --2- 172.21.15.53:0/2089233837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4501598c0 0x7fd45006e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.430+0000 7fd44f7fe640 1 -- 172.21.15.53:0/2089233837 >> v1:172.21.15.53:6789/0 conn(0x7fd45006ec40 legacy=0x7fd4500c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.430+0000 7fd44dffb640 1 -- 172.21.15.53:0/2089233837 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4176688117 0 0) 0x7fd45006f1c0 con 0x7fd45006ec40 2024-05-10T10:41:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.430+0000 7fd44f7fe640 1 -- 172.21.15.53:0/2089233837 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45006fa60 con 0x7fd4500c7390 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.430+0000 7fd44f7fe640 1 --2- 172.21.15.53:0/2089233837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd4500cee40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd4480097a0 tx=0x7fd44802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1194aba029080cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd44dffb640 1 -- 172.21.15.53:0/2089233837 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448037070 con 0x7fd4500c7390 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd44dffb640 1 -- 172.21.15.53:0/2089233837 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd448032440 con 0x7fd4500c7390 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd44dffb640 1 -- 172.21.15.53:0/2089233837 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448032810 con 0x7fd4500c7390 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 -- 172.21.15.53:0/2089233837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 msgr2=0x7fd4500cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 --2- 172.21.15.53:0/2089233837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd4500cee40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd4480097a0 tx=0x7fd44802a180 comp rx=0 tx=0).stop 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 -- 172.21.15.53:0/2089233837 shutdown_connections 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 --2- 172.21.15.53:0/2089233837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd4500cee40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 --2- 172.21.15.53:0/2089233837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4501598c0 0x7fd45006e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 -- 172.21.15.53:0/2089233837 >> 172.21.15.53:0/2089233837 conn(0x7fd4500b2900 msgr2=0x7fd4500b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:26.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 -- 172.21.15.53:0/2089233837 shutdown_connections 2024-05-10T10:41:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.431+0000 7fd4554da640 1 -- 172.21.15.53:0/2089233837 wait complete. 2024-05-10T10:41:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 Processor -- start 2024-05-10T10:41:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 -- start start 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd45006ec40 0x7fd4500b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd45014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4501598c0 0x7fd450155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd450159ca0 con 0x7fd4500c7390 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd45015c6e0 con 0x7fd4501598c0 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd4554da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd450159e20 con 0x7fd45006ec40 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd45014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd44e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd45014f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41012/0 (socket says 172.21.15.53:41012) 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.433+0000 7fd44e7fc640 1 -- 172.21.15.53:0/2128365272 learned_addr learned my addr 172.21.15.53:0/2128365272 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd44effd640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd45006ec40 0x7fd4500b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd44e7fc640 1 -- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd45006ec40 msgr2=0x7fd4500b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd44e7fc640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd45006ec40 0x7fd4500b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd44e7fc640 1 -- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4501598c0 msgr2=0x7fd450155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd44e7fc640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4501598c0 0x7fd450155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd44e7fc640 1 -- 172.21.15.53:0/2128365272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4500b0e40 con 0x7fd4500c7390 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd44e7fc640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd45014f7c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c4d0 tx=0x7fd44400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd4377fe640 1 -- 172.21.15.53:0/2128365272 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444018070 con 0x7fd4500c7390 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd4377fe640 1 -- 172.21.15.53:0/2128365272 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd444007500 con 0x7fd4500c7390 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4500b1120 con 0x7fd4500c7390 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd4377fe640 1 -- 172.21.15.53:0/2128365272 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444004a20 con 0x7fd4500c7390 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.434+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4500b16e0 con 0x7fd4500c7390 2024-05-10T10:41:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.435+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45006a220 con 0x7fd4500c7390 2024-05-10T10:41:26.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.436+0000 7fd4377fe640 1 -- 172.21.15.53:0/2128365272 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd44401b070 con 0x7fd4500c7390 2024-05-10T10:41:26.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.436+0000 7fd4377fe640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd43c0762d0 0x7fd43c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.436+0000 7fd4377fe640 1 -- 172.21.15.53:0/2128365272 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7fd444096b20 con 0x7fd4500c7390 2024-05-10T10:41:26.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.437+0000 7fd44effd640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd43c0762d0 0x7fd43c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.437+0000 7fd44effd640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd43c0762d0 0x7fd43c078790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd440009e90 tx=0x7fd44000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:26.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.441+0000 7fd4377fe640 1 -- 172.21.15.53:0/2128365272 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd4440602b0 con 0x7fd4500c7390 2024-05-10T10:41:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.643+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd45006e4e0 con 0x7fd4500c7390 2024-05-10T10:41:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.644+0000 7fd4377fe640 1 -- 172.21.15.53:0/2128365272 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7fd44405fbb0 con 0x7fd4500c7390 2024-05-10T10:41:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:41:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd43c0762d0 msgr2=0x7fd43c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd43c0762d0 0x7fd43c078790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd440009e90 tx=0x7fd44000b040 comp rx=0 tx=0).stop 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 msgr2=0x7fd45014f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd45014f7c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c4d0 tx=0x7fd44400c990 comp rx=0 tx=0).stop 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 shutdown_connections 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd43c0762d0 0x7fd43c078790 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4501598c0 0x7fd450155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4500c7390 0x7fd45014f7c0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 --2- 172.21.15.53:0/2128365272 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd45006ec40 0x7fd4500b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 >> 172.21.15.53:0/2128365272 conn(0x7fd4500b2900 msgr2=0x7fd4500b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 shutdown_connections 2024-05-10T10:41:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.646+0000 7fd4554da640 1 -- 172.21.15.53:0/2128365272 wait complete. 2024-05-10T10:41:26.658 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:41:26.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:26.659 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:41:26.712 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:41:26.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:26.712 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:41:26.742 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.886+0000 7f59baa62640 1 Processor -- start 2024-05-10T10:41:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.886+0000 7f59baa62640 1 -- start start 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.886+0000 7f59baa62640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59b415ce60 0x7f59b40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59baa62640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b40c6a60 con 0x7f59b41611e0 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59baa62640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b40cd090 con 0x7f59b40c65f0 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59baa62640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f59b40cd370 con 0x7f59b415ce60 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59b3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59b415ce60 0x7f59b40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59b8fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f59b41611e0 0x7f59b415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49528/0 (socket says 172.21.15.53:49528) 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59b8fff640 1 -- 172.21.15.53:0/2571298676 learned_addr learned my addr 172.21.15.53:0/2571298676 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59b3fff640 1 -- 172.21.15.53:0/2571298676 >> v1:172.21.15.135:6789/0 conn(0x7f59b40c65f0 legacy=0x7f59b4160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59b3fff640 1 -- 172.21.15.53:0/2571298676 >> v1:172.21.15.53:6789/0 conn(0x7f59b41611e0 legacy=0x7f59b415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59b2ffd640 1 -- 172.21.15.53:0/2571298676 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833650321 0 0) 0x7f59b40c6a60 con 0x7f59b41611e0 2024-05-10T10:41:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.887+0000 7f59b3fff640 1 -- 172.21.15.53:0/2571298676 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b40cd650 con 0x7f59b415ce60 2024-05-10T10:41:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.888+0000 7f59b3fff640 1 --2- 172.21.15.53:0/2571298676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b415ce60 0x7f59b40cb300 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f59a40097a0 tx=0x7f59a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd88f7b7e05e9963 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.888+0000 7f59b2ffd640 1 -- 172.21.15.53:0/2571298676 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4037070 con 0x7f59b415ce60 2024-05-10T10:41:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.888+0000 7f59b2ffd640 1 -- 172.21.15.53:0/2571298676 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a402ae50 con 0x7f59b415ce60 2024-05-10T10:41:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.888+0000 7f59b2ffd640 1 -- 172.21.15.53:0/2571298676 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4032660 con 0x7f59b415ce60 2024-05-10T10:41:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.889+0000 7f59baa62640 1 -- 172.21.15.53:0/2571298676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b415ce60 msgr2=0x7f59b40cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.889+0000 7f59baa62640 1 --2- 172.21.15.53:0/2571298676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b415ce60 0x7f59b40cb300 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f59a40097a0 tx=0x7f59a402a140 comp rx=0 tx=0).stop 2024-05-10T10:41:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.889+0000 7f59baa62640 1 -- 172.21.15.53:0/2571298676 shutdown_connections 2024-05-10T10:41:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.889+0000 7f59baa62640 1 --2- 172.21.15.53:0/2571298676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b415ce60 0x7f59b40cb300 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.889+0000 7f59baa62640 1 -- 172.21.15.53:0/2571298676 >> 172.21.15.53:0/2571298676 conn(0x7f59b40b2900 msgr2=0x7f59b40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.889+0000 7f59baa62640 1 -- 172.21.15.53:0/2571298676 shutdown_connections 2024-05-10T10:41:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.889+0000 7f59baa62640 1 -- 172.21.15.53:0/2571298676 wait complete. 2024-05-10T10:41:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.890+0000 7f59baa62640 1 Processor -- start 2024-05-10T10:41:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.890+0000 7f59baa62640 1 -- start start 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59baa62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b40c65f0 0x7f59b4152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59baa62640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59b415ce60 0x7f59b40b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59baa62640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b41611e0 0x7f59b40b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59baa62640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40cde50 con 0x7f59b41611e0 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59baa62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40ccd60 con 0x7f59b40c65f0 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59baa62640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f59b40cdfd0 con 0x7f59b415ce60 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b41611e0 0x7f59b40b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b41611e0 0x7f59b40b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41030/0 (socket says 172.21.15.53:41030) 2024-05-10T10:41:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 -- 172.21.15.53:0/535593369 learned_addr learned my addr 172.21.15.53:0/535593369 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b3fff640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b40c65f0 0x7f59b4152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b37fe640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59b415ce60 0x7f59b40b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 -- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59b415ce60 msgr2=0x7f59b40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59b415ce60 0x7f59b40b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 -- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b40c65f0 msgr2=0x7f59b4152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b40c65f0 0x7f59b4152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b8fff640 1 -- 172.21.15.53:0/535593369 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b40b0450 con 0x7f59b41611e0 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b3fff640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b40c65f0 0x7f59b4152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.891+0000 7f59b37fe640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59b415ce60 0x7f59b40b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.892+0000 7f59b8fff640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b41611e0 0x7f59b40b0bd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c780 tx=0x7f59ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.892+0000 7f59b0ff9640 1 -- 172.21.15.53:0/535593369 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac018070 con 0x7f59b41611e0 2024-05-10T10:41:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.892+0000 7f59b0ff9640 1 -- 172.21.15.53:0/535593369 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac00a640 con 0x7f59b41611e0 2024-05-10T10:41:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.892+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b409e720 con 0x7f59b41611e0 2024-05-10T10:41:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.892+0000 7f59b0ff9640 1 -- 172.21.15.53:0/535593369 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00aa70 con 0x7f59b41611e0 2024-05-10T10:41:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.892+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b40af970 con 0x7f59b41611e0 2024-05-10T10:41:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.893+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b406a220 con 0x7f59b41611e0 2024-05-10T10:41:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.894+0000 7f59b0ff9640 1 -- 172.21.15.53:0/535593369 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f59ac01b070 con 0x7f59b41611e0 2024-05-10T10:41:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.894+0000 7f59b0ff9640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59a00762d0 0x7f59a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.894+0000 7f59b0ff9640 1 -- 172.21.15.53:0/535593369 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f59ac096cf0 con 0x7f59b41611e0 2024-05-10T10:41:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.894+0000 7f59b3fff640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59a00762d0 0x7f59a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.895+0000 7f59b3fff640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59a00762d0 0x7f59a0078790 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f59a4034000 tx=0x7f59a402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:26.899+0000 7f59b0ff9640 1 -- 172.21.15.53:0/535593369 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f59ac09b070 con 0x7f59b41611e0 2024-05-10T10:41:27.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.127+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59b40d1990 con 0x7f59b41611e0 2024-05-10T10:41:27.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.127+0000 7f59b0ff9640 1 -- 172.21.15.53:0/535593369 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5189 (secure 0 0 0) 0x7f59ac060520 con 0x7f59b41611e0 2024-05-10T10:41:27.132 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:27.132 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:41:18.164463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/4122299129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":4122299129},{"type":"v1","addr":"172.21.15.53:6835","nonce":4122299129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7479,"qdb_cluster":[7479]},"id":7}]} 2024-05-10T10:41:27.133 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 82 2024-05-10T10:41:27.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59a00762d0 msgr2=0x7f59a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59a00762d0 0x7f59a0078790 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f59a4034000 tx=0x7f59a402e000 comp rx=0 tx=0).stop 2024-05-10T10:41:27.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b41611e0 msgr2=0x7f59b40b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b41611e0 0x7f59b40b0bd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c780 tx=0x7f59ac00cc40 comp rx=0 tx=0).stop 2024-05-10T10:41:27.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 shutdown_connections 2024-05-10T10:41:27.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f59a00762d0 0x7f59a0078790 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59b41611e0 0x7f59b40b0bd0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59b415ce60 0x7f59b40b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 --2- 172.21.15.53:0/535593369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b40c65f0 0x7f59b4152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 >> 172.21.15.53:0/535593369 conn(0x7f59b40b2900 msgr2=0x7f59b415d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.129+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 shutdown_connections 2024-05-10T10:41:27.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.130+0000 7f59baa62640 1 -- 172.21.15.53:0/535593369 wait complete. 2024-05-10T10:41:27.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-10T10:41:27.270 INFO:teuthology.orchestra.run.smithi053.stdout:{"message":"","return_code":0} 2024-05-10T10:41:27.271 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T10:41:27.271 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:41:27.271 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:41:27.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:41:27.289 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:41:27.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:27.289 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:41:27.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:41:27.344 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-10T10:41:27.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-10T10:41:27.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:27.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-10T10:41:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.492+0000 7f24486e8640 1 Processor -- start 2024-05-10T10:41:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.493+0000 7f24486e8640 1 -- start start 2024-05-10T10:41:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f24486e8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24400d3920 0x7f24400ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f24486e8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f24400cff10 0x7f24400ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f24486e8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24400d0d60 con 0x7f244015d2d0 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f24486e8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f24400d1040 con 0x7f24400cff10 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f24486e8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f24400cdd60 con 0x7f24400d3920 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2446c85640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f24400cff10 0x7f24400ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2446484640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f244015d2d0 0x7f24400b2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49530/0 (socket says 172.21.15.53:49530) 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2446484640 1 -- 172.21.15.53:0/1159000501 learned_addr learned my addr 172.21.15.53:0/1159000501 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2445c83640 1 --2- 172.21.15.53:0/1159000501 >> v2:172.21.15.135:3300/0 conn(0x7f24400d3920 0x7f24400ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2446c85640 1 -- 172.21.15.53:0/1159000501 >> v2:172.21.15.135:3300/0 conn(0x7f24400d3920 msgr2=0x7f24400ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2446c85640 1 --2- 172.21.15.53:0/1159000501 >> v2:172.21.15.135:3300/0 conn(0x7f24400d3920 0x7f24400ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2446c85640 1 -- 172.21.15.53:0/1159000501 >> v1:172.21.15.53:6789/0 conn(0x7f244015d2d0 legacy=0x7f24400b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.494+0000 7f2446c85640 1 -- 172.21.15.53:0/1159000501 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24400ce040 con 0x7f24400cff10 2024-05-10T10:41:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.495+0000 7f2446c85640 1 --2- 172.21.15.53:0/1159000501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400cff10 0x7f24400ceaa0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f24400d2d60 tx=0x7f243802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43733faef8e8415f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.495+0000 7f2445482640 1 -- 172.21.15.53:0/1159000501 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438007740 con 0x7f24400cff10 2024-05-10T10:41:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.495+0000 7f2445482640 1 -- 172.21.15.53:0/1159000501 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24380078e0 con 0x7f24400cff10 2024-05-10T10:41:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.495+0000 7f2445482640 1 -- 172.21.15.53:0/1159000501 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438007cb0 con 0x7f24400cff10 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 -- 172.21.15.53:0/1159000501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400cff10 msgr2=0x7f24400ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 --2- 172.21.15.53:0/1159000501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400cff10 0x7f24400ceaa0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f24400d2d60 tx=0x7f243802a680 comp rx=0 tx=0).stop 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 -- 172.21.15.53:0/1159000501 shutdown_connections 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 --2- 172.21.15.53:0/1159000501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400cff10 0x7f24400ceaa0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 --2- 172.21.15.53:0/1159000501 >> v2:172.21.15.135:3300/0 conn(0x7f24400d3920 0x7f24400ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 -- 172.21.15.53:0/1159000501 >> 172.21.15.53:0/1159000501 conn(0x7f24400d2e00 msgr2=0x7f24400d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 -- 172.21.15.53:0/1159000501 shutdown_connections 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.496+0000 7f24486e8640 1 -- 172.21.15.53:0/1159000501 wait complete. 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.497+0000 7f24486e8640 1 Processor -- start 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.497+0000 7f24486e8640 1 -- start start 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f24486e8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24400cff10 0x7f24400c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f24486e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400d3920 0x7f24400cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f24486e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244015d2d0 0x7f2440155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f24486e8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24400cda10 con 0x7f24400d3920 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f24486e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f244015d6b0 con 0x7f244015d2d0 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f24486e8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f24400b2e70 con 0x7f24400cff10 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400d3920 0x7f24400cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400d3920 0x7f24400cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41046/0 (socket says 172.21.15.53:41046) 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 -- 172.21.15.53:0/963528663 learned_addr learned my addr 172.21.15.53:0/963528663 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2446c85640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244015d2d0 0x7f2440155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2446484640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24400cff10 0x7f24400c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 -- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24400cff10 msgr2=0x7f24400c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24400cff10 0x7f24400c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 -- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244015d2d0 msgr2=0x7f2440155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244015d2d0 0x7f2440155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 -- 172.21.15.53:0/963528663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24400cb8c0 con 0x7f24400d3920 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.498+0000 7f2445c83640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400d3920 0x7f24400cb000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f243400d800 tx=0x7f243400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.499+0000 7f2432ffd640 1 -- 172.21.15.53:0/963528663 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434004020 con 0x7f24400d3920 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.499+0000 7f2432ffd640 1 -- 172.21.15.53:0/963528663 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24340041c0 con 0x7f24400d3920 2024-05-10T10:41:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.499+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24400cbba0 con 0x7f24400d3920 2024-05-10T10:41:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.499+0000 7f2432ffd640 1 -- 172.21.15.53:0/963528663 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434004390 con 0x7f24400d3920 2024-05-10T10:41:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.499+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24400cc160 con 0x7f24400d3920 2024-05-10T10:41:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.501+0000 7f2432ffd640 1 -- 172.21.15.53:0/963528663 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f243401b020 con 0x7f24400d3920 2024-05-10T10:41:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.501+0000 7f2432ffd640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24240762d0 0x7f2424078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.502+0000 7f2432ffd640 1 -- 172.21.15.53:0/963528663 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f2434097400 con 0x7f24400d3920 2024-05-10T10:41:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.502+0000 7f2446484640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24240762d0 0x7f2424078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.502+0000 7f244b40a740 1 Processor -- start 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.502+0000 7f244b40a740 1 -- start start 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.502+0000 7f2446484640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24240762d0 0x7f2424078790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f24400b2460 tx=0x7f243c007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f244b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dbcbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dbfb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfbc00 0x558c3dbd0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f244b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558c3db34cf0 con 0x558c3dc19220 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c3daede10 con 0x558c3dbfc340 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558c3db30cb0 con 0x558c3dbfbc00 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dbcbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dbcbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41052/0 (socket says 172.21.15.53:41052) 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 -- 172.21.15.53:0/1928061483 learned_addr learned my addr 172.21.15.53:0/1928061483 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2445c83640 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dbfb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446c85640 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfbc00 0x558c3dbd0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 -- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfbc00 msgr2=0x558c3dbd0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfbc00 0x558c3dbd0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 -- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 msgr2=0x558c3dbfb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dbfb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 -- 172.21.15.53:0/1928061483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3d90e7e0 con 0x558c3dc19220 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2446484640 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dbcbbb0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f243c0017b0 tx=0x7f243c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852dc31e6a1514d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.503+0000 7f2423fff640 1 -- 172.21.15.53:0/1928061483 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c026070 con 0x558c3dc19220 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f2423fff640 1 -- 172.21.15.53:0/1928061483 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c013970 con 0x558c3dc19220 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f2423fff640 1 -- 172.21.15.53:0/1928061483 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c013d40 con 0x558c3dc19220 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 -- 172.21.15.53:0/1928061483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 msgr2=0x558c3dbcbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dbcbbb0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f243c0017b0 tx=0x7f243c013000 comp rx=0 tx=0).stop 2024-05-10T10:41:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 -- 172.21.15.53:0/1928061483 shutdown_connections 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfbc00 0x558c3dbd0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dbfb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 --2- 172.21.15.53:0/1928061483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dbcbbb0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 -- 172.21.15.53:0/1928061483 >> 172.21.15.53:0/1928061483 conn(0x558c3dbcde20 msgr2=0x558c3dc166c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 -- 172.21.15.53:0/1928061483 shutdown_connections 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.504+0000 7f244b40a740 1 -- 172.21.15.53:0/1928061483 wait complete. 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.505+0000 7f244b40a740 1 -- start start 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.505+0000 7f244b40a740 1 Processor -- start 2024-05-10T10:41:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f244b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dbfc340 0x558c3dbf6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dbfc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc0f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f244b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558c3dbcffd0 con 0x558c3dbfc340 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c3daf1940 con 0x558c3dc19220 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558c3dbcf3c0 con 0x558c3dc26be0 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f244b40a740 10 client.0 _ll_register_callbacks cb 0x558c3daf4440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dbfc340 0x558c3dbf6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dbfc340 0x558c3dbf6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41056/0 (socket says 172.21.15.53:41056) 2024-05-10T10:41:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 -- 172.21.15.53:0/3850836745 learned_addr learned my addr 172.21.15.53:0/3850836745 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2445c83640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dbfc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446c85640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc0f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 -- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 msgr2=0x558c3dc0f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc0f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 -- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 msgr2=0x558c3dbfc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dbfc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446484640 1 -- 172.21.15.53:0/3850836745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3d96ef70 con 0x558c3dbfc340 2024-05-10T10:41:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2446c85640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc0f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.506+0000 7f2445c83640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dbfc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f2446484640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dbfc340 0x558c3dbf6cf0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f243c0137c0 tx=0x7f243c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddd1d9dc814d725 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f24217fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f24217fa640 1 -- 172.21.15.53:0/3850836745 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c030060 con 0x558c3dbfc340 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f24217fa640 1 -- 172.21.15.53:0/3850836745 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f243c021aa0 con 0x558c3dbfc340 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243c013970 con 0x558c3dbfc340 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f24217fa640 1 -- 172.21.15.53:0/3850836745 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f243c036050 con 0x558c3dbfc340 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 10 client.7683 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x558c3d806170 con 0x558c3dbfc340 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f24217fa640 1 -- 172.21.15.53:0/3850836745 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c013970 con 0x558c3dbfc340 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f24217fa640 1 -- 172.21.15.53:0/3850836745 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f243c039020 con 0x558c3dbfc340 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 10 client.7683 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 10 client.7683 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 20 client.7683 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc26fc0 0x558c3dbff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 4 client.7683 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c3db42900 con 0x558c3dc26fc0 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.507+0000 7f2445c83640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc26fc0 0x558c3dbff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f2445c83640 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc26fc0 0x558c3dbff080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec5581c01248c8e server_cookie=1ca7b27298cacd4a in_seq=0 out_seq=0 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f24217fa640 10 client.7683 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f24217fa640 1 -- 172.21.15.53:0/3850836745 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c3db42900 con 0x558c3dc26fc0 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f24217fa640 10 client.7683 handle_command_reply: tid=0 2024-05-10T10:41:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 client.7683 shutdown 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dbfc340 msgr2=0x558c3dbf6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dbfc340 0x558c3dbf6cf0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f243c0137c0 tx=0x7f243c000bc0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 shutdown_connections 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc26fc0 0x558c3dbff080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc0f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dbfc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f24217fa640 0 client.7683 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 --2- 172.21.15.53:0/3850836745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dbfc340 0x558c3dbf6cf0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.509+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 >> 172.21.15.53:0/3850836745 conn(0x558c3dbfbc00 msgr2=0x558c3dbcf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.510+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 shutdown_connections 2024-05-10T10:41:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.510+0000 7f244b40a740 1 -- 172.21.15.53:0/3850836745 wait complete. 2024-05-10T10:41:27.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.510+0000 7f244b40a740 20 client.7683 trim_cache size 0 max 16384 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.513+0000 7f77f2704640 1 Processor -- start 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.514+0000 7f77f2704640 1 -- start start 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.514+0000 7f77f2704640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f77ec162810 0x7f77ec0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.514+0000 7f77f2704640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0b3450 0x7f77ec0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.514+0000 7f77f2704640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77ec0cc300 con 0x7f77ec0d5ea0 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.514+0000 7f77f2704640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f77ec0b64d0 con 0x7f77ec162810 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.514+0000 7f77f2704640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec0b5c30 con 0x7f77ec0b3450 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.514+0000 7f77ebfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f77ec162810 0x7f77ec0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77eb7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f77ec0d5ea0 0x7f77ec0b23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49544/0 (socket says 172.21.15.53:49544) 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77ebfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f77ec162810 0x7f77ec0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41060/0 (socket says 172.21.15.53:41060) 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77eb7fe640 1 -- 172.21.15.53:0/167500435 learned_addr learned my addr 172.21.15.53:0/167500435 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77ebfff640 1 -- 172.21.15.53:0/167500435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0b3450 msgr2=0x7f77ec0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77f0ca1640 1 --2- 172.21.15.53:0/167500435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0b3450 0x7f77ec0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77ebfff640 1 --2- 172.21.15.53:0/167500435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0b3450 0x7f77ec0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77eaffd640 1 -- 172.21.15.53:0/167500435 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967175752 0 0) 0x7f77e0001110 con 0x7f77ec0d5ea0 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77ebfff640 1 -- 172.21.15.53:0/167500435 >> v1:172.21.15.53:6789/0 conn(0x7f77ec0d5ea0 legacy=0x7f77ec0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77ebfff640 1 -- 172.21.15.53:0/167500435 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec0b5f10 con 0x7f77ec162810 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.515+0000 7f77f0ca1640 1 --2- 172.21.15.53:0/167500435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0b3450 0x7f77ec0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77ebfff640 1 --2- 172.21.15.53:0/167500435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 0x7f77ec0b1e80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009790 tx=0x7f77dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665c875a700420e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77eaffd640 1 -- 172.21.15.53:0/167500435 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc037070 con 0x7f77ec162810 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77eaffd640 1 -- 172.21.15.53:0/167500435 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77dc032440 con 0x7f77ec162810 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77eaffd640 1 -- 172.21.15.53:0/167500435 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc032810 con 0x7f77ec162810 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 -- 172.21.15.53:0/167500435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 msgr2=0x7f77ec0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 --2- 172.21.15.53:0/167500435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 0x7f77ec0b1e80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009790 tx=0x7f77dc02a180 comp rx=0 tx=0).stop 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 -- 172.21.15.53:0/167500435 shutdown_connections 2024-05-10T10:41:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 --2- 172.21.15.53:0/167500435 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0b3450 0x7f77ec0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 --2- 172.21.15.53:0/167500435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 0x7f77ec0b1e80 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 -- 172.21.15.53:0/167500435 >> 172.21.15.53:0/167500435 conn(0x7f77ec09e590 msgr2=0x7f77ec0d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 -- 172.21.15.53:0/167500435 shutdown_connections 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.516+0000 7f77f2704640 1 -- 172.21.15.53:0/167500435 wait complete. 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.518+0000 7f77f2704640 1 Processor -- start 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.518+0000 7f77f2704640 1 -- start start 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.518+0000 7f77f2704640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77ec0b3450 0x7f77ec150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f2704640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0d5ea0 0x7f77ec0d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f2704640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 0x7f77ec0d2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f2704640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec0b2f30 con 0x7f77ec162810 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f2704640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec0b61f0 con 0x7f77ec0b3450 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f2704640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f77ec0b58e0 con 0x7f77ec0d5ea0 2024-05-10T10:41:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 0x7f77ec0d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 0x7f77ec0d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41066/0 (socket says 172.21.15.53:41066) 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 -- 172.21.15.53:0/811655640 learned_addr learned my addr 172.21.15.53:0/811655640 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77ebfff640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77ec0b3450 0x7f77ec150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77eb7fe640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0d5ea0 0x7f77ec0d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 -- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0d5ea0 msgr2=0x7f77ec0d1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0d5ea0 0x7f77ec0d1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 -- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77ec0b3450 msgr2=0x7f77ec150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77ec0b3450 0x7f77ec150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 -- 172.21.15.53:0/811655640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec0d0a80 con 0x7f77ec162810 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77eb7fe640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f77ec0d5ea0 0x7f77ec0d1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f0ca1640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77ec162810 0x7f77ec0d2290 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f77e400cd50 tx=0x7f77e400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e400f680 con 0x7f77ec162810 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e400f840 con 0x7f77ec162810 2024-05-10T10:41:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f2704640 1 -- 172.21.15.53:0/811655640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec0d0d60 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e400fc30 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.519+0000 7f77f2704640 1 -- 172.21.15.53:0/811655640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec0d1320 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.520+0000 7f77f2704640 1 -- 172.21.15.53:0/811655640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f77ec06a220 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.522+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f77e40213f0 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.522+0000 7f77e8ff9640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f77d80762d0 0x7f77d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.522+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f77e40988b0 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.522+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f77e4026070 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.522+0000 7f77ebfff640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f77d80762d0 0x7f77d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.523+0000 7f77f2704640 1 -- 172.21.15.53:0/811655640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f77ec06cbb0 con 0x7f77ec162810 2024-05-10T10:41:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.523+0000 7f77ebfff640 1 --2- 172.21.15.53:0/811655640 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f77d80762d0 0x7f77d8078790 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f77dc034000 tx=0x7f77dc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.530+0000 7f244b40a740 1 Processor -- start 2024-05-10T10:41:27.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.530+0000 7f244b40a740 1 -- start start 2024-05-10T10:41:27.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dc23d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f244b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dc24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.542 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc2a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f244b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2440201ef0 con 0x558c3dc19220 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2440200f40 con 0x558c3dbfc340 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f24400c74c0 con 0x558c3dc26be0 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dc24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dc24260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41078/0 (socket says 172.21.15.53:41078) 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 -- 172.21.15.53:0/3014587563 learned_addr learned my addr 172.21.15.53:0/3014587563 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 -- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 msgr2=0x558c3dc2a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2446c85640 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc2a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc2a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 -- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 msgr2=0x558c3dc23d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2446484640 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dc23d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dc23d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.531+0000 7f2445c83640 1 -- 172.21.15.53:0/3014587563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3dc10d90 con 0x558c3dc19220 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.532+0000 7f2445c83640 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dc24260 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2434060570 tx=0x7f2434097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32be1a39f299eeb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.532+0000 7f24237fe640 1 -- 172.21.15.53:0/3014587563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434093070 con 0x558c3dc19220 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.532+0000 7f24237fe640 1 -- 172.21.15.53:0/3014587563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2434069300 con 0x558c3dc19220 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.532+0000 7f24237fe640 1 -- 172.21.15.53:0/3014587563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24340970b0 con 0x558c3dc19220 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.532+0000 7f244b40a740 1 -- 172.21.15.53:0/3014587563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 msgr2=0x558c3dc24260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.532+0000 7f244b40a740 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dc24260 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2434060570 tx=0x7f2434097040 comp rx=0 tx=0).stop 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.532+0000 7f244b40a740 1 -- 172.21.15.53:0/3014587563 shutdown_connections 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dc26be0 0x558c3dc2a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc19220 0x558c3dc24260 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 --2- 172.21.15.53:0/3014587563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dbfc340 0x558c3dc23d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 -- 172.21.15.53:0/3014587563 >> 172.21.15.53:0/3014587563 conn(0x558c3dbfbc00 msgr2=0x558c3dc15a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 -- 172.21.15.53:0/3014587563 shutdown_connections 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 -- 172.21.15.53:0/3014587563 wait complete. 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 -- start start 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.533+0000 7f244b40a740 1 Processor -- start 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfc340 0x558c3dbf8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f244b40a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dc2e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f244b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc26be0 0x558c3dc2e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f244b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2440200920 con 0x558c3dc26be0 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c3dbf7490 con 0x558c3dc19220 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f244b40a740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558c3dc2f1d0 con 0x558c3dbfc340 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f244b40a740 10 client.0 _ll_register_callbacks cb 0x7f243c024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446c85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc26be0 0x558c3dc2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446c85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc26be0 0x558c3dc2e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41080/0 (socket says 172.21.15.53:41080) 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446c85640 1 -- 172.21.15.53:0/4048189381 learned_addr learned my addr 172.21.15.53:0/4048189381 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2445c83640 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dc2e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446484640 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfc340 0x558c3dbf8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446c85640 1 -- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfc340 msgr2=0x558c3dbf8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446c85640 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfc340 0x558c3dbf8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446c85640 1 -- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 msgr2=0x558c3dc2e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.534+0000 7f2446c85640 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dc2e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f2446c85640 1 -- 172.21.15.53:0/4048189381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3dbcfde0 con 0x558c3dc26be0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f2446c85640 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc26be0 0x558c3dc2e910 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e80 tx=0x7f243802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1d282189f771cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f24217fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f24217fa640 1 -- 172.21.15.53:0/4048189381 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243804d060 con 0x558c3dc26be0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2434069300 con 0x558c3dc26be0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f24217fa640 1 -- 172.21.15.53:0/4048189381 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f2438048660 con 0x558c3dc26be0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f24217fa640 1 -- 172.21.15.53:0/4048189381 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2438048860 con 0x558c3dc26be0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 10 client.7698 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x558c3dbcc020 con 0x558c3dc26be0 2024-05-10T10:41:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f24217fa640 1 -- 172.21.15.53:0/4048189381 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2434069300 con 0x558c3dc26be0 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f24217fa640 1 -- 172.21.15.53:0/4048189381 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f2438048a00 con 0x558c3dc26be0 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 10 client.7698 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 10 client.7698 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 20 client.7698 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc94240 0x558c3dc2c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 4 client.7698 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.535+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x558c3db3c770 con 0x558c3dc94240 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.536+0000 7f2446484640 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc94240 0x558c3dc2c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.536+0000 7f24217fa640 10 client.7698 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.536+0000 7f2446484640 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc94240 0x558c3dc2c1a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6117d65d43cdd8e server_cookie=ac43422c3650629b in_seq=0 out_seq=0 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.536+0000 7f24217fa640 1 -- 172.21.15.53:0/4048189381 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558c3db3c770 con 0x558c3dc94240 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.536+0000 7f24217fa640 10 client.7698 handle_command_reply: tid=0 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.536+0000 7f244b40a740 1 client.7698 shutdown 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc26be0 msgr2=0x558c3dc2e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc26be0 0x558c3dc2e910 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e80 tx=0x7f243802c000 comp rx=0 tx=0).stop 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 shutdown_connections 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x558c3dc94240 0x558c3dc2c1a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f24217fa640 0 client.7698 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c3dc26be0 0x558c3dc2e910 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3dc19220 0x558c3dc2e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 --2- 172.21.15.53:0/4048189381 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558c3dbfc340 0x558c3dbf8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 >> 172.21.15.53:0/4048189381 conn(0x558c3dbfbc00 msgr2=0x558c3dc18570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 shutdown_connections 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 1 -- 172.21.15.53:0/4048189381 wait complete. 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.537+0000 7f244b40a740 20 client.7698 trim_cache size 0 max 16384 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "01f54a22-2cb5-43a0-86ab-2c395f546db8", 2024-05-10T10:41:27.550 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:41:27.551 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:41:27.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24240762d0 msgr2=0x7f2424078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.552 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24240762d0 0x7f2424078790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f24400b2460 tx=0x7f243c007e80 comp rx=0 tx=0).stop 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400d3920 msgr2=0x7f24400cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400d3920 0x7f24400cb000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f243400d800 tx=0x7f243400dcc0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 shutdown_connections 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24240762d0 0x7f2424078790 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f244015d2d0 0x7f2440155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24400d3920 0x7f24400cb000 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 --2- 172.21.15.53:0/963528663 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24400cff10 0x7f24400c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 >> 172.21.15.53:0/963528663 conn(0x7f24400d2e00 msgr2=0x7f24400b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 shutdown_connections 2024-05-10T10:41:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.538+0000 7f24486e8640 1 -- 172.21.15.53:0/963528663 wait complete. 2024-05-10T10:41:27.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:27.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-10T10:41:27.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.659+0000 7f6698fe4640 1 Processor -- start 2024-05-10T10:41:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.659+0000 7f6698fe4640 1 -- start start 2024-05-10T10:41:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6698fe4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162900 0x7f66940d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6698fe4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6694162ce0 0x7f66940b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6698fe4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66940d4190 con 0x7f66940d3950 2024-05-10T10:41:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6698fe4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f66940cfc40 con 0x7f6694162ce0 2024-05-10T10:41:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6698fe4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f66940d3eb0 con 0x7f6694162900 2024-05-10T10:41:27.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6692d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f66940d3950 0x7f66940d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49552/0 (socket says 172.21.15.53:49552) 2024-05-10T10:41:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6692d9d640 1 -- 172.21.15.53:0/559932260 learned_addr learned my addr 172.21.15.53:0/559932260 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f668bfff640 1 --2- 172.21.15.53:0/559932260 >> v2:172.21.15.135:3300/0 conn(0x7f6694162ce0 0x7f66940b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f669259c640 1 --2- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162900 0x7f66940d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6691d9b640 1 -- 172.21.15.53:0/559932260 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494440524 0 0) 0x7f668c001490 con 0x7f66940d3950 2024-05-10T10:41:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f6691d9b640 1 -- 172.21.15.53:0/559932260 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6680003520 con 0x7f66940d3950 2024-05-10T10:41:27.670 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f668bfff640 1 -- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162900 msgr2=0x7f66940d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f668bfff640 1 --2- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162900 0x7f66940d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f668bfff640 1 -- 172.21.15.53:0/559932260 >> v1:172.21.15.53:6789/0 conn(0x7f66940d3950 legacy=0x7f66940d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f668bfff640 1 -- 172.21.15.53:0/559932260 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66940cfee0 con 0x7f6694162ce0 2024-05-10T10:41:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.660+0000 7f669259c640 1 --2- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162900 0x7f66940d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:41:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.661+0000 7f668bfff640 1 --2- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162ce0 0x7f66940b48e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f667c0097a0 tx=0x7f667c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2228b414b3eb31c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.661+0000 7f6691d9b640 1 -- 172.21.15.53:0/559932260 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c037070 con 0x7f6694162ce0 2024-05-10T10:41:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.661+0000 7f6691d9b640 1 -- 172.21.15.53:0/559932260 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667c02ae50 con 0x7f6694162ce0 2024-05-10T10:41:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.661+0000 7f6691d9b640 1 -- 172.21.15.53:0/559932260 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c032640 con 0x7f6694162ce0 2024-05-10T10:41:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 -- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162ce0 msgr2=0x7f66940b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 --2- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162ce0 0x7f66940b48e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f667c0097a0 tx=0x7f667c02a100 comp rx=0 tx=0).stop 2024-05-10T10:41:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 -- 172.21.15.53:0/559932260 shutdown_connections 2024-05-10T10:41:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 --2- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162ce0 0x7f66940b48e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 --2- 172.21.15.53:0/559932260 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162900 0x7f66940d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 -- 172.21.15.53:0/559932260 >> 172.21.15.53:0/559932260 conn(0x7f669409e430 msgr2=0x7f66940d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 -- 172.21.15.53:0/559932260 shutdown_connections 2024-05-10T10:41:27.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.662+0000 7f6698fe4640 1 -- 172.21.15.53:0/559932260 wait complete. 2024-05-10T10:41:27.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.663+0000 7f6698fe4640 1 Processor -- start 2024-05-10T10:41:27.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.663+0000 7f6698fe4640 1 -- start start 2024-05-10T10:41:27.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f6698fe4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66940d3950 0x7f66940b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f6698fe4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162900 0x7f6694152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f6698fe4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162ce0 0x7f66941534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f6698fe4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66940b4300 con 0x7f66940d3950 2024-05-10T10:41:27.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f6698fe4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66940b4180 con 0x7f6694162900 2024-05-10T10:41:27.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f6698fe4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f66940d10d0 con 0x7f6694162ce0 2024-05-10T10:41:27.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f669259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66940d3950 0x7f66940b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f669259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66940d3950 0x7f66940b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41092/0 (socket says 172.21.15.53:41092) 2024-05-10T10:41:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f669259c640 1 -- 172.21.15.53:0/2081100192 learned_addr learned my addr 172.21.15.53:0/2081100192 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.664+0000 7f668bfff640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162900 0x7f6694152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f669259c640 1 -- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162ce0 msgr2=0x7f66941534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f6692d9d640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162ce0 0x7f66941534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f669259c640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162ce0 0x7f66941534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f669259c640 1 -- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162900 msgr2=0x7f6694152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f669259c640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162900 0x7f6694152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f669259c640 1 -- 172.21.15.53:0/2081100192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66940cabf0 con 0x7f66940d3950 2024-05-10T10:41:27.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f668bfff640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162900 0x7f6694152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:27.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f6692d9d640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162ce0 0x7f66941534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:27.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f669259c640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66940d3950 0x7f66940b2070 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f668400e960 tx=0x7f668400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f668affd640 1 -- 172.21.15.53:0/2081100192 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684017070 con 0x7f66940d3950 2024-05-10T10:41:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f668affd640 1 -- 172.21.15.53:0/2081100192 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668400ca00 con 0x7f66940d3950 2024-05-10T10:41:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f668affd640 1 -- 172.21.15.53:0/2081100192 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668400cdd0 con 0x7f66940d3950 2024-05-10T10:41:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66940caed0 con 0x7f66940d3950 2024-05-10T10:41:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.665+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66940cb490 con 0x7f66940d3950 2024-05-10T10:41:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.667+0000 7f668affd640 1 -- 172.21.15.53:0/2081100192 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f668401a070 con 0x7f66940d3950 2024-05-10T10:41:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.667+0000 7f668affd640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6674076300 0x7f66740787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.667+0000 7f668affd640 1 -- 172.21.15.53:0/2081100192 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f6684097900 con 0x7f66940d3950 2024-05-10T10:41:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.668+0000 7f668bfff640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6674076300 0x7f66740787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.668+0000 7f668bfff640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6674076300 0x7f66740787c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f667c034000 tx=0x7f667c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.668+0000 7f669bdd9740 1 Processor -- start 2024-05-10T10:41:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.668+0000 7f669bdd9740 1 -- start start 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e22af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e53280 0x55bb69e52600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e52b40 0x55bb69e27950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f669bdd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bb69def690 con 0x55bb69e52b40 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb69d8cf30 con 0x55bb69e53280 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55bb69d45b20 con 0x55bb69e70160 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f6692d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e52b40 0x55bb69e27950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f6692d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e52b40 0x55bb69e27950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41106/0 (socket says 172.21.15.53:41106) 2024-05-10T10:41:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f6692d9d640 1 -- 172.21.15.53:0/313652036 learned_addr learned my addr 172.21.15.53:0/313652036 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f669259c640 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e22af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.669+0000 7f668bfff640 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e53280 0x55bb69e52600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6692d9d640 1 -- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 msgr2=0x55bb69e22af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6692d9d640 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e22af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6692d9d640 1 -- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e53280 msgr2=0x55bb69e52600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6692d9d640 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e53280 0x55bb69e52600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6692d9d640 1 -- 172.21.15.53:0/313652036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb69d3de60 con 0x55bb69e52b40 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f668bfff640 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e53280 0x55bb69e52600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6692d9d640 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e52b40 0x55bb69e27950 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f668c0021b0 tx=0x7f668c00d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1269e8c384edf7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6673fff640 1 -- 172.21.15.53:0/313652036 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c018070 con 0x55bb69e52b40 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6673fff640 1 -- 172.21.15.53:0/313652036 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c013670 con 0x55bb69e52b40 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f6673fff640 1 -- 172.21.15.53:0/313652036 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c013a20 con 0x55bb69e52b40 2024-05-10T10:41:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f669bdd9740 1 -- 172.21.15.53:0/313652036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e52b40 msgr2=0x55bb69e27950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.670+0000 7f669bdd9740 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e52b40 0x55bb69e27950 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f668c0021b0 tx=0x7f668c00d5e0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 -- 172.21.15.53:0/313652036 shutdown_connections 2024-05-10T10:41:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e52b40 0x55bb69e27950 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e53280 0x55bb69e52600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 --2- 172.21.15.53:0/313652036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e22af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 -- 172.21.15.53:0/313652036 >> 172.21.15.53:0/313652036 conn(0x55bb69e24d60 msgr2=0x55bb69e6d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 -- 172.21.15.53:0/313652036 shutdown_connections 2024-05-10T10:41:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 -- 172.21.15.53:0/313652036 wait complete. 2024-05-10T10:41:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 -- start start 2024-05-10T10:41:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.671+0000 7f669bdd9740 1 Processor -- start 2024-05-10T10:41:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69e43060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669bdd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bb69d495e0 con 0x55bb69e53280 2024-05-10T10:41:27.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb69d88ef0 con 0x55bb69e6e650 2024-05-10T10:41:27.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55bb69e26f10 con 0x55bb69e70160 2024-05-10T10:41:27.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669bdd9740 10 client.0 _ll_register_callbacks cb 0x55bb69d44320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:27.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69e43060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69e43060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41120/0 (socket says 172.21.15.53:41120) 2024-05-10T10:41:27.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669259c640 1 -- 172.21.15.53:0/301578968 learned_addr learned my addr 172.21.15.53:0/301578968 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f668bfff640 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e80af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f6692d9d640 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669259c640 1 -- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 msgr2=0x55bb69e80af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.672+0000 7f669259c640 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e80af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669259c640 1 -- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 msgr2=0x55bb69e6ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669259c640 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e6ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669259c640 1 -- 172.21.15.53:0/301578968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb69b687d0 con 0x55bb69e53280 2024-05-10T10:41:27.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669259c640 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69e43060 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6684095d00 tx=0x7f66840605c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf9569b616861b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f66717fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f66717fa640 1 -- 172.21.15.53:0/301578968 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684021070 con 0x55bb69e53280 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f66717fa640 1 -- 172.21.15.53:0/301578968 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f668405fc90 con 0x55bb69e53280 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668c013670 con 0x55bb69e53280 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f66717fa640 1 -- 172.21.15.53:0/301578968 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f6684064de0 con 0x55bb69e53280 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 10 client.7710 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55bb69a9e3f0 con 0x55bb69e53280 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f66717fa640 1 -- 172.21.15.53:0/301578968 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c013670 con 0x55bb69e53280 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f66717fa640 1 -- 172.21.15.53:0/301578968 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f668405c3a0 con 0x55bb69e53280 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 10 client.7710 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 10 client.7710 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 20 client.7710 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:27.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e53c40 0x55bb69e54020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 4 client.7710 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb69d9ab40 con 0x55bb69e53c40 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.673+0000 7f668bfff640 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e53c40 0x55bb69e54020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.674+0000 7f66717fa640 10 client.7710 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.674+0000 7f668bfff640 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e53c40 0x55bb69e54020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f322bb748dfa98b5 server_cookie=47f4fd19aa568647 in_seq=0 out_seq=0 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.674+0000 7f66717fa640 1 -- 172.21.15.53:0/301578968 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb69d9ab40 con 0x55bb69e53c40 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.674+0000 7f66717fa640 10 client.7710 handle_command_reply: tid=0 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.674+0000 7f669bdd9740 1 client.7710 shutdown 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.674+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 msgr2=0x55bb69e43060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.674+0000 7f669bdd9740 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69e43060 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6684095d00 tx=0x7f66840605c0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 shutdown_connections 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e53c40 0x55bb69e54020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e6ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e70160 0x55bb69e80af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 --2- 172.21.15.53:0/301578968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69e43060 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f66717fa640 0 client.7710 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 >> 172.21.15.53:0/301578968 conn(0x55bb69e52b40 msgr2=0x55bb69e26710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 shutdown_connections 2024-05-10T10:41:27.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 1 -- 172.21.15.53:0/301578968 wait complete. 2024-05-10T10:41:27.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.675+0000 7f669bdd9740 20 client.7710 trim_cache size 0 max 16384 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 Processor -- start 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 -- start start 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e53280 0x55bb69e70ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e7fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6694201fb0 con 0x55bb69e70160 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6694200fd0 con 0x55bb69e6e650 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f66940c9500 con 0x55bb69e53280 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.693+0000 7f6692d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41122/0 (socket says 172.21.15.53:41122) 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 -- 172.21.15.53:0/1132325821 learned_addr learned my addr 172.21.15.53:0/1132325821 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f668bfff640 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e7fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f669259c640 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e53280 0x55bb69e70ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 -- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e53280 msgr2=0x55bb69e70ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e53280 0x55bb69e70ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 -- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 msgr2=0x55bb69e7fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e7fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 -- 172.21.15.53:0/1132325821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb69a5e0f0 con 0x55bb69e70160 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f668bfff640 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e7fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f6692d9d640 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f668c016040 tx=0x7f668c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a65013fb0d8345b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f66737fe640 1 -- 172.21.15.53:0/1132325821 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c018040 con 0x55bb69e70160 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f66737fe640 1 -- 172.21.15.53:0/1132325821 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c013670 con 0x55bb69e70160 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.694+0000 7f66737fe640 1 -- 172.21.15.53:0/1132325821 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c013a40 con 0x55bb69e70160 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 -- 172.21.15.53:0/1132325821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e70160 msgr2=0x55bb69e4d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f668c016040 tx=0x7f668c00b000 comp rx=0 tx=0).stop 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 -- 172.21.15.53:0/1132325821 shutdown_connections 2024-05-10T10:41:27.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e6e650 0x55bb69e7fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 --2- 172.21.15.53:0/1132325821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e53280 0x55bb69e70ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 -- 172.21.15.53:0/1132325821 >> 172.21.15.53:0/1132325821 conn(0x55bb69e52b40 msgr2=0x55bb69e42470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 -- 172.21.15.53:0/1132325821 shutdown_connections 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 -- 172.21.15.53:0/1132325821 wait complete. 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 -- start start 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.695+0000 7f669bdd9740 1 Processor -- start 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69eeaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e6e650 0x55bb69e7c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669bdd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669bdd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bb69e287d0 con 0x55bb69e53280 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6694200b20 con 0x55bb69e70160 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669bdd9740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55bb69da2850 con 0x55bb69e6e650 2024-05-10T10:41:27.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669bdd9740 10 client.0 _ll_register_callbacks cb 0x7f6684069140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69eeaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69eeaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41124/0 (socket says 172.21.15.53:41124) 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.696+0000 7f669259c640 1 -- 172.21.15.53:0/489596243 learned_addr learned my addr 172.21.15.53:0/489596243 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f6692d9d640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f668bfff640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e6e650 0x55bb69e7c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669259c640 1 -- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e6e650 msgr2=0x55bb69e7c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669259c640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e6e650 0x55bb69e7c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669259c640 1 -- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e70160 msgr2=0x55bb69e4d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669259c640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669259c640 1 -- 172.21.15.53:0/489596243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb69e28490 con 0x55bb69e53280 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f668bfff640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e6e650 0x55bb69e7c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669259c640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69eeaca0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f668405cab0 tx=0x7f6684095bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67211255bc291cc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f66717fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f66717fa640 1 -- 172.21.15.53:0/489596243 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684021040 con 0x55bb69e53280 2024-05-10T10:41:27.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f66717fa640 1 -- 172.21.15.53:0/489596243 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f668405fe90 con 0x55bb69e53280 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668c013670 con 0x55bb69e53280 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f66717fa640 1 -- 172.21.15.53:0/489596243 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f6684060800 con 0x55bb69e53280 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 10 client.7716 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55bb69eeb1e0 con 0x55bb69e53280 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f66717fa640 1 -- 172.21.15.53:0/489596243 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c013670 con 0x55bb69e53280 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f66717fa640 1 -- 172.21.15.53:0/489596243 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f668405f700 con 0x55bb69e53280 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 10 client.7716 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 10 client.7716 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 20 client.7716 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e7c800 0x55bb69e4dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 4 client.7716 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.697+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb69d949b0 con 0x55bb69e7c800 2024-05-10T10:41:27.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f668bfff640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e7c800 0x55bb69e4dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f66717fa640 10 client.7716 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f668bfff640 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e7c800 0x55bb69e4dc70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978a4d679398bd67 server_cookie=4dc0858efe001eb0 in_seq=0 out_seq=0 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f66717fa640 1 -- 172.21.15.53:0/489596243 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55bb69d949b0 con 0x55bb69e7c800 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f66717fa640 10 client.7716 handle_command_reply: tid=0 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 client.7716 shutdown 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 msgr2=0x55bb69eeaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69eeaca0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f668405cab0 tx=0x7f6684095bd0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 shutdown_connections 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x55bb69e7c800 0x55bb69e4dc70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb69e70160 0x55bb69e4d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bb69e6e650 0x55bb69e7c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f66717fa640 0 client.7716 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 --2- 172.21.15.53:0/489596243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb69e53280 0x55bb69eeaca0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 >> 172.21.15.53:0/489596243 conn(0x55bb69e52b40 msgr2=0x55bb69e219a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 shutdown_connections 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.698+0000 7f669bdd9740 1 -- 172.21.15.53:0/489596243 wait complete. 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f669bdd9740 20 client.7716 trim_cache size 0 max 16384 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stdout: "01f54a22-2cb5-43a0-86ab-2c395f546db8": { 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "01f54a22-2cb5-43a0-86ab-2c395f546db8", 2024-05-10T10:41:27.712 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair" 2024-05-10T10:41:27.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:27.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:41:27.713 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:41:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6674076300 msgr2=0x7f66740787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6674076300 0x7f66740787c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f667c034000 tx=0x7f667c02e000 comp rx=0 tx=0).stop 2024-05-10T10:41:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66940d3950 msgr2=0x7f66940b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:27.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66940d3950 0x7f66940b2070 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f668400e960 tx=0x7f668400ee20 comp rx=0 tx=0).stop 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 shutdown_connections 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6674076300 0x7f66740787c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6694162ce0 0x7f66941534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6694162900 0x7f6694152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 --2- 172.21.15.53:0/2081100192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66940d3950 0x7f66940b2070 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 >> 172.21.15.53:0/2081100192 conn(0x7f669409e430 msgr2=0x7f66940d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 shutdown_connections 2024-05-10T10:41:27.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:27.699+0000 7f6698fe4640 1 -- 172.21.15.53:0/2081100192 wait complete. 2024-05-10T10:41:27.715 INFO:tasks.cephfs.filesystem:scrub status for tag:01f54a22-2cb5-43a0-86ab-2c395f546db8 - {'path': '/', 'tag': '01f54a22-2cb5-43a0-86ab-2c395f546db8', 'options': 'recursive,repair'} 2024-05-10T10:41:28.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:28.829+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 7 ==== log(4 entries from seq 1 at 2024-05-10T10:41:27.538061+0000) ==== 695+0+0 (secure 0 0 0) 0x7f77e4096d30 con 0x7f77ec162810 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: cluster: 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: id: 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: health: HEALTH_OK 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: services: 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: mgr: x(active, since 8m), standbys: y 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: mds: 1/1 daemons up, 3 standby 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: data: 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: volumes: 1/1 healthy 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: pools: 3 pools, 97 pgs 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: objects: 26 objects, 6.6 MiB 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: usage: 263 MiB used, 720 GiB / 720 GiB avail 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: pgs: 97 active+clean 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: io: 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: client: 769 KiB/s wr, 0 op/s rd, 0 op/s wr 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:41:27.538061+0000 mds.c [INF] scrub queued for path: / 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:41:27.538064+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:41:27.538068+0000 mds.c [INF] scrub summary: active paths [/] 2024-05-10T10:41:28.832 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:41:27.539276+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-10T10:41:31.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:31.833+0000 7f77e8ff9640 1 -- 172.21.15.53:0/811655640 <== mon.0 v2:172.21.15.53:3300/0 8 ==== log(1 entries from seq 5 at 2024-05-10T10:41:30.454571+0000) ==== 189+0+0 (secure 0 0 0) 0x7f77e4099000 con 0x7f77ec162810 2024-05-10T10:41:31.835 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:41:30.454571+0000 mds.c [INF] scrub summary: idle 2024-05-10T10:41:45.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:41:45.584 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:41:57.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:41:57.717 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-10T10:41:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.835+0000 7fef5ddac640 1 Processor -- start 2024-05-10T10:41:57.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.835+0000 7fef5ddac640 1 -- start start 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef5ddac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef5815f3b0 0x7fef58163930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef5ddac640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef58163e70 con 0x7fef580d0e90 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef5ddac640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef5815f790 con 0x7fef580cfdd0 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef5ddac640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fef580cf160 con 0x7fef5815f3b0 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef577fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef5815f3b0 0x7fef58163930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef57fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fef580d0e90 0x7fef5806ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51064/0 (socket says 172.21.15.53:51064) 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef57fff640 1 -- 172.21.15.53:0/963434012 learned_addr learned my addr 172.21.15.53:0/963434012 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef577fe640 1 -- 172.21.15.53:0/963434012 >> v1:172.21.15.135:6789/0 conn(0x7fef580cfdd0 legacy=0x7fef580d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef577fe640 1 -- 172.21.15.53:0/963434012 >> v1:172.21.15.53:6789/0 conn(0x7fef580d0e90 legacy=0x7fef5806ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.836+0000 7fef577fe640 1 -- 172.21.15.53:0/963434012 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef580ce550 con 0x7fef5815f3b0 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.837+0000 7fef577fe640 1 --2- 172.21.15.53:0/963434012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef5815f3b0 0x7fef58163930 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fef480097a0 tx=0x7fef4802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b066eb7c284347fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.837+0000 7fef567fc640 1 -- 172.21.15.53:0/963434012 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48037070 con 0x7fef5815f3b0 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.837+0000 7fef567fc640 1 -- 172.21.15.53:0/963434012 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4802ae50 con 0x7fef5815f3b0 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.837+0000 7fef567fc640 1 -- 172.21.15.53:0/963434012 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48032660 con 0x7fef5815f3b0 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.838+0000 7fef5ddac640 1 -- 172.21.15.53:0/963434012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef5815f3b0 msgr2=0x7fef58163930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.838+0000 7fef5ddac640 1 --2- 172.21.15.53:0/963434012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef5815f3b0 0x7fef58163930 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fef480097a0 tx=0x7fef4802a140 comp rx=0 tx=0).stop 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.838+0000 7fef5ddac640 1 -- 172.21.15.53:0/963434012 shutdown_connections 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.838+0000 7fef5ddac640 1 --2- 172.21.15.53:0/963434012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef5815f3b0 0x7fef58163930 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.838+0000 7fef5ddac640 1 -- 172.21.15.53:0/963434012 >> 172.21.15.53:0/963434012 conn(0x7fef580b3c60 msgr2=0x7fef580b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.838+0000 7fef5ddac640 1 -- 172.21.15.53:0/963434012 shutdown_connections 2024-05-10T10:41:57.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.838+0000 7fef5ddac640 1 -- 172.21.15.53:0/963434012 wait complete. 2024-05-10T10:41:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.839+0000 7fef5ddac640 1 Processor -- start 2024-05-10T10:41:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.839+0000 7fef5ddac640 1 -- start start 2024-05-10T10:41:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef5ddac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef580cfdd0 0x7fef581600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef5ddac640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef580d0e90 0x7fef5814a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef5ddac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5815f3b0 0x7fef5815da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef5ddac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef580cedf0 con 0x7fef580cfdd0 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef5ddac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef580ceb10 con 0x7fef5815f3b0 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef5ddac640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fef5806d540 con 0x7fef580d0e90 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef580cfdd0 0x7fef581600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef580cfdd0 0x7fef581600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47838/0 (socket says 172.21.15.53:47838) 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 -- 172.21.15.53:0/1587537952 learned_addr learned my addr 172.21.15.53:0/1587537952 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef57fff640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5815f3b0 0x7fef5815da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef56ffd640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef580d0e90 0x7fef5814a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 -- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef580d0e90 msgr2=0x7fef5814a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef580d0e90 0x7fef5814a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 -- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5815f3b0 msgr2=0x7fef5815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5815f3b0 0x7fef5815da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 -- 172.21.15.53:0/1587537952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef58158a70 con 0x7fef580cfdd0 2024-05-10T10:41:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.840+0000 7fef577fe640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef580cfdd0 0x7fef581600e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fef480097a0 tx=0x7fef480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.841+0000 7fef3ffff640 1 -- 172.21.15.53:0/1587537952 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48042070 con 0x7fef580cfdd0 2024-05-10T10:41:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.841+0000 7fef3ffff640 1 -- 172.21.15.53:0/1587537952 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4803e670 con 0x7fef580cfdd0 2024-05-10T10:41:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.841+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef58158d50 con 0x7fef580cfdd0 2024-05-10T10:41:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.841+0000 7fef3ffff640 1 -- 172.21.15.53:0/1587537952 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48037040 con 0x7fef580cfdd0 2024-05-10T10:41:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.841+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5815a9f0 con 0x7fef580cfdd0 2024-05-10T10:41:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.842+0000 7fef3ffff640 1 -- 172.21.15.53:0/1587537952 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef48045070 con 0x7fef580cfdd0 2024-05-10T10:41:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.843+0000 7fef3ffff640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.843+0000 7fef3ffff640 1 -- 172.21.15.53:0/1587537952 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7fef480b7d20 con 0x7fef580cfdd0 2024-05-10T10:41:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.843+0000 7fef56ffd640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef60b9c740 1 Processor -- start 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef56ffd640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fef4c007fd0 tx=0x7fef4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef60b9c740 1 -- start start 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38dc160 0x5618a388eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a38be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38beb40 0x5618a3893950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef60b9c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5618a385b690 con 0x5618a38beb40 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.844+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5618a37f8f30 con 0x5618a38bf280 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5618a37b1b20 con 0x5618a38dc160 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38beb40 0x5618a3893950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a38be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38beb40 0x5618a3893950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47840/0 (socket says 172.21.15.53:47840) 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 -- 172.21.15.53:0/3418661282 learned_addr learned my addr 172.21.15.53:0/3418661282 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:57.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef577fe640 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38dc160 0x5618a388eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 -- 172.21.15.53:0/3418661282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38dc160 msgr2=0x5618a388eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38dc160 0x5618a388eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 -- 172.21.15.53:0/3418661282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 msgr2=0x5618a38be600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a38be600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 -- 172.21.15.53:0/3418661282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a37a9e60 con 0x5618a38beb40 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef57fff640 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38beb40 0x5618a3893950 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fef50001ff0 tx=0x7fef5000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cdaf6e1a3aa7d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef3d7fa640 1 -- 172.21.15.53:0/3418661282 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50019070 con 0x5618a38beb40 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef3d7fa640 1 -- 172.21.15.53:0/3418661282 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5000ee50 con 0x5618a38beb40 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.845+0000 7fef3d7fa640 1 -- 172.21.15.53:0/3418661282 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50014680 con 0x5618a38beb40 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 -- 172.21.15.53:0/3418661282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38beb40 msgr2=0x5618a3893950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38beb40 0x5618a3893950 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fef50001ff0 tx=0x7fef5000e530 comp rx=0 tx=0).stop 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 -- 172.21.15.53:0/3418661282 shutdown_connections 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38beb40 0x5618a3893950 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a38be600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3418661282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38dc160 0x5618a388eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 -- 172.21.15.53:0/3418661282 >> 172.21.15.53:0/3418661282 conn(0x5618a3890d60 msgr2=0x5618a38d9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 -- 172.21.15.53:0/3418661282 shutdown_connections 2024-05-10T10:41:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.846+0000 7fef60b9c740 1 -- 172.21.15.53:0/3418661282 wait complete. 2024-05-10T10:41:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.847+0000 7fef60b9c740 1 -- start start 2024-05-10T10:41:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.847+0000 7fef60b9c740 1 Processor -- start 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a38ec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a38933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef60b9c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5618a37b55e0 con 0x5618a38c1100 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5618a37f4ef0 con 0x5618a38dc160 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5618a3892f10 con 0x5618a38bf280 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef60b9c740 10 client.0 _ll_register_callbacks cb 0x5618a37b0320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a38933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a38933a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47844/0 (socket says 172.21.15.53:47844) 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 -- 172.21.15.53:0/2565124947 learned_addr learned my addr 172.21.15.53:0/2565124947 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef56ffd640 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.854 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef577fe640 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a38ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 -- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 msgr2=0x5618a38ec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a38ec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 -- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 msgr2=0x5618a38c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 -- 172.21.15.53:0/2565124947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a35d47d0 con 0x5618a38c1100 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef57fff640 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a38933a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fef50009010 tx=0x7fef50001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8867de989519c0d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef367fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef367fc640 1 -- 172.21.15.53:0/2565124947 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50024020 con 0x5618a38c1100 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.848+0000 7fef367fc640 1 -- 172.21.15.53:0/2565124947 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7fef50014c40 con 0x5618a38c1100 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef5000ee50 con 0x5618a38c1100 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef367fc640 1 -- 172.21.15.53:0/2565124947 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fef5002e050 con 0x5618a38c1100 2024-05-10T10:41:57.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 10 client.7731 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5618a350a3f0 con 0x5618a38c1100 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef367fc640 1 -- 172.21.15.53:0/2565124947 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5000ee50 con 0x5618a38c1100 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef367fc640 1 -- 172.21.15.53:0/2565124947 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7fef50031020 con 0x5618a38c1100 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 10 client.7731 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 10 client.7731 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 20 client.7731 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38bf770 0x5618a38bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 4 client.7731 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618a3806b40 con 0x5618a38bf770 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef577fe640 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38bf770 0x5618a38bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef367fc640 10 client.7731 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.849+0000 7fef577fe640 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38bf770 0x5618a38bfbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5efe79c52de0aba1 server_cookie=57559a510cb76e98 in_seq=0 out_seq=0 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef367fc640 1 -- 172.21.15.53:0/2565124947 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618a3806b40 con 0x5618a38bf770 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef367fc640 10 client.7731 handle_command_reply: tid=0 2024-05-10T10:41:57.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 client.7731 shutdown 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 msgr2=0x5618a38933a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a38933a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fef50009010 tx=0x7fef50001250 comp rx=0 tx=0).stop 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 shutdown_connections 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38bf770 0x5618a38bfbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a38933a0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef367fc640 0 client.7731 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 --2- 172.21.15.53:0/2565124947 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a38ec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 >> 172.21.15.53:0/2565124947 conn(0x5618a38beb40 msgr2=0x5618a388f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 shutdown_connections 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.850+0000 7fef60b9c740 1 -- 172.21.15.53:0/2565124947 wait complete. 2024-05-10T10:41:57.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.851+0000 7fef60b9c740 20 client.7731 trim_cache size 0 max 16384 2024-05-10T10:41:57.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 Processor -- start 2024-05-10T10:41:57.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 -- start start 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a3894550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a3894a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef58201f70 con 0x5618a38c1100 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef58200f50 con 0x5618a38dc160 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.869+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fef581f9020 con 0x5618a38bf280 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a3894a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a3894a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47846/0 (socket says 172.21.15.53:47846) 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 -- 172.21.15.53:0/508996881 learned_addr learned my addr 172.21.15.53:0/508996881 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef57fff640 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef577fe640 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a3894550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 -- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 msgr2=0x5618a3894550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a3894550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 -- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 msgr2=0x5618a38b94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38b94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 -- 172.21.15.53:0/508996881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a34ca0f0 con 0x5618a38c1100 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef57fff640 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38b94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef56ffd640 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a3894a90 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fef4c005240 tx=0x7fef4c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d3cd8fe76c7fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef3cff9640 1 -- 172.21.15.53:0/508996881 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c028070 con 0x5618a38c1100 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef3cff9640 1 -- 172.21.15.53:0/508996881 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c003800 con 0x5618a38c1100 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.870+0000 7fef3cff9640 1 -- 172.21.15.53:0/508996881 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0039b0 con 0x5618a38c1100 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 -- 172.21.15.53:0/508996881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 msgr2=0x5618a3894a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a3894a90 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fef4c005240 tx=0x7fef4c011000 comp rx=0 tx=0).stop 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 -- 172.21.15.53:0/508996881 shutdown_connections 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38dc160 0x5618a38b94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38c1100 0x5618a3894a90 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 --2- 172.21.15.53:0/508996881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38bf280 0x5618a3894550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 -- 172.21.15.53:0/508996881 >> 172.21.15.53:0/508996881 conn(0x5618a38beb40 msgr2=0x5618a38929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:57.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 -- 172.21.15.53:0/508996881 shutdown_connections 2024-05-10T10:41:57.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.871+0000 7fef60b9c740 1 -- 172.21.15.53:0/508996881 wait complete. 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.872+0000 7fef60b9c740 1 -- start start 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.872+0000 7fef60b9c740 1 Processor -- start 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a39404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38c1100 0x5618a38d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38dc160 0x5618a38d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef58200a60 con 0x5618a38dc160 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5618a3891b40 con 0x5618a38bf280 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5618a38b9b60 con 0x5618a38c1100 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 10 client.0 _ll_register_callbacks cb 0x7fef50017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38dc160 0x5618a38d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38dc160 0x5618a38d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47860/0 (socket says 172.21.15.53:47860) 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 -- 172.21.15.53:0/3716521224 learned_addr learned my addr 172.21.15.53:0/3716521224 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef56ffd640 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38c1100 0x5618a38d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef577fe640 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a39404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 -- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38c1100 msgr2=0x5618a38d7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38c1100 0x5618a38d7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 -- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 msgr2=0x5618a39404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a39404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 -- 172.21.15.53:0/3716521224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a38eaa90 con 0x5618a38dc160 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef57fff640 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38dc160 0x5618a38d0fa0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fef50001b80 tx=0x7fef50009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6127228b1ef73 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef367fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef367fc640 1 -- 172.21.15.53:0/3716521224 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50037060 con 0x5618a38dc160 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef367fc640 1 -- 172.21.15.53:0/3716521224 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7fef500294c0 con 0x5618a38dc160 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef4c003800 con 0x5618a38dc160 2024-05-10T10:41:57.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef367fc640 1 -- 172.21.15.53:0/3716521224 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fef5003d050 con 0x5618a38dc160 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 10 client.7737 fetch_fsmap learned FSMap version 82 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.873+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5618a38d7ec0 con 0x5618a38dc160 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef367fc640 1 -- 172.21.15.53:0/3716521224 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c003800 con 0x5618a38dc160 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef367fc640 1 -- 172.21.15.53:0/3716521224 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7fef50040020 con 0x5618a38dc160 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef60b9c740 10 client.7737 fetch_fsmap finished waiting for FSMap version 82 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef60b9c740 10 client.7737 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef60b9c740 20 client.7737 populate_metadata read hostname 'smithi053' 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38d14e0 0x5618a38fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef60b9c740 4 client.7737 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 --> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618a38009b0 con 0x5618a38d14e0 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef577fe640 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38d14e0 0x5618a38fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef367fc640 10 client.7737 ms_handle_connect on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef577fe640 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38d14e0 0x5618a38fbe40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abfcc7c6114eced7 server_cookie=7e921a67d9c0a423 in_seq=0 out_seq=0 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef367fc640 1 -- 172.21.15.53:0/3716521224 <== mds.0 v2:172.21.15.53:6834/4122299129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5618a38009b0 con 0x5618a38d14e0 2024-05-10T10:41:57.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef367fc640 10 client.7737 handle_command_reply: tid=0 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.874+0000 7fef60b9c740 1 client.7737 shutdown 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38dc160 msgr2=0x5618a38d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38dc160 0x5618a38d0fa0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fef50001b80 tx=0x7fef50009040 comp rx=0 tx=0).stop 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 shutdown_connections 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:6834/4122299129,v1:172.21.15.53:6835/4122299129] conn(0x5618a38d14e0 0x5618a38fbe40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5618a38dc160 0x5618a38d0fa0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5618a38c1100 0x5618a38d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 --2- 172.21.15.53:0/3716521224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5618a38bf280 0x5618a39404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 >> 172.21.15.53:0/3716521224 conn(0x5618a38beb40 msgr2=0x5618a38d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef367fc640 0 client.7737 ms_handle_reset on v2:172.21.15.53:6834/4122299129 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 shutdown_connections 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 1 -- 172.21.15.53:0/3716521224 wait complete. 2024-05-10T10:41:57.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef60b9c740 20 client.7737 trim_cache size 0 max 16384 2024-05-10T10:41:57.882 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:41:57.882 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:41:57.882 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:41:57.882 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 msgr2=0x7fef44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fef4c007fd0 tx=0x7fef4c009040 comp rx=0 tx=0).stop 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef580cfdd0 msgr2=0x7fef581600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef580cfdd0 0x7fef581600e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fef480097a0 tx=0x7fef480046c0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 shutdown_connections 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5815f3b0 0x7fef5815da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.875+0000 7fef5ddac640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef580d0e90 0x7fef5814a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.876+0000 7fef5ddac640 1 --2- 172.21.15.53:0/1587537952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef580cfdd0 0x7fef581600e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.876+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 >> 172.21.15.53:0/1587537952 conn(0x7fef580b3c60 msgr2=0x7fef5806f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:57.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.876+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 shutdown_connections 2024-05-10T10:41:57.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:57.876+0000 7fef5ddac640 1 -- 172.21.15.53:0/1587537952 wait complete. 2024-05-10T10:41:57.887 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:41:57.988 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.092+0000 7f842611b640 1 Processor -- start 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.092+0000 7f842611b640 1 -- start start 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f842611b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8420162750 0x7f84200d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f842611b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f84200d5cb0 0x7f84200d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f842611b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84200b45f0 con 0x7f8420162b30 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f842611b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f84200b4f20 con 0x7f8420162750 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f842611b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f84200d0fe0 con 0x7f84200d5cb0 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f841f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8420162750 0x7f84200d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f841f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8420162750 0x7f84200d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47874/0 (socket says 172.21.15.53:47874) 2024-05-10T10:41:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f841f7fe640 1 -- 172.21.15.53:0/1464551285 learned_addr learned my addr 172.21.15.53:0/1464551285 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.093+0000 7f841effd640 1 --2- 172.21.15.53:0/1464551285 >> v2:172.21.15.135:3300/0 conn(0x7f84200d5cb0 0x7f84200d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.094+0000 7f841f7fe640 1 -- 172.21.15.53:0/1464551285 >> v2:172.21.15.135:3300/0 conn(0x7f84200d5cb0 msgr2=0x7f84200d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.094+0000 7f841f7fe640 1 --2- 172.21.15.53:0/1464551285 >> v2:172.21.15.135:3300/0 conn(0x7f84200d5cb0 0x7f84200d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.094+0000 7f841f7fe640 1 -- 172.21.15.53:0/1464551285 >> v1:172.21.15.135:6789/0 conn(0x7f8420162b30 legacy=0x7f84200d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.094+0000 7f841f7fe640 1 -- 172.21.15.53:0/1464551285 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84200b48d0 con 0x7f8420162750 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.094+0000 7f841f7fe640 1 --2- 172.21.15.53:0/1464551285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162750 0x7f84200d0670 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f84200b4e90 tx=0x7f841002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2df40dc72f8def server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.094+0000 7f841e7fc640 1 -- 172.21.15.53:0/1464551285 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410037070 con 0x7f8420162750 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f841e7fc640 1 -- 172.21.15.53:0/1464551285 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8410032440 con 0x7f8420162750 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f841e7fc640 1 -- 172.21.15.53:0/1464551285 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410032810 con 0x7f8420162750 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 -- 172.21.15.53:0/1464551285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162750 msgr2=0x7f84200d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 --2- 172.21.15.53:0/1464551285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162750 0x7f84200d0670 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f84200b4e90 tx=0x7f841002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 -- 172.21.15.53:0/1464551285 shutdown_connections 2024-05-10T10:41:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 --2- 172.21.15.53:0/1464551285 >> v2:172.21.15.135:3300/0 conn(0x7f84200d5cb0 0x7f84200d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 --2- 172.21.15.53:0/1464551285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162750 0x7f84200d0670 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 -- 172.21.15.53:0/1464551285 >> 172.21.15.53:0/1464551285 conn(0x7f842009e3b0 msgr2=0x7f84200d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 -- 172.21.15.53:0/1464551285 shutdown_connections 2024-05-10T10:41:58.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.095+0000 7f842611b640 1 -- 172.21.15.53:0/1464551285 wait complete. 2024-05-10T10:41:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.096+0000 7f842611b640 1 Processor -- start 2024-05-10T10:41:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f842611b640 1 -- start start 2024-05-10T10:41:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f842611b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8420162750 0x7f84200b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f842611b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162b30 0x7f8420159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f842611b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84201596b0 0x7f84200b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f842611b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84200d0d30 con 0x7f8420162b30 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f842611b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84200d0bb0 con 0x7f84201596b0 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f842611b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f84200b63c0 con 0x7f8420162750 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f841effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162b30 0x7f8420159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f841effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162b30 0x7f8420159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47886/0 (socket says 172.21.15.53:47886) 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.097+0000 7f841effd640 1 -- 172.21.15.53:0/2281409941 learned_addr learned my addr 172.21.15.53:0/2281409941 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841f7fe640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8420162750 0x7f84200b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841ffff640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84201596b0 0x7f84200b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841effd640 1 -- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8420162750 msgr2=0x7f84200b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841effd640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8420162750 0x7f84200b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841effd640 1 -- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84201596b0 msgr2=0x7f84200b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841effd640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84201596b0 0x7f84200b87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841effd640 1 -- 172.21.15.53:0/2281409941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84200cc120 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f841effd640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162b30 0x7f8420159170 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f841400e8b0 tx=0x7f841400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f8407fff640 1 -- 172.21.15.53:0/2281409941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414017070 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f8407fff640 1 -- 172.21.15.53:0/2281409941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841400c940 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84200cc3c0 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f8407fff640 1 -- 172.21.15.53:0/2281409941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841400cd30 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.098+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84200cc980 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.099+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842006a220 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.100+0000 7f8407fff640 1 -- 172.21.15.53:0/2281409941 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f841401a070 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.100+0000 7f8407fff640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f840c0762d0 0x7f840c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.100+0000 7f8407fff640 1 -- 172.21.15.53:0/2281409941 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6159+0+0 (secure 0 0 0) 0x7f8414097ba0 con 0x7f8420162b30 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.100+0000 7f841f7fe640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f840c0762d0 0x7f840c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.101+0000 7f841f7fe640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f840c0762d0 0x7f840c078790 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8410034000 tx=0x7f841002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.105+0000 7f8407fff640 1 -- 172.21.15.53:0/2281409941 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8414061350 con 0x7f8420162b30 2024-05-10T10:41:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.338+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f84200d1570 con 0x7f8420162b30 2024-05-10T10:41:58.339 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:41:58.338+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:41:58.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.340+0000 7f8407fff640 1 -- 172.21.15.53:0/2281409941 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f8414060c50 con 0x7f8420162b30 2024-05-10T10:41:58.340 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:41:58.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f840c0762d0 msgr2=0x7f840c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f840c0762d0 0x7f840c078790 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8410034000 tx=0x7f841002e000 comp rx=0 tx=0).stop 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162b30 msgr2=0x7f8420159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162b30 0x7f8420159170 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f841400e8b0 tx=0x7f841400ed70 comp rx=0 tx=0).stop 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 shutdown_connections 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f840c0762d0 0x7f840c078790 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84201596b0 0x7f84200b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8420162b30 0x7f8420159170 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 --2- 172.21.15.53:0/2281409941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8420162750 0x7f84200b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 >> 172.21.15.53:0/2281409941 conn(0x7f842009e3b0 msgr2=0x7f84200d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.343+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 shutdown_connections 2024-05-10T10:41:58.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.344+0000 7f842611b640 1 -- 172.21.15.53:0/2281409941 wait complete. 2024-05-10T10:41:58.348 INFO:tasks.ceph.mds.c.smithi053.stderr: -6948> 2024-05-10T10:41:21.403+0000 7f17a2b40640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:41:58.359 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-05-10T10:41:58.384 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:41:58.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 -- start start 2024-05-10T10:41:58.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 Processor -- start 2024-05-10T10:41:58.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b160710c00 0x55b1614de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b160711800 0x55b1614de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b160711c00 0x55b1614deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b1614ae180 con 0x55b160710c00 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b1614ae300 con 0x55b160711800 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24efe0280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b1614ae480 con 0x55b160711c00 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24d78e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b160710c00 0x55b1614de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24d78e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b160710c00 0x55b1614de000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47914/0 (socket says 172.21.15.53:47914) 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.403+0000 7fc24d78e640 1 -- 172.21.15.53:0/2220414149 learned_addr learned my addr 172.21.15.53:0/2220414149 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b160711800 0x55b1614de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24df8f640 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b160711c00 0x55b1614deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24d78e640 1 -- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b160711c00 msgr2=0x55b1614deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24d78e640 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b160711c00 0x55b1614deb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24d78e640 1 -- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b160711800 msgr2=0x55b1614de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24d78e640 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b160711800 0x55b1614de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24d78e640 1 -- 172.21.15.53:0/2220414149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b160746780 con 0x55b160710c00 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b160711800 0x55b1614de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:58.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24df8f640 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b160711c00 0x55b1614deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24d78e640 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b160710c00 0x55b1614de000 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x55b16152d710 tx=0x55b1614b0a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7530bf31d1f1668 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24af89640 1 -- 172.21.15.53:0/2220414149 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b1607369c0 con 0x55b160710c00 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24af89640 1 -- 172.21.15.53:0/2220414149 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b16073cfc0 con 0x55b160710c00 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24af89640 1 -- 172.21.15.53:0/2220414149 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b160736680 con 0x55b160710c00 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24efe0280 1 -- 172.21.15.53:0/2220414149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1607463c0 con 0x55b160710c00 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24efe0280 1 -- 172.21.15.53:0/2220414149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b160747680 con 0x55b160710c00 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24efe0280 4 waiting for MDS map... 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24af89640 1 -- 172.21.15.53:0/2220414149 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x55b1615e8200 con 0x55b160710c00 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24af89640 1 -- 172.21.15.53:0/2220414149 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x55b16076b180 con 0x55b160710c00 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24efe0280 4 Got MDS map 83 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.404+0000 7fc24efe0280 10 main: JournalTool::main 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.405+0000 7fc24efe0280 4 main: JournalTool: connecting to RADOS... 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.405+0000 7fc24efe0280 1 Processor -- start 2024-05-10T10:41:58.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.405+0000 7fc24efe0280 1 -- start start 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24efe0280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b1614ae600 con 0x55b16164c800 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24efe0280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b1614ae780 con 0x55b16164c400 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24efe0280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b1614ae900 con 0x55b16164c000 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47916/0 (socket says 172.21.15.53:47916) 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 -- 172.21.15.53:0/321901299 learned_addr learned my addr 172.21.15.53:0/321901299 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24d78e640 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24df8f640 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 -- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c000 msgr2=0x55b1614df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.413 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 -- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c400 msgr2=0x55b1614df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 -- 172.21.15.53:0/321901299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1607474a0 con 0x55b16164c800 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24d78e640 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c800 0x55b1614dfb80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x55b1615f4810 tx=0x55b1615fb8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f034ced67691a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc248f85640 1 -- 172.21.15.53:0/321901299 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b1607369c0 con 0x55b16164c800 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc248f85640 1 -- 172.21.15.53:0/321901299 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b16073ca80 con 0x55b16164c800 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.406+0000 7fc248f85640 1 -- 172.21.15.53:0/321901299 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b160736680 con 0x55b16164c800 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 -- 172.21.15.53:0/321901299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c800 msgr2=0x55b1614dfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c800 0x55b1614dfb80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x55b1615f4810 tx=0x55b1615fb8c0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 -- 172.21.15.53:0/321901299 shutdown_connections 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.414 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 --2- 172.21.15.53:0/321901299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 -- 172.21.15.53:0/321901299 >> 172.21.15.53:0/321901299 conn(0x55b1614efc00 msgr2=0x55b16080b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 -- 172.21.15.53:0/321901299 shutdown_connections 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 -- 172.21.15.53:0/321901299 wait complete. 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 Processor -- start 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.407+0000 7fc24efe0280 1 -- start start 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24efe0280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24efe0280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b1614afc80 con 0x55b16164c000 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24efe0280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b1614afb00 con 0x55b16164c800 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24efe0280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b1614af980 con 0x55b16164c400 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47924/0 (socket says 172.21.15.53:47924) 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 -- 172.21.15.53:0/2078480954 learned_addr learned my addr 172.21.15.53:0/2078480954 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24d78e640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 -- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c400 msgr2=0x55b1614df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 -- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c800 msgr2=0x55b1614dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24df8f640 1 -- 172.21.15.53:0/2078480954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b160746000 con 0x55b16164c000 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b16164c800 0x55b1614dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.408+0000 7fc24d78e640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b16164c400 0x55b1614df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.409+0000 7fc24df8f640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c000 0x55b1614df080 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x55b1615f5ad0 tx=0x55b1615fb9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.409+0000 7fc247782640 1 -- 172.21.15.53:0/2078480954 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b1607369c0 con 0x55b16164c000 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.409+0000 7fc247782640 1 -- 172.21.15.53:0/2078480954 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b16073cc40 con 0x55b16164c000 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.409+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b160746960 con 0x55b16164c000 2024-05-10T10:41:58.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.409+0000 7fc247782640 1 -- 172.21.15.53:0/2078480954 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b160736680 con 0x55b16164c000 2024-05-10T10:41:58.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.409+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b160747860 con 0x55b16164c000 2024-05-10T10:41:58.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.409+0000 7fc24efe0280 4 main: JournalTool: resolving pool 15 2024-05-10T10:41:58.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.410+0000 7fc247782640 1 -- 172.21.15.53:0/2078480954 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55b1615dac00 con 0x55b16164c000 2024-05-10T10:41:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.410+0000 7fc247782640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55b1615dcc00 0x55b1614e0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.410+0000 7fc247782640 1 -- 172.21.15.53:0/2078480954 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x55b161653a00 con 0x55b16164c000 2024-05-10T10:41:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.410+0000 7fc24efe0280 4 main: JournalTool: creating IoCtx.. 2024-05-10T10:41:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24efe0280 4 main: Executing for rank 0 2024-05-10T10:41:58.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24efe0280 4 Filtering by inode '1099511627777' 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b16164cc00 0x55b1614e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55b16164d000 con 0x55b16164cc00 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24d78e640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55b1615dcc00 0x55b1614e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b16164cc00 0x55b1614e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24d78e640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55b1615dcc00 0x55b1614e0100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55b16152dbf0 tx=0x55b1607c8d80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.411+0000 7fc24cf8d640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b16164cc00 0x55b1614e0680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.414+0000 7fc24cf8d640 1 -- 172.21.15.53:0/2078480954 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55b16081e000 con 0x55b16164cc00 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.414+0000 7fc24efe0280 4 Pointer 400.00000000 is readable 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.414+0000 7fc24efe0280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.414+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55b16164dc00 con 0x55b16164cc00 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24cf8d640 1 -- 172.21.15.53:0/2078480954 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55b16081e000 con 0x55b16164cc00 2024-05-10T10:41:58.420 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 10 Header 0x400000 0x407d21 0x40809d 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 10 Starting journal scan from offset 0x407d21 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55b16164d400 con 0x55b16164cc00 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24cf8d640 1 -- 172.21.15.53:0/2078480954 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55b16081e000 con 0x55b16164cc00 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 10 Parsing data, 0x37c bytes available 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 10 Attempting decode at 0x407d21 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 10 Consumed 0x37c bytes 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 10 Valid entry at 0x407d21 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55b16164d800 0x55b1614e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55b16164d000 con 0x55b16164d800 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.415+0000 7fc24df8f640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55b16164d800 0x55b1614e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24df8f640 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55b16164d800 0x55b1614e0c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.421 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24df8f640 1 -- 172.21.15.53:0/2078480954 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b16081e500 con 0x55b16164d800 2024-05-10T10:41:58.422 INFO:teuthology.orchestra.run.smithi053.stdout:Events by type: 2024-05-10T10:41:58.422 INFO:teuthology.orchestra.run.smithi053.stdout:Errors: 0 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 4 Reached end of journal objects 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 4 Scanned objects, 0 missing, 1 valid 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 4 Events scanned, 0 gaps 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 4 Found 1 valid events 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 4 Selected 0 events events for processing 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 4 0 events 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b16164cc00 msgr2=0x55b1614e0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55b16164cc00 0x55b1614e0680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55b16164d800 msgr2=0x55b1614e0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55b16164d800 0x55b1614e0c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55b1615dcc00 msgr2=0x55b1614e0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55b1615dcc00 0x55b1614e0100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55b16152dbf0 tx=0x55b1607c8d80 comp rx=0 tx=0).stop 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c000 msgr2=0x55b1614df080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24cf8d640 1 -- 172.21.15.53:0/2078480954 reap_dead start 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c000 0x55b1614df080 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x55b1615f5ad0 tx=0x55b1615fb9e0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 shutdown_connections 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2078480954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16164c000 0x55b1614df080 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 >> 172.21.15.53:0/2078480954 conn(0x55b1614efc00 msgr2=0x55b16080b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 shutdown_connections 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.416+0000 7fc24efe0280 1 -- 172.21.15.53:0/2078480954 wait complete. 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 -- 172.21.15.53:0/2220414149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b160710c00 msgr2=0x55b1614de000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b160710c00 0x55b1614de000 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x55b16152d710 tx=0x55b1614b0a20 comp rx=0 tx=0).stop 2024-05-10T10:41:58.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 -- 172.21.15.53:0/2220414149 shutdown_connections 2024-05-10T10:41:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b160711c00 0x55b1614deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b160711800 0x55b1614de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 --2- 172.21.15.53:0/2220414149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b160710c00 0x55b1614de000 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 -- 172.21.15.53:0/2220414149 >> 172.21.15.53:0/2220414149 conn(0x55b160711000 msgr2=0x55b16080a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 -- 172.21.15.53:0/2220414149 shutdown_connections 2024-05-10T10:41:58.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.417+0000 7fc24efe0280 1 -- 172.21.15.53:0/2220414149 wait complete. 2024-05-10T10:41:58.426 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.067174, result: Events by type: Errors: 0 2024-05-10T10:41:58.426 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-05-10T10:41:58.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 -- start start 2024-05-10T10:41:58.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 Processor -- start 2024-05-10T10:41:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba66142c00 0x55ba66f10000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba66143800 0x55ba66f10580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba66143c00 0x55ba66f10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ba66ee0180 con 0x55ba66142c00 2024-05-10T10:41:58.515 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba66ee0300 con 0x55ba66143c00 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa21881a280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ba66ee0480 con 0x55ba66143800 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa216fc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba66142c00 0x55ba66f10000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa216fc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba66142c00 0x55ba66f10000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47936/0 (socket says 172.21.15.53:47936) 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.508+0000 7fa216fc6640 1 -- 172.21.15.53:0/3477846898 learned_addr learned my addr 172.21.15.53:0/3477846898 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa2167c5640 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba66143800 0x55ba66f10580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa2177c7640 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba66143c00 0x55ba66f10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa216fc6640 1 -- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba66143800 msgr2=0x55ba66f10580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa216fc6640 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba66143800 0x55ba66f10580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa216fc6640 1 -- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba66143c00 msgr2=0x55ba66f10b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa216fc6640 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba66143c00 0x55ba66f10b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa216fc6640 1 -- 172.21.15.53:0/3477846898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba66178780 con 0x55ba66142c00 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa216fc6640 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba66142c00 0x55ba66f10000 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x55ba66f650e0 tx=0x55ba66ee2a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb75c80f7587706 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa2147c1640 1 -- 172.21.15.53:0/3477846898 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba661689c0 con 0x55ba66142c00 2024-05-10T10:41:58.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa2147c1640 1 -- 172.21.15.53:0/3477846898 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ba6616e700 con 0x55ba66142c00 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa2147c1640 1 -- 172.21.15.53:0/3477846898 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba66168680 con 0x55ba66142c00 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa21881a280 1 -- 172.21.15.53:0/3477846898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba661794a0 con 0x55ba66142c00 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa21881a280 1 -- 172.21.15.53:0/3477846898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ba66178000 con 0x55ba66142c00 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa21881a280 4 waiting for MDS map... 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa2147c1640 1 -- 172.21.15.53:0/3477846898 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x55ba67024200 con 0x55ba66142c00 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa2147c1640 1 -- 172.21.15.53:0/3477846898 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x55ba6619d180 con 0x55ba66142c00 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa21881a280 4 Got MDS map 83 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa21881a280 10 main: JournalTool::main 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.509+0000 7fa21881a280 4 main: JournalTool: connecting to RADOS... 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.510+0000 7fa21881a280 1 Processor -- start 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.510+0000 7fa21881a280 1 -- start start 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.510+0000 7fa21881a280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e000 0x55ba66f11080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa21881a280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e400 0x55ba66f11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa21881a280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 0x55ba66f11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa21881a280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ba66ee0600 con 0x55ba6706e000 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa21881a280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba66ee0780 con 0x55ba6706e400 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa21881a280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ba66ee0900 con 0x55ba6706e800 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e000 0x55ba66f11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e000 0x55ba66f11080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47942/0 (socket says 172.21.15.53:47942) 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 -- 172.21.15.53:0/852714686 learned_addr learned my addr 172.21.15.53:0/852714686 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2167c5640 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 0x55ba66f11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa216fc6640 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e400 0x55ba66f11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 -- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 msgr2=0x55ba66f11b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 0x55ba66f11b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 -- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e400 msgr2=0x55ba66f11600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e400 0x55ba66f11600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 -- 172.21.15.53:0/852714686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba661783c0 con 0x55ba6706e000 2024-05-10T10:41:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2177c7640 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e000 0x55ba66f11080 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x55ba66f44900 tx=0x55ba67029800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2a10fdaadd4f27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2127bd640 1 -- 172.21.15.53:0/852714686 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba661689c0 con 0x55ba6706e000 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2127bd640 1 -- 172.21.15.53:0/852714686 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ba6616efc0 con 0x55ba6706e000 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.511+0000 7fa2127bd640 1 -- 172.21.15.53:0/852714686 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba66168680 con 0x55ba6706e000 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 -- 172.21.15.53:0/852714686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e000 msgr2=0x55ba66f11080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e000 0x55ba66f11080 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x55ba66f44900 tx=0x55ba67029800 comp rx=0 tx=0).stop 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 -- 172.21.15.53:0/852714686 shutdown_connections 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 0x55ba66f11b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e400 0x55ba66f11600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 --2- 172.21.15.53:0/852714686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e000 0x55ba66f11080 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 -- 172.21.15.53:0/852714686 >> 172.21.15.53:0/852714686 conn(0x55ba66f31c00 msgr2=0x55ba6623d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 -- 172.21.15.53:0/852714686 shutdown_connections 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 -- 172.21.15.53:0/852714686 wait complete. 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 Processor -- start 2024-05-10T10:41:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.512+0000 7fa21881a280 1 -- start start 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa21881a280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e000 0x55ba66f11080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa21881a280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e400 0x55ba66f11600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa21881a280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 0x55ba66f11b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa21881a280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ba66ee1c80 con 0x55ba6706e400 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa21881a280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba66ee1b00 con 0x55ba6706e000 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa21881a280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ba66ee1980 con 0x55ba6706e800 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e400 0x55ba66f11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e400 0x55ba66f11600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47948/0 (socket says 172.21.15.53:47948) 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 -- 172.21.15.53:0/3268115048 learned_addr learned my addr 172.21.15.53:0/3268115048 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa2177c7640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e000 0x55ba66f11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa2167c5640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 0x55ba66f11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 -- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 msgr2=0x55ba66f11b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba6706e800 0x55ba66f11b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 -- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e000 msgr2=0x55ba66f11080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6706e000 0x55ba66f11080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.513+0000 7fa216fc6640 1 -- 172.21.15.53:0/3268115048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba66178960 con 0x55ba6706e400 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.514+0000 7fa216fc6640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e400 0x55ba66f11600 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x55ba66f44510 tx=0x55ba661fad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.514+0000 7fa210fba640 1 -- 172.21.15.53:0/3268115048 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba66168b60 con 0x55ba6706e400 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.514+0000 7fa210fba640 1 -- 172.21.15.53:0/3268115048 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ba6616ee00 con 0x55ba6706e400 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.514+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba66179680 con 0x55ba6706e400 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.514+0000 7fa210fba640 1 -- 172.21.15.53:0/3268115048 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba66168820 con 0x55ba6706e400 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.514+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba66179860 con 0x55ba6706e400 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.514+0000 7fa21881a280 4 main: JournalTool: resolving pool 15 2024-05-10T10:41:58.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.515+0000 7fa210fba640 1 -- 172.21.15.53:0/3268115048 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55ba67082c00 con 0x55ba6706e400 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.515+0000 7fa210fba640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ba6708f000 0x55ba66f12100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.515+0000 7fa210fba640 1 -- 172.21.15.53:0/3268115048 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x55ba67075200 con 0x55ba6706e400 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.515+0000 7fa21881a280 4 main: JournalTool: creating IoCtx.. 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.515+0000 7fa21881a280 4 main: Executing for rank 0 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.515+0000 7fa21881a280 4 Filtering by inode '1099511627778' 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.515+0000 7fa21881a280 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ba6706ec00 0x55ba66f12680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55ba6706f000 con 0x55ba6706ec00 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa2177c7640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ba6708f000 0x55ba66f12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.524 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa2167c5640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ba6706ec00 0x55ba66f12680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa2167c5640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ba6706ec00 0x55ba66f12680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa2177c7640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ba6708f000 0x55ba66f12100 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55ba66f44900 tx=0x55ba66ee2780 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa2167c5640 1 -- 172.21.15.53:0/3268115048 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55ba66250000 con 0x55ba6706ec00 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa21881a280 4 Pointer 400.00000000 is readable 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa21881a280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.516+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55ba6708f400 con 0x55ba6706ec00 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa2167c5640 1 -- 172.21.15.53:0/3268115048 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55ba66250000 con 0x55ba6706ec00 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 10 Header 0x400000 0x407d21 0x40809d 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 10 Starting journal scan from offset 0x407d21 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55ba6708f800 con 0x55ba6706ec00 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa2167c5640 1 -- 172.21.15.53:0/3268115048 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55ba66250000 con 0x55ba6706ec00 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 10 Parsing data, 0x37c bytes available 2024-05-10T10:41:58.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 10 Attempting decode at 0x407d21 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 10 Consumed 0x37c bytes 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 10 Valid entry at 0x407d21 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ba6708fc00 0x55ba66f12c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) -- 0x55ba6708f400 con 0x55ba6708fc00 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.517+0000 7fa2177c7640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ba6708fc00 0x55ba66f12c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa2177c7640 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ba6708fc00 0x55ba66f12c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa2177c7640 1 -- 172.21.15.53:0/3268115048 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ba66250500 con 0x55ba6708fc00 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stdout:Events by type: 2024-05-10T10:41:58.526 INFO:teuthology.orchestra.run.smithi053.stdout:Errors: 0 2024-05-10T10:41:58.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 4 Reached end of journal objects 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 4 Scanned objects, 0 missing, 1 valid 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 4 Events scanned, 0 gaps 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 4 Found 1 valid events 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 4 Selected 0 events events for processing 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 4 0 events 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ba6706ec00 msgr2=0x55ba66f12680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ba6706ec00 0x55ba66f12680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ba6708fc00 msgr2=0x55ba66f12c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ba6708fc00 0x55ba66f12c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ba6708f000 msgr2=0x55ba66f12100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ba6708f000 0x55ba66f12100 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55ba66f44900 tx=0x55ba66ee2780 comp rx=0 tx=0).stop 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e400 msgr2=0x55ba66f11600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3268115048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba6706e400 0x55ba66f11600 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x55ba66f44510 tx=0x55ba661fad80 comp rx=0 tx=0).stop 2024-05-10T10:41:58.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa2167c5640 1 -- 172.21.15.53:0/3268115048 reap_dead start 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa2167c5640 1 -- 172.21.15.53:0/3268115048 reap_dead start 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 shutdown_connections 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 >> 172.21.15.53:0/3268115048 conn(0x55ba66f31c00 msgr2=0x55ba6623d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 shutdown_connections 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3268115048 wait complete. 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3477846898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba66142c00 msgr2=0x55ba66f10000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba66142c00 0x55ba66f10000 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x55ba66f650e0 tx=0x55ba66ee2a20 comp rx=0 tx=0).stop 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3477846898 shutdown_connections 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba66143c00 0x55ba66f10b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ba66143800 0x55ba66f10580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 --2- 172.21.15.53:0/3477846898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba66142c00 0x55ba66f10000 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.518+0000 7fa21881a280 1 -- 172.21.15.53:0/3477846898 >> 172.21.15.53:0/3477846898 conn(0x55ba66143000 msgr2=0x55ba6623c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.519+0000 7fa21881a280 1 -- 172.21.15.53:0/3477846898 shutdown_connections 2024-05-10T10:41:58.530 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.519+0000 7fa21881a280 1 -- 172.21.15.53:0/3477846898 wait complete. 2024-05-10T10:41:58.531 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.104395, result: Events by type: Errors: 0 2024-05-10T10:41:58.531 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-05-10T10:41:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.575+0000 7f84ad00df40 1 Processor -- start 2024-05-10T10:41:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.575+0000 7f84ad00df40 1 -- start start 2024-05-10T10:41:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84ad00df40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92a5630 0x5603b9330610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84ad00df40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92bf800 0x5603b93da0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84ad00df40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b932f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84ad00df40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5603b921df50 con 0x5603b9334c00 2024-05-10T10:41:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84ad00df40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603b9221f90 con 0x5603b92a5630 2024-05-10T10:41:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84ad00df40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5603b92a5fe0 con 0x5603b92bf800 2024-05-10T10:41:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b932f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b932f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47964/0 (socket says 172.21.15.53:47964) 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 -- 172.21.15.53:0/3837850633 learned_addr learned my addr 172.21.15.53:0/3837850633 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84a9697640 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92bf800 0x5603b93da0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84a9e98640 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92a5630 0x5603b9330610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 -- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92bf800 msgr2=0x5603b93da0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92bf800 0x5603b93da0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 -- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92a5630 msgr2=0x5603b9330610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92a5630 0x5603b9330610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 -- 172.21.15.53:0/3837850633 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b92be820 con 0x5603b9334c00 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84a9697640 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92bf800 0x5603b93da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84aa699640 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b932f3e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f84a400a4a0 tx=0x7f84a404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadabbe75166d2c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.576+0000 7f84a3fff640 1 -- 172.21.15.53:0/3837850633 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4063070 con 0x5603b9334c00 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84a3fff640 1 -- 172.21.15.53:0/3837850633 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a40548a0 con 0x5603b9334c00 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84a3fff640 1 -- 172.21.15.53:0/3837850633 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4054c70 con 0x5603b9334c00 2024-05-10T10:41:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 -- 172.21.15.53:0/3837850633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 msgr2=0x5603b932f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b932f3e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f84a400a4a0 tx=0x7f84a404e000 comp rx=0 tx=0).stop 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 -- 172.21.15.53:0/3837850633 shutdown_connections 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b932f3e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92bf800 0x5603b93da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 --2- 172.21.15.53:0/3837850633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92a5630 0x5603b9330610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 -- 172.21.15.53:0/3837850633 >> 172.21.15.53:0/3837850633 conn(0x5603b92757c0 msgr2=0x5603b92be0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 -- 172.21.15.53:0/3837850633 shutdown_connections 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.577+0000 7f84ad00df40 1 -- 172.21.15.53:0/3837850633 wait complete. 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 Processor -- start 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 -- start start 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92a5630 0x5603b9348180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92bf800 0x5603b9332f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b92b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5603b93325e0 con 0x5603b9334c00 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603b934a3e0 con 0x5603b92bf800 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84ad00df40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5603b93328c0 con 0x5603b92a5630 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84aa699640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b92b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84aa699640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b92b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47968/0 (socket says 172.21.15.53:47968) 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.578+0000 7f84aa699640 1 -- 172.21.15.53:0/2136084959 learned_addr learned my addr 172.21.15.53:0/2136084959 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84a9e98640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92a5630 0x5603b9348180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84a9697640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92bf800 0x5603b9332f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84aa699640 1 -- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92a5630 msgr2=0x5603b9348180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84aa699640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5603b92a5630 0x5603b9348180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84aa699640 1 -- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92bf800 msgr2=0x5603b9332f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84aa699640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603b92bf800 0x5603b9332f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84aa699640 1 -- 172.21.15.53:0/2136084959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b9348ee0 con 0x5603b9334c00 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84aa699640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b92b7dc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f84a4025780 tx=0x7f84a4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84a2ffd640 1 -- 172.21.15.53:0/2136084959 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4063050 con 0x5603b9334c00 2024-05-10T10:41:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84a2ffd640 1 -- 172.21.15.53:0/2136084959 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a4068650 con 0x5603b9334c00 2024-05-10T10:41:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603b92b63f0 con 0x5603b9334c00 2024-05-10T10:41:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84a2ffd640 1 -- 172.21.15.53:0/2136084959 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4068a20 con 0x5603b9334c00 2024-05-10T10:41:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.579+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b93497c0 con 0x5603b9334c00 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.581+0000 7f84a2ffd640 1 -- 172.21.15.53:0/2136084959 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f84a4066070 con 0x5603b9334c00 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.581+0000 7f84a2ffd640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f848c074cb0 0x7f848c077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.581+0000 7f84a2ffd640 1 -- 172.21.15.53:0/2136084959 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7f84a40dbe00 con 0x5603b9334c00 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.581+0000 7f84a9e98640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f848c074cb0 0x7f848c077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.581+0000 7f84ad00df40 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5603b92b8300 0x5603b934d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.582+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5603b9333460 con 0x5603b92b8300 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.582+0000 7f84a9697640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5603b92b8300 0x5603b934d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.582+0000 7f84a9e98640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f848c074cb0 0x7f848c077170 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5603b9333940 tx=0x7f849c001840 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.582+0000 7f84a9697640 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5603b92b8300 0x5603b934d380 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84a9697640 1 -- 172.21.15.53:0/2136084959 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v106'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f849400e3e0 con 0x5603b92b8300 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5603b92b8300 msgr2=0x5603b934d380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x5603b92b8300 0x5603b934d380 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f848c074cb0 msgr2=0x7f848c077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f848c074cb0 0x7f848c077170 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5603b9333940 tx=0x7f849c001840 comp rx=0 tx=0).stop 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 msgr2=0x5603b92b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 --2- 172.21.15.53:0/2136084959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603b9334c00 0x5603b92b7dc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f84a4025780 tx=0x7f84a4003330 comp rx=0 tx=0).stop 2024-05-10T10:41:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84aa699640 1 -- 172.21.15.53:0/2136084959 reap_dead start 2024-05-10T10:41:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 shutdown_connections 2024-05-10T10:41:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.585+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 >> 172.21.15.53:0/2136084959 conn(0x5603b92757c0 msgr2=0x5603b92be0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.586+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 shutdown_connections 2024-05-10T10:41:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.586+0000 7f84ad00df40 1 -- 172.21.15.53:0/2136084959 wait complete. 2024-05-10T10:41:58.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-10T10:41:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.736+0000 7f936dd6a640 1 Processor -- start 2024-05-10T10:41:58.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.736+0000 7f936dd6a640 1 -- start start 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.737+0000 7f936dd6a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 0x7f93680d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.737+0000 7f936dd6a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f93680b4aa0 0x7f93680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.737+0000 7f936dd6a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93680d41d0 con 0x7f93680d1a10 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.737+0000 7f936dd6a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f93680cfd50 con 0x7f93680b4aa0 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.737+0000 7f936dd6a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f93680b5d90 con 0x7f93680d3df0 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.737+0000 7f9367fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f93680d1a10 0x7f93680d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51094/0 (socket says 172.21.15.53:51094) 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.737+0000 7f9367fff640 1 -- 172.21.15.53:0/2938824590 learned_addr learned my addr 172.21.15.53:0/2938824590 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93677fe640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 0x7f93680d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f9366ffd640 1 --2- 172.21.15.53:0/2938824590 >> v2:172.21.15.135:3300/0 conn(0x7f93680b4aa0 0x7f93680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1898471404 0 0) 0x7f93680d41d0 con 0x7f93680d1a10 2024-05-10T10:41:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9350003520 con 0x7f93680d1a10 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3086111758 0 0) 0x7f9350003520 con 0x7f93680d1a10 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 msgr2=0x7f93680d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 0x7f93680d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 >> v2:172.21.15.135:3300/0 conn(0x7f93680b4aa0 msgr2=0x7f93680b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 --2- 172.21.15.53:0/2938824590 >> v2:172.21.15.135:3300/0 conn(0x7f93680b4aa0 0x7f93680b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93680b54f0 con 0x7f93680d1a10 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f93600019f0 con 0x7f93680d1a10 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 >> v1:172.21.15.53:6789/0 conn(0x7f93680d1a10 legacy=0x7f93680d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93500039b0 0x7f9350003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9350004800 0x7f9350004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 0x7f93500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93680b5d90 con 0x7f93500039b0 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f935000b9c0 con 0x7f9350004800 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f935000bb40 con 0x7f93680d3df0 2024-05-10T10:41:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93677fe640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93500039b0 0x7f9350003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f9366ffd640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9350004800 0x7f9350004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f9367fff640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 0x7f93500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93677fe640 1 -- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 msgr2=0x7f93500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93677fe640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93680d3df0 0x7f93500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93677fe640 1 -- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9350004800 msgr2=0x7f9350004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93677fe640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9350004800 0x7f9350004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f93677fe640 1 -- 172.21.15.53:0/2938824590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9350003520 con 0x7f93500039b0 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.738+0000 7f9366ffd640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9350004800 0x7f9350004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.739+0000 7f93677fe640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93500039b0 0x7f9350003dd0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9358009ac0 tx=0x7f9350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15ab5c1af3e5823 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.739+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358036070 con 0x7f93500039b0 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.739+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9358031400 con 0x7f93500039b0 2024-05-10T10:41:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.739+0000 7f93667fc640 1 -- 172.21.15.53:0/2938824590 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358009690 con 0x7f93500039b0 2024-05-10T10:41:58.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.740+0000 7f936dd6a640 1 -- 172.21.15.53:0/2938824590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93500039b0 msgr2=0x7f9350003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.740+0000 7f936dd6a640 1 --2- 172.21.15.53:0/2938824590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93500039b0 0x7f9350003dd0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9358009ac0 tx=0x7f9350005830 comp rx=0 tx=0).stop 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.740+0000 7f9367fff640 1 -- 172.21.15.53:0/2938824590 reap_dead start 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.740+0000 7f936dd6a640 1 -- 172.21.15.53:0/2938824590 shutdown_connections 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.740+0000 7f936dd6a640 1 -- 172.21.15.53:0/2938824590 >> 172.21.15.53:0/2938824590 conn(0x7f936809e450 msgr2=0x7f93680d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.740+0000 7f936dd6a640 1 -- 172.21.15.53:0/2938824590 shutdown_connections 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.740+0000 7f936dd6a640 1 -- 172.21.15.53:0/2938824590 wait complete. 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.741+0000 7f936dd6a640 1 Processor -- start 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.741+0000 7f936dd6a640 1 -- start start 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f936dd6a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93500039b0 0x7f93680a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f936dd6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93680ce340 0x7f93681606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f936dd6a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93680caa10 0x7f93680cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f936dd6a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93680cff80 con 0x7f93680ce340 2024-05-10T10:41:58.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f936dd6a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93680b5ab0 con 0x7f93680caa10 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f936dd6a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f93680b6120 con 0x7f93500039b0 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93680ce340 0x7f93681606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93680ce340 0x7f93681606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47990/0 (socket says 172.21.15.53:47990) 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 -- 172.21.15.53:0/3705651606 learned_addr learned my addr 172.21.15.53:0/3705651606 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f93677fe640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93500039b0 0x7f93680a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9367fff640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93680caa10 0x7f93680cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 -- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93500039b0 msgr2=0x7f93680a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93500039b0 0x7f93680a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 -- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93680caa10 msgr2=0x7f93680cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93680caa10 0x7f93680cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9366ffd640 1 -- 172.21.15.53:0/3705651606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93680ce7c0 con 0x7f93680ce340 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.742+0000 7f9367fff640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93680caa10 0x7f93680cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.743+0000 7f9366ffd640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93680ce340 0x7f93681606e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f935c011700 tx=0x7f935c011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.743+0000 7f934ffff640 1 -- 172.21.15.53:0/3705651606 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c00f9c0 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.743+0000 7f934ffff640 1 -- 172.21.15.53:0/3705651606 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c00fb60 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.743+0000 7f934ffff640 1 -- 172.21.15.53:0/3705651606 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c015460 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.743+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93680cea90 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.743+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93680cf050 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.744+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93680d3560 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.745+0000 7f934ffff640 1 -- 172.21.15.53:0/3705651606 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f935c015600 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.745+0000 7f934ffff640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9354076300 0x7f93540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.745+0000 7f934ffff640 1 -- 172.21.15.53:0/3705651606 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7f935c098010 con 0x7f93680ce340 2024-05-10T10:41:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.745+0000 7f93677fe640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9354076300 0x7f93540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:58.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.746+0000 7f93677fe640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9354076300 0x7f93540787c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9358033000 tx=0x7f9358008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:58.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.750+0000 7f934ffff640 1 -- 172.21.15.53:0/3705651606 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f93680d3560 con 0x7f93680ce340 2024-05-10T10:41:58.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:58.979+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f936806a220 con 0x7f93680ce340 2024-05-10T10:41:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.341+0000 7f934ffff640 1 -- 172.21.15.53:0/3705651606 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v84) ==== 162+0+0 (secure 0 0 0) 0x7f935c019030 con 0x7f93680ce340 2024-05-10T10:41:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-10T10:41:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.343+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9354076300 msgr2=0x7f93540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.343+0000 7f936dd6a640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9354076300 0x7f93540787c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9358033000 tx=0x7f9358008030 comp rx=0 tx=0).stop 2024-05-10T10:41:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.343+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93680ce340 msgr2=0x7f93681606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.343+0000 7f936dd6a640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93680ce340 0x7f93681606e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f935c011700 tx=0x7f935c011bc0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 shutdown_connections 2024-05-10T10:41:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9354076300 0x7f93540787c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93680caa10 0x7f93680cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93680ce340 0x7f93681606e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 --2- 172.21.15.53:0/3705651606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93500039b0 0x7f93680a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 >> 172.21.15.53:0/3705651606 conn(0x7f936809e450 msgr2=0x7f93680b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 shutdown_connections 2024-05-10T10:41:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.344+0000 7f936dd6a640 1 -- 172.21.15.53:0/3705651606 wait complete. 2024-05-10T10:41:59.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.465+0000 7fa68d228640 1 Processor -- start 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.465+0000 7fa68d228640 1 -- start start 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.466+0000 7fa68d228640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa68815f3b0 0x7fa6880d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.466+0000 7fa68d228640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6880d30b0 con 0x7fa6880d5d30 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.466+0000 7fa68d228640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6880d3390 con 0x7fa68815f790 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.466+0000 7fa68d228640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa6880c8c80 con 0x7fa68815f3b0 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.466+0000 7fa68759e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa6880d5d30 0x7fa6880d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51110/0 (socket says 172.21.15.53:51110) 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.466+0000 7fa68759e640 1 -- 172.21.15.53:0/2718441003 learned_addr learned my addr 172.21.15.53:0/2718441003 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa686d9d640 1 --2- 172.21.15.53:0/2718441003 >> v2:172.21.15.135:3300/0 conn(0x7fa68815f3b0 0x7fa6880d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 846326792 0 0) 0x7fa6880d30b0 con 0x7fa6880d5d30 2024-05-10T10:41:59.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa674003520 con 0x7fa6880d5d30 2024-05-10T10:41:59.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3544055913 0 0) 0x7fa674003520 con 0x7fa6880d5d30 2024-05-10T10:41:59.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 >> v2:172.21.15.135:3300/0 conn(0x7fa68815f3b0 msgr2=0x7fa6880d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:59.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 --2- 172.21.15.53:0/2718441003 >> v2:172.21.15.135:3300/0 conn(0x7fa68815f3b0 0x7fa6880d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 >> v1:172.21.15.135:6789/0 conn(0x7fa68815f790 legacy=0x7fa6880c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6880c8f60 con 0x7fa6880d5d30 2024-05-10T10:41:59.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fa680001670 con 0x7fa6880d5d30 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 >> v1:172.21.15.53:6789/0 conn(0x7fa6880d5d30 legacy=0x7fa6880d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6740039b0 0x7fa674003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa674004800 0x7fa674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740051c0 0x7fa67400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880c8c80 con 0x7fa6740039b0 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa67400c060 con 0x7fa6740051c0 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa67400c260 con 0x7fa674004800 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa686d9d640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6740039b0 0x7fa674003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa68659c640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa674004800 0x7fa674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa68759e640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740051c0 0x7fa67400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa686d9d640 1 -- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa674004800 msgr2=0x7fa674004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa686d9d640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa674004800 0x7fa674004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa686d9d640 1 -- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740051c0 msgr2=0x7fa67400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa686d9d640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740051c0 0x7fa67400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa686d9d640 1 -- 172.21.15.53:0/2718441003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa674003520 con 0x7fa6740039b0 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa68759e640 1 -- 172.21.15.53:0/2718441003 reap_dead start 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.467+0000 7fa68659c640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa674004800 0x7fa674004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.468+0000 7fa686d9d640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6740039b0 0x7fa674003dd0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa678009fc0 tx=0x7fa674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a0d230e40e3616 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.468+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678009c10 con 0x7fa6740039b0 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.468+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa678036070 con 0x7fa6740039b0 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.468+0000 7fa685d9b640 1 -- 172.21.15.53:0/2718441003 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678031880 con 0x7fa6740039b0 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.469+0000 7fa68d228640 1 -- 172.21.15.53:0/2718441003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6740039b0 msgr2=0x7fa674003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.469+0000 7fa68d228640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6740039b0 0x7fa674003dd0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa678009fc0 tx=0x7fa674005830 comp rx=0 tx=0).stop 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.469+0000 7fa68d228640 1 -- 172.21.15.53:0/2718441003 shutdown_connections 2024-05-10T10:41:59.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.469+0000 7fa68d228640 1 --2- 172.21.15.53:0/2718441003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6740039b0 0x7fa674003dd0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.469+0000 7fa68d228640 1 -- 172.21.15.53:0/2718441003 >> 172.21.15.53:0/2718441003 conn(0x7fa6880d5130 msgr2=0x7fa6880d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:59.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.469+0000 7fa68d228640 1 -- 172.21.15.53:0/2718441003 shutdown_connections 2024-05-10T10:41:59.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.469+0000 7fa68d228640 1 -- 172.21.15.53:0/2718441003 wait complete. 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.470+0000 7fa68d228640 1 Processor -- start 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.470+0000 7fa68d228640 1 -- start start 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68d228640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740039b0 0x7fa6881475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68d228640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6880cc200 0x7fa6880cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68d228640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa688153cb0 0x7fa688154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68d228640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880d3870 con 0x7fa6880cc200 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68d228640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880d36f0 con 0x7fa6740039b0 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68d228640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880d39f0 con 0x7fa688153cb0 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6880cc200 0x7fa6880cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6880cc200 0x7fa6880cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48016/0 (socket says 172.21.15.53:48016) 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.471+0000 7fa68659c640 1 -- 172.21.15.53:0/3018481608 learned_addr learned my addr 172.21.15.53:0/3018481608 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:41:59.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa686d9d640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740039b0 0x7fa6881475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68759e640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa688153cb0 0x7fa688154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68659c640 1 -- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa688153cb0 msgr2=0x7fa688154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68659c640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa688153cb0 0x7fa688154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68659c640 1 -- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740039b0 msgr2=0x7fa6881475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68659c640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740039b0 0x7fa6881475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68659c640 1 -- 172.21.15.53:0/3018481608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6880b9af0 con 0x7fa6880cc200 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68759e640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa688153cb0 0x7fa688154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa686d9d640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740039b0 0x7fa6881475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68659c640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6880cc200 0x7fa6880cc680 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e4b0 tx=0x7fa67000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa67f7fe640 1 -- 172.21.15.53:0/3018481608 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700052e0 con 0x7fa6880cc200 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa67f7fe640 1 -- 172.21.15.53:0/3018481608 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa670018070 con 0x7fa6880cc200 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6880b9dd0 con 0x7fa6880cc200 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa67f7fe640 1 -- 172.21.15.53:0/3018481608 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670003830 con 0x7fa6880cc200 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.472+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6880ba390 con 0x7fa6880cc200 2024-05-10T10:41:59.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.473+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68806a220 con 0x7fa6880cc200 2024-05-10T10:41:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.474+0000 7fa67f7fe640 1 -- 172.21.15.53:0/3018481608 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa6700039d0 con 0x7fa6880cc200 2024-05-10T10:41:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.474+0000 7fa67f7fe640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:41:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.474+0000 7fa67f7fe640 1 -- 172.21.15.53:0/3018481608 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fa670097430 con 0x7fa6880cc200 2024-05-10T10:41:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.475+0000 7fa686d9d640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:41:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.475+0000 7fa686d9d640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa678035000 tx=0x7fa67802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:41:59.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.479+0000 7fa67f7fe640 1 -- 172.21.15.53:0/3018481608 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa670060ae0 con 0x7fa6880cc200 2024-05-10T10:41:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.708+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa688154670 con 0x7fa6880cc200 2024-05-10T10:41:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.708+0000 7fa67f7fe640 1 -- 172.21.15.53:0/3018481608 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5184 (secure 0 0 0) 0x7fa6700603e0 con 0x7fa6880cc200 2024-05-10T10:41:59.709 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:41:59.710 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:41:59.344396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7749},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7749":{"gid":7749,"name":"c","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.53:6835/1219905896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1219905896},{"type":"v1","addr":"172.21.15.53:6835","nonce":1219905896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-10T10:41:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 85 2024-05-10T10:41:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.710+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 msgr2=0x7fa668078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.710+0000 7fa68d228640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa678035000 tx=0x7fa67802f000 comp rx=0 tx=0).stop 2024-05-10T10:41:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.710+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6880cc200 msgr2=0x7fa6880cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:41:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.710+0000 7fa68d228640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6880cc200 0x7fa6880cc680 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e4b0 tx=0x7fa67000e970 comp rx=0 tx=0).stop 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 shutdown_connections 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa6680762d0 0x7fa668078790 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa688153cb0 0x7fa688154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6880cc200 0x7fa6880cc680 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 --2- 172.21.15.53:0/3018481608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6740039b0 0x7fa6881475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 >> 172.21.15.53:0/3018481608 conn(0x7fa6880d5130 msgr2=0x7fa6880d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 shutdown_connections 2024-05-10T10:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:41:59.711+0000 7fa68d228640 1 -- 172.21.15.53:0/3018481608 wait complete. 2024-05-10T10:42:00.126 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:41:15.140801+0000', 'modified': '2024-05-10T10:41:59.344396+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 106, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7749}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7749': {'gid': 7749, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/1219905896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1219905896}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1219905896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:42:00.126 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7749:up:replay 2024-05-10T10:42:01.128 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:01.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.244+0000 7f0d3b2b0640 1 Processor -- start 2024-05-10T10:42:01.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.244+0000 7f0d3b2b0640 1 -- start start 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3b2b0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415fad0 0x7f0d3406ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3b2b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0d3415feb0 0x7f0d34072850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3b2b0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0d34072d90 0x7f0d3415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3b2b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3406d410 con 0x7f0d3415feb0 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3b2b0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3415cb20 con 0x7f0d34072d90 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3b2b0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3415c840 con 0x7f0d3415fad0 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0d3415feb0 0x7f0d34072850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:01.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0d3415feb0 0x7f0d34072850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48032/0 (socket says 172.21.15.53:48032) 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 -- 172.21.15.53:0/3889170121 learned_addr learned my addr 172.21.15.53:0/3889170121 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3984d640 1 --2- 172.21.15.53:0/3889170121 >> v2:172.21.15.135:3300/0 conn(0x7f0d34072d90 0x7f0d3415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3904c640 1 --2- 172.21.15.53:0/3889170121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415fad0 0x7f0d3406ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 -- 172.21.15.53:0/3889170121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415fad0 msgr2=0x7f0d3406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 --2- 172.21.15.53:0/3889170121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415fad0 0x7f0d3406ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 -- 172.21.15.53:0/3889170121 >> v2:172.21.15.135:3300/0 conn(0x7f0d34072d90 msgr2=0x7f0d3415bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 --2- 172.21.15.53:0/3889170121 >> v2:172.21.15.135:3300/0 conn(0x7f0d34072d90 0x7f0d3415bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3884b640 1 -- 172.21.15.53:0/3889170121 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3406c3f0 con 0x7f0d3415feb0 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3984d640 1 --2- 172.21.15.53:0/3889170121 >> v2:172.21.15.135:3300/0 conn(0x7f0d34072d90 0x7f0d3415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.245+0000 7f0d3904c640 1 --2- 172.21.15.53:0/3889170121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415fad0 0x7f0d3406ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.246+0000 7f0d3884b640 1 --2- 172.21.15.53:0/3889170121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415feb0 0x7f0d34072850 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0d340d0160 tx=0x7f0d2402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98a9e4e51175dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.246+0000 7f0d2bfff640 1 -- 172.21.15.53:0/3889170121 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24037070 con 0x7f0d3415feb0 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.246+0000 7f0d2bfff640 1 -- 172.21.15.53:0/3889170121 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d24032440 con 0x7f0d3415feb0 2024-05-10T10:42:01.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.246+0000 7f0d2bfff640 1 -- 172.21.15.53:0/3889170121 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24032810 con 0x7f0d3415feb0 2024-05-10T10:42:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/3889170121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415feb0 msgr2=0x7f0d34072850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/3889170121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415feb0 0x7f0d34072850 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0d340d0160 tx=0x7f0d2402a140 comp rx=0 tx=0).stop 2024-05-10T10:42:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/3889170121 shutdown_connections 2024-05-10T10:42:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/3889170121 >> v2:172.21.15.135:3300/0 conn(0x7f0d34072d90 0x7f0d3415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/3889170121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415feb0 0x7f0d34072850 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/3889170121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415fad0 0x7f0d3406ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/3889170121 >> 172.21.15.53:0/3889170121 conn(0x7f0d340d1510 msgr2=0x7f0d340d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/3889170121 shutdown_connections 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.247+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/3889170121 wait complete. 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.248+0000 7f0d3b2b0640 1 Processor -- start 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.248+0000 7f0d3b2b0640 1 -- start start 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3b2b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d34072d90 0x7f0d3415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3b2b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415fad0 0x7f0d34147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3b2b0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415cca0 0x7f0d340ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3b2b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3415c0c0 con 0x7f0d3415fad0 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3b2b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3406c9b0 con 0x7f0d34072d90 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3b2b0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3415c240 con 0x7f0d3415cca0 2024-05-10T10:42:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3884b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415fad0 0x7f0d34147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3884b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415fad0 0x7f0d34147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48048/0 (socket says 172.21.15.53:48048) 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3884b640 1 -- 172.21.15.53:0/654249764 learned_addr learned my addr 172.21.15.53:0/654249764 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3904c640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d34072d90 0x7f0d3415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.249+0000 7f0d3984d640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415cca0 0x7f0d340ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3884b640 1 -- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415cca0 msgr2=0x7f0d340ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3884b640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415cca0 0x7f0d340ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3884b640 1 -- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d34072d90 msgr2=0x7f0d3415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3884b640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d34072d90 0x7f0d3415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3884b640 1 -- 172.21.15.53:0/654249764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3415d660 con 0x7f0d3415fad0 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3984d640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415cca0 0x7f0d340ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3884b640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415fad0 0x7f0d34147b50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0d24000c00 tx=0x7f0d240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d29ffb640 1 -- 172.21.15.53:0/654249764 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24037040 con 0x7f0d3415fad0 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d29ffb640 1 -- 172.21.15.53:0/654249764 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d24042070 con 0x7f0d3415fad0 2024-05-10T10:42:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3415d940 con 0x7f0d3415fad0 2024-05-10T10:42:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d29ffb640 1 -- 172.21.15.53:0/654249764 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24030bb0 con 0x7f0d3415fad0 2024-05-10T10:42:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.250+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3415df00 con 0x7f0d3415fad0 2024-05-10T10:42:01.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.251+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3406a220 con 0x7f0d3415fad0 2024-05-10T10:42:01.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.252+0000 7f0d29ffb640 1 -- 172.21.15.53:0/654249764 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d24047460 con 0x7f0d3415fad0 2024-05-10T10:42:01.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.253+0000 7f0d29ffb640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d18076300 0x7f0d180787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:01.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.253+0000 7f0d29ffb640 1 -- 172.21.15.53:0/654249764 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7f0d240b83a0 con 0x7f0d3415fad0 2024-05-10T10:42:01.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.253+0000 7f0d3904c640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d18076300 0x7f0d180787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:01.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.253+0000 7f0d3904c640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d18076300 0x7f0d180787c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000ab60 tx=0x7f0d30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:01.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.257+0000 7f0d29ffb640 1 -- 172.21.15.53:0/654249764 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0d24081ac0 con 0x7f0d3415fad0 2024-05-10T10:42:01.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.488+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d3406da70 con 0x7f0d3415fad0 2024-05-10T10:42:01.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.488+0000 7f0d29ffb640 1 -- 172.21.15.53:0/654249764 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5184 (secure 0 0 0) 0x7f0d240813c0 con 0x7f0d3415fad0 2024-05-10T10:42:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:01.489 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:42:00.349650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7749},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7749":{"gid":7749,"name":"c","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.53:6835/1219905896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1219905896},{"type":"v1","addr":"172.21.15.53:6835","nonce":1219905896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-10T10:42:01.489 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 87 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d18076300 msgr2=0x7f0d180787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d18076300 0x7f0d180787c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000ab60 tx=0x7f0d30008040 comp rx=0 tx=0).stop 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415fad0 msgr2=0x7f0d34147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415fad0 0x7f0d34147b50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0d24000c00 tx=0x7f0d240046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 shutdown_connections 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0d18076300 0x7f0d180787c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0d3415cca0 0x7f0d340ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d3415fad0 0x7f0d34147b50 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 --2- 172.21.15.53:0/654249764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d34072d90 0x7f0d3415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 >> 172.21.15.53:0/654249764 conn(0x7f0d340d1510 msgr2=0x7f0d3406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 shutdown_connections 2024-05-10T10:42:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:01.490+0000 7f0d3b2b0640 1 -- 172.21.15.53:0/654249764 wait complete. 2024-05-10T10:42:01.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:41:15.140801+0000', 'modified': '2024-05-10T10:42:00.349650+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 106, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7749}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7749': {'gid': 7749, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.53:6835/1219905896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1219905896}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1219905896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:42:01.905 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7749:up:rejoin 2024-05-10T10:42:02.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.021+0000 7fecd58a9640 1 Processor -- start 2024-05-10T10:42:03.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.021+0000 7fecd58a9640 1 -- start start 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7fecd58a9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fecd015d650 0x7fecd00cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7fecd58a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00ce590 con 0x7fecd00b43a0 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7fecd58a9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00d0500 con 0x7fecd00d3980 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7fecd58a9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fecd015da30 con 0x7fecd015d650 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7feccf7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fecd00b43a0 0x7fecd00b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54530/0 (socket says 172.21.15.53:54530) 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7feccf7fe640 1 -- 172.21.15.53:0/2154618487 learned_addr learned my addr 172.21.15.53:0/2154618487 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7fecceffd640 1 --2- 172.21.15.53:0/2154618487 >> v2:172.21.15.135:3300/0 conn(0x7fecd015d650 0x7fecd00cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468417474 0 0) 0x7fecc8001490 con 0x7fecd00b43a0 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.022+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecb8003520 con 0x7fecd00b43a0 2024-05-10T10:42:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 978218301 0 0) 0x7fecb8003520 con 0x7fecd00b43a0 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 >> v2:172.21.15.135:3300/0 conn(0x7fecd015d650 msgr2=0x7fecd00cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 --2- 172.21.15.53:0/2154618487 >> v2:172.21.15.135:3300/0 conn(0x7fecd015d650 0x7fecd00cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 >> v1:172.21.15.135:6789/0 conn(0x7fecd00d3980 legacy=0x7fecd00cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00ce810 con 0x7fecd00b43a0 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fecc8001bc0 con 0x7fecd00b43a0 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 >> v1:172.21.15.53:6789/0 conn(0x7fecd00b43a0 legacy=0x7fecd00b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80039b0 0x7fecb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80051c0 0x7fecb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fecb800c030 con 0x7fecb8004800 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecb800c1e0 con 0x7fecb80039b0 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fecb800c3e0 con 0x7fecb80051c0 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7fecce7fc640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7fecceffd640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80039b0 0x7fecb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccf7fe640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80051c0 0x7fecb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7fecce7fc640 1 -- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80051c0 msgr2=0x7fecb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7fecce7fc640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80051c0 0x7fecb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7fecce7fc640 1 -- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80039b0 msgr2=0x7fecb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7fecce7fc640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb80039b0 0x7fecb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7fecce7fc640 1 -- 172.21.15.53:0/2154618487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb8003520 con 0x7fecb8004800 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccf7fe640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecb80051c0 0x7fecb800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.023+0000 7feccf7fe640 1 -- 172.21.15.53:0/2154618487 reap_dead start 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.024+0000 7fecce7fc640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecb8004c80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001750 tx=0x7fecb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7da7b0df22285d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.024+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4001370 con 0x7fecb8004800 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.024+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc40052d0 con 0x7fecb8004800 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.024+0000 7feccdffb640 1 -- 172.21.15.53:0/2154618487 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc402abc0 con 0x7fecb8004800 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.025+0000 7fecd58a9640 1 -- 172.21.15.53:0/2154618487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 msgr2=0x7fecb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.025+0000 7fecd58a9640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecb8004c80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001750 tx=0x7fecb8005830 comp rx=0 tx=0).stop 2024-05-10T10:42:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.025+0000 7fecd58a9640 1 -- 172.21.15.53:0/2154618487 shutdown_connections 2024-05-10T10:42:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.025+0000 7fecd58a9640 1 --2- 172.21.15.53:0/2154618487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecb8004c80 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.025+0000 7fecd58a9640 1 -- 172.21.15.53:0/2154618487 >> 172.21.15.53:0/2154618487 conn(0x7fecd00d2df0 msgr2=0x7fecd00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.025+0000 7fecd58a9640 1 -- 172.21.15.53:0/2154618487 shutdown_connections 2024-05-10T10:42:03.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.025+0000 7fecd58a9640 1 -- 172.21.15.53:0/2154618487 wait complete. 2024-05-10T10:42:03.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.026+0000 7fecd58a9640 1 Processor -- start 2024-05-10T10:42:03.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.026+0000 7fecd58a9640 1 -- start start 2024-05-10T10:42:03.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecd58a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecd01f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecd58a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00c7790 0x7fecd00c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecd58a9640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd00c7c10 0x7fecd00c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecd58a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fecd00cf540 con 0x7fecb8004800 2024-05-10T10:42:03.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecd58a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecd00cedd0 con 0x7fecd00c7790 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecd58a9640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fecd00cf6c0 con 0x7fecd00c7c10 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecceffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecd01f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecceffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecd01f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58534/0 (socket says 172.21.15.53:58534) 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecceffd640 1 -- 172.21.15.53:0/3742914070 learned_addr learned my addr 172.21.15.53:0/3742914070 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecce7fc640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00c7790 0x7fecd00c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7feccf7fe640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd00c7c10 0x7fecd00c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecceffd640 1 -- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd00c7c10 msgr2=0x7fecd00c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecceffd640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd00c7c10 0x7fecd00c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecceffd640 1 -- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00c7790 msgr2=0x7fecd00c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.027+0000 7fecceffd640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00c7790 0x7fecd00c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.028+0000 7fecceffd640 1 -- 172.21.15.53:0/3742914070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00c66f0 con 0x7fecb8004800 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.028+0000 7fecceffd640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecd01f02a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fecc0011ad0 tx=0x7fecc0011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.028+0000 7fecb7fff640 1 -- 172.21.15.53:0/3742914070 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc000fb50 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.028+0000 7fecb7fff640 1 -- 172.21.15.53:0/3742914070 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc000fcf0 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.028+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd00c69d0 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.028+0000 7fecb7fff640 1 -- 172.21.15.53:0/3742914070 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc000fea0 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.028+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd00c6f90 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.029+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd006a220 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.030+0000 7fecb7fff640 1 -- 172.21.15.53:0/3742914070 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fecc0012040 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.030+0000 7fecb7fff640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecbc0762d0 0x7fecbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.030+0000 7fecb7fff640 1 -- 172.21.15.53:0/3742914070 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fecc0097d70 con 0x7fecb8004800 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.030+0000 7fecce7fc640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecbc0762d0 0x7fecbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.031+0000 7fecce7fc640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecbc0762d0 0x7fecbc078790 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fecc4032000 tx=0x7fecc402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.035+0000 7fecb7fff640 1 -- 172.21.15.53:0/3742914070 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fecc0019030 con 0x7fecb8004800 2024-05-10T10:42:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.266+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecd00712c0 con 0x7fecb8004800 2024-05-10T10:42:03.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.267+0000 7fecb7fff640 1 -- 172.21.15.53:0/3742914070 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5191 (secure 0 0 0) 0x7fecc0060ef0 con 0x7fecb8004800 2024-05-10T10:42:03.269 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:03.269 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:42:02.348221+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7749},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7749":{"gid":7749,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6835/1219905896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1219905896},{"type":"v1","addr":"172.21.15.53:6835","nonce":1219905896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7749,"qdb_cluster":[7749]},"id":7}]} 2024-05-10T10:42:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 88 2024-05-10T10:42:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecbc0762d0 msgr2=0x7fecbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecbc0762d0 0x7fecbc078790 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fecc4032000 tx=0x7fecc402e000 comp rx=0 tx=0).stop 2024-05-10T10:42:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 msgr2=0x7fecd01f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecd01f02a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fecc0011ad0 tx=0x7fecc0011f90 comp rx=0 tx=0).stop 2024-05-10T10:42:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 shutdown_connections 2024-05-10T10:42:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fecbc0762d0 0x7fecbc078790 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fecd00c7c10 0x7fecd00c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecd00c7790 0x7fecd00c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 --2- 172.21.15.53:0/3742914070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecb8004800 0x7fecd01f02a0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 >> 172.21.15.53:0/3742914070 conn(0x7fecd00d2df0 msgr2=0x7fecd00716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 shutdown_connections 2024-05-10T10:42:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.269+0000 7fecd58a9640 1 -- 172.21.15.53:0/3742914070 wait complete. 2024-05-10T10:42:03.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:41:15.140801+0000', 'modified': '2024-05-10T10:42:02.348221+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 106, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7749}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7749': {'gid': 7749, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6835/1219905896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1219905896}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1219905896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7749, 'qdb_cluster': [7749]} 2024-05-10T10:42:03.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:42:03.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T10:42:03.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.797+0000 7ff41dd5f640 1 Processor -- start 2024-05-10T10:42:03.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.797+0000 7ff41dd5f640 1 -- start start 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.797+0000 7ff41dd5f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff418162940 0x7ff4180b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.797+0000 7ff41dd5f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4180d2d20 con 0x7ff4180d3d80 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.797+0000 7ff41dd5f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4180d3000 con 0x7ff4180d59d0 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.797+0000 7ff41dd5f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff418162d20 con 0x7ff418162940 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff417fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff4180d3d80 0x7ff4180d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54542/0 (socket says 172.21.15.53:54542) 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff417fff640 1 -- 172.21.15.53:0/3015044654 learned_addr learned my addr 172.21.15.53:0/3015044654 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3817733255 0 0) 0x7ff4180d2d20 con 0x7ff4180d3d80 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff400003520 con 0x7ff4180d3d80 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4177fe640 1 --2- 172.21.15.53:0/3015044654 >> v2:172.21.15.135:3300/0 conn(0x7ff418162940 0x7ff4180b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2593122537 0 0) 0x7ff400003520 con 0x7ff4180d3d80 2024-05-10T10:42:03.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 >> v2:172.21.15.135:3300/0 conn(0x7ff418162940 msgr2=0x7ff4180b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 --2- 172.21.15.53:0/3015044654 >> v2:172.21.15.135:3300/0 conn(0x7ff418162940 0x7ff4180b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 >> v1:172.21.15.135:6789/0 conn(0x7ff4180d59d0 legacy=0x7ff4180d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4180d32e0 con 0x7ff4180d3d80 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7ff4100019f0 con 0x7ff4180d3d80 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 >> v1:172.21.15.53:6789/0 conn(0x7ff4180d3d80 legacy=0x7ff4180d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4000039b0 0x7ff400003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff400004800 0x7ff400004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4000051c0 0x7ff40000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000c130 con 0x7ff400004800 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000c2e0 con 0x7ff4000039b0 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff40000c4e0 con 0x7ff4000051c0 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff416ffd640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff400004800 0x7ff400004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.798+0000 7ff4177fe640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4000039b0 0x7ff400003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff417fff640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4000051c0 0x7ff40000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff416ffd640 1 -- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4000051c0 msgr2=0x7ff40000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff416ffd640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4000051c0 0x7ff40000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff416ffd640 1 -- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4000039b0 msgr2=0x7ff400003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff416ffd640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4000039b0 0x7ff400003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff416ffd640 1 -- 172.21.15.53:0/3015044654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400003520 con 0x7ff400004800 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff417fff640 1 -- 172.21.15.53:0/3015044654 reap_dead start 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.799+0000 7ff416ffd640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff400004800 0x7ff400004c80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff40c001750 tx=0x7ff400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119ea027c3cf174a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c001370 con 0x7ff400004800 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c0052d0 con 0x7ff400004800 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff4167fc640 1 -- 172.21.15.53:0/3015044654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c02abc0 con 0x7ff400004800 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff41dd5f640 1 -- 172.21.15.53:0/3015044654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff400004800 msgr2=0x7ff400004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff400004800 0x7ff400004c80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff40c001750 tx=0x7ff400005830 comp rx=0 tx=0).stop 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff41dd5f640 1 -- 172.21.15.53:0/3015044654 shutdown_connections 2024-05-10T10:42:03.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/3015044654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff400004800 0x7ff400004c80 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff41dd5f640 1 -- 172.21.15.53:0/3015044654 >> 172.21.15.53:0/3015044654 conn(0x7ff41809e380 msgr2=0x7ff4180d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff41dd5f640 1 -- 172.21.15.53:0/3015044654 shutdown_connections 2024-05-10T10:42:03.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.800+0000 7ff41dd5f640 1 -- 172.21.15.53:0/3015044654 wait complete. 2024-05-10T10:42:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.801+0000 7ff41dd5f640 1 Processor -- start 2024-05-10T10:42:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff41dd5f640 1 -- start start 2024-05-10T10:42:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff41dd5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff400004800 0x7ff41814ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff41dd5f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4180cf370 0x7ff4180b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff41dd5f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4180cc180 0x7ff4180b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff41dd5f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4180d2390 con 0x7ff4180cf370 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff41dd5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4180cfc50 con 0x7ff400004800 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff41dd5f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff4180d2510 con 0x7ff4180cc180 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff416ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4180cf370 0x7ff4180b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff416ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4180cf370 0x7ff4180b96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58554/0 (socket says 172.21.15.53:58554) 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.802+0000 7ff416ffd640 1 -- 172.21.15.53:0/1994679938 learned_addr learned my addr 172.21.15.53:0/1994679938 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff416ffd640 1 -- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4180cc180 msgr2=0x7ff4180b9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff417fff640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4180cc180 0x7ff4180b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff400004800 0x7ff41814ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff416ffd640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4180cc180 0x7ff4180b9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff416ffd640 1 -- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff400004800 msgr2=0x7ff41814ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff416ffd640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff400004800 0x7ff41814ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff416ffd640 1 -- 172.21.15.53:0/1994679938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41814bfd0 con 0x7ff4180cf370 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff416ffd640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4180cf370 0x7ff4180b96f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff40c000c00 tx=0x7ff40c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff3fffff640 1 -- 172.21.15.53:0/1994679938 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c03f020 con 0x7ff4180cf370 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff3fffff640 1 -- 172.21.15.53:0/1994679938 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c005480 con 0x7ff4180cf370 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff3fffff640 1 -- 172.21.15.53:0/1994679938 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c007a30 con 0x7ff4180cf370 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4180c8ff0 con 0x7ff4180cf370 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.803+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4180c9290 con 0x7ff4180cf370 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.805+0000 7ff3fffff640 1 -- 172.21.15.53:0/1994679938 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff40c042020 con 0x7ff4180cf370 2024-05-10T10:42:03.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.806+0000 7ff3fffff640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff404076300 0x7ff4040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.806+0000 7ff3fffff640 1 -- 172.21.15.53:0/1994679938 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7ff40c0b73f0 con 0x7ff4180cf370 2024-05-10T10:42:03.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.806+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff404076300 0x7ff4040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.806+0000 7ff420a76740 1 Processor -- start 2024-05-10T10:42:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.806+0000 7ff420a76740 1 -- start start 2024-05-10T10:42:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.806+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff404076300 0x7ff4040787c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff408009a00 tx=0x7ff408008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469bf50 0x560dd469e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff420a76740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd466ebb0 0x560dd46723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff420a76740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560dd45d88d0 con 0x560dd466ebb0 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560dd4591c10 con 0x560dd469c330 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560dd45d4890 con 0x560dd469bf50 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd466ebb0 0x560dd46723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd466ebb0 0x560dd46723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58558/0 (socket says 172.21.15.53:58558) 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 -- 172.21.15.53:0/501671320 learned_addr learned my addr 172.21.15.53:0/501671320 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff4177fe640 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469bf50 0x560dd469e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff416ffd640 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 -- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469bf50 msgr2=0x560dd469e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469bf50 0x560dd469e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 -- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 msgr2=0x560dd469eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.807+0000 7ff417fff640 1 -- 172.21.15.53:0/501671320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd45995d0 con 0x560dd466ebb0 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff417fff640 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd466ebb0 0x560dd46723c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff4180cc110 tx=0x7ff410004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235d4d07f76403e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff3fdffb640 1 -- 172.21.15.53:0/501671320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41001a070 con 0x560dd466ebb0 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff3fdffb640 1 -- 172.21.15.53:0/501671320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410001250 con 0x560dd466ebb0 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff3fdffb640 1 -- 172.21.15.53:0/501671320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410007790 con 0x560dd466ebb0 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff420a76740 1 -- 172.21.15.53:0/501671320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd466ebb0 msgr2=0x560dd46723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff420a76740 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd466ebb0 0x560dd46723c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff4180cc110 tx=0x7ff410004f30 comp rx=0 tx=0).stop 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff420a76740 1 -- 172.21.15.53:0/501671320 shutdown_connections 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff420a76740 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd466ebb0 0x560dd46723c0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff420a76740 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff420a76740 1 --2- 172.21.15.53:0/501671320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469bf50 0x560dd469e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.808+0000 7ff420a76740 1 -- 172.21.15.53:0/501671320 >> 172.21.15.53:0/501671320 conn(0x560dd4670e20 msgr2=0x560dd46b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.809+0000 7ff420a76740 1 -- 172.21.15.53:0/501671320 shutdown_connections 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.809+0000 7ff420a76740 1 -- 172.21.15.53:0/501671320 wait complete. 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.809+0000 7ff420a76740 1 -- start start 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.809+0000 7ff420a76740 1 Processor -- start 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff420a76740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd4696920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd46ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd46b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff420a76740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560dd4673fd0 con 0x560dd469bf50 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560dd4595760 con 0x560dd469c330 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560dd46742b0 con 0x560dd46b5a20 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff420a76740 10 client.0 _ll_register_callbacks cb 0x560dd45cecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd4696920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd4696920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58562/0 (socket says 172.21.15.53:58562) 2024-05-10T10:42:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 -- 172.21.15.53:0/1827456371 learned_addr learned my addr 172.21.15.53:0/1827456371 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff417fff640 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd46b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff416ffd640 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd46ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 -- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 msgr2=0x560dd46b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd46b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 -- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 msgr2=0x560dd46ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd46ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.810+0000 7ff4177fe640 1 -- 172.21.15.53:0/1827456371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd42ea2d0 con 0x560dd469bf50 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd4696920 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff408016720 tx=0x7ff408004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41a3afebbc8f4c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff3f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1827456371 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408015600 con 0x560dd469bf50 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1827456371 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7ff4080157a0 con 0x560dd469bf50 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff410001250 con 0x560dd469bf50 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1827456371 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff408034050 con 0x560dd469bf50 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 10 client.7821 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x560dd43b46b0 con 0x560dd469bf50 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1827456371 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410001250 con 0x560dd469bf50 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1827456371 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7ff4080159a0 con 0x560dd469bf50 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 10 client.7821 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 10 client.7821 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 20 client.7821 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd46bcdd0 0x560dd46bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 4 client.7821 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd455c760 con 0x560dd46bcdd0 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.811+0000 7ff416ffd640 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd46bcdd0 0x560dd46bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.812+0000 7ff416ffd640 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd46bcdd0 0x560dd46bd1b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3003f99ee9e33983 server_cookie=6354bdb11392164c in_seq=0 out_seq=0 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.812+0000 7ff3f67fc640 10 client.7821 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.812+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1827456371 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560dd455c760 con 0x560dd46bcdd0 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.812+0000 7ff3f67fc640 10 client.7821 handle_command_reply: tid=0 2024-05-10T10:42:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.812+0000 7ff420a76740 1 client.7821 shutdown 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.812+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 msgr2=0x560dd4696920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd4696920 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff408016720 tx=0x7ff408004730 comp rx=0 tx=0).stop 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 shutdown_connections 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd46b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd46bcdd0 0x560dd46bd1b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd46ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 --2- 172.21.15.53:0/1827456371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd4696920 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 >> 172.21.15.53:0/1827456371 conn(0x560dd466ebb0 msgr2=0x560dd46b7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff3f67fc640 0 client.7821 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 shutdown_connections 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 1 -- 172.21.15.53:0/1827456371 wait complete. 2024-05-10T10:42:03.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.813+0000 7ff420a76740 20 client.7821 trim_cache size 0 max 16384 2024-05-10T10:42:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.831+0000 7ff420a76740 1 Processor -- start 2024-05-10T10:42:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.831+0000 7ff420a76740 1 -- start start 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff420a76740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd466de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd469ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff420a76740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4182026c0 con 0x560dd469bf50 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff418201700 con 0x560dd469c330 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff418200ac0 con 0x560dd46b5a20 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff4177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd466de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff4177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd466de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58572/0 (socket says 172.21.15.53:58572) 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff4177fe640 1 -- 172.21.15.53:0/3688483393 learned_addr learned my addr 172.21.15.53:0/3688483393 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff417fff640 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd469ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.832+0000 7ff416ffd640 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff4177fe640 1 -- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 msgr2=0x560dd469ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff4177fe640 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd469ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff4177fe640 1 -- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 msgr2=0x560dd469a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff4177fe640 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff4177fe640 1 -- 172.21.15.53:0/3688483393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd46b1c60 con 0x560dd469bf50 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff417fff640 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd469ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff4177fe640 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd466de60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff408013a10 tx=0x7ff408025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b489bcb6ab6788d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff3fd7fa640 1 -- 172.21.15.53:0/3688483393 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40800f040 con 0x560dd469bf50 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff3fd7fa640 1 -- 172.21.15.53:0/3688483393 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40803d020 con 0x560dd469bf50 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff3fd7fa640 1 -- 172.21.15.53:0/3688483393 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408015800 con 0x560dd469bf50 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff420a76740 1 -- 172.21.15.53:0/3688483393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 msgr2=0x560dd466de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.833+0000 7ff420a76740 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd466de60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff408013a10 tx=0x7ff408025000 comp rx=0 tx=0).stop 2024-05-10T10:42:03.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 -- 172.21.15.53:0/3688483393 shutdown_connections 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd46b5a20 0x560dd469ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469c330 0x560dd469a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 --2- 172.21.15.53:0/3688483393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd469bf50 0x560dd466de60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 -- 172.21.15.53:0/3688483393 >> 172.21.15.53:0/3688483393 conn(0x560dd466ebb0 msgr2=0x560dd46a24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 -- 172.21.15.53:0/3688483393 shutdown_connections 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 -- 172.21.15.53:0/3688483393 wait complete. 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 -- start start 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.834+0000 7ff420a76740 1 Processor -- start 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469bf50 0x560dd4736ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff420a76740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469c330 0x560dd469b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff420a76740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd46b5a20 0x560dd469a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff420a76740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560dd46b1980 con 0x560dd46b5a20 2024-05-10T10:42:03.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4182010b0 con 0x560dd469bf50 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff420a76740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x560dd46b0de0 con 0x560dd469c330 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff420a76740 10 client.0 _ll_register_callbacks cb 0x7ff408013f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd46b5a20 0x560dd469a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd46b5a20 0x560dd469a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58574/0 (socket says 172.21.15.53:58574) 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 -- 172.21.15.53:0/1098503397 learned_addr learned my addr 172.21.15.53:0/1098503397 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff416ffd640 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469c330 0x560dd469b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469bf50 0x560dd4736ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 -- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469c330 msgr2=0x560dd469b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469c330 0x560dd469b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 -- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469bf50 msgr2=0x560dd4736ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469bf50 0x560dd4736ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.835+0000 7ff417fff640 1 -- 172.21.15.53:0/1098503397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd46b6c50 con 0x560dd46b5a20 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff417fff640 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd46b5a20 0x560dd469a650 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff410004f70 tx=0x7ff410001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67226ae86cd31dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff3f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1098503397 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41001a040 con 0x560dd46b5a20 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1098503397 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7ff410021810 con 0x560dd46b5a20 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff40803d020 con 0x560dd46b5a20 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1098503397 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff41002f050 con 0x560dd46b5a20 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 10 client.7827 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x560dd46b4070 con 0x560dd46b5a20 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1098503397 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40803d020 con 0x560dd46b5a20 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1098503397 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7ff410032020 con 0x560dd46b5a20 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 10 client.7827 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 10 client.7827 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 20 client.7827 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd469ace0 0x560dd46dc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 4 client.7827 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "status"}) -- 0x560dd46baf90 con 0x560dd469ace0 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.836+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd469ace0 0x560dd46dc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff4177fe640 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd469ace0 0x560dd46dc1a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea5ec1e43cd5130 server_cookie=58a7feb597e7e280 in_seq=0 out_seq=0 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff3f67fc640 10 client.7827 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff3f67fc640 1 -- 172.21.15.53:0/1098503397 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x560dd46baf90 con 0x560dd469ace0 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff3f67fc640 10 client.7827 handle_command_reply: tid=0 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 client.7827 shutdown 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd46b5a20 msgr2=0x560dd469a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd46b5a20 0x560dd469a650 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff410004f70 tx=0x7ff410001dc0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 shutdown_connections 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dd46b5a20 0x560dd469a650 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560dd469c330 0x560dd469b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x560dd469ace0 0x560dd46dc1a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 --2- 172.21.15.53:0/1098503397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dd469bf50 0x560dd4736ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff3f67fc640 0 client.7827 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 >> 172.21.15.53:0/1098503397 conn(0x560dd466ebb0 msgr2=0x560dd4675ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 shutdown_connections 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 1 -- 172.21.15.53:0/1098503397 wait complete. 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.837+0000 7ff420a76740 20 client.7827 trim_cache size 0 max 16384 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 7749, 2024-05-10T10:42:03.846 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 4.4895282730000003, 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 88, 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 106, 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 106, 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 5.4522848069999998 2024-05-10T10:42:03.847 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:03.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff404076300 msgr2=0x7ff4040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff404076300 0x7ff4040787c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff408009a00 tx=0x7ff408008040 comp rx=0 tx=0).stop 2024-05-10T10:42:03.848 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4180cf370 msgr2=0x7ff4180b96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4180cf370 0x7ff4180b96f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff40c000c00 tx=0x7ff40c001120 comp rx=0 tx=0).stop 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 shutdown_connections 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff404076300 0x7ff4040787c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4180cc180 0x7ff4180b9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4180cf370 0x7ff4180b96f0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 --2- 172.21.15.53:0/1994679938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff400004800 0x7ff41814ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 >> 172.21.15.53:0/1994679938 conn(0x7ff41809e380 msgr2=0x7ff41815da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 shutdown_connections 2024-05-10T10:42:03.849 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.838+0000 7ff41dd5f640 1 -- 172.21.15.53:0/1994679938 wait complete. 2024-05-10T10:42:03.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-10T10:42:03.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:03.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-10T10:42:03.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.959+0000 7fe88a072640 1 Processor -- start 2024-05-10T10:42:03.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.959+0000 7fe88a072640 1 -- start start 2024-05-10T10:42:03.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe88a072640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8840b4210 0x7fe88406e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe88a072640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 0x7fe8840cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe88a072640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe884162ff0 0x7fe884163460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe88a072640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe8840d3c70 con 0x7fe8840b4210 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe88a072640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe8840d0120 con 0x7fe884162ff0 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe88a072640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe8840d1580 con 0x7fe8840cf780 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8840b4210 0x7fe88406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8840b4210 0x7fe88406e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58584/0 (socket says 172.21.15.53:58584) 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 -- 172.21.15.53:0/2809577579 learned_addr learned my addr 172.21.15.53:0/2809577579 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe882ffd640 1 --2- 172.21.15.53:0/2809577579 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 0x7fe8840cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe883fff640 1 --2- 172.21.15.53:0/2809577579 >> v2:172.21.15.135:3300/0 conn(0x7fe884162ff0 0x7fe884163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 -- 172.21.15.53:0/2809577579 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 msgr2=0x7fe8840cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 --2- 172.21.15.53:0/2809577579 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 0x7fe8840cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 -- 172.21.15.53:0/2809577579 >> v2:172.21.15.135:3300/0 conn(0x7fe884162ff0 msgr2=0x7fe884163460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 --2- 172.21.15.53:0/2809577579 >> v2:172.21.15.135:3300/0 conn(0x7fe884162ff0 0x7fe884163460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe8837fe640 1 -- 172.21.15.53:0/2809577579 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8840d1860 con 0x7fe8840b4210 2024-05-10T10:42:03.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.960+0000 7fe883fff640 1 --2- 172.21.15.53:0/2809577579 >> v2:172.21.15.135:3300/0 conn(0x7fe884162ff0 0x7fe884163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:03.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.961+0000 7fe8837fe640 1 --2- 172.21.15.53:0/2809577579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe88406e130 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe8840735c0 tx=0x7fe87402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670a44c00a2aa3a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.961+0000 7fe8827fc640 1 -- 172.21.15.53:0/2809577579 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874037070 con 0x7fe8840b4210 2024-05-10T10:42:03.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.961+0000 7fe8827fc640 1 -- 172.21.15.53:0/2809577579 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe874032440 con 0x7fe8840b4210 2024-05-10T10:42:03.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.961+0000 7fe8827fc640 1 -- 172.21.15.53:0/2809577579 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874032810 con 0x7fe8840b4210 2024-05-10T10:42:03.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 -- 172.21.15.53:0/2809577579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 msgr2=0x7fe88406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 --2- 172.21.15.53:0/2809577579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe88406e130 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe8840735c0 tx=0x7fe87402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 -- 172.21.15.53:0/2809577579 shutdown_connections 2024-05-10T10:42:03.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 --2- 172.21.15.53:0/2809577579 >> v2:172.21.15.135:3300/0 conn(0x7fe884162ff0 0x7fe884163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 --2- 172.21.15.53:0/2809577579 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 0x7fe8840cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 --2- 172.21.15.53:0/2809577579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe88406e130 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 -- 172.21.15.53:0/2809577579 >> 172.21.15.53:0/2809577579 conn(0x7fe8840d2e00 msgr2=0x7fe8840d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 -- 172.21.15.53:0/2809577579 shutdown_connections 2024-05-10T10:42:03.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.962+0000 7fe88a072640 1 -- 172.21.15.53:0/2809577579 wait complete. 2024-05-10T10:42:03.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.963+0000 7fe88a072640 1 Processor -- start 2024-05-10T10:42:03.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.963+0000 7fe88a072640 1 -- start start 2024-05-10T10:42:03.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe88a072640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe884154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe88a072640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 0x7fe8840c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe88a072640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe884162ff0 0x7fe884161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe88a072640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8840d2100 con 0x7fe8840b4210 2024-05-10T10:42:03.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe88a072640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8840d1e20 con 0x7fe884162ff0 2024-05-10T10:42:03.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe88a072640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe8840b45f0 con 0x7fe8840cf780 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe884154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe884154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58590/0 (socket says 172.21.15.53:58590) 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.964+0000 7fe8837fe640 1 -- 172.21.15.53:0/598298595 learned_addr learned my addr 172.21.15.53:0/598298595 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe883fff640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe884162ff0 0x7fe884161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe8837fe640 1 -- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 msgr2=0x7fe8840c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe8837fe640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 0x7fe8840c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe8837fe640 1 -- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe884162ff0 msgr2=0x7fe884161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe8837fe640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe884162ff0 0x7fe884161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe8837fe640 1 -- 172.21.15.53:0/598298595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8840c6a00 con 0x7fe8840b4210 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe883fff640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe884162ff0 0x7fe884161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe8837fe640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe884154e20 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe87402a220 tx=0x7fe8740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe86bfff640 1 -- 172.21.15.53:0/598298595 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874041020 con 0x7fe8840b4210 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe86bfff640 1 -- 172.21.15.53:0/598298595 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87403d620 con 0x7fe8840b4210 2024-05-10T10:42:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8840c6ca0 con 0x7fe8840b4210 2024-05-10T10:42:03.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe86bfff640 1 -- 172.21.15.53:0/598298595 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874037040 con 0x7fe8840b4210 2024-05-10T10:42:03.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.965+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8840c7260 con 0x7fe8840b4210 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.967+0000 7fe86bfff640 1 -- 172.21.15.53:0/598298595 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe874044020 con 0x7fe8840b4210 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.968+0000 7fe86bfff640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8700762d0 0x7fe870078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.968+0000 7fe86bfff640 1 -- 172.21.15.53:0/598298595 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fe8740b7fb0 con 0x7fe8840b4210 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.968+0000 7fe882ffd640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8700762d0 0x7fe870078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.968+0000 7fe88cd8f740 1 Processor -- start 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.968+0000 7fe88cd8f740 1 -- start start 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe882ffd640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8700762d0 0x7fe870078790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe884160de0 tx=0x7fe878009460 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a2359d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d4c00 0x55a2359a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a23590dcf0 con 0x55a2359d5340 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a2358c6e10 con 0x55a2359f2220 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a235909cb0 con 0x55a2359d4c00 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a2359d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a2359d46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58600/0 (socket says 172.21.15.53:58600) 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe882ffd640 1 -- 172.21.15.53:0/3382750754 learned_addr learned my addr 172.21.15.53:0/3382750754 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe8837fe640 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.969+0000 7fe883fff640 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d4c00 0x55a2359a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe882ffd640 1 -- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d4c00 msgr2=0x55a2359a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe882ffd640 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d4c00 0x55a2359a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe882ffd640 1 -- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 msgr2=0x55a2359a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe882ffd640 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe882ffd640 1 -- 172.21.15.53:0/3382750754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2356e77e0 con 0x55a2359d5340 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe8837fe640 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe882ffd640 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a2359d46c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe87800ea90 tx=0x7fe878011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1500e5af07cb48c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe8697fa640 1 -- 172.21.15.53:0/3382750754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87801b040 con 0x55a2359d5340 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe8697fa640 1 -- 172.21.15.53:0/3382750754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878024070 con 0x55a2359d5340 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.970+0000 7fe8697fa640 1 -- 172.21.15.53:0/3382750754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878015c00 con 0x55a2359d5340 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3382750754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 msgr2=0x55a2359d46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a2359d46c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe87800ea90 tx=0x7fe878011000 comp rx=0 tx=0).stop 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3382750754 shutdown_connections 2024-05-10T10:42:03.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d4c00 0x55a2359a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a2359d46c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3382750754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3382750754 >> 172.21.15.53:0/3382750754 conn(0x55a2359a6e20 msgr2=0x55a2359ef6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3382750754 shutdown_connections 2024-05-10T10:42:03.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.971+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3382750754 wait complete. 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.972+0000 7fe88cd8f740 1 -- start start 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.972+0000 7fe88cd8f740 1 Processor -- start 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.973+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359d5340 0x55a2359d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.973+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359f2220 0x55a2359d79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.973+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.973+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a2359a8fd0 con 0x55a2359f2220 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.973+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a2358ca940 con 0x55a2359d5340 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.973+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a2359a83c0 con 0x55a2359d1400 2024-05-10T10:42:03.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe88cd8f740 10 client.0 _ll_register_callbacks cb 0x55a2358cd440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359f2220 0x55a2359d79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359f2220 0x55a2359d79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58610/0 (socket says 172.21.15.53:58610) 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 -- 172.21.15.53:0/1724831108 learned_addr learned my addr 172.21.15.53:0/1724831108 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe883fff640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe8837fe640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359d5340 0x55a2359d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 -- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 msgr2=0x55a2359d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 -- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359d5340 msgr2=0x55a2359d4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359d5340 0x55a2359d4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 -- 172.21.15.53:0/1724831108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a235747f70 con 0x55a2359f2220 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe8837fe640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359d5340 0x55a2359d4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe882ffd640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359f2220 0x55a2359d79c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fe878015df0 tx=0x7fe878009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5836bf2f03a9b1ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe8627fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:03.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe8627fc640 1 -- 172.21.15.53:0/1724831108 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878032430 con 0x55a2359f2220 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe8627fc640 1 -- 172.21.15.53:0/1724831108 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fe878029400 con 0x55a2359f2220 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.974+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe878024070 con 0x55a2359f2220 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe8627fc640 1 -- 172.21.15.53:0/1724831108 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe87803a050 con 0x55a2359f2220 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 10 client.7839 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55a2355df170 con 0x55a2359f2220 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe8627fc640 1 -- 172.21.15.53:0/1724831108 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878024070 con 0x55a2359f2220 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe8627fc640 1 -- 172.21.15.53:0/1724831108 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fe87803d020 con 0x55a2359f2220 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 10 client.7839 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 10 client.7839 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 20 client.7839 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359c2cd0 0x55a2359d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 4 client.7839 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:03.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a23591b900 con 0x55a2359c2cd0 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe8837fe640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359c2cd0 0x55a2359d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe8837fe640 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359c2cd0 0x55a2359d2680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39fa3b594eb4b324 server_cookie=c7c9ed3e14c8ab58 in_seq=0 out_seq=0 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.975+0000 7fe8627fc640 10 client.7839 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe8627fc640 1 -- 172.21.15.53:0/1724831108 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a23591b900 con 0x55a2359c2cd0 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe8627fc640 10 client.7839 handle_command_reply: tid=0 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 client.7839 shutdown 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359f2220 msgr2=0x55a2359d79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359f2220 0x55a2359d79c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fe878015df0 tx=0x7fe878009190 comp rx=0 tx=0).stop 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 shutdown_connections 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359f2220 0x55a2359d79c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359c2cd0 0x55a2359d2680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/1724831108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359d5340 0x55a2359d4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 >> 172.21.15.53:0/1724831108 conn(0x55a2359d4c00 msgr2=0x55a2359a85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe8627fc640 0 client.7839 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:03.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 shutdown_connections 2024-05-10T10:42:03.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 1 -- 172.21.15.53:0/1724831108 wait complete. 2024-05-10T10:42:03.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.976+0000 7fe88cd8f740 20 client.7839 trim_cache size 0 max 16384 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.981+0000 7f6f3a059640 1 Processor -- start 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.981+0000 7f6f3a059640 1 -- start start 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f3a059640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6f34162810 0x7f6f340b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f3a059640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6f340b3450 0x7f6f340b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f3a059640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f340cc300 con 0x7f6f340d5ea0 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f3a059640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6f340b64d0 con 0x7f6f340b3450 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f3a059640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6f340b5c30 con 0x7f6f34162810 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f2bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6f340d5ea0 0x7f6f340b23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54544/0 (socket says 172.21.15.53:54544) 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f2bfff640 1 -- 172.21.15.53:0/3545420390 learned_addr learned my addr 172.21.15.53:0/3545420390 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.982+0000 7f6f33fff640 1 --2- 172.21.15.53:0/3545420390 >> v2:172.21.15.53:3300/0 conn(0x7f6f340b3450 0x7f6f340b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.983+0000 7f6f337fe640 1 --2- 172.21.15.53:0/3545420390 >> v2:172.21.15.135:3300/0 conn(0x7f6f34162810 0x7f6f340b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.983+0000 7f6f33fff640 1 -- 172.21.15.53:0/3545420390 >> v2:172.21.15.135:3300/0 conn(0x7f6f34162810 msgr2=0x7f6f340b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.983+0000 7f6f33fff640 1 --2- 172.21.15.53:0/3545420390 >> v2:172.21.15.135:3300/0 conn(0x7f6f34162810 0x7f6f340b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.983+0000 7f6f33fff640 1 -- 172.21.15.53:0/3545420390 >> v1:172.21.15.53:6789/0 conn(0x7f6f340d5ea0 legacy=0x7f6f340b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.983+0000 7f6f33fff640 1 -- 172.21.15.53:0/3545420390 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f340b5f10 con 0x7f6f340b3450 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.983+0000 7f6f32ffd640 1 -- 172.21.15.53:0/3545420390 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399789586 0 0) 0x7f6f1c001110 con 0x7f6f340d5ea0 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.983+0000 7f6f337fe640 1 --2- 172.21.15.53:0/3545420390 >> v2:172.21.15.135:3300/0 conn(0x7f6f34162810 0x7f6f340b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.984+0000 7f6f33fff640 1 --2- 172.21.15.53:0/3545420390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340b3450 0x7f6f340b38e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6f3408fab0 tx=0x7f6f2c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cc08b97d742d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.984+0000 7f6f32ffd640 1 -- 172.21.15.53:0/3545420390 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c007740 con 0x7f6f340b3450 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.984+0000 7f6f32ffd640 1 -- 172.21.15.53:0/3545420390 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c0078e0 con 0x7f6f340b3450 2024-05-10T10:42:03.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.984+0000 7f6f32ffd640 1 -- 172.21.15.53:0/3545420390 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c007cb0 con 0x7f6f340b3450 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 -- 172.21.15.53:0/3545420390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340b3450 msgr2=0x7f6f340b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 --2- 172.21.15.53:0/3545420390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340b3450 0x7f6f340b38e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6f3408fab0 tx=0x7f6f2c02a680 comp rx=0 tx=0).stop 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 -- 172.21.15.53:0/3545420390 shutdown_connections 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 --2- 172.21.15.53:0/3545420390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340b3450 0x7f6f340b38e0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 --2- 172.21.15.53:0/3545420390 >> v2:172.21.15.135:3300/0 conn(0x7f6f34162810 0x7f6f340b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 -- 172.21.15.53:0/3545420390 >> 172.21.15.53:0/3545420390 conn(0x7f6f3409e590 msgr2=0x7f6f340d5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 -- 172.21.15.53:0/3545420390 shutdown_connections 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.985+0000 7f6f3a059640 1 -- 172.21.15.53:0/3545420390 wait complete. 2024-05-10T10:42:03.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.986+0000 7f6f3a059640 1 Processor -- start 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.986+0000 7f6f3a059640 1 -- start start 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f3a059640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f340b3450 0x7f6f34151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f3a059640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340d5ea0 0x7f6f3414f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f3a059640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f34162810 0x7f6f3414f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f3a059640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f340b2f30 con 0x7f6f340d5ea0 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f3a059640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6f340b61f0 con 0x7f6f34162810 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f3a059640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f6f340b58e0 con 0x7f6f340b3450 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340d5ea0 0x7f6f3414f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340d5ea0 0x7f6f3414f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58628/0 (socket says 172.21.15.53:58628) 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 -- 172.21.15.53:0/2969836088 learned_addr learned my addr 172.21.15.53:0/2969836088 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f337fe640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f340b3450 0x7f6f34151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f33fff640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f34162810 0x7f6f3414f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 -- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f340b3450 msgr2=0x7f6f34151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6f340b3450 0x7f6f34151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 -- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f34162810 msgr2=0x7f6f3414f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f34162810 0x7f6f3414f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f2bfff640 1 -- 172.21.15.53:0/2969836088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f341502e0 con 0x7f6f340d5ea0 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.987+0000 7f6f33fff640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f34162810 0x7f6f3414f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.988+0000 7f6f2bfff640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f340d5ea0 0x7f6f3414f190 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c430 tx=0x7f6f1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.988+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0074e0 con 0x7f6f340d5ea0 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.988+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1c0047e0 con 0x7f6f340d5ea0 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.988+0000 7f6f3a059640 1 -- 172.21.15.53:0/2969836088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341505c0 con 0x7f6f340d5ea0 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.988+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c004bd0 con 0x7f6f340d5ea0 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.988+0000 7f6f3a059640 1 -- 172.21.15.53:0/2969836088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f34150b80 con 0x7f6f340d5ea0 2024-05-10T10:42:03.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.988+0000 7f6f3a059640 1 -- 172.21.15.53:0/2969836088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f6f34162bf0 con 0x7f6f340d5ea0 2024-05-10T10:42:03.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.989+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6f1c011070 con 0x7f6f340d5ea0 2024-05-10T10:42:03.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.990+0000 7f6f30ff9640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6f140762d0 0x7f6f14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:03.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.990+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7f6f1c097c70 con 0x7f6f340d5ea0 2024-05-10T10:42:03.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.990+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+443 (secure 0 0 0) 0x7f6f34162bf0 con 0x7f6f340d5ea0 2024-05-10T10:42:03.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.990+0000 7f6f337fe640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6f140762d0 0x7f6f14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:03.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.990+0000 7f6f3a059640 1 -- 172.21.15.53:0/2969836088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f6f3406a220 con 0x7f6f340d5ea0 2024-05-10T10:42:03.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.991+0000 7f6f337fe640 1 --2- 172.21.15.53:0/2969836088 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f6f140762d0 0x7f6f14078790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6f24007990 tx=0x7f6f24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:03.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.997+0000 7fe88cd8f740 1 Processor -- start 2024-05-10T10:42:03.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.997+0000 7fe88cd8f740 1 -- start start 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a235a13480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe884202000 con 0x55a2359d5340 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841609c0 con 0x55a2359f2220 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe88415fd20 con 0x55a2359d1400 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a235a13480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a235a13480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58632/0 (socket says 172.21.15.53:58632) 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 -- 172.21.15.53:0/3395484138 learned_addr learned my addr 172.21.15.53:0/3395484138 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe8837fe640 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe883fff640 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 -- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 msgr2=0x55a2359f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359f18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 -- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 msgr2=0x55a2359ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 -- 172.21.15.53:0/3395484138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2359ac010 con 0x55a2359d5340 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.998+0000 7fe882ffd640 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a235a13480 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fe878029940 tx=0x7fe87802b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc20ce0f2b2a001 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe868ff9640 1 -- 172.21.15.53:0/3395484138 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878043070 con 0x55a2359d5340 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe868ff9640 1 -- 172.21.15.53:0/3395484138 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878024040 con 0x55a2359d5340 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe868ff9640 1 -- 172.21.15.53:0/3395484138 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878029620 con 0x55a2359d5340 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3395484138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 msgr2=0x55a235a13480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a235a13480 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fe878029940 tx=0x7fe87802b070 comp rx=0 tx=0).stop 2024-05-10T10:42:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3395484138 shutdown_connections 2024-05-10T10:42:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d5340 0x55a235a13480 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/3395484138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d1400 0x55a2359f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3395484138 >> 172.21.15.53:0/3395484138 conn(0x55a2359d4c00 msgr2=0x55a2359d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3395484138 shutdown_connections 2024-05-10T10:42:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:03.999+0000 7fe88cd8f740 1 -- 172.21.15.53:0/3395484138 wait complete. 2024-05-10T10:42:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.000+0000 7fe88cd8f740 1 -- start start 2024-05-10T10:42:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.000+0000 7fe88cd8f740 1 Processor -- start 2024-05-10T10:42:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d1400 0x55a2359e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d5340 0x55a235a6d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe88cd8f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a2359a5480 con 0x55a2359d1400 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe884160370 con 0x55a2359f2220 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe88cd8f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a23599f560 con 0x55a2359d5340 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe88cd8f740 10 client.0 _ll_register_callbacks cb 0x7fe8740094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d1400 0x55a2359e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d1400 0x55a2359e5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58644/0 (socket says 172.21.15.53:58644) 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 -- 172.21.15.53:0/902715022 learned_addr learned my addr 172.21.15.53:0/902715022 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe883fff640 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe882ffd640 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d5340 0x55a235a6d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 -- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d5340 msgr2=0x55a235a6d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d5340 0x55a235a6d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 -- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 msgr2=0x55a2359eb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359eb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 -- 172.21.15.53:0/902715022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2359d3480 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8837fe640 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d1400 0x55a2359e5050 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fe8740807b0 tx=0x7fe874081710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c254008ea8946c83 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8627fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.001+0000 7fe8627fc640 1 -- 172.21.15.53:0/902715022 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740b4070 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe8627fc640 1 -- 172.21.15.53:0/902715022 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fe874089a90 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe878024040 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe8627fc640 1 -- 172.21.15.53:0/902715022 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe874089d20 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 10 client.7854 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55a2359a4720 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe8627fc640 1 -- 172.21.15.53:0/902715022 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878024040 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe8627fc640 1 -- 172.21.15.53:0/902715022 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fe874084040 con 0x55a2359d1400 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 10 client.7854 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 10 client.7854 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:42:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 20 client.7854 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359a3a60 0x55a2359ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 4 client.7854 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55a235915770 con 0x55a2359a3a60 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe882ffd640 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359a3a60 0x55a2359ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe8627fc640 10 client.7854 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.002+0000 7fe882ffd640 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359a3a60 0x55a2359ebef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1639a370eb4c6a1d server_cookie=2d6940cb7c958f34 in_seq=0 out_seq=0 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe8627fc640 1 -- 172.21.15.53:0/902715022 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a235915770 con 0x55a2359a3a60 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe8627fc640 10 client.7854 handle_command_reply: tid=0 2024-05-10T10:42:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 client.7854 shutdown 2024-05-10T10:42:04.009 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:42:04.003+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-10T10:42:04.009 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:42:04.003+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-10T10:42:04.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d1400 msgr2=0x55a2359e5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d1400 0x55a2359e5050 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fe8740807b0 tx=0x7fe874081710 comp rx=0 tx=0).stop 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 shutdown_connections 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a2359a3a60 0x55a2359ebef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2359f2220 0x55a2359eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe8627fc640 0 client.7854 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a2359d5340 0x55a235a6d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 --2- 172.21.15.53:0/902715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2359d1400 0x55a2359e5050 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.003+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 >> 172.21.15.53:0/902715022 conn(0x55a2359d4c00 msgr2=0x55a2359d7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.004+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 shutdown_connections 2024-05-10T10:42:04.012 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:42:04.004+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-10T10:42:04.012 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:42:04.004+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-10T10:42:04.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.004+0000 7fe88cd8f740 1 -- 172.21.15.53:0/902715022 wait complete. 2024-05-10T10:42:04.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.004+0000 7fe88cd8f740 20 client.7854 trim_cache size 0 max 16384 2024-05-10T10:42:04.012 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0, 2024-05-10T10:42:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_tag": "e283ec97-3c43-4c08-9401-65509dafdbef", 2024-05-10T10:42:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "mode": "asynchronous" 2024-05-10T10:42:04.013 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8700762d0 msgr2=0x7fe870078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8700762d0 0x7fe870078790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe884160de0 tx=0x7fe878009460 comp rx=0 tx=0).stop 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 msgr2=0x7fe884154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe884154e20 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe87402a220 tx=0x7fe8740046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 shutdown_connections 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe8700762d0 0x7fe870078790 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe884162ff0 0x7fe884161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8840cf780 0x7fe8840c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 --2- 172.21.15.53:0/598298595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8840b4210 0x7fe884154e20 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 >> 172.21.15.53:0/598298595 conn(0x7fe8840d2e00 msgr2=0x7fe8840714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 shutdown_connections 2024-05-10T10:42:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.005+0000 7fe88a072640 1 -- 172.21.15.53:0/598298595 wait complete. 2024-05-10T10:42:04.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:04.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-10T10:42:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.122+0000 7fd3bba99640 1 Processor -- start 2024-05-10T10:42:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.122+0000 7fd3bba99640 1 -- start start 2024-05-10T10:42:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3bba99640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd3b406df20 0x7fd3b40739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3bba99640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b4073f20 0x7fd3b415d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3bba99640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b4157a40 con 0x7fd3b406c2c0 2024-05-10T10:42:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3bba99640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b4161990 con 0x7fd3b406df20 2024-05-10T10:42:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3bba99640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b4160d80 con 0x7fd3b4073f20 2024-05-10T10:42:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3ba036640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b4073f20 0x7fd3b415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3b9034640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd3b406df20 0x7fd3b40739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3ba036640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b4073f20 0x7fd3b415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:43120/0 (socket says 172.21.15.53:43120) 2024-05-10T10:42:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3ba036640 1 -- 172.21.15.53:0/367358363 learned_addr learned my addr 172.21.15.53:0/367358363 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3b9835640 1 --1- 172.21.15.53:0/367358363 >> v1:172.21.15.135:6789/0 conn(0x7fd3b406c2c0 0x7fd3b406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:38740/0 (socket says 172.21.15.53:38740) 2024-05-10T10:42:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3ba036640 1 -- 172.21.15.53:0/367358363 >> v2:172.21.15.135:3300/0 conn(0x7fd3b406df20 msgr2=0x7fd3b40739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3ba036640 1 --2- 172.21.15.53:0/367358363 >> v2:172.21.15.135:3300/0 conn(0x7fd3b406df20 0x7fd3b40739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3ba036640 1 -- 172.21.15.53:0/367358363 >> v1:172.21.15.135:6789/0 conn(0x7fd3b406c2c0 legacy=0x7fd3b406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3ba036640 1 -- 172.21.15.53:0/367358363 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b4161060 con 0x7fd3b4073f20 2024-05-10T10:42:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.123+0000 7fd3b9034640 1 --2- 172.21.15.53:0/367358363 >> v2:172.21.15.135:3300/0 conn(0x7fd3b406df20 0x7fd3b40739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.124+0000 7fd3ba036640 1 --2- 172.21.15.53:0/367358363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b4073f20 0x7fd3b415d810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e50 tx=0x7fd3ac02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb186b279420aff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.124+0000 7fd3b8833640 1 -- 172.21.15.53:0/367358363 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac036070 con 0x7fd3b4073f20 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.124+0000 7fd3b8833640 1 -- 172.21.15.53:0/367358363 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac031600 con 0x7fd3b4073f20 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.124+0000 7fd3b8833640 1 -- 172.21.15.53:0/367358363 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0319d0 con 0x7fd3b4073f20 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 -- 172.21.15.53:0/367358363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b4073f20 msgr2=0x7fd3b415d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 --2- 172.21.15.53:0/367358363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b4073f20 0x7fd3b415d810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e50 tx=0x7fd3ac02a710 comp rx=0 tx=0).stop 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 -- 172.21.15.53:0/367358363 shutdown_connections 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 --2- 172.21.15.53:0/367358363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b4073f20 0x7fd3b415d810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 --2- 172.21.15.53:0/367358363 >> v2:172.21.15.135:3300/0 conn(0x7fd3b406df20 0x7fd3b40739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 -- 172.21.15.53:0/367358363 >> 172.21.15.53:0/367358363 conn(0x7fd3b40d0c90 msgr2=0x7fd3b40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 -- 172.21.15.53:0/367358363 shutdown_connections 2024-05-10T10:42:04.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.125+0000 7fd3bba99640 1 -- 172.21.15.53:0/367358363 wait complete. 2024-05-10T10:42:04.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.126+0000 7fd3bba99640 1 Processor -- start 2024-05-10T10:42:04.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.126+0000 7fd3bba99640 1 -- start start 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3bba99640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b406c2c0 0x7fd3b4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3bba99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3b406df20 0x7fd3b40c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3bba99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3b4073f20 0x7fd3b41e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3bba99640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b40cd440 con 0x7fd3b406df20 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3bba99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4161620 con 0x7fd3b4073f20 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3bba99640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b4072180 con 0x7fd3b406c2c0 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3b406df20 0x7fd3b40c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3b406df20 0x7fd3b40c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58648/0 (socket says 172.21.15.53:58648) 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 -- 172.21.15.53:0/3082083730 learned_addr learned my addr 172.21.15.53:0/3082083730 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9835640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b406c2c0 0x7fd3b4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3ba036640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3b4073f20 0x7fd3b41e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 -- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b406c2c0 msgr2=0x7fd3b4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b406c2c0 0x7fd3b4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 -- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3b4073f20 msgr2=0x7fd3b41e7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3b4073f20 0x7fd3b41e7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.127+0000 7fd3b9034640 1 -- 172.21.15.53:0/3082083730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b41e7e30 con 0x7fd3b406df20 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.128+0000 7fd3b9034640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3b406df20 0x7fd3b40c5740 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400e960 tx=0x7fd3a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.128+0000 7fd3aa7fc640 1 -- 172.21.15.53:0/3082083730 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4017070 con 0x7fd3b406df20 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.128+0000 7fd3aa7fc640 1 -- 172.21.15.53:0/3082083730 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a400ca00 con 0x7fd3b406df20 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.128+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b40c7d60 con 0x7fd3b406df20 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.128+0000 7fd3aa7fc640 1 -- 172.21.15.53:0/3082083730 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400cdd0 con 0x7fd3b406df20 2024-05-10T10:42:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.128+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b40c8320 con 0x7fd3b406df20 2024-05-10T10:42:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.130+0000 7fd3aa7fc640 1 -- 172.21.15.53:0/3082083730 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd3a401a070 con 0x7fd3b406df20 2024-05-10T10:42:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.130+0000 7fd3aa7fc640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd3980762d0 0x7fd398078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.130+0000 7fd3aa7fc640 1 -- 172.21.15.53:0/3082083730 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fd3a40979f0 con 0x7fd3b406df20 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.131+0000 7fd3b9835640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd3980762d0 0x7fd398078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.131+0000 7fd3b9835640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd3980762d0 0x7fd398078790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd3b41e8110 tx=0x7fd3b0007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.131+0000 7fd3be7b8740 1 Processor -- start 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.131+0000 7fd3be7b8740 1 -- start start 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4f450 0x557561e53250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e27160 0x557561e4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e52600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557561def690 con 0x557561e4fd70 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557561d8cf30 con 0x557561e27160 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557561d45b20 con 0x557561e4f450 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e52600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e52600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58654/0 (socket says 172.21.15.53:58654) 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 -- 172.21.15.53:0/3386802920 learned_addr learned my addr 172.21.15.53:0/3386802920 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3b9835640 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4f450 0x557561e53250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3b9034640 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e27160 0x557561e4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 -- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4f450 msgr2=0x557561e53250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4f450 0x557561e53250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 -- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e27160 msgr2=0x557561e4f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e27160 0x557561e4f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.132+0000 7fd3ba036640 1 -- 172.21.15.53:0/3386802920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557561d3de60 con 0x557561e4fd70 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3ba036640 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e52600 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009f70 tx=0x7fd3ac02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf98ce4b13bf160c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3977fe640 1 -- 172.21.15.53:0/3386802920 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac036040 con 0x557561e4fd70 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3977fe640 1 -- 172.21.15.53:0/3386802920 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac037040 con 0x557561e4fd70 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3977fe640 1 -- 172.21.15.53:0/3386802920 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0318c0 con 0x557561e4fd70 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3be7b8740 1 -- 172.21.15.53:0/3386802920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 msgr2=0x557561e52600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e52600 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009f70 tx=0x7fd3ac02e000 comp rx=0 tx=0).stop 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3be7b8740 1 -- 172.21.15.53:0/3386802920 shutdown_connections 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.133+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e52600 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.134+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e27160 0x557561e4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.134+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/3386802920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4f450 0x557561e53250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.134+0000 7fd3be7b8740 1 -- 172.21.15.53:0/3386802920 >> 172.21.15.53:0/3386802920 conn(0x557561e24d60 msgr2=0x557561e6d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.134+0000 7fd3be7b8740 1 -- 172.21.15.53:0/3386802920 shutdown_connections 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.134+0000 7fd3be7b8740 1 -- 172.21.15.53:0/3386802920 wait complete. 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.134+0000 7fd3be7b8740 1 -- start start 2024-05-10T10:42:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.134+0000 7fd3be7b8740 1 Processor -- start 2024-05-10T10:42:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4f450 0x557561e463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4c9a0 0x557561e4ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557561d495e0 con 0x557561e4f450 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557561d88ef0 con 0x557561e4c9a0 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557561e230b0 con 0x557561e4fd70 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3be7b8740 10 client.0 _ll_register_callbacks cb 0x557561d44320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3b9835640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4f450 0x557561e463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3b9835640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4f450 0x557561e463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58662/0 (socket says 172.21.15.53:58662) 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3b9835640 1 -- 172.21.15.53:0/2766908324 learned_addr learned my addr 172.21.15.53:0/2766908324 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3ba036640 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4c9a0 0x557561e4ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.135+0000 7fd3b9034640 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3b9835640 1 -- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 msgr2=0x557561e6efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3b9835640 1 -- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4c9a0 msgr2=0x557561e4ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4c9a0 0x557561e4ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3b9835640 1 -- 172.21.15.53:0/2766908324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557561b687d0 con 0x557561e4f450 2024-05-10T10:42:04.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4f450 0x557561e463e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0013620 tx=0x7fd3b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69d3f5920e192d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd394ff9640 1 -- 172.21.15.53:0/2766908324 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0020580 con 0x557561e4f450 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd394ff9640 1 -- 172.21.15.53:0/2766908324 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fd3b0020720 con 0x557561e4f450 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ac037040 con 0x557561e4f450 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd394ff9640 1 -- 172.21.15.53:0/2766908324 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd3b002e050 con 0x557561e4f450 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 10 client.7863 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x557561a9e3f0 con 0x557561e4f450 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd394ff9640 1 -- 172.21.15.53:0/2766908324 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac037040 con 0x557561e4f450 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd394ff9640 1 -- 172.21.15.53:0/2766908324 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fd3b0031020 con 0x557561e4f450 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 10 client.7863 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 10 client.7863 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 20 client.7863 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e85980 0x557561e85d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 4 client.7863 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:04.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.136+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557561d9ab40 con 0x557561e85980 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.137+0000 7fd3b9034640 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e85980 0x557561e85d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.137+0000 7fd394ff9640 10 client.7863 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.137+0000 7fd3b9034640 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e85980 0x557561e85d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a4f10ddd0948f5 server_cookie=5a40edbc506d912a in_seq=0 out_seq=0 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.137+0000 7fd394ff9640 1 -- 172.21.15.53:0/2766908324 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557561d9ab40 con 0x557561e85980 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.137+0000 7fd394ff9640 10 client.7863 handle_command_reply: tid=0 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.137+0000 7fd3be7b8740 1 client.7863 shutdown 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4f450 msgr2=0x557561e463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4f450 0x557561e463e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0013620 tx=0x7fd3b0000bc0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 shutdown_connections 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e85980 0x557561e85d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4c9a0 0x557561e4ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2766908324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4f450 0x557561e463e0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd394ff9640 0 client.7863 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 >> 172.21.15.53:0/2766908324 conn(0x557561e27160 msgr2=0x557561e28210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 shutdown_connections 2024-05-10T10:42:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2766908324 wait complete. 2024-05-10T10:42:04.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.138+0000 7fd3be7b8740 20 client.7863 trim_cache size 0 max 16384 2024-05-10T10:42:04.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.156+0000 7fd3be7b8740 1 Processor -- start 2024-05-10T10:42:04.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.156+0000 7fd3be7b8740 1 -- start start 2024-05-10T10:42:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561e4e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41fdcc0 con 0x557561e4c9a0 2024-05-10T10:42:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b40d00c0 con 0x557561e4f450 2024-05-10T10:42:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b41f7080 con 0x557561e4fd70 2024-05-10T10:42:04.160 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58678/0 (socket says 172.21.15.53:58678) 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 -- 172.21.15.53:0/2328057316 learned_addr learned my addr 172.21.15.53:0/2328057316 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9034640 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561e4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3ba036640 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 -- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 msgr2=0x557561e6ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 -- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 msgr2=0x557561e4e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561e4e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 -- 172.21.15.53:0/2328057316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557561a5e0f0 con 0x557561e4c9a0 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9034640 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561e4e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3ba036640 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4c9a0 0x557561e221f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007670 tx=0x7fd3b0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7804a483153c69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd396ffd640 1 -- 172.21.15.53:0/2328057316 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0013df0 con 0x557561e4c9a0 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd396ffd640 1 -- 172.21.15.53:0/2328057316 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0026070 con 0x557561e4c9a0 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.157+0000 7fd396ffd640 1 -- 172.21.15.53:0/2328057316 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0020580 con 0x557561e4c9a0 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2328057316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4c9a0 msgr2=0x557561e221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4c9a0 0x557561e221f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007670 tx=0x7fd3b0021000 comp rx=0 tx=0).stop 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2328057316 shutdown_connections 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4fd70 0x557561e6ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561e4e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2328057316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2328057316 >> 172.21.15.53:0/2328057316 conn(0x557561e27160 msgr2=0x557561e95120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2328057316 shutdown_connections 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2328057316 wait complete. 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.158+0000 7fd3be7b8740 1 -- start start 2024-05-10T10:42:04.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 1 Processor -- start 2024-05-10T10:42:04.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561ee9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.166 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557561e704d0 con 0x557561e4fd70 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b40cfa70 con 0x557561e4f450 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x557561e4fb60 con 0x557561e4c9a0 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.159+0000 7fd3be7b8740 10 client.0 _ll_register_callbacks cb 0x7fd3b00048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e26300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58682/0 (socket says 172.21.15.53:58682) 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 -- 172.21.15.53:0/2314704595 learned_addr learned my addr 172.21.15.53:0/2314704595 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3b9034640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561ee9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 -- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4c9a0 msgr2=0x557561e221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 -- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 msgr2=0x557561ee9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561ee9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 -- 172.21.15.53:0/2314704595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557561e71500 con 0x557561e4fd70 2024-05-10T10:42:04.167 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3b9034640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561ee9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3ba036640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e26300 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e50 tx=0x7fd3ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790904d81a17b2d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd394ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd394ff9640 1 -- 172.21.15.53:0/2314704595 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac036040 con 0x557561e4fd70 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd394ff9640 1 -- 172.21.15.53:0/2314704595 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fd3ac0318d0 con 0x557561e4fd70 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b0026070 con 0x557561e4fd70 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd394ff9640 1 -- 172.21.15.53:0/2314704595 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd3ac055050 con 0x557561e4fd70 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3be7b8740 10 client.7869 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x557561eef9f0 con 0x557561e4fd70 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.160+0000 7fd394ff9640 1 -- 172.21.15.53:0/2314704595 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0026070 con 0x557561e4fd70 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd394ff9640 1 -- 172.21.15.53:0/2314704595 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fd3ac058020 con 0x557561e4fd70 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 10 client.7869 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 10 client.7869 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:42:04.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 20 client.7869 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e4e570 0x557561e26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 4 client.7869 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557561d949b0 con 0x557561e4e570 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e4e570 0x557561e26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd394ff9640 10 client.7869 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3b9835640 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e4e570 0x557561e26840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0732135cb7652 server_cookie=88253979672e20d1 in_seq=0 out_seq=0 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd394ff9640 1 -- 172.21.15.53:0/2314704595 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x557561d949b0 con 0x557561e4e570 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd394ff9640 10 client.7869 handle_command_reply: tid=0 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 client.7869 shutdown 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 msgr2=0x557561e26300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e26300 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e50 tx=0x7fd3ac004ab0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 shutdown_connections 2024-05-10T10:42:04.169 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557561e4fd70 0x557561e26300 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x557561e4e570 0x557561e26840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557561e4f450 0x557561ee9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 --2- 172.21.15.53:0/2314704595 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x557561e4c9a0 0x557561e221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd394ff9640 0 client.7869 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.161+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 >> 172.21.15.53:0/2314704595 conn(0x557561e27160 msgr2=0x557561e4d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 shutdown_connections 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3be7b8740 1 -- 172.21.15.53:0/2314704595 wait complete. 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3be7b8740 20 client.7869 trim_cache size 0 max 16384 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": { 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "e283ec97-3c43-4c08-9401-65509dafdbef": { 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "path": "/", 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "tag": "e283ec97-3c43-4c08-9401-65509dafdbef", 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "options": "recursive,repair" 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:04.174 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd3980762d0 msgr2=0x7fd398078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd3980762d0 0x7fd398078790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd3b41e8110 tx=0x7fd3b0007e80 comp rx=0 tx=0).stop 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3b406df20 msgr2=0x7fd3b40c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3b406df20 0x7fd3b40c5740 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400e960 tx=0x7fd3a400ee20 comp rx=0 tx=0).stop 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 shutdown_connections 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd3980762d0 0x7fd398078790 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3b4073f20 0x7fd3b41e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3b406df20 0x7fd3b40c5740 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 --2- 172.21.15.53:0/3082083730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3b406c2c0 0x7fd3b4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:04.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 >> 172.21.15.53:0/3082083730 conn(0x7fd3b40d0c90 msgr2=0x7fd3b40b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 shutdown_connections 2024-05-10T10:42:04.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.162+0000 7fd3bba99640 1 -- 172.21.15.53:0/3082083730 wait complete. 2024-05-10T10:42:04.177 INFO:tasks.cephfs.filesystem:scrub status for tag:e283ec97-3c43-4c08-9401-65509dafdbef - {'path': '/', 'tag': 'e283ec97-3c43-4c08-9401-65509dafdbef', 'options': 'recursive,repair'} 2024-05-10T10:42:04.351 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:42:04.349+0000 7f0bf79ca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:42:04.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:04.646+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 7 ==== log(8 entries from seq 1 at 2024-05-10T10:42:04.004687+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f6f1c096090 con 0x7f6f340d5ea0 2024-05-10T10:42:04.648 INFO:teuthology.orchestra.run.smithi053.stdout: cluster: 2024-05-10T10:42:04.648 INFO:teuthology.orchestra.run.smithi053.stdout: id: 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:42:04.648 INFO:teuthology.orchestra.run.smithi053.stdout: health: HEALTH_OK 2024-05-10T10:42:04.648 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:04.648 INFO:teuthology.orchestra.run.smithi053.stdout: services: 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: mgr: x(active, since 8m), standbys: y 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: mds: 1/1 daemons up, 3 standby 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: data: 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: volumes: 1/1 healthy 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: pools: 3 pools, 97 pgs 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: objects: 30 objects, 19 MiB 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: pgs: 97 active+clean 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:04.649 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.004687+0000 mds.c [INF] scrub queued for path: / 2024-05-10T10:42:04.650 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.004690+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-10T10:42:04.650 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.004694+0000 mds.c [INF] scrub summary: active paths [/] 2024-05-10T10:42:04.650 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.005218+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-05-10T10:42:04.650 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.005222+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-05-10T10:42:04.650 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.005632+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-05-10T10:42:04.650 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.005639+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-05-10T10:42:04.650 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:04.005968+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-10T10:42:09.655 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:09.653+0000 7f6f30ff9640 1 -- 172.21.15.53:0/2969836088 <== mon.0 v2:172.21.15.53:3300/0 8 ==== log(1 entries from seq 9 at 2024-05-10T10:42:08.388374+0000) ==== 189+0+0 (secure 0 0 0) 0x7f6f1c004d70 con 0x7f6f340d5ea0 2024-05-10T10:42:09.655 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:42:08.388374+0000 mds.c [INF] scrub summary: idle 2024-05-10T10:42:15.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:42:15.732 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:42:34.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:34.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-10T10:42:34.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.295+0000 7fd318f3f640 1 Processor -- start 2024-05-10T10:42:34.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.295+0000 7fd318f3f640 1 -- start start 2024-05-10T10:42:34.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd318f3f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd3140cf3e0 0x7fd3140cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd318f3f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140d2190 0x7fd3140cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd318f3f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31406e720 con 0x7fd3140cd500 2024-05-10T10:42:34.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd318f3f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd31406de80 con 0x7fd3140cf3e0 2024-05-10T10:42:34.297 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd318f3f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd31406e160 con 0x7fd3140d2190 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd31259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd3140cf3e0 0x7fd3140cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd31259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd3140cf3e0 0x7fd3140cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60436/0 (socket says 172.21.15.53:60436) 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd31259c640 1 -- 172.21.15.53:0/2814938547 learned_addr learned my addr 172.21.15.53:0/2814938547 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd312d9d640 1 --2- 172.21.15.53:0/2814938547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140d2190 0x7fd3140cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd31259c640 1 -- 172.21.15.53:0/2814938547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140d2190 msgr2=0x7fd3140cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd31259c640 1 --2- 172.21.15.53:0/2814938547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140d2190 0x7fd3140cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.296+0000 7fd31259c640 1 -- 172.21.15.53:0/2814938547 >> v1:172.21.15.135:6789/0 conn(0x7fd3140cd500 legacy=0x7fd3140b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.297+0000 7fd31259c640 1 -- 172.21.15.53:0/2814938547 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31406e440 con 0x7fd3140cf3e0 2024-05-10T10:42:34.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.297+0000 7fd31259c640 1 --2- 172.21.15.53:0/2814938547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cf3e0 0x7fd3140cf8c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd304009870 tx=0x7fd30402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1502b3bff4c38cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.297+0000 7fd31159a640 1 -- 172.21.15.53:0/2814938547 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304037070 con 0x7fd3140cf3e0 2024-05-10T10:42:34.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.297+0000 7fd31159a640 1 -- 172.21.15.53:0/2814938547 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd304032440 con 0x7fd3140cf3e0 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.297+0000 7fd31159a640 1 -- 172.21.15.53:0/2814938547 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304032810 con 0x7fd3140cf3e0 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 -- 172.21.15.53:0/2814938547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cf3e0 msgr2=0x7fd3140cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 --2- 172.21.15.53:0/2814938547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cf3e0 0x7fd3140cf8c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd304009870 tx=0x7fd30402a210 comp rx=0 tx=0).stop 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 -- 172.21.15.53:0/2814938547 shutdown_connections 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 --2- 172.21.15.53:0/2814938547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140d2190 0x7fd3140cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 --2- 172.21.15.53:0/2814938547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cf3e0 0x7fd3140cf8c0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 -- 172.21.15.53:0/2814938547 >> 172.21.15.53:0/2814938547 conn(0x7fd3140d2eb0 msgr2=0x7fd3140d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 -- 172.21.15.53:0/2814938547 shutdown_connections 2024-05-10T10:42:34.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.298+0000 7fd318f3f640 1 -- 172.21.15.53:0/2814938547 wait complete. 2024-05-10T10:42:34.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 Processor -- start 2024-05-10T10:42:34.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 -- start start 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cd500 0x7fd314149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140cf3e0 0x7fd3140c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3140d2190 0x7fd31415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140b34e0 con 0x7fd3140cd500 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140b4510 con 0x7fd3140d2190 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd318f3f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd3140b3660 con 0x7fd3140cf3e0 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd31259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cd500 0x7fd314149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd31259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cd500 0x7fd314149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60440/0 (socket says 172.21.15.53:60440) 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.300+0000 7fd31259c640 1 -- 172.21.15.53:0/1809347951 learned_addr learned my addr 172.21.15.53:0/1809347951 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd312d9d640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3140d2190 0x7fd31415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140cf3e0 0x7fd3140c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd31259c640 1 -- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140cf3e0 msgr2=0x7fd3140c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd31259c640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140cf3e0 0x7fd3140c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd31259c640 1 -- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3140d2190 msgr2=0x7fd31415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd31259c640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3140d2190 0x7fd31415fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd31259c640 1 -- 172.21.15.53:0/1809347951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3140c7a30 con 0x7fd3140cd500 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140cf3e0 0x7fd3140c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd31259c640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cd500 0x7fd314149c40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd304000c00 tx=0x7fd3040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd302ffd640 1 -- 172.21.15.53:0/1809347951 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304041020 con 0x7fd3140cd500 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd302ffd640 1 -- 172.21.15.53:0/1809347951 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3040306c0 con 0x7fd3140cd500 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3140c7cf0 con 0x7fd3140cd500 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd302ffd640 1 -- 172.21.15.53:0/1809347951 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304037040 con 0x7fd3140cd500 2024-05-10T10:42:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.301+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3140c82b0 con 0x7fd3140cd500 2024-05-10T10:42:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.304+0000 7fd302ffd640 1 -- 172.21.15.53:0/1809347951 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd304044020 con 0x7fd3140cd500 2024-05-10T10:42:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.304+0000 7fd302ffd640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2f40762d0 0x7fd2f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.304+0000 7fd302ffd640 1 -- 172.21.15.53:0/1809347951 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fd3040b8040 con 0x7fd3140cd500 2024-05-10T10:42:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.304+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2f40762d0 0x7fd2f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.304+0000 7fd31bc58740 1 Processor -- start 2024-05-10T10:42:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd31bc58740 1 -- start start 2024-05-10T10:42:34.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2f40762d0 0x7fd2f4078790 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd31415d290 tx=0x7fd308009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb92160 0x55a1fcb44af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcb74600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb74b40 0x55a1fcb49950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd31bc58740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a1fcb11690 con 0x55a1fcb74b40 2024-05-10T10:42:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a1fcaaef30 con 0x55a1fcb75280 2024-05-10T10:42:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.305+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a1fca67b20 con 0x55a1fcb92160 2024-05-10T10:42:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb74b40 0x55a1fcb49950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb74b40 0x55a1fcb49950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60444/0 (socket says 172.21.15.53:60444) 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 -- 172.21.15.53:0/1782410704 learned_addr learned my addr 172.21.15.53:0/1782410704 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd31259c640 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb92160 0x55a1fcb44af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcb74600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 -- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb92160 msgr2=0x55a1fcb44af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb92160 0x55a1fcb44af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 -- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 msgr2=0x55a1fcb74600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcb74600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 -- 172.21.15.53:0/1782410704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1fca5fe60 con 0x55a1fcb74b40 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd312d9d640 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb74b40 0x55a1fcb49950 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd30c007410 tx=0x7fd30c0078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db02809d968d78d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd2f3fff640 1 -- 172.21.15.53:0/1782410704 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0147e0 con 0x55a1fcb74b40 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd2f3fff640 1 -- 172.21.15.53:0/1782410704 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c01a070 con 0x55a1fcb74b40 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.306+0000 7fd2f3fff640 1 -- 172.21.15.53:0/1782410704 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c014b80 con 0x55a1fcb74b40 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 -- 172.21.15.53:0/1782410704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb74b40 msgr2=0x55a1fcb49950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb74b40 0x55a1fcb49950 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd30c007410 tx=0x7fd30c0078d0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 -- 172.21.15.53:0/1782410704 shutdown_connections 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb74b40 0x55a1fcb49950 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcb74600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1782410704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb92160 0x55a1fcb44af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 -- 172.21.15.53:0/1782410704 >> 172.21.15.53:0/1782410704 conn(0x55a1fcb46d60 msgr2=0x55a1fcb8f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 -- 172.21.15.53:0/1782410704 shutdown_connections 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.307+0000 7fd31bc58740 1 -- 172.21.15.53:0/1782410704 wait complete. 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.308+0000 7fd31bc58740 1 -- start start 2024-05-10T10:42:34.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.308+0000 7fd31bc58740 1 Processor -- start 2024-05-10T10:42:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb75280 0x55a1fcb86510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb92160 0x55a1fcb78400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcb64fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a1fca6b5e0 con 0x55a1fcb75280 2024-05-10T10:42:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a1fcaaaef0 con 0x55a1fcb92160 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a1fcb48f10 con 0x55a1fcb76e30 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 10 client.0 _ll_register_callbacks cb 0x55a1fca66320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb75280 0x55a1fcb86510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb75280 0x55a1fcb86510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60456/0 (socket says 172.21.15.53:60456) 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 -- 172.21.15.53:0/1554225782 learned_addr learned my addr 172.21.15.53:0/1554225782 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd312d9d640 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcb64fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb92160 0x55a1fcb78400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 -- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 msgr2=0x55a1fcb64fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcb64fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 -- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb92160 msgr2=0x55a1fcb78400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb92160 0x55a1fcb78400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 -- 172.21.15.53:0/1554225782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1fc88a7d0 con 0x55a1fcb75280 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31259c640 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb75280 0x55a1fcb86510 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd304080d80 tx=0x7fd304081620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676ec49b94a219ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd2f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd2f17fa640 1 -- 172.21.15.53:0/1554225782 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040804e0 con 0x55a1fcb75280 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd2f17fa640 1 -- 172.21.15.53:0/1554225782 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fd3040b0dc0 con 0x55a1fcb75280 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.309+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c01a070 con 0x55a1fcb75280 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd2f17fa640 1 -- 172.21.15.53:0/1554225782 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd304085430 con 0x55a1fcb75280 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 10 client.7881 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55a1fc7c03f0 con 0x55a1fcb75280 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd2f17fa640 1 -- 172.21.15.53:0/1554225782 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c01a070 con 0x55a1fcb75280 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd2f17fa640 1 -- 172.21.15.53:0/1554225782 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fd30403a030 con 0x55a1fcb75280 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 10 client.7881 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 10 client.7881 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 20 client.7881 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb71150 0x55a1fcb9f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 4 client.7881 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1fcabcb40 con 0x55a1fcb71150 2024-05-10T10:42:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb71150 0x55a1fcb9f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd2f17fa640 10 client.7881 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.310+0000 7fd311d9b640 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb71150 0x55a1fcb9f500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab58327e2957cf5e server_cookie=967ff23ef9f77b5 in_seq=0 out_seq=0 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd2f17fa640 1 -- 172.21.15.53:0/1554225782 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a1fcabcb40 con 0x55a1fcb71150 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd2f17fa640 10 client.7881 handle_command_reply: tid=0 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 client.7881 shutdown 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb75280 msgr2=0x55a1fcb86510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb75280 0x55a1fcb86510 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd304080d80 tx=0x7fd304081620 comp rx=0 tx=0).stop 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 shutdown_connections 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcb64fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb71150 0x55a1fcb9f500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb92160 0x55a1fcb78400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 --2- 172.21.15.53:0/1554225782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb75280 0x55a1fcb86510 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 >> 172.21.15.53:0/1554225782 conn(0x55a1fcb74b40 msgr2=0x55a1fcb44230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd2f17fa640 0 client.7881 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:34.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 shutdown_connections 2024-05-10T10:42:34.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 1 -- 172.21.15.53:0/1554225782 wait complete. 2024-05-10T10:42:34.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.311+0000 7fd31bc58740 20 client.7881 trim_cache size 0 max 16384 2024-05-10T10:42:34.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 Processor -- start 2024-05-10T10:42:34.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 -- start start 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb75280 0x55a1fcb6ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb76e30 0x55a1fcb6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcb86a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3142021d0 con 0x55a1fcb92160 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3142011b0 con 0x55a1fcb76e30 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd31414d150 con 0x55a1fcb75280 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd312d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcb86a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd312d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcb86a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60458/0 (socket says 172.21.15.53:60458) 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.330+0000 7fd312d9d640 1 -- 172.21.15.53:0/2416722029 learned_addr learned my addr 172.21.15.53:0/2416722029 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd31259c640 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb75280 0x55a1fcb6ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd311d9b640 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb76e30 0x55a1fcb6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd312d9d640 1 -- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb75280 msgr2=0x55a1fcb6ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd312d9d640 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb75280 0x55a1fcb6ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd312d9d640 1 -- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb76e30 msgr2=0x55a1fcb6f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd312d9d640 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb76e30 0x55a1fcb6f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd312d9d640 1 -- 172.21.15.53:0/2416722029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1fcb47ea0 con 0x55a1fcb92160 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd312d9d640 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcb86a80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd30c016000 tx=0x7fd30c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c45db3e4b6ef8f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd2f37fe640 1 -- 172.21.15.53:0/2416722029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0145e0 con 0x55a1fcb92160 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd2f37fe640 1 -- 172.21.15.53:0/2416722029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c01a040 con 0x55a1fcb92160 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.331+0000 7fd2f37fe640 1 -- 172.21.15.53:0/2416722029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0149e0 con 0x55a1fcb92160 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 -- 172.21.15.53:0/2416722029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 msgr2=0x55a1fcb86a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcb86a80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd30c016000 tx=0x7fd30c00c000 comp rx=0 tx=0).stop 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 -- 172.21.15.53:0/2416722029 shutdown_connections 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcb86a80 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb76e30 0x55a1fcb6f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2416722029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb75280 0x55a1fcb6ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 -- 172.21.15.53:0/2416722029 >> 172.21.15.53:0/2416722029 conn(0x55a1fcb74b40 msgr2=0x55a1fcb45830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 -- 172.21.15.53:0/2416722029 shutdown_connections 2024-05-10T10:42:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 -- 172.21.15.53:0/2416722029 wait complete. 2024-05-10T10:42:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 -- start start 2024-05-10T10:42:34.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.332+0000 7fd31bc58740 1 Processor -- start 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcc0bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcba8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd31bc58740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcba8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd31bc58740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd314200b70 con 0x55a1fcb92160 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a1fcb40250 con 0x55a1fcb75280 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd31bc58740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55a1fcc0fc80 con 0x55a1fcb76e30 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd31bc58740 10 client.0 _ll_register_callbacks cb 0x7fd30407d750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:34.339 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.333+0000 7fd312d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcba8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcba8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60462/0 (socket says 172.21.15.53:60462) 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 -- 172.21.15.53:0/2338915026 learned_addr learned my addr 172.21.15.53:0/2338915026 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd311d9b640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcba8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd31259c640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcc0bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 -- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 msgr2=0x55a1fcba8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcba8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 -- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 msgr2=0x55a1fcc0bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcc0bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 -- 172.21.15.53:0/2338915026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1fc7800f0 con 0x55a1fcb92160 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd311d9b640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcba8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd31259c640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcc0bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd312d9d640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcba8eb0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f80 tx=0x7fd30c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be156205e36ebbd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd2f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:34.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd2f17fa640 1 -- 172.21.15.53:0/2338915026 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c037020 con 0x55a1fcb92160 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd2f17fa640 1 -- 172.21.15.53:0/2338915026 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7fd30c033470 con 0x55a1fcb92160 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c01a040 con 0x55a1fcb92160 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd2f17fa640 1 -- 172.21.15.53:0/2338915026 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd30c041050 con 0x55a1fcb92160 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd31bc58740 10 client.7887 fetch_fsmap learned FSMap version 88 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.334+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55a1fcb448a0 con 0x55a1fcb92160 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd2f17fa640 1 -- 172.21.15.53:0/2338915026 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c01a040 con 0x55a1fcb92160 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd2f17fa640 1 -- 172.21.15.53:0/2338915026 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fd30c044020 con 0x55a1fcb92160 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 10 client.7887 fetch_fsmap finished waiting for FSMap version 88 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 10 client.7887 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 20 client.7887 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb75770 0x55a1fcb40250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 4 client.7887 mds_command: new command op to 7749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 --> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a1fcab69b0 con 0x55a1fcb75770 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31259c640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb75770 0x55a1fcb40250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd2f17fa640 10 client.7887 ms_handle_connect on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31259c640 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb75770 0x55a1fcb40250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3418516d00b3093e server_cookie=6b320edd1a151915 in_seq=0 out_seq=0 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd2f17fa640 1 -- 172.21.15.53:0/2338915026 <== mds.0 v2:172.21.15.53:6834/1219905896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a1fcab69b0 con 0x55a1fcb75770 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd2f17fa640 10 client.7887 handle_command_reply: tid=0 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 client.7887 shutdown 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 msgr2=0x55a1fcba8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcba8eb0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f80 tx=0x7fd30c004ab0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 shutdown_connections 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a1fcb92160 0x55a1fcba8eb0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55a1fcb76e30 0x55a1fcba8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.53:6834/1219905896,v1:172.21.15.53:6835/1219905896] conn(0x55a1fcb75770 0x55a1fcb40250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 --2- 172.21.15.53:0/2338915026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1fcb75280 0x55a1fcc0bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 >> 172.21.15.53:0/2338915026 conn(0x55a1fcb74b40 msgr2=0x55a1fcb71710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.335+0000 7fd2f17fa640 0 client.7887 ms_handle_reset on v2:172.21.15.53:6834/1219905896 2024-05-10T10:42:34.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 shutdown_connections 2024-05-10T10:42:34.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd31bc58740 1 -- 172.21.15.53:0/2338915026 wait complete. 2024-05-10T10:42:34.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd31bc58740 20 client.7887 trim_cache size 0 max 16384 2024-05-10T10:42:34.343 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "status": "no active scrubs running", 2024-05-10T10:42:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: "scrubs": {} 2024-05-10T10:42:34.343 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:34.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2f40762d0 msgr2=0x7fd2f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2f40762d0 0x7fd2f4078790 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd31415d290 tx=0x7fd308009040 comp rx=0 tx=0).stop 2024-05-10T10:42:34.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cd500 msgr2=0x7fd314149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cd500 0x7fd314149c40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd304000c00 tx=0x7fd3040046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 shutdown_connections 2024-05-10T10:42:34.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2f40762d0 0x7fd2f4078790 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3140d2190 0x7fd31415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3140cf3e0 0x7fd3140c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 --2- 172.21.15.53:0/1809347951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3140cd500 0x7fd314149c40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 >> 172.21.15.53:0/1809347951 conn(0x7fd3140d2eb0 msgr2=0x7fd3140b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 shutdown_connections 2024-05-10T10:42:34.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.336+0000 7fd318f3f640 1 -- 172.21.15.53:0/1809347951 wait complete. 2024-05-10T10:42:34.348 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T10:42:34.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:42:34.554 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.554+0000 7f3dd9228640 1 Processor -- start 2024-05-10T10:42:34.555 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.554+0000 7f3dd9228640 1 -- start start 2024-05-10T10:42:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd9228640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3dd40c7110 0x7f3dd40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd9228640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd406f1c0 con 0x7f3dd40c6c80 2024-05-10T10:42:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd9228640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd406f4a0 con 0x7f3dd41598c0 2024-05-10T10:42:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd9228640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd406f780 con 0x7f3dd40c7110 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dcbfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3dd40c6c80 0x7f3dd40cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59204/0 (socket says 172.21.15.53:59204) 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dcbfff640 1 -- 172.21.15.53:0/1816059313 learned_addr learned my addr 172.21.15.53:0/1816059313 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd359e640 1 --2- 172.21.15.53:0/1816059313 >> v2:172.21.15.135:3300/0 conn(0x7f3dd40c7110 0x7f3dd40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558012305 0 0) 0x7f3dd406f1c0 con 0x7f3dd40c6c80 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dc0003520 con 0x7f3dd40c6c80 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1744238479 0 0) 0x7f3dc0003520 con 0x7f3dd40c6c80 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 >> v2:172.21.15.135:3300/0 conn(0x7f3dd40c7110 msgr2=0x7f3dd40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 --2- 172.21.15.53:0/1816059313 >> v2:172.21.15.135:3300/0 conn(0x7f3dd40c7110 0x7f3dd40cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 >> v1:172.21.15.135:6789/0 conn(0x7f3dd41598c0 legacy=0x7f3dd406e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd406fa60 con 0x7f3dd40c6c80 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f3dbc001670 con 0x7f3dd40c6c80 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 >> v1:172.21.15.53:6789/0 conn(0x7f3dd40c6c80 legacy=0x7f3dd40cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.555+0000 7f3dd259c640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00039b0 0x7f3dc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd259c640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dc0004800 0x7f3dc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd259c640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dc00051c0 0x7f3dc000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd406f780 con 0x7f3dc00051c0 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc000c060 con 0x7f3dc00039b0 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc000c260 con 0x7f3dc0004800 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd359e640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dc00051c0 0x7f3dc000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd2d9d640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00039b0 0x7f3dc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dcbfff640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dc0004800 0x7f3dc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd359e640 1 -- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dc0004800 msgr2=0x7f3dc0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd359e640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dc0004800 0x7f3dc0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd359e640 1 -- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00039b0 msgr2=0x7f3dc0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd359e640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00039b0 0x7f3dc0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.556+0000 7f3dd359e640 1 -- 172.21.15.53:0/1816059313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc0003520 con 0x7f3dc00051c0 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd359e640 1 -- 172.21.15.53:0/1816059313 reap_dead start 2024-05-10T10:42:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd359e640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dc00051c0 0x7f3dc000b9c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009fc0 tx=0x7f3dc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0f5b28f816b349 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc035020 con 0x7f3dc00051c0 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc003bd0 con 0x7f3dc00051c0 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd259c640 1 -- 172.21.15.53:0/1816059313 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc009c10 con 0x7f3dc00051c0 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd9228640 1 -- 172.21.15.53:0/1816059313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dc00051c0 msgr2=0x7f3dc000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd9228640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dc00051c0 0x7f3dc000b9c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009fc0 tx=0x7f3dc0005830 comp rx=0 tx=0).stop 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd9228640 1 -- 172.21.15.53:0/1816059313 shutdown_connections 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd9228640 1 --2- 172.21.15.53:0/1816059313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dc00051c0 0x7f3dc000b9c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd9228640 1 -- 172.21.15.53:0/1816059313 >> 172.21.15.53:0/1816059313 conn(0x7f3dd40b2900 msgr2=0x7f3dd40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd9228640 1 -- 172.21.15.53:0/1816059313 shutdown_connections 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.557+0000 7f3dd9228640 1 -- 172.21.15.53:0/1816059313 wait complete. 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 Processor -- start 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 -- start start 2024-05-10T10:42:34.564 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00051c0 0x7f3dd415e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dd40cdd00 0x7f3dd414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dd40ce160 0x7f3dd414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd415c6e0 con 0x7f3dd40ce160 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd40c7560 con 0x7f3dc00051c0 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.559+0000 7f3dd9228640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd4159ca0 con 0x7f3dd40cdd00 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dd40ce160 0x7f3dd414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dd40ce160 0x7f3dd414d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60482/0 (socket says 172.21.15.53:60482) 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 -- 172.21.15.53:0/2127245471 learned_addr learned my addr 172.21.15.53:0/2127245471 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dcbfff640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dd40cdd00 0x7f3dd414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd2d9d640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00051c0 0x7f3dd415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 -- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dd40cdd00 msgr2=0x7f3dd414d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dd40cdd00 0x7f3dd414d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 -- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00051c0 msgr2=0x7f3dd415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00051c0 0x7f3dd415e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 -- 172.21.15.53:0/2127245471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd40cc0e0 con 0x7f3dd40ce160 2024-05-10T10:42:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd359e640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dd40ce160 0x7f3dd414d830 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc02af10 tx=0x7f3dcc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dcb7fe640 1 -- 172.21.15.53:0/2127245471 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc035030 con 0x7f3dd40ce160 2024-05-10T10:42:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dcb7fe640 1 -- 172.21.15.53:0/2127245471 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc043070 con 0x7f3dd40ce160 2024-05-10T10:42:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dcb7fe640 1 -- 172.21.15.53:0/2127245471 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc02d070 con 0x7f3dd40ce160 2024-05-10T10:42:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd40cc3c0 con 0x7f3dd40ce160 2024-05-10T10:42:34.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.560+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd40cc980 con 0x7f3dd40ce160 2024-05-10T10:42:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.561+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd406a220 con 0x7f3dd40ce160 2024-05-10T10:42:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.562+0000 7f3dcb7fe640 1 -- 172.21.15.53:0/2127245471 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3dcc005d10 con 0x7f3dd40ce160 2024-05-10T10:42:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.563+0000 7f3dcb7fe640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db4076300 0x7f3db40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.563+0000 7f3dcb7fe640 1 -- 172.21.15.53:0/2127245471 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6245+0+0 (secure 0 0 0) 0x7f3dcc0b7f50 con 0x7f3dd40ce160 2024-05-10T10:42:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.563+0000 7f3dd2d9d640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db4076300 0x7f3db40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.567 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.563+0000 7f3dd2d9d640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db4076300 0x7f3db40787c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4000c00 tx=0x7f3dc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.567+0000 7f3dcb7fe640 1 -- 172.21.15.53:0/2127245471 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3dcc04c020 con 0x7f3dd40ce160 2024-05-10T10:42:34.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.798+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3dd40cf3a0 con 0x7f3dd40ce160 2024-05-10T10:42:34.799 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:42:34.798+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:42:34.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.800+0000 7f3dcb7fe640 1 -- 172.21.15.53:0/2127245471 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7f3dcc0810d0 con 0x7f3dd40ce160 2024-05-10T10:42:34.800 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:42:34.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.802+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db4076300 msgr2=0x7f3db40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.802+0000 7f3dd9228640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db4076300 0x7f3db40787c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4000c00 tx=0x7f3dc4009040 comp rx=0 tx=0).stop 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.802+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dd40ce160 msgr2=0x7f3dd414d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.802+0000 7f3dd9228640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dd40ce160 0x7f3dd414d830 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc02af10 tx=0x7f3dcc009750 comp rx=0 tx=0).stop 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 shutdown_connections 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3db4076300 0x7f3db40787c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dd40ce160 0x7f3dd414d830 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dd40cdd00 0x7f3dd414d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 --2- 172.21.15.53:0/2127245471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dc00051c0 0x7f3dd415e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 >> 172.21.15.53:0/2127245471 conn(0x7f3dd40b2900 msgr2=0x7f3dd4157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 shutdown_connections 2024-05-10T10:42:34.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.803+0000 7f3dd9228640 1 -- 172.21.15.53:0/2127245471 wait complete. 2024-05-10T10:42:34.805 INFO:tasks.ceph.mds.c.smithi053.stderr: -1154> 2024-05-10T10:42:04.003+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-10T10:42:34.805 INFO:tasks.ceph.mds.c.smithi053.stderr: -1151> 2024-05-10T10:42:04.003+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-10T10:42:34.805 INFO:tasks.ceph.mds.c.smithi053.stderr: -1110> 2024-05-10T10:42:04.004+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-10T10:42:34.805 INFO:tasks.ceph.mds.c.smithi053.stderr: -1107> 2024-05-10T10:42:04.004+0000 7f0bf39c2640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-10T10:42:34.805 INFO:tasks.ceph.mds.c.smithi053.stderr: -995> 2024-05-10T10:42:04.349+0000 7f0bf79ca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:42:34.816 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-05-10T10:42:34.832 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:42:34.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 -- start start 2024-05-10T10:42:34.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 Processor -- start 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f01229cc00 0x55f01306a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f01229d800 0x55f01306a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f01229dc00 0x55f01306ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f01307a180 con 0x55f01229d800 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f01307a300 con 0x55f01229cc00 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f01307a480 con 0x55f01229dc00 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a5292640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f01229d800 0x55f01306a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a5292640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f01229d800 0x55f01306a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60502/0 (socket says 172.21.15.53:60502) 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.858+0000 7fa3a5292640 1 -- 172.21.15.53:0/921348505 learned_addr learned my addr 172.21.15.53:0/921348505 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a6294640 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f01229dc00 0x55f01306ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a5a93640 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f01229cc00 0x55f01306a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a5292640 1 -- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f01229dc00 msgr2=0x55f01306ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a5292640 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f01229dc00 0x55f01306ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a5292640 1 -- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f01229cc00 msgr2=0x55f01306a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a5292640 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f01229cc00 0x55f01306a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a5292640 1 -- 172.21.15.53:0/921348505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0122d2780 con 0x55f01229d800 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a6294640 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f01229dc00 0x55f01306ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a5292640 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f01229d800 0x55f01306a580 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x55f0130cb320 tx=0x55f01303e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e46c0f215304eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a328e640 1 -- 172.21.15.53:0/921348505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f0122c2680 con 0x55f01229d800 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a328e640 1 -- 172.21.15.53:0/921348505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f0122c8700 con 0x55f01229d800 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a328e640 1 -- 172.21.15.53:0/921348505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f0122c2b60 con 0x55f01229d800 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a72e7280 1 -- 172.21.15.53:0/921348505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0122d34a0 con 0x55f01229d800 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a72e7280 1 -- 172.21.15.53:0/921348505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55f0122d2000 con 0x55f01229d800 2024-05-10T10:42:34.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a72e7280 4 waiting for MDS map... 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a328e640 1 -- 172.21.15.53:0/921348505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x55f013178600 con 0x55f01229d800 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a328e640 1 -- 172.21.15.53:0/921348505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 89) ==== 2885+0+0 (secure 0 0 0) 0x55f0122f7180 con 0x55f01229d800 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a72e7280 4 Got MDS map 89 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.859+0000 7fa3a72e7280 10 main: main 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.860+0000 7fa3a72e7280 4 main: connecting to RADOS... 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.860+0000 7fa3a72e7280 1 Processor -- start 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.860+0000 7fa3a72e7280 1 -- start start 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175400 0x55f01306b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 0x55f01306bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f01307a600 con 0x55f013175400 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f01307a780 con 0x55f013175c00 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f01307a900 con 0x55f013175800 2024-05-10T10:42:34.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175400 0x55f01306b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175400 0x55f01306b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60518/0 (socket says 172.21.15.53:60518) 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 -- 172.21.15.53:0/1086795857 learned_addr learned my addr 172.21.15.53:0/1086795857 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a5a93640 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a5292640 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 0x55f01306bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 -- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175800 msgr2=0x55f01306b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 -- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 msgr2=0x55f01306bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 0x55f01306bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 -- 172.21.15.53:0/1086795857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0122d23c0 con 0x55f013175400 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a5a93640 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a6294640 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175400 0x55f01306b080 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x55f0131d2600 tx=0x55f01303c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f72ef0387c6e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a128a640 1 -- 172.21.15.53:0/1086795857 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f0122c29c0 con 0x55f013175400 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a128a640 1 -- 172.21.15.53:0/1086795857 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f0122c8fc0 con 0x55f013175400 2024-05-10T10:42:34.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.861+0000 7fa3a128a640 1 -- 172.21.15.53:0/1086795857 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f0122c2680 con 0x55f013175400 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1086795857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175400 msgr2=0x55f01306b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175400 0x55f01306b080 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x55f0131d2600 tx=0x55f01303c660 comp rx=0 tx=0).stop 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1086795857 shutdown_connections 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 0x55f01306bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1086795857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175400 0x55f01306b080 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1086795857 >> 172.21.15.53:0/1086795857 conn(0x55f013175000 msgr2=0x55f012389000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1086795857 shutdown_connections 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1086795857 wait complete. 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 Processor -- start 2024-05-10T10:42:34.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.862+0000 7fa3a72e7280 1 -- start start 2024-05-10T10:42:34.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175400 0x55f01306b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a72e7280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 0x55f01306bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f01307bb00 con 0x55f013175800 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f01307b980 con 0x55f013175c00 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a72e7280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55f01307b800 con 0x55f013175400 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175800 0x55f01306b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60524/0 (socket says 172.21.15.53:60524) 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 -- 172.21.15.53:0/1955682547 learned_addr learned my addr 172.21.15.53:0/1955682547 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a6294640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175400 0x55f01306b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5292640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 0x55f01306bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 -- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175400 msgr2=0x55f01306b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f013175400 0x55f01306b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 -- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 msgr2=0x55f01306bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f013175c00 0x55f01306bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.863+0000 7fa3a5a93640 1 -- 172.21.15.53:0/1955682547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0122d2780 con 0x55f013175800 2024-05-10T10:42:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.864+0000 7fa3a5a93640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175800 0x55f01306b600 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55f0131d3440 tx=0x55f01318f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.864+0000 7fa39fa87640 1 -- 172.21.15.53:0/1955682547 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f0122c29c0 con 0x55f013175800 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.864+0000 7fa39fa87640 1 -- 172.21.15.53:0/1955682547 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f0122c8e00 con 0x55f013175800 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.864+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f0122d34a0 con 0x55f013175800 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.864+0000 7fa39fa87640 1 -- 172.21.15.53:0/1955682547 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f0122c2680 con 0x55f013175800 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.864+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0122d2960 con 0x55f013175800 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.864+0000 7fa3a72e7280 4 main: resolving pool 15 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.865+0000 7fa39fa87640 1 -- 172.21.15.53:0/1955682547 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55f013170c00 con 0x55f013175800 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.865+0000 7fa39fa87640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55f0130a4c00 0x55f01306c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.865+0000 7fa39fa87640 1 -- 172.21.15.53:0/1955682547 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x55f013172a00 con 0x55f013175800 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.866+0000 7fa3a72e7280 4 main: creating IoCtx.. 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.866+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55f0130a5000 0x55f01306c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.866+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107) -- 0x55f0130a5400 con 0x55f0130a5000 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.866+0000 7fa3a5292640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55f0130a5000 0x55f01306c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.866+0000 7fa3a6294640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55f0130a4c00 0x55f01306c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.866+0000 7fa3a5292640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55f0130a5000 0x55f01306c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.866+0000 7fa3a6294640 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55f0130a4c00 0x55f01306c100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55f0131d2600 tx=0x55f01303c4e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a5292640 1 -- 172.21.15.53:0/1955682547 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x55f013024000 con 0x55f0130a5000 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: "0": { 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: "data": { 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: "version": 5, 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: "inotable": { 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: "projected_free": [ 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: "start": 1099511627779, 2024-05-10T10:42:34.873 INFO:teuthology.orchestra.run.smithi053.stdout: "len": 1099511627773 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: "free": [ 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: "start": 1099511627779, 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: "len": 1099511627773 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: "result": 0 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:34.874 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:34.875 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55f0130a5000 msgr2=0x55f01306c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55f0130a5000 0x55f01306c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55f0130a4c00 msgr2=0x55f01306c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55f0130a4c00 0x55f01306c100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55f0131d2600 tx=0x55f01303c4e0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175800 msgr2=0x55f01306b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/1955682547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f013175800 0x55f01306b600 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55f0131d3440 tx=0x55f01318f800 comp rx=0 tx=0).stop 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a5a93640 1 -- 172.21.15.53:0/1955682547 reap_dead start 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 shutdown_connections 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 >> 172.21.15.53:0/1955682547 conn(0x55f013175000 msgr2=0x55f012389000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 shutdown_connections 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.868+0000 7fa3a72e7280 1 -- 172.21.15.53:0/1955682547 wait complete. 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 -- 172.21.15.53:0/921348505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f01229d800 msgr2=0x55f01306a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f01229d800 0x55f01306a580 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x55f0130cb320 tx=0x55f01303e120 comp rx=0 tx=0).stop 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 -- 172.21.15.53:0/921348505 shutdown_connections 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55f01229dc00 0x55f01306ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f01229d800 0x55f01306a580 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 --2- 172.21.15.53:0/921348505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f01229cc00 0x55f01306a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 -- 172.21.15.53:0/921348505 >> 172.21.15.53:0/921348505 conn(0x55f01229d000 msgr2=0x55f012388800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 -- 172.21.15.53:0/921348505 shutdown_connections 2024-05-10T10:42:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:34.869+0000 7fa3a72e7280 1 -- 172.21.15.53:0/921348505 wait complete. 2024-05-10T10:42:34.878 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.061921, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-05-10T10:42:34.878 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:42:34.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:42:34.962 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:42:35.004 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:42:35.004 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:42:35.004 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:35.004 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:35.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:42:35.021 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:42:35.021 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:42:35.021 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:35.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:42:35.063 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:42:35.063 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:35.063 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:35.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:42:35.079 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:42:35.080 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:42:35.080 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:35.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:42:35.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.225+0000 7f2018c93640 1 Processor -- start 2024-05-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.225+0000 7f2018c93640 1 -- start start 2024-05-10T10:42:35.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f2018c93640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f201406c990 0x7f2014163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f2018c93640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2014163ba0 con 0x7f201415f620 2024-05-10T10:42:35.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f2018c93640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2014160190 con 0x7f2014070bd0 2024-05-10T10:42:35.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f2018c93640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f201406cd70 con 0x7f201406c990 2024-05-10T10:42:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f201259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f201406c990 0x7f2014163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f2011d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f201415f620 0x7f20140704c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59212/0 (socket says 172.21.15.53:59212) 2024-05-10T10:42:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f2011d9b640 1 -- 172.21.15.53:0/3051541153 learned_addr learned my addr 172.21.15.53:0/3051541153 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.226+0000 7f201259c640 1 -- 172.21.15.53:0/3051541153 >> v1:172.21.15.135:6789/0 conn(0x7f2014070bd0 legacy=0x7f20140cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.227+0000 7f201259c640 1 -- 172.21.15.53:0/3051541153 >> v1:172.21.15.53:6789/0 conn(0x7f201415f620 legacy=0x7f20140704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.227+0000 7f201259c640 1 -- 172.21.15.53:0/3051541153 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2014160470 con 0x7f201406c990 2024-05-10T10:42:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.227+0000 7f201159a640 1 -- 172.21.15.53:0/3051541153 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 836505466 0 0) 0x7f2014163ba0 con 0x7f201415f620 2024-05-10T10:42:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.227+0000 7f201259c640 1 --2- 172.21.15.53:0/3051541153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201406c990 0x7f2014163660 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f20040097a0 tx=0x7f200402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576566f827024d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.227+0000 7f201159a640 1 -- 172.21.15.53:0/3051541153 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004037070 con 0x7f201406c990 2024-05-10T10:42:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.227+0000 7f201159a640 1 -- 172.21.15.53:0/3051541153 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004032440 con 0x7f201406c990 2024-05-10T10:42:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.227+0000 7f201159a640 1 -- 172.21.15.53:0/3051541153 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004032810 con 0x7f201406c990 2024-05-10T10:42:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.228+0000 7f2018c93640 1 -- 172.21.15.53:0/3051541153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201406c990 msgr2=0x7f2014163660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.228+0000 7f2018c93640 1 --2- 172.21.15.53:0/3051541153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201406c990 0x7f2014163660 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f20040097a0 tx=0x7f200402a230 comp rx=0 tx=0).stop 2024-05-10T10:42:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.228+0000 7f2018c93640 1 -- 172.21.15.53:0/3051541153 shutdown_connections 2024-05-10T10:42:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.228+0000 7f2018c93640 1 --2- 172.21.15.53:0/3051541153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201406c990 0x7f2014163660 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.228+0000 7f2018c93640 1 -- 172.21.15.53:0/3051541153 >> 172.21.15.53:0/3051541153 conn(0x7f20140b3980 msgr2=0x7f20140b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.228+0000 7f2018c93640 1 -- 172.21.15.53:0/3051541153 shutdown_connections 2024-05-10T10:42:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.228+0000 7f2018c93640 1 -- 172.21.15.53:0/3051541153 wait complete. 2024-05-10T10:42:35.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.229+0000 7f2018c93640 1 Processor -- start 2024-05-10T10:42:35.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.229+0000 7f2018c93640 1 -- start start 2024-05-10T10:42:35.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2018c93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201406c990 0x7f20140b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2018c93640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2014070bd0 0x7f20140af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2018c93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201415f620 0x7f20140afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2018c93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2014073e70 con 0x7f201415f620 2024-05-10T10:42:35.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2018c93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2014163140 con 0x7f201406c990 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2018c93640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f201415fab0 con 0x7f2014070bd0 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2012d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201415f620 0x7f20140afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2012d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201415f620 0x7f20140afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60538/0 (socket says 172.21.15.53:60538) 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f2012d9d640 1 -- 172.21.15.53:0/3324122086 learned_addr learned my addr 172.21.15.53:0/3324122086 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.230+0000 7f201259c640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201406c990 0x7f20140b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2011d9b640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2014070bd0 0x7f20140af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2012d9d640 1 -- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2014070bd0 msgr2=0x7f20140af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2012d9d640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2014070bd0 0x7f20140af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2012d9d640 1 -- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201406c990 msgr2=0x7f20140b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2012d9d640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201406c990 0x7f20140b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2012d9d640 1 -- 172.21.15.53:0/3324122086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201414c600 con 0x7f201415f620 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2012d9d640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201415f620 0x7f20140afcb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f200c00c7a0 tx=0x7f200c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2002ffd640 1 -- 172.21.15.53:0/3324122086 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c003b70 con 0x7f201415f620 2024-05-10T10:42:35.233 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2002ffd640 1 -- 172.21.15.53:0/3324122086 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c003d30 con 0x7f201415f620 2024-05-10T10:42:35.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20140b0380 con 0x7f201415f620 2024-05-10T10:42:35.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2002ffd640 1 -- 172.21.15.53:0/3324122086 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c00aa90 con 0x7f201415f620 2024-05-10T10:42:35.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.231+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201414c8a0 con 0x7f201415f620 2024-05-10T10:42:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.232+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201406a220 con 0x7f201415f620 2024-05-10T10:42:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.233+0000 7f2002ffd640 1 -- 172.21.15.53:0/3324122086 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f200c00ac30 con 0x7f201415f620 2024-05-10T10:42:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.233+0000 7f2002ffd640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ff40762d0 0x7f1ff4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.234+0000 7f2002ffd640 1 -- 172.21.15.53:0/3324122086 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7f200c097770 con 0x7f201415f620 2024-05-10T10:42:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.234+0000 7f201259c640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ff40762d0 0x7f1ff4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.234+0000 7f201259c640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ff40762d0 0x7f1ff4078790 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2004034000 tx=0x7f200402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.238+0000 7f2002ffd640 1 -- 172.21.15.53:0/3324122086 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f200c060e60 con 0x7f201415f620 2024-05-10T10:42:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.467+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20140d27c0 con 0x7f201415f620 2024-05-10T10:42:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.467+0000 7f2002ffd640 1 -- 172.21.15.53:0/3324122086 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+4351 (secure 0 0 0) 0x7f200c060760 con 0x7f201415f620 2024-05-10T10:42:35.470 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:35.470 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:42:34.799111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-10T10:42:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 89 2024-05-10T10:42:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.469+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ff40762d0 msgr2=0x7f1ff4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.469+0000 7f2018c93640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ff40762d0 0x7f1ff4078790 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2004034000 tx=0x7f200402e000 comp rx=0 tx=0).stop 2024-05-10T10:42:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.469+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201415f620 msgr2=0x7f20140afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.469+0000 7f2018c93640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201415f620 0x7f20140afcb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f200c00c7a0 tx=0x7f200c00cc60 comp rx=0 tx=0).stop 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 shutdown_connections 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ff40762d0 0x7f1ff4078790 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201415f620 0x7f20140afcb0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2014070bd0 0x7f20140af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 --2- 172.21.15.53:0/3324122086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201406c990 0x7f20140b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 >> 172.21.15.53:0/3324122086 conn(0x7f20140b3980 msgr2=0x7f20140d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 shutdown_connections 2024-05-10T10:42:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.470+0000 7f2018c93640 1 -- 172.21.15.53:0/3324122086 wait complete. 2024-05-10T10:42:35.482 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:35.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.585+0000 7f6215c15640 1 Processor -- start 2024-05-10T10:42:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.585+0000 7f6215c15640 1 -- start start 2024-05-10T10:42:35.587 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f6215c15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6210162750 0x7f62100d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f6215c15640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f62100d5cb0 0x7f62100d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f6215c15640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162b30 0x7f62100d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f6215c15640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f62100b4f20 con 0x7f6210162750 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f6215c15640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f62100d0fe0 con 0x7f62100d5cb0 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f6215c15640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f62100b45f0 con 0x7f6210162b30 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6210162750 0x7f62100d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6210162750 0x7f62100d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60540/0 (socket says 172.21.15.53:60540) 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 -- 172.21.15.53:0/2252447558 learned_addr learned my addr 172.21.15.53:0/2252447558 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620ffff640 1 --2- 172.21.15.53:0/2252447558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162b30 0x7f62100d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620effd640 1 --2- 172.21.15.53:0/2252447558 >> v2:172.21.15.135:3300/0 conn(0x7f62100d5cb0 0x7f62100d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 -- 172.21.15.53:0/2252447558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162b30 msgr2=0x7f62100d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 --2- 172.21.15.53:0/2252447558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162b30 0x7f62100d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.588 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 -- 172.21.15.53:0/2252447558 >> v2:172.21.15.135:3300/0 conn(0x7f62100d5cb0 msgr2=0x7f62100d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:35.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 --2- 172.21.15.53:0/2252447558 >> v2:172.21.15.135:3300/0 conn(0x7f62100d5cb0 0x7f62100d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.586+0000 7f620f7fe640 1 -- 172.21.15.53:0/2252447558 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62100b48d0 con 0x7f6210162750 2024-05-10T10:42:35.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.587+0000 7f620f7fe640 1 --2- 172.21.15.53:0/2252447558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6210162750 0x7f62100d0670 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f62100b4e90 tx=0x7f620002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8c5f45d556243c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.587+0000 7f620e7fc640 1 -- 172.21.15.53:0/2252447558 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200037070 con 0x7f6210162750 2024-05-10T10:42:35.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.587+0000 7f620e7fc640 1 -- 172.21.15.53:0/2252447558 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6200032440 con 0x7f6210162750 2024-05-10T10:42:35.589 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.587+0000 7f620e7fc640 1 -- 172.21.15.53:0/2252447558 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200032810 con 0x7f6210162750 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 -- 172.21.15.53:0/2252447558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6210162750 msgr2=0x7f62100d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 --2- 172.21.15.53:0/2252447558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6210162750 0x7f62100d0670 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f62100b4e90 tx=0x7f620002a140 comp rx=0 tx=0).stop 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 -- 172.21.15.53:0/2252447558 shutdown_connections 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 --2- 172.21.15.53:0/2252447558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162b30 0x7f62100d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 --2- 172.21.15.53:0/2252447558 >> v2:172.21.15.135:3300/0 conn(0x7f62100d5cb0 0x7f62100d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 --2- 172.21.15.53:0/2252447558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6210162750 0x7f62100d0670 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 -- 172.21.15.53:0/2252447558 >> 172.21.15.53:0/2252447558 conn(0x7f621009e3b0 msgr2=0x7f62100d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 -- 172.21.15.53:0/2252447558 shutdown_connections 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.588+0000 7f6215c15640 1 -- 172.21.15.53:0/2252447558 wait complete. 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.589+0000 7f6215c15640 1 Processor -- start 2024-05-10T10:42:35.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.589+0000 7f6215c15640 1 -- start start 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f6215c15640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162750 0x7f62101515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f6215c15640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6210162b30 0x7f62100ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f6215c15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100cc120 0x7f62100b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f6215c15640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62100d0d30 con 0x7f62100cc120 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f6215c15640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62100d0bb0 con 0x7f6210162b30 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f6215c15640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f62100d1660 con 0x7f6210162750 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100cc120 0x7f62100b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100cc120 0x7f62100b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60554/0 (socket says 172.21.15.53:60554) 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 -- 172.21.15.53:0/629524292 learned_addr learned my addr 172.21.15.53:0/629524292 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620f7fe640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162750 0x7f62101515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620effd640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6210162b30 0x7f62100ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 -- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162750 msgr2=0x7f62101515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162750 0x7f62101515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 -- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6210162b30 msgr2=0x7f62100ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6210162b30 0x7f62100ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 -- 172.21.15.53:0/629524292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62101f8340 con 0x7f62100cc120 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.590+0000 7f620ffff640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100cc120 0x7f62100b3560 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6208007e60 tx=0x7f620800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.591+0000 7f61f7fff640 1 -- 172.21.15.53:0/629524292 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208014420 con 0x7f62100cc120 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.591+0000 7f61f7fff640 1 -- 172.21.15.53:0/629524292 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6208019070 con 0x7f62100cc120 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.591+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62101f8620 con 0x7f62100cc120 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.591+0000 7f61f7fff640 1 -- 172.21.15.53:0/629524292 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208014820 con 0x7f62100cc120 2024-05-10T10:42:35.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.591+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62101f8be0 con 0x7f62100cc120 2024-05-10T10:42:35.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.592+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621006a220 con 0x7f62100cc120 2024-05-10T10:42:35.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.592+0000 7f61f7fff640 1 -- 172.21.15.53:0/629524292 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62080149c0 con 0x7f62100cc120 2024-05-10T10:42:35.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.593+0000 7f61f7fff640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61fc0762d0 0x7f61fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.593+0000 7f61f7fff640 1 -- 172.21.15.53:0/629524292 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7f6208097f80 con 0x7f62100cc120 2024-05-10T10:42:35.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.593+0000 7f620f7fe640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61fc0762d0 0x7f61fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.593+0000 7f620f7fe640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61fc0762d0 0x7f61fc078790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6200034000 tx=0x7f620002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.597+0000 7f61f7fff640 1 -- 172.21.15.53:0/629524292 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6208061020 con 0x7f62100cc120 2024-05-10T10:42:35.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.830+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62101f8de0 con 0x7f62100cc120 2024-05-10T10:42:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.830+0000 7f61f7fff640 1 -- 172.21.15.53:0/629524292 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5176 (secure 0 0 0) 0x7f6208065320 con 0x7f62100cc120 2024-05-10T10:42:35.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:35.831 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1980081186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1980081186},{"type":"v1","addr":"172.21.15.53:6835","nonce":1980081186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:42:34.799111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-10T10:42:35.831 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 90 2024-05-10T10:42:35.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61fc0762d0 msgr2=0x7f61fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61fc0762d0 0x7f61fc078790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6200034000 tx=0x7f620002e000 comp rx=0 tx=0).stop 2024-05-10T10:42:35.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100cc120 msgr2=0x7f62100b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100cc120 0x7f62100b3560 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6208007e60 tx=0x7f620800e4b0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 shutdown_connections 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f61fc0762d0 0x7f61fc078790 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62100cc120 0x7f62100b3560 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6210162b30 0x7f62100ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 --2- 172.21.15.53:0/629524292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6210162750 0x7f62101515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 >> 172.21.15.53:0/629524292 conn(0x7f621009e3b0 msgr2=0x7f62100d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 shutdown_connections 2024-05-10T10:42:35.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.832+0000 7f6215c15640 1 -- 172.21.15.53:0/629524292 wait complete. 2024-05-10T10:42:35.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:42:35.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.947+0000 7f9c78904640 1 Processor -- start 2024-05-10T10:42:35.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.947+0000 7f9c78904640 1 -- start start 2024-05-10T10:42:35.948 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.948+0000 7f9c78904640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9c7415ce60 0x7f9c740cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.948+0000 7f9c78904640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c741611e0 0x7f9c7415d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.948+0000 7f9c78904640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c740c6a60 con 0x7f9c740c65f0 2024-05-10T10:42:35.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.948+0000 7f9c78904640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9c740cd090 con 0x7f9c7415ce60 2024-05-10T10:42:35.949 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.948+0000 7f9c78904640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9c740cd370 con 0x7f9c741611e0 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c7259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9c7415ce60 0x7f9c740cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c72d9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c741611e0 0x7f9c7415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c7259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9c7415ce60 0x7f9c740cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:40532/0 (socket says 172.21.15.53:40532) 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c7259c640 1 -- 172.21.15.53:0/617686157 learned_addr learned my addr 172.21.15.53:0/617686157 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c7259c640 1 -- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c741611e0 msgr2=0x7f9c7415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c7259c640 1 --2- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c741611e0 0x7f9c7415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c7259c640 1 -- 172.21.15.53:0/617686157 >> v1:172.21.15.135:6789/0 conn(0x7f9c740c65f0 legacy=0x7f9c74160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c7259c640 1 -- 172.21.15.53:0/617686157 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c740cd650 con 0x7f9c7415ce60 2024-05-10T10:42:35.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.949+0000 7f9c72d9d640 1 --2- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c741611e0 0x7f9c7415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.950+0000 7f9c7259c640 1 --2- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c7415ce60 0x7f9c740cb300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009e50 tx=0x7f9c6402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6f055367d16e1ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.950+0000 7f9c7159a640 1 -- 172.21.15.53:0/617686157 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64036070 con 0x7f9c7415ce60 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.950+0000 7f9c7159a640 1 -- 172.21.15.53:0/617686157 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c64031600 con 0x7f9c7415ce60 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.950+0000 7f9c7159a640 1 -- 172.21.15.53:0/617686157 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640319d0 con 0x7f9c7415ce60 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.950+0000 7f9c78904640 1 -- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c7415ce60 msgr2=0x7f9c740cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.950+0000 7f9c78904640 1 --2- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c7415ce60 0x7f9c740cb300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009e50 tx=0x7f9c6402a680 comp rx=0 tx=0).stop 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.951+0000 7f9c78904640 1 -- 172.21.15.53:0/617686157 shutdown_connections 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.951+0000 7f9c78904640 1 --2- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c741611e0 0x7f9c7415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.951+0000 7f9c78904640 1 --2- 172.21.15.53:0/617686157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c7415ce60 0x7f9c740cb300 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.951+0000 7f9c78904640 1 -- 172.21.15.53:0/617686157 >> 172.21.15.53:0/617686157 conn(0x7f9c740b2900 msgr2=0x7f9c740b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.951+0000 7f9c78904640 1 -- 172.21.15.53:0/617686157 shutdown_connections 2024-05-10T10:42:35.952 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.951+0000 7f9c78904640 1 -- 172.21.15.53:0/617686157 wait complete. 2024-05-10T10:42:35.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.952+0000 7f9c78904640 1 Processor -- start 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.952+0000 7f9c78904640 1 -- start start 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c78904640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c740c65f0 0x7f9c74157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c78904640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415ce60 0x7f9c7409bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c78904640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c741611e0 0x7f9c7409df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c78904640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9c740cde50 con 0x7f9c7415ce60 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c78904640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c740ccd60 con 0x7f9c741611e0 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c78904640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9c740cdfd0 con 0x7f9c740c65f0 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415ce60 0x7f9c7409bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415ce60 0x7f9c7409bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60566/0 (socket says 172.21.15.53:60566) 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 -- 172.21.15.53:0/1879004528 learned_addr learned my addr 172.21.15.53:0/1879004528 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c7259c640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c740c65f0 0x7f9c74157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c72d9d640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c741611e0 0x7f9c7409df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 -- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c740c65f0 msgr2=0x7f9c74157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c740c65f0 0x7f9c74157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 -- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c741611e0 msgr2=0x7f9c7409df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c741611e0 0x7f9c7409df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 -- 172.21.15.53:0/1879004528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7409c910 con 0x7f9c7415ce60 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.953+0000 7f9c71d9b640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415ce60 0x7f9c7409bec0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00c430 tx=0x7f9c5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.954+0000 7f9c6affd640 1 -- 172.21.15.53:0/1879004528 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c017070 con 0x7f9c7415ce60 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.954+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c740c7e20 con 0x7f9c7415ce60 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.954+0000 7f9c6affd640 1 -- 172.21.15.53:0/1879004528 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c5c0052d0 con 0x7f9c7415ce60 2024-05-10T10:42:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.954+0000 7f9c6affd640 1 -- 172.21.15.53:0/1879004528 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c003850 con 0x7f9c7415ce60 2024-05-10T10:42:35.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.954+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c740c83e0 con 0x7f9c7415ce60 2024-05-10T10:42:35.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.955+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7406a220 con 0x7f9c7415ce60 2024-05-10T10:42:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.955+0000 7f9c6affd640 1 -- 172.21.15.53:0/1879004528 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9c5c01a070 con 0x7f9c7415ce60 2024-05-10T10:42:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.956+0000 7f9c6affd640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9c540762d0 0x7f9c54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.956+0000 7f9c6affd640 1 -- 172.21.15.53:0/1879004528 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7f9c5c096b00 con 0x7f9c7415ce60 2024-05-10T10:42:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.956+0000 7f9c7259c640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9c540762d0 0x7f9c54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.957+0000 7f9c7259c640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9c540762d0 0x7f9c54078790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9c64033000 tx=0x7f9c64009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:35.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:35.960+0000 7f9c6affd640 1 -- 172.21.15.53:0/1879004528 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9c5c05fb20 con 0x7f9c7415ce60 2024-05-10T10:42:36.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.152+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c7415dea0 con 0x7f9c7415ce60 2024-05-10T10:42:36.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.153+0000 7f9c6affd640 1 -- 172.21.15.53:0/1879004528 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+17241 (secure 0 0 0) 0x7f9c5c068160 con 0x7f9c7415ce60 2024-05-10T10:42:36.154 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:36.154 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":107,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:42:34.798937+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:41:12.805545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T10:41:14.096305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]},{"pgid":"16.31","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/1219905896":"2024-05-11T10:42:34.798914+0000","172.21.15.53:6834/1219905896":"2024-05-11T10:42:34.798914+0000","172.21.15.53:6835/4122299129":"2024-05-11T10:41:58.338869+0000","172.21.15.53:6834/4122299129":"2024-05-11T10:41:58.338869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:42:36.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9c540762d0 msgr2=0x7f9c54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9c540762d0 0x7f9c54078790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9c64033000 tx=0x7f9c64009000 comp rx=0 tx=0).stop 2024-05-10T10:42:36.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415ce60 msgr2=0x7f9c7409bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415ce60 0x7f9c7409bec0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00c430 tx=0x7f9c5c00c830 comp rx=0 tx=0).stop 2024-05-10T10:42:36.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 shutdown_connections 2024-05-10T10:42:36.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9c540762d0 0x7f9c54078790 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c741611e0 0x7f9c7409df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415ce60 0x7f9c7409bec0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 --2- 172.21.15.53:0/1879004528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9c740c65f0 0x7f9c74157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 >> 172.21.15.53:0/1879004528 conn(0x7f9c740b2900 msgr2=0x7f9c740ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:36.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 shutdown_connections 2024-05-10T10:42:36.157 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.155+0000 7f9c78904640 1 -- 172.21.15.53:0/1879004528 wait complete. 2024-05-10T10:42:36.167 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:42:36.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.271+0000 7f214edda640 1 Processor -- start 2024-05-10T10:42:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.272+0000 7f214edda640 1 -- start start 2024-05-10T10:42:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.272+0000 7f214edda640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f214815f290 0x7f21480d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.272+0000 7f214edda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.272+0000 7f214edda640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214806f3c0 con 0x7f21480702e0 2024-05-10T10:42:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.272+0000 7f214edda640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f21480d14c0 con 0x7f214815f290 2024-05-10T10:42:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.272+0000 7f214edda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f21480d2300 con 0x7f21480d3450 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.272+0000 7f214cb76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f214815f290 0x7f21480d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214cb76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f214815f290 0x7f21480d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60576/0 (socket says 172.21.15.53:60576) 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f2147fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f21480702e0 0x7f21480d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59224/0 (socket says 172.21.15.53:59224) 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214cb76640 1 -- 172.21.15.53:0/1675760886 learned_addr learned my addr 172.21.15.53:0/1675760886 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214d377640 1 --2- 172.21.15.53:0/1675760886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214cb76640 1 -- 172.21.15.53:0/1675760886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 msgr2=0x7f214806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214cb76640 1 --2- 172.21.15.53:0/1675760886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214806d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214cb76640 1 -- 172.21.15.53:0/1675760886 >> v1:172.21.15.53:6789/0 conn(0x7f21480702e0 legacy=0x7f21480d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214cb76640 1 -- 172.21.15.53:0/1675760886 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21480d16f0 con 0x7f214815f290 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f214d377640 1 --2- 172.21.15.53:0/1675760886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214806d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.273+0000 7f21477fe640 1 -- 172.21.15.53:0/1675760886 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851893898 0 0) 0x7f214806f3c0 con 0x7f21480702e0 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214cb76640 1 --2- 172.21.15.53:0/1675760886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214815f290 0x7f21480d0f80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f2140009e50 tx=0x7f214002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65893ed914d6e61f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f21477fe640 1 -- 172.21.15.53:0/1675760886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140036070 con 0x7f214815f290 2024-05-10T10:42:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f21477fe640 1 -- 172.21.15.53:0/1675760886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2140031600 con 0x7f214815f290 2024-05-10T10:42:36.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f21477fe640 1 -- 172.21.15.53:0/1675760886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21400319d0 con 0x7f214815f290 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 -- 172.21.15.53:0/1675760886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214815f290 msgr2=0x7f21480d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 --2- 172.21.15.53:0/1675760886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214815f290 0x7f21480d0f80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f2140009e50 tx=0x7f214002a7a0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 -- 172.21.15.53:0/1675760886 shutdown_connections 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 --2- 172.21.15.53:0/1675760886 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214806d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 --2- 172.21.15.53:0/1675760886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214815f290 0x7f21480d0f80 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 -- 172.21.15.53:0/1675760886 >> 172.21.15.53:0/1675760886 conn(0x7f214809ba20 msgr2=0x7f214809be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 -- 172.21.15.53:0/1675760886 shutdown_connections 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.274+0000 7f214edda640 1 -- 172.21.15.53:0/1675760886 wait complete. 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 Processor -- start 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 -- start start 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21480702e0 0x7f21481f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214809eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f214815f290 0x7f21481f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f21480d1f90 con 0x7f21480702e0 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f21480d1cb0 con 0x7f214815f290 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.276+0000 7f214edda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f214815f670 con 0x7f21480d3450 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21480702e0 0x7f21481f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21480702e0 0x7f21481f84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60586/0 (socket says 172.21.15.53:60586) 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 -- 172.21.15.53:0/3075752834 learned_addr learned my addr 172.21.15.53:0/3075752834 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f2147fff640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214809eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214d377640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f214815f290 0x7f21481f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 -- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 msgr2=0x7f214809eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214809eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 -- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f214815f290 msgr2=0x7f21481f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f214815f290 0x7f21481f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 -- 172.21.15.53:0/3075752834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214809f530 con 0x7f21480702e0 2024-05-10T10:42:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214d377640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f214815f290 0x7f21481f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:36.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214cb76640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21480702e0 0x7f21481f84e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2140034040 tx=0x7f2140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:36.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f21457fa640 1 -- 172.21.15.53:0/3075752834 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140036050 con 0x7f21480702e0 2024-05-10T10:42:36.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f21457fa640 1 -- 172.21.15.53:0/3075752834 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2140037040 con 0x7f21480702e0 2024-05-10T10:42:36.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21480c7b50 con 0x7f21480702e0 2024-05-10T10:42:36.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f21457fa640 1 -- 172.21.15.53:0/3075752834 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140009510 con 0x7f21480702e0 2024-05-10T10:42:36.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.277+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21480c8110 con 0x7f21480702e0 2024-05-10T10:42:36.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.278+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21480693f0 con 0x7f21480702e0 2024-05-10T10:42:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.279+0000 7f21457fa640 1 -- 172.21.15.53:0/3075752834 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2140044020 con 0x7f21480702e0 2024-05-10T10:42:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.279+0000 7f21457fa640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2128076300 0x7f21280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.279+0000 7f21457fa640 1 -- 172.21.15.53:0/3075752834 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7f21400b7ca0 con 0x7f21480702e0 2024-05-10T10:42:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.280+0000 7f2147fff640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2128076300 0x7f21280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.280+0000 7f2147fff640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2128076300 0x7f21280787c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f214809c7e0 tx=0x7f2138007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:36.284 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.284+0000 7f21457fa640 1 -- 172.21.15.53:0/3075752834 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f214004b020 con 0x7f21480702e0 2024-05-10T10:42:36.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.513+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2148161a90 con 0x7f21480702e0 2024-05-10T10:42:36.514 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.513+0000 7f21457fa640 1 -- 172.21.15.53:0/3075752834 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5176 (secure 0 0 0) 0x7f2140081420 con 0x7f21480702e0 2024-05-10T10:42:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:36.514 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1980081186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1980081186},{"type":"v1","addr":"172.21.15.53:6835","nonce":1980081186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:42:34.799111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-10T10:42:36.514 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 90 2024-05-10T10:42:36.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2128076300 msgr2=0x7f21280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2128076300 0x7f21280787c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f214809c7e0 tx=0x7f2138007c00 comp rx=0 tx=0).stop 2024-05-10T10:42:36.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21480702e0 msgr2=0x7f21481f84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.516 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21480702e0 0x7f21481f84e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2140034040 tx=0x7f2140004a40 comp rx=0 tx=0).stop 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 shutdown_connections 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2128076300 0x7f21280787c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f214815f290 0x7f21481f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21480d3450 0x7f214809eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 --2- 172.21.15.53:0/3075752834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21480702e0 0x7f21481f84e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 >> 172.21.15.53:0/3075752834 conn(0x7f214809ba20 msgr2=0x7f2148070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.515+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 shutdown_connections 2024-05-10T10:42:36.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.516+0000 7f214edda640 1 -- 172.21.15.53:0/3075752834 wait complete. 2024-05-10T10:42:36.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.631+0000 7fac12425640 1 Processor -- start 2024-05-10T10:42:36.632 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.631+0000 7fac12425640 1 -- start start 2024-05-10T10:42:36.634 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac12425640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fac0c0b3910 0x7fac0c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac12425640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0cf3b0 0x7fac0c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac12425640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac0c160c90 con 0x7fac0c1608b0 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac12425640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fac0c0ce5f0 con 0x7fac0c0b3910 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac12425640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fac0c0ce8d0 con 0x7fac0c0cf3b0 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fac0c0b3910 0x7fac0c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fac0c1608b0 0x7fac0c0d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59226/0 (socket says 172.21.15.53:59226) 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0bfff640 1 -- 172.21.15.53:0/967283079 learned_addr learned my addr 172.21.15.53:0/967283079 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac109c2640 1 --2- 172.21.15.53:0/967283079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0cf3b0 0x7fac0c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0b7fe640 1 -- 172.21.15.53:0/967283079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0cf3b0 msgr2=0x7fac0c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0b7fe640 1 --2- 172.21.15.53:0/967283079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0cf3b0 0x7fac0c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0b7fe640 1 -- 172.21.15.53:0/967283079 >> v1:172.21.15.53:6789/0 conn(0x7fac0c1608b0 legacy=0x7fac0c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0b7fe640 1 -- 172.21.15.53:0/967283079 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0c0cebb0 con 0x7fac0c0b3910 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.632+0000 7fac0affd640 1 -- 172.21.15.53:0/967283079 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98850501 0 0) 0x7fabfc001110 con 0x7fac0c1608b0 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.633+0000 7fac0b7fe640 1 --2- 172.21.15.53:0/967283079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c0b3910 0x7fac0c0d1400 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d034674b001b33d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.633+0000 7fac0affd640 1 -- 172.21.15.53:0/967283079 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00037070 con 0x7fac0c0b3910 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.633+0000 7fac0affd640 1 -- 172.21.15.53:0/967283079 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac00032440 con 0x7fac0c0b3910 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.633+0000 7fac0affd640 1 -- 172.21.15.53:0/967283079 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00032810 con 0x7fac0c0b3910 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 -- 172.21.15.53:0/967283079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c0b3910 msgr2=0x7fac0c0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 --2- 172.21.15.53:0/967283079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c0b3910 0x7fac0c0d1400 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fac000097a0 tx=0x7fac0002a180 comp rx=0 tx=0).stop 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 -- 172.21.15.53:0/967283079 shutdown_connections 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 --2- 172.21.15.53:0/967283079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0cf3b0 0x7fac0c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 --2- 172.21.15.53:0/967283079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c0b3910 0x7fac0c0d1400 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 -- 172.21.15.53:0/967283079 >> 172.21.15.53:0/967283079 conn(0x7fac0c0d2df0 msgr2=0x7fac0c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:36.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 -- 172.21.15.53:0/967283079 shutdown_connections 2024-05-10T10:42:36.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.634+0000 7fac12425640 1 -- 172.21.15.53:0/967283079 wait complete. 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.635+0000 7fac12425640 1 Processor -- start 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.635+0000 7fac12425640 1 -- start start 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac12425640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0b3910 0x7fac0c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac12425640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0c0cf3b0 0x7fac0c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac12425640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c1608b0 0x7fac0c0c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac12425640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c0d1f10 con 0x7fac0c1608b0 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac12425640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c0d1d90 con 0x7fac0c0cf3b0 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac12425640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fac0c0d1940 con 0x7fac0c0b3910 2024-05-10T10:42:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac109c2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c1608b0 0x7fac0c0c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac109c2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c1608b0 0x7fac0c0c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60606/0 (socket says 172.21.15.53:60606) 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac109c2640 1 -- 172.21.15.53:0/2207542004 learned_addr learned my addr 172.21.15.53:0/2207542004 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac0bfff640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0b3910 0x7fac0c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac0b7fe640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0c0cf3b0 0x7fac0c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac109c2640 1 -- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0b3910 msgr2=0x7fac0c0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.636+0000 7fac109c2640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0b3910 0x7fac0c0c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac109c2640 1 -- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0c0cf3b0 msgr2=0x7fac0c1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac109c2640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0c0cf3b0 0x7fac0c1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac109c2640 1 -- 172.21.15.53:0/2207542004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0c1f4600 con 0x7fac0c1608b0 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac0bfff640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0b3910 0x7fac0c0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac0b7fe640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0c0cf3b0 0x7fac0c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac109c2640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c1608b0 0x7fac0c0c8360 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fac04007e60 tx=0x7fac0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:36.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac08ff9640 1 -- 172.21.15.53:0/2207542004 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04014420 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac08ff9640 1 -- 172.21.15.53:0/2207542004 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac04019070 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0c0c7b80 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac08ff9640 1 -- 172.21.15.53:0/2207542004 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04014820 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.637+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac0c0ca060 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.638+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac0c06a220 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.638+0000 7fac08ff9640 1 -- 172.21.15.53:0/2207542004 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac040149c0 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.639+0000 7fac08ff9640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabf80762d0 0x7fabf8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.639+0000 7fac08ff9640 1 -- 172.21.15.53:0/2207542004 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7fac040980e0 con 0x7fac0c1608b0 2024-05-10T10:42:36.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.639+0000 7fac0bfff640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabf80762d0 0x7fabf8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.643 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.640+0000 7fac0bfff640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabf80762d0 0x7fabf8078790 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fabfc000f60 tx=0x7fabfc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:36.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.643+0000 7fac08ff9640 1 -- 172.21.15.53:0/2207542004 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fac0c1f4600 con 0x7fac0c1608b0 2024-05-10T10:42:36.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.873+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac0c0c6290 con 0x7fac0c1608b0 2024-05-10T10:42:36.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.874+0000 7fac08ff9640 1 -- 172.21.15.53:0/2207542004 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5176 (secure 0 0 0) 0x7fac04061170 con 0x7fac0c1608b0 2024-05-10T10:42:36.875 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:36.875 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1980081186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1980081186},{"type":"v1","addr":"172.21.15.53:6835","nonce":1980081186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:41:15.140801+0000","modified":"2024-05-10T10:42:34.799111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-10T10:42:36.875 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 90 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabf80762d0 msgr2=0x7fabf8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabf80762d0 0x7fabf8078790 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fabfc000f60 tx=0x7fabfc009040 comp rx=0 tx=0).stop 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c1608b0 msgr2=0x7fac0c0c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c1608b0 0x7fac0c0c8360 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fac04007e60 tx=0x7fac0400e4b0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 shutdown_connections 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabf80762d0 0x7fabf8078790 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac0c1608b0 0x7fac0c0c8360 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0c0cf3b0 0x7fac0c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 --2- 172.21.15.53:0/2207542004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac0c0b3910 0x7fac0c0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 >> 172.21.15.53:0/2207542004 conn(0x7fac0c0d2df0 msgr2=0x7fac0c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:36.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 shutdown_connections 2024-05-10T10:42:36.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.876+0000 7fac12425640 1 -- 172.21.15.53:0/2207542004 wait complete. 2024-05-10T10:42:36.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:42:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.992+0000 7fabfc415640 1 Processor -- start 2024-05-10T10:42:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.992+0000 7fabfc415640 1 -- start start 2024-05-10T10:42:36.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabfc415640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fabf40b5520 0x7fabf40d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabfc415640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fabf40b5900 0x7fabf40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:36.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabfc415640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf40d41f0 con 0x7fabf40b62e0 2024-05-10T10:42:36.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabfc415640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fabf40d5c60 con 0x7fabf40b5900 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabfc415640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fabf40d5eb0 con 0x7fabf40b5520 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabf99b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fabf40b5900 0x7fabf40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabf99b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fabf40b5900 0x7fabf40d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60616/0 (socket says 172.21.15.53:60616) 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabf99b0640 1 -- 172.21.15.53:0/2134833088 learned_addr learned my addr 172.21.15.53:0/2134833088 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.993+0000 7fabfa1b1640 1 --2- 172.21.15.53:0/2134833088 >> v2:172.21.15.135:3300/0 conn(0x7fabf40b5520 0x7fabf40d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.994+0000 7fabf99b0640 1 -- 172.21.15.53:0/2134833088 >> v2:172.21.15.135:3300/0 conn(0x7fabf40b5520 msgr2=0x7fabf40d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.994+0000 7fabf99b0640 1 --2- 172.21.15.53:0/2134833088 >> v2:172.21.15.135:3300/0 conn(0x7fabf40b5520 0x7fabf40d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.994+0000 7fabf99b0640 1 -- 172.21.15.53:0/2134833088 >> v1:172.21.15.135:6789/0 conn(0x7fabf40b62e0 legacy=0x7fabf40d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:36.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.994+0000 7fabf99b0640 1 -- 172.21.15.53:0/2134833088 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf40d0a30 con 0x7fabf40b5900 2024-05-10T10:42:36.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.994+0000 7fabf99b0640 1 --2- 172.21.15.53:0/2134833088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40d1de0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fabf4074950 tx=0x7fabe402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a59fc9a0a758fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:36.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.994+0000 7fabf91af640 1 -- 172.21.15.53:0/2134833088 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4037070 con 0x7fabf40b5900 2024-05-10T10:42:36.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabf91af640 1 -- 172.21.15.53:0/2134833088 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe4032440 con 0x7fabf40b5900 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabf91af640 1 -- 172.21.15.53:0/2134833088 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4032810 con 0x7fabf40b5900 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 -- 172.21.15.53:0/2134833088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 msgr2=0x7fabf40d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 --2- 172.21.15.53:0/2134833088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40d1de0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fabf4074950 tx=0x7fabe402a230 comp rx=0 tx=0).stop 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 -- 172.21.15.53:0/2134833088 shutdown_connections 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 --2- 172.21.15.53:0/2134833088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40d1de0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 --2- 172.21.15.53:0/2134833088 >> v2:172.21.15.135:3300/0 conn(0x7fabf40b5520 0x7fabf40d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 -- 172.21.15.53:0/2134833088 >> 172.21.15.53:0/2134833088 conn(0x7fabf40d5170 msgr2=0x7fabf40d5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 -- 172.21.15.53:0/2134833088 shutdown_connections 2024-05-10T10:42:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.995+0000 7fabfc415640 1 -- 172.21.15.53:0/2134833088 wait complete. 2024-05-10T10:42:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.996+0000 7fabfc415640 1 Processor -- start 2024-05-10T10:42:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.996+0000 7fabfc415640 1 -- start start 2024-05-10T10:42:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabfc415640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf40b5520 0x7fabf4154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabfc415640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabfc415640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf40b62e0 0x7fabf406e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabfc415640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fabf40b9300 con 0x7fabf40b5900 2024-05-10T10:42:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabfc415640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fabf40b9180 con 0x7fabf40b62e0 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabfc415640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fabf40b9480 con 0x7fabf40b5520 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabf99b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabf99b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60624/0 (socket says 172.21.15.53:60624) 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabf99b0640 1 -- 172.21.15.53:0/2150963589 learned_addr learned my addr 172.21.15.53:0/2150963589 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.997+0000 7fabfa1b1640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf40b5520 0x7fabf4154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabf99b0640 1 -- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf40b5520 msgr2=0x7fabf4154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabf99b0640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf40b5520 0x7fabf4154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabfa9b2640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf40b62e0 0x7fabf406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabf99b0640 1 -- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf40b62e0 msgr2=0x7fabf406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabf99b0640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf40b62e0 0x7fabf406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabf99b0640 1 -- 172.21.15.53:0/2150963589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41f4d20 con 0x7fabf40b5900 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabfa9b2640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf40b62e0 0x7fabf406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabf99b0640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40a19b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fabe4000c00 tx=0x7fabe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabeaffd640 1 -- 172.21.15.53:0/2150963589 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4041020 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabeaffd640 1 -- 172.21.15.53:0/2150963589 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe40306c0 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabeaffd640 1 -- 172.21.15.53:0/2150963589 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4037040 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41f4ff0 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.998+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41f55b0 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:36.999+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf40693f0 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.000+0000 7fabeaffd640 1 -- 172.21.15.53:0/2150963589 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fabe4044020 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.000+0000 7fabeaffd640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabd80762d0 0x7fabd8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.000+0000 7fabeaffd640 1 -- 172.21.15.53:0/2150963589 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7fabe40b80b0 con 0x7fabf40b5900 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.000+0000 7fabfa1b1640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabd80762d0 0x7fabd8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.001+0000 7fabfa1b1640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabd80762d0 0x7fabd8078790 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fabf0006fd0 tx=0x7fabf0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.004+0000 7fabeaffd640 1 -- 172.21.15.53:0/2150963589 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fabe404b030 con 0x7fabf40b5900 2024-05-10T10:42:37.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.196+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabf4075090 con 0x7fabf40b5900 2024-05-10T10:42:37.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.197+0000 7fabeaffd640 1 -- 172.21.15.53:0/2150963589 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+17241 (secure 0 0 0) 0x7fabe40817a0 con 0x7fabf40b5900 2024-05-10T10:42:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:37.198 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":107,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:42:34.798937+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:41:12.805545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T10:41:14.096305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]},{"pgid":"16.31","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/1219905896":"2024-05-11T10:42:34.798914+0000","172.21.15.53:6834/1219905896":"2024-05-11T10:42:34.798914+0000","172.21.15.53:6835/4122299129":"2024-05-11T10:41:58.338869+0000","172.21.15.53:6834/4122299129":"2024-05-11T10:41:58.338869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:42:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabd80762d0 msgr2=0x7fabd8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabd80762d0 0x7fabd8078790 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fabf0006fd0 tx=0x7fabf0008040 comp rx=0 tx=0).stop 2024-05-10T10:42:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 msgr2=0x7fabf40a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40a19b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fabe4000c00 tx=0x7fabe40046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 shutdown_connections 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fabd80762d0 0x7fabd8078790 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf40b62e0 0x7fabf406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf40b5900 0x7fabf40a19b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 --2- 172.21.15.53:0/2150963589 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf40b5520 0x7fabf4154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 >> 172.21.15.53:0/2150963589 conn(0x7fabf40d5170 msgr2=0x7fabf40b9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 shutdown_connections 2024-05-10T10:42:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.199+0000 7fabfc415640 1 -- 172.21.15.53:0/2150963589 wait complete. 2024-05-10T10:42:37.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:42:37.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.316+0000 7f41d6324640 1 Processor -- start 2024-05-10T10:42:37.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.316+0000 7f41d6324640 1 -- start start 2024-05-10T10:42:37.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41d6324640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f41d0163730 0x7f41d00ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41d6324640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 0x7f41d006c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41d6324640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d00cb190 con 0x7f41d006fa60 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41d6324640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f41d0163bc0 con 0x7f41d0163730 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41d6324640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41d0160240 con 0x7f41d00cad80 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41cffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f41d006fa60 0x7f41d015f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59240/0 (socket says 172.21.15.53:59240) 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41cffff640 1 -- 172.21.15.53:0/2389086681 learned_addr learned my addr 172.21.15.53:0/2389086681 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41d48c1640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 0x7f41d006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2389086681 >> v2:172.21.15.135:3300/0 conn(0x7f41d0163730 0x7f41d00ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556965848 0 0) 0x7f41c0001110 con 0x7f41d006fa60 2024-05-10T10:42:37.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.317+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41bc003520 con 0x7f41d006fa60 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 976128326 0 0) 0x7f41bc003520 con 0x7f41d006fa60 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 msgr2=0x7f41d006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 0x7f41d006c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 >> v2:172.21.15.135:3300/0 conn(0x7f41d0163730 msgr2=0x7f41d00ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 --2- 172.21.15.53:0/2389086681 >> v2:172.21.15.135:3300/0 conn(0x7f41d0163730 0x7f41d00ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d0160520 con 0x7f41d006fa60 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f41c0001840 con 0x7f41d006fa60 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2389086681 >> v2:172.21.15.135:3300/0 conn(0x7f41d0163730 0x7f41d00ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 >> v1:172.21.15.53:6789/0 conn(0x7f41d006fa60 legacy=0x7f41d015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41bc0039b0 0x7f41bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 0x7f41bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0163bc0 con 0x7f41bc004d40 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0160240 con 0x7f41bc0039b0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc00b9c0 con 0x7f41d00cad80 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41d48c1640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41cffff640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41bc0039b0 0x7f41bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 0x7f41bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41d48c1640 1 -- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 msgr2=0x7f41bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41d48c1640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d00cad80 0x7f41bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41d48c1640 1 -- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41bc0039b0 msgr2=0x7f41bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41d48c1640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41bc0039b0 0x7f41bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.318+0000 7f41d48c1640 1 -- 172.21.15.53:0/2389086681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc003520 con 0x7f41bc004d40 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.319+0000 7f41d48c1640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41bc0051c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009fa0 tx=0x7f41bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a49ccd55d26d2d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.319+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8009bf0 con 0x7f41bc004d40 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.319+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c8036070 con 0x7f41bc004d40 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.319+0000 7f41cf7fe640 1 -- 172.21.15.53:0/2389086681 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8030850 con 0x7f41bc004d40 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.320+0000 7f41d6324640 1 -- 172.21.15.53:0/2389086681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 msgr2=0x7f41bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.320+0000 7f41d6324640 1 --2- 172.21.15.53:0/2389086681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41bc0051c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009fa0 tx=0x7f41bc005830 comp rx=0 tx=0).stop 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.320+0000 7f41d48c1640 1 -- 172.21.15.53:0/2389086681 reap_dead start 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.320+0000 7f41d6324640 1 -- 172.21.15.53:0/2389086681 shutdown_connections 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.320+0000 7f41d6324640 1 -- 172.21.15.53:0/2389086681 >> 172.21.15.53:0/2389086681 conn(0x7f41d00d1510 msgr2=0x7f41d00d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.320+0000 7f41d6324640 1 -- 172.21.15.53:0/2389086681 shutdown_connections 2024-05-10T10:42:37.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.320+0000 7f41d6324640 1 -- 172.21.15.53:0/2389086681 wait complete. 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.321+0000 7f41d6324640 1 Processor -- start 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.321+0000 7f41d6324640 1 -- start start 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41d6324640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41d0148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41d6324640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d00cce00 0x7f41d00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41d6324640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d01f8a70 0x7f41d01f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41d6324640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41d006eae0 con 0x7f41bc004d40 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41d6324640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0163210 con 0x7f41d00cce00 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41d6324640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41d015fe40 con 0x7f41d01f8a70 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41d0148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41d0148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60644/0 (socket says 172.21.15.53:60644) 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 -- 172.21.15.53:0/2227389431 learned_addr learned my addr 172.21.15.53:0/2227389431 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d00cce00 0x7f41d00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 -- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d01f8a70 msgr2=0x7f41d01f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d01f8a70 0x7f41d01f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 -- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d00cce00 msgr2=0x7f41d00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d00cce00 0x7f41d00cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 -- 172.21.15.53:0/2227389431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d01f97a0 con 0x7f41bc004d40 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d00cce00 0x7f41d00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:37.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cffff640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41d0148e80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f41c0007340 tx=0x7f41c0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cd7fa640 1 -- 172.21.15.53:0/2227389431 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c001a070 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cd7fa640 1 -- 172.21.15.53:0/2227389431 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c000f940 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41cd7fa640 1 -- 172.21.15.53:0/2227389431 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c000fd10 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.322+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d00aa6d0 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.323+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d01493c0 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.323+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d006a220 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.324+0000 7f41cd7fa640 1 -- 172.21.15.53:0/2227389431 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41c001d070 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.324+0000 7f41cd7fa640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41b00762d0 0x7f41b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.325+0000 7f41cd7fa640 1 -- 172.21.15.53:0/2227389431 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7f41c0097270 con 0x7f41bc004d40 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.325+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41b00762d0 0x7f41b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.325+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41b00762d0 0x7f41b0078790 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f41d01497a0 tx=0x7f41b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.329+0000 7f41cd7fa640 1 -- 172.21.15.53:0/2227389431 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f41d00cb190 con 0x7f41bc004d40 2024-05-10T10:42:37.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.561+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f41d006dcc0 con 0x7f41bc004d40 2024-05-10T10:42:37.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.812+0000 7f41cd7fa640 1 -- 172.21.15.53:0/2227389431 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f41c00609f0 con 0x7f41bc004d40 2024-05-10T10:42:37.812 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41b00762d0 msgr2=0x7f41b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41b00762d0 0x7f41b0078790 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f41d01497a0 tx=0x7f41b8008040 comp rx=0 tx=0).stop 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 msgr2=0x7f41d0148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41d0148e80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f41c0007340 tx=0x7f41c0007800 comp rx=0 tx=0).stop 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 shutdown_connections 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f41b00762d0 0x7f41b0078790 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f41d01f8a70 0x7f41d01f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41d00cce00 0x7f41d00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 --2- 172.21.15.53:0/2227389431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41bc004d40 0x7f41d0148e80 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 >> 172.21.15.53:0/2227389431 conn(0x7f41d00d1510 msgr2=0x7f41d0162230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 shutdown_connections 2024-05-10T10:42:37.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.814+0000 7f41d6324640 1 -- 172.21.15.53:0/2227389431 wait complete. 2024-05-10T10:42:37.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:42:37.933 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.933+0000 7f972351b640 1 Processor -- start 2024-05-10T10:42:37.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.933+0000 7f972351b640 1 -- start start 2024-05-10T10:42:37.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f972351b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c162900 0x7f971c0d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.934 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f972351b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971c0d4190 con 0x7f971c0d3950 2024-05-10T10:42:37.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f972351b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971c0cfc40 con 0x7f971c162ce0 2024-05-10T10:42:37.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f972351b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f971c0d3eb0 con 0x7f971c162900 2024-05-10T10:42:37.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f9721ab8640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f971c0d3950 0x7f971c0d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59250/0 (socket says 172.21.15.53:59250) 2024-05-10T10:42:37.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f9721ab8640 1 -- 172.21.15.53:0/2652569685 learned_addr learned my addr 172.21.15.53:0/2652569685 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:37.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f97212b7640 1 --2- 172.21.15.53:0/2652569685 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c162900 0x7f971c0d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491340234 0 0) 0x7f9714001490 con 0x7f971c0d3950 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.934+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9708003520 con 0x7f971c0d3950 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.935+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416675927 0 0) 0x7f971c0cfc40 con 0x7f971c162ce0 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.935+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9714001490 con 0x7f971c162ce0 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.935+0000 7f97212b7640 1 -- 172.21.15.53:0/2652569685 >> v1:172.21.15.135:6789/0 conn(0x7f971c162ce0 legacy=0x7f971c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.935+0000 7f97212b7640 1 -- 172.21.15.53:0/2652569685 >> v1:172.21.15.53:6789/0 conn(0x7f971c0d3950 legacy=0x7f971c0d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.935+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 308235274 0 0) 0x7f9708003520 con 0x7f971c0d3950 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.935+0000 7f97212b7640 1 -- 172.21.15.53:0/2652569685 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c0cfee0 con 0x7f971c162900 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f97212b7640 1 --2- 172.21.15.53:0/2652569685 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c162900 0x7f971c0d3410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97180097d0 tx=0x7f971802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b146b6bc6df60b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718037070 con 0x7f971c162900 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971802ab90 con 0x7f971c162900 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f9713fff640 1 -- 172.21.15.53:0/2652569685 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714001bc0 con 0x7f971c162900 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f972351b640 1 -- 172.21.15.53:0/2652569685 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c162900 msgr2=0x7f971c0d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f972351b640 1 --2- 172.21.15.53:0/2652569685 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c162900 0x7f971c0d3410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97180097d0 tx=0x7f971802a100 comp rx=0 tx=0).stop 2024-05-10T10:42:37.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f972351b640 1 -- 172.21.15.53:0/2652569685 shutdown_connections 2024-05-10T10:42:37.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f972351b640 1 --2- 172.21.15.53:0/2652569685 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c162900 0x7f971c0d3410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f972351b640 1 -- 172.21.15.53:0/2652569685 >> 172.21.15.53:0/2652569685 conn(0x7f971c09e430 msgr2=0x7f971c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:37.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f972351b640 1 -- 172.21.15.53:0/2652569685 shutdown_connections 2024-05-10T10:42:37.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.936+0000 7f972351b640 1 -- 172.21.15.53:0/2652569685 wait complete. 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 Processor -- start 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 -- start start 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c0d3950 0x7f971c160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c162900 0x7f971c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c162ce0 0x7f971c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0b4300 con 0x7f971c162ce0 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0b4180 con 0x7f971c162900 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.938+0000 7f972351b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f971c0b4ff0 con 0x7f971c0d3950 2024-05-10T10:42:37.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c162ce0 0x7f971c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c162ce0 0x7f971c157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60660/0 (socket says 172.21.15.53:60660) 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 -- 172.21.15.53:0/1982448625 learned_addr learned my addr 172.21.15.53:0/1982448625 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f97212b7640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c0d3950 0x7f971c160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9720ab6640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c162900 0x7f971c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 -- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c0d3950 msgr2=0x7f971c160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c0d3950 0x7f971c160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 -- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c162900 msgr2=0x7f971c1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c162900 0x7f971c1576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 -- 172.21.15.53:0/1982448625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c15bf30 con 0x7f971c162ce0 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9720ab6640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c162900 0x7f971c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9721ab8640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c162ce0 0x7f971c157c20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9714001c20 tx=0x7f971400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9711ffb640 1 -- 172.21.15.53:0/1982448625 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714017070 con 0x7f971c162ce0 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c15c210 con 0x7f971c162ce0 2024-05-10T10:42:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9711ffb640 1 -- 172.21.15.53:0/1982448625 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9714012600 con 0x7f971c162ce0 2024-05-10T10:42:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f9711ffb640 1 -- 172.21.15.53:0/1982448625 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140129d0 con 0x7f971c162ce0 2024-05-10T10:42:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.939+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c15c7d0 con 0x7f971c162ce0 2024-05-10T10:42:37.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.940+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c15c9d0 con 0x7f971c162ce0 2024-05-10T10:42:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.941+0000 7f9711ffb640 1 -- 172.21.15.53:0/1982448625 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f971401a070 con 0x7f971c162ce0 2024-05-10T10:42:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.942+0000 7f9711ffb640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f97000762d0 0x7f9700078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.942+0000 7f9711ffb640 1 -- 172.21.15.53:0/1982448625 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7f9714097a00 con 0x7f971c162ce0 2024-05-10T10:42:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.942+0000 7f97212b7640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f97000762d0 0x7f9700078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:37.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.942+0000 7f97212b7640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f97000762d0 0x7f9700078790 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9718034000 tx=0x7f971802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:37.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:37.946+0000 7f9711ffb640 1 -- 172.21.15.53:0/1982448625 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f971c0d4190 con 0x7f971c162ce0 2024-05-10T10:42:38.179 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.179+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f971c06a220 con 0x7f971c162ce0 2024-05-10T10:42:38.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.817+0000 7f9711ffb640 1 -- 172.21.15.53:0/1982448625 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) ==== 104+0+0 (secure 0 0 0) 0x7f9714061060 con 0x7f971c162ce0 2024-05-10T10:42:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f97000762d0 msgr2=0x7f9700078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f97000762d0 0x7f9700078790 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9718034000 tx=0x7f971802e000 comp rx=0 tx=0).stop 2024-05-10T10:42:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c162ce0 msgr2=0x7f971c157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c162ce0 0x7f971c157c20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9714001c20 tx=0x7f971400a430 comp rx=0 tx=0).stop 2024-05-10T10:42:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 shutdown_connections 2024-05-10T10:42:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f97000762d0 0x7f9700078790 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:38.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c162ce0 0x7f971c157c20 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c162900 0x7f971c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 --2- 172.21.15.53:0/1982448625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c0d3950 0x7f971c160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 >> 172.21.15.53:0/1982448625 conn(0x7f971c09e430 msgr2=0x7f971c0d5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 shutdown_connections 2024-05-10T10:42:38.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.820+0000 7f972351b640 1 -- 172.21.15.53:0/1982448625 wait complete. 2024-05-10T10:42:38.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:42:38.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.949+0000 7efd0dfc8640 1 Processor -- start 2024-05-10T10:42:38.950 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.949+0000 7efd0dfc8640 1 -- start start 2024-05-10T10:42:38.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd0dfc8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd08162e00 0x7efd08163290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:38.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd0dfc8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd080716f0 con 0x7efd0815f370 2024-05-10T10:42:38.951 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd0dfc8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd080d2510 con 0x7efd081639a0 2024-05-10T10:42:38.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd0dfc8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7efd08070220 con 0x7efd08162e00 2024-05-10T10:42:38.953 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd08162e00 0x7efd08163290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:38.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd08162e00 0x7efd08163290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60670/0 (socket says 172.21.15.53:60670) 2024-05-10T10:42:38.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd07fff640 1 -- 172.21.15.53:0/750985267 learned_addr learned my addr 172.21.15.53:0/750985267 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:38.954 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd077fe640 1 --1- 172.21.15.53:0/750985267 >> v1:172.21.15.53:6789/0 conn(0x7efd0815f370 0x7efd080d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59252/0 (socket says 172.21.15.53:59252) 2024-05-10T10:42:38.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd07fff640 1 -- 172.21.15.53:0/750985267 >> v1:172.21.15.135:6789/0 conn(0x7efd081639a0 legacy=0x7efd08070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd07fff640 1 -- 172.21.15.53:0/750985267 >> v1:172.21.15.53:6789/0 conn(0x7efd0815f370 legacy=0x7efd080d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd07fff640 1 -- 172.21.15.53:0/750985267 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd08070500 con 0x7efd08162e00 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.950+0000 7efd06ffd640 1 -- 172.21.15.53:0/750985267 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204219680 0 0) 0x7efd080716f0 con 0x7efd0815f370 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.951+0000 7efd07fff640 1 --2- 172.21.15.53:0/750985267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd08162e00 0x7efd08163290 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e50 tx=0x7efd0002a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b54c4fce7501e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.951+0000 7efd06ffd640 1 -- 172.21.15.53:0/750985267 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00037070 con 0x7efd08162e00 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.951+0000 7efd06ffd640 1 -- 172.21.15.53:0/750985267 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd00032630 con 0x7efd08162e00 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.951+0000 7efd06ffd640 1 -- 172.21.15.53:0/750985267 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0002e050 con 0x7efd08162e00 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.952+0000 7efd0dfc8640 1 -- 172.21.15.53:0/750985267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd08162e00 msgr2=0x7efd08163290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.952+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/750985267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd08162e00 0x7efd08163290 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e50 tx=0x7efd0002a840 comp rx=0 tx=0).stop 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.952+0000 7efd0dfc8640 1 -- 172.21.15.53:0/750985267 shutdown_connections 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.952+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/750985267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd08162e00 0x7efd08163290 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:38.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.952+0000 7efd0dfc8640 1 -- 172.21.15.53:0/750985267 >> 172.21.15.53:0/750985267 conn(0x7efd080d2e10 msgr2=0x7efd080d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.952+0000 7efd0dfc8640 1 -- 172.21.15.53:0/750985267 shutdown_connections 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.952+0000 7efd0dfc8640 1 -- 172.21.15.53:0/750985267 wait complete. 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.953+0000 7efd0dfc8640 1 Processor -- start 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.953+0000 7efd0dfc8640 1 -- start start 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd0dfc8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0815f370 0x7efd080a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd0dfc8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08162e00 0x7efd0809cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd0dfc8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd081639a0 0x7efd0809d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd0dfc8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efd080cd460 con 0x7efd0815f370 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd0dfc8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd08070ac0 con 0x7efd08162e00 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd0dfc8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efd080cd5e0 con 0x7efd081639a0 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0815f370 0x7efd080a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0815f370 0x7efd080a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60672/0 (socket says 172.21.15.53:60672) 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 -- 172.21.15.53:0/121941740 learned_addr learned my addr 172.21.15.53:0/121941740 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:38.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd07fff640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd081639a0 0x7efd0809d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efcfffff640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08162e00 0x7efd0809cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 -- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd081639a0 msgr2=0x7efd0809d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd081639a0 0x7efd0809d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 -- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08162e00 msgr2=0x7efd0809cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08162e00 0x7efd0809cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 -- 172.21.15.53:0/121941740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0814c0f0 con 0x7efd0815f370 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd077fe640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0815f370 0x7efd080a05f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c4d0 tx=0x7efcf800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd04ff9640 1 -- 172.21.15.53:0/121941740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8018070 con 0x7efd0815f370 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd04ff9640 1 -- 172.21.15.53:0/121941740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf80074e0 con 0x7efd0815f370 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0814c3d0 con 0x7efd0815f370 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.954+0000 7efd04ff9640 1 -- 172.21.15.53:0/121941740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8004a20 con 0x7efd0815f370 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.955+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0809eba0 con 0x7efd0815f370 2024-05-10T10:42:38.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.956+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd080685d0 con 0x7efd0815f370 2024-05-10T10:42:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.956+0000 7efd04ff9640 1 -- 172.21.15.53:0/121941740 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efcf801b070 con 0x7efd0815f370 2024-05-10T10:42:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.957+0000 7efd04ff9640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efce80762d0 0x7efce8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.957+0000 7efd04ff9640 1 -- 172.21.15.53:0/121941740 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6331+0+0 (secure 0 0 0) 0x7efcf8096b20 con 0x7efd0815f370 2024-05-10T10:42:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.957+0000 7efcfffff640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efce80762d0 0x7efce8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.957+0000 7efcfffff640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efce80762d0 0x7efce8078790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efd080d3550 tx=0x7efcf0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:38.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:38.961+0000 7efd04ff9640 1 -- 172.21.15.53:0/121941740 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7efd080685d0 con 0x7efd0815f370 2024-05-10T10:42:39.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.151+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efd0806a220 con 0x7efd0815f370 2024-05-10T10:42:39.823 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.823+0000 7efd04ff9640 1 -- 172.21.15.53:0/121941740 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7efcf80602a0 con 0x7efd0815f370 2024-05-10T10:42:39.824 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:42:39.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.826+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efce80762d0 msgr2=0x7efce8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.826+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efce80762d0 0x7efce8078790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efd080d3550 tx=0x7efcf0009040 comp rx=0 tx=0).stop 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.826+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0815f370 msgr2=0x7efd080a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.826+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0815f370 0x7efd080a05f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c4d0 tx=0x7efcf800c990 comp rx=0 tx=0).stop 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 shutdown_connections 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efce80762d0 0x7efce8078790 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efd081639a0 0x7efd0809d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08162e00 0x7efd0809cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 --2- 172.21.15.53:0/121941740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0815f370 0x7efd080a05f0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 >> 172.21.15.53:0/121941740 conn(0x7efd080d2e10 msgr2=0x7efd0806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 shutdown_connections 2024-05-10T10:42:39.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.827+0000 7efd0dfc8640 1 -- 172.21.15.53:0/121941740 wait complete. 2024-05-10T10:42:39.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:42:39.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.956+0000 7f942af24640 1 Processor -- start 2024-05-10T10:42:39.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.956+0000 7f942af24640 1 -- start start 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f942af24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94240b48a0 0x7f94240b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f942af24640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f942af24640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94240d0830 con 0x7f9424162990 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f942af24640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f94240d10d0 con 0x7f94240b48a0 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f942af24640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f94240d0b10 con 0x7f94240d1aa0 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f94294c1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f9423fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94240b48a0 0x7f94240b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f9428cc0640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9424162990 0x7f94240d59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:39740/0 (socket says 172.21.15.53:39740) 2024-05-10T10:42:39.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.957+0000 7f9428cc0640 1 -- 172.21.15.53:0/1784642534 learned_addr learned my addr 172.21.15.53:0/1784642534 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:39.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.958+0000 7f94237fe640 1 -- 172.21.15.53:0/1784642534 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827971294 0 0) 0x7f94240d0830 con 0x7f9424162990 2024-05-10T10:42:39.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.958+0000 7f94237fe640 1 -- 172.21.15.53:0/1784642534 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f940c003520 con 0x7f9424162990 2024-05-10T10:42:39.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.958+0000 7f9423fff640 1 -- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 msgr2=0x7f94240d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:39.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.958+0000 7f9423fff640 1 --2- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.958+0000 7f9423fff640 1 -- 172.21.15.53:0/1784642534 >> v1:172.21.15.135:6789/0 conn(0x7f9424162990 legacy=0x7f94240d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.958+0000 7f9423fff640 1 -- 172.21.15.53:0/1784642534 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94240d0df0 con 0x7f94240b48a0 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.958+0000 7f94294c1640 1 --2- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f9423fff640 1 --2- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94240b48a0 0x7f94240b5f30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f94180097a0 tx=0x7f941802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c4bc174120cb7d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f94237fe640 1 -- 172.21.15.53:0/1784642534 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418037070 con 0x7f94240b48a0 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f94237fe640 1 -- 172.21.15.53:0/1784642534 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9418032440 con 0x7f94240b48a0 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f94237fe640 1 -- 172.21.15.53:0/1784642534 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94180327f0 con 0x7f94240b48a0 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f942af24640 1 -- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94240b48a0 msgr2=0x7f94240b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f942af24640 1 --2- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94240b48a0 0x7f94240b5f30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f94180097a0 tx=0x7f941802a180 comp rx=0 tx=0).stop 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f942af24640 1 -- 172.21.15.53:0/1784642534 shutdown_connections 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f942af24640 1 --2- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f942af24640 1 --2- 172.21.15.53:0/1784642534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94240b48a0 0x7f94240b5f30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.959+0000 7f942af24640 1 -- 172.21.15.53:0/1784642534 >> 172.21.15.53:0/1784642534 conn(0x7f942409e3f0 msgr2=0x7f94240d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.960+0000 7f942af24640 1 -- 172.21.15.53:0/1784642534 shutdown_connections 2024-05-10T10:42:39.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.960+0000 7f942af24640 1 -- 172.21.15.53:0/1784642534 wait complete. 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.961+0000 7f942af24640 1 Processor -- start 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.961+0000 7f942af24640 1 -- start start 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f942af24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94240b48a0 0x7f94240ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f942af24640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f942af24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9424162990 0x7f9424159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f942af24640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f94240d3320 con 0x7f94240b48a0 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f942af24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9424162d70 con 0x7f9424162990 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f942af24640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f94240d34a0 con 0x7f94240d1aa0 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94240b48a0 0x7f94240ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94240b48a0 0x7f94240ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60674/0 (socket says 172.21.15.53:60674) 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 -- 172.21.15.53:0/703421231 learned_addr learned my addr 172.21.15.53:0/703421231 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:39.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f94294c1640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9424162990 0x7f9424159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9423fff640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 -- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 msgr2=0x7f94240ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 -- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9424162990 msgr2=0x7f9424159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9424162990 0x7f9424159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 -- 172.21.15.53:0/703421231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94240cd8e0 con 0x7f94240b48a0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.962+0000 7f9428cc0640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94240b48a0 0x7f94240ceef0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f941400c390 tx=0x7f941400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.963+0000 7f94217fa640 1 -- 172.21.15.53:0/703421231 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414017070 con 0x7f94240b48a0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.963+0000 7f94217fa640 1 -- 172.21.15.53:0/703421231 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9414005ce0 con 0x7f94240b48a0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.963+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94240b89c0 con 0x7f94240b48a0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.963+0000 7f94217fa640 1 -- 172.21.15.53:0/703421231 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94140072f0 con 0x7f94240b48a0 2024-05-10T10:42:39.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.963+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94240b9260 con 0x7f94240b48a0 2024-05-10T10:42:39.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.964+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942406a220 con 0x7f94240b48a0 2024-05-10T10:42:39.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.964+0000 7f94217fa640 1 -- 172.21.15.53:0/703421231 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f941401a070 con 0x7f94240b48a0 2024-05-10T10:42:39.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.965+0000 7f94217fa640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94100762d0 0x7f9410078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:39.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.965+0000 7f94217fa640 1 -- 172.21.15.53:0/703421231 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5893+0+0 (secure 0 0 0) 0x7f94140968c0 con 0x7f94240b48a0 2024-05-10T10:42:39.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.965+0000 7f9423fff640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94100762d0 0x7f9410078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:39.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.966+0000 7f9423fff640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94100762d0 0x7f9410078790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9418034000 tx=0x7f941802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:39.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:39.969+0000 7f94217fa640 1 -- 172.21.15.53:0/703421231 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f941405fb20 con 0x7f94240b48a0 2024-05-10T10:42:40.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.161+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f94240d2710 con 0x7f94240b48a0 2024-05-10T10:42:40.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.828+0000 7f94217fa640 1 -- 172.21.15.53:0/703421231 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f9414063e20 con 0x7f94240b48a0 2024-05-10T10:42:40.829 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:42:40.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94100762d0 msgr2=0x7f9410078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94100762d0 0x7f9410078790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9418034000 tx=0x7f941802e000 comp rx=0 tx=0).stop 2024-05-10T10:42:40.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94240b48a0 msgr2=0x7f94240ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94240b48a0 0x7f94240ceef0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f941400c390 tx=0x7f941400c790 comp rx=0 tx=0).stop 2024-05-10T10:42:40.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 shutdown_connections 2024-05-10T10:42:40.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f94100762d0 0x7f9410078790 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9424162990 0x7f9424159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94240d1aa0 0x7f94240ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 --2- 172.21.15.53:0/703421231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94240b48a0 0x7f94240ceef0 secure :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f941400c390 tx=0x7f941400c790 comp rx=0 tx=0).stop 2024-05-10T10:42:40.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.832+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 >> 172.21.15.53:0/703421231 conn(0x7f942409e3f0 msgr2=0x7f94240d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.833+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 shutdown_connections 2024-05-10T10:42:40.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.833+0000 7f942af24640 1 -- 172.21.15.53:0/703421231 wait complete. 2024-05-10T10:42:40.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T10:42:40.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.960+0000 7fac7ed9d640 1 Processor -- start 2024-05-10T10:42:40.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.960+0000 7fac7ed9d640 1 -- start start 2024-05-10T10:42:40.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fac780cf240 0x7fac780cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7ed9d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac7815cfb0 con 0x7fac78160530 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7ed9d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac7815d290 con 0x7fac7806f030 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7ed9d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fac780ccb60 con 0x7fac780cf240 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac6ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fac78160530 0x7fac7815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59262/0 (socket says 172.21.15.53:59262) 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac6ffff640 1 -- 172.21.15.53:0/1121998530 learned_addr learned my addr 172.21.15.53:0/1121998530 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7d33a640 1 --2- 172.21.15.53:0/1121998530 >> v2:172.21.15.53:3300/0 conn(0x7fac780cf240 0x7fac780cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7d33a640 1 -- 172.21.15.53:0/1121998530 >> v1:172.21.15.135:6789/0 conn(0x7fac7806f030 legacy=0x7fac7806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7d33a640 1 -- 172.21.15.53:0/1121998530 >> v1:172.21.15.53:6789/0 conn(0x7fac78160530 legacy=0x7fac7815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.961+0000 7fac7d33a640 1 -- 172.21.15.53:0/1121998530 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7815fd30 con 0x7fac780cf240 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.962+0000 7fac7d33a640 1 --2- 172.21.15.53:0/1121998530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac780cf240 0x7fac780cf6d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fac700097a0 tx=0x7fac7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40fff603c52c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.962+0000 7fac6f7fe640 1 -- 172.21.15.53:0/1121998530 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70037070 con 0x7fac780cf240 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.962+0000 7fac6f7fe640 1 -- 172.21.15.53:0/1121998530 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac70032440 con 0x7fac780cf240 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.962+0000 7fac6f7fe640 1 -- 172.21.15.53:0/1121998530 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70032810 con 0x7fac780cf240 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.963+0000 7fac7ed9d640 1 -- 172.21.15.53:0/1121998530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac780cf240 msgr2=0x7fac780cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.963+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/1121998530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac780cf240 0x7fac780cf6d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fac700097a0 tx=0x7fac7002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.963+0000 7fac7ed9d640 1 -- 172.21.15.53:0/1121998530 shutdown_connections 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.963+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/1121998530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac780cf240 0x7fac780cf6d0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.963+0000 7fac7ed9d640 1 -- 172.21.15.53:0/1121998530 >> 172.21.15.53:0/1121998530 conn(0x7fac780d0c90 msgr2=0x7fac780d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:40.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.963+0000 7fac7ed9d640 1 -- 172.21.15.53:0/1121998530 shutdown_connections 2024-05-10T10:42:40.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.963+0000 7fac7ed9d640 1 -- 172.21.15.53:0/1121998530 wait complete. 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.964+0000 7fac7ed9d640 1 Processor -- start 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.964+0000 7fac7ed9d640 1 -- start start 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac7806f030 0x7fac780c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac780cf240 0x7fac780b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac78160530 0x7fac780b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7ed9d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac7815fa40 con 0x7fac7806f030 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7ed9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fac78160990 con 0x7fac780cf240 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7ed9d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fac780cb910 con 0x7fac78160530 2024-05-10T10:42:40.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac7806f030 0x7fac780c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac7806f030 0x7fac780c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60686/0 (socket says 172.21.15.53:60686) 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 -- 172.21.15.53:0/969256258 learned_addr learned my addr 172.21.15.53:0/969256258 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac6ffff640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac780cf240 0x7fac780b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7d33a640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac78160530 0x7fac780b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 -- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac78160530 msgr2=0x7fac780b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac78160530 0x7fac780b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 -- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac780cf240 msgr2=0x7fac780b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac780cf240 0x7fac780b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 -- 172.21.15.53:0/969256258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac780c98a0 con 0x7fac7806f030 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac6ffff640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac780cf240 0x7fac780b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7d33a640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac78160530 0x7fac780b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac7cb39640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac7806f030 0x7fac780c8ea0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fac7400ba80 tx=0x7fac7400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.965+0000 7fac6d7fa640 1 -- 172.21.15.53:0/969256258 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7400cc00 con 0x7fac7806f030 2024-05-10T10:42:40.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.966+0000 7fac6d7fa640 1 -- 172.21.15.53:0/969256258 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7400cda0 con 0x7fac7806f030 2024-05-10T10:42:40.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.966+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac780c9b80 con 0x7fac7806f030 2024-05-10T10:42:40.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.966+0000 7fac6d7fa640 1 -- 172.21.15.53:0/969256258 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74003bb0 con 0x7fac7806f030 2024-05-10T10:42:40.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.966+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac780ca140 con 0x7fac7806f030 2024-05-10T10:42:40.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.967+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7806a220 con 0x7fac7806f030 2024-05-10T10:42:40.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.967+0000 7fac6d7fa640 1 -- 172.21.15.53:0/969256258 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac74014440 con 0x7fac7806f030 2024-05-10T10:42:40.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.968+0000 7fac6d7fa640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fac640762d0 0x7fac64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:40.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.968+0000 7fac6d7fa640 1 -- 172.21.15.53:0/969256258 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7fac74097500 con 0x7fac7806f030 2024-05-10T10:42:40.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.968+0000 7fac6ffff640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fac640762d0 0x7fac64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:40.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.969+0000 7fac6ffff640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fac640762d0 0x7fac64078790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fac780b3540 tx=0x7fac68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:40.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:40.972+0000 7fac6d7fa640 1 -- 172.21.15.53:0/969256258 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fac74060ee0 con 0x7fac7806f030 2024-05-10T10:42:41.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.170+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fac780cf020 con 0x7fac7806f030 2024-05-10T10:42:41.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.173+0000 7fac6d7fa640 1 -- 172.21.15.53:0/969256258 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fac740607e0 con 0x7fac7806f030 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fac640762d0 msgr2=0x7fac64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fac640762d0 0x7fac64078790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fac780b3540 tx=0x7fac68009040 comp rx=0 tx=0).stop 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac7806f030 msgr2=0x7fac780c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac7806f030 0x7fac780c8ea0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fac7400ba80 tx=0x7fac7400bf40 comp rx=0 tx=0).stop 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 shutdown_connections 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fac640762d0 0x7fac64078790 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac78160530 0x7fac780b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac780cf240 0x7fac780b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 --2- 172.21.15.53:0/969256258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac7806f030 0x7fac780c8ea0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 >> 172.21.15.53:0/969256258 conn(0x7fac780d0c90 msgr2=0x7fac7815db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 shutdown_connections 2024-05-10T10:42:41.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.175+0000 7fac7ed9d640 1 -- 172.21.15.53:0/969256258 wait complete. 2024-05-10T10:42:41.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-10T10:42:41.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.290+0000 7fb7efc09640 1 Processor -- start 2024-05-10T10:42:41.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.290+0000 7fb7efc09640 1 -- start start 2024-05-10T10:42:41.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.291+0000 7fb7efc09640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb7e80b8860 0x7fb7e806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.291+0000 7fb7efc09640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80cfd80 0x7fb7e80ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.291+0000 7fb7efc09640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e815fc90 con 0x7fb7e80d0230 2024-05-10T10:42:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.291+0000 7fb7efc09640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e806ee50 con 0x7fb7e80b8860 2024-05-10T10:42:41.292 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.291+0000 7fb7efc09640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e80b8c40 con 0x7fb7e80cfd80 2024-05-10T10:42:41.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed9a5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb7e80b8860 0x7fb7e806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:41.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ee1a6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80cfd80 0x7fb7e80ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:41.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed9a5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb7e80b8860 0x7fb7e806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:40662/0 (socket says 172.21.15.53:40662) 2024-05-10T10:42:41.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ee1a6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80cfd80 0x7fb7e80ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:41694/0 (socket says 172.21.15.53:41694) 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed9a5640 1 -- 172.21.15.53:0/4052792696 learned_addr learned my addr 172.21.15.53:0/4052792696 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed1a4640 1 --1- 172.21.15.53:0/4052792696 >> v1:172.21.15.135:6789/0 conn(0x7fb7e80d0230 0x7fb7e80cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:39752/0 (socket says 172.21.15.53:39752) 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed9a5640 1 -- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80cfd80 msgr2=0x7fb7e80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed9a5640 1 --2- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80cfd80 0x7fb7e80ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed9a5640 1 -- 172.21.15.53:0/4052792696 >> v1:172.21.15.135:6789/0 conn(0x7fb7e80d0230 legacy=0x7fb7e80cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ed9a5640 1 -- 172.21.15.53:0/4052792696 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e80cd570 con 0x7fb7e80b8860 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.292+0000 7fb7ee1a6640 1 --2- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80cfd80 0x7fb7e80ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7ed9a5640 1 --2- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80b8860 0x7fb7e806e520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e402a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=184d0028ed57616e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7ec9a3640 1 -- 172.21.15.53:0/4052792696 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4037070 con 0x7fb7e80b8860 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7ec9a3640 1 -- 172.21.15.53:0/4052792696 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e402ad60 con 0x7fb7e80b8860 2024-05-10T10:42:41.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7ec9a3640 1 -- 172.21.15.53:0/4052792696 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4032660 con 0x7fb7e80b8860 2024-05-10T10:42:41.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7efc09640 1 -- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80b8860 msgr2=0x7fb7e806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.295 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7efc09640 1 --2- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80b8860 0x7fb7e806e520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e402a0d0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7efc09640 1 -- 172.21.15.53:0/4052792696 shutdown_connections 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7efc09640 1 --2- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80cfd80 0x7fb7e80ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7efc09640 1 --2- 172.21.15.53:0/4052792696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80b8860 0x7fb7e806e520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.293+0000 7fb7efc09640 1 -- 172.21.15.53:0/4052792696 >> 172.21.15.53:0/4052792696 conn(0x7fb7e80d2df0 msgr2=0x7fb7e80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.294+0000 7fb7efc09640 1 -- 172.21.15.53:0/4052792696 shutdown_connections 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.294+0000 7fb7efc09640 1 -- 172.21.15.53:0/4052792696 wait complete. 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.295+0000 7fb7efc09640 1 Processor -- start 2024-05-10T10:42:41.296 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.295+0000 7fb7efc09640 1 -- start start 2024-05-10T10:42:41.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.295+0000 7fb7efc09640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80b8860 0x7fb7e81600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:41.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7efc09640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80cfd80 0x7fb7e8162820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:41.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7efc09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e80d0230 0x7fb7e80c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:41.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7efc09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e80cde10 con 0x7fb7e80d0230 2024-05-10T10:42:41.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7efc09640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e80cdb30 con 0x7fb7e80cfd80 2024-05-10T10:42:41.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7efc09640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e806ea60 con 0x7fb7e80b8860 2024-05-10T10:42:41.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e80d0230 0x7fb7e80c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e80d0230 0x7fb7e80c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60690/0 (socket says 172.21.15.53:60690) 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 -- 172.21.15.53:0/689489159 learned_addr learned my addr 172.21.15.53:0/689489159 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ed9a5640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80b8860 0x7fb7e81600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ed1a4640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80cfd80 0x7fb7e8162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 -- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80b8860 msgr2=0x7fb7e81600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80b8860 0x7fb7e81600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 -- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80cfd80 msgr2=0x7fb7e8162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80cfd80 0x7fb7e8162820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 -- 172.21.15.53:0/689489159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e80c92f0 con 0x7fb7e80d0230 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ed9a5640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80b8860 0x7fb7e81600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7ee1a6640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e80d0230 0x7fb7e80c8ab0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000efc0 tx=0x7fb7e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7de7fc640 1 -- 172.21.15.53:0/689489159 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0018020 con 0x7fb7e80d0230 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7de7fc640 1 -- 172.21.15.53:0/689489159 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e0014480 con 0x7fb7e80d0230 2024-05-10T10:42:41.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e80c9590 con 0x7fb7e80d0230 2024-05-10T10:42:41.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.296+0000 7fb7de7fc640 1 -- 172.21.15.53:0/689489159 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0014850 con 0x7fb7e80d0230 2024-05-10T10:42:41.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.297+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e815b5c0 con 0x7fb7e80d0230 2024-05-10T10:42:41.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.298+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e806a220 con 0x7fb7e80d0230 2024-05-10T10:42:41.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.298+0000 7fb7de7fc640 1 -- 172.21.15.53:0/689489159 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb7e0005d10 con 0x7fb7e80d0230 2024-05-10T10:42:41.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.298+0000 7fb7de7fc640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7cc0762d0 0x7fb7cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:41.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.299+0000 7fb7de7fc640 1 -- 172.21.15.53:0/689489159 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7fb7e00982b0 con 0x7fb7e80d0230 2024-05-10T10:42:41.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.299+0000 7fb7ed9a5640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7cc0762d0 0x7fb7cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:41.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.299+0000 7fb7ed9a5640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7cc0762d0 0x7fb7cc078790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4034000 tx=0x7fb7e402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:41.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.303+0000 7fb7de7fc640 1 -- 172.21.15.53:0/689489159 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb7e0022030 con 0x7fb7e80d0230 2024-05-10T10:42:41.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.575+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fb7e80d1e20 con 0x7fb7e80d0230 2024-05-10T10:42:41.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.829+0000 7fb7de7fc640 1 -- 172.21.15.53:0/689489159 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v414) ==== 146+0+0 (secure 0 0 0) 0x7fb7e0010040 con 0x7fb7e80d0230 2024-05-10T10:42:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7cc0762d0 msgr2=0x7fb7cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7cc0762d0 0x7fb7cc078790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4034000 tx=0x7fb7e402e000 comp rx=0 tx=0).stop 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e80d0230 msgr2=0x7fb7e80c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e80d0230 0x7fb7e80c8ab0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000efc0 tx=0x7fb7e000c490 comp rx=0 tx=0).stop 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 shutdown_connections 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb7cc0762d0 0x7fb7cc078790 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e80d0230 0x7fb7e80c8ab0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7e80cfd80 0x7fb7e8162820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 --2- 172.21.15.53:0/689489159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7e80b8860 0x7fb7e81600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 >> 172.21.15.53:0/689489159 conn(0x7fb7e80d2df0 msgr2=0x7fb7e80d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 shutdown_connections 2024-05-10T10:42:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:41.832+0000 7fb7efc09640 1 -- 172.21.15.53:0/689489159 wait complete. 2024-05-10T10:42:41.845 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-10T10:42:42.007 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-10T10:42:42.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:42:42.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.117+0000 7f9eaf187640 1 Processor -- start 2024-05-10T10:42:42.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.117+0000 7f9eaf187640 1 -- start start 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.117+0000 7f9eaf187640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ea815f790 0x7f9ea80cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.117+0000 7f9eaf187640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea80d30b0 con 0x7f9ea815f3b0 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.117+0000 7f9eaf187640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea80d3390 con 0x7f9ea80d5d30 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.117+0000 7f9eaf187640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea80c8c80 con 0x7f9ea815f790 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9eacf23640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9ea815f3b0 0x7f9ea80d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59276/0 (socket says 172.21.15.53:59276) 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9eacf23640 1 -- 172.21.15.53:0/3566632175 learned_addr learned my addr 172.21.15.53:0/3566632175 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea7fff640 1 --2- 172.21.15.53:0/3566632175 >> v2:172.21.15.135:3300/0 conn(0x7f9ea815f790 0x7f9ea80cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121876686 0 0) 0x7f9ea80d30b0 con 0x7f9ea815f3b0 2024-05-10T10:42:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e94003520 con 0x7f9ea815f3b0 2024-05-10T10:42:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3116489027 0 0) 0x7f9e94003520 con 0x7f9ea815f3b0 2024-05-10T10:42:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 >> v2:172.21.15.135:3300/0 conn(0x7f9ea815f790 msgr2=0x7f9ea80cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 --2- 172.21.15.53:0/3566632175 >> v2:172.21.15.135:3300/0 conn(0x7f9ea815f790 0x7f9ea80cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 >> v1:172.21.15.135:6789/0 conn(0x7f9ea80d5d30 legacy=0x7f9ea80d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea80c8f60 con 0x7f9ea815f3b0 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f9e90001670 con 0x7f9ea815f3b0 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 >> v1:172.21.15.53:6789/0 conn(0x7f9ea815f3b0 legacy=0x7f9ea80d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9e94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e94004800 0x7f9e94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e940051c0 0x7f9e9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80c8c80 con 0x7f9e940039b0 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9400c060 con 0x7f9e94004800 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9400c260 con 0x7f9e940051c0 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9eacf23640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9e94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.118+0000 7f9ea7fff640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e94004800 0x7f9e94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9ead724640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e940051c0 0x7f9e9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9eacf23640 1 -- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e940051c0 msgr2=0x7f9e9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9eacf23640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e940051c0 0x7f9e9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9eacf23640 1 -- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e94004800 msgr2=0x7f9e94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9eacf23640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e94004800 0x7f9e94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9eacf23640 1 -- 172.21.15.53:0/3566632175 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e94003520 con 0x7f9e940039b0 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9ead724640 1 -- 172.21.15.53:0/3566632175 reap_dead start 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.119+0000 7f9eacf23640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9e94003dd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c240 tx=0x7f9e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0063dcbf370b740 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90007cf0 con 0x7f9e940039b0 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e90005af0 con 0x7f9e940039b0 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9ea77fe640 1 -- 172.21.15.53:0/3566632175 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90007070 con 0x7f9e940039b0 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9eaf187640 1 -- 172.21.15.53:0/3566632175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 msgr2=0x7f9e94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9eaf187640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9e94003dd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c240 tx=0x7f9e94005830 comp rx=0 tx=0).stop 2024-05-10T10:42:42.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9eaf187640 1 -- 172.21.15.53:0/3566632175 shutdown_connections 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9eaf187640 1 --2- 172.21.15.53:0/3566632175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9e94003dd0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9eaf187640 1 -- 172.21.15.53:0/3566632175 >> 172.21.15.53:0/3566632175 conn(0x7f9ea80d5130 msgr2=0x7f9ea80d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9eaf187640 1 -- 172.21.15.53:0/3566632175 shutdown_connections 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.120+0000 7f9eaf187640 1 -- 172.21.15.53:0/3566632175 wait complete. 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.121+0000 7f9eaf187640 1 Processor -- start 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.121+0000 7f9eaf187640 1 -- start start 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eaf187640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9ea8152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eaf187640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ea80b85a0 0x7f9ea80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eaf187640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ea80ca9e0 0x7f9ea80cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eaf187640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80d3870 con 0x7f9e940039b0 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eaf187640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80d36f0 con 0x7f9ea80b85a0 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eaf187640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea80d39f0 con 0x7f9ea80ca9e0 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eacf23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9ea8152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eacf23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9ea8152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60704/0 (socket says 172.21.15.53:60704) 2024-05-10T10:42:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.122+0000 7f9eacf23640 1 -- 172.21.15.53:0/972401810 learned_addr learned my addr 172.21.15.53:0/972401810 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9ea7fff640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ea80b85a0 0x7f9ea80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9ead724640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ea80ca9e0 0x7f9ea80cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eacf23640 1 -- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ea80ca9e0 msgr2=0x7f9ea80cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eacf23640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ea80ca9e0 0x7f9ea80cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eacf23640 1 -- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ea80b85a0 msgr2=0x7f9ea80b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eacf23640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ea80b85a0 0x7f9ea80b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eacf23640 1 -- 172.21.15.53:0/972401810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea8152b50 con 0x7f9e940039b0 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9ead724640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ea80ca9e0 0x7f9ea80cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9ea7fff640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ea80b85a0 0x7f9ea80b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eacf23640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9ea8152610 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9e90001970 tx=0x7f9ea80d3560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9ea57fa640 1 -- 172.21.15.53:0/972401810 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90037040 con 0x7f9e940039b0 2024-05-10T10:42:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9ea57fa640 1 -- 172.21.15.53:0/972401810 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e90041070 con 0x7f9e940039b0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea80cd460 con 0x7f9e940039b0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9ea57fa640 1 -- 172.21.15.53:0/972401810 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900047e0 con 0x7f9e940039b0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.123+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea80cda20 con 0x7f9e940039b0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.124+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea806a220 con 0x7f9e940039b0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.125+0000 7f9ea57fa640 1 -- 172.21.15.53:0/972401810 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9e90004980 con 0x7f9e940039b0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.125+0000 7f9ea57fa640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e88076300 0x7f9e880787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.126+0000 7f9ea57fa640 1 -- 172.21.15.53:0/972401810 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7f9e900b7300 con 0x7f9e940039b0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.126+0000 7f9ea7fff640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e88076300 0x7f9e880787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.126+0000 7f9ea7fff640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e88076300 0x7f9e880787c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9ea806e1c0 tx=0x7f9e9800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.130+0000 7f9ea57fa640 1 -- 172.21.15.53:0/972401810 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9e90004c70 con 0x7f9e940039b0 2024-05-10T10:42:42.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.320+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ea815dce0 con 0x7f9e940039b0 2024-05-10T10:42:42.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.321+0000 7f9ea57fa640 1 -- 172.21.15.53:0/972401810 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13433 (secure 0 0 0) 0x7f9e90080d80 con 0x7f9e940039b0 2024-05-10T10:42:42.322 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:42.322 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":109,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:42:40.824072+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/1219905896":"2024-05-11T10:42:34.798914+0000","172.21.15.53:6834/1219905896":"2024-05-11T10:42:34.798914+0000","172.21.15.53:6835/4122299129":"2024-05-11T10:41:58.338869+0000","172.21.15.53:6834/4122299129":"2024-05-11T10:41:58.338869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:42:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e88076300 msgr2=0x7f9e880787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e88076300 0x7f9e880787c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9ea806e1c0 tx=0x7f9e9800b040 comp rx=0 tx=0).stop 2024-05-10T10:42:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 msgr2=0x7f9ea8152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9ea8152610 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9e90001970 tx=0x7f9ea80d3560 comp rx=0 tx=0).stop 2024-05-10T10:42:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 shutdown_connections 2024-05-10T10:42:42.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9e88076300 0x7f9e880787c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ea80ca9e0 0x7f9ea80cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ea80b85a0 0x7f9ea80b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 --2- 172.21.15.53:0/972401810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e940039b0 0x7f9ea8152610 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 >> 172.21.15.53:0/972401810 conn(0x7f9ea80d5130 msgr2=0x7f9ea815e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 shutdown_connections 2024-05-10T10:42:42.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.323+0000 7f9eaf187640 1 -- 172.21.15.53:0/972401810 wait complete. 2024-05-10T10:42:42.335 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T10:33:28.171977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T10:42:42.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T10:42:42.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.441+0000 7fe9a78cc640 1 Processor -- start 2024-05-10T10:42:42.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.442+0000 7fe9a78cc640 1 -- start start 2024-05-10T10:42:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.442+0000 7fe9a78cc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9a015f410 0x7fe9a00d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.442+0000 7fe9a78cc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00b56a0 0x7fe9a00d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.442+0000 7fe9a78cc640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a00d2800 con 0x7fe9a00d0f10 2024-05-10T10:42:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.442+0000 7fe9a78cc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a00d2ae0 con 0x7fe9a015f410 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.442+0000 7fe9a78cc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a00d01d0 con 0x7fe9a00b56a0 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5668640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9a015f410 0x7fe9a00d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5668640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9a015f410 0x7fe9a00d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60718/0 (socket says 172.21.15.53:60718) 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5668640 1 -- 172.21.15.53:0/3225308489 learned_addr learned my addr 172.21.15.53:0/3225308489 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5e69640 1 --2- 172.21.15.53:0/3225308489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00b56a0 0x7fe9a00d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5668640 1 -- 172.21.15.53:0/3225308489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00b56a0 msgr2=0x7fe9a00d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5668640 1 --2- 172.21.15.53:0/3225308489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00b56a0 0x7fe9a00d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5668640 1 -- 172.21.15.53:0/3225308489 >> v1:172.21.15.135:6789/0 conn(0x7fe9a00d0f10 legacy=0x7fe9a00b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5668640 1 -- 172.21.15.53:0/3225308489 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a00d04b0 con 0x7fe9a015f410 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.443+0000 7fe9a5e69640 1 --2- 172.21.15.53:0/3225308489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00b56a0 0x7fe9a00d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe9a5668640 1 --2- 172.21.15.53:0/3225308489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a015f410 0x7fe9a00d09d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe998009870 tx=0x7fe99802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69d0f48e83c32bb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe99ffff640 1 -- 172.21.15.53:0/3225308489 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998037070 con 0x7fe9a015f410 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe99ffff640 1 -- 172.21.15.53:0/3225308489 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe998032440 con 0x7fe9a015f410 2024-05-10T10:42:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe99ffff640 1 -- 172.21.15.53:0/3225308489 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998032810 con 0x7fe9a015f410 2024-05-10T10:42:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe9a78cc640 1 -- 172.21.15.53:0/3225308489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a015f410 msgr2=0x7fe9a00d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/3225308489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a015f410 0x7fe9a00d09d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe998009870 tx=0x7fe99802a180 comp rx=0 tx=0).stop 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe9a78cc640 1 -- 172.21.15.53:0/3225308489 shutdown_connections 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/3225308489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00b56a0 0x7fe9a00d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/3225308489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a015f410 0x7fe9a00d09d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.444+0000 7fe9a78cc640 1 -- 172.21.15.53:0/3225308489 >> 172.21.15.53:0/3225308489 conn(0x7fe9a008ed30 msgr2=0x7fe9a009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.445+0000 7fe9a78cc640 1 -- 172.21.15.53:0/3225308489 shutdown_connections 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.445+0000 7fe9a78cc640 1 -- 172.21.15.53:0/3225308489 wait complete. 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 Processor -- start 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 -- start start 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a00d0f10 0x7fe9a0158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a015f410 0x7fe9a00b8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00ba9b0 0x7fe9a00c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00cffb0 con 0x7fe9a00d0f10 2024-05-10T10:42:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00cfe30 con 0x7fe9a015f410 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a78cc640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a00d3c10 con 0x7fe9a00ba9b0 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.446+0000 7fe9a5668640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a00d0f10 0x7fe9a0158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a00d0f10 0x7fe9a0158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60734/0 (socket says 172.21.15.53:60734) 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 -- 172.21.15.53:0/1546152854 learned_addr learned my addr 172.21.15.53:0/1546152854 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a4e67640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a015f410 0x7fe9a00b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5e69640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00ba9b0 0x7fe9a00c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 -- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00ba9b0 msgr2=0x7fe9a00c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00ba9b0 0x7fe9a00c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 -- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a015f410 msgr2=0x7fe9a00b8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a015f410 0x7fe9a00b8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 -- 172.21.15.53:0/1546152854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a00ca090 con 0x7fe9a00d0f10 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a5668640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a00d0f10 0x7fe9a0158130 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe998000c00 tx=0x7fe9980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe99dffb640 1 -- 172.21.15.53:0/1546152854 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998037040 con 0x7fe9a00d0f10 2024-05-10T10:42:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe99dffb640 1 -- 172.21.15.53:0/1546152854 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe998042070 con 0x7fe9a00d0f10 2024-05-10T10:42:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a00ca370 con 0x7fe9a00d0f10 2024-05-10T10:42:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe99dffb640 1 -- 172.21.15.53:0/1546152854 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998030bd0 con 0x7fe9a00d0f10 2024-05-10T10:42:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.447+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a00bae60 con 0x7fe9a00d0f10 2024-05-10T10:42:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.448+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a00693f0 con 0x7fe9a00d0f10 2024-05-10T10:42:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.449+0000 7fe99dffb640 1 -- 172.21.15.53:0/1546152854 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe998047460 con 0x7fe9a00d0f10 2024-05-10T10:42:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.449+0000 7fe99dffb640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe98c0762d0 0x7fe98c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.450+0000 7fe99dffb640 1 -- 172.21.15.53:0/1546152854 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7fe9980b80f0 con 0x7fe9a00d0f10 2024-05-10T10:42:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.450+0000 7fe9a4e67640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe98c0762d0 0x7fe98c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.450+0000 7fe9a4e67640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe98c0762d0 0x7fe98c078790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe994004ea0 tx=0x7fe994009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.454+0000 7fe99dffb640 1 -- 172.21.15.53:0/1546152854 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe9a00693f0 con 0x7fe9a00d0f10 2024-05-10T10:42:42.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.645+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9a006a220 con 0x7fe9a00d0f10 2024-05-10T10:42:42.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.645+0000 7fe99dffb640 1 -- 172.21.15.53:0/1546152854 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fe998081c00 con 0x7fe9a00d0f10 2024-05-10T10:42:42.646 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2024-05-10T10:42:42.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe98c0762d0 msgr2=0x7fe98c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe98c0762d0 0x7fe98c078790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe994004ea0 tx=0x7fe994009040 comp rx=0 tx=0).stop 2024-05-10T10:42:42.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a00d0f10 msgr2=0x7fe9a0158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a00d0f10 0x7fe9a0158130 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe998000c00 tx=0x7fe9980046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 shutdown_connections 2024-05-10T10:42:42.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe98c0762d0 0x7fe98c078790 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe9a00ba9b0 0x7fe9a00c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a015f410 0x7fe9a00b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 --2- 172.21.15.53:0/1546152854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9a00d0f10 0x7fe9a0158130 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 >> 172.21.15.53:0/1546152854 conn(0x7fe9a008ed30 msgr2=0x7fe9a00d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:42.650 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 shutdown_connections 2024-05-10T10:42:42.651 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.647+0000 7fe9a78cc640 1 -- 172.21.15.53:0/1546152854 wait complete. 2024-05-10T10:42:42.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-10T10:42:42.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.764+0000 7f7c1b9dd640 1 Processor -- start 2024-05-10T10:42:42.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.764+0000 7f7c1b9dd640 1 -- start start 2024-05-10T10:42:42.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c1b9dd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7c1415f3b0 0x7f7c140d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c1b9dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c140d5d30 0x7f7c140d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c1b9dd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c140d30b0 con 0x7f7c1415f790 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c1b9dd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7c140d3390 con 0x7f7c1415f3b0 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c1b9dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7c140c8c80 con 0x7f7c140d5d30 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19779640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7c1415f3b0 0x7f7c140d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19779640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7c1415f3b0 0x7f7c140d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48644/0 (socket says 172.21.15.53:48644) 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19779640 1 -- 172.21.15.53:0/3072715578 learned_addr learned my addr 172.21.15.53:0/3072715578 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19f7a640 1 --2- 172.21.15.53:0/3072715578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c140d5d30 0x7f7c140d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19779640 1 -- 172.21.15.53:0/3072715578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c140d5d30 msgr2=0x7f7c140d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19779640 1 --2- 172.21.15.53:0/3072715578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c140d5d30 0x7f7c140d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19779640 1 -- 172.21.15.53:0/3072715578 >> v1:172.21.15.135:6789/0 conn(0x7f7c1415f790 legacy=0x7f7c140c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.765+0000 7f7c19779640 1 -- 172.21.15.53:0/3072715578 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c140c8f60 con 0x7f7c1415f3b0 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.766+0000 7f7c19779640 1 --2- 172.21.15.53:0/3072715578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1415f3b0 0x7f7c140d36f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009870 tx=0x7f7c1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2d424d580394f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.766+0000 7f7c0bfff640 1 -- 172.21.15.53:0/3072715578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10037070 con 0x7f7c1415f3b0 2024-05-10T10:42:42.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.766+0000 7f7c0bfff640 1 -- 172.21.15.53:0/3072715578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c10032440 con 0x7f7c1415f3b0 2024-05-10T10:42:42.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.766+0000 7f7c0bfff640 1 -- 172.21.15.53:0/3072715578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10032810 con 0x7f7c1415f3b0 2024-05-10T10:42:42.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/3072715578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1415f3b0 msgr2=0x7f7c140d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/3072715578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1415f3b0 0x7f7c140d36f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009870 tx=0x7f7c1002a270 comp rx=0 tx=0).stop 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/3072715578 shutdown_connections 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/3072715578 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c140d5d30 0x7f7c140d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/3072715578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1415f3b0 0x7f7c140d36f0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/3072715578 >> 172.21.15.53:0/3072715578 conn(0x7f7c140d5130 msgr2=0x7f7c140d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/3072715578 shutdown_connections 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.767+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/3072715578 wait complete. 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.768+0000 7f7c1b9dd640 1 Processor -- start 2024-05-10T10:42:42.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.768+0000 7f7c1b9dd640 1 -- start start 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c1b9dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c140d5d30 0x7f7c14160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c1b9dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c1415f3b0 0x7f7c141550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c1b9dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1406e060 0x7f7c1406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c1b9dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140d0e60 con 0x7f7c1406e060 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c1b9dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140d0ce0 con 0x7f7c140d5d30 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c1b9dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7c140d0fe0 con 0x7f7c1415f3b0 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1406e060 0x7f7c1406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1406e060 0x7f7c1406e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48652/0 (socket says 172.21.15.53:48652) 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 -- 172.21.15.53:0/2660093580 learned_addr learned my addr 172.21.15.53:0/2660093580 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19779640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c140d5d30 0x7f7c14160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c18f78640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c1415f3b0 0x7f7c141550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 -- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c1415f3b0 msgr2=0x7f7c141550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c1415f3b0 0x7f7c141550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 -- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c140d5d30 msgr2=0x7f7c14160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:42.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c140d5d30 0x7f7c14160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 -- 172.21.15.53:0/2660093580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c141610a0 con 0x7f7c1406e060 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c18f78640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c1415f3b0 0x7f7c141550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19779640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c140d5d30 0x7f7c14160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c19f7a640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1406e060 0x7f7c1406e4f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c007e40 tx=0x7f7c0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c09ffb640 1 -- 172.21.15.53:0/2660093580 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c014620 con 0x7f7c1406e060 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c09ffb640 1 -- 172.21.15.53:0/2660093580 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c019070 con 0x7f7c1406e060 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c14161380 con 0x7f7c1406e060 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.769+0000 7f7c09ffb640 1 -- 172.21.15.53:0/2660093580 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c014a20 con 0x7f7c1406e060 2024-05-10T10:42:42.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.770+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1415fe60 con 0x7f7c1406e060 2024-05-10T10:42:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.771+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1406a220 con 0x7f7c1406e060 2024-05-10T10:42:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.771+0000 7f7c09ffb640 1 -- 172.21.15.53:0/2660093580 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7c0c014bc0 con 0x7f7c1406e060 2024-05-10T10:42:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.772+0000 7f7c09ffb640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7bf80762d0 0x7f7bf8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.772+0000 7f7c09ffb640 1 -- 172.21.15.53:0/2660093580 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7f7c0c097dc0 con 0x7f7c1406e060 2024-05-10T10:42:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.772+0000 7f7c19779640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7bf80762d0 0x7f7bf8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:42.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.773+0000 7f7c19779640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7bf80762d0 0x7f7bf8078790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7c10034000 tx=0x7f7c1002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:42.777 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:42.776+0000 7f7c09ffb640 1 -- 172.21.15.53:0/2660093580 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7c0c061840 con 0x7f7c1406e060 2024-05-10T10:42:43.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.054+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f7c140cf4d0 con 0x7f7c1406e060 2024-05-10T10:42:43.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.835+0000 7f7c09ffb640 1 -- 172.21.15.53:0/2660093580 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v416) ==== 147+0+0 (secure 0 0 0) 0x7f7c0c061140 con 0x7f7c1406e060 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7bf80762d0 msgr2=0x7f7bf8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7bf80762d0 0x7f7bf8078790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7c10034000 tx=0x7f7c1002e000 comp rx=0 tx=0).stop 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1406e060 msgr2=0x7f7c1406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1406e060 0x7f7c1406e4f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c007e40 tx=0x7f7c0c00e4b0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 shutdown_connections 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7bf80762d0 0x7f7bf8078790 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c1406e060 0x7f7c1406e4f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c1415f3b0 0x7f7c141550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 --2- 172.21.15.53:0/2660093580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c140d5d30 0x7f7c14160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 >> 172.21.15.53:0/2660093580 conn(0x7f7c140d5130 msgr2=0x7f7c140c9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 shutdown_connections 2024-05-10T10:42:43.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.838+0000 7f7c1b9dd640 1 -- 172.21.15.53:0/2660093580 wait complete. 2024-05-10T10:42:43.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T10:42:43.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.966+0000 7f78506f8640 1 Processor -- start 2024-05-10T10:42:43.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.966+0000 7f78506f8640 1 -- start start 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f78506f8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f784806f030 0x7f7848160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f78506f8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f784806f410 0x7f784815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f78506f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f78506f8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f78480ccb60 con 0x7f784806f410 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f78506f8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f784815cfb0 con 0x7f784806f030 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f78506f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f784815d290 con 0x7f784815c9e0 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f784806f410 0x7f784815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f784806f410 0x7f784815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48660/0 (socket says 172.21.15.53:48660) 2024-05-10T10:42:43.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 -- 172.21.15.53:0/1995531988 learned_addr learned my addr 172.21.15.53:0/1995531988 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784e494640 1 --2- 172.21.15.53:0/1995531988 >> v2:172.21.15.135:3300/0 conn(0x7f784806f030 0x7f7848160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784ec95640 1 --2- 172.21.15.53:0/1995531988 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 -- 172.21.15.53:0/1995531988 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 msgr2=0x7f78480cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 --2- 172.21.15.53:0/1995531988 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 -- 172.21.15.53:0/1995531988 >> v2:172.21.15.135:3300/0 conn(0x7f784806f030 msgr2=0x7f7848160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 --2- 172.21.15.53:0/1995531988 >> v2:172.21.15.135:3300/0 conn(0x7f784806f030 0x7f7848160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.967+0000 7f784dc93640 1 -- 172.21.15.53:0/1995531988 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784815fd30 con 0x7f784806f410 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.968+0000 7f784dc93640 1 --2- 172.21.15.53:0/1995531988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f410 0x7f784815c4a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f78480d1cf0 tx=0x7f783802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e141193f6cc04266 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.968+0000 7f784d492640 1 -- 172.21.15.53:0/1995531988 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838037070 con 0x7f784806f410 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.968+0000 7f784d492640 1 -- 172.21.15.53:0/1995531988 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7838032440 con 0x7f784806f410 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.968+0000 7f784d492640 1 -- 172.21.15.53:0/1995531988 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838032810 con 0x7f784806f410 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 -- 172.21.15.53:0/1995531988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f410 msgr2=0x7f784815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 --2- 172.21.15.53:0/1995531988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f410 0x7f784815c4a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f78480d1cf0 tx=0x7f783802a1e0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 -- 172.21.15.53:0/1995531988 shutdown_connections 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 --2- 172.21.15.53:0/1995531988 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 --2- 172.21.15.53:0/1995531988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f410 0x7f784815c4a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 --2- 172.21.15.53:0/1995531988 >> v2:172.21.15.135:3300/0 conn(0x7f784806f030 0x7f7848160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 -- 172.21.15.53:0/1995531988 >> 172.21.15.53:0/1995531988 conn(0x7f78480d0c90 msgr2=0x7f78480d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 -- 172.21.15.53:0/1995531988 shutdown_connections 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.969+0000 7f78506f8640 1 -- 172.21.15.53:0/1995531988 wait complete. 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.970+0000 7f78506f8640 1 Processor -- start 2024-05-10T10:42:43.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.970+0000 7f78506f8640 1 -- start start 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f78506f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f030 0x7f78480b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f78506f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f784806f410 0x7f7848157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f78506f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f78506f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f78480cb910 con 0x7f784806f030 2024-05-10T10:42:43.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f78506f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f784815fa40 con 0x7f784806f410 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f78506f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7848159d70 con 0x7f784815c9e0 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f030 0x7f78480b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f030 0x7f78480b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48672/0 (socket says 172.21.15.53:48672) 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 -- 172.21.15.53:0/401970195 learned_addr learned my addr 172.21.15.53:0/401970195 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784ec95640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784dc93640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f784806f410 0x7f7848157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 -- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 msgr2=0x7f78480c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 -- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f784806f410 msgr2=0x7f7848157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f784806f410 0x7f7848157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.971+0000 7f784e494640 1 -- 172.21.15.53:0/401970195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78480b5390 con 0x7f784806f030 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.972+0000 7f784e494640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f030 0x7f78480b4e50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f784400d830 tx=0x7f784400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.972+0000 7f783effd640 1 -- 172.21.15.53:0/401970195 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844018030 con 0x7f784806f030 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.972+0000 7f783effd640 1 -- 172.21.15.53:0/401970195 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784400be10 con 0x7f784806f030 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.972+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78480b5660 con 0x7f784806f030 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.972+0000 7f783effd640 1 -- 172.21.15.53:0/401970195 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440098b0 con 0x7f784806f030 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.972+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78480b5c20 con 0x7f784806f030 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.973+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784815e750 con 0x7f784806f030 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.973+0000 7f783effd640 1 -- 172.21.15.53:0/401970195 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f784401b070 con 0x7f784806f030 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.974+0000 7f783effd640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f782c0762d0 0x7f782c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:43.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.974+0000 7f783effd640 1 -- 172.21.15.53:0/401970195 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7f78440975f0 con 0x7f784806f030 2024-05-10T10:42:43.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.974+0000 7f784dc93640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f782c0762d0 0x7f782c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:43.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.975+0000 7f784dc93640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f782c0762d0 0x7f782c078790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7838034000 tx=0x7f783802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:43.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:43.978+0000 7f783effd640 1 -- 172.21.15.53:0/401970195 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7844061100 con 0x7f784806f030 2024-05-10T10:42:44.170 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.170+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f784806a220 con 0x7f784806f030 2024-05-10T10:42:44.173 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.173+0000 7f783effd640 1 -- 172.21.15.53:0/401970195 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f7844060a00 con 0x7f784806f030 2024-05-10T10:42:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f782c0762d0 msgr2=0x7f782c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f782c0762d0 0x7f782c078790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7838034000 tx=0x7f783802e000 comp rx=0 tx=0).stop 2024-05-10T10:42:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f030 msgr2=0x7f78480b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f030 0x7f78480b4e50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f784400d830 tx=0x7f784400dcf0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 shutdown_connections 2024-05-10T10:42:44.176 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f782c0762d0 0x7f782c078790 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f784815c9e0 0x7f78480c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f784806f410 0x7f7848157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 --2- 172.21.15.53:0/401970195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f784806f030 0x7f78480b4e50 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 >> 172.21.15.53:0/401970195 conn(0x7f78480d0c90 msgr2=0x7f784815a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 shutdown_connections 2024-05-10T10:42:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.175+0000 7f78506f8640 1 -- 172.21.15.53:0/401970195 wait complete. 2024-05-10T10:42:44.187 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:42:44.188 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:42:44.227 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:42:44.266 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:42:44.266 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:44.267 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:44.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:42:44.284 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:42:44.284 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:44.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:42:44.325 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:44.325 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:44.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:42:44.341 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:42:44.341 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:44.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:42:44.383 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T10:42:44.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.486+0000 7fef6759e640 1 Processor -- start 2024-05-10T10:42:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.486+0000 7fef6759e640 1 -- start start 2024-05-10T10:42:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.487+0000 7fef6759e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fef600d3bd0 0x7fef600d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.487+0000 7fef6759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef600d0b90 con 0x7fef6015f1f0 2024-05-10T10:42:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.487+0000 7fef6759e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef600b4470 con 0x7fef600d5a00 2024-05-10T10:42:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.487+0000 7fef6759e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fef600d2ff0 con 0x7fef600d3bd0 2024-05-10T10:42:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.487+0000 7fef6659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fef6015f1f0 0x7fef600d34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36338/0 (socket says 172.21.15.53:36338) 2024-05-10T10:42:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.487+0000 7fef6659c640 1 -- 172.21.15.53:0/2983768753 learned_addr learned my addr 172.21.15.53:0/2983768753 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883131618 0 0) 0x7fef600d0b90 con 0x7fef6015f1f0 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef4c003520 con 0x7fef6015f1f0 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef65d9b640 1 --2- 172.21.15.53:0/2983768753 >> v2:172.21.15.135:3300/0 conn(0x7fef600d3bd0 0x7fef600d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1961894102 0 0) 0x7fef4c003520 con 0x7fef6015f1f0 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 >> v2:172.21.15.135:3300/0 conn(0x7fef600d3bd0 msgr2=0x7fef600d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 --2- 172.21.15.53:0/2983768753 >> v2:172.21.15.135:3300/0 conn(0x7fef600d3bd0 0x7fef600d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 >> v1:172.21.15.135:6789/0 conn(0x7fef600d5a00 legacy=0x7fef600b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef600b4750 con 0x7fef6015f1f0 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fef580019f0 con 0x7fef6015f1f0 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 >> v1:172.21.15.53:6789/0 conn(0x7fef6015f1f0 legacy=0x7fef600d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef4c0039b0 0x7fef4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c004800 0x7fef4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c0052e0 0x7fef4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef600d2ff0 con 0x7fef4c0052e0 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c00c1b0 con 0x7fef4c004800 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c00c3b0 con 0x7fef4c0039b0 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef66d9d640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c0052e0 0x7fef4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef65d9b640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c004800 0x7fef4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.488+0000 7fef6659c640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef4c0039b0 0x7fef4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef66d9d640 1 -- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef4c0039b0 msgr2=0x7fef4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef66d9d640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef4c0039b0 0x7fef4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef66d9d640 1 -- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c004800 msgr2=0x7fef4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef66d9d640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c004800 0x7fef4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef66d9d640 1 -- 172.21.15.53:0/2983768753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c003520 con 0x7fef4c0052e0 2024-05-10T10:42:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef66d9d640 1 -- 172.21.15.53:0/2983768753 reap_dead start 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef66d9d640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c0052e0 0x7fef4c00b9c0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fef5c001720 tx=0x7fef4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea9a6c44d6382a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c001370 con 0x7fef4c0052e0 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c0052d0 con 0x7fef4c0052e0 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.489+0000 7fef6559a640 1 -- 172.21.15.53:0/2983768753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c02abc0 con 0x7fef4c0052e0 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.490+0000 7fef6759e640 1 -- 172.21.15.53:0/2983768753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c0052e0 msgr2=0x7fef4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.490+0000 7fef6759e640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c0052e0 0x7fef4c00b9c0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fef5c001720 tx=0x7fef4c005830 comp rx=0 tx=0).stop 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.490+0000 7fef6759e640 1 -- 172.21.15.53:0/2983768753 shutdown_connections 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.490+0000 7fef6759e640 1 --2- 172.21.15.53:0/2983768753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c0052e0 0x7fef4c00b9c0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.490+0000 7fef6759e640 1 -- 172.21.15.53:0/2983768753 >> 172.21.15.53:0/2983768753 conn(0x7fef600d4ee0 msgr2=0x7fef600d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.490+0000 7fef6759e640 1 -- 172.21.15.53:0/2983768753 shutdown_connections 2024-05-10T10:42:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.490+0000 7fef6759e640 1 -- 172.21.15.53:0/2983768753 wait complete. 2024-05-10T10:42:44.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.491+0000 7fef6759e640 1 Processor -- start 2024-05-10T10:42:44.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.491+0000 7fef6759e640 1 -- start start 2024-05-10T10:42:44.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef6759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c0052e0 0x7fef6015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef6759e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef600cdbc0 0x7fef600ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef6759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef600b9cb0 0x7fef600ba130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef6759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef600d0500 con 0x7fef600b9cb0 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef6759e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef600d5e90 con 0x7fef4c0052e0 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef6759e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fef600d0680 con 0x7fef600cdbc0 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef66d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef600b9cb0 0x7fef600ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef66d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef600b9cb0 0x7fef600ba130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48682/0 (socket says 172.21.15.53:48682) 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef66d9d640 1 -- 172.21.15.53:0/976733268 learned_addr learned my addr 172.21.15.53:0/976733268 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef66d9d640 1 -- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef600cdbc0 msgr2=0x7fef600ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.492+0000 7fef66d9d640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef600cdbc0 0x7fef600ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef66d9d640 1 -- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c0052e0 msgr2=0x7fef6015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef6659c640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c0052e0 0x7fef6015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef66d9d640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c0052e0 0x7fef6015cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef66d9d640 1 -- 172.21.15.53:0/976733268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef601f4c70 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef66d9d640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef600b9cb0 0x7fef600ba130 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005040 tx=0x7fef5c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef56ffd640 1 -- 172.21.15.53:0/976733268 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c007a30 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef56ffd640 1 -- 172.21.15.53:0/976733268 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c005480 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef600cea40 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef56ffd640 1 -- 172.21.15.53:0/976733268 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c03d650 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.493+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef600cc410 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.494+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6006a220 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.495+0000 7fef56ffd640 1 -- 172.21.15.53:0/976733268 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef5c032440 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.495+0000 7fef56ffd640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.495+0000 7fef56ffd640 1 -- 172.21.15.53:0/976733268 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7fef5c0b6e10 con 0x7fef600b9cb0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.496+0000 7fef6659c640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.496+0000 7fef6659c640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fef580013d0 tx=0x7fef58006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:44.500 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.499+0000 7fef56ffd640 1 -- 172.21.15.53:0/976733268 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fef5c080920 con 0x7fef600b9cb0 2024-05-10T10:42:44.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.729+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef6006e0e0 con 0x7fef600b9cb0 2024-05-10T10:42:44.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.730+0000 7fef56ffd640 1 -- 172.21.15.53:0/976733268 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+3875 (secure 0 0 0) 0x7fef5c080220 con 0x7fef600b9cb0 2024-05-10T10:42:44.730 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:44.731 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3661048242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3661048242},{"type":"v1","addr":"172.21.15.135:6836","nonce":3661048242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/3650836231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3650836231},{"type":"v1","addr":"172.21.15.135:6837","nonce":3650836231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2246810847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2246810847},{"type":"v1","addr":"172.21.15.53:6833","nonce":2246810847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7896,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1980081186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1980081186},{"type":"v1","addr":"172.21.15.53:6835","nonce":1980081186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-05-10T10:42:44.731 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 92 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 msgr2=0x7fef44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fef580013d0 tx=0x7fef58006040 comp rx=0 tx=0).stop 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef600b9cb0 msgr2=0x7fef600ba130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef600b9cb0 0x7fef600ba130 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005040 tx=0x7fef5c001190 comp rx=0 tx=0).stop 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 shutdown_connections 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef600b9cb0 0x7fef600ba130 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef600cdbc0 0x7fef600ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 --2- 172.21.15.53:0/976733268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef4c0052e0 0x7fef6015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 >> 172.21.15.53:0/976733268 conn(0x7fef600d4ee0 msgr2=0x7fef600d52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 shutdown_connections 2024-05-10T10:42:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.732+0000 7fef6759e640 1 -- 172.21.15.53:0/976733268 wait complete. 2024-05-10T10:42:44.744 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T10:42:44.744 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T10:42:44.744 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:42:44.744 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:42:44.745 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T10:42:44.745 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T10:42:44.745 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:42:44.745 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:42:44.745 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T10:42:44.746 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T10:42:44.746 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:42:44.746 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:42:44.746 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T10:42:44.746 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T10:42:44.746 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:42:44.747 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:42:44.754 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:42:44.754 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T10:42:44.755 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:42:44.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T10:42:44.757 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:42:44.757 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T10:42:44.758 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:42:44.758 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T10:42:44.797 INFO:tasks.ceph.mds.a:Started 2024-05-10T10:42:44.797 DEBUG:teuthology.parallel:result is None 2024-05-10T10:42:44.798 INFO:tasks.ceph.mds.c:Started 2024-05-10T10:42:44.798 DEBUG:teuthology.parallel:result is None 2024-05-10T10:42:44.800 INFO:tasks.ceph.mds.b:Started 2024-05-10T10:42:44.800 DEBUG:teuthology.parallel:result is None 2024-05-10T10:42:44.801 INFO:tasks.ceph.mds.d:Started 2024-05-10T10:42:44.801 DEBUG:teuthology.parallel:result is None 2024-05-10T10:42:44.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T10:42:44.875 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:42:44.875 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2024-05-10T10:42:44.881 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2024-05-10T10:42:44.881 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:42:44.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.912+0000 7f76e4746640 1 Processor -- start 2024-05-10T10:42:44.912 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.912+0000 7f76e4746640 1 -- start start 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e4746640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e4746640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f76dc0d5cb0 0x7f76dc0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e4746640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f76dc162b30 0x7f76dc0d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e4746640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f76dc0b4f20 con 0x7f76dc0d5cb0 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e4746640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f76dc0d0fe0 con 0x7f76dc162b30 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e4746640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc0b45f0 con 0x7f76dc162750 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f76dc0d5cb0 0x7f76dc0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f76dc0d5cb0 0x7f76dc0d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48726/0 (socket says 172.21.15.53:48726) 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 -- 172.21.15.53:0/220528950 learned_addr learned my addr 172.21.15.53:0/220528950 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e2ce3640 1 --2- 172.21.15.53:0/220528950 >> v2:172.21.15.135:3300/0 conn(0x7f76dc162b30 0x7f76dc0d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e24e2640 1 --2- 172.21.15.53:0/220528950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 -- 172.21.15.53:0/220528950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 msgr2=0x7f76dc0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 --2- 172.21.15.53:0/220528950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 -- 172.21.15.53:0/220528950 >> v2:172.21.15.135:3300/0 conn(0x7f76dc162b30 msgr2=0x7f76dc0d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 --2- 172.21.15.53:0/220528950 >> v2:172.21.15.135:3300/0 conn(0x7f76dc162b30 0x7f76dc0d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.913+0000 7f76e1ce1640 1 -- 172.21.15.53:0/220528950 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc0b48d0 con 0x7f76dc0d5cb0 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.914+0000 7f76e1ce1640 1 --2- 172.21.15.53:0/220528950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc0d5cb0 0x7f76dc0d3630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0cc0d0 tx=0x7f76cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367e9762080fa386 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.914+0000 7f76e14e0640 1 -- 172.21.15.53:0/220528950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc037070 con 0x7f76dc0d5cb0 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.914+0000 7f76e14e0640 1 -- 172.21.15.53:0/220528950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc032440 con 0x7f76dc0d5cb0 2024-05-10T10:42:44.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.914+0000 7f76e14e0640 1 -- 172.21.15.53:0/220528950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc032810 con 0x7f76dc0d5cb0 2024-05-10T10:42:44.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 -- 172.21.15.53:0/220528950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc0d5cb0 msgr2=0x7f76dc0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 --2- 172.21.15.53:0/220528950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc0d5cb0 0x7f76dc0d3630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0cc0d0 tx=0x7f76cc02a140 comp rx=0 tx=0).stop 2024-05-10T10:42:44.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 -- 172.21.15.53:0/220528950 shutdown_connections 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 --2- 172.21.15.53:0/220528950 >> v2:172.21.15.135:3300/0 conn(0x7f76dc162b30 0x7f76dc0d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 --2- 172.21.15.53:0/220528950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc0d5cb0 0x7f76dc0d3630 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 --2- 172.21.15.53:0/220528950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 -- 172.21.15.53:0/220528950 >> 172.21.15.53:0/220528950 conn(0x7f76dc09e3b0 msgr2=0x7f76dc0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 -- 172.21.15.53:0/220528950 shutdown_connections 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.915+0000 7f76e4746640 1 -- 172.21.15.53:0/220528950 wait complete. 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.916+0000 7f76e4746640 1 Processor -- start 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.916+0000 7f76e4746640 1 -- start start 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e4746640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e4746640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc162b30 0x7f76dc153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e4746640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76dc15c720 0x7f76dc0c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e4746640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc0d0d30 con 0x7f76dc162b30 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e4746640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc0d0bb0 con 0x7f76dc15c720 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e4746640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc0d1660 con 0x7f76dc162750 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc162b30 0x7f76dc153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc162b30 0x7f76dc153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48730/0 (socket says 172.21.15.53:48730) 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 -- 172.21.15.53:0/3355393925 learned_addr learned my addr 172.21.15.53:0/3355393925 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e24e2640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e2ce3640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76dc15c720 0x7f76dc0c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 -- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 msgr2=0x7f76dc0b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 -- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76dc15c720 msgr2=0x7f76dc0c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76dc15c720 0x7f76dc0c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:44.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e1ce1640 1 -- 172.21.15.53:0/3355393925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc0b8180 con 0x7f76dc162b30 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.917+0000 7f76e2ce3640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76dc15c720 0x7f76dc0c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.918+0000 7f76e1ce1640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc162b30 0x7f76dc153460 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f76cc02a180 tx=0x7f76cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.918+0000 7f76d2ffd640 1 -- 172.21.15.53:0/3355393925 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc041020 con 0x7f76dc162b30 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.918+0000 7f76d2ffd640 1 -- 172.21.15.53:0/3355393925 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc03d620 con 0x7f76dc162b30 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.918+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc0cdc30 con 0x7f76dc162b30 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.918+0000 7f76d2ffd640 1 -- 172.21.15.53:0/3355393925 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc037040 con 0x7f76dc162b30 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.918+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc0ce1f0 con 0x7f76dc162b30 2024-05-10T10:42:44.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.919+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76dc06a220 con 0x7f76dc162b30 2024-05-10T10:42:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.919+0000 7f76d2ffd640 1 -- 172.21.15.53:0/3355393925 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f76cc044020 con 0x7f76dc162b30 2024-05-10T10:42:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.920+0000 7f76d2ffd640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76c00762d0 0x7f76c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.920+0000 7f76d2ffd640 1 -- 172.21.15.53:0/3355393925 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5421+0+0 (secure 0 0 0) 0x7f76cc0b7b20 con 0x7f76dc162b30 2024-05-10T10:42:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.920+0000 7f76e24e2640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76c00762d0 0x7f76c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:44.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.921+0000 7f76e24e2640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76c00762d0 0x7f76c0078790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009990 tx=0x7f76d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:44.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:44.924+0000 7f76d2ffd640 1 -- 172.21.15.53:0/3355393925 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f76cc04b030 con 0x7f76dc162b30 2024-05-10T10:42:45.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:45.127+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f76dc15eb80 con 0x7f76dc162b30 2024-05-10T10:42:45.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:42:45.795 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:42:46.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.200+0000 7f76d2ffd640 1 -- 172.21.15.53:0/3355393925 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) ==== 97+0+0 (secure 0 0 0) 0x7f76cc081630 con 0x7f76dc162b30 2024-05-10T10:42:46.200 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2024-05-10T10:42:46.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.202+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76c00762d0 msgr2=0x7f76c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.202+0000 7f76e4746640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76c00762d0 0x7f76c0078790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009990 tx=0x7f76d8008040 comp rx=0 tx=0).stop 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.202+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc162b30 msgr2=0x7f76dc153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.202+0000 7f76e4746640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc162b30 0x7f76dc153460 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f76cc02a180 tx=0x7f76cc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 shutdown_connections 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f76c00762d0 0x7f76c0078790 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76dc15c720 0x7f76dc0c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76dc162b30 0x7f76dc153460 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 --2- 172.21.15.53:0/3355393925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f76dc162750 0x7f76dc0b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 >> 172.21.15.53:0/3355393925 conn(0x7f76dc09e3b0 msgr2=0x7f76dc06da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 shutdown_connections 2024-05-10T10:42:46.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.203+0000 7f76e4746640 1 -- 172.21.15.53:0/3355393925 wait complete. 2024-05-10T10:42:46.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T10:42:46.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.327+0000 7ff86e732640 1 Processor -- start 2024-05-10T10:42:46.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.327+0000 7ff86e732640 1 -- start start 2024-05-10T10:42:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff86e732640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff86815b6d0 0x7ff8680c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff86e732640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff86815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff86e732640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8680c5d10 con 0x7ff8680cf5d0 2024-05-10T10:42:46.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff86e732640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff8680cc380 con 0x7ff86815b6d0 2024-05-10T10:42:46.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff86e732640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680c5ff0 con 0x7ff8680cf170 2024-05-10T10:42:46.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff867fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff86815b6d0 0x7ff8680c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff8677fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff86815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff86cccf640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff8680cf5d0 0x7ff86806dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:49890/0 (socket says 172.21.15.53:49890) 2024-05-10T10:42:46.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff86cccf640 1 -- 172.21.15.53:0/3085930386 learned_addr learned my addr 172.21.15.53:0/3085930386 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff867fff640 1 -- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 msgr2=0x7ff86815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff867fff640 1 --2- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff86815f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff867fff640 1 -- 172.21.15.53:0/3085930386 >> v1:172.21.15.135:6789/0 conn(0x7ff8680cf5d0 legacy=0x7ff86806dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff867fff640 1 -- 172.21.15.53:0/3085930386 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8680c62d0 con 0x7ff86815b6d0 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff866ffd640 1 -- 172.21.15.53:0/3085930386 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 489745390 0 0) 0x7ff8680c5d10 con 0x7ff8680cf5d0 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.328+0000 7ff8677fe640 1 --2- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff86815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.329+0000 7ff867fff640 1 --2- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 0x7ff8680c6db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097a0 tx=0x7ff85802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e5f55ad908de81 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.329+0000 7ff866ffd640 1 -- 172.21.15.53:0/3085930386 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858037070 con 0x7ff86815b6d0 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.329+0000 7ff866ffd640 1 -- 172.21.15.53:0/3085930386 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff858032440 con 0x7ff86815b6d0 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.329+0000 7ff866ffd640 1 -- 172.21.15.53:0/3085930386 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858032810 con 0x7ff86815b6d0 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 -- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 msgr2=0x7ff8680c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 --2- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 0x7ff8680c6db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8580097a0 tx=0x7ff85802a210 comp rx=0 tx=0).stop 2024-05-10T10:42:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 -- 172.21.15.53:0/3085930386 shutdown_connections 2024-05-10T10:42:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 --2- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff86815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 --2- 172.21.15.53:0/3085930386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 0x7ff8680c6db0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 -- 172.21.15.53:0/3085930386 >> 172.21.15.53:0/3085930386 conn(0x7ff8680b2900 msgr2=0x7ff8680b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 -- 172.21.15.53:0/3085930386 shutdown_connections 2024-05-10T10:42:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.330+0000 7ff86e732640 1 -- 172.21.15.53:0/3085930386 wait complete. 2024-05-10T10:42:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.331+0000 7ff86e732640 1 Processor -- start 2024-05-10T10:42:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.331+0000 7ff86e732640 1 -- start start 2024-05-10T10:42:46.334 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff86e732640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff868167730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff86e732640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8680cf5d0 0x7ff868071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff86e732640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 0x7ff868072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff86e732640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680cd170 con 0x7ff8680cf5d0 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff86e732640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680c6890 con 0x7ff86815b6d0 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff86e732640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680cd2f0 con 0x7ff8680cf170 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff8677fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8680cf5d0 0x7ff868071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff8677fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8680cf5d0 0x7ff868071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48746/0 (socket says 172.21.15.53:48746) 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff8677fe640 1 -- 172.21.15.53:0/2257738305 learned_addr learned my addr 172.21.15.53:0/2257738305 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff867fff640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff868167730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.332+0000 7ff86cccf640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 0x7ff868072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff8677fe640 1 -- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 msgr2=0x7ff868167730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff8677fe640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff868167730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff8677fe640 1 -- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 msgr2=0x7ff868072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff8677fe640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 0x7ff868072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff8677fe640 1 -- 172.21.15.53:0/2257738305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86815c830 con 0x7ff8680cf5d0 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff8677fe640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8680cf5d0 0x7ff868071df0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00ebe0 tx=0x7ff85c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff864ff9640 1 -- 172.21.15.53:0/2257738305 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c017070 con 0x7ff8680cf5d0 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff864ff9640 1 -- 172.21.15.53:0/2257738305 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff85c00cc30 con 0x7ff8680cf5d0 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff864ff9640 1 -- 172.21.15.53:0/2257738305 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c013470 con 0x7ff8680cf5d0 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff868072920 con 0x7ff8680cf5d0 2024-05-10T10:42:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.333+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86815cd90 con 0x7ff8680cf5d0 2024-05-10T10:42:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.334+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86806a220 con 0x7ff8680cf5d0 2024-05-10T10:42:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.335+0000 7ff864ff9640 1 -- 172.21.15.53:0/2257738305 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff85c01a070 con 0x7ff8680cf5d0 2024-05-10T10:42:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.335+0000 7ff864ff9640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff8540762d0 0x7ff854078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.336+0000 7ff864ff9640 1 -- 172.21.15.53:0/2257738305 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7ff85c0976a0 con 0x7ff8680cf5d0 2024-05-10T10:42:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.336+0000 7ff867fff640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff8540762d0 0x7ff854078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.336+0000 7ff867fff640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff8540762d0 0x7ff854078790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff858034000 tx=0x7ff85802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:46.340 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.340+0000 7ff864ff9640 1 -- 172.21.15.53:0/2257738305 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff86815c830 con 0x7ff8680cf5d0 2024-05-10T10:42:46.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.607+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff86815ee60 con 0x7ff8680cf5d0 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.608+0000 7ff864ff9640 1 -- 172.21.15.53:0/2257738305 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7ff85c097020 con 0x7ff8680cf5d0 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mf19yAxAAp32WNDMi4GHvoCyh725kwg==", 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.609 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mdyWRDxAAGP+jfw/2DP7lh0/FLbaylQ==", 2024-05-10T10:42:46.610 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbDmlChAAX7dUe1y78rc/jVifHhNxhA==", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mbuqqFhAAzHVMceG7ceq7aVXiBjkVew==", 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDP9z1mlIdkOxAAhs9cwCPIGJEGt1wg7Tr9SA==", 2024-05-10T10:42:46.612 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDS9z1mvRNvOhAAr20oCsk43bDyYvrgSUl5vA==", 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDV9z1m9NNLORAAINe5Dz8uXiJq3HOGl6yNkw==", 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDY9z1m0D9GNxAAFHuPNvIz9jwE4K8CcKVYcA==", 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.614 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDe9z1m9TKqORAAg3c+jAbAghx5c1kd/LvR8g==", 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2024-05-10T10:42:46.615 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDh9z1mnwVFORAAtNYNg5Pk33HVPNheUtRo7A==", 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDk9z1m+ZW1OBAAYCR/jUHLgps2VpLqtHBCOA==", 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDn9z1mRhmMNxAAWESbSYd9zruxJH7QtAa9xg==", 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg==", 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.617 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mB0PBIxAAWtW2R3RU5J052mX+2yvK1g==", 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1maySOIBAA45MPGRgOMwVANYnDl4FoPw==", 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1mwhxWKRAAHo0VZyCFgor1QbdcC4vvcw==", 2024-05-10T10:42:46.619 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1mKY2tIhAAzPSpOQLOFy1GEKdl5bPBEw==", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9z1m4wccABAANNzshDiyxuNaPofk9NUDiw==", 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDN9z1m5SDsNxAAGfQAb5xAJJ9qgFPoFxbjXQ==", 2024-05-10T10:42:46.621 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2024-05-10T10:42:46.622 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2024-05-10T10:42:46.622 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2024-05-10T10:42:46.622 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2024-05-10T10:42:46.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-10T10:42:46.622 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:46.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff8540762d0 msgr2=0x7ff854078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff8540762d0 0x7ff854078790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff858034000 tx=0x7ff85802e000 comp rx=0 tx=0).stop 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8680cf5d0 msgr2=0x7ff868071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8680cf5d0 0x7ff868071df0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00ebe0 tx=0x7ff85c00c520 comp rx=0 tx=0).stop 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 shutdown_connections 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff8540762d0 0x7ff854078790 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86815b6d0 0x7ff868072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8680cf5d0 0x7ff868071df0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 --2- 172.21.15.53:0/2257738305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8680cf170 0x7ff868167730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 >> 172.21.15.53:0/2257738305 conn(0x7ff8680b2900 msgr2=0x7ff8680cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 shutdown_connections 2024-05-10T10:42:46.624 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.610+0000 7ff86e732640 1 -- 172.21.15.53:0/2257738305 wait complete. 2024-05-10T10:42:46.625 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T10:42:46.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.728+0000 7f7c459dd640 1 Processor -- start 2024-05-10T10:42:46.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c459dd640 1 -- start start 2024-05-10T10:42:46.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c459dd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7c4015f9d0 0x7f7c400d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c459dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c400cfe00 0x7f7c400d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c459dd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c4006eb40 con 0x7f7c4015f5f0 2024-05-10T10:42:46.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c459dd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4006ede0 con 0x7f7c4015f9d0 2024-05-10T10:42:46.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c459dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4006f0c0 con 0x7f7c400cfe00 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c3e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7c4015f9d0 0x7f7c400d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.729+0000 7f7c3effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7c4015f5f0 0x7f7c400cf6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36354/0 (socket says 172.21.15.53:36354) 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3effd640 1 -- 172.21.15.53:0/2503248881 learned_addr learned my addr 172.21.15.53:0/2503248881 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3f7fe640 1 --2- 172.21.15.53:0/2503248881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c400cfe00 0x7f7c400d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3e7fc640 1 -- 172.21.15.53:0/2503248881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c400cfe00 msgr2=0x7f7c400d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3e7fc640 1 --2- 172.21.15.53:0/2503248881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c400cfe00 0x7f7c400d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3dffb640 1 -- 172.21.15.53:0/2503248881 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2177307244 0 0) 0x7f7c4006eb40 con 0x7f7c4015f5f0 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3e7fc640 1 -- 172.21.15.53:0/2503248881 >> v1:172.21.15.53:6789/0 conn(0x7f7c4015f5f0 legacy=0x7f7c400cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3e7fc640 1 -- 172.21.15.53:0/2503248881 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4006f3a0 con 0x7f7c4015f9d0 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.730+0000 7f7c3f7fe640 1 --2- 172.21.15.53:0/2503248881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c400cfe00 0x7f7c400d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c3e7fc640 1 --2- 172.21.15.53:0/2503248881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f9d0 0x7f7c400d1b90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7c340097a0 tx=0x7f7c3402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54cec34051e4e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:46.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c3dffb640 1 -- 172.21.15.53:0/2503248881 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34037070 con 0x7f7c4015f9d0 2024-05-10T10:42:46.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c3dffb640 1 -- 172.21.15.53:0/2503248881 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3402ae50 con 0x7f7c4015f9d0 2024-05-10T10:42:46.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c3dffb640 1 -- 172.21.15.53:0/2503248881 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34032660 con 0x7f7c4015f9d0 2024-05-10T10:42:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 -- 172.21.15.53:0/2503248881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f9d0 msgr2=0x7f7c400d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 --2- 172.21.15.53:0/2503248881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f9d0 0x7f7c400d1b90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7c340097a0 tx=0x7f7c3402a100 comp rx=0 tx=0).stop 2024-05-10T10:42:46.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 -- 172.21.15.53:0/2503248881 shutdown_connections 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 --2- 172.21.15.53:0/2503248881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c400cfe00 0x7f7c400d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 --2- 172.21.15.53:0/2503248881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f9d0 0x7f7c400d1b90 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 -- 172.21.15.53:0/2503248881 >> 172.21.15.53:0/2503248881 conn(0x7f7c400d2e70 msgr2=0x7f7c400d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 -- 172.21.15.53:0/2503248881 shutdown_connections 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.731+0000 7f7c459dd640 1 -- 172.21.15.53:0/2503248881 wait complete. 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 Processor -- start 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 -- start start 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c400cfe00 0x7f7c400b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f5f0 0x7f7c4014e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c4015f9d0 0x7f7c4014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c400cdc00 con 0x7f7c4015f5f0 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c400cda80 con 0x7f7c400cfe00 2024-05-10T10:42:46.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.733+0000 7f7c459dd640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7c400cdd80 con 0x7f7c4015f9d0 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f5f0 0x7f7c4014e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f5f0 0x7f7c4014e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48760/0 (socket says 172.21.15.53:48760) 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 -- 172.21.15.53:0/4140953117 learned_addr learned my addr 172.21.15.53:0/4140953117 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3effd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c400cfe00 0x7f7c400b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3f7fe640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c4015f9d0 0x7f7c4014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 -- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c4015f9d0 msgr2=0x7f7c4014ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c4015f9d0 0x7f7c4014ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 -- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c400cfe00 msgr2=0x7f7c400b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c400cfe00 0x7f7c400b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 -- 172.21.15.53:0/4140953117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4014f5d0 con 0x7f7c4015f5f0 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3f7fe640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c4015f9d0 0x7f7c4014ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c3e7fc640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f5f0 0x7f7c4014e950 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7c34035040 tx=0x7f7c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c27fff640 1 -- 172.21.15.53:0/4140953117 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34041070 con 0x7f7c4015f5f0 2024-05-10T10:42:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c27fff640 1 -- 172.21.15.53:0/4140953117 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3403c670 con 0x7f7c4015f5f0 2024-05-10T10:42:46.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4014f8b0 con 0x7f7c4015f5f0 2024-05-10T10:42:46.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c27fff640 1 -- 172.21.15.53:0/4140953117 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34037040 con 0x7f7c4015f5f0 2024-05-10T10:42:46.738 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.734+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c400b68e0 con 0x7f7c4015f5f0 2024-05-10T10:42:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.735+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c400693f0 con 0x7f7c4015f5f0 2024-05-10T10:42:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.736+0000 7f7c27fff640 1 -- 172.21.15.53:0/4140953117 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7c3403a070 con 0x7f7c4015f5f0 2024-05-10T10:42:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.737+0000 7f7c27fff640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7c2c0762d0 0x7f7c2c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.737+0000 7f7c27fff640 1 -- 172.21.15.53:0/4140953117 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f7c340b7700 con 0x7f7c4015f5f0 2024-05-10T10:42:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.737+0000 7f7c3effd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7c2c0762d0 0x7f7c2c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.737+0000 7f7c3effd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7c2c0762d0 0x7f7c2c078790 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7c300012e0 tx=0x7f7c30001700 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:46.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.741+0000 7f7c27fff640 1 -- 172.21.15.53:0/4140953117 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7c34080c60 con 0x7f7c4015f5f0 2024-05-10T10:42:46.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.938+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7c400d04e0 con 0x7f7c4015f5f0 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.938+0000 7f7c27fff640 1 -- 172.21.15.53:0/4140953117 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7f7c34084f60 con 0x7f7c4015f5f0 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-10T10:42:46.939 INFO:teuthology.orchestra.run.smithi053.stdout:] 2024-05-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7c2c0762d0 msgr2=0x7f7c2c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7c2c0762d0 0x7f7c2c078790 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7c300012e0 tx=0x7f7c30001700 comp rx=0 tx=0).stop 2024-05-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f5f0 msgr2=0x7f7c4014e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f5f0 0x7f7c4014e950 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7c34035040 tx=0x7f7c340046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 shutdown_connections 2024-05-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7c2c0762d0 0x7f7c2c078790 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c4015f9d0 0x7f7c4014ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c4015f5f0 0x7f7c4014e950 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 --2- 172.21.15.53:0/4140953117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c400cfe00 0x7f7c400b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 >> 172.21.15.53:0/4140953117 conn(0x7f7c400d2e70 msgr2=0x7f7c40071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 shutdown_connections 2024-05-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:46.940+0000 7f7c459dd640 1 -- 172.21.15.53:0/4140953117 wait complete. 2024-05-10T10:42:46.952 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T10:42:46.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T10:42:47.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.060+0000 7f3ba2e99640 1 Processor -- start 2024-05-10T10:42:47.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.060+0000 7f3ba2e99640 1 -- start start 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba2e99640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b9c06e820 0x7f3b9c0d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba2e99640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b9c0cbb00 0x7f3b9c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba2e99640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9c0d12a0 con 0x7f3b9c070020 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba2e99640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c0cf120 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba2e99640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c0cf400 con 0x7f3b9c06e820 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba1436640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b9c0cbb00 0x7f3b9c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba1436640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b9c0cbb00 0x7f3b9c06d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48776/0 (socket says 172.21.15.53:48776) 2024-05-10T10:42:47.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba1436640 1 -- 172.21.15.53:0/235507962 learned_addr learned my addr 172.21.15.53:0/235507962 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:47.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba1436640 1 -- 172.21.15.53:0/235507962 >> v2:172.21.15.135:3300/0 conn(0x7f3b9c06e820 msgr2=0x7f3b9c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:47.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba0c35640 1 --2- 172.21.15.53:0/235507962 >> v2:172.21.15.135:3300/0 conn(0x7f3b9c06e820 0x7f3b9c0d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:47.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba1436640 1 --2- 172.21.15.53:0/235507962 >> v2:172.21.15.135:3300/0 conn(0x7f3b9c06e820 0x7f3b9c0d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:47.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba1436640 1 -- 172.21.15.53:0/235507962 >> v1:172.21.15.135:6789/0 conn(0x7f3b9c070020 legacy=0x7f3b9c06ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:47.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba1436640 1 -- 172.21.15.53:0/235507962 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c0cf6e0 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.061+0000 7f3ba0c35640 1 --2- 172.21.15.53:0/235507962 >> v2:172.21.15.135:3300/0 conn(0x7f3b9c06e820 0x7f3b9c0d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:47.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.062+0000 7f3ba1436640 1 --2- 172.21.15.53:0/235507962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c06d090 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b41a0 tx=0x7f3b9402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c37c8fdd3fbcaac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:47.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.062+0000 7f3b9b7fe640 1 -- 172.21.15.53:0/235507962 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94036070 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.062+0000 7f3b9b7fe640 1 -- 172.21.15.53:0/235507962 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b94031600 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.062+0000 7f3b9b7fe640 1 -- 172.21.15.53:0/235507962 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940319d0 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 -- 172.21.15.53:0/235507962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 msgr2=0x7f3b9c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/235507962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c06d090 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b41a0 tx=0x7f3b9402a6f0 comp rx=0 tx=0).stop 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 -- 172.21.15.53:0/235507962 shutdown_connections 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/235507962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c06d090 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/235507962 >> v2:172.21.15.135:3300/0 conn(0x7f3b9c06e820 0x7f3b9c0d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 -- 172.21.15.53:0/235507962 >> 172.21.15.53:0/235507962 conn(0x7f3b9c0cdf30 msgr2=0x7f3b9c0ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 -- 172.21.15.53:0/235507962 shutdown_connections 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.063+0000 7f3ba2e99640 1 -- 172.21.15.53:0/235507962 wait complete. 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.064+0000 7f3ba2e99640 1 Processor -- start 2024-05-10T10:42:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.064+0000 7f3ba2e99640 1 -- start start 2024-05-10T10:42:47.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba2e99640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b9c06e820 0x7f3b9c0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:47.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba2e99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b9c070020 0x7f3b9c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:47.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba2e99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba2e99640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0cbf90 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba2e99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c070590 con 0x7f3b9c070020 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba2e99640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c1607a0 con 0x7f3b9c06e820 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c0c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48780/0 (socket says 172.21.15.53:48780) 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 -- 172.21.15.53:0/2041081040 learned_addr learned my addr 172.21.15.53:0/2041081040 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3b9bfff640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b9c070020 0x7f3b9c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba0c35640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b9c06e820 0x7f3b9c0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 -- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b9c06e820 msgr2=0x7f3b9c0b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b9c06e820 0x7f3b9c0b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 -- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b9c070020 msgr2=0x7f3b9c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b9c070020 0x7f3b9c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.065+0000 7f3ba1436640 1 -- 172.21.15.53:0/2041081040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c159a90 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.066+0000 7f3ba1436640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c0c92b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3b94034040 tx=0x7f3b94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:47.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.066+0000 7f3b997fa640 1 -- 172.21.15.53:0/2041081040 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94036050 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.066+0000 7f3b997fa640 1 -- 172.21.15.53:0/2041081040 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b94042070 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.066+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c0b1fd0 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.066+0000 7f3b997fa640 1 -- 172.21.15.53:0/2041081040 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94009510 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.066+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c0c8900 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.067+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9c06a220 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.067+0000 7f3b997fa640 1 -- 172.21.15.53:0/2041081040 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b94007bd0 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.068+0000 7f3b997fa640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b800762d0 0x7f3b80078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:47.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.068+0000 7f3b997fa640 1 -- 172.21.15.53:0/2041081040 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f3b940b8290 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.068+0000 7f3ba0c35640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b800762d0 0x7f3b80078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:47.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.069+0000 7f3ba0c35640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b800762d0 0x7f3b80078790 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000aac0 tx=0x7f3b90008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:47.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.072+0000 7f3b997fa640 1 -- 172.21.15.53:0/2041081040 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3b9c159a90 con 0x7f3b9c0cbb00 2024-05-10T10:42:47.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:47.262+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3b9c0cec50 con 0x7f3b9c0cbb00 2024-05-10T10:42:48.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.202+0000 7f3b997fa640 1 -- 172.21.15.53:0/2041081040 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f3b94081760 con 0x7f3b9c0cbb00 2024-05-10T10:42:48.203 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2024-05-10T10:42:48.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.205+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b800762d0 msgr2=0x7f3b80078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.205+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b800762d0 0x7f3b80078790 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000aac0 tx=0x7f3b90008040 comp rx=0 tx=0).stop 2024-05-10T10:42:48.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.205+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 msgr2=0x7f3b9c0c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.206 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.205+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c0c92b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3b94034040 tx=0x7f3b94004a40 comp rx=0 tx=0).stop 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 shutdown_connections 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3b800762d0 0x7f3b80078790 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b9c0cbb00 0x7f3b9c0c92b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b9c070020 0x7f3b9c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 --2- 172.21.15.53:0/2041081040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b9c06e820 0x7f3b9c0b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 >> 172.21.15.53:0/2041081040 conn(0x7f3b9c0cdf30 msgr2=0x7f3b9c0757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 shutdown_connections 2024-05-10T10:42:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.206+0000 7f3ba2e99640 1 -- 172.21.15.53:0/2041081040 wait complete. 2024-05-10T10:42:48.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T10:42:48.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.340+0000 7f36d957f640 1 Processor -- start 2024-05-10T10:42:48.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.340+0000 7f36d957f640 1 -- start start 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.341+0000 7f36d957f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cf3e0 0x7f36d40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.341+0000 7f36d957f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f36d40d2190 0x7f36d40cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.341+0000 7f36d957f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d406e720 con 0x7f36d40cd500 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.341+0000 7f36d957f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f36d406de80 con 0x7f36d40d2190 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.341+0000 7f36d957f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f36d406e160 con 0x7f36d40cf3e0 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d2ffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cf3e0 0x7f36d40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f36d40d2190 0x7f36d40cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d27fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f36d40cd500 0x7f36d40b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:49920/0 (socket says 172.21.15.53:49920) 2024-05-10T10:42:48.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d27fc640 1 -- 172.21.15.53:0/16902347 learned_addr learned my addr 172.21.15.53:0/16902347 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:48.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d1ffb640 1 -- 172.21.15.53:0/16902347 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1159231901 0 0) 0x7f36d406e720 con 0x7f36d40cd500 2024-05-10T10:42:48.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d1ffb640 1 -- 172.21.15.53:0/16902347 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36bc003520 con 0x7f36d40cd500 2024-05-10T10:42:48.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d2ffd640 1 -- 172.21.15.53:0/16902347 >> v2:172.21.15.135:3300/0 conn(0x7f36d40d2190 msgr2=0x7f36d40cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:48.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d2ffd640 1 --2- 172.21.15.53:0/16902347 >> v2:172.21.15.135:3300/0 conn(0x7f36d40d2190 0x7f36d40cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d2ffd640 1 -- 172.21.15.53:0/16902347 >> v1:172.21.15.135:6789/0 conn(0x7f36d40cd500 legacy=0x7f36d40b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d2ffd640 1 -- 172.21.15.53:0/16902347 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d406e440 con 0x7f36d40cf3e0 2024-05-10T10:42:48.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.342+0000 7f36d37fe640 1 --2- 172.21.15.53:0/16902347 >> v2:172.21.15.135:3300/0 conn(0x7f36d40d2190 0x7f36d40cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:48.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.343+0000 7f36d2ffd640 1 --2- 172.21.15.53:0/16902347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cf3e0 0x7f36d40cf8c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36c40097a0 tx=0x7f36c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae841d9e0d5b29df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.343+0000 7f36d1ffb640 1 -- 172.21.15.53:0/16902347 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4037070 con 0x7f36d40cf3e0 2024-05-10T10:42:48.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.343+0000 7f36d1ffb640 1 -- 172.21.15.53:0/16902347 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c4032440 con 0x7f36d40cf3e0 2024-05-10T10:42:48.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.343+0000 7f36d1ffb640 1 -- 172.21.15.53:0/16902347 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c40327f0 con 0x7f36d40cf3e0 2024-05-10T10:42:48.348 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 -- 172.21.15.53:0/16902347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cf3e0 msgr2=0x7f36d40cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 --2- 172.21.15.53:0/16902347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cf3e0 0x7f36d40cf8c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36c40097a0 tx=0x7f36c402a180 comp rx=0 tx=0).stop 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 -- 172.21.15.53:0/16902347 shutdown_connections 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 --2- 172.21.15.53:0/16902347 >> v2:172.21.15.135:3300/0 conn(0x7f36d40d2190 0x7f36d40cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 --2- 172.21.15.53:0/16902347 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cf3e0 0x7f36d40cf8c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 -- 172.21.15.53:0/16902347 >> 172.21.15.53:0/16902347 conn(0x7f36d40d2eb0 msgr2=0x7f36d40d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 -- 172.21.15.53:0/16902347 shutdown_connections 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.344+0000 7f36d957f640 1 -- 172.21.15.53:0/16902347 wait complete. 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.345+0000 7f36d957f640 1 Processor -- start 2024-05-10T10:42:48.349 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.345+0000 7f36d957f640 1 -- start start 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d957f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cd500 0x7f36d41600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d957f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d40cf3e0 0x7f36d4149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d957f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d40d2190 0x7f36d40cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d957f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36d40b34e0 con 0x7f36d40cf3e0 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d957f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36d40b4510 con 0x7f36d40d2190 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d957f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f36d40b3660 con 0x7f36d40cd500 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d40cf3e0 0x7f36d4149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d40cf3e0 0x7f36d4149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48796/0 (socket says 172.21.15.53:48796) 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 -- 172.21.15.53:0/1754827697 learned_addr learned my addr 172.21.15.53:0/1754827697 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d2ffd640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cd500 0x7f36d41600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.350 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d37fe640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d40d2190 0x7f36d40cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 -- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cd500 msgr2=0x7f36d41600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cd500 0x7f36d41600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 -- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d40d2190 msgr2=0x7f36d40cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d40d2190 0x7f36d40cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 -- 172.21.15.53:0/1754827697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d40cb510 con 0x7f36d40cf3e0 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d2ffd640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cd500 0x7f36d41600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d37fe640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d40d2190 0x7f36d40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.346+0000 7f36d27fc640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d40cf3e0 0x7f36d4149c40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c430 tx=0x7f36c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.347+0000 7f36bb7fe640 1 -- 172.21.15.53:0/1754827697 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8017070 con 0x7f36d40cf3e0 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.347+0000 7f36bb7fe640 1 -- 172.21.15.53:0/1754827697 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c8005d00 con 0x7f36d40cf3e0 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.347+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d40cb7b0 con 0x7f36d40cf3e0 2024-05-10T10:42:48.351 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.347+0000 7f36bb7fe640 1 -- 172.21.15.53:0/1754827697 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8007310 con 0x7f36d40cf3e0 2024-05-10T10:42:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.347+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d40cbd70 con 0x7f36d40cf3e0 2024-05-10T10:42:48.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.348+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d406a220 con 0x7f36d40cf3e0 2024-05-10T10:42:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.349+0000 7f36bb7fe640 1 -- 172.21.15.53:0/1754827697 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36c801a070 con 0x7f36d40cf3e0 2024-05-10T10:42:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.349+0000 7f36bb7fe640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36c00762d0 0x7f36c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.349+0000 7f36bb7fe640 1 -- 172.21.15.53:0/1754827697 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5619+0+0 (secure 0 0 0) 0x7f36c8096880 con 0x7f36d40cf3e0 2024-05-10T10:42:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.349+0000 7f36d2ffd640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36c00762d0 0x7f36c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:48.353 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.350+0000 7f36d2ffd640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36c00762d0 0x7f36c0078790 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f36c4034000 tx=0x7f36c402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:48.354 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.354+0000 7f36bb7fe640 1 -- 172.21.15.53:0/1754827697 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f36c805fbe0 con 0x7f36d40cf3e0 2024-05-10T10:42:48.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:48.546+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f36d40cd1e0 con 0x7f36d40cf3e0 2024-05-10T10:42:49.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.210+0000 7f36bb7fe640 1 -- 172.21.15.53:0/1754827697 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7f36c8063ee0 con 0x7f36d40cf3e0 2024-05-10T10:42:49.211 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2024-05-10T10:42:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36c00762d0 msgr2=0x7f36c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36c00762d0 0x7f36c0078790 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f36c4034000 tx=0x7f36c402e000 comp rx=0 tx=0).stop 2024-05-10T10:42:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d40cf3e0 msgr2=0x7f36d4149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d40cf3e0 0x7f36d4149c40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c430 tx=0x7f36c800c830 comp rx=0 tx=0).stop 2024-05-10T10:42:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 shutdown_connections 2024-05-10T10:42:49.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36c00762d0 0x7f36c0078790 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d40d2190 0x7f36d40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d40cf3e0 0x7f36d4149c40 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 --2- 172.21.15.53:0/1754827697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36d40cd500 0x7f36d41600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 >> 172.21.15.53:0/1754827697 conn(0x7f36d40d2eb0 msgr2=0x7f36d4162480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 shutdown_connections 2024-05-10T10:42:49.215 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.213+0000 7f36d957f640 1 -- 172.21.15.53:0/1754827697 wait complete. 2024-05-10T10:42:49.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T10:42:49.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.356+0000 7f4b94d71640 1 Processor -- start 2024-05-10T10:42:49.357 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.356+0000 7f4b94d71640 1 -- start start 2024-05-10T10:42:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.357+0000 7f4b94d71640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4b900b4f90 0x7f4b900b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.357+0000 7f4b94d71640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b900d2800 con 0x7f4b900d36d0 2024-05-10T10:42:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.357+0000 7f4b94d71640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b900d2ae0 con 0x7f4b9015f410 2024-05-10T10:42:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.357+0000 7f4b94d71640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4b900d01d0 con 0x7f4b900b4f90 2024-05-10T10:42:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.358+0000 7f4b8dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4b900b4f90 0x7f4b900b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.358+0000 7f4b8ed9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4b900d36d0 0x7f4b900d1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36358/0 (socket says 172.21.15.53:36358) 2024-05-10T10:42:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.358+0000 7f4b8ed9d640 1 -- 172.21.15.53:0/2746158009 learned_addr learned my addr 172.21.15.53:0/2746158009 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.358+0000 7f4b8dd9b640 1 -- 172.21.15.53:0/2746158009 >> v1:172.21.15.135:6789/0 conn(0x7f4b9015f410 legacy=0x7f4b900d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.358+0000 7f4b8dd9b640 1 -- 172.21.15.53:0/2746158009 >> v1:172.21.15.53:6789/0 conn(0x7f4b900d36d0 legacy=0x7f4b900d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.358+0000 7f4b8d59a640 1 -- 172.21.15.53:0/2746158009 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166632779 0 0) 0x7f4b900d2800 con 0x7f4b900d36d0 2024-05-10T10:42:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.358+0000 7f4b8dd9b640 1 -- 172.21.15.53:0/2746158009 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b900d04b0 con 0x7f4b900b4f90 2024-05-10T10:42:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b8dd9b640 1 --2- 172.21.15.53:0/2746158009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b5410 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4b840097a0 tx=0x7f4b8402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79577a49d6defc1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b8d59a640 1 -- 172.21.15.53:0/2746158009 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84037070 con 0x7f4b900b4f90 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b8d59a640 1 -- 172.21.15.53:0/2746158009 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b84032440 con 0x7f4b900b4f90 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b8d59a640 1 -- 172.21.15.53:0/2746158009 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84032810 con 0x7f4b900b4f90 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b94d71640 1 -- 172.21.15.53:0/2746158009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 msgr2=0x7f4b900b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b94d71640 1 --2- 172.21.15.53:0/2746158009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b5410 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4b840097a0 tx=0x7f4b8402a1a0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b94d71640 1 -- 172.21.15.53:0/2746158009 shutdown_connections 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b94d71640 1 --2- 172.21.15.53:0/2746158009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b5410 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b94d71640 1 -- 172.21.15.53:0/2746158009 >> 172.21.15.53:0/2746158009 conn(0x7f4b9008ed30 msgr2=0x7f4b9009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.359+0000 7f4b94d71640 1 -- 172.21.15.53:0/2746158009 shutdown_connections 2024-05-10T10:42:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.360+0000 7f4b94d71640 1 -- 172.21.15.53:0/2746158009 wait complete. 2024-05-10T10:42:49.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.361+0000 7f4b94d71640 1 Processor -- start 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.361+0000 7f4b94d71640 1 -- start start 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b94d71640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b94d71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b900d36d0 0x7f4b9015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b94d71640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b9015f410 0x7f4b900ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b94d71640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b900cffb0 con 0x7f4b900b4f90 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b94d71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b900cfe30 con 0x7f4b900d36d0 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b94d71640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4b900d10e0 con 0x7f4b9015f410 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48826/0 (socket says 172.21.15.53:48826) 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 -- 172.21.15.53:0/1059765185 learned_addr learned my addr 172.21.15.53:0/1059765185 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8dd9b640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b900d36d0 0x7f4b9015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8ed9d640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b9015f410 0x7f4b900ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 -- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b9015f410 msgr2=0x7f4b900ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b9015f410 0x7f4b900ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 -- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b900d36d0 msgr2=0x7f4b9015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b900d36d0 0x7f4b9015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 -- 172.21.15.53:0/1059765185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b900a0530 con 0x7f4b900b4f90 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b8e59c640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b88c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c400 tx=0x7f4b8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b7effd640 1 -- 172.21.15.53:0/1059765185 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80017070 con 0x7f4b900b4f90 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b7effd640 1 -- 172.21.15.53:0/1059765185 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b800052d0 con 0x7f4b900b4f90 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b900b9060 con 0x7f4b900b4f90 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.362+0000 7f4b7effd640 1 -- 172.21.15.53:0/1059765185 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80003830 con 0x7f4b900b4f90 2024-05-10T10:42:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.363+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b900b9620 con 0x7f4b900b4f90 2024-05-10T10:42:49.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.363+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b900cf930 con 0x7f4b900b4f90 2024-05-10T10:42:49.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.364+0000 7f4b7effd640 1 -- 172.21.15.53:0/1059765185 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b8001a070 con 0x7f4b900b4f90 2024-05-10T10:42:49.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.365+0000 7f4b7effd640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b700762d0 0x7f4b70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:49.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.365+0000 7f4b7effd640 1 -- 172.21.15.53:0/1059765185 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6001+0+0 (secure 0 0 0) 0x7f4b80096a20 con 0x7f4b900b4f90 2024-05-10T10:42:49.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.365+0000 7f4b8dd9b640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b700762d0 0x7f4b70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:49.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.365+0000 7f4b8dd9b640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b700762d0 0x7f4b70078790 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b84034000 tx=0x7f4b8402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:49.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.370+0000 7f4b7effd640 1 -- 172.21.15.53:0/1059765185 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4b8005fbf0 con 0x7f4b900b4f90 2024-05-10T10:42:49.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:49.598+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4b9006a220 con 0x7f4b900b4f90 2024-05-10T10:42:50.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.212+0000 7f4b7effd640 1 -- 172.21.15.53:0/1059765185 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v94) ==== 174+0+0 (secure 0 0 0) 0x7f4b80068230 con 0x7f4b900b4f90 2024-05-10T10:42:50.213 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 17 and data pool 18 2024-05-10T10:42:50.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b700762d0 msgr2=0x7f4b70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b700762d0 0x7f4b70078790 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b84034000 tx=0x7f4b8402e000 comp rx=0 tx=0).stop 2024-05-10T10:42:50.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 msgr2=0x7f4b900b88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b88c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c400 tx=0x7f4b8000c8c0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 shutdown_connections 2024-05-10T10:42:50.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b700762d0 0x7f4b70078790 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b9015f410 0x7f4b900ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b900d36d0 0x7f4b9015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 --2- 172.21.15.53:0/1059765185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b900b4f90 0x7f4b900b88c0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 >> 172.21.15.53:0/1059765185 conn(0x7f4b9008ed30 msgr2=0x7f4b900b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:50.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 shutdown_connections 2024-05-10T10:42:50.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.216+0000 7f4b94d71640 1 -- 172.21.15.53:0/1059765185 wait complete. 2024-05-10T10:42:50.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:42:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.358+0000 7f090c6e8640 1 Processor -- start 2024-05-10T10:42:50.359 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.359+0000 7f090c6e8640 1 -- start start 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.359+0000 7f090c6e8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0904162600 0x7f090406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090c6e8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0904162de0 0x7f09040ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090c6e8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09040cb190 con 0x7f09040cad80 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090c6e8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f09041635d0 con 0x7f0904162de0 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090c6e8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f09041638b0 con 0x7f0904162600 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f0909c83640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0904162de0 0x7f09040ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090ac85640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f09040cad80 0x7f0904072850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36368/0 (socket says 172.21.15.53:36368) 2024-05-10T10:42:50.361 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090ac85640 1 -- 172.21.15.53:0/93081282 learned_addr learned my addr 172.21.15.53:0/93081282 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:50.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090a484640 1 --2- 172.21.15.53:0/93081282 >> v2:172.21.15.135:3300/0 conn(0x7f0904162600 0x7f090406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f0909c83640 1 -- 172.21.15.53:0/93081282 >> v2:172.21.15.135:3300/0 conn(0x7f0904162600 msgr2=0x7f090406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:50.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f0909c83640 1 --2- 172.21.15.53:0/93081282 >> v2:172.21.15.135:3300/0 conn(0x7f0904162600 0x7f090406ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f0909c83640 1 -- 172.21.15.53:0/93081282 >> v1:172.21.15.53:6789/0 conn(0x7f09040cad80 legacy=0x7f0904072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f0909c83640 1 -- 172.21.15.53:0/93081282 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0904163b90 con 0x7f0904162de0 2024-05-10T10:42:50.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.360+0000 7f090a484640 1 --2- 172.21.15.53:0/93081282 >> v2:172.21.15.135:3300/0 conn(0x7f0904162600 0x7f090406ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:50.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.361+0000 7f0909c83640 1 --2- 172.21.15.53:0/93081282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f09040ca840 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f09040cf090 tx=0x7f08f802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2d3741865be26f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.361+0000 7f0909482640 1 -- 172.21.15.53:0/93081282 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8037070 con 0x7f0904162de0 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.361+0000 7f0909482640 1 -- 172.21.15.53:0/93081282 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f8032440 con 0x7f0904162de0 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.361+0000 7f0909482640 1 -- 172.21.15.53:0/93081282 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f802ae60 con 0x7f0904162de0 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.361+0000 7f090c6e8640 1 -- 172.21.15.53:0/93081282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 msgr2=0x7f09040ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.361+0000 7f090c6e8640 1 --2- 172.21.15.53:0/93081282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f09040ca840 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f09040cf090 tx=0x7f08f802a110 comp rx=0 tx=0).stop 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.362+0000 7f090c6e8640 1 -- 172.21.15.53:0/93081282 shutdown_connections 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.362+0000 7f090c6e8640 1 --2- 172.21.15.53:0/93081282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f09040ca840 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.362+0000 7f090c6e8640 1 --2- 172.21.15.53:0/93081282 >> v2:172.21.15.135:3300/0 conn(0x7f0904162600 0x7f090406ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.362+0000 7f090c6e8640 1 -- 172.21.15.53:0/93081282 >> 172.21.15.53:0/93081282 conn(0x7f09040d1510 msgr2=0x7f09040d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.362+0000 7f090c6e8640 1 -- 172.21.15.53:0/93081282 shutdown_connections 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.362+0000 7f090c6e8640 1 -- 172.21.15.53:0/93081282 wait complete. 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.363+0000 7f090c6e8640 1 Processor -- start 2024-05-10T10:42:50.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.363+0000 7f090c6e8640 1 -- start start 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09040cad80 0x7f090414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0904162600 0x7f090415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f0904148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09041629e0 con 0x7f0904162de0 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0904163250 con 0x7f09040cad80 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0904162b60 con 0x7f0904162600 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f0904148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f0904148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48846/0 (socket says 172.21.15.53:48846) 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 -- 172.21.15.53:0/3263925029 learned_addr learned my addr 172.21.15.53:0/3263925029 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090a484640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09040cad80 0x7f090414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f0909c83640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0904162600 0x7f090415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 -- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0904162600 msgr2=0x7f090415d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0904162600 0x7f090415d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 -- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09040cad80 msgr2=0x7f090414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09040cad80 0x7f090414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 -- 172.21.15.53:0/3263925029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0904159f50 con 0x7f0904162de0 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090a484640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09040cad80 0x7f090414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090ac85640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f0904148eb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c370 tx=0x7f08fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f0902ffd640 1 -- 172.21.15.53:0/3263925029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc017070 con 0x7f0904162de0 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f0902ffd640 1 -- 172.21.15.53:0/3263925029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc005ce0 con 0x7f0904162de0 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f0902ffd640 1 -- 172.21.15.53:0/3263925029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0072d0 con 0x7f0904162de0 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09040b42b0 con 0x7f0904162de0 2024-05-10T10:42:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.364+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09040b4b50 con 0x7f0904162de0 2024-05-10T10:42:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.366+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f090406a220 con 0x7f0904162de0 2024-05-10T10:42:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.366+0000 7f0902ffd640 1 -- 172.21.15.53:0/3263925029 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f08fc01a070 con 0x7f0904162de0 2024-05-10T10:42:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.367+0000 7f0902ffd640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08f00762d0 0x7f08f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.367+0000 7f0902ffd640 1 -- 172.21.15.53:0/3263925029 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7f08fc096a50 con 0x7f0904162de0 2024-05-10T10:42:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.367+0000 7f090a484640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08f00762d0 0x7f08f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.367+0000 7f090a484640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08f00762d0 0x7f08f0078790 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e550 tx=0x7f08f4007ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:50.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.371+0000 7f0902ffd640 1 -- 172.21.15.53:0/3263925029 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f09040cb190 con 0x7f0904162de0 2024-05-10T10:42:50.565 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.565+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f090406d450 con 0x7f0904162de0 2024-05-10T10:42:50.566 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.566+0000 7f0902ffd640 1 -- 172.21.15.53:0/3263925029 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16965 (secure 0 0 0) 0x7f08fc05fb30 con 0x7f0904162de0 2024-05-10T10:42:50.567 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:50.567 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":117,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:42:49.599029+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:42:47.266983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-10T10:42:48.551069+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08f00762d0 msgr2=0x7f08f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08f00762d0 0x7f08f0078790 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e550 tx=0x7f08f4007ab0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 msgr2=0x7f0904148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f0904148eb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c370 tx=0x7f08fc00c830 comp rx=0 tx=0).stop 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 shutdown_connections 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08f00762d0 0x7f08f0078790 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0904162de0 0x7f0904148eb0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0904162600 0x7f090415d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 --2- 172.21.15.53:0/3263925029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09040cad80 0x7f090414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 >> 172.21.15.53:0/3263925029 conn(0x7f09040d1510 msgr2=0x7f09040b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 shutdown_connections 2024-05-10T10:42:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.568+0000 7f090c6e8640 1 -- 172.21.15.53:0/3263925029 wait complete. 2024-05-10T10:42:50.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:42:50.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.683+0000 7f4cc8db6640 1 Processor -- start 2024-05-10T10:42:50.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.683+0000 7f4cc8db6640 1 -- start start 2024-05-10T10:42:50.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.683+0000 7f4cc8db6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4cc40d0230 0x7f4cc40cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.683+0000 7f4cc8db6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc415fc90 con 0x7f4cc40cfbb0 2024-05-10T10:42:50.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.683+0000 7f4cc8db6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc40b8c40 con 0x7f4cc40b8860 2024-05-10T10:42:50.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.683+0000 7f4cc8db6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4cc406ec30 con 0x7f4cc40d0230 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.684+0000 7f4cbbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4cc40d0230 0x7f4cc40cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.684+0000 7f4cc2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4cc40cfbb0 0x7f4cc40ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36384/0 (socket says 172.21.15.53:36384) 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.684+0000 7f4cc2d9d640 1 -- 172.21.15.53:0/1005639298 learned_addr learned my addr 172.21.15.53:0/1005639298 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.684+0000 7f4cbbfff640 1 -- 172.21.15.53:0/1005639298 >> v1:172.21.15.135:6789/0 conn(0x7f4cc40b8860 legacy=0x7f4cc406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.684+0000 7f4cbbfff640 1 -- 172.21.15.53:0/1005639298 >> v1:172.21.15.53:6789/0 conn(0x7f4cc40cfbb0 legacy=0x7f4cc40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.684+0000 7f4cbbfff640 1 -- 172.21.15.53:0/1005639298 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc40cd570 con 0x7f4cc40d0230 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.684+0000 7f4cc1d9b640 1 -- 172.21.15.53:0/1005639298 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 895994471 0 0) 0x7f4cc415fc90 con 0x7f4cc40cfbb0 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cbbfff640 1 --2- 172.21.15.53:0/1005639298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc40cf670 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0097a0 tx=0x7f4cac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4fdc58b6db64d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc1d9b640 1 -- 172.21.15.53:0/1005639298 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac037070 con 0x7f4cc40d0230 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc1d9b640 1 -- 172.21.15.53:0/1005639298 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cac02ae50 con 0x7f4cc40d0230 2024-05-10T10:42:50.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc1d9b640 1 -- 172.21.15.53:0/1005639298 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac032660 con 0x7f4cc40d0230 2024-05-10T10:42:50.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc8db6640 1 -- 172.21.15.53:0/1005639298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 msgr2=0x7f4cc40cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/1005639298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc40cf670 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0097a0 tx=0x7f4cac02a140 comp rx=0 tx=0).stop 2024-05-10T10:42:50.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc8db6640 1 -- 172.21.15.53:0/1005639298 shutdown_connections 2024-05-10T10:42:50.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/1005639298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc40cf670 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc8db6640 1 -- 172.21.15.53:0/1005639298 >> 172.21.15.53:0/1005639298 conn(0x7f4cc40d2df0 msgr2=0x7f4cc40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc8db6640 1 -- 172.21.15.53:0/1005639298 shutdown_connections 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.685+0000 7f4cc8db6640 1 -- 172.21.15.53:0/1005639298 wait complete. 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.687+0000 7f4cc8db6640 1 Processor -- start 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.687+0000 7f4cc8db6640 1 -- start start 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.687+0000 7f4cc8db6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4cc40b8860 0x7f4cc41f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.687+0000 7f4cc8db6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc40cfbb0 0x7f4cc4159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc8db6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc8db6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc40cdb30 con 0x7f4cc40d0230 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc8db6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc406ee50 con 0x7f4cc40cfbb0 2024-05-10T10:42:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc8db6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc40cde10 con 0x7f4cc40b8860 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48862/0 (socket says 172.21.15.53:48862) 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 -- 172.21.15.53:0/760572320 learned_addr learned my addr 172.21.15.53:0/760572320 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cbbfff640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc40cfbb0 0x7f4cc4159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc259c640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4cc40b8860 0x7f4cc41f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 -- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4cc40b8860 msgr2=0x7f4cc41f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4cc40b8860 0x7f4cc41f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 -- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc40cfbb0 msgr2=0x7f4cc4159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc40cfbb0 0x7f4cc4159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 -- 172.21.15.53:0/760572320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc414c640 con 0x7f4cc40d0230 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc259c640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4cc40b8860 0x7f4cc41f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc2d9d640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc415a270 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c7a0 tx=0x7f4cbc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:50.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cbaffd640 1 -- 172.21.15.53:0/760572320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc018070 con 0x7f4cc40d0230 2024-05-10T10:42:50.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cbaffd640 1 -- 172.21.15.53:0/760572320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc00a640 con 0x7f4cc40d0230 2024-05-10T10:42:50.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc414c920 con 0x7f4cc40d0230 2024-05-10T10:42:50.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cbaffd640 1 -- 172.21.15.53:0/760572320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00aab0 con 0x7f4cc40d0230 2024-05-10T10:42:50.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.688+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc414cee0 con 0x7f4cc40d0230 2024-05-10T10:42:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.689+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc406a220 con 0x7f4cc40d0230 2024-05-10T10:42:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.690+0000 7f4cbaffd640 1 -- 172.21.15.53:0/760572320 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4cbc01b070 con 0x7f4cc40d0230 2024-05-10T10:42:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.690+0000 7f4cbaffd640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ca40762d0 0x7f4ca4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.691+0000 7f4cbaffd640 1 -- 172.21.15.53:0/760572320 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7f4cbc096cb0 con 0x7f4cc40d0230 2024-05-10T10:42:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.691+0000 7f4cc259c640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ca40762d0 0x7f4ca4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.691+0000 7f4cc259c640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ca40762d0 0x7f4ca4078790 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4003eb0 tx=0x7f4cb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:50.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.695+0000 7f4cbaffd640 1 -- 172.21.15.53:0/760572320 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4cbc05fe30 con 0x7f4cc40d0230 2024-05-10T10:42:50.886 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.886+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cc406f3d0 con 0x7f4cc40d0230 2024-05-10T10:42:50.887 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.887+0000 7f4cbaffd640 1 -- 172.21.15.53:0/760572320 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16965 (secure 0 0 0) 0x7f4cbc064130 con 0x7f4cc40d0230 2024-05-10T10:42:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:50.888 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":117,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:42:49.599029+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:42:47.266983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-10T10:42:48.551069+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:42:50.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ca40762d0 msgr2=0x7f4ca4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ca40762d0 0x7f4ca4078790 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4003eb0 tx=0x7f4cb4009040 comp rx=0 tx=0).stop 2024-05-10T10:42:50.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 msgr2=0x7f4cc415a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:50.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc415a270 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c7a0 tx=0x7f4cbc00cc60 comp rx=0 tx=0).stop 2024-05-10T10:42:50.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 shutdown_connections 2024-05-10T10:42:50.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4ca40762d0 0x7f4ca4078790 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.890 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4cc40d0230 0x7f4cc415a270 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc40cfbb0 0x7f4cc4159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 --2- 172.21.15.53:0/760572320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4cc40b8860 0x7f4cc41f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:50.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 >> 172.21.15.53:0/760572320 conn(0x7f4cc40d2df0 msgr2=0x7f4cc40d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:50.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 shutdown_connections 2024-05-10T10:42:50.891 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:50.889+0000 7f4cc8db6640 1 -- 172.21.15.53:0/760572320 wait complete. 2024-05-10T10:42:50.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T10:42:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.007+0000 7f72391da640 1 Processor -- start 2024-05-10T10:42:51.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.007+0000 7f72391da640 1 -- start start 2024-05-10T10:42:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f72391da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f723406c2c0 0x7f72340d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f72391da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 0x7f7234167130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f72391da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7234164b10 con 0x7f72340cb770 2024-05-10T10:42:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f72391da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f72340ccec0 con 0x7f723406c2c0 2024-05-10T10:42:51.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f72391da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f72341645e0 con 0x7f72340c4190 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f722a59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f72340cb770 0x7f723406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36386/0 (socket says 172.21.15.53:36386) 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f722a59c640 1 -- 172.21.15.53:0/273539864 learned_addr learned my addr 172.21.15.53:0/273539864 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f7232d9d640 1 --2- 172.21.15.53:0/273539864 >> v2:172.21.15.135:3300/0 conn(0x7f723406c2c0 0x7f72340d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723359e640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 0x7f7234167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461260501 0 0) 0x7f7234164b10 con 0x7f72340cb770 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7220003520 con 0x7f72340cb770 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393845521 0 0) 0x7f7220003520 con 0x7f72340cb770 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 msgr2=0x7f7234167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 0x7f7234167130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 >> v2:172.21.15.135:3300/0 conn(0x7f723406c2c0 msgr2=0x7f72340d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 --2- 172.21.15.53:0/273539864 >> v2:172.21.15.135:3300/0 conn(0x7f723406c2c0 0x7f72340d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340cd0f0 con 0x7f72340cb770 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f721c001670 con 0x7f72340cb770 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723359e640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 0x7f7234167130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:51.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 >> v1:172.21.15.53:6789/0 conn(0x7f72340cb770 legacy=0x7f723406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 0x7f72200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7220003ef0 0x7f7220004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220004d40 0x7f72200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f72341645e0 con 0x7f7220003ef0 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f722000b9c0 con 0x7f7220004d40 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f722000bb40 con 0x7f72340c4190 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.008+0000 7f7232d9d640 1 --2- 172.21.15.53:0/273539864 >> v2:172.21.15.135:3300/0 conn(0x7f723406c2c0 0x7f72340d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f722a59c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7220003ef0 0x7f7220004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f723359e640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220004d40 0x7f72200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f7232d9d640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 0x7f72200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f722a59c640 1 -- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 msgr2=0x7f72200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f722a59c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4190 0x7f72200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f722a59c640 1 -- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220004d40 msgr2=0x7f72200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:51.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f722a59c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220004d40 0x7f72200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.009+0000 7f722a59c640 1 -- 172.21.15.53:0/273539864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7220003520 con 0x7f7220003ef0 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f722a59c640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7220003ef0 0x7f7220004800 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f721c00c290 tx=0x7f7220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c9f8ad46a626d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c007d50 con 0x7f7220003ef0 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f721c005af0 con 0x7f7220003ef0 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f723259c640 1 -- 172.21.15.53:0/273539864 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c007070 con 0x7f7220003ef0 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f72391da640 1 -- 172.21.15.53:0/273539864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7220003ef0 msgr2=0x7f7220004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f72391da640 1 --2- 172.21.15.53:0/273539864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7220003ef0 0x7f7220004800 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f721c00c290 tx=0x7f7220005830 comp rx=0 tx=0).stop 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f723359e640 1 -- 172.21.15.53:0/273539864 reap_dead start 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f72391da640 1 -- 172.21.15.53:0/273539864 shutdown_connections 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f72391da640 1 -- 172.21.15.53:0/273539864 >> 172.21.15.53:0/273539864 conn(0x7f72340d0c90 msgr2=0x7f72340d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f72391da640 1 -- 172.21.15.53:0/273539864 shutdown_connections 2024-05-10T10:42:51.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.010+0000 7f72391da640 1 -- 172.21.15.53:0/273539864 wait complete. 2024-05-10T10:42:51.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.012+0000 7f72391da640 1 Processor -- start 2024-05-10T10:42:51.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.012+0000 7f72391da640 1 -- start start 2024-05-10T10:42:51.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220003ef0 0x7f72340b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4620 0x7f72341e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72341ea0b0 0x7f72341ea560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7234164830 con 0x7f72341ea0b0 2024-05-10T10:42:51.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72340cb2e0 con 0x7f7220003ef0 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f72340cd390 con 0x7f72340c4620 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72341ea0b0 0x7f72341ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72341ea0b0 0x7f72341ea560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48884/0 (socket says 172.21.15.53:48884) 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 -- 172.21.15.53:0/677102141 learned_addr learned my addr 172.21.15.53:0/677102141 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f722a59c640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4620 0x7f72341e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f7232d9d640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220003ef0 0x7f72340b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 -- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4620 msgr2=0x7f72341e9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4620 0x7f72341e9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 -- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220003ef0 msgr2=0x7f72340b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220003ef0 0x7f72340b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 -- 172.21.15.53:0/677102141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72341ec8e0 con 0x7f72341ea0b0 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f723359e640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72341ea0b0 0x7f72341ea560 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f72340d0c10 tx=0x7f722c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f722bfff640 1 -- 172.21.15.53:0/677102141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c00fd10 con 0x7f72341ea0b0 2024-05-10T10:42:51.018 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f722bfff640 1 -- 172.21.15.53:0/677102141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c003990 con 0x7f72341ea0b0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f722bfff640 1 -- 172.21.15.53:0/677102141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c003d80 con 0x7f72341ea0b0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341ec600 con 0x7f72341ea0b0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.013+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72340b4690 con 0x7f72341ea0b0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.014+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723406a220 con 0x7f72341ea0b0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.015+0000 7f722bfff640 1 -- 172.21.15.53:0/677102141 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f722c004020 con 0x7f72341ea0b0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.016+0000 7f722bfff640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7214076300 0x7f72140787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.016+0000 7f722bfff640 1 -- 172.21.15.53:0/677102141 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7f722c098000 con 0x7f72341ea0b0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.016+0000 7f7232d9d640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7214076300 0x7f72140787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:51.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.016+0000 7f7232d9d640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7214076300 0x7f72140787c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7224009ae0 tx=0x7f7224008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:51.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.020+0000 7f722bfff640 1 -- 172.21.15.53:0/677102141 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f722c019030 con 0x7f72341ea0b0 2024-05-10T10:42:51.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:51.250+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f72340b6c10 con 0x7f72341ea0b0 2024-05-10T10:42:52.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.238+0000 7f722bfff640 1 -- 172.21.15.53:0/677102141 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) ==== 118+0+0 (secure 0 0 0) 0x7f722c010040 con 0x7f72341ea0b0 2024-05-10T10:42:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7214076300 msgr2=0x7f72140787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7214076300 0x7f72140787c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7224009ae0 tx=0x7f7224008040 comp rx=0 tx=0).stop 2024-05-10T10:42:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72341ea0b0 msgr2=0x7f72341ea560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72341ea0b0 0x7f72341ea560 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f72340d0c10 tx=0x7f722c009f90 comp rx=0 tx=0).stop 2024-05-10T10:42:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 shutdown_connections 2024-05-10T10:42:52.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7214076300 0x7f72140787c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72341ea0b0 0x7f72341ea560 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72340c4620 0x7f72341e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 --2- 172.21.15.53:0/677102141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7220003ef0 0x7f72340b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 >> 172.21.15.53:0/677102141 conn(0x7f72340d0c90 msgr2=0x7f72340b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 shutdown_connections 2024-05-10T10:42:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.241+0000 7f72391da640 1 -- 172.21.15.53:0/677102141 wait complete. 2024-05-10T10:42:52.255 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T10:42:52.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T10:42:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.368+0000 7fd26b2da640 1 Processor -- start 2024-05-10T10:42:52.369 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.368+0000 7fd26b2da640 1 -- start start 2024-05-10T10:42:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd26b2da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd26b2da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2640d36d0 0x7fd2640d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd26b2da640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2640d2800 con 0x7fd26415f410 2024-05-10T10:42:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd26b2da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd2640d2ae0 con 0x7fd2640d36d0 2024-05-10T10:42:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd26b2da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd2640d01d0 con 0x7fd2640b4f90 2024-05-10T10:42:52.370 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd269877640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2640d36d0 0x7fd2640d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd269877640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd2640d36d0 0x7fd2640d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48898/0 (socket says 172.21.15.53:48898) 2024-05-10T10:42:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd269877640 1 -- 172.21.15.53:0/2155123516 learned_addr learned my addr 172.21.15.53:0/2155123516 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd268875640 1 --2- 172.21.15.53:0/2155123516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd269877640 1 -- 172.21.15.53:0/2155123516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 msgr2=0x7fd2640b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd269877640 1 --2- 172.21.15.53:0/2155123516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd269877640 1 -- 172.21.15.53:0/2155123516 >> v1:172.21.15.135:6789/0 conn(0x7fd26415f410 legacy=0x7fd2640d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:52.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.369+0000 7fd269877640 1 -- 172.21.15.53:0/2155123516 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2640d04b0 con 0x7fd2640d36d0 2024-05-10T10:42:52.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.370+0000 7fd269877640 1 --2- 172.21.15.53:0/2155123516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2640d36d0 0x7fd2640d3b60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009870 tx=0x7fd25c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89527177e1e78d17 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.370+0000 7fd25bfff640 1 -- 172.21.15.53:0/2155123516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c037070 con 0x7fd2640d36d0 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.370+0000 7fd25bfff640 1 -- 172.21.15.53:0/2155123516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c032440 con 0x7fd2640d36d0 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.370+0000 7fd25bfff640 1 -- 172.21.15.53:0/2155123516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c032810 con 0x7fd2640d36d0 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 -- 172.21.15.53:0/2155123516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2640d36d0 msgr2=0x7fd2640d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2155123516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2640d36d0 0x7fd2640d3b60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009870 tx=0x7fd25c02a1c0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 -- 172.21.15.53:0/2155123516 shutdown_connections 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2155123516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2640d36d0 0x7fd2640d3b60 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2155123516 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 -- 172.21.15.53:0/2155123516 >> 172.21.15.53:0/2155123516 conn(0x7fd26408ed30 msgr2=0x7fd26409e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 -- 172.21.15.53:0/2155123516 shutdown_connections 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.371+0000 7fd26b2da640 1 -- 172.21.15.53:0/2155123516 wait complete. 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.372+0000 7fd26b2da640 1 Processor -- start 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.372+0000 7fd26b2da640 1 -- start start 2024-05-10T10:42:52.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd26b2da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd26b2da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2640d36d0 0x7fd26414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd26b2da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd26415f410 0x7fd26414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd26b2da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd2640cffb0 con 0x7fd26415f410 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd26b2da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2640cfe30 con 0x7fd2640d36d0 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd26b2da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd2640d10e0 con 0x7fd2640b4f90 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd269877640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd26415f410 0x7fd26414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd269877640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd26415f410 0x7fd26414b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48900/0 (socket says 172.21.15.53:48900) 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd269877640 1 -- 172.21.15.53:0/2516743917 learned_addr learned my addr 172.21.15.53:0/2516743917 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd268875640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2640d36d0 0x7fd26414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.373+0000 7fd269076640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:52.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd269877640 1 -- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 msgr2=0x7fd2640cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd269877640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd269877640 1 -- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2640d36d0 msgr2=0x7fd26414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd269877640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2640d36d0 0x7fd26414a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd269877640 1 -- 172.21.15.53:0/2516743917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2640c96d0 con 0x7fd26415f410 2024-05-10T10:42:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd269877640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd26415f410 0x7fd26414b280 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd25c02a200 tx=0x7fd25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:52.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd259ffb640 1 -- 172.21.15.53:0/2516743917 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c037040 con 0x7fd26415f410 2024-05-10T10:42:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd259ffb640 1 -- 172.21.15.53:0/2516743917 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c042070 con 0x7fd26415f410 2024-05-10T10:42:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd259ffb640 1 -- 172.21.15.53:0/2516743917 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c030920 con 0x7fd26415f410 2024-05-10T10:42:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2640c99b0 con 0x7fd26415f410 2024-05-10T10:42:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.374+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2640c9f70 con 0x7fd26415f410 2024-05-10T10:42:52.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.375+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26406a220 con 0x7fd26415f410 2024-05-10T10:42:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.376+0000 7fd259ffb640 1 -- 172.21.15.53:0/2516743917 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd25c02ad10 con 0x7fd26415f410 2024-05-10T10:42:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.376+0000 7fd259ffb640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2480762d0 0x7fd248078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.376+0000 7fd259ffb640 1 -- 172.21.15.53:0/2516743917 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd25c0b7cd0 con 0x7fd26415f410 2024-05-10T10:42:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.376+0000 7fd269076640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2480762d0 0x7fd248078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:52.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.377+0000 7fd269076640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2480762d0 0x7fd248078790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd260001e70 tx=0x7fd260009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:52.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.380+0000 7fd259ffb640 1 -- 172.21.15.53:0/2516743917 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd25c080dc0 con 0x7fd26415f410 2024-05-10T10:42:52.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:52.608+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd2640d6090 con 0x7fd26415f410 2024-05-10T10:42:53.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.242+0000 7fd259ffb640 1 -- 172.21.15.53:0/2516743917 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) ==== 122+0+0 (secure 0 0 0) 0x7fd25c0850c0 con 0x7fd26415f410 2024-05-10T10:42:53.247 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2480762d0 msgr2=0x7fd248078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2480762d0 0x7fd248078790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd260001e70 tx=0x7fd260009040 comp rx=0 tx=0).stop 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd26415f410 msgr2=0x7fd26414b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd26415f410 0x7fd26414b280 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd25c02a200 tx=0x7fd25c0046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 shutdown_connections 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd2480762d0 0x7fd248078790 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd26415f410 0x7fd26414b280 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2640d36d0 0x7fd26414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 --2- 172.21.15.53:0/2516743917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2640b4f90 0x7fd2640cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 >> 172.21.15.53:0/2516743917 conn(0x7fd26408ed30 msgr2=0x7fd2640757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 shutdown_connections 2024-05-10T10:42:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.246+0000 7fd26b2da640 1 -- 172.21.15.53:0/2516743917 wait complete. 2024-05-10T10:42:53.262 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T10:42:53.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.373+0000 7f34c2590640 1 Processor -- start 2024-05-10T10:42:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.373+0000 7f34c2590640 1 -- start start 2024-05-10T10:42:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34c2590640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f34bc15f790 0x7f34bc0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34c2590640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 0x7f34bc0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34c2590640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34bc0d30b0 con 0x7f34bc15f3b0 2024-05-10T10:42:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34c2590640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f34bc0d3390 con 0x7f34bc15f790 2024-05-10T10:42:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34c2590640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc0c8c80 con 0x7f34bc0d5d30 2024-05-10T10:42:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bbfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f34bc15f3b0 0x7f34bc0d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49192/0 (socket says 172.21.15.53:49192) 2024-05-10T10:42:53.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bbfff640 1 -- 172.21.15.53:0/3238773845 learned_addr learned my addr 172.21.15.53:0/3238773845 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:53.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34b3fff640 1 --2- 172.21.15.53:0/3238773845 >> v2:172.21.15.135:3300/0 conn(0x7f34bc15f790 0x7f34bc0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.378 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824297670 0 0) 0x7f34bc0d30b0 con 0x7f34bc15f3b0 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34c0b2d640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 0x7f34bc0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34a8003520 con 0x7f34bc15f3b0 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1727929614 0 0) 0x7f34a8003520 con 0x7f34bc15f3b0 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 msgr2=0x7f34bc0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 0x7f34bc0d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 >> v2:172.21.15.135:3300/0 conn(0x7f34bc15f790 msgr2=0x7f34bc0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 --2- 172.21.15.53:0/3238773845 >> v2:172.21.15.135:3300/0 conn(0x7f34bc15f790 0x7f34bc0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc0c8f60 con 0x7f34bc15f3b0 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f34ac001670 con 0x7f34bc15f3b0 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 >> v1:172.21.15.53:6789/0 conn(0x7f34bc15f3b0 legacy=0x7f34bc0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 0x7f34a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34a8003f30 0x7f34a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.379 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a8004d40 0x7f34a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc0d3390 con 0x7f34a8004d40 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc0c8c80 con 0x7f34a8003f30 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.374+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f34a800b9c0 con 0x7f34bc0d5d30 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34c0b2d640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a8004d40 0x7f34a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34b3fff640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34a8003f30 0x7f34a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34bbfff640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 0x7f34a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34c0b2d640 1 -- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 msgr2=0x7f34a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34c0b2d640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0d5d30 0x7f34a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34c0b2d640 1 -- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34a8003f30 msgr2=0x7f34a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34c0b2d640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34a8003f30 0x7f34a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.375+0000 7f34c0b2d640 1 -- 172.21.15.53:0/3238773845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a8003520 con 0x7f34a8004d40 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c0b2d640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a8004d40 0x7f34a80051c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009fc0 tx=0x7f34a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6b96305af69613 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:53.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4009c10 con 0x7f34a8004d40 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b4036070 con 0x7f34a8004d40 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34bb7fe640 1 -- 172.21.15.53:0/3238773845 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4031670 con 0x7f34a8004d40 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c2590640 1 -- 172.21.15.53:0/3238773845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a8004d40 msgr2=0x7f34a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c2590640 1 --2- 172.21.15.53:0/3238773845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a8004d40 0x7f34a80051c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009fc0 tx=0x7f34a8005830 comp rx=0 tx=0).stop 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c0b2d640 1 -- 172.21.15.53:0/3238773845 reap_dead start 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c2590640 1 -- 172.21.15.53:0/3238773845 shutdown_connections 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c2590640 1 -- 172.21.15.53:0/3238773845 >> 172.21.15.53:0/3238773845 conn(0x7f34bc0d5130 msgr2=0x7f34bc0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c2590640 1 -- 172.21.15.53:0/3238773845 shutdown_connections 2024-05-10T10:42:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.376+0000 7f34c2590640 1 -- 172.21.15.53:0/3238773845 wait complete. 2024-05-10T10:42:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 Processor -- start 2024-05-10T10:42:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 -- start start 2024-05-10T10:42:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34bc0ba7c0 0x7f34bc154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0ccbf0 0x7f34bc157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc157fc0 0x7f34bc1f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc0d3f80 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc0d3e00 con 0x7f34bc157fc0 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.378+0000 7f34c2590640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc0ca770 con 0x7f34bc0ccbf0 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34bc0ba7c0 0x7f34bc154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34bc0ba7c0 0x7f34bc154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45432/0 (socket says 172.21.15.53:45432) 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 -- 172.21.15.53:0/1338374151 learned_addr learned my addr 172.21.15.53:0/1338374151 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34c0b2d640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc157fc0 0x7f34bc1f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 -- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0ccbf0 msgr2=0x7f34bc157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34b3fff640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0ccbf0 0x7f34bc157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0ccbf0 0x7f34bc157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 -- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc157fc0 msgr2=0x7f34bc1f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc157fc0 0x7f34bc1f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 -- 172.21.15.53:0/1338374151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc0b86f0 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34c0b2d640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc157fc0 0x7f34bc1f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34b3fff640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0ccbf0 0x7f34bc157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34bbfff640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34bc0ba7c0 0x7f34bc154800 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f34ac004840 tx=0x7f34ac007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34b97fa640 1 -- 172.21.15.53:0/1338374151 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00fa20 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34b97fa640 1 -- 172.21.15.53:0/1338374151 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ac018070 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34b97fa640 1 -- 172.21.15.53:0/1338374151 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00fe20 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc0b89d0 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.379+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc0cd050 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.380+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc160bd0 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.381+0000 7f34b97fa640 1 -- 172.21.15.53:0/1338374151 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f34ac01d460 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.381+0000 7f34b97fa640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f349c0762d0 0x7f349c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.381+0000 7f34b97fa640 1 -- 172.21.15.53:0/1338374151 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f34ac097950 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.382+0000 7f34b3fff640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f349c0762d0 0x7f349c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.382+0000 7f34b3fff640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f349c0762d0 0x7f349c078790 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0b8170 tx=0x7f34a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:53.387 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.386+0000 7f34b97fa640 1 -- 172.21.15.53:0/1338374151 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f34bc160bd0 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.613+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34bc06ddc0 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.614+0000 7f34b97fa640 1 -- 172.21.15.53:0/1338374151 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5174 (secure 0 0 0) 0x7f34bc06ddc0 con 0x7f34bc0ba7c0 2024-05-10T10:42:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:53.616 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8025,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/38461894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":38461894},{"type":"v1","addr":"172.21.15.53:6834","nonce":38461894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:42:52.609796+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/776479349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":776479349},{"type":"v1","addr":"172.21.15.135:6837","nonce":776479349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8034,"qdb_cluster":[8034]},"id":8}]} 2024-05-10T10:42:53.616 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 98 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f349c0762d0 msgr2=0x7f349c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f349c0762d0 0x7f349c078790 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0b8170 tx=0x7f34a4008040 comp rx=0 tx=0).stop 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34bc0ba7c0 msgr2=0x7f34bc154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34bc0ba7c0 0x7f34bc154800 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f34ac004840 tx=0x7f34ac007720 comp rx=0 tx=0).stop 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 shutdown_connections 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f349c0762d0 0x7f349c078790 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc157fc0 0x7f34bc1f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34bc0ccbf0 0x7f34bc157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 --2- 172.21.15.53:0/1338374151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34bc0ba7c0 0x7f34bc154800 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 >> 172.21.15.53:0/1338374151 conn(0x7f34bc0d5130 msgr2=0x7f34bc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:53.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 shutdown_connections 2024-05-10T10:42:53.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.616+0000 7f34c2590640 1 -- 172.21.15.53:0/1338374151 wait complete. 2024-05-10T10:42:53.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:42:53.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.731+0000 7fa82cad1640 1 Processor -- start 2024-05-10T10:42:53.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.731+0000 7fa82cad1640 1 -- start start 2024-05-10T10:42:53.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.732+0000 7fa82cad1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa82815ce60 0x7fa8280cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.732+0000 7fa82cad1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8280c6a60 con 0x7fa8281611e0 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.732+0000 7fa82cad1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8280cd090 con 0x7fa8280c65f0 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.732+0000 7fa82cad1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa8280cd370 con 0x7fa82815ce60 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa82659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa82815ce60 0x7fa8280cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa826d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa8281611e0 0x7fa82815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49200/0 (socket says 172.21.15.53:49200) 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa826d9d640 1 -- 172.21.15.53:0/87315265 learned_addr learned my addr 172.21.15.53:0/87315265 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa82659c640 1 --2- 172.21.15.53:0/87315265 >> v2:172.21.15.53:3300/0 conn(0x7fa82815ce60 0x7fa8280cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45436/0 (socket says 172.21.15.53:45436) 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa82659c640 1 -- 172.21.15.53:0/87315265 >> v1:172.21.15.135:6789/0 conn(0x7fa8280c65f0 legacy=0x7fa828160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa82659c640 1 -- 172.21.15.53:0/87315265 >> v1:172.21.15.53:6789/0 conn(0x7fa8281611e0 legacy=0x7fa82815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa82659c640 1 -- 172.21.15.53:0/87315265 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8280cd650 con 0x7fa82815ce60 2024-05-10T10:42:53.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.733+0000 7fa82559a640 1 -- 172.21.15.53:0/87315265 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1156778884 0 0) 0x7fa8280c6a60 con 0x7fa8281611e0 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.734+0000 7fa82659c640 1 --2- 172.21.15.53:0/87315265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82815ce60 0x7fa8280cb300 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa8180097a0 tx=0x7fa81802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6aaf4284871517 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.734+0000 7fa82559a640 1 -- 172.21.15.53:0/87315265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818037070 con 0x7fa82815ce60 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.734+0000 7fa82559a640 1 -- 172.21.15.53:0/87315265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818032440 con 0x7fa82815ce60 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.734+0000 7fa82559a640 1 -- 172.21.15.53:0/87315265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818032810 con 0x7fa82815ce60 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.734+0000 7fa82cad1640 1 -- 172.21.15.53:0/87315265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82815ce60 msgr2=0x7fa8280cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.734+0000 7fa82cad1640 1 --2- 172.21.15.53:0/87315265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82815ce60 0x7fa8280cb300 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa8180097a0 tx=0x7fa81802a1a0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.735+0000 7fa82cad1640 1 -- 172.21.15.53:0/87315265 shutdown_connections 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.735+0000 7fa82cad1640 1 --2- 172.21.15.53:0/87315265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa82815ce60 0x7fa8280cb300 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.735+0000 7fa82cad1640 1 -- 172.21.15.53:0/87315265 >> 172.21.15.53:0/87315265 conn(0x7fa8280b2900 msgr2=0x7fa8280b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.735+0000 7fa82cad1640 1 -- 172.21.15.53:0/87315265 shutdown_connections 2024-05-10T10:42:53.737 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.735+0000 7fa82cad1640 1 -- 172.21.15.53:0/87315265 wait complete. 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.736+0000 7fa82cad1640 1 Processor -- start 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.736+0000 7fa82cad1640 1 -- start start 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82cad1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8280c65f0 0x7fa8280af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82cad1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82815ce60 0x7fa82809ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82cad1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8281611e0 0x7fa828152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82cad1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280cde50 con 0x7fa8280c65f0 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82cad1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280ccd60 con 0x7fa82815ce60 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82cad1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa8280cdfd0 con 0x7fa8281611e0 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8280c65f0 0x7fa8280af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8280c65f0 0x7fa8280af6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45440/0 (socket says 172.21.15.53:45440) 2024-05-10T10:42:53.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 -- 172.21.15.53:0/2453332154 learned_addr learned my addr 172.21.15.53:0/2453332154 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa825d9b640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82815ce60 0x7fa82809ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa826d9d640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8281611e0 0x7fa828152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 -- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8281611e0 msgr2=0x7fa828152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8281611e0 0x7fa828152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 -- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82815ce60 msgr2=0x7fa82809ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82815ce60 0x7fa82809ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 -- 172.21.15.53:0/2453332154 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8280afcd0 con 0x7fa8280c65f0 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa825d9b640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82815ce60 0x7fa82809ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa826d9d640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8281611e0 0x7fa828152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.737+0000 7fa82659c640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8280c65f0 0x7fa8280af6f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa8180097a0 tx=0x7fa8180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.738+0000 7fa816ffd640 1 -- 172.21.15.53:0/2453332154 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818037050 con 0x7fa8280c65f0 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.738+0000 7fa816ffd640 1 -- 172.21.15.53:0/2453332154 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818038040 con 0x7fa8280c65f0 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.738+0000 7fa816ffd640 1 -- 172.21.15.53:0/2453332154 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818004490 con 0x7fa8280c65f0 2024-05-10T10:42:53.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.738+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8280affb0 con 0x7fa8280c65f0 2024-05-10T10:42:53.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.738+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8280b0570 con 0x7fa8280c65f0 2024-05-10T10:42:53.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.739+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82806a220 con 0x7fa8280c65f0 2024-05-10T10:42:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.739+0000 7fa816ffd640 1 -- 172.21.15.53:0/2453332154 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa818030900 con 0x7fa8280c65f0 2024-05-10T10:42:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.740+0000 7fa816ffd640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa808076300 0x7fa8080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.740+0000 7fa816ffd640 1 -- 172.21.15.53:0/2453332154 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa8180b8260 con 0x7fa8280c65f0 2024-05-10T10:42:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.740+0000 7fa825d9b640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa808076300 0x7fa8080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:53.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.741+0000 7fa825d9b640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa808076300 0x7fa8080787c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa82815e1a0 tx=0x7fa81c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:53.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.744+0000 7fa816ffd640 1 -- 172.21.15.53:0/2453332154 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa818081ac0 con 0x7fa8280c65f0 2024-05-10T10:42:53.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.936+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8280c7ad0 con 0x7fa8280c65f0 2024-05-10T10:42:53.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.937+0000 7fa816ffd640 1 -- 172.21.15.53:0/2453332154 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16952 (secure 0 0 0) 0x7fa8180813c0 con 0x7fa8280c65f0 2024-05-10T10:42:53.939 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:53.939 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":118,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:42:51.225258+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:42:47.266983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-10T10:42:48.551069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:42:53.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa808076300 msgr2=0x7fa8080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa808076300 0x7fa8080787c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa82815e1a0 tx=0x7fa81c009040 comp rx=0 tx=0).stop 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8280c65f0 msgr2=0x7fa8280af6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8280c65f0 0x7fa8280af6f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa8180097a0 tx=0x7fa8180046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 shutdown_connections 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa808076300 0x7fa8080787c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8281611e0 0x7fa828152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa82815ce60 0x7fa82809ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 --2- 172.21.15.53:0/2453332154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8280c65f0 0x7fa8280af6f0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.939+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 >> 172.21.15.53:0/2453332154 conn(0x7fa8280b2900 msgr2=0x7fa82815d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.940+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 shutdown_connections 2024-05-10T10:42:53.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:53.940+0000 7fa82cad1640 1 -- 172.21.15.53:0/2453332154 wait complete. 2024-05-10T10:42:53.953 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:42:53.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:42:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.055+0000 7fcafacd1640 1 Processor -- start 2024-05-10T10:42:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.056+0000 7fcafacd1640 1 -- start start 2024-05-10T10:42:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.056+0000 7fcafacd1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcaf415d290 0x7fcaf4071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.056+0000 7fcafacd1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf40d3510 0x7fcaf40ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.056+0000 7fcafacd1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf4072330 con 0x7fcaf4071880 2024-05-10T10:42:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.056+0000 7fcafacd1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf4071d70 con 0x7fcaf415d290 2024-05-10T10:42:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.056+0000 7fcafacd1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf4072050 con 0x7fcaf40d3510 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf8a6d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcaf415d290 0x7fcaf4071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf8a6d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcaf415d290 0x7fcaf4071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45456/0 (socket says 172.21.15.53:45456) 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcaf4071880 0x7fcaf40d2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49216/0 (socket says 172.21.15.53:49216) 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf8a6d640 1 -- 172.21.15.53:0/1483534646 learned_addr learned my addr 172.21.15.53:0/1483534646 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf8a6d640 1 -- 172.21.15.53:0/1483534646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf40d3510 msgr2=0x7fcaf40ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf926e640 1 --2- 172.21.15.53:0/1483534646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf40d3510 0x7fcaf40ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf8a6d640 1 --2- 172.21.15.53:0/1483534646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf40d3510 0x7fcaf40ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf8a6d640 1 -- 172.21.15.53:0/1483534646 >> v1:172.21.15.53:6789/0 conn(0x7fcaf4071880 legacy=0x7fcaf40d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf8a6d640 1 -- 172.21.15.53:0/1483534646 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf40cda70 con 0x7fcaf415d290 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf37fe640 1 -- 172.21.15.53:0/1483534646 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2820530516 0 0) 0x7fcaf4072330 con 0x7fcaf4071880 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.057+0000 7fcaf926e640 1 --2- 172.21.15.53:0/1483534646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf40d3510 0x7fcaf40ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcaf8a6d640 1 --2- 172.21.15.53:0/1483534646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf415d290 0x7fcaf4071340 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009870 tx=0x7fcaec02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c02e3a21366952 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcaf37fe640 1 -- 172.21.15.53:0/1483534646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec037070 con 0x7fcaf415d290 2024-05-10T10:42:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcaf37fe640 1 -- 172.21.15.53:0/1483534646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec032440 con 0x7fcaf415d290 2024-05-10T10:42:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcaf37fe640 1 -- 172.21.15.53:0/1483534646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec032810 con 0x7fcaf415d290 2024-05-10T10:42:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 -- 172.21.15.53:0/1483534646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf415d290 msgr2=0x7fcaf4071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 --2- 172.21.15.53:0/1483534646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf415d290 0x7fcaf4071340 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009870 tx=0x7fcaec02a250 comp rx=0 tx=0).stop 2024-05-10T10:42:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 -- 172.21.15.53:0/1483534646 shutdown_connections 2024-05-10T10:42:54.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 --2- 172.21.15.53:0/1483534646 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf40d3510 0x7fcaf40ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 --2- 172.21.15.53:0/1483534646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf415d290 0x7fcaf4071340 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 -- 172.21.15.53:0/1483534646 >> 172.21.15.53:0/1483534646 conn(0x7fcaf409ba20 msgr2=0x7fcaf409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 -- 172.21.15.53:0/1483534646 shutdown_connections 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.058+0000 7fcafacd1640 1 -- 172.21.15.53:0/1483534646 wait complete. 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 Processor -- start 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 -- start start 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf4071880 0x7fcaf41f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf40d3510 0x7fcaf40c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaf415d290 0x7fcaf415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40d1c20 con 0x7fcaf40d3510 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40d1aa0 con 0x7fcaf415d290 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.060+0000 7fcafacd1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf415b3d0 con 0x7fcaf4071880 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf40d3510 0x7fcaf40c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf40d3510 0x7fcaf40c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45466/0 (socket says 172.21.15.53:45466) 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 -- 172.21.15.53:0/3580099877 learned_addr learned my addr 172.21.15.53:0/3580099877 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf926e640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaf415d290 0x7fcaf415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf8a6d640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf4071880 0x7fcaf41f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 -- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf4071880 msgr2=0x7fcaf41f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf4071880 0x7fcaf41f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 -- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaf415d290 msgr2=0x7fcaf415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaf415d290 0x7fcaf415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 -- 172.21.15.53:0/3580099877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf40b8c70 con 0x7fcaf40d3510 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf926e640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaf415d290 0x7fcaf415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf3fff640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf40d3510 0x7fcaf40c8220 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c410 tx=0x7fcadc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf17fa640 1 -- 172.21.15.53:0/3580099877 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0074e0 con 0x7fcaf40d3510 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf17fa640 1 -- 172.21.15.53:0/3580099877 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc0047e0 con 0x7fcaf40d3510 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcaf17fa640 1 -- 172.21.15.53:0/3580099877 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc004bd0 con 0x7fcaf40d3510 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf40b7ec0 con 0x7fcaf40d3510 2024-05-10T10:42:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.061+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf40b8440 con 0x7fcaf40d3510 2024-05-10T10:42:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.062+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf40693f0 con 0x7fcaf40d3510 2024-05-10T10:42:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.063+0000 7fcaf17fa640 1 -- 172.21.15.53:0/3580099877 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcadc011070 con 0x7fcaf40d3510 2024-05-10T10:42:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.063+0000 7fcaf17fa640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcad40762d0 0x7fcad4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.064+0000 7fcaf17fa640 1 -- 172.21.15.53:0/3580099877 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fcadc0974a0 con 0x7fcaf40d3510 2024-05-10T10:42:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.064+0000 7fcaf8a6d640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcad40762d0 0x7fcad4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.064+0000 7fcaf8a6d640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcad40762d0 0x7fcad4078790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcaec034000 tx=0x7fcaec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.068+0000 7fcaf17fa640 1 -- 172.21.15.53:0/3580099877 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fcadc060b50 con 0x7fcaf40d3510 2024-05-10T10:42:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.254+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcaf40b26c0 con 0x7fcad40762d0 2024-05-10T10:42:54.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.261+0000 7fcaf17fa640 1 -- 172.21.15.53:0/3580099877 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240244 (secure 0 0 0) 0x7fcaf40b26c0 con 0x7fcad40762d0 2024-05-10T10:42:54.265 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:54.266 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:42:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcad40762d0 msgr2=0x7fcad4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcad40762d0 0x7fcad4078790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcaec034000 tx=0x7fcaec02e000 comp rx=0 tx=0).stop 2024-05-10T10:42:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf40d3510 msgr2=0x7fcaf40c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf40d3510 0x7fcaf40c8220 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c410 tx=0x7fcadc00c8d0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 shutdown_connections 2024-05-10T10:42:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcad40762d0 0x7fcad4078790 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaf415d290 0x7fcaf415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf40d3510 0x7fcaf40c8220 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 --2- 172.21.15.53:0/3580099877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcaf4071880 0x7fcaf41f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 >> 172.21.15.53:0/3580099877 conn(0x7fcaf409ba20 msgr2=0x7fcaf40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 shutdown_connections 2024-05-10T10:42:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.264+0000 7fcafacd1640 1 -- 172.21.15.53:0/3580099877 wait complete. 2024-05-10T10:42:54.277 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-10T10:42:52.295813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261740,"kb_used_data":12264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712980,"statfs":{"total":773094113280,"available":772826091520,"internally_reserved":0,"allocated":12558336,"data_stored":6445160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.086891"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.215198+0000","last_change":"2024-05-10T10:42:48.215198+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.215198+0000","last_undegraded":"2024-05-10T10:42:48.215198+0000","last_fullsized":"2024-05-10T10:42:48.215198+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.212026+0000","last_change":"2024-05-10T10:42:48.212026+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.212026+0000","last_undegraded":"2024-05-10T10:42:48.212026+0000","last_fullsized":"2024-05-10T10:42:48.212026+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.210412+0000","last_change":"2024-05-10T10:42:48.210412+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.210412+0000","last_undegraded":"2024-05-10T10:42:48.210412+0000","last_fullsized":"2024-05-10T10:42:48.210412+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.212158+0000","last_change":"2024-05-10T10:42:48.212158+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.212158+0000","last_undegraded":"2024-05-10T10:42:48.212158+0000","last_fullsized":"2024-05-10T10:42:48.212158+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.210804+0000","last_change":"2024-05-10T10:42:48.210804+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.210804+0000","last_undegraded":"2024-05-10T10:42:48.210804+0000","last_fullsized":"2024-05-10T10:42:48.210804+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209273+0000","last_change":"2024-05-10T10:42:48.209273+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209273+0000","last_undegraded":"2024-05-10T10:42:48.209273+0000","last_fullsized":"2024-05-10T10:42:48.209273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.210932+0000","last_change":"2024-05-10T10:42:48.210932+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.210932+0000","last_undegraded":"2024-05-10T10:42:48.210932+0000","last_fullsized":"2024-05-10T10:42:48.210932+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209283+0000","last_change":"2024-05-10T10:42:48.209283+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209283+0000","last_undegraded":"2024-05-10T10:42:48.209283+0000","last_fullsized":"2024-05-10T10:42:48.209283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.214602+0000","last_change":"2024-05-10T10:42:48.214602+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.214602+0000","last_undegraded":"2024-05-10T10:42:48.214602+0000","last_fullsized":"2024-05-10T10:42:48.214602+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.211814+0000","last_change":"2024-05-10T10:42:48.211814+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.211814+0000","last_undegraded":"2024-05-10T10:42:48.211814+0000","last_fullsized":"2024-05-10T10:42:48.211814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.211887+0000","last_change":"2024-05-10T10:42:48.211887+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.211887+0000","last_undegraded":"2024-05-10T10:42:48.211887+0000","last_fullsized":"2024-05-10T10:42:48.211887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.208262+0000","last_change":"2024-05-10T10:42:48.208262+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.208262+0000","last_undegraded":"2024-05-10T10:42:48.208262+0000","last_fullsized":"2024-05-10T10:42:48.208262+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.207591+0000","last_change":"2024-05-10T10:42:48.207591+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.207591+0000","last_undegraded":"2024-05-10T10:42:48.207591+0000","last_fullsized":"2024-05-10T10:42:48.207591+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.207902+0000","last_change":"2024-05-10T10:42:48.207902+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.207902+0000","last_undegraded":"2024-05-10T10:42:48.207902+0000","last_fullsized":"2024-05-10T10:42:48.207902+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.207532+0000","last_change":"2024-05-10T10:42:48.207532+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.207532+0000","last_undegraded":"2024-05-10T10:42:48.207532+0000","last_fullsized":"2024-05-10T10:42:48.207532+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.212663+0000","last_change":"2024-05-10T10:42:48.212663+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.212663+0000","last_undegraded":"2024-05-10T10:42:48.212663+0000","last_fullsized":"2024-05-10T10:42:48.212663+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.213778+0000","last_change":"2024-05-10T10:42:48.213778+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.213778+0000","last_undegraded":"2024-05-10T10:42:48.213778+0000","last_fullsized":"2024-05-10T10:42:48.213778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209664+0000","last_change":"2024-05-10T10:42:48.209664+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209664+0000","last_undegraded":"2024-05-10T10:42:48.209664+0000","last_fullsized":"2024-05-10T10:42:48.209664+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209996+0000","last_change":"2024-05-10T10:42:48.209996+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209996+0000","last_undegraded":"2024-05-10T10:42:48.209996+0000","last_fullsized":"2024-05-10T10:42:48.209996+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209291+0000","last_change":"2024-05-10T10:42:48.209291+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209291+0000","last_undegraded":"2024-05-10T10:42:48.209291+0000","last_fullsized":"2024-05-10T10:42:48.209291+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209526+0000","last_change":"2024-05-10T10:42:48.209526+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209526+0000","last_undegraded":"2024-05-10T10:42:48.209526+0000","last_fullsized":"2024-05-10T10:42:48.209526+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.206994+0000","last_change":"2024-05-10T10:42:48.206994+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.206994+0000","last_undegraded":"2024-05-10T10:42:48.206994+0000","last_fullsized":"2024-05-10T10:42:48.206994+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.210942+0000","last_change":"2024-05-10T10:42:48.210942+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.210942+0000","last_undegraded":"2024-05-10T10:42:48.210942+0000","last_fullsized":"2024-05-10T10:42:48.210942+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209297+0000","last_change":"2024-05-10T10:42:48.209297+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209297+0000","last_undegraded":"2024-05-10T10:42:48.209297+0000","last_fullsized":"2024-05-10T10:42:48.209297+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.214597+0000","last_change":"2024-05-10T10:42:48.214597+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.214597+0000","last_undegraded":"2024-05-10T10:42:48.214597+0000","last_fullsized":"2024-05-10T10:42:48.214597+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.214588+0000","last_change":"2024-05-10T10:42:48.214588+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.214588+0000","last_undegraded":"2024-05-10T10:42:48.214588+0000","last_fullsized":"2024-05-10T10:42:48.214588+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209350+0000","last_change":"2024-05-10T10:42:48.209350+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209350+0000","last_undegraded":"2024-05-10T10:42:48.209350+0000","last_fullsized":"2024-05-10T10:42:48.209350+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.210261+0000","last_change":"2024-05-10T10:42:48.210261+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.210261+0000","last_undegraded":"2024-05-10T10:42:48.210261+0000","last_fullsized":"2024-05-10T10:42:48.210261+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.211068+0000","last_change":"2024-05-10T10:42:48.211068+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.211068+0000","last_undegraded":"2024-05-10T10:42:48.211068+0000","last_fullsized":"2024-05-10T10:42:48.211068+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.209293+0000","last_change":"2024-05-10T10:42:48.209293+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.209293+0000","last_undegraded":"2024-05-10T10:42:48.209293+0000","last_fullsized":"2024-05-10T10:42:48.209293+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.210476+0000","last_change":"2024-05-10T10:42:48.210476+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.210476+0000","last_undegraded":"2024-05-10T10:42:48.210476+0000","last_fullsized":"2024-05-10T10:42:48.210476+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T10:42:49.203714+0000","last_change":"2024-05-10T10:42:49.203714+0000","last_active":"2024-05-10T10:42:49.203714+0000","last_peered":"2024-05-10T10:42:49.203714+0000","last_clean":"2024-05-10T10:42:49.203714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:49.203714+0000","last_undegraded":"2024-05-10T10:42:49.203714+0000","last_fullsized":"2024-05-10T10:42:49.203714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-05-10T10:42:48.206987+0000","last_change":"2024-05-10T10:42:48.206987+0000","last_active":"2024-05-10T10:42:48.201581+0000","last_peered":"2024-05-10T10:42:48.201581+0000","last_clean":"2024-05-10T10:42:48.201581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T10:42:48.206987+0000","last_undegraded":"2024-05-10T10:42:48.206987+0000","last_fullsized":"2024-05-10T10:42:48.206987+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":304,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-10T10:42:48.211255+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:42:48.211255+0000","last_peered":"2024-05-10T10:42:48.211255+0000","last_clean":"2024-05-10T10:42:48.211255+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:42:48.211255+0000","last_undegraded":"2024-05-10T10:42:48.211255+0000","last_fullsized":"2024-05-10T10:42:48.211255+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2015232,"data_stored":1248421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":658053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":658053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":658053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":658053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":658053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":658053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2015232,"data_stored":1248421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:42:54.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:42:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.420+0000 7fa35a759640 1 Processor -- start 2024-05-10T10:42:54.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.420+0000 7fa35a759640 1 -- start start 2024-05-10T10:42:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa35a759640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa354163730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa35a759640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3540caf50 0x7fa35415fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa35a759640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa354160240 con 0x7fa35415f730 2024-05-10T10:42:54.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa35a759640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa354163c70 con 0x7fa3540caf50 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa35a759640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa354160520 con 0x7fa35406fa60 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa358cf6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3540caf50 0x7fa35415fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa358cf6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3540caf50 0x7fa35415fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45470/0 (socket says 172.21.15.53:45470) 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa358cf6640 1 -- 172.21.15.53:0/269939504 learned_addr learned my addr 172.21.15.53:0/269939504 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.421+0000 7fa353fff640 1 --2- 172.21.15.53:0/269939504 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa354163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.422+0000 7fa358cf6640 1 -- 172.21.15.53:0/269939504 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 msgr2=0x7fa354163730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.422+0000 7fa358cf6640 1 --2- 172.21.15.53:0/269939504 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa354163730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.422+0000 7fa358cf6640 1 -- 172.21.15.53:0/269939504 >> v1:172.21.15.135:6789/0 conn(0x7fa35415f730 legacy=0x7fa3540ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.422+0000 7fa358cf6640 1 -- 172.21.15.53:0/269939504 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa354162f30 con 0x7fa3540caf50 2024-05-10T10:42:54.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.422+0000 7fa353fff640 1 --2- 172.21.15.53:0/269939504 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa354163730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:54.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.422+0000 7fa358cf6640 1 --2- 172.21.15.53:0/269939504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35415fb90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009e50 tx=0x7fa34c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb46472db8eb0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.422+0000 7fa352ffd640 1 -- 172.21.15.53:0/269939504 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c036070 con 0x7fa3540caf50 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa352ffd640 1 -- 172.21.15.53:0/269939504 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c031600 con 0x7fa3540caf50 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa352ffd640 1 -- 172.21.15.53:0/269939504 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0319d0 con 0x7fa3540caf50 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 -- 172.21.15.53:0/269939504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 msgr2=0x7fa35415fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 --2- 172.21.15.53:0/269939504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35415fb90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009e50 tx=0x7fa34c02a680 comp rx=0 tx=0).stop 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 -- 172.21.15.53:0/269939504 shutdown_connections 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 --2- 172.21.15.53:0/269939504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35415fb90 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 --2- 172.21.15.53:0/269939504 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa354163730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 -- 172.21.15.53:0/269939504 >> 172.21.15.53:0/269939504 conn(0x7fa3540d1510 msgr2=0x7fa3540d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 -- 172.21.15.53:0/269939504 shutdown_connections 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.423+0000 7fa35a759640 1 -- 172.21.15.53:0/269939504 wait complete. 2024-05-10T10:42:54.426 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.424+0000 7fa35a759640 1 Processor -- start 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa35a759640 1 -- start start 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa35a759640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa3540b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa35a759640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa35a759640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa35415f730 0x7fa3540b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa35a759640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa35406c560 con 0x7fa3540caf50 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa35a759640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa35406eae0 con 0x7fa35415f730 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa35a759640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa35406c6e0 con 0x7fa35406fa60 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35414f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45472/0 (socket says 172.21.15.53:45472) 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.425+0000 7fa3537fe640 1 -- 172.21.15.53:0/2442724293 learned_addr learned my addr 172.21.15.53:0/2442724293 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa358cf6640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa35415f730 0x7fa3540b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa353fff640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa3540b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.429 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa3537fe640 1 -- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 msgr2=0x7fa3540b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa3537fe640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa3540b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa3537fe640 1 -- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa35415f730 msgr2=0x7fa3540b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa3537fe640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa35415f730 0x7fa3540b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa3537fe640 1 -- 172.21.15.53:0/2442724293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa354150790 con 0x7fa3540caf50 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa358cf6640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa35415f730 0x7fa3540b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa3537fe640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35414f730 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c3c0 tx=0x7fa34800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa350ff9640 1 -- 172.21.15.53:0/2442724293 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480052d0 con 0x7fa3540caf50 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa350ff9640 1 -- 172.21.15.53:0/2442724293 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348005490 con 0x7fa3540caf50 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3540b4d50 con 0x7fa3540caf50 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa350ff9640 1 -- 172.21.15.53:0/2442724293 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348007260 con 0x7fa3540caf50 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.426+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35415c980 con 0x7fa3540caf50 2024-05-10T10:42:54.430 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.427+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35406a220 con 0x7fa3540caf50 2024-05-10T10:42:54.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.428+0000 7fa350ff9640 1 -- 172.21.15.53:0/2442724293 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa348007400 con 0x7fa3540caf50 2024-05-10T10:42:54.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.428+0000 7fa350ff9640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3400762d0 0x7fa340078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.429+0000 7fa350ff9640 1 -- 172.21.15.53:0/2442724293 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa348096c40 con 0x7fa3540caf50 2024-05-10T10:42:54.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.429+0000 7fa353fff640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3400762d0 0x7fa340078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.431 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.429+0000 7fa353fff640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3400762d0 0x7fa340078790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa34400a990 tx=0x7fa344008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.434 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.433+0000 7fa350ff9640 1 -- 172.21.15.53:0/2442724293 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa348060400 con 0x7fa3540caf50 2024-05-10T10:42:54.623 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.622+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa35406dcc0 con 0x7fa3400762d0 2024-05-10T10:42:54.629 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.628+0000 7fa350ff9640 1 -- 172.21.15.53:0/2442724293 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252476 (secure 0 0 0) 0x7fa35406dcc0 con 0x7fa3400762d0 2024-05-10T10:42:54.632 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:54.633 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:42:54.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3400762d0 msgr2=0x7fa340078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3400762d0 0x7fa340078790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa34400a990 tx=0x7fa344008040 comp rx=0 tx=0).stop 2024-05-10T10:42:54.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 msgr2=0x7fa35414f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.635 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35414f730 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c3c0 tx=0x7fa34800c880 comp rx=0 tx=0).stop 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 shutdown_connections 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa3400762d0 0x7fa340078790 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa35415f730 0x7fa3540b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3540caf50 0x7fa35414f730 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 --2- 172.21.15.53:0/2442724293 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa35406fa60 0x7fa3540b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 >> 172.21.15.53:0/2442724293 conn(0x7fa3540d1510 msgr2=0x7fa3540ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 shutdown_connections 2024-05-10T10:42:54.636 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.632+0000 7fa35a759640 1 -- 172.21.15.53:0/2442724293 wait complete. 2024-05-10T10:42:54.644 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-05-10T10:42:54.296232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262204,"kb_used_data":12712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712516,"statfs":{"total":773094113280,"available":772825616384,"internally_reserved":0,"allocated":13017088,"data_stored":6663860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001511"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350032+0000","last_change":"2024-05-10T10:42:50.238566+0000","last_active":"2024-05-10T10:42:51.350032+0000","last_peered":"2024-05-10T10:42:51.350032+0000","last_clean":"2024-05-10T10:42:51.350032+0000","last_became_active":"2024-05-10T10:42:50.238307+0000","last_became_peered":"2024-05-10T10:42:50.238307+0000","last_unstale":"2024-05-10T10:42:51.350032+0000","last_undegraded":"2024-05-10T10:42:51.350032+0000","last_fullsized":"2024-05-10T10:42:51.350032+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:45.272396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349807+0000","last_change":"2024-05-10T10:42:50.239901+0000","last_active":"2024-05-10T10:42:51.349807+0000","last_peered":"2024-05-10T10:42:51.349807+0000","last_clean":"2024-05-10T10:42:51.349807+0000","last_became_active":"2024-05-10T10:42:50.239619+0000","last_became_peered":"2024-05-10T10:42:50.239619+0000","last_unstale":"2024-05-10T10:42:51.349807+0000","last_undegraded":"2024-05-10T10:42:51.349807+0000","last_fullsized":"2024-05-10T10:42:51.349807+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:07:43.746287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349963+0000","last_change":"2024-05-10T10:42:50.245991+0000","last_active":"2024-05-10T10:42:51.349963+0000","last_peered":"2024-05-10T10:42:51.349963+0000","last_clean":"2024-05-10T10:42:51.349963+0000","last_became_active":"2024-05-10T10:42:50.245619+0000","last_became_peered":"2024-05-10T10:42:50.245619+0000","last_unstale":"2024-05-10T10:42:51.349963+0000","last_undegraded":"2024-05-10T10:42:51.349963+0000","last_fullsized":"2024-05-10T10:42:51.349963+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:35.725138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235450+0000","last_change":"2024-05-10T10:42:50.227312+0000","last_active":"2024-05-10T10:42:51.235450+0000","last_peered":"2024-05-10T10:42:51.235450+0000","last_clean":"2024-05-10T10:42:51.235450+0000","last_became_active":"2024-05-10T10:42:50.227053+0000","last_became_peered":"2024-05-10T10:42:50.227053+0000","last_unstale":"2024-05-10T10:42:51.235450+0000","last_undegraded":"2024-05-10T10:42:51.235450+0000","last_fullsized":"2024-05-10T10:42:51.235450+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:09:33.941842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349669+0000","last_change":"2024-05-10T10:42:50.242701+0000","last_active":"2024-05-10T10:42:51.349669+0000","last_peered":"2024-05-10T10:42:51.349669+0000","last_clean":"2024-05-10T10:42:51.349669+0000","last_became_active":"2024-05-10T10:42:50.242414+0000","last_became_peered":"2024-05-10T10:42:50.242414+0000","last_unstale":"2024-05-10T10:42:51.349669+0000","last_undegraded":"2024-05-10T10:42:51.349669+0000","last_fullsized":"2024-05-10T10:42:51.349669+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:59.374542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349536+0000","last_change":"2024-05-10T10:42:50.247276+0000","last_active":"2024-05-10T10:42:51.349536+0000","last_peered":"2024-05-10T10:42:51.349536+0000","last_clean":"2024-05-10T10:42:51.349536+0000","last_became_active":"2024-05-10T10:42:50.246762+0000","last_became_peered":"2024-05-10T10:42:50.246762+0000","last_unstale":"2024-05-10T10:42:51.349536+0000","last_undegraded":"2024-05-10T10:42:51.349536+0000","last_fullsized":"2024-05-10T10:42:51.349536+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:23:49.100190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349192+0000","last_change":"2024-05-10T10:42:50.238720+0000","last_active":"2024-05-10T10:42:51.349192+0000","last_peered":"2024-05-10T10:42:51.349192+0000","last_clean":"2024-05-10T10:42:51.349192+0000","last_became_active":"2024-05-10T10:42:50.238417+0000","last_became_peered":"2024-05-10T10:42:50.238417+0000","last_unstale":"2024-05-10T10:42:51.349192+0000","last_undegraded":"2024-05-10T10:42:51.349192+0000","last_fullsized":"2024-05-10T10:42:51.349192+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:32.516389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349037+0000","last_change":"2024-05-10T10:42:50.245947+0000","last_active":"2024-05-10T10:42:51.349037+0000","last_peered":"2024-05-10T10:42:51.349037+0000","last_clean":"2024-05-10T10:42:51.349037+0000","last_became_active":"2024-05-10T10:42:50.245629+0000","last_became_peered":"2024-05-10T10:42:50.245629+0000","last_unstale":"2024-05-10T10:42:51.349037+0000","last_undegraded":"2024-05-10T10:42:51.349037+0000","last_fullsized":"2024-05-10T10:42:51.349037+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:09:33.941842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236106+0000","last_change":"2024-05-10T10:42:50.242999+0000","last_active":"2024-05-10T10:42:51.236106+0000","last_peered":"2024-05-10T10:42:51.236106+0000","last_clean":"2024-05-10T10:42:51.236106+0000","last_became_active":"2024-05-10T10:42:50.242596+0000","last_became_peered":"2024-05-10T10:42:50.242596+0000","last_unstale":"2024-05-10T10:42:51.236106+0000","last_undegraded":"2024-05-10T10:42:51.236106+0000","last_fullsized":"2024-05-10T10:42:51.236106+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:03:05.787535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349399+0000","last_change":"2024-05-10T10:42:50.237123+0000","last_active":"2024-05-10T10:42:51.349399+0000","last_peered":"2024-05-10T10:42:51.349399+0000","last_clean":"2024-05-10T10:42:51.349399+0000","last_became_active":"2024-05-10T10:42:50.236789+0000","last_became_peered":"2024-05-10T10:42:50.236789+0000","last_unstale":"2024-05-10T10:42:51.349399+0000","last_undegraded":"2024-05-10T10:42:51.349399+0000","last_fullsized":"2024-05-10T10:42:51.349399+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:36:26.650391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349897+0000","last_change":"2024-05-10T10:42:50.236378+0000","last_active":"2024-05-10T10:42:51.349897+0000","last_peered":"2024-05-10T10:42:51.349897+0000","last_clean":"2024-05-10T10:42:51.349897+0000","last_became_active":"2024-05-10T10:42:50.235735+0000","last_became_peered":"2024-05-10T10:42:50.235735+0000","last_unstale":"2024-05-10T10:42:51.349897+0000","last_undegraded":"2024-05-10T10:42:51.349897+0000","last_fullsized":"2024-05-10T10:42:51.349897+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:36:26.650391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349460+0000","last_change":"2024-05-10T10:42:50.237738+0000","last_active":"2024-05-10T10:42:51.349460+0000","last_peered":"2024-05-10T10:42:51.349460+0000","last_clean":"2024-05-10T10:42:51.349460+0000","last_became_active":"2024-05-10T10:42:50.237469+0000","last_became_peered":"2024-05-10T10:42:50.237469+0000","last_unstale":"2024-05-10T10:42:51.349460+0000","last_undegraded":"2024-05-10T10:42:51.349460+0000","last_fullsized":"2024-05-10T10:42:51.349460+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:27:58.325784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349753+0000","last_change":"2024-05-10T10:42:50.225692+0000","last_active":"2024-05-10T10:42:51.349753+0000","last_peered":"2024-05-10T10:42:51.349753+0000","last_clean":"2024-05-10T10:42:51.349753+0000","last_became_active":"2024-05-10T10:42:50.225373+0000","last_became_peered":"2024-05-10T10:42:50.225373+0000","last_unstale":"2024-05-10T10:42:51.349753+0000","last_undegraded":"2024-05-10T10:42:51.349753+0000","last_fullsized":"2024-05-10T10:42:51.349753+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:45.272396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350107+0000","last_change":"2024-05-10T10:42:50.245136+0000","last_active":"2024-05-10T10:42:51.350107+0000","last_peered":"2024-05-10T10:42:51.350107+0000","last_clean":"2024-05-10T10:42:51.350107+0000","last_became_active":"2024-05-10T10:42:50.244876+0000","last_became_peered":"2024-05-10T10:42:50.244876+0000","last_unstale":"2024-05-10T10:42:51.350107+0000","last_undegraded":"2024-05-10T10:42:51.350107+0000","last_fullsized":"2024-05-10T10:42:51.350107+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:09.166913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349369+0000","last_change":"2024-05-10T10:42:50.236115+0000","last_active":"2024-05-10T10:42:51.349369+0000","last_peered":"2024-05-10T10:42:51.349369+0000","last_clean":"2024-05-10T10:42:51.349369+0000","last_became_active":"2024-05-10T10:42:50.235807+0000","last_became_peered":"2024-05-10T10:42:50.235807+0000","last_unstale":"2024-05-10T10:42:51.349369+0000","last_undegraded":"2024-05-10T10:42:51.349369+0000","last_fullsized":"2024-05-10T10:42:51.349369+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:53.798157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349006+0000","last_change":"2024-05-10T10:42:50.241681+0000","last_active":"2024-05-10T10:42:51.349006+0000","last_peered":"2024-05-10T10:42:51.349006+0000","last_clean":"2024-05-10T10:42:51.349006+0000","last_became_active":"2024-05-10T10:42:50.241359+0000","last_became_peered":"2024-05-10T10:42:50.241359+0000","last_unstale":"2024-05-10T10:42:51.349006+0000","last_undegraded":"2024-05-10T10:42:51.349006+0000","last_fullsized":"2024-05-10T10:42:51.349006+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:30:53.362674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349524+0000","last_change":"2024-05-10T10:42:50.230323+0000","last_active":"2024-05-10T10:42:51.349524+0000","last_peered":"2024-05-10T10:42:51.349524+0000","last_clean":"2024-05-10T10:42:51.349524+0000","last_became_active":"2024-05-10T10:42:50.230038+0000","last_became_peered":"2024-05-10T10:42:50.230038+0000","last_unstale":"2024-05-10T10:42:51.349524+0000","last_undegraded":"2024-05-10T10:42:51.349524+0000","last_fullsized":"2024-05-10T10:42:51.349524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:44:05.030791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349949+0000","last_change":"2024-05-10T10:42:50.238943+0000","last_active":"2024-05-10T10:42:51.349949+0000","last_peered":"2024-05-10T10:42:51.349949+0000","last_clean":"2024-05-10T10:42:51.349949+0000","last_became_active":"2024-05-10T10:42:50.237417+0000","last_became_peered":"2024-05-10T10:42:50.237417+0000","last_unstale":"2024-05-10T10:42:51.349949+0000","last_undegraded":"2024-05-10T10:42:51.349949+0000","last_fullsized":"2024-05-10T10:42:51.349949+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:03:05.787535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350234+0000","last_change":"2024-05-10T10:42:50.240137+0000","last_active":"2024-05-10T10:42:51.350234+0000","last_peered":"2024-05-10T10:42:51.350234+0000","last_clean":"2024-05-10T10:42:51.350234+0000","last_became_active":"2024-05-10T10:42:50.239880+0000","last_became_peered":"2024-05-10T10:42:50.239880+0000","last_unstale":"2024-05-10T10:42:51.350234+0000","last_undegraded":"2024-05-10T10:42:51.350234+0000","last_fullsized":"2024-05-10T10:42:51.350234+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:53.798157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.234105+0000","last_change":"2024-05-10T10:42:50.240014+0000","last_active":"2024-05-10T10:42:51.234105+0000","last_peered":"2024-05-10T10:42:51.234105+0000","last_clean":"2024-05-10T10:42:51.234105+0000","last_became_active":"2024-05-10T10:42:50.239721+0000","last_became_peered":"2024-05-10T10:42:50.239721+0000","last_unstale":"2024-05-10T10:42:51.234105+0000","last_undegraded":"2024-05-10T10:42:51.234105+0000","last_fullsized":"2024-05-10T10:42:51.234105+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:28:19.207470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236262+0000","last_change":"2024-05-10T10:42:50.230300+0000","last_active":"2024-05-10T10:42:51.236262+0000","last_peered":"2024-05-10T10:42:51.236262+0000","last_clean":"2024-05-10T10:42:51.236262+0000","last_became_active":"2024-05-10T10:42:50.230079+0000","last_became_peered":"2024-05-10T10:42:50.230079+0000","last_unstale":"2024-05-10T10:42:51.236262+0000","last_undegraded":"2024-05-10T10:42:51.236262+0000","last_fullsized":"2024-05-10T10:42:51.236262+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:18:35.725138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350000+0000","last_change":"2024-05-10T10:42:50.237222+0000","last_active":"2024-05-10T10:42:51.350000+0000","last_peered":"2024-05-10T10:42:51.350000+0000","last_clean":"2024-05-10T10:42:51.350000+0000","last_became_active":"2024-05-10T10:42:50.236956+0000","last_became_peered":"2024-05-10T10:42:50.236956+0000","last_unstale":"2024-05-10T10:42:51.350000+0000","last_undegraded":"2024-05-10T10:42:51.350000+0000","last_fullsized":"2024-05-10T10:42:51.350000+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:44:05.030791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235510+0000","last_change":"2024-05-10T10:42:50.229044+0000","last_active":"2024-05-10T10:42:51.235510+0000","last_peered":"2024-05-10T10:42:51.235510+0000","last_clean":"2024-05-10T10:42:51.235510+0000","last_became_active":"2024-05-10T10:42:50.228793+0000","last_became_peered":"2024-05-10T10:42:50.228793+0000","last_unstale":"2024-05-10T10:42:51.235510+0000","last_undegraded":"2024-05-10T10:42:51.235510+0000","last_fullsized":"2024-05-10T10:42:51.235510+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:57.210515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235676+0000","last_change":"2024-05-10T10:42:50.240848+0000","last_active":"2024-05-10T10:42:51.235676+0000","last_peered":"2024-05-10T10:42:51.235676+0000","last_clean":"2024-05-10T10:42:51.235676+0000","last_became_active":"2024-05-10T10:42:50.240555+0000","last_became_peered":"2024-05-10T10:42:50.240555+0000","last_unstale":"2024-05-10T10:42:51.235676+0000","last_undegraded":"2024-05-10T10:42:51.235676+0000","last_fullsized":"2024-05-10T10:42:51.235676+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:39:07.995037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235511+0000","last_change":"2024-05-10T10:42:50.234668+0000","last_active":"2024-05-10T10:42:51.235511+0000","last_peered":"2024-05-10T10:42:51.235511+0000","last_clean":"2024-05-10T10:42:51.235511+0000","last_became_active":"2024-05-10T10:42:50.234360+0000","last_became_peered":"2024-05-10T10:42:50.234360+0000","last_unstale":"2024-05-10T10:42:51.235511+0000","last_undegraded":"2024-05-10T10:42:51.235511+0000","last_fullsized":"2024-05-10T10:42:51.235511+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:45.192670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.234270+0000","last_change":"2024-05-10T10:42:50.228098+0000","last_active":"2024-05-10T10:42:51.234270+0000","last_peered":"2024-05-10T10:42:51.234270+0000","last_clean":"2024-05-10T10:42:51.234270+0000","last_became_active":"2024-05-10T10:42:50.227804+0000","last_became_peered":"2024-05-10T10:42:50.227804+0000","last_unstale":"2024-05-10T10:42:51.234270+0000","last_undegraded":"2024-05-10T10:42:51.234270+0000","last_fullsized":"2024-05-10T10:42:51.234270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:24:00.993441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350349+0000","last_change":"2024-05-10T10:42:50.239444+0000","last_active":"2024-05-10T10:42:51.350349+0000","last_peered":"2024-05-10T10:42:51.350349+0000","last_clean":"2024-05-10T10:42:51.350349+0000","last_became_active":"2024-05-10T10:42:50.239170+0000","last_became_peered":"2024-05-10T10:42:50.239170+0000","last_unstale":"2024-05-10T10:42:51.350349+0000","last_undegraded":"2024-05-10T10:42:51.350349+0000","last_fullsized":"2024-05-10T10:42:51.350349+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:29.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349608+0000","last_change":"2024-05-10T10:42:50.235771+0000","last_active":"2024-05-10T10:42:51.349608+0000","last_peered":"2024-05-10T10:42:51.349608+0000","last_clean":"2024-05-10T10:42:51.349608+0000","last_became_active":"2024-05-10T10:42:50.235507+0000","last_became_peered":"2024-05-10T10:42:50.235507+0000","last_unstale":"2024-05-10T10:42:51.349608+0000","last_undegraded":"2024-05-10T10:42:51.349608+0000","last_fullsized":"2024-05-10T10:42:51.349608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:53:35.328648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.234373+0000","last_change":"2024-05-10T10:42:50.238137+0000","last_active":"2024-05-10T10:42:51.234373+0000","last_peered":"2024-05-10T10:42:51.234373+0000","last_clean":"2024-05-10T10:42:51.234373+0000","last_became_active":"2024-05-10T10:42:50.237863+0000","last_became_peered":"2024-05-10T10:42:50.237863+0000","last_unstale":"2024-05-10T10:42:51.234373+0000","last_undegraded":"2024-05-10T10:42:51.234373+0000","last_fullsized":"2024-05-10T10:42:51.234373+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:57:24.698453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350139+0000","last_change":"2024-05-10T10:42:50.235534+0000","last_active":"2024-05-10T10:42:51.350139+0000","last_peered":"2024-05-10T10:42:51.350139+0000","last_clean":"2024-05-10T10:42:51.350139+0000","last_became_active":"2024-05-10T10:42:50.235219+0000","last_became_peered":"2024-05-10T10:42:50.235219+0000","last_unstale":"2024-05-10T10:42:51.350139+0000","last_undegraded":"2024-05-10T10:42:51.350139+0000","last_fullsized":"2024-05-10T10:42:51.350139+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:04:24.464674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235313+0000","last_change":"2024-05-10T10:42:50.224980+0000","last_active":"2024-05-10T10:42:51.235313+0000","last_peered":"2024-05-10T10:42:51.235313+0000","last_clean":"2024-05-10T10:42:51.235313+0000","last_became_active":"2024-05-10T10:42:50.224325+0000","last_became_peered":"2024-05-10T10:42:50.224325+0000","last_unstale":"2024-05-10T10:42:51.235313+0000","last_undegraded":"2024-05-10T10:42:51.235313+0000","last_fullsized":"2024-05-10T10:42:51.235313+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:18.341207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235545+0000","last_change":"2024-05-10T10:42:50.240752+0000","last_active":"2024-05-10T10:42:51.235545+0000","last_peered":"2024-05-10T10:42:51.235545+0000","last_clean":"2024-05-10T10:42:51.235545+0000","last_became_active":"2024-05-10T10:42:50.240437+0000","last_became_peered":"2024-05-10T10:42:50.240437+0000","last_unstale":"2024-05-10T10:42:51.235545+0000","last_undegraded":"2024-05-10T10:42:51.235545+0000","last_fullsized":"2024-05-10T10:42:51.235545+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:58:15.802823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349926+0000","last_change":"2024-05-10T10:42:50.239973+0000","last_active":"2024-05-10T10:42:51.349926+0000","last_peered":"2024-05-10T10:42:51.349926+0000","last_clean":"2024-05-10T10:42:51.349926+0000","last_became_active":"2024-05-10T10:42:50.239657+0000","last_became_peered":"2024-05-10T10:42:50.239657+0000","last_unstale":"2024-05-10T10:42:51.349926+0000","last_undegraded":"2024-05-10T10:42:51.349926+0000","last_fullsized":"2024-05-10T10:42:51.349926+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:18.341207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350028+0000","last_change":"2024-05-10T10:42:49.230322+0000","last_active":"2024-05-10T10:42:51.350028+0000","last_peered":"2024-05-10T10:42:51.350028+0000","last_clean":"2024-05-10T10:42:51.350028+0000","last_became_active":"2024-05-10T10:42:49.230034+0000","last_became_peered":"2024-05-10T10:42:49.230034+0000","last_unstale":"2024-05-10T10:42:51.350028+0000","last_undegraded":"2024-05-10T10:42:51.350028+0000","last_fullsized":"2024-05-10T10:42:51.350028+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:31:39.278453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350276+0000","last_change":"2024-05-10T10:42:50.247623+0000","last_active":"2024-05-10T10:42:51.350276+0000","last_peered":"2024-05-10T10:42:51.350276+0000","last_clean":"2024-05-10T10:42:51.350276+0000","last_became_active":"2024-05-10T10:42:50.247177+0000","last_became_peered":"2024-05-10T10:42:50.247177+0000","last_unstale":"2024-05-10T10:42:51.350276+0000","last_undegraded":"2024-05-10T10:42:51.350276+0000","last_fullsized":"2024-05-10T10:42:51.350276+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:53:09.063408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349451+0000","last_change":"2024-05-10T10:42:49.230282+0000","last_active":"2024-05-10T10:42:51.349451+0000","last_peered":"2024-05-10T10:42:51.349451+0000","last_clean":"2024-05-10T10:42:51.349451+0000","last_became_active":"2024-05-10T10:42:49.229873+0000","last_became_peered":"2024-05-10T10:42:49.229873+0000","last_unstale":"2024-05-10T10:42:51.349451+0000","last_undegraded":"2024-05-10T10:42:51.349451+0000","last_fullsized":"2024-05-10T10:42:51.349451+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:31:39.278453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236071+0000","last_change":"2024-05-10T10:42:50.230684+0000","last_active":"2024-05-10T10:42:51.236071+0000","last_peered":"2024-05-10T10:42:51.236071+0000","last_clean":"2024-05-10T10:42:51.236071+0000","last_became_active":"2024-05-10T10:42:50.230411+0000","last_became_peered":"2024-05-10T10:42:50.230411+0000","last_unstale":"2024-05-10T10:42:51.236071+0000","last_undegraded":"2024-05-10T10:42:51.236071+0000","last_fullsized":"2024-05-10T10:42:51.236071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:22:00.423221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235711+0000","last_change":"2024-05-10T10:42:49.228051+0000","last_active":"2024-05-10T10:42:51.235711+0000","last_peered":"2024-05-10T10:42:51.235711+0000","last_clean":"2024-05-10T10:42:51.235711+0000","last_became_active":"2024-05-10T10:42:49.227600+0000","last_became_peered":"2024-05-10T10:42:49.227600+0000","last_unstale":"2024-05-10T10:42:51.235711+0000","last_undegraded":"2024-05-10T10:42:51.235711+0000","last_fullsized":"2024-05-10T10:42:51.235711+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:44:04.028657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236264+0000","last_change":"2024-05-10T10:42:50.231048+0000","last_active":"2024-05-10T10:42:51.236264+0000","last_peered":"2024-05-10T10:42:51.236264+0000","last_clean":"2024-05-10T10:42:51.236264+0000","last_became_active":"2024-05-10T10:42:50.230808+0000","last_became_peered":"2024-05-10T10:42:50.230808+0000","last_unstale":"2024-05-10T10:42:51.236264+0000","last_undegraded":"2024-05-10T10:42:51.236264+0000","last_fullsized":"2024-05-10T10:42:51.236264+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:29.045471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349979+0000","last_change":"2024-05-10T10:42:49.229425+0000","last_active":"2024-05-10T10:42:51.349979+0000","last_peered":"2024-05-10T10:42:51.349979+0000","last_clean":"2024-05-10T10:42:51.349979+0000","last_became_active":"2024-05-10T10:42:49.229063+0000","last_became_peered":"2024-05-10T10:42:49.229063+0000","last_unstale":"2024-05-10T10:42:51.349979+0000","last_undegraded":"2024-05-10T10:42:51.349979+0000","last_fullsized":"2024-05-10T10:42:51.349979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:40:58.006026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349004+0000","last_change":"2024-05-10T10:42:50.240996+0000","last_active":"2024-05-10T10:42:51.349004+0000","last_peered":"2024-05-10T10:42:51.349004+0000","last_clean":"2024-05-10T10:42:51.349004+0000","last_became_active":"2024-05-10T10:42:50.240412+0000","last_became_peered":"2024-05-10T10:42:50.240412+0000","last_unstale":"2024-05-10T10:42:51.349004+0000","last_undegraded":"2024-05-10T10:42:51.349004+0000","last_fullsized":"2024-05-10T10:42:51.349004+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:53.798157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349733+0000","last_change":"2024-05-10T10:42:49.226058+0000","last_active":"2024-05-10T10:42:51.349733+0000","last_peered":"2024-05-10T10:42:51.349733+0000","last_clean":"2024-05-10T10:42:51.349733+0000","last_became_active":"2024-05-10T10:42:49.225805+0000","last_became_peered":"2024-05-10T10:42:49.225805+0000","last_unstale":"2024-05-10T10:42:51.349733+0000","last_undegraded":"2024-05-10T10:42:51.349733+0000","last_fullsized":"2024-05-10T10:42:51.349733+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:56.778602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349387+0000","last_change":"2024-05-10T10:42:50.235504+0000","last_active":"2024-05-10T10:42:51.349387+0000","last_peered":"2024-05-10T10:42:51.349387+0000","last_clean":"2024-05-10T10:42:51.349387+0000","last_became_active":"2024-05-10T10:42:50.235207+0000","last_became_peered":"2024-05-10T10:42:50.235207+0000","last_unstale":"2024-05-10T10:42:51.349387+0000","last_undegraded":"2024-05-10T10:42:51.349387+0000","last_fullsized":"2024-05-10T10:42:51.349387+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:29.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349587+0000","last_change":"2024-05-10T10:42:49.231627+0000","last_active":"2024-05-10T10:42:51.349587+0000","last_peered":"2024-05-10T10:42:51.349587+0000","last_clean":"2024-05-10T10:42:51.349587+0000","last_became_active":"2024-05-10T10:42:49.231278+0000","last_became_peered":"2024-05-10T10:42:49.231278+0000","last_unstale":"2024-05-10T10:42:51.349587+0000","last_undegraded":"2024-05-10T10:42:51.349587+0000","last_fullsized":"2024-05-10T10:42:51.349587+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:22:35.793344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350245+0000","last_change":"2024-05-10T10:42:50.232469+0000","last_active":"2024-05-10T10:42:51.350245+0000","last_peered":"2024-05-10T10:42:51.350245+0000","last_clean":"2024-05-10T10:42:51.350245+0000","last_became_active":"2024-05-10T10:42:50.232152+0000","last_became_peered":"2024-05-10T10:42:50.232152+0000","last_unstale":"2024-05-10T10:42:51.350245+0000","last_undegraded":"2024-05-10T10:42:51.350245+0000","last_fullsized":"2024-05-10T10:42:51.350245+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:12:48.684679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235370+0000","last_change":"2024-05-10T10:42:49.230133+0000","last_active":"2024-05-10T10:42:51.235370+0000","last_peered":"2024-05-10T10:42:51.235370+0000","last_clean":"2024-05-10T10:42:51.235370+0000","last_became_active":"2024-05-10T10:42:49.229845+0000","last_became_peered":"2024-05-10T10:42:49.229845+0000","last_unstale":"2024-05-10T10:42:51.235370+0000","last_undegraded":"2024-05-10T10:42:51.235370+0000","last_fullsized":"2024-05-10T10:42:51.235370+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:15:37.432658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349892+0000","last_change":"2024-05-10T10:42:50.243857+0000","last_active":"2024-05-10T10:42:51.349892+0000","last_peered":"2024-05-10T10:42:51.349892+0000","last_clean":"2024-05-10T10:42:51.349892+0000","last_became_active":"2024-05-10T10:42:50.243568+0000","last_became_peered":"2024-05-10T10:42:50.243568+0000","last_unstale":"2024-05-10T10:42:51.349892+0000","last_undegraded":"2024-05-10T10:42:51.349892+0000","last_fullsized":"2024-05-10T10:42:51.349892+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:21.427949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349364+0000","last_change":"2024-05-10T10:42:49.235133+0000","last_active":"2024-05-10T10:42:51.349364+0000","last_peered":"2024-05-10T10:42:51.349364+0000","last_clean":"2024-05-10T10:42:51.349364+0000","last_became_active":"2024-05-10T10:42:49.234754+0000","last_became_peered":"2024-05-10T10:42:49.234754+0000","last_unstale":"2024-05-10T10:42:51.349364+0000","last_undegraded":"2024-05-10T10:42:51.349364+0000","last_fullsized":"2024-05-10T10:42:51.349364+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:13.240000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349936+0000","last_change":"2024-05-10T10:42:50.238996+0000","last_active":"2024-05-10T10:42:51.349936+0000","last_peered":"2024-05-10T10:42:51.349936+0000","last_clean":"2024-05-10T10:42:51.349936+0000","last_became_active":"2024-05-10T10:42:50.238717+0000","last_became_peered":"2024-05-10T10:42:50.238717+0000","last_unstale":"2024-05-10T10:42:51.349936+0000","last_undegraded":"2024-05-10T10:42:51.349936+0000","last_fullsized":"2024-05-10T10:42:51.349936+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:11.454848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350164+0000","last_change":"2024-05-10T10:42:49.228687+0000","last_active":"2024-05-10T10:42:51.350164+0000","last_peered":"2024-05-10T10:42:51.350164+0000","last_clean":"2024-05-10T10:42:51.350164+0000","last_became_active":"2024-05-10T10:42:49.228377+0000","last_became_peered":"2024-05-10T10:42:49.228377+0000","last_unstale":"2024-05-10T10:42:51.350164+0000","last_undegraded":"2024-05-10T10:42:51.350164+0000","last_fullsized":"2024-05-10T10:42:51.350164+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:33.500764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236752+0000","last_change":"2024-05-10T10:42:50.231397+0000","last_active":"2024-05-10T10:42:51.236752+0000","last_peered":"2024-05-10T10:42:51.236752+0000","last_clean":"2024-05-10T10:42:51.236752+0000","last_became_active":"2024-05-10T10:42:50.228202+0000","last_became_peered":"2024-05-10T10:42:50.228202+0000","last_unstale":"2024-05-10T10:42:51.236752+0000","last_undegraded":"2024-05-10T10:42:51.236752+0000","last_fullsized":"2024-05-10T10:42:51.236752+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:09:33.941842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236126+0000","last_change":"2024-05-10T10:42:49.224613+0000","last_active":"2024-05-10T10:42:51.236126+0000","last_peered":"2024-05-10T10:42:51.236126+0000","last_clean":"2024-05-10T10:42:51.236126+0000","last_became_active":"2024-05-10T10:42:49.223701+0000","last_became_peered":"2024-05-10T10:42:49.223701+0000","last_unstale":"2024-05-10T10:42:51.236126+0000","last_undegraded":"2024-05-10T10:42:51.236126+0000","last_fullsized":"2024-05-10T10:42:51.236126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:28:36.941117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350092+0000","last_change":"2024-05-10T10:42:50.244354+0000","last_active":"2024-05-10T10:42:51.350092+0000","last_peered":"2024-05-10T10:42:51.350092+0000","last_clean":"2024-05-10T10:42:51.350092+0000","last_became_active":"2024-05-10T10:42:50.244094+0000","last_became_peered":"2024-05-10T10:42:50.244094+0000","last_unstale":"2024-05-10T10:42:51.350092+0000","last_undegraded":"2024-05-10T10:42:51.350092+0000","last_fullsized":"2024-05-10T10:42:51.350092+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:29.045471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236599+0000","last_change":"2024-05-10T10:42:49.229887+0000","last_active":"2024-05-10T10:42:51.236599+0000","last_peered":"2024-05-10T10:42:51.236599+0000","last_clean":"2024-05-10T10:42:51.236599+0000","last_became_active":"2024-05-10T10:42:49.229581+0000","last_became_peered":"2024-05-10T10:42:49.229581+0000","last_unstale":"2024-05-10T10:42:51.236599+0000","last_undegraded":"2024-05-10T10:42:51.236599+0000","last_fullsized":"2024-05-10T10:42:51.236599+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:35:07.840789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235631+0000","last_change":"2024-05-10T10:42:50.227466+0000","last_active":"2024-05-10T10:42:51.235631+0000","last_peered":"2024-05-10T10:42:51.235631+0000","last_clean":"2024-05-10T10:42:51.235631+0000","last_became_active":"2024-05-10T10:42:50.227118+0000","last_became_peered":"2024-05-10T10:42:50.227118+0000","last_unstale":"2024-05-10T10:42:51.235631+0000","last_undegraded":"2024-05-10T10:42:51.235631+0000","last_fullsized":"2024-05-10T10:42:51.235631+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:46:39.070417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349843+0000","last_change":"2024-05-10T10:42:49.228264+0000","last_active":"2024-05-10T10:42:51.349843+0000","last_peered":"2024-05-10T10:42:51.349843+0000","last_clean":"2024-05-10T10:42:51.349843+0000","last_became_active":"2024-05-10T10:42:49.227923+0000","last_became_peered":"2024-05-10T10:42:49.227923+0000","last_unstale":"2024-05-10T10:42:51.349843+0000","last_undegraded":"2024-05-10T10:42:51.349843+0000","last_fullsized":"2024-05-10T10:42:51.349843+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:58.372409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349550+0000","last_change":"2024-05-10T10:42:50.248537+0000","last_active":"2024-05-10T10:42:51.349550+0000","last_peered":"2024-05-10T10:42:51.349550+0000","last_clean":"2024-05-10T10:42:51.349550+0000","last_became_active":"2024-05-10T10:42:50.248148+0000","last_became_peered":"2024-05-10T10:42:50.248148+0000","last_unstale":"2024-05-10T10:42:51.349550+0000","last_undegraded":"2024-05-10T10:42:51.349550+0000","last_fullsized":"2024-05-10T10:42:51.349550+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:29:33.695112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349457+0000","last_change":"2024-05-10T10:42:49.233462+0000","last_active":"2024-05-10T10:42:51.349457+0000","last_peered":"2024-05-10T10:42:51.349457+0000","last_clean":"2024-05-10T10:42:51.349457+0000","last_became_active":"2024-05-10T10:42:49.233188+0000","last_became_peered":"2024-05-10T10:42:49.233188+0000","last_unstale":"2024-05-10T10:42:51.349457+0000","last_undegraded":"2024-05-10T10:42:51.349457+0000","last_fullsized":"2024-05-10T10:42:51.349457+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:44:04.028657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349208+0000","last_change":"2024-05-10T10:42:50.237854+0000","last_active":"2024-05-10T10:42:51.349208+0000","last_peered":"2024-05-10T10:42:51.349208+0000","last_clean":"2024-05-10T10:42:51.349208+0000","last_became_active":"2024-05-10T10:42:50.237574+0000","last_became_peered":"2024-05-10T10:42:50.237574+0000","last_unstale":"2024-05-10T10:42:51.349208+0000","last_undegraded":"2024-05-10T10:42:51.349208+0000","last_fullsized":"2024-05-10T10:42:51.349208+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:59.374542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350293+0000","last_change":"2024-05-10T10:42:49.225663+0000","last_active":"2024-05-10T10:42:51.350293+0000","last_peered":"2024-05-10T10:42:51.350293+0000","last_clean":"2024-05-10T10:42:51.350293+0000","last_became_active":"2024-05-10T10:42:49.225183+0000","last_became_peered":"2024-05-10T10:42:49.225183+0000","last_unstale":"2024-05-10T10:42:51.350293+0000","last_undegraded":"2024-05-10T10:42:51.350293+0000","last_fullsized":"2024-05-10T10:42:51.350293+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:36:25.648258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.234157+0000","last_change":"2024-05-10T10:42:50.232720+0000","last_active":"2024-05-10T10:42:51.234157+0000","last_peered":"2024-05-10T10:42:51.234157+0000","last_clean":"2024-05-10T10:42:51.234157+0000","last_became_active":"2024-05-10T10:42:50.231581+0000","last_became_peered":"2024-05-10T10:42:50.231581+0000","last_unstale":"2024-05-10T10:42:51.234157+0000","last_undegraded":"2024-05-10T10:42:51.234157+0000","last_fullsized":"2024-05-10T10:42:51.234157+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:10:58.741822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349515+0000","last_change":"2024-05-10T10:42:49.226642+0000","last_active":"2024-05-10T10:42:51.349515+0000","last_peered":"2024-05-10T10:42:51.349515+0000","last_clean":"2024-05-10T10:42:51.349515+0000","last_became_active":"2024-05-10T10:42:49.226196+0000","last_became_peered":"2024-05-10T10:42:49.226196+0000","last_unstale":"2024-05-10T10:42:51.349515+0000","last_undegraded":"2024-05-10T10:42:51.349515+0000","last_fullsized":"2024-05-10T10:42:51.349515+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:52.796024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236510+0000","last_change":"2024-05-10T10:42:50.240028+0000","last_active":"2024-05-10T10:42:51.236510+0000","last_peered":"2024-05-10T10:42:51.236510+0000","last_clean":"2024-05-10T10:42:51.236510+0000","last_became_active":"2024-05-10T10:42:50.239727+0000","last_became_peered":"2024-05-10T10:42:50.239727+0000","last_unstale":"2024-05-10T10:42:51.236510+0000","last_undegraded":"2024-05-10T10:42:51.236510+0000","last_fullsized":"2024-05-10T10:42:51.236510+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:11:41.319783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349586+0000","last_change":"2024-05-10T10:42:49.235268+0000","last_active":"2024-05-10T10:42:51.349586+0000","last_peered":"2024-05-10T10:42:51.349586+0000","last_clean":"2024-05-10T10:42:51.349586+0000","last_became_active":"2024-05-10T10:42:49.234989+0000","last_became_peered":"2024-05-10T10:42:49.234989+0000","last_unstale":"2024-05-10T10:42:51.349586+0000","last_undegraded":"2024-05-10T10:42:51.349586+0000","last_fullsized":"2024-05-10T10:42:51.349586+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:56.778602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350067+0000","last_change":"2024-05-10T10:42:50.236598+0000","last_active":"2024-05-10T10:42:51.350067+0000","last_peered":"2024-05-10T10:42:51.350067+0000","last_clean":"2024-05-10T10:42:51.350067+0000","last_became_active":"2024-05-10T10:42:50.236110+0000","last_became_peered":"2024-05-10T10:42:50.236110+0000","last_unstale":"2024-05-10T10:42:51.350067+0000","last_undegraded":"2024-05-10T10:42:51.350067+0000","last_fullsized":"2024-05-10T10:42:51.350067+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:21.427949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.234314+0000","last_change":"2024-05-10T10:42:49.231721+0000","last_active":"2024-05-10T10:42:51.234314+0000","last_peered":"2024-05-10T10:42:51.234314+0000","last_clean":"2024-05-10T10:42:51.234314+0000","last_became_active":"2024-05-10T10:42:49.231451+0000","last_became_peered":"2024-05-10T10:42:49.231451+0000","last_unstale":"2024-05-10T10:42:51.234314+0000","last_undegraded":"2024-05-10T10:42:51.234314+0000","last_fullsized":"2024-05-10T10:42:51.234314+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:31:47.239226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350346+0000","last_change":"2024-05-10T10:42:50.239152+0000","last_active":"2024-05-10T10:42:51.350346+0000","last_peered":"2024-05-10T10:42:51.350346+0000","last_clean":"2024-05-10T10:42:51.350346+0000","last_became_active":"2024-05-10T10:42:50.238725+0000","last_became_peered":"2024-05-10T10:42:50.238725+0000","last_unstale":"2024-05-10T10:42:51.350346+0000","last_undegraded":"2024-05-10T10:42:51.350346+0000","last_fullsized":"2024-05-10T10:42:51.350346+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:15:10.589628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235838+0000","last_change":"2024-05-10T10:42:49.228198+0000","last_active":"2024-05-10T10:42:51.235838+0000","last_peered":"2024-05-10T10:42:51.235838+0000","last_clean":"2024-05-10T10:42:51.235838+0000","last_became_active":"2024-05-10T10:42:49.227902+0000","last_became_peered":"2024-05-10T10:42:49.227902+0000","last_unstale":"2024-05-10T10:42:51.235838+0000","last_undegraded":"2024-05-10T10:42:51.235838+0000","last_fullsized":"2024-05-10T10:42:51.235838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T17:07:44.270262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235723+0000","last_change":"2024-05-10T10:42:50.233463+0000","last_active":"2024-05-10T10:42:51.235723+0000","last_peered":"2024-05-10T10:42:51.235723+0000","last_clean":"2024-05-10T10:42:51.235723+0000","last_became_active":"2024-05-10T10:42:50.233178+0000","last_became_peered":"2024-05-10T10:42:50.233178+0000","last_unstale":"2024-05-10T10:42:51.235723+0000","last_undegraded":"2024-05-10T10:42:51.235723+0000","last_fullsized":"2024-05-10T10:42:51.235723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:39:33.675525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235502+0000","last_change":"2024-05-10T10:42:49.234665+0000","last_active":"2024-05-10T10:42:51.235502+0000","last_peered":"2024-05-10T10:42:51.235502+0000","last_clean":"2024-05-10T10:42:51.235502+0000","last_became_active":"2024-05-10T10:42:49.234235+0000","last_became_peered":"2024-05-10T10:42:49.234235+0000","last_unstale":"2024-05-10T10:42:51.235502+0000","last_undegraded":"2024-05-10T10:42:51.235502+0000","last_fullsized":"2024-05-10T10:42:51.235502+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:58.372409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349159+0000","last_change":"2024-05-10T10:42:50.239279+0000","last_active":"2024-05-10T10:42:51.349159+0000","last_peered":"2024-05-10T10:42:51.349159+0000","last_clean":"2024-05-10T10:42:51.349159+0000","last_became_active":"2024-05-10T10:42:50.238994+0000","last_became_peered":"2024-05-10T10:42:50.238994+0000","last_unstale":"2024-05-10T10:42:51.349159+0000","last_undegraded":"2024-05-10T10:42:51.349159+0000","last_fullsized":"2024-05-10T10:42:51.349159+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:03.783211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350144+0000","last_change":"2024-05-10T10:42:49.234410+0000","last_active":"2024-05-10T10:42:51.350144+0000","last_peered":"2024-05-10T10:42:51.350144+0000","last_clean":"2024-05-10T10:42:51.350144+0000","last_became_active":"2024-05-10T10:42:49.234125+0000","last_became_peered":"2024-05-10T10:42:49.234125+0000","last_unstale":"2024-05-10T10:42:51.350144+0000","last_undegraded":"2024-05-10T10:42:51.350144+0000","last_fullsized":"2024-05-10T10:42:51.350144+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:15:37.486795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349315+0000","last_change":"2024-05-10T10:42:50.233392+0000","last_active":"2024-05-10T10:42:51.349315+0000","last_peered":"2024-05-10T10:42:51.349315+0000","last_clean":"2024-05-10T10:42:51.349315+0000","last_became_active":"2024-05-10T10:42:50.233083+0000","last_became_peered":"2024-05-10T10:42:50.233083+0000","last_unstale":"2024-05-10T10:42:51.349315+0000","last_undegraded":"2024-05-10T10:42:51.349315+0000","last_fullsized":"2024-05-10T10:42:51.349315+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:21:57.210515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235753+0000","last_change":"2024-05-10T10:42:49.228871+0000","last_active":"2024-05-10T10:42:51.235753+0000","last_peered":"2024-05-10T10:42:51.235753+0000","last_clean":"2024-05-10T10:42:51.235753+0000","last_became_active":"2024-05-10T10:42:49.228443+0000","last_became_peered":"2024-05-10T10:42:49.228443+0000","last_unstale":"2024-05-10T10:42:51.235753+0000","last_undegraded":"2024-05-10T10:42:51.235753+0000","last_fullsized":"2024-05-10T10:42:51.235753+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T20:36:25.648258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350345+0000","last_change":"2024-05-10T10:42:50.243689+0000","last_active":"2024-05-10T10:42:51.350345+0000","last_peered":"2024-05-10T10:42:51.350345+0000","last_clean":"2024-05-10T10:42:51.350345+0000","last_became_active":"2024-05-10T10:42:50.243424+0000","last_became_peered":"2024-05-10T10:42:50.243424+0000","last_unstale":"2024-05-10T10:42:51.350345+0000","last_undegraded":"2024-05-10T10:42:51.350345+0000","last_fullsized":"2024-05-10T10:42:51.350345+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:58.397989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349607+0000","last_change":"2024-05-10T10:42:49.235791+0000","last_active":"2024-05-10T10:42:51.349607+0000","last_peered":"2024-05-10T10:42:51.349607+0000","last_clean":"2024-05-10T10:42:51.349607+0000","last_became_active":"2024-05-10T10:42:49.235479+0000","last_became_peered":"2024-05-10T10:42:49.235479+0000","last_unstale":"2024-05-10T10:42:51.349607+0000","last_undegraded":"2024-05-10T10:42:51.349607+0000","last_fullsized":"2024-05-10T10:42:51.349607+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:28.321788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235431+0000","last_change":"2024-05-10T10:42:50.225070+0000","last_active":"2024-05-10T10:42:51.235431+0000","last_peered":"2024-05-10T10:42:51.235431+0000","last_clean":"2024-05-10T10:42:51.235431+0000","last_became_active":"2024-05-10T10:42:50.224790+0000","last_became_peered":"2024-05-10T10:42:50.224790+0000","last_unstale":"2024-05-10T10:42:51.235431+0000","last_undegraded":"2024-05-10T10:42:51.235431+0000","last_fullsized":"2024-05-10T10:42:51.235431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:28:27.898696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235010+0000","last_change":"2024-05-10T10:42:49.227802+0000","last_active":"2024-05-10T10:42:51.235010+0000","last_peered":"2024-05-10T10:42:51.235010+0000","last_clean":"2024-05-10T10:42:51.235010+0000","last_became_active":"2024-05-10T10:42:49.227510+0000","last_became_peered":"2024-05-10T10:42:49.227510+0000","last_unstale":"2024-05-10T10:42:51.235010+0000","last_undegraded":"2024-05-10T10:42:51.235010+0000","last_fullsized":"2024-05-10T10:42:51.235010+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:20.425815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349642+0000","last_change":"2024-05-10T10:42:50.241563+0000","last_active":"2024-05-10T10:42:51.349642+0000","last_peered":"2024-05-10T10:42:51.349642+0000","last_clean":"2024-05-10T10:42:51.349642+0000","last_became_active":"2024-05-10T10:42:50.241222+0000","last_became_peered":"2024-05-10T10:42:50.241222+0000","last_unstale":"2024-05-10T10:42:51.349642+0000","last_undegraded":"2024-05-10T10:42:51.349642+0000","last_fullsized":"2024-05-10T10:42:51.349642+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:05:38.167541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350375+0000","last_change":"2024-05-10T10:42:49.233357+0000","last_active":"2024-05-10T10:42:51.350375+0000","last_peered":"2024-05-10T10:42:51.350375+0000","last_clean":"2024-05-10T10:42:51.350375+0000","last_became_active":"2024-05-10T10:42:49.233016+0000","last_became_peered":"2024-05-10T10:42:49.233016+0000","last_unstale":"2024-05-10T10:42:51.350375+0000","last_undegraded":"2024-05-10T10:42:51.350375+0000","last_fullsized":"2024-05-10T10:42:51.350375+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:30:47.657643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349150+0000","last_change":"2024-05-10T10:42:50.240959+0000","last_active":"2024-05-10T10:42:51.349150+0000","last_peered":"2024-05-10T10:42:51.349150+0000","last_clean":"2024-05-10T10:42:51.349150+0000","last_became_active":"2024-05-10T10:42:50.240674+0000","last_became_peered":"2024-05-10T10:42:50.240674+0000","last_unstale":"2024-05-10T10:42:51.349150+0000","last_undegraded":"2024-05-10T10:42:51.349150+0000","last_fullsized":"2024-05-10T10:42:51.349150+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:29.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350140+0000","last_change":"2024-05-10T10:42:49.234499+0000","last_active":"2024-05-10T10:42:51.350140+0000","last_peered":"2024-05-10T10:42:51.350140+0000","last_clean":"2024-05-10T10:42:51.350140+0000","last_became_active":"2024-05-10T10:42:49.234130+0000","last_became_peered":"2024-05-10T10:42:49.234130+0000","last_unstale":"2024-05-10T10:42:51.350140+0000","last_undegraded":"2024-05-10T10:42:51.350140+0000","last_fullsized":"2024-05-10T10:42:51.350140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:56.778602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235854+0000","last_change":"2024-05-10T10:42:50.227887+0000","last_active":"2024-05-10T10:42:51.235854+0000","last_peered":"2024-05-10T10:42:51.235854+0000","last_clean":"2024-05-10T10:42:51.235854+0000","last_became_active":"2024-05-10T10:42:50.227591+0000","last_became_peered":"2024-05-10T10:42:50.227591+0000","last_unstale":"2024-05-10T10:42:51.235854+0000","last_undegraded":"2024-05-10T10:42:51.235854+0000","last_fullsized":"2024-05-10T10:42:51.235854+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:07:43.746287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350267+0000","last_change":"2024-05-10T10:42:49.229012+0000","last_active":"2024-05-10T10:42:51.350267+0000","last_peered":"2024-05-10T10:42:51.350267+0000","last_clean":"2024-05-10T10:42:51.350267+0000","last_became_active":"2024-05-10T10:42:49.228561+0000","last_became_peered":"2024-05-10T10:42:49.228561+0000","last_unstale":"2024-05-10T10:42:51.350267+0000","last_undegraded":"2024-05-10T10:42:51.350267+0000","last_fullsized":"2024-05-10T10:42:51.350267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:51:51.580978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.234311+0000","last_change":"2024-05-10T10:42:50.230423+0000","last_active":"2024-05-10T10:42:51.234311+0000","last_peered":"2024-05-10T10:42:51.234311+0000","last_clean":"2024-05-10T10:42:51.234311+0000","last_became_active":"2024-05-10T10:42:50.230151+0000","last_became_peered":"2024-05-10T10:42:50.230151+0000","last_unstale":"2024-05-10T10:42:51.234311+0000","last_undegraded":"2024-05-10T10:42:51.234311+0000","last_fullsized":"2024-05-10T10:42:51.234311+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:36.952065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349656+0000","last_change":"2024-05-10T10:42:49.234279+0000","last_active":"2024-05-10T10:42:51.349656+0000","last_peered":"2024-05-10T10:42:51.349656+0000","last_clean":"2024-05-10T10:42:51.349656+0000","last_became_active":"2024-05-10T10:42:49.233996+0000","last_became_peered":"2024-05-10T10:42:49.233996+0000","last_unstale":"2024-05-10T10:42:51.349656+0000","last_undegraded":"2024-05-10T10:42:51.349656+0000","last_fullsized":"2024-05-10T10:42:51.349656+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:28:37.834557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349310+0000","last_change":"2024-05-10T10:42:50.236955+0000","last_active":"2024-05-10T10:42:51.349310+0000","last_peered":"2024-05-10T10:42:51.349310+0000","last_clean":"2024-05-10T10:42:51.349310+0000","last_became_active":"2024-05-10T10:42:50.236696+0000","last_became_peered":"2024-05-10T10:42:50.236696+0000","last_unstale":"2024-05-10T10:42:51.349310+0000","last_undegraded":"2024-05-10T10:42:51.349310+0000","last_fullsized":"2024-05-10T10:42:51.349310+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:17:48.807245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349861+0000","last_change":"2024-05-10T10:42:49.230118+0000","last_active":"2024-05-10T10:42:51.349861+0000","last_peered":"2024-05-10T10:42:51.349861+0000","last_clean":"2024-05-10T10:42:51.349861+0000","last_became_active":"2024-05-10T10:42:49.229865+0000","last_became_peered":"2024-05-10T10:42:49.229865+0000","last_unstale":"2024-05-10T10:42:51.349861+0000","last_undegraded":"2024-05-10T10:42:51.349861+0000","last_fullsized":"2024-05-10T10:42:51.349861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T22:31:39.278453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349456+0000","last_change":"2024-05-10T10:42:50.244986+0000","last_active":"2024-05-10T10:42:51.349456+0000","last_peered":"2024-05-10T10:42:51.349456+0000","last_clean":"2024-05-10T10:42:51.349456+0000","last_became_active":"2024-05-10T10:42:50.244699+0000","last_became_peered":"2024-05-10T10:42:50.244699+0000","last_unstale":"2024-05-10T10:42:51.349456+0000","last_undegraded":"2024-05-10T10:42:51.349456+0000","last_fullsized":"2024-05-10T10:42:51.349456+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:44.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349752+0000","last_change":"2024-05-10T10:42:49.226517+0000","last_active":"2024-05-10T10:42:51.349752+0000","last_peered":"2024-05-10T10:42:51.349752+0000","last_clean":"2024-05-10T10:42:51.349752+0000","last_became_active":"2024-05-10T10:42:49.226266+0000","last_became_peered":"2024-05-10T10:42:49.226266+0000","last_unstale":"2024-05-10T10:42:51.349752+0000","last_undegraded":"2024-05-10T10:42:51.349752+0000","last_fullsized":"2024-05-10T10:42:51.349752+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:55:33.500764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235686+0000","last_change":"2024-05-10T10:42:50.233005+0000","last_active":"2024-05-10T10:42:51.235686+0000","last_peered":"2024-05-10T10:42:51.235686+0000","last_clean":"2024-05-10T10:42:51.235686+0000","last_became_active":"2024-05-10T10:42:50.232751+0000","last_became_peered":"2024-05-10T10:42:50.232751+0000","last_unstale":"2024-05-10T10:42:51.235686+0000","last_undegraded":"2024-05-10T10:42:51.235686+0000","last_fullsized":"2024-05-10T10:42:51.235686+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:29:44.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349246+0000","last_change":"2024-05-10T10:42:49.233728+0000","last_active":"2024-05-10T10:42:51.349246+0000","last_peered":"2024-05-10T10:42:51.349246+0000","last_clean":"2024-05-10T10:42:51.349246+0000","last_became_active":"2024-05-10T10:42:49.230416+0000","last_became_peered":"2024-05-10T10:42:49.230416+0000","last_unstale":"2024-05-10T10:42:51.349246+0000","last_undegraded":"2024-05-10T10:42:51.349246+0000","last_fullsized":"2024-05-10T10:42:51.349246+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T21:27:22.007461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.350038+0000","last_change":"2024-05-10T10:42:50.228773+0000","last_active":"2024-05-10T10:42:51.350038+0000","last_peered":"2024-05-10T10:42:51.350038+0000","last_clean":"2024-05-10T10:42:51.350038+0000","last_became_active":"2024-05-10T10:42:50.228468+0000","last_became_peered":"2024-05-10T10:42:50.228468+0000","last_unstale":"2024-05-10T10:42:51.350038+0000","last_undegraded":"2024-05-10T10:42:51.350038+0000","last_fullsized":"2024-05-10T10:42:51.350038+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T18:28:27.898696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235136+0000","last_change":"2024-05-10T10:42:49.231268+0000","last_active":"2024-05-10T10:42:51.235136+0000","last_peered":"2024-05-10T10:42:51.235136+0000","last_clean":"2024-05-10T10:42:51.235136+0000","last_became_active":"2024-05-10T10:42:49.230932+0000","last_became_peered":"2024-05-10T10:42:49.230932+0000","last_unstale":"2024-05-10T10:42:51.235136+0000","last_undegraded":"2024-05-10T10:42:51.235136+0000","last_fullsized":"2024-05-10T10:42:51.235136+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T19:38:17.339074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.236407+0000","last_change":"2024-05-10T10:42:50.229783+0000","last_active":"2024-05-10T10:42:51.236407+0000","last_peered":"2024-05-10T10:42:51.236407+0000","last_clean":"2024-05-10T10:42:51.236407+0000","last_became_active":"2024-05-10T10:42:50.229548+0000","last_became_peered":"2024-05-10T10:42:50.229548+0000","last_unstale":"2024-05-10T10:42:51.236407+0000","last_undegraded":"2024-05-10T10:42:51.236407+0000","last_fullsized":"2024-05-10T10:42:51.236407+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:49.203714+0000","last_clean_scrub_stamp":"2024-05-10T10:42:49.203714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:12:09.166913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.349574+0000","last_change":"2024-05-10T10:42:49.224769+0000","last_active":"2024-05-10T10:42:51.349574+0000","last_peered":"2024-05-10T10:42:51.349574+0000","last_clean":"2024-05-10T10:42:51.349574+0000","last_became_active":"2024-05-10T10:42:49.223891+0000","last_became_peered":"2024-05-10T10:42:49.223891+0000","last_unstale":"2024-05-10T10:42:51.349574+0000","last_undegraded":"2024-05-10T10:42:51.349574+0000","last_fullsized":"2024-05-10T10:42:51.349574+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:42:48.201581+0000","last_clean_scrub_stamp":"2024-05-10T10:42:48.201581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:10.452714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":310,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-10T10:42:51.235306+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:42:51.235306+0000","last_peered":"2024-05-10T10:42:51.235306+0000","last_clean":"2024-05-10T10:42:51.235306+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:42:51.235306+0000","last_undegraded":"2024-05-10T10:42:51.235306+0000","last_fullsized":"2024-05-10T10:42:51.235306+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1275932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1474560,"data_stored":685370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":684948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":686400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":686434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1470464,"data_stored":684788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1474560,"data_stored":684810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2068480,"data_stored":1275178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:42:54.648 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:42:54.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T10:42:54.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.795+0000 7fcf6a9af640 1 Processor -- start 2024-05-10T10:42:54.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.795+0000 7fcf6a9af640 1 -- start start 2024-05-10T10:42:54.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf6a9af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 0x7fcf640b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf6a9af640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6406e720 con 0x7fcf640cffd0 2024-05-10T10:42:54.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf6a9af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6406de80 con 0x7fcf640cf3e0 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf6a9af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6406e160 con 0x7fcf640cd500 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf5bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcf640cffd0 0x7fcf640d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49224/0 (socket says 172.21.15.53:49224) 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf5bfff640 1 -- 172.21.15.53:0/2818210187 learned_addr learned my addr 172.21.15.53:0/2818210187 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923335409 0 0) 0x7fcf6406e720 con 0x7fcf640cffd0 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf50003520 con 0x7fcf640cffd0 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.796+0000 7fcf68f4c640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 0x7fcf640b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1385934783 0 0) 0x7fcf50003520 con 0x7fcf640cffd0 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 msgr2=0x7fcf640b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 0x7fcf640b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 >> v1:172.21.15.135:6789/0 conn(0x7fcf640cf3e0 legacy=0x7fcf640cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6406e440 con 0x7fcf640cffd0 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fcf4c001670 con 0x7fcf640cffd0 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 >> v1:172.21.15.53:6789/0 conn(0x7fcf640cffd0 legacy=0x7fcf640d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 0x7fcf500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf50003f60 0x7fcf50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6406e160 con 0x7fcf50004d40 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5000baa0 con 0x7fcf50003f60 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5000bca0 con 0x7fcf640cd500 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf68f4c640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf63fff640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 0x7fcf500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf5bfff640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf50003f60 0x7fcf50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf68f4c640 1 -- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 msgr2=0x7fcf500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf68f4c640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640cd500 0x7fcf500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf68f4c640 1 -- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf50003f60 msgr2=0x7fcf50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf68f4c640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf50003f60 0x7fcf50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.797+0000 7fcf68f4c640 1 -- 172.21.15.53:0/2818210187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf50003520 con 0x7fcf50004d40 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.798+0000 7fcf68f4c640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf500051c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fcf64073ea0 tx=0x7fcf50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7a9b2d64db8da7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.798+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c037070 con 0x7fcf50004d40 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.798+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5c02e050 con 0x7fcf50004d40 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.798+0000 7fcf637fe640 1 -- 172.21.15.53:0/2818210187 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c009c10 con 0x7fcf50004d40 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.799+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2818210187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 msgr2=0x7fcf500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.799+0000 7fcf6a9af640 1 --2- 172.21.15.53:0/2818210187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf500051c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fcf64073ea0 tx=0x7fcf50005830 comp rx=0 tx=0).stop 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.799+0000 7fcf68f4c640 1 -- 172.21.15.53:0/2818210187 reap_dead start 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.799+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2818210187 shutdown_connections 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.799+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2818210187 >> 172.21.15.53:0/2818210187 conn(0x7fcf640d2eb0 msgr2=0x7fcf640d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.799+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2818210187 shutdown_connections 2024-05-10T10:42:54.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.799+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2818210187 wait complete. 2024-05-10T10:42:54.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.800+0000 7fcf6a9af640 1 Processor -- start 2024-05-10T10:42:54.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.800+0000 7fcf6a9af640 1 -- start start 2024-05-10T10:42:54.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf64155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640b50a0 0x7fcf640b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf640b66c0 0x7fcf640b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf640b3de0 con 0x7fcf50004d40 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf640b4510 con 0x7fcf640b66c0 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcf640b3f60 con 0x7fcf640b50a0 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf64155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf64155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45488/0 (socket says 172.21.15.53:45488) 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 -- 172.21.15.53:0/2254431126 learned_addr learned my addr 172.21.15.53:0/2254431126 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf5bfff640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640b50a0 0x7fcf640b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf68f4c640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf640b66c0 0x7fcf640b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 -- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640b50a0 msgr2=0x7fcf640b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640b50a0 0x7fcf640b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 -- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf640b66c0 msgr2=0x7fcf640b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf640b66c0 0x7fcf640b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 -- 172.21.15.53:0/2254431126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf640b5ef0 con 0x7fcf50004d40 2024-05-10T10:42:54.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf63fff640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf64155d10 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400f820 tx=0x7fcf5400fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf617fa640 1 -- 172.21.15.53:0/2254431126 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf540050d0 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf617fa640 1 -- 172.21.15.53:0/2254431126 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf54018070 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf64156250 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf617fa640 1 -- 172.21.15.53:0/2254431126 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400d730 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.801+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf640cb100 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.802+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6406a220 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.803+0000 7fcf617fa640 1 -- 172.21.15.53:0/2254431126 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcf5400d8d0 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.804+0000 7fcf617fa640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcf44076300 0x7fcf440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.804+0000 7fcf617fa640 1 -- 172.21.15.53:0/2254431126 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fcf540974f0 con 0x7fcf50004d40 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.804+0000 7fcf5bfff640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcf44076300 0x7fcf440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:54.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.804+0000 7fcf5bfff640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcf44076300 0x7fcf440787c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcf640b86a0 tx=0x7fcf4c001190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:54.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:54.808+0000 7fcf617fa640 1 -- 172.21.15.53:0/2254431126 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fcf54060cb0 con 0x7fcf50004d40 2024-05-10T10:42:55.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.074+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcf64162700 con 0x7fcf50004d40 2024-05-10T10:42:55.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.076+0000 7fcf617fa640 1 -- 172.21.15.53:0/2254431126 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fcf540605b0 con 0x7fcf50004d40 2024-05-10T10:42:55.077 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-05-10T10:42:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9z1mxtPVGhAAeroXMC+2kXTpTuWGlIJzfg== 2024-05-10T10:42:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2024-05-10T10:42:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2024-05-10T10:42:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-10T10:42:55.078 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2024-05-10T10:42:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcf44076300 msgr2=0x7fcf440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcf44076300 0x7fcf440787c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcf640b86a0 tx=0x7fcf4c001190 comp rx=0 tx=0).stop 2024-05-10T10:42:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 msgr2=0x7fcf64155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf64155d10 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400f820 tx=0x7fcf5400fc20 comp rx=0 tx=0).stop 2024-05-10T10:42:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 shutdown_connections 2024-05-10T10:42:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcf44076300 0x7fcf440787c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf640b66c0 0x7fcf640b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf640b50a0 0x7fcf640b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 --2- 172.21.15.53:0/2254431126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50004d40 0x7fcf64155d10 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 >> 172.21.15.53:0/2254431126 conn(0x7fcf640d2eb0 msgr2=0x7fcf640cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 shutdown_connections 2024-05-10T10:42:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.078+0000 7fcf6a9af640 1 -- 172.21.15.53:0/2254431126 wait complete. 2024-05-10T10:42:55.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:42:55.194 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.193+0000 7f821c346640 1 Processor -- start 2024-05-10T10:42:55.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.193+0000 7f821c346640 1 -- start start 2024-05-10T10:42:55.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821c346640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.195 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821c346640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82140c65f0 0x7f8214160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821c346640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82140c6a30 con 0x7f8214161010 2024-05-10T10:42:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821c346640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8214161480 con 0x7f82140c65f0 2024-05-10T10:42:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821c346640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f82140cd090 con 0x7f821415ce60 2024-05-10T10:42:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f82198e1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82140c65f0 0x7f8214160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821a0e2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821a8e3640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8214161010 0x7f82140add50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:40028/0 (socket says 172.21.15.53:40028) 2024-05-10T10:42:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821a8e3640 1 -- 172.21.15.53:0/551329252 learned_addr learned my addr 172.21.15.53:0/551329252 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f82190e0640 1 -- 172.21.15.53:0/551329252 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692400996 0 0) 0x7f82140c6a30 con 0x7f8214161010 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.194+0000 7f821a0e2640 1 -- 172.21.15.53:0/551329252 >> v2:172.21.15.135:3300/0 conn(0x7f82140c65f0 msgr2=0x7f8214160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.195+0000 7f821a0e2640 1 --2- 172.21.15.53:0/551329252 >> v2:172.21.15.135:3300/0 conn(0x7f82140c65f0 0x7f8214160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.195+0000 7f821a0e2640 1 -- 172.21.15.53:0/551329252 >> v1:172.21.15.135:6789/0 conn(0x7f8214161010 legacy=0x7f82140add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.195+0000 7f821a0e2640 1 -- 172.21.15.53:0/551329252 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82140cd370 con 0x7f821415ce60 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.195+0000 7f821a0e2640 1 --2- 172.21.15.53:0/551329252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140cb300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82100097a0 tx=0x7f821002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32aee7cfa72ee7c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f82190e0640 1 -- 172.21.15.53:0/551329252 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210037070 con 0x7f821415ce60 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f82190e0640 1 -- 172.21.15.53:0/551329252 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821002ae50 con 0x7f821415ce60 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f82190e0640 1 -- 172.21.15.53:0/551329252 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210032660 con 0x7f821415ce60 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 -- 172.21.15.53:0/551329252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 msgr2=0x7f82140cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 --2- 172.21.15.53:0/551329252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140cb300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82100097a0 tx=0x7f821002a100 comp rx=0 tx=0).stop 2024-05-10T10:42:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 -- 172.21.15.53:0/551329252 shutdown_connections 2024-05-10T10:42:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 --2- 172.21.15.53:0/551329252 >> v2:172.21.15.135:3300/0 conn(0x7f82140c65f0 0x7f8214160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 --2- 172.21.15.53:0/551329252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140cb300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 -- 172.21.15.53:0/551329252 >> 172.21.15.53:0/551329252 conn(0x7f82140b2900 msgr2=0x7f82140b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 -- 172.21.15.53:0/551329252 shutdown_connections 2024-05-10T10:42:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.196+0000 7f821c346640 1 -- 172.21.15.53:0/551329252 wait complete. 2024-05-10T10:42:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.197+0000 7f821c346640 1 Processor -- start 2024-05-10T10:42:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.197+0000 7f821c346640 1 -- start start 2024-05-10T10:42:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821c346640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140c65f0 0x7f82140b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821c346640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821c346640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8214161010 0x7f82141f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821c346640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82140ccd60 con 0x7f8214161010 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821c346640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82140cd930 con 0x7f82140c65f0 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821c346640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f821415e250 con 0x7f821415ce60 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821a8e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8214161010 0x7f82141f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821a8e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8214161010 0x7f82141f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45496/0 (socket says 172.21.15.53:45496) 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821a8e3640 1 -- 172.21.15.53:0/1201248104 learned_addr learned my addr 172.21.15.53:0/1201248104 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f821a0e2640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140c65f0 0x7f82140b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.198+0000 7f82198e1640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821a8e3640 1 -- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 msgr2=0x7f82140b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821a8e3640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821a8e3640 1 -- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140c65f0 msgr2=0x7f82140b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821a8e3640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140c65f0 0x7f82140b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821a8e3640 1 -- 172.21.15.53:0/1201248104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82140b5e00 con 0x7f8214161010 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821a8e3640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8214161010 0x7f82141f2b80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c7b0 tx=0x7f820c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f820affd640 1 -- 172.21.15.53:0/1201248104 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c018070 con 0x7f8214161010 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f820affd640 1 -- 172.21.15.53:0/1201248104 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820c00a640 con 0x7f8214161010 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82140b4fc0 con 0x7f8214161010 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f820affd640 1 -- 172.21.15.53:0/1201248104 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c00aa30 con 0x7f8214161010 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.199+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821409bc70 con 0x7f8214161010 2024-05-10T10:42:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.200+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821406a220 con 0x7f8214161010 2024-05-10T10:42:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.201+0000 7f820affd640 1 -- 172.21.15.53:0/1201248104 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f820c01b070 con 0x7f8214161010 2024-05-10T10:42:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.201+0000 7f820affd640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f81f80762d0 0x7f81f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.201+0000 7f820affd640 1 -- 172.21.15.53:0/1201248104 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f820c096c50 con 0x7f8214161010 2024-05-10T10:42:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.201+0000 7f821a0e2640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f81f80762d0 0x7f81f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.202+0000 7f821a0e2640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f81f80762d0 0x7f81f8078790 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8210034000 tx=0x7f821002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.205+0000 7f820affd640 1 -- 172.21.15.53:0/1201248104 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f820c09c020 con 0x7f8214161010 2024-05-10T10:42:55.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.434+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82140c7300 con 0x7f8214161010 2024-05-10T10:42:55.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.434+0000 7f820affd640 1 -- 172.21.15.53:0/1201248104 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5174 (secure 0 0 0) 0x7f820c05fe60 con 0x7f8214161010 2024-05-10T10:42:55.436 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:42:55.436 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8025,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/38461894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":38461894},{"type":"v1","addr":"172.21.15.53:6834","nonce":38461894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:42:52.609796+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8034":{"gid":8034,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/776479349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":776479349},{"type":"v1","addr":"172.21.15.135:6837","nonce":776479349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8034,"qdb_cluster":[8034]},"id":8}]} 2024-05-10T10:42:55.437 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 98 2024-05-10T10:42:55.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f81f80762d0 msgr2=0x7f81f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f81f80762d0 0x7f81f8078790 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8210034000 tx=0x7f821002e000 comp rx=0 tx=0).stop 2024-05-10T10:42:55.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8214161010 msgr2=0x7f82141f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8214161010 0x7f82141f2b80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c7b0 tx=0x7f820c00cc70 comp rx=0 tx=0).stop 2024-05-10T10:42:55.438 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 shutdown_connections 2024-05-10T10:42:55.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f81f80762d0 0x7f81f8078790 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8214161010 0x7f82141f2b80 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f821415ce60 0x7f82140b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 --2- 172.21.15.53:0/1201248104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140c65f0 0x7f82140b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 >> 172.21.15.53:0/1201248104 conn(0x7f82140b2900 msgr2=0x7f821415e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:55.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 shutdown_connections 2024-05-10T10:42:55.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.436+0000 7f821c346640 1 -- 172.21.15.53:0/1201248104 wait complete. 2024-05-10T10:42:55.847 INFO:tasks.ceph.mds.d.smithi135.stderr:2024-05-10T10:42:55.846+0000 7f868b251640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:42:55.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:42:49.598991+0000', 'modified': '2024-05-10T10:42:52.609796+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8034}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8034': {'gid': 8034, 'name': 'd', 'rank': 0, 'incarnation': 94, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6837/776479349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6835', 'nonce': 776479349}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 776479349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8034, 'qdb_cluster': [8034]} 2024-05-10T10:42:55.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:42:55.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-10T10:42:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.959+0000 7fc4717f7640 1 Processor -- start 2024-05-10T10:42:55.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.959+0000 7fc4717f7640 1 -- start start 2024-05-10T10:42:55.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc4717f7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc46c15d290 0x7fc46c071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc4717f7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc4717f7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46c072330 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc4717f7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc46c071d70 con 0x7fc46c15d290 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc4717f7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc46c072050 con 0x7fc46c071880 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc46affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc46c15d290 0x7fc46c071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc46affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc46c15d290 0x7fc46c071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45500/0 (socket says 172.21.15.53:45500) 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.960+0000 7fc46affd640 1 -- 172.21.15.53:0/3240781761 learned_addr learned my addr 172.21.15.53:0/3240781761 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.961+0000 7fc46affd640 1 -- 172.21.15.53:0/3240781761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 msgr2=0x7fc46c0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.961+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/3240781761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.961+0000 7fc46affd640 1 --2- 172.21.15.53:0/3240781761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0d2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.961+0000 7fc46affd640 1 -- 172.21.15.53:0/3240781761 >> v1:172.21.15.135:6789/0 conn(0x7fc46c0d1aa0 legacy=0x7fc46c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.961+0000 7fc46affd640 1 -- 172.21.15.53:0/3240781761 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c0cda70 con 0x7fc46c15d290 2024-05-10T10:42:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.961+0000 7fc46affd640 1 --2- 172.21.15.53:0/3240781761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c15d290 0x7fc46c071340 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009e50 tx=0x7fc45c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426c559c2320ddd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc469ffb640 1 -- 172.21.15.53:0/3240781761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c036070 con 0x7fc46c15d290 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc469ffb640 1 -- 172.21.15.53:0/3240781761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c031600 con 0x7fc46c15d290 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc469ffb640 1 -- 172.21.15.53:0/3240781761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0319d0 con 0x7fc46c15d290 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 -- 172.21.15.53:0/3240781761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c15d290 msgr2=0x7fc46c071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 --2- 172.21.15.53:0/3240781761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c15d290 0x7fc46c071340 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009e50 tx=0x7fc45c02a6c0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 -- 172.21.15.53:0/3240781761 shutdown_connections 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 --2- 172.21.15.53:0/3240781761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 --2- 172.21.15.53:0/3240781761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c15d290 0x7fc46c071340 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 -- 172.21.15.53:0/3240781761 >> 172.21.15.53:0/3240781761 conn(0x7fc46c09ba20 msgr2=0x7fc46c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:55.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 -- 172.21.15.53:0/3240781761 shutdown_connections 2024-05-10T10:42:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.962+0000 7fc4717f7640 1 -- 172.21.15.53:0/3240781761 wait complete. 2024-05-10T10:42:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.963+0000 7fc4717f7640 1 Processor -- start 2024-05-10T10:42:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc4717f7640 1 -- start start 2024-05-10T10:42:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc4717f7640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc4717f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c0d1aa0 0x7fc46c1554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc4717f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc46c15d290 0x7fc46c0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc4717f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c0cf400 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc4717f7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c0cf280 con 0x7fc46c15d290 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc4717f7640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc46c0cf580 con 0x7fc46c071880 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c0d1aa0 0x7fc46c1554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c0d1aa0 0x7fc46c1554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45514/0 (socket says 172.21.15.53:45514) 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc46a7fc640 1 -- 172.21.15.53:0/751860524 learned_addr learned my addr 172.21.15.53:0/751860524 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.964+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc46c15d290 0x7fc46c0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc46affd640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc46a7fc640 1 -- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 msgr2=0x7fc46c0c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc46a7fc640 1 -- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc46c15d290 msgr2=0x7fc46c0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc46c15d290 0x7fc46c0cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc46a7fc640 1 -- 172.21.15.53:0/751860524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c0c77d0 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c0d1aa0 0x7fc46c1554b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc45400c7c0 tx=0x7fc45400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc4637fe640 1 -- 172.21.15.53:0/751860524 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454015070 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc4637fe640 1 -- 172.21.15.53:0/751860524 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc454007820 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c0c7ab0 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc4637fe640 1 -- 172.21.15.53:0/751860524 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454007bd0 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.965+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c0c84c0 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.967+0000 7fc4637fe640 1 -- 172.21.15.53:0/751860524 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc454018070 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.968+0000 7fc4637fe640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc44c076300 0x7fc44c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.968+0000 7fc4637fe640 1 -- 172.21.15.53:0/751860524 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc4540974f0 con 0x7fc46c0d1aa0 2024-05-10T10:42:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.968+0000 7fc46affd640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc44c076300 0x7fc44c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.968+0000 7fc4745ef740 1 Processor -- start 2024-05-10T10:42:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc46affd640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc44c076300 0x7fc44c0787c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc45c033000 tx=0x7fc45c009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc4745ef740 1 -- start start 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6246f50 0x563bc6249760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc6249ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6219bb0 0x563bc621d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc4745ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bc61838d0 con 0x563bc6246f50 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563bc613cc10 con 0x563bc6247330 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563bc617f890 con 0x563bc6219bb0 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.969+0000 7fc46affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6246f50 0x563bc6249760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6246f50 0x563bc6249760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45520/0 (socket says 172.21.15.53:45520) 2024-05-10T10:42:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 -- 172.21.15.53:0/2016373761 learned_addr learned my addr 172.21.15.53:0/2016373761 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc6249ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6219bb0 0x563bc621d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 -- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6219bb0 msgr2=0x563bc621d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6219bb0 0x563bc621d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 -- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 msgr2=0x563bc6249ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc6249ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 -- 172.21.15.53:0/2016373761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc61445d0 con 0x563bc6246f50 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc46affd640 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6246f50 0x563bc6249760 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009680 tx=0x7fc45c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2b660f39c68210 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc460ff9640 1 -- 172.21.15.53:0/2016373761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c036040 con 0x563bc6246f50 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc460ff9640 1 -- 172.21.15.53:0/2016373761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c037040 con 0x563bc6246f50 2024-05-10T10:42:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.970+0000 7fc460ff9640 1 -- 172.21.15.53:0/2016373761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c03db50 con 0x563bc6246f50 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 -- 172.21.15.53:0/2016373761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6246f50 msgr2=0x563bc6249760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6246f50 0x563bc6249760 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009680 tx=0x7fc45c02e000 comp rx=0 tx=0).stop 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 -- 172.21.15.53:0/2016373761 shutdown_connections 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6219bb0 0x563bc621d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc6249ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 --2- 172.21.15.53:0/2016373761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6246f50 0x563bc6249760 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 -- 172.21.15.53:0/2016373761 >> 172.21.15.53:0/2016373761 conn(0x563bc621be20 msgr2=0x563bc6264760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 -- 172.21.15.53:0/2016373761 shutdown_connections 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.971+0000 7fc4745ef740 1 -- 172.21.15.53:0/2016373761 wait complete. 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 -- start start 2024-05-10T10:42:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 Processor -- start 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6247330 0x563bc62464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc625c020 0x563bc6245520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bc621efd0 con 0x563bc6247330 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563bc6140760 con 0x563bc625c020 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563bc621f2b0 con 0x563bc6246f50 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.972+0000 7fc4745ef740 10 client.0 _ll_register_callbacks cb 0x563bc6179cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6247330 0x563bc62464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6247330 0x563bc62464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45536/0 (socket says 172.21.15.53:45536) 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 -- 172.21.15.53:0/25632565 learned_addr learned my addr 172.21.15.53:0/25632565 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc625c020 0x563bc6245520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46affd640 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 -- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 msgr2=0x563bc625e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 -- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc625c020 msgr2=0x563bc6245520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc625c020 0x563bc6245520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 -- 172.21.15.53:0/25632565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc5e952d0 con 0x563bc6247330 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6247330 0x563bc62464a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc45405cc50 tx=0x7fc45405fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e946f3e8bf115d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc449ffb640 1 -- 172.21.15.53:0/25632565 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45401f040 con 0x563bc6247330 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc449ffb640 1 -- 172.21.15.53:0/25632565 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc45405fac0 con 0x563bc6247330 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45c037040 con 0x563bc6247330 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc449ffb640 1 -- 172.21.15.53:0/25632565 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fc454095050 con 0x563bc6247330 2024-05-10T10:42:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc4745ef740 10 client.8154 fetch_fsmap learned FSMap version 98 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x563bc5f5f6b0 con 0x563bc6247330 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.973+0000 7fc449ffb640 1 -- 172.21.15.53:0/25632565 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c037040 con 0x563bc6247330 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc449ffb640 1 -- 172.21.15.53:0/25632565 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7fc454065af0 con 0x563bc6247330 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc4745ef740 10 client.8154 fetch_fsmap finished waiting for FSMap version 98 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc4745ef740 10 client.8154 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc4745ef740 20 client.8154 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc4745ef740 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6268360 0x563bc6283100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc4745ef740 4 client.8154 mds_command: new command op to 8034 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 --> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bc6107760 con 0x563bc6268360 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc46affd640 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6268360 0x563bc6283100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc46affd640 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6268360 0x563bc6283100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bbea242525da3e server_cookie=3a8646cba10b9a4d in_seq=0 out_seq=0 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.974+0000 7fc449ffb640 10 client.8154 ms_handle_connect on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc449ffb640 1 -- 172.21.15.53:0/25632565 <== mds.0 v2:172.21.15.135:6835/776479349 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563bc6107760 con 0x563bc6268360 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc449ffb640 10 client.8154 handle_command_reply: tid=0 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 client.8154 shutdown 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6247330 msgr2=0x563bc62464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6247330 0x563bc62464a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc45405cc50 tx=0x7fc45405fec0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 shutdown_connections 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6268360 0x563bc6283100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc625c020 0x563bc6245520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc6247330 0x563bc62464a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc449ffb640 0 client.8154 ms_handle_reset on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 --2- 172.21.15.53:0/25632565 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 >> 172.21.15.53:0/25632565 conn(0x563bc6219bb0 msgr2=0x563bc6249750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 shutdown_connections 2024-05-10T10:42:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.975+0000 7fc4745ef740 1 -- 172.21.15.53:0/25632565 wait complete. 2024-05-10T10:42:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.976+0000 7fc4745ef740 20 client.8154 trim_cache size 0 max 16384 2024-05-10T10:42:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.994+0000 7fc4745ef740 1 Processor -- start 2024-05-10T10:42:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.994+0000 7fc4745ef740 1 -- start start 2024-05-10T10:42:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6246f50 0x563bc6244ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6247330 0x563bc625b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc625b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc4745ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1fec40 con 0x563bc625c020 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c0cbab0 con 0x563bc6246f50 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc46c1fe5d0 con 0x563bc6247330 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc625b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc625b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45540/0 (socket says 172.21.15.53:45540) 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 -- 172.21.15.53:0/1626683476 learned_addr learned my addr 172.21.15.53:0/1626683476 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46affd640 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6246f50 0x563bc6244ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6247330 0x563bc625b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 -- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6247330 msgr2=0x563bc625b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6247330 0x563bc625b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 -- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6246f50 msgr2=0x563bc6244ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6246f50 0x563bc6244ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46b7fe640 1 -- 172.21.15.53:0/1626683476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc621ba30 con 0x563bc625c020 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.995+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6247330 0x563bc625b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc625b930 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc464004fd0 tx=0x7fc464007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb21fbdb1bb73f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc44bfff640 1 -- 172.21.15.53:0/1626683476 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46401a070 con 0x563bc625c020 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc44bfff640 1 -- 172.21.15.53:0/1626683476 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc464014b70 con 0x563bc625c020 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc44bfff640 1 -- 172.21.15.53:0/1626683476 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46401f460 con 0x563bc625c020 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 -- 172.21.15.53:0/1626683476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 msgr2=0x563bc625b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc625b930 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc464004fd0 tx=0x7fc464007db0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 -- 172.21.15.53:0/1626683476 shutdown_connections 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc625b930 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6247330 0x563bc625b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 --2- 172.21.15.53:0/1626683476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6246f50 0x563bc6244ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 -- 172.21.15.53:0/1626683476 >> 172.21.15.53:0/1626683476 conn(0x563bc6219bb0 msgr2=0x563bc626c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 -- 172.21.15.53:0/1626683476 shutdown_connections 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.996+0000 7fc4745ef740 1 -- 172.21.15.53:0/1626683476 wait complete. 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.997+0000 7fc4745ef740 1 -- start start 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.997+0000 7fc4745ef740 1 Processor -- start 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc625b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc4745ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc6264550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc4745ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bc6265a10 con 0x563bc625c020 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563bc6265d30 con 0x563bc6247330 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc4745ef740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x563bc6261070 con 0x563bc6246f50 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc4745ef740 10 client.0 _ll_register_callbacks cb 0x7fc454060040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc6264550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc6264550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45542/0 (socket says 172.21.15.53:45542) 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 -- 172.21.15.53:0/3180650821 learned_addr learned my addr 172.21.15.53:0/3180650821 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46a7fc640 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc625b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46affd640 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 -- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 msgr2=0x563bc625b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 -- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 msgr2=0x563bc625b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc625b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 -- 172.21.15.53:0/3180650821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc62742e0 con 0x563bc625c020 2024-05-10T10:42:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc46b7fe640 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc6264550 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc464004fa0 tx=0x7fc4640036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c3d6e10b93021b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.998+0000 7fc449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc449ffb640 1 -- 172.21.15.53:0/3180650821 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46401fc20 con 0x563bc625c020 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc449ffb640 1 -- 172.21.15.53:0/3180650821 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc46401fdc0 con 0x563bc625c020 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc464014b70 con 0x563bc625c020 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc449ffb640 1 -- 172.21.15.53:0/3180650821 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fc464032050 con 0x563bc625c020 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 10 client.8160 fetch_fsmap learned FSMap version 98 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x563bc621aec0 con 0x563bc625c020 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc449ffb640 1 -- 172.21.15.53:0/3180650821 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc464014b70 con 0x563bc625c020 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc449ffb640 1 -- 172.21.15.53:0/3180650821 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7fc464035020 con 0x563bc625c020 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 10 client.8160 fetch_fsmap finished waiting for FSMap version 98 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 10 client.8160 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 20 client.8160 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6264eb0 0x563bc624c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 4 client.8160 mds_command: new command op to 8034 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 --> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] -- command(tid 0: {"prefix": "status"}) -- 0x563bc61914e0 con 0x563bc6264eb0 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:55.999+0000 7fc46affd640 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6264eb0 0x563bc624c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc449ffb640 10 client.8160 ms_handle_connect on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc46affd640 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6264eb0 0x563bc624c2a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49eccb880a9abfbd server_cookie=3c276fde11a5f2d2 in_seq=0 out_seq=0 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc449ffb640 1 -- 172.21.15.53:0/3180650821 <== mds.0 v2:172.21.15.135:6835/776479349 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563bc61914e0 con 0x563bc6264eb0 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc449ffb640 10 client.8160 handle_command_reply: tid=0 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 client.8160 shutdown 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 msgr2=0x563bc6264550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc6264550 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc464004fa0 tx=0x7fc4640036f0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 shutdown_connections 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x563bc6264eb0 0x563bc624c2a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bc625c020 0x563bc6264550 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563bc6247330 0x563bc625b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 --2- 172.21.15.53:0/3180650821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x563bc6246f50 0x563bc625b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc449ffb640 0 client.8160 ms_handle_reset on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 >> 172.21.15.53:0/3180650821 conn(0x563bc6219bb0 msgr2=0x563bc62452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 shutdown_connections 2024-05-10T10:42:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 1 -- 172.21.15.53:0/3180650821 wait complete. 2024-05-10T10:42:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.000+0000 7fc4745ef740 20 client.8160 trim_cache size 0 max 16384 2024-05-10T10:42:56.010 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 8034, 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 5.7885879750000004, 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 98, 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 118, 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 118, 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 11.11870588 2024-05-10T10:42:56.011 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc44c076300 msgr2=0x7fc44c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc44c076300 0x7fc44c0787c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc45c033000 tx=0x7fc45c009000 comp rx=0 tx=0).stop 2024-05-10T10:42:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c0d1aa0 msgr2=0x7fc46c1554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c0d1aa0 0x7fc46c1554b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc45400c7c0 tx=0x7fc45400cc80 comp rx=0 tx=0).stop 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 shutdown_connections 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc44c076300 0x7fc44c0787c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc46c15d290 0x7fc46c0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc46c0d1aa0 0x7fc46c1554b0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 --2- 172.21.15.53:0/751860524 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc46c071880 0x7fc46c0c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 >> 172.21.15.53:0/751860524 conn(0x7fc46c09ba20 msgr2=0x7fc46c0b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 shutdown_connections 2024-05-10T10:42:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.001+0000 7fc4717f7640 1 -- 172.21.15.53:0/751860524 wait complete. 2024-05-10T10:42:56.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:42:56.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.116+0000 7fc2fc9a4640 1 Processor -- start 2024-05-10T10:42:56.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.116+0000 7fc2fc9a4640 1 -- start start 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2fc9a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 0x7fc2f80c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2fc9a4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f806f1c0 con 0x7fc2f80cee40 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2fc9a4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f806f4a0 con 0x7fc2f81598c0 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2fc9a4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f80c7620 con 0x7fc2f80c6c80 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2f6d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc2f80cee40 0x7fc2f80cf2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49230/0 (socket says 172.21.15.53:49230) 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2f6d9d640 1 -- 172.21.15.53:0/2643444889 learned_addr learned my addr 172.21.15.53:0/2643444889 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2f5d9b640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 0x7fc2f80c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 820858551 0 0) 0x7fc2f806f1c0 con 0x7fc2f80cee40 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.117+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2e0003520 con 0x7fc2f80cee40 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280563915 0 0) 0x7fc2f806f4a0 con 0x7fc2f81598c0 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2f806f1c0 con 0x7fc2f81598c0 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4065131949 0 0) 0x7fc2e0003520 con 0x7fc2f80cee40 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 msgr2=0x7fc2f80c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 0x7fc2f80c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 >> v1:172.21.15.135:6789/0 conn(0x7fc2f81598c0 legacy=0x7fc2f806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f806f780 con 0x7fc2f80cee40 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fc2f00019f0 con 0x7fc2f80cee40 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f5d9b640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 0x7fc2f80c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 >> v1:172.21.15.53:6789/0 conn(0x7fc2f80cee40 legacy=0x7fc2f80cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 0x7fc2e0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2e00040b0 0x7fc2e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000ba70 con 0x7fc2e0004d40 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000bbf0 con 0x7fc2e00040b0 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e000bdf0 con 0x7fc2f80c6c80 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f6d9d640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f659c640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 0x7fc2e0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f5d9b640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2e00040b0 0x7fc2e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f6d9d640 1 -- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 msgr2=0x7fc2e0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f6d9d640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f80c6c80 0x7fc2e0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f6d9d640 1 -- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2e00040b0 msgr2=0x7fc2e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f6d9d640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2e00040b0 0x7fc2e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.118+0000 7fc2f6d9d640 1 -- 172.21.15.53:0/2643444889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e0003520 con 0x7fc2e0004d40 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.119+0000 7fc2f6d9d640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2e00051c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000a7f0 tx=0x7fc2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e21a3828c69135 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.119+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000a410 con 0x7fc2e0004d40 2024-05-10T10:42:56.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.119+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f002f070 con 0x7fc2e0004d40 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.119+0000 7fc2f559a640 1 -- 172.21.15.53:0/2643444889 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f002bc30 con 0x7fc2e0004d40 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.120+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/2643444889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 msgr2=0x7fc2e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.120+0000 7fc2fc9a4640 1 --2- 172.21.15.53:0/2643444889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2e00051c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000a7f0 tx=0x7fc2e0005830 comp rx=0 tx=0).stop 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.120+0000 7fc2f6d9d640 1 -- 172.21.15.53:0/2643444889 reap_dead start 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.120+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/2643444889 shutdown_connections 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.120+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/2643444889 >> 172.21.15.53:0/2643444889 conn(0x7fc2f80b2900 msgr2=0x7fc2f80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.120+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/2643444889 shutdown_connections 2024-05-10T10:42:56.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.120+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/2643444889 wait complete. 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.121+0000 7fc2fc9a4640 1 Processor -- start 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.121+0000 7fc2fc9a4640 1 -- start start 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2fc9a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2f8156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2fc9a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f80c9e30 0x7fc2f815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2fc9a4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815e8f0 0x7fc2f80cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2fc9a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f815c6e0 con 0x7fc2e0004d40 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2fc9a4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80ce920 con 0x7fc2f80c9e30 2024-05-10T10:42:56.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2fc9a4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f806ee10 con 0x7fc2f815e8f0 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2f8156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2f8156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45554/0 (socket says 172.21.15.53:45554) 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 -- 172.21.15.53:0/3749507110 learned_addr learned my addr 172.21.15.53:0/3749507110 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f6d9d640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815e8f0 0x7fc2f80cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f5d9b640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f80c9e30 0x7fc2f815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 -- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815e8f0 msgr2=0x7fc2f80cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815e8f0 0x7fc2f80cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 -- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f80c9e30 msgr2=0x7fc2f815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f80c9e30 0x7fc2f815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f659c640 1 -- 172.21.15.53:0/3749507110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f80c8d90 con 0x7fc2e0004d40 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.122+0000 7fc2f5d9b640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f80c9e30 0x7fc2f815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:56.129 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.123+0000 7fc2f659c640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2f8156990 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800f7a0 tx=0x7fc2e800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.123+0000 7fc2e6ffd640 1 -- 172.21.15.53:0/3749507110 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80052e0 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.123+0000 7fc2e6ffd640 1 -- 172.21.15.53:0/3749507110 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8018070 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.123+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f80c9070 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.123+0000 7fc2e6ffd640 1 -- 172.21.15.53:0/3749507110 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e800d970 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.123+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f80c9630 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.124+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f80afc70 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.124+0000 7fc2e6ffd640 1 -- 172.21.15.53:0/3749507110 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc2e800db10 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.125+0000 7fc2e6ffd640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2d8076300 0x7fc2d80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.125+0000 7fc2e6ffd640 1 -- 172.21.15.53:0/3749507110 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc2e8097730 con 0x7fc2e0004d40 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.125+0000 7fc2f5d9b640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2d8076300 0x7fc2d80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.126+0000 7fc2f5d9b640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2d8076300 0x7fc2d80787c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ade10 tx=0x7fc2ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:56.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.129+0000 7fc2e6ffd640 1 -- 172.21.15.53:0/3749507110 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc2e8060f10 con 0x7fc2e0004d40 2024-05-10T10:42:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.360+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2f806a220 con 0x7fc2e0004d40 2024-05-10T10:42:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.360+0000 7fc2e6ffd640 1 -- 172.21.15.53:0/3749507110 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7fc2e8060810 con 0x7fc2e0004d40 2024-05-10T10:42:56.362 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:42:56.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2d8076300 msgr2=0x7fc2d80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2d8076300 0x7fc2d80787c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ade10 tx=0x7fc2ec008040 comp rx=0 tx=0).stop 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 msgr2=0x7fc2f8156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2f8156990 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800f7a0 tx=0x7fc2e800fc60 comp rx=0 tx=0).stop 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 shutdown_connections 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc2d8076300 0x7fc2d80787c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2f815e8f0 0x7fc2f80cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2f80c9e30 0x7fc2f815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 --2- 172.21.15.53:0/3749507110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e0004d40 0x7fc2f8156990 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 >> 172.21.15.53:0/3749507110 conn(0x7fc2f80b2900 msgr2=0x7fc2f815f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 shutdown_connections 2024-05-10T10:42:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56.363+0000 7fc2fc9a4640 1 -- 172.21.15.53:0/3749507110 wait complete. 2024-05-10T10:42:56.375 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:42:56.375 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:42:56.375 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:42:56.376 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:42:56.376 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:56.376 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:42:56.376 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:42:56.376 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:42:56.376 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:42:56.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:56.376 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:56.393 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42193sec preferred_lft 42193sec 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:42:56.394 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:42:56.395 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:56.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:56.396 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:42:56.457 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:42:56.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:56.459 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:42:56.516 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:42:56.516 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:42:56.516 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:42:56.516 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:42:56.516 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:42:56.517 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:42:56.517 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:42:56.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:42:56.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:42:56.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:42:56.517 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:42:56.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:42:56.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:56.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:56.646 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:42:56.662 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:42:56.662 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:42:56.662 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:42:56.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:56.663 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:42:56.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:42:56.728 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:56.784 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:42:56.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:42:56.786 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:56.840 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:42:56.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:56.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:56.925 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:56.925 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:42:56.925 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:42:56.934 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:56.934 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:42:56.934 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:42:56.934 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:42:56.934 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:42:56.939 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:56.939 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:42:56.940 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:42:56.940 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:42:56.940 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:42:56.940 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:42:56.940 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:42:56.940 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:42:56.940 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:42:56.941 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:42:56.941 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:42:56.941 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:42:56.941 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:42:56.942 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:42:56.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:56.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:56.969 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:42:57.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:42:57.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.075 INFO:teuthology.orchestra.run.smithi053.stdout:client6578 2024-05-10T10:42:57.075 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:57.076 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:57.092 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:57.092 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:57.092 DEBUG:teuthology.orchestra.run.smithi053:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-05-10T10:42:57.092 DEBUG:teuthology.orchestra.run.smithi053:> parent = os.path.join(mount_point, "parent") 2024-05-10T10:42:57.092 DEBUG:teuthology.orchestra.run.smithi053:> os.mkdir(parent) 2024-05-10T10:42:57.093 DEBUG:teuthology.orchestra.run.smithi053:> flushed = os.path.join(parent, "flushed") 2024-05-10T10:42:57.093 DEBUG:teuthology.orchestra.run.smithi053:> os.mkdir(flushed) 2024-05-10T10:42:57.093 DEBUG:teuthology.orchestra.run.smithi053:> for f in ["alpha", "bravo", "charlie"]: 2024-05-10T10:42:57.093 DEBUG:teuthology.orchestra.run.smithi053:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-05-10T10:42:57.093 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:42:57.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:57.219 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-10T10:42:57.281 INFO:teuthology.orchestra.run.smithi053.stdout:./ 2024-05-10T10:42:57.281 INFO:teuthology.orchestra.run.smithi053.stdout:./parent 2024-05-10T10:42:57.281 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed 2024-05-10T10:42:57.281 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/charlie 2024-05-10T10:42:57.281 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/alpha 2024-05-10T10:42:57.281 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/bravo 2024-05-10T10:42:57.282 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:57.282 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:57.282 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:42:57.282 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:42:57.282 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-10T10:42:57.282 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:42:57.388 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:42:57.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.398 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:57.398 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:57.398 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:42:57.398 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:42:57.398 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-10T10:42:57.398 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:42:57.506 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2024-05-10T10:42:57.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.515 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:57.515 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:57.515 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:42:57.515 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:42:57.515 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-10T10:42:57.515 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:42:57.623 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2024-05-10T10:42:57.626 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.632 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:57.632 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:57.632 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:42:57.632 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:42:57.632 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-10T10:42:57.632 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:42:57.740 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628280 2024-05-10T10:42:57.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.749 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:57.749 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:57.749 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:42:57.749 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:42:57.749 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-10T10:42:57.750 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:42:57.857 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627778 2024-05-10T10:42:57.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.866 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:57.866 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:57.866 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:42:57.866 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:42:57.866 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-10T10:42:57.866 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:42:57.974 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628279 2024-05-10T10:42:57.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:57.982 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:57.982 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:58.039 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:58.039 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:58.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:42:58.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.239+0000 7fc04e711640 1 Processor -- start 2024-05-10T10:42:58.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.239+0000 7fc04e711640 1 -- start start 2024-05-10T10:42:58.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.240+0000 7fc04e711640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc04815f3b0 0x7fc0480d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.240+0000 7fc04e711640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0480d5d30 0x7fc0480d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.240+0000 7fc04e711640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0480d30b0 con 0x7fc04815f790 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.240+0000 7fc04e711640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc0480d3390 con 0x7fc04815f3b0 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.240+0000 7fc04e711640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc0480c8c80 con 0x7fc0480d5d30 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc04ccae640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0480d5d30 0x7fc0480d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc047fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc04815f3b0 0x7fc0480d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc0477fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc04815f790 0x7fc0480c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:40052/0 (socket says 172.21.15.53:40052) 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc0477fe640 1 -- 172.21.15.53:0/1501162470 learned_addr learned my addr 172.21.15.53:0/1501162470 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc04ccae640 1 -- 172.21.15.53:0/1501162470 >> v2:172.21.15.135:3300/0 conn(0x7fc04815f3b0 msgr2=0x7fc0480d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc04ccae640 1 --2- 172.21.15.53:0/1501162470 >> v2:172.21.15.135:3300/0 conn(0x7fc04815f3b0 0x7fc0480d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc04ccae640 1 -- 172.21.15.53:0/1501162470 >> v1:172.21.15.135:6789/0 conn(0x7fc04815f790 legacy=0x7fc0480c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc04ccae640 1 -- 172.21.15.53:0/1501162470 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0480c8f60 con 0x7fc0480d5d30 2024-05-10T10:42:58.243 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.241+0000 7fc047fff640 1 --2- 172.21.15.53:0/1501162470 >> v2:172.21.15.135:3300/0 conn(0x7fc04815f3b0 0x7fc0480d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:58.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.242+0000 7fc04ccae640 1 --2- 172.21.15.53:0/1501162470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0480d5d30 0x7fc0480d3c30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc040009d90 tx=0x7fc04002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57cec588b0db9545 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.242+0000 7fc046ffd640 1 -- 172.21.15.53:0/1501162470 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040007740 con 0x7fc0480d5d30 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.242+0000 7fc046ffd640 1 -- 172.21.15.53:0/1501162470 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0400078e0 con 0x7fc0480d5d30 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.242+0000 7fc046ffd640 1 -- 172.21.15.53:0/1501162470 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040007cb0 con 0x7fc0480d5d30 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 -- 172.21.15.53:0/1501162470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0480d5d30 msgr2=0x7fc0480d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 --2- 172.21.15.53:0/1501162470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0480d5d30 0x7fc0480d3c30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc040009d90 tx=0x7fc04002a660 comp rx=0 tx=0).stop 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 -- 172.21.15.53:0/1501162470 shutdown_connections 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 --2- 172.21.15.53:0/1501162470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0480d5d30 0x7fc0480d3c30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 --2- 172.21.15.53:0/1501162470 >> v2:172.21.15.135:3300/0 conn(0x7fc04815f3b0 0x7fc0480d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 -- 172.21.15.53:0/1501162470 >> 172.21.15.53:0/1501162470 conn(0x7fc0480d5130 msgr2=0x7fc0480d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 -- 172.21.15.53:0/1501162470 shutdown_connections 2024-05-10T10:42:58.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.243+0000 7fc04e711640 1 -- 172.21.15.53:0/1501162470 wait complete. 2024-05-10T10:42:58.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.244+0000 7fc04e711640 1 Processor -- start 2024-05-10T10:42:58.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.244+0000 7fc04e711640 1 -- start start 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc04e711640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0480d5d30 0x7fc0481551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc04e711640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc04815f3b0 0x7fc0480cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc04e711640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc048151f20 0x7fc04815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc04e711640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc0480d0e60 con 0x7fc0480d5d30 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc04e711640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0480d0ce0 con 0x7fc048151f20 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc04e711640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc0480d0fe0 con 0x7fc04815f3b0 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0480d5d30 0x7fc0481551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0480d5d30 0x7fc0481551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45568/0 (socket says 172.21.15.53:45568) 2024-05-10T10:42:58.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 -- 172.21.15.53:0/3220760244 learned_addr learned my addr 172.21.15.53:0/3220760244 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc0477fe640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc04815f3b0 0x7fc0480cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc04ccae640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc048151f20 0x7fc04815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 -- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc04815f3b0 msgr2=0x7fc0480cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc04815f3b0 0x7fc0480cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 -- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc048151f20 msgr2=0x7fc04815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc048151f20 0x7fc04815ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.245+0000 7fc047fff640 1 -- 172.21.15.53:0/3220760244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0480ce4a0 con 0x7fc0480d5d30 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.246+0000 7fc047fff640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0480d5d30 0x7fc0481551a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fc03800ebe0 tx=0x7fc03800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.246+0000 7fc044ff9640 1 -- 172.21.15.53:0/3220760244 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038017070 con 0x7fc0480d5d30 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.246+0000 7fc044ff9640 1 -- 172.21.15.53:0/3220760244 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03800cbd0 con 0x7fc0480d5d30 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.246+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc048155830 con 0x7fc0480d5d30 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.246+0000 7fc044ff9640 1 -- 172.21.15.53:0/3220760244 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038013470 con 0x7fc0480d5d30 2024-05-10T10:42:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.246+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc048155df0 con 0x7fc0480d5d30 2024-05-10T10:42:58.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.247+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04806a220 con 0x7fc0480d5d30 2024-05-10T10:42:58.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.248+0000 7fc044ff9640 1 -- 172.21.15.53:0/3220760244 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc03801a070 con 0x7fc0480d5d30 2024-05-10T10:42:58.250 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.248+0000 7fc044ff9640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0340762d0 0x7fc034078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.248+0000 7fc044ff9640 1 -- 172.21.15.53:0/3220760244 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc038097540 con 0x7fc0480d5d30 2024-05-10T10:42:58.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.249+0000 7fc0477fe640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0340762d0 0x7fc034078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.251 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.249+0000 7fc0477fe640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0340762d0 0x7fc034078790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc0480baef0 tx=0x7fc03c007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.253+0000 7fc044ff9640 1 -- 172.21.15.53:0/3220760244 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc038060750 con 0x7fc0480d5d30 2024-05-10T10:42:58.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.458+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc048075da0 con 0x7fc0480d5d30 2024-05-10T10:42:58.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.458+0000 7fc044ff9640 1 -- 172.21.15.53:0/3220760244 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7fc038064a50 con 0x7fc0480d5d30 2024-05-10T10:42:58.460 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:42:58.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0340762d0 msgr2=0x7fc034078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0340762d0 0x7fc034078790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc0480baef0 tx=0x7fc03c007e80 comp rx=0 tx=0).stop 2024-05-10T10:42:58.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0480d5d30 msgr2=0x7fc0481551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0480d5d30 0x7fc0481551a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fc03800ebe0 tx=0x7fc03800c520 comp rx=0 tx=0).stop 2024-05-10T10:42:58.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 shutdown_connections 2024-05-10T10:42:58.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc0340762d0 0x7fc034078790 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc048151f20 0x7fc04815ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc04815f3b0 0x7fc0480cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 --2- 172.21.15.53:0/3220760244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0480d5d30 0x7fc0481551a0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 >> 172.21.15.53:0/3220760244 conn(0x7fc0480d5130 msgr2=0x7fc0480ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 shutdown_connections 2024-05-10T10:42:58.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.460+0000 7fc04e711640 1 -- 172.21.15.53:0/3220760244 wait complete. 2024-05-10T10:42:58.473 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:42:58.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:58.473 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:58.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:42:58.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:58.527 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:42:58.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-10T10:42:58.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.698+0000 7f016a1af640 1 Processor -- start 2024-05-10T10:42:58.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.698+0000 7f016a1af640 1 -- start start 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f016a1af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 0x7f01640cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f016a1af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f01640cd500 0x7f01640cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f016a1af640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016406e720 con 0x7f01640b2dd0 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f016a1af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f016406de80 con 0x7f01640cd500 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f016a1af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f016406e160 con 0x7f01640cf3e0 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f0163fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f01640b2dd0 0x7f01640b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49242/0 (socket says 172.21.15.53:49242) 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f0163fff640 1 -- 172.21.15.53:0/745237654 learned_addr learned my addr 172.21.15.53:0/745237654 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3397294191 0 0) 0x7f016406e720 con 0x7f01640b2dd0 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f015affd640 1 --2- 172.21.15.53:0/745237654 >> v2:172.21.15.135:3300/0 conn(0x7f01640cd500 0x7f01640cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0150003520 con 0x7f01640b2dd0 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.699+0000 7f01637fe640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 0x7f01640cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3045337698 0 0) 0x7f0150003520 con 0x7f01640b2dd0 2024-05-10T10:42:58.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 msgr2=0x7f01640cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 0x7f01640cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 >> v2:172.21.15.135:3300/0 conn(0x7f01640cd500 msgr2=0x7f01640cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 --2- 172.21.15.53:0/745237654 >> v2:172.21.15.135:3300/0 conn(0x7f01640cd500 0x7f01640cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016406e440 con 0x7f01640b2dd0 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f015c0019f0 con 0x7f01640b2dd0 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f015affd640 1 --2- 172.21.15.53:0/745237654 >> v2:172.21.15.135:3300/0 conn(0x7f01640cd500 0x7f01640cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 >> v1:172.21.15.53:6789/0 conn(0x7f01640b2dd0 legacy=0x7f01640b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0150004800 0x7f0150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 0x7f01500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f016406de80 con 0x7f0150004800 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f016406e160 con 0x7f01500039b0 2024-05-10T10:42:58.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f015000ba70 con 0x7f01640cf3e0 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f015affd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0150004800 0x7f0150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f01637fe640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f0163fff640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 0x7f01500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f015affd640 1 -- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 msgr2=0x7f01500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f015affd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01640cf3e0 0x7f01500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f015affd640 1 -- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01500039b0 msgr2=0x7f0150003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f015affd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01500039b0 0x7f0150003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.700+0000 7f015affd640 1 -- 172.21.15.53:0/745237654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0150003520 con 0x7f0150004800 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.701+0000 7f015affd640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0150004800 0x7f0150004c80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f014c006fd0 tx=0x7f0150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8690492c3b29004a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.701+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c00f040 con 0x7f0150004800 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.701+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c02ad00 con 0x7f0150004800 2024-05-10T10:42:58.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.701+0000 7f0162ffd640 1 -- 172.21.15.53:0/745237654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c004250 con 0x7f0150004800 2024-05-10T10:42:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.702+0000 7f016a1af640 1 -- 172.21.15.53:0/745237654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0150004800 msgr2=0x7f0150004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.702+0000 7f016a1af640 1 --2- 172.21.15.53:0/745237654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0150004800 0x7f0150004c80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f014c006fd0 tx=0x7f0150005830 comp rx=0 tx=0).stop 2024-05-10T10:42:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.702+0000 7f0163fff640 1 -- 172.21.15.53:0/745237654 reap_dead start 2024-05-10T10:42:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.702+0000 7f016a1af640 1 -- 172.21.15.53:0/745237654 shutdown_connections 2024-05-10T10:42:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.702+0000 7f016a1af640 1 -- 172.21.15.53:0/745237654 >> 172.21.15.53:0/745237654 conn(0x7f01640d2eb0 msgr2=0x7f01640d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.702+0000 7f016a1af640 1 -- 172.21.15.53:0/745237654 shutdown_connections 2024-05-10T10:42:58.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.702+0000 7f016a1af640 1 -- 172.21.15.53:0/745237654 wait complete. 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.703+0000 7f016a1af640 1 Processor -- start 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.703+0000 7f016a1af640 1 -- start start 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f016a1af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0150004800 0x7f01641f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f016a1af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01640b8020 0x7f01641521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f016a1af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01640cc770 0x7f01640ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f016a1af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01640cf8c0 con 0x7f01640b8020 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f016a1af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01640b4510 con 0x7f01640cc770 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f016a1af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f01640cfa40 con 0x7f0150004800 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01640b8020 0x7f01641521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01640b8020 0x7f01641521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45580/0 (socket says 172.21.15.53:45580) 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 -- 172.21.15.53:0/453402379 learned_addr learned my addr 172.21.15.53:0/453402379 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f0163fff640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01640cc770 0x7f01640ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f01637fe640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0150004800 0x7f01641f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 -- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0150004800 msgr2=0x7f01641f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0150004800 0x7f01641f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 -- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01640cc770 msgr2=0x7f01640ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01640cc770 0x7f01640ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.704+0000 7f015affd640 1 -- 172.21.15.53:0/453402379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01641526e0 con 0x7f01640b8020 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.705+0000 7f015affd640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01640b8020 0x7f01641521a0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f014c007010 tx=0x7f014c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.705+0000 7f0160ff9640 1 -- 172.21.15.53:0/453402379 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c041070 con 0x7f01640b8020 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.705+0000 7f0160ff9640 1 -- 172.21.15.53:0/453402379 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c00f000 con 0x7f01640b8020 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.705+0000 7f0160ff9640 1 -- 172.21.15.53:0/453402379 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0044d0 con 0x7f01640b8020 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.705+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01640b84a0 con 0x7f01640b8020 2024-05-10T10:42:58.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.705+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0164152c40 con 0x7f01640b8020 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.707+0000 7f0160ff9640 1 -- 172.21.15.53:0/453402379 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f014c044070 con 0x7f01640b8020 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.707+0000 7f0160ff9640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0144076300 0x7f01440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.707+0000 7f0160ff9640 1 -- 172.21.15.53:0/453402379 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f014c0b7e20 con 0x7f01640b8020 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.707+0000 7f01637fe640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0144076300 0x7f01440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.708+0000 7f016cf9f740 1 Processor -- start 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.708+0000 7f016cf9f740 1 -- start start 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.708+0000 7f01637fe640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0144076300 0x7f01440787c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0154006fd0 tx=0x7f0154008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236da600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236dab40 0x5583236af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f016cf9f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558323677690 con 0x5583236f8160 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558323614f30 con 0x5583236db280 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5583235cdb20 con 0x5583236dab40 2024-05-10T10:42:58.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236aaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45590/0 (socket says 172.21.15.53:45590) 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 -- 172.21.15.53:0/2217439388 learned_addr learned my addr 172.21.15.53:0/2217439388 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f015affd640 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f0163fff640 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236dab40 0x5583236af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 -- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236dab40 msgr2=0x5583236af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236dab40 0x5583236af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 -- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 msgr2=0x5583236da600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236da600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f01637fe640 1 -- 172.21.15.53:0/2217439388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583235c5e60 con 0x5583236f8160 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f0163fff640 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236dab40 0x5583236af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.709+0000 7f015affd640 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236da600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.710+0000 7f01637fe640 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236aaaf0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f015400b8a0 tx=0x7f0154015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a9cfcca66c639f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.710+0000 7f01597fa640 1 -- 172.21.15.53:0/2217439388 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015400be20 con 0x5583236f8160 2024-05-10T10:42:58.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.710+0000 7f01597fa640 1 -- 172.21.15.53:0/2217439388 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0154027070 con 0x5583236f8160 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.710+0000 7f01597fa640 1 -- 172.21.15.53:0/2217439388 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154015bd0 con 0x5583236f8160 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.710+0000 7f016cf9f740 1 -- 172.21.15.53:0/2217439388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 msgr2=0x5583236aaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.710+0000 7f016cf9f740 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236aaaf0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f015400b8a0 tx=0x7f0154015000 comp rx=0 tx=0).stop 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 -- 172.21.15.53:0/2217439388 shutdown_connections 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236dab40 0x5583236af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236da600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 --2- 172.21.15.53:0/2217439388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236aaaf0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 -- 172.21.15.53:0/2217439388 >> 172.21.15.53:0/2217439388 conn(0x5583236acd60 msgr2=0x5583236f5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 -- 172.21.15.53:0/2217439388 shutdown_connections 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 -- 172.21.15.53:0/2217439388 wait complete. 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 -- start start 2024-05-10T10:42:58.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.711+0000 7f016cf9f740 1 Processor -- start 2024-05-10T10:42:58.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583237055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f016cf9f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5583235d15e0 con 0x5583236f8160 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558323610ef0 con 0x5583236db280 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5583236aef10 con 0x558323705140 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f016cf9f740 10 client.0 _ll_register_callbacks cb 0x5583235cc320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45596/0 (socket says 172.21.15.53:45596) 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 -- 172.21.15.53:0/4045037715 learned_addr learned my addr 172.21.15.53:0/4045037715 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 -- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 msgr2=0x5583237055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f0163fff640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583237055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f01637fe640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583237055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 -- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 msgr2=0x5583236de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f015affd640 1 -- 172.21.15.53:0/4045037715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583233f07d0 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f0163fff640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583237055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.712+0000 7f01637fe640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f015affd640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236cad80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f014c003fd0 tx=0x7f014c081490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f66fb0fe66a507 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f01427fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f01427fc640 1 -- 172.21.15.53:0/4045037715 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c04b040 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f01427fc640 1 -- 172.21.15.53:0/4045037715 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f014c0b2c50 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0154027070 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f01427fc640 1 -- 172.21.15.53:0/4045037715 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f014c0b2e50 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 10 client.8181 fetch_fsmap learned FSMap version 98 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x5583233263f0 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f01427fc640 1 -- 172.21.15.53:0/4045037715 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0154027070 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f01427fc640 1 -- 172.21.15.53:0/4045037715 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f014c07fbc0 con 0x5583236f8160 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 10 client.8181 fetch_fsmap finished waiting for FSMap version 98 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 10 client.8181 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-10T10:42:58.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 20 client.8181 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236dc8f0 0x5583236ed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 4 client.8181 mds_command: new command op to 8034 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 --> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558323622b40 con 0x5583236dc8f0 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.713+0000 7f01637fe640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236dc8f0 0x5583236ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.714+0000 7f01637fe640 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236dc8f0 0x5583236ed180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2f6cb9b659acd0 server_cookie=72d063f4d36501d6 in_seq=0 out_seq=0 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.714+0000 7f01427fc640 10 client.8181 ms_handle_connect on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f01427fc640 1 -- 172.21.15.53:0/4045037715 <== mds.0 v2:172.21.15.135:6835/776479349 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558323622b40 con 0x5583236dc8f0 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f01427fc640 10 client.8181 handle_command_reply: tid=0 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 client.8181 shutdown 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 msgr2=0x5583236cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236cad80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f014c003fd0 tx=0x7f014c081490 comp rx=0 tx=0).stop 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 shutdown_connections 2024-05-10T10:42:58.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236dc8f0 0x5583236ed180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583237055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236f8160 0x5583236cad80 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f01427fc640 0 client.8181 ms_handle_reset on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 --2- 172.21.15.53:0/4045037715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 >> 172.21.15.53:0/4045037715 conn(0x5583236dab40 msgr2=0x5583236aa230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 shutdown_connections 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 1 -- 172.21.15.53:0/4045037715 wait complete. 2024-05-10T10:42:58.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.715+0000 7f016cf9f740 20 client.8181 trim_cache size 0 max 16384 2024-05-10T10:42:58.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.730+0000 7f016cf9f740 1 Processor -- start 2024-05-10T10:42:58.731 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.730+0000 7f016cf9f740 1 -- start start 2024-05-10T10:42:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236f8160 0x5583236f9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558323705140 0x558323717d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f016cf9f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01642025a0 con 0x558323705140 2024-05-10T10:42:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0164201580 con 0x5583236db280 2024-05-10T10:42:58.734 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f01642008e0 con 0x5583236f8160 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558323705140 0x558323717d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558323705140 0x558323717d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45602/0 (socket says 172.21.15.53:45602) 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 -- 172.21.15.53:0/3959854273 learned_addr learned my addr 172.21.15.53:0/3959854273 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f01637fe640 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f015affd640 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236f8160 0x5583236f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 -- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236f8160 msgr2=0x5583236f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236f8160 0x5583236f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 -- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 msgr2=0x5583236d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 -- 172.21.15.53:0/3959854273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583232e60f0 con 0x558323705140 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.731+0000 7f0163fff640 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558323705140 0x558323717d20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f0164073530 tx=0x7f015c007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e078d04eb65b69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f0158ff9640 1 -- 172.21.15.53:0/3959854273 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c010040 con 0x558323705140 2024-05-10T10:42:58.735 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f0158ff9640 1 -- 172.21.15.53:0/3959854273 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c015610 con 0x558323705140 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f0158ff9640 1 -- 172.21.15.53:0/3959854273 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c015a00 con 0x558323705140 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 -- 172.21.15.53:0/3959854273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558323705140 msgr2=0x558323717d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558323705140 0x558323717d20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f0164073530 tx=0x7f015c007420 comp rx=0 tx=0).stop 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 -- 172.21.15.53:0/3959854273 shutdown_connections 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558323705140 0x558323717d20 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5583236f8160 0x5583236f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 --2- 172.21.15.53:0/3959854273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236db280 0x5583236d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 -- 172.21.15.53:0/3959854273 >> 172.21.15.53:0/3959854273 conn(0x5583236dab40 msgr2=0x5583236de4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 -- 172.21.15.53:0/3959854273 shutdown_connections 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.732+0000 7f016cf9f740 1 -- 172.21.15.53:0/3959854273 wait complete. 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.733+0000 7f016cf9f740 1 -- start start 2024-05-10T10:42:58.736 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.733+0000 7f016cf9f740 1 Processor -- start 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236db280 0x558323772f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236f8160 0x5583236f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583236eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5583236f4180 con 0x5583236db280 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0164200f30 con 0x5583236f8160 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5583236ee400 con 0x558323705140 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 10 client.0 _ll_register_callbacks cb 0x7f014c07cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:42:58.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236db280 0x558323772f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236db280 0x558323772f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45608/0 (socket says 172.21.15.53:45608) 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 -- 172.21.15.53:0/514302673 learned_addr learned my addr 172.21.15.53:0/514302673 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f015affd640 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236f8160 0x5583236f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 -- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 msgr2=0x5583236eaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583236eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 -- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236f8160 msgr2=0x5583236f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236f8160 0x5583236f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 -- 172.21.15.53:0/514302673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583236c8d60 con 0x5583236db280 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f015affd640 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236f8160 0x5583236f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01637fe640 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236db280 0x558323772f30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f015400b870 tx=0x7f01540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc823869c0a7454 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01427fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01427fc640 1 -- 172.21.15.53:0/514302673 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015403a030 con 0x5583236db280 2024-05-10T10:42:58.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f01427fc640 1 -- 172.21.15.53:0/514302673 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f0154015c60 con 0x5583236db280 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.734+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015c015610 con 0x5583236db280 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f01427fc640 1 -- 172.21.15.53:0/514302673 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f0154006bd0 con 0x5583236db280 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 10 client.8187 fetch_fsmap learned FSMap version 98 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x5583236f8e70 con 0x5583236db280 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f01427fc640 1 -- 172.21.15.53:0/514302673 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c015610 con 0x5583236db280 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f01427fc640 1 -- 172.21.15.53:0/514302673 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f0154036340 con 0x5583236db280 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 10 client.8187 fetch_fsmap finished waiting for FSMap version 98 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 10 client.8187 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 20 client.8187 populate_metadata read hostname 'smithi053' 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236f7280 0x558323705520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 4 client.8187 mds_command: new command op to 8034 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 --> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55832361c9b0 con 0x5583236f7280 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.735+0000 7f015affd640 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236f7280 0x558323705520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.736+0000 7f015affd640 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236f7280 0x558323705520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf51c17963ddb1a server_cookie=b27c92b9d71a5c0a in_seq=0 out_seq=0 2024-05-10T10:42:58.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.736+0000 7f01427fc640 10 client.8187 ms_handle_connect on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.748+0000 7f01427fc640 1 -- 172.21.15.53:0/514302673 <== mds.0 v2:172.21.15.135:6835/776479349 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55832361c9b0 con 0x5583236f7280 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.748+0000 7f01427fc640 10 client.8187 handle_command_reply: tid=0 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.748+0000 7f016cf9f740 1 client.8187 shutdown 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236db280 msgr2=0x558323772f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236db280 0x558323772f30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f015400b870 tx=0x7f01540046c0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 shutdown_connections 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:6835/776479349,v1:172.21.15.135:6837/776479349] conn(0x5583236f7280 0x558323705520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x558323705140 0x5583236eaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f01427fc640 0 client.8187 ms_handle_reset on v2:172.21.15.135:6835/776479349 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583236f8160 0x5583236f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 --2- 172.21.15.53:0/514302673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5583236db280 0x558323772f30 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 >> 172.21.15.53:0/514302673 conn(0x5583236dab40 msgr2=0x5583236de4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 shutdown_connections 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 1 -- 172.21.15.53:0/514302673 wait complete. 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016cf9f740 20 client.8187 trim_cache size 0 max 16384 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "message": "", 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0 2024-05-10T10:42:58.752 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:42:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0144076300 msgr2=0x7f01440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0144076300 0x7f01440787c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0154006fd0 tx=0x7f0154008040 comp rx=0 tx=0).stop 2024-05-10T10:42:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01640b8020 msgr2=0x7f01641521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01640b8020 0x7f01641521a0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f014c007010 tx=0x7f014c004730 comp rx=0 tx=0).stop 2024-05-10T10:42:58.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 shutdown_connections 2024-05-10T10:42:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0144076300 0x7f01440787c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01640cc770 0x7f01640ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01640b8020 0x7f01641521a0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 --2- 172.21.15.53:0/453402379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0150004800 0x7f01641f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.749+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 >> 172.21.15.53:0/453402379 conn(0x7f01640d2eb0 msgr2=0x7f016406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.750+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 shutdown_connections 2024-05-10T10:42:58.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.750+0000 7f016a1af640 1 -- 172.21.15.53:0/453402379 wait complete. 2024-05-10T10:42:58.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:42:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.864+0000 7f7ee0b5f640 1 Processor -- start 2024-05-10T10:42:58.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.864+0000 7f7ee0b5f640 1 -- start start 2024-05-10T10:42:58.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ee0b5f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7edc0d5cb0 0x7f7edc0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ee0b5f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7edc0b45f0 con 0x7f7edc162b30 2024-05-10T10:42:58.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ee0b5f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7edc0b48d0 con 0x7f7edc162750 2024-05-10T10:42:58.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ee0b5f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7edc0b4f20 con 0x7f7edc0d5cb0 2024-05-10T10:42:58.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7edad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7edc162b30 0x7f7edc0b53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49258/0 (socket says 172.21.15.53:49258) 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7edad9d640 1 -- 172.21.15.53:0/2289513420 learned_addr learned my addr 172.21.15.53:0/2289513420 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed9d9b640 1 --2- 172.21.15.53:0/2289513420 >> v2:172.21.15.135:3300/0 conn(0x7f7edc0d5cb0 0x7f7edc0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599509381 0 0) 0x7f7edc0b45f0 con 0x7f7edc162b30 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ec4003520 con 0x7f7edc162b30 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4279409665 0 0) 0x7f7ec4003520 con 0x7f7edc162b30 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 >> v2:172.21.15.135:3300/0 conn(0x7f7edc0d5cb0 msgr2=0x7f7edc0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 --2- 172.21.15.53:0/2289513420 >> v2:172.21.15.135:3300/0 conn(0x7f7edc0d5cb0 0x7f7edc0d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 >> v1:172.21.15.135:6789/0 conn(0x7f7edc162750 legacy=0x7f7edc0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc0b4bb0 con 0x7f7edc162b30 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f7ed40019f0 con 0x7f7edc162b30 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 >> v1:172.21.15.53:6789/0 conn(0x7f7edc162b30 legacy=0x7f7edc0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.865+0000 7f7ed959a640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec40039b0 0x7f7ec4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed959a640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4004800 0x7f7ec4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed959a640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec40051c0 0x7f7ec400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0b4f20 con 0x7f7ec4004800 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec400c160 con 0x7f7ec40051c0 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec400c360 con 0x7f7ec40039b0 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed9d9b640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4004800 0x7f7ec4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7eda59c640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec40039b0 0x7f7ec4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7edad9d640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec40051c0 0x7f7ec400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed9d9b640 1 -- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec40039b0 msgr2=0x7f7ec4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed9d9b640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec40039b0 0x7f7ec4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed9d9b640 1 -- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec40051c0 msgr2=0x7f7ec400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed9d9b640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec40051c0 0x7f7ec400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7ed9d9b640 1 -- 172.21.15.53:0/2289513420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec4003520 con 0x7f7ec4004800 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7edad9d640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec40051c0 0x7f7ec400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.866+0000 7f7edad9d640 1 -- 172.21.15.53:0/2289513420 reap_dead start 2024-05-10T10:42:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ed9d9b640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4004800 0x7f7ec4004c80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7edc06cd40 tx=0x7f7ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46dc332f2a0dfb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0036070 con 0x7f7ec4004800 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0031400 con 0x7f7ec4004800 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ed959a640 1 -- 172.21.15.53:0/2289513420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0009690 con 0x7f7ec4004800 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/2289513420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4004800 msgr2=0x7f7ec4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4004800 0x7f7ec4004c80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7edc06cd40 tx=0x7f7ec4005830 comp rx=0 tx=0).stop 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/2289513420 shutdown_connections 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/2289513420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4004800 0x7f7ec4004c80 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/2289513420 >> 172.21.15.53:0/2289513420 conn(0x7f7edc09e3b0 msgr2=0x7f7edc0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/2289513420 shutdown_connections 2024-05-10T10:42:58.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.867+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/2289513420 wait complete. 2024-05-10T10:42:58.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.869+0000 7f7ee0b5f640 1 Processor -- start 2024-05-10T10:42:58.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.869+0000 7f7ee0b5f640 1 -- start start 2024-05-10T10:42:58.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec4004800 0x7f7edc1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7edc158cf0 0x7f7edc0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7edc158310 0x7f7edc15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0d0a00 con 0x7f7edc158310 2024-05-10T10:42:58.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0d0880 con 0x7f7edc158cf0 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc0d0b80 con 0x7f7ec4004800 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7edc158310 0x7f7edc15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7edc158310 0x7f7edc15a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45622/0 (socket says 172.21.15.53:45622) 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 -- 172.21.15.53:0/262286073 learned_addr learned my addr 172.21.15.53:0/262286073 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ed9d9b640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7edc158cf0 0x7f7edc0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7eda59c640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec4004800 0x7f7edc1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 -- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec4004800 msgr2=0x7f7edc1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec4004800 0x7f7edc1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 -- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7edc158cf0 msgr2=0x7f7edc0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7edc158cf0 0x7f7edc0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 -- 172.21.15.53:0/262286073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc0cb500 con 0x7f7edc158310 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7eda59c640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec4004800 0x7f7edc1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7edad9d640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7edc158310 0x7f7edc15a810 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0d41b0 tx=0x7f7ed400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ecaffd640 1 -- 172.21.15.53:0/262286073 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400eb30 con 0x7f7edc158310 2024-05-10T10:42:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ecaffd640 1 -- 172.21.15.53:0/262286073 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed4018070 con 0x7f7edc158310 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc0c9f20 con 0x7f7edc158310 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ecaffd640 1 -- 172.21.15.53:0/262286073 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4013420 con 0x7f7edc158310 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.870+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc0ca4e0 con 0x7f7edc158310 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.871+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7edc068c40 con 0x7f7edc158310 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.872+0000 7f7ecaffd640 1 -- 172.21.15.53:0/262286073 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7ed40135c0 con 0x7f7edc158310 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.872+0000 7f7ecaffd640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7ebc0762d0 0x7f7ebc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.873+0000 7f7ecaffd640 1 -- 172.21.15.53:0/262286073 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f7ed40979c0 con 0x7f7edc158310 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.873+0000 7f7eda59c640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7ebc0762d0 0x7f7ebc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:42:58.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.873+0000 7f7eda59c640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7ebc0762d0 0x7f7ebc078790 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0b4460 tx=0x7f7ecc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:42:58.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:58.877+0000 7f7ecaffd640 1 -- 172.21.15.53:0/262286073 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7edc068c40 con 0x7f7edc158310 2024-05-10T10:42:59.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.107+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7edc0b3c90 con 0x7f7edc158310 2024-05-10T10:42:59.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.107+0000 7f7ecaffd640 1 -- 172.21.15.53:0/262286073 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7f7ed4099e20 con 0x7f7edc158310 2024-05-10T10:42:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7ebc0762d0 msgr2=0x7f7ebc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7ebc0762d0 0x7f7ebc078790 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0b4460 tx=0x7f7ecc009040 comp rx=0 tx=0).stop 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7edc158310 msgr2=0x7f7edc15a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7edc158310 0x7f7edc15a810 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0d41b0 tx=0x7f7ed400e530 comp rx=0 tx=0).stop 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 shutdown_connections 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7ebc0762d0 0x7f7ebc078790 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7edc158310 0x7f7edc15a810 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7edc158cf0 0x7f7edc0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 --2- 172.21.15.53:0/262286073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ec4004800 0x7f7edc1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:42:59.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 >> 172.21.15.53:0/262286073 conn(0x7f7edc09e3b0 msgr2=0x7f7edc0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:42:59.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 shutdown_connections 2024-05-10T10:42:59.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59.109+0000 7f7ee0b5f640 1 -- 172.21.15.53:0/262286073 wait complete. 2024-05-10T10:42:59.121 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:42:59.122 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:42:59.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.122 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:42:59.139 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42190sec preferred_lft 42190sec 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:42:59.140 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:42:59.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.142 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:42:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:42:59.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.205 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:42:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:42:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:42:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:42:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:42:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:42:59.262 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:42:59.263 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:42:59.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:42:59.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:42:59.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:42:59.263 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:42:59.352 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:42:59.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:59.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.392 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:42:59.408 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:42:59.408 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:42:59.408 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:42:59.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:42:59.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:42:59.474 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:59.530 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:42:59.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:42:59.530 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:59.586 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:42:59.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:59.671 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:59.672 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:42:59.672 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:42:59.680 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:59.681 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:42:59.681 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:42:59.681 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:42:59.681 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:42:59.688 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:42:59.688 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:42:59.688 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:42:59.688 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:42:59.689 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:42:59.690 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:42:59.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:42:59.716 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:42:59.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:42:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:42:59.816 INFO:teuthology.orchestra.run.smithi053.stdout:client8196 2024-05-10T10:42:59.816 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:42:59.816 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:42:59.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.833 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-05-10T10:42:59.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:42:59.894 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-05-10T10:42:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:8+0 records in 2024-05-10T10:42:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:8+0 records out 2024-05-10T10:42:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0231745 s, 362 MB/s 2024-05-10T10:42:59.977 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:42:59.977 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:42:59.977 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:42:59.977 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:42:59.977 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-10T10:42:59.978 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:00.084 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627779 2024-05-10T10:43:00.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:00.093 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:00.093 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:00.094 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:00.094 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:00.094 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-10T10:43:00.094 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:00.201 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627780 2024-05-10T10:43:00.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:00.210 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:00.210 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:00.267 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:00.267 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:00.325 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:43:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.468+0000 7fa140ee8640 1 Processor -- start 2024-05-10T10:43:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.469+0000 7fa140ee8640 1 -- start start 2024-05-10T10:43:00.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.469+0000 7fa140ee8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa13c15f3b0 0x7fa13c0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:00.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.469+0000 7fa140ee8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c15f790 0x7fa13c0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:00.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.469+0000 7fa140ee8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13c0d30b0 con 0x7fa13c0d5d30 2024-05-10T10:43:00.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.469+0000 7fa140ee8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa13c0d3390 con 0x7fa13c15f3b0 2024-05-10T10:43:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.469+0000 7fa140ee8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c0c8c80 con 0x7fa13c15f790 2024-05-10T10:43:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa13a59c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa13c15f3b0 0x7fa13c0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa139d9b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c15f790 0x7fa13c0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa13ad9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa13c0d5d30 0x7fa13c0c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:40080/0 (socket says 172.21.15.53:40080) 2024-05-10T10:43:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa13ad9d640 1 -- 172.21.15.53:0/987332397 learned_addr learned my addr 172.21.15.53:0/987332397 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa139d9b640 1 -- 172.21.15.53:0/987332397 >> v2:172.21.15.135:3300/0 conn(0x7fa13c15f3b0 msgr2=0x7fa13c0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa139d9b640 1 --2- 172.21.15.53:0/987332397 >> v2:172.21.15.135:3300/0 conn(0x7fa13c15f3b0 0x7fa13c0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa139d9b640 1 -- 172.21.15.53:0/987332397 >> v1:172.21.15.135:6789/0 conn(0x7fa13c0d5d30 legacy=0x7fa13c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa139d9b640 1 -- 172.21.15.53:0/987332397 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c0c8f60 con 0x7fa13c15f790 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.470+0000 7fa13a59c640 1 --2- 172.21.15.53:0/987332397 >> v2:172.21.15.135:3300/0 conn(0x7fa13c15f3b0 0x7fa13c0d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.471+0000 7fa139d9b640 1 --2- 172.21.15.53:0/987332397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c15f790 0x7fa13c0d3c30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa1240097a0 tx=0x7fa12402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615d14cdc68ed804 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.471+0000 7fa13959a640 1 -- 172.21.15.53:0/987332397 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124037070 con 0x7fa13c15f790 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.471+0000 7fa13959a640 1 -- 172.21.15.53:0/987332397 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124032440 con 0x7fa13c15f790 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.471+0000 7fa13959a640 1 -- 172.21.15.53:0/987332397 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124032810 con 0x7fa13c15f790 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 -- 172.21.15.53:0/987332397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c15f790 msgr2=0x7fa13c0d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 --2- 172.21.15.53:0/987332397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c15f790 0x7fa13c0d3c30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa1240097a0 tx=0x7fa12402a180 comp rx=0 tx=0).stop 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 -- 172.21.15.53:0/987332397 shutdown_connections 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 --2- 172.21.15.53:0/987332397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c15f790 0x7fa13c0d3c30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 --2- 172.21.15.53:0/987332397 >> v2:172.21.15.135:3300/0 conn(0x7fa13c15f3b0 0x7fa13c0d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 -- 172.21.15.53:0/987332397 >> 172.21.15.53:0/987332397 conn(0x7fa13c0d5130 msgr2=0x7fa13c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 -- 172.21.15.53:0/987332397 shutdown_connections 2024-05-10T10:43:00.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.472+0000 7fa140ee8640 1 -- 172.21.15.53:0/987332397 wait complete. 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.473+0000 7fa140ee8640 1 Processor -- start 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.473+0000 7fa140ee8640 1 -- start start 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa140ee8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa13c0d5d30 0x7fa13c154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa140ee8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa13c15f3b0 0x7fa13c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa140ee8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c0ce910 0x7fa13c15fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa140ee8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c0d0e60 con 0x7fa13c15f3b0 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa140ee8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c0d0ce0 con 0x7fa13c0d5d30 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa140ee8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c0d0fe0 con 0x7fa13c0ce910 2024-05-10T10:43:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa13c15f3b0 0x7fa13c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa13c15f3b0 0x7fa13c0ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45630/0 (socket says 172.21.15.53:45630) 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 -- 172.21.15.53:0/3611989108 learned_addr learned my addr 172.21.15.53:0/3611989108 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa13ad9d640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c0ce910 0x7fa13c15fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa13a59c640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa13c0d5d30 0x7fa13c154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 -- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c0ce910 msgr2=0x7fa13c15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c0ce910 0x7fa13c15fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 -- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa13c0d5d30 msgr2=0x7fa13c154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa13c0d5d30 0x7fa13c154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.474+0000 7fa139d9b640 1 -- 172.21.15.53:0/3611989108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c155260 con 0x7fa13c15f3b0 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa13ad9d640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c0ce910 0x7fa13c15fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa13a59c640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa13c0d5d30 0x7fa13c154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa139d9b640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa13c15f3b0 0x7fa13c0ce3d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa124035040 tx=0x7fa1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa132ffd640 1 -- 172.21.15.53:0/3611989108 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124037040 con 0x7fa13c15f3b0 2024-05-10T10:43:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa132ffd640 1 -- 172.21.15.53:0/3611989108 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124030500 con 0x7fa13c15f3b0 2024-05-10T10:43:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c155540 con 0x7fa13c15f3b0 2024-05-10T10:43:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa132ffd640 1 -- 172.21.15.53:0/3611989108 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124030910 con 0x7fa13c15f3b0 2024-05-10T10:43:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.475+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c155b00 con 0x7fa13c15f3b0 2024-05-10T10:43:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.476+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c06a220 con 0x7fa13c15f3b0 2024-05-10T10:43:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.477+0000 7fa132ffd640 1 -- 172.21.15.53:0/3611989108 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa124044020 con 0x7fa13c15f3b0 2024-05-10T10:43:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.477+0000 7fa132ffd640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa11c0762d0 0x7fa11c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.478+0000 7fa132ffd640 1 -- 172.21.15.53:0/3611989108 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa1240b84f0 con 0x7fa13c15f3b0 2024-05-10T10:43:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.478+0000 7fa13a59c640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa11c0762d0 0x7fa11c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.478+0000 7fa13a59c640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa11c0762d0 0x7fa11c078790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009ba0 tx=0x7fa12c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:00.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.481+0000 7fa132ffd640 1 -- 172.21.15.53:0/3611989108 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa124081d30 con 0x7fa13c15f3b0 2024-05-10T10:43:00.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.688+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa13c0cb410 con 0x7fa13c15f3b0 2024-05-10T10:43:00.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.688+0000 7fa132ffd640 1 -- 172.21.15.53:0/3611989108 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7fa124081630 con 0x7fa13c15f3b0 2024-05-10T10:43:00.690 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa11c0762d0 msgr2=0x7fa11c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa11c0762d0 0x7fa11c078790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009ba0 tx=0x7fa12c008040 comp rx=0 tx=0).stop 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa13c15f3b0 msgr2=0x7fa13c0ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa13c15f3b0 0x7fa13c0ce3d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa124035040 tx=0x7fa1240046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 shutdown_connections 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa11c0762d0 0x7fa11c078790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa13c0ce910 0x7fa13c15fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa13c15f3b0 0x7fa13c0ce3d0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 --2- 172.21.15.53:0/3611989108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa13c0d5d30 0x7fa13c154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:00.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 >> 172.21.15.53:0/3611989108 conn(0x7fa13c0d5130 msgr2=0x7fa13c0ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:00.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 shutdown_connections 2024-05-10T10:43:00.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:00.690+0000 7fa140ee8640 1 -- 172.21.15.53:0/3611989108 wait complete. 2024-05-10T10:43:00.703 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:43:00.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:00.703 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:00.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:00.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:00.777 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:43:00.803 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:00.804 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:00.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:43:00.860 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:00.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:00.860 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:43:00.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:43:00.917 INFO:teuthology.orchestra.run.smithi053.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-10T10:43:00.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:43:01.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.060+0000 7fb22b5b6640 1 Processor -- start 2024-05-10T10:43:01.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.060+0000 7fb22b5b6640 1 -- start start 2024-05-10T10:43:01.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb22b5b6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb22415f620 0x7fb22415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb22b5b6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2240704c0 0x7fb224070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb22b5b6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb224160190 con 0x7fb22406c990 2024-05-10T10:43:01.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb22b5b6640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb22406cd70 con 0x7fb2240704c0 2024-05-10T10:43:01.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb22b5b6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb224160470 con 0x7fb22415f620 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb229b53640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2240704c0 0x7fb224070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb228b51640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb22415f620 0x7fb22415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb229352640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb22406c990 0x7fb224163660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:40096/0 (socket says 172.21.15.53:40096) 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb229352640 1 -- 172.21.15.53:0/356245129 learned_addr learned my addr 172.21.15.53:0/356245129 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb21bfff640 1 -- 172.21.15.53:0/356245129 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1134992384 0 0) 0x7fb224160190 con 0x7fb22406c990 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb21bfff640 1 -- 172.21.15.53:0/356245129 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb210003520 con 0x7fb22406c990 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb228b51640 1 -- 172.21.15.53:0/356245129 >> v2:172.21.15.135:3300/0 conn(0x7fb2240704c0 msgr2=0x7fb224070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb228b51640 1 --2- 172.21.15.53:0/356245129 >> v2:172.21.15.135:3300/0 conn(0x7fb2240704c0 0x7fb224070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb228b51640 1 -- 172.21.15.53:0/356245129 >> v1:172.21.15.135:6789/0 conn(0x7fb22406c990 legacy=0x7fb224163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb228b51640 1 -- 172.21.15.53:0/356245129 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb224162e60 con 0x7fb22415f620 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.061+0000 7fb229b53640 1 --2- 172.21.15.53:0/356245129 >> v2:172.21.15.135:3300/0 conn(0x7fb2240704c0 0x7fb224070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.062+0000 7fb228b51640 1 --2- 172.21.15.53:0/356245129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb22415f620 0x7fb22415fa80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2140097a0 tx=0x7fb21402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a19ad268f2383a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.062+0000 7fb21bfff640 1 -- 172.21.15.53:0/356245129 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214037070 con 0x7fb22415f620 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.062+0000 7fb21bfff640 1 -- 172.21.15.53:0/356245129 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb214032440 con 0x7fb22415f620 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.062+0000 7fb21bfff640 1 -- 172.21.15.53:0/356245129 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140327f0 con 0x7fb22415f620 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 -- 172.21.15.53:0/356245129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb22415f620 msgr2=0x7fb22415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/356245129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb22415f620 0x7fb22415fa80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2140097a0 tx=0x7fb21402a180 comp rx=0 tx=0).stop 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 -- 172.21.15.53:0/356245129 shutdown_connections 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/356245129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb22415f620 0x7fb22415fa80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/356245129 >> v2:172.21.15.135:3300/0 conn(0x7fb2240704c0 0x7fb224070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 -- 172.21.15.53:0/356245129 >> 172.21.15.53:0/356245129 conn(0x7fb2240b3980 msgr2=0x7fb2240b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 -- 172.21.15.53:0/356245129 shutdown_connections 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.063+0000 7fb22b5b6640 1 -- 172.21.15.53:0/356245129 wait complete. 2024-05-10T10:43:01.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.064+0000 7fb22b5b6640 1 Processor -- start 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.064+0000 7fb22b5b6640 1 -- start start 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb22b5b6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb22406c990 0x7fb2240b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb22b5b6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2240704c0 0x7fb2241f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb22b5b6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb22415f620 0x7fb2240c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb22b5b6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb224073e70 con 0x7fb22415f620 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb22b5b6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb224070e90 con 0x7fb22406c990 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb22b5b6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb2240cf920 con 0x7fb2240704c0 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb229b53640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb22415f620 0x7fb2240c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb229352640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb22406c990 0x7fb2240b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb228b51640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2240704c0 0x7fb2241f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb229b53640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb22415f620 0x7fb2240c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45640/0 (socket says 172.21.15.53:45640) 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb229b53640 1 -- 172.21.15.53:0/1456011698 learned_addr learned my addr 172.21.15.53:0/1456011698 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb229352640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb22406c990 0x7fb2240b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:57394/0 (socket says 172.21.15.53:57394) 2024-05-10T10:43:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb228b51640 1 -- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb22406c990 msgr2=0x7fb2240b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb228b51640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb22406c990 0x7fb2240b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb228b51640 1 -- 172.21.15.53:0/1456011698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb22415f620 msgr2=0x7fb2240c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb228b51640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb22415f620 0x7fb2240c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb228b51640 1 -- 172.21.15.53:0/1456011698 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2240d2b40 con 0x7fb2240704c0 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.065+0000 7fb228b51640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2240704c0 0x7fb2241f4220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb214031000 tx=0x7fb2140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.066+0000 7fb219ffb640 1 -- 172.21.15.53:0/1456011698 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214037040 con 0x7fb2240704c0 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.066+0000 7fb219ffb640 1 -- 172.21.15.53:0/1456011698 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb214003c90 con 0x7fb2240704c0 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.066+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2240d2860 con 0x7fb2240704c0 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.066+0000 7fb219ffb640 1 -- 172.21.15.53:0/1456011698 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214006e10 con 0x7fb2240704c0 2024-05-10T10:43:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.066+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2240b7110 con 0x7fb2240704c0 2024-05-10T10:43:01.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.067+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22406a220 con 0x7fb2240704c0 2024-05-10T10:43:01.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.067+0000 7fb219ffb640 1 -- 172.21.15.53:0/1456011698 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb214044020 con 0x7fb2240704c0 2024-05-10T10:43:01.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.068+0000 7fb219ffb640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2080762d0 0x7fb208078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.068+0000 7fb219ffb640 1 -- 172.21.15.53:0/1456011698 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fb2140b7680 con 0x7fb2240704c0 2024-05-10T10:43:01.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.068+0000 7fb229352640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2080762d0 0x7fb208078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.069+0000 7fb229352640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2080762d0 0x7fb208078790 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb220005880 tx=0x7fb220009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.073+0000 7fb219ffb640 1 -- 172.21.15.53:0/1456011698 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb21404b020 con 0x7fb2240704c0 2024-05-10T10:43:01.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.302+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb224075220 con 0x7fb2240704c0 2024-05-10T10:43:01.305 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:43:01.303+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:43:01.309 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.308+0000 7fb219ffb640 1 -- 172.21.15.53:0/1456011698 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7fb214080760 con 0x7fb2240704c0 2024-05-10T10:43:01.310 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:43:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2080762d0 msgr2=0x7fb208078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2080762d0 0x7fb208078790 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb220005880 tx=0x7fb220009040 comp rx=0 tx=0).stop 2024-05-10T10:43:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2240704c0 msgr2=0x7fb2241f4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2240704c0 0x7fb2241f4220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb214031000 tx=0x7fb2140046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 shutdown_connections 2024-05-10T10:43:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb2080762d0 0x7fb208078790 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb22415f620 0x7fb2240c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2240704c0 0x7fb2241f4220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 --2- 172.21.15.53:0/1456011698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb22406c990 0x7fb2240b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 >> 172.21.15.53:0/1456011698 conn(0x7fb2240b3980 msgr2=0x7fb2240d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:01.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 shutdown_connections 2024-05-10T10:43:01.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.310+0000 7fb22b5b6640 1 -- 172.21.15.53:0/1456011698 wait complete. 2024-05-10T10:43:01.313 INFO:tasks.ceph.mds.d.smithi135.stderr: -8244> 2024-05-10T10:42:55.846+0000 7f868b251640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:43:01.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:01.326 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:01.346 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.370 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.371 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-05-10T10:43:01.372 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:01.373 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-05-10T10:43:01.374 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.375 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:01.376 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-05-10T10:43:01.377 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:01.378 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: mon down mkfs grace = 300 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.379 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-05-10T10:43:01.380 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = False 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:01.381 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.382 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.382 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-05-10T10:43:01.382 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-05-10T10:43:01.382 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:01.382 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:01.383 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-05-10T10:43:01.384 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-05-10T10:43:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug trim objects = true 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 20 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 20 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 10 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.389 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: mon down mkfs grace = 300 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = "" 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.392 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = True 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2024-05-10T10:43:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = False 2024-05-10T10:43:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.396 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:01.438 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:01.480 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-05-10T10:43:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-05-10T10:43:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-05-10T10:43:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-05-10T10:43:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-05-10T10:43:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-05-10T10:43:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: mon down mkfs grace = 300 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-05-10T10:43:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-05-10T10:43:01.490 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = False 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = False 2024-05-10T10:43:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.492 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-05-10T10:43:01.492 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-05-10T10:43:01.492 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:01.492 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:01.493 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-05-10T10:43:01.494 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:01.495 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-05-10T10:43:01.496 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug trim objects = true 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.497 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 20 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 20 2024-05-10T10:43:01.498 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 10 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:01.499 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:01.500 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:01.501 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: mon down mkfs grace = 300 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:01.502 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = "" 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2024-05-10T10:43:01.503 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = False 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = False 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:01.504 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:01.505 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-05-10T10:43:01.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.588+0000 7f135588cf40 1 Processor -- start 2024-05-10T10:43:01.590 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.588+0000 7f135588cf40 1 -- start start 2024-05-10T10:43:01.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f135588cf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd23f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f135588cf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd242330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f135588cf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd242870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f135588cf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564ddd1d0430 con 0x564ddd1b23f0 2024-05-10T10:43:01.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f135588cf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ddd12ff50 con 0x564ddd25d4c0 2024-05-10T10:43:01.594 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f135588cf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x564ddd1d0f60 con 0x564ddd25d8a0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd23f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd23f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45670/0 (socket says 172.21.15.53:45670) 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 -- 172.21.15.53:0/1756310023 learned_addr learned my addr 172.21.15.53:0/1756310023 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f134ab85640 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd242330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353b87640 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd242870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 -- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 msgr2=0x564ddd242870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd242870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 -- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 msgr2=0x564ddd242330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd242330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.589+0000 7f1353386640 1 -- 172.21.15.53:0/1756310023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ddd253660 con 0x564ddd1b23f0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f1353386640 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd23f620 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f134400a770 tx=0x7f134400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184db73ccb3b8937 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f1352384640 1 -- 172.21.15.53:0/1756310023 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344018070 con 0x564ddd1b23f0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f1352384640 1 -- 172.21.15.53:0/1756310023 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1344007600 con 0x564ddd1b23f0 2024-05-10T10:43:01.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f1352384640 1 -- 172.21.15.53:0/1756310023 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440079b0 con 0x564ddd1b23f0 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 -- 172.21.15.53:0/1756310023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 msgr2=0x564ddd23f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd23f620 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f134400a770 tx=0x7f134400a860 comp rx=0 tx=0).stop 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 -- 172.21.15.53:0/1756310023 shutdown_connections 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd242870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd242330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 --2- 172.21.15.53:0/1756310023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd23f620 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 -- 172.21.15.53:0/1756310023 >> 172.21.15.53:0/1756310023 conn(0x564ddd1b4880 msgr2=0x564ddd186710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 -- 172.21.15.53:0/1756310023 shutdown_connections 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.590+0000 7f135588cf40 1 -- 172.21.15.53:0/1756310023 wait complete. 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.591+0000 7f135588cf40 1 Processor -- start 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.591+0000 7f135588cf40 1 -- start start 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.591+0000 7f135588cf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd238c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f135588cf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd1b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f135588cf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd1b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f135588cf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564ddd2ed030 con 0x564ddd1b23f0 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f135588cf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ddd1b6790 con 0x564ddd25d4c0 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f135588cf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x564ddd242db0 con 0x564ddd25d8a0 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd238c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd238c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45684/0 (socket says 172.21.15.53:45684) 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 -- 172.21.15.53:0/3251384654 learned_addr learned my addr 172.21.15.53:0/3251384654 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353b87640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd1b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f134ab85640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd1b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 -- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 msgr2=0x564ddd1b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd1b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 -- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 msgr2=0x564ddd1b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd1b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 -- 172.21.15.53:0/3251384654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ddd1b3380 con 0x564ddd1b23f0 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353b87640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x564ddd25d8a0 0x564ddd1b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:01.597 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f134ab85640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ddd25d4c0 0x564ddd1b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:01.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1353386640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd238c20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1344000c00 tx=0x7f1344003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1351382640 1 -- 172.21.15.53:0/3251384654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344021020 con 0x564ddd1b23f0 2024-05-10T10:43:01.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1351382640 1 -- 172.21.15.53:0/3251384654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1344019040 con 0x564ddd1b23f0 2024-05-10T10:43:01.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ddd1869a0 con 0x564ddd1b23f0 2024-05-10T10:43:01.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f1351382640 1 -- 172.21.15.53:0/3251384654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344018070 con 0x564ddd1b23f0 2024-05-10T10:43:01.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.592+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ddd1b3640 con 0x564ddd1b23f0 2024-05-10T10:43:01.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.594+0000 7f1351382640 1 -- 172.21.15.53:0/3251384654 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1344024020 con 0x564ddd1b23f0 2024-05-10T10:43:01.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.594+0000 7f1351382640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1334074cb0 0x7f1334077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.594+0000 7f1351382640 1 -- 172.21.15.53:0/3251384654 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f1344097240 con 0x564ddd1b23f0 2024-05-10T10:43:01.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.594+0000 7f135588cf40 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x564ddd2422f0 0x564ddd259420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.594+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x564ddd23dbf0 con 0x564ddd2422f0 2024-05-10T10:43:01.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.595+0000 7f134ab85640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1334074cb0 0x7f1334077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.595+0000 7f1353b87640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x564ddd2422f0 0x564ddd259420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.595+0000 7f1353b87640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x564ddd2422f0 0x564ddd259420 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.600 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.595+0000 7f134ab85640 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1334074cb0 0x7f1334077170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f133c056000 tx=0x7f133c050000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.598+0000 7f1353b87640 1 -- 172.21.15.53:0/3251384654 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v119'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f134c001630 con 0x564ddd2422f0 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.598+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x564ddd2422f0 msgr2=0x564ddd259420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.598+0000 7f135588cf40 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x564ddd2422f0 0x564ddd259420 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1334074cb0 msgr2=0x7f1334077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1334074cb0 0x7f1334077170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f133c056000 tx=0x7f133c050000 comp rx=0 tx=0).stop 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 msgr2=0x564ddd238c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 --2- 172.21.15.53:0/3251384654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ddd1b23f0 0x564ddd238c20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1344000c00 tx=0x7f1344003020 comp rx=0 tx=0).stop 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f1353b87640 1 -- 172.21.15.53:0/3251384654 reap_dead start 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 shutdown_connections 2024-05-10T10:43:01.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 >> 172.21.15.53:0/3251384654 conn(0x564ddd1b4880 msgr2=0x564ddd240af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:01.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 shutdown_connections 2024-05-10T10:43:01.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.599+0000 7f135588cf40 1 -- 172.21.15.53:0/3251384654 wait complete. 2024-05-10T10:43:01.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-10T10:43:01.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.707+0000 7ff594045640 1 Processor -- start 2024-05-10T10:43:01.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.708+0000 7ff594045640 1 -- start start 2024-05-10T10:43:01.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.708+0000 7ff594045640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff58c0d1a50 0x7ff58c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.708+0000 7ff594045640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0c9cc0 0x7ff58c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.708+0000 7ff594045640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58c0d07d0 con 0x7ff58c0b6140 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.708+0000 7ff594045640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff58c0d0ab0 con 0x7ff58c0d1a50 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.708+0000 7ff594045640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c0d0d90 con 0x7ff58c0c9cc0 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.709+0000 7ff591de1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff58c0d1a50 0x7ff58c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.709+0000 7ff591de1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff58c0d1a50 0x7ff58c15f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45686/0 (socket says 172.21.15.53:45686) 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.709+0000 7ff591de1640 1 -- 172.21.15.53:0/2376807002 learned_addr learned my addr 172.21.15.53:0/2376807002 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.709+0000 7ff591de1640 1 -- 172.21.15.53:0/2376807002 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0c9cc0 msgr2=0x7ff58c15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.709+0000 7ff591de1640 1 --2- 172.21.15.53:0/2376807002 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0c9cc0 0x7ff58c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.709+0000 7ff591de1640 1 -- 172.21.15.53:0/2376807002 >> v1:172.21.15.135:6789/0 conn(0x7ff58c0b6140 legacy=0x7ff58c0d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:01.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.709+0000 7ff591de1640 1 -- 172.21.15.53:0/2376807002 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c0d5eb0 con 0x7ff58c0d1a50 2024-05-10T10:43:01.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff591de1640 1 --2- 172.21.15.53:0/2376807002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c15f190 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ff588009870 tx=0x7ff58802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0280dfceac04a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff590ddf640 1 -- 172.21.15.53:0/2376807002 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588037070 con 0x7ff58c0d1a50 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff590ddf640 1 -- 172.21.15.53:0/2376807002 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff588032440 con 0x7ff58c0d1a50 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff590ddf640 1 -- 172.21.15.53:0/2376807002 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588032810 con 0x7ff58c0d1a50 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 -- 172.21.15.53:0/2376807002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 msgr2=0x7ff58c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 --2- 172.21.15.53:0/2376807002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c15f190 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ff588009870 tx=0x7ff58802a210 comp rx=0 tx=0).stop 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 -- 172.21.15.53:0/2376807002 shutdown_connections 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 --2- 172.21.15.53:0/2376807002 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0c9cc0 0x7ff58c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 --2- 172.21.15.53:0/2376807002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c15f190 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 -- 172.21.15.53:0/2376807002 >> 172.21.15.53:0/2376807002 conn(0x7ff58c0d5080 msgr2=0x7ff58c0d5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 -- 172.21.15.53:0/2376807002 shutdown_connections 2024-05-10T10:43:01.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.710+0000 7ff594045640 1 -- 172.21.15.53:0/2376807002 wait complete. 2024-05-10T10:43:01.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 Processor -- start 2024-05-10T10:43:01.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 -- start start 2024-05-10T10:43:01.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0b6140 0x7ff58c0cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff58c0c9cc0 0x7ff58c1f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c0b6810 con 0x7ff58c0d1a50 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c0b6690 con 0x7ff58c0c9cc0 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff594045640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c0d1e30 con 0x7ff58c0b6140 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.712+0000 7ff5925e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c0d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45696/0 (socket says 172.21.15.53:45696) 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 -- 172.21.15.53:0/4215824959 learned_addr learned my addr 172.21.15.53:0/4215824959 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 -- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0b6140 msgr2=0x7ff58c0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0b6140 0x7ff58c0cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 -- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff58c0c9cc0 msgr2=0x7ff58c1f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff58c0c9cc0 0x7ff58c1f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 -- 172.21.15.53:0/4215824959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c0b95e0 con 0x7ff58c0d1a50 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5925e2640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c0d3440 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7ff58400b9e0 tx=0x7ff58400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5827fc640 1 -- 172.21.15.53:0/4215824959 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584019070 con 0x7ff58c0d1a50 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5827fc640 1 -- 172.21.15.53:0/4215824959 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58400cb20 con 0x7ff58c0d1a50 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c1f8a40 con 0x7ff58c0d1a50 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff5827fc640 1 -- 172.21.15.53:0/4215824959 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584014440 con 0x7ff58c0d1a50 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.713+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1f8fc0 con 0x7ff58c0d1a50 2024-05-10T10:43:01.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.714+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c069470 con 0x7ff58c0d1a50 2024-05-10T10:43:01.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.715+0000 7ff5827fc640 1 -- 172.21.15.53:0/4215824959 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff58401c070 con 0x7ff58c0d1a50 2024-05-10T10:43:01.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.715+0000 7ff5827fc640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5700762d0 0x7ff570078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:01.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.716+0000 7ff5827fc640 1 -- 172.21.15.53:0/4215824959 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7ff584097590 con 0x7ff58c0d1a50 2024-05-10T10:43:01.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.716+0000 7ff591de1640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5700762d0 0x7ff570078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:01.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.716+0000 7ff591de1640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5700762d0 0x7ff570078790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff588034000 tx=0x7ff58802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:01.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.720+0000 7ff5827fc640 1 -- 172.21.15.53:0/4215824959 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff584060c70 con 0x7ff58c0d1a50 2024-05-10T10:43:01.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:01.953+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff58c162600 con 0x7ff58c0d1a50 2024-05-10T10:43:02.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.306+0000 7ff5827fc640 1 -- 172.21.15.53:0/4215824959 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v100) ==== 162+0+0 (secure 0 0 0) 0x7ff584060570 con 0x7ff58c0d1a50 2024-05-10T10:43:02.308 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-10T10:43:02.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5700762d0 msgr2=0x7ff570078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5700762d0 0x7ff570078790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff588034000 tx=0x7ff58802c000 comp rx=0 tx=0).stop 2024-05-10T10:43:02.310 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 msgr2=0x7ff58c0d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c0d3440 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7ff58400b9e0 tx=0x7ff58400bea0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 shutdown_connections 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff5700762d0 0x7ff570078790 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c0d1a50 0x7ff58c0d3440 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff58c0c9cc0 0x7ff58c1f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.308+0000 7ff594045640 1 --2- 172.21.15.53:0/4215824959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58c0b6140 0x7ff58c0cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.309+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 >> 172.21.15.53:0/4215824959 conn(0x7ff58c0d5080 msgr2=0x7ff58c06cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.309+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 shutdown_connections 2024-05-10T10:43:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.309+0000 7ff594045640 1 -- 172.21.15.53:0/4215824959 wait complete. 2024-05-10T10:43:02.325 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:02.432 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.431+0000 7efee3e20640 1 Processor -- start 2024-05-10T10:43:02.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.431+0000 7efee3e20640 1 -- start start 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee3e20640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efedc15f3e0 0x7efedc0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee3e20640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee3e20640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efedc0d0ff0 con 0x7efedc0d0750 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee3e20640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7efedc0d0d10 con 0x7efedc15f3e0 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee3e20640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efedc0b49f0 con 0x7efedc0b54d0 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee1bbc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efedc15f3e0 0x7efedc0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee13bb640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efedc0d0750 0x7efedc0d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49278/0 (socket says 172.21.15.53:49278) 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee13bb640 1 -- 172.21.15.53:0/1341557782 learned_addr learned my addr 172.21.15.53:0/1341557782 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee1bbc640 1 -- 172.21.15.53:0/1341557782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 msgr2=0x7efedc0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee23bd640 1 --2- 172.21.15.53:0/1341557782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee1bbc640 1 --2- 172.21.15.53:0/1341557782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee0bba640 1 -- 172.21.15.53:0/1341557782 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 579640990 0 0) 0x7efedc0d0ff0 con 0x7efedc0d0750 2024-05-10T10:43:02.435 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee1bbc640 1 -- 172.21.15.53:0/1341557782 >> v1:172.21.15.53:6789/0 conn(0x7efedc0d0750 legacy=0x7efedc0d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.432+0000 7efee1bbc640 1 -- 172.21.15.53:0/1341557782 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc0b4cd0 con 0x7efedc15f3e0 2024-05-10T10:43:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.433+0000 7efee1bbc640 1 --2- 172.21.15.53:0/1341557782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0d1a00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efed00097a0 tx=0x7efed002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df63e1faaf8b5a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.433+0000 7efee0bba640 1 -- 172.21.15.53:0/1341557782 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0037070 con 0x7efedc15f3e0 2024-05-10T10:43:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.433+0000 7efee0bba640 1 -- 172.21.15.53:0/1341557782 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed0032440 con 0x7efedc15f3e0 2024-05-10T10:43:02.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.433+0000 7efee0bba640 1 -- 172.21.15.53:0/1341557782 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0032810 con 0x7efedc15f3e0 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 -- 172.21.15.53:0/1341557782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 msgr2=0x7efedc0d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 --2- 172.21.15.53:0/1341557782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0d1a00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efed00097a0 tx=0x7efed002a180 comp rx=0 tx=0).stop 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 -- 172.21.15.53:0/1341557782 shutdown_connections 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 --2- 172.21.15.53:0/1341557782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 --2- 172.21.15.53:0/1341557782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0d1a00 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 -- 172.21.15.53:0/1341557782 >> 172.21.15.53:0/1341557782 conn(0x7efedc0d4ff0 msgr2=0x7efedc0d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 -- 172.21.15.53:0/1341557782 shutdown_connections 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.434+0000 7efee3e20640 1 -- 172.21.15.53:0/1341557782 wait complete. 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.435+0000 7efee3e20640 1 Processor -- start 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.435+0000 7efee3e20640 1 -- start start 2024-05-10T10:43:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee3e20640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee3e20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc0d0750 0x7efedc0b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee3e20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee3e20640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efedc15f7c0 con 0x7efedc15f3e0 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee3e20640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0d3f90 con 0x7efedc0d0750 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee3e20640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7efedc15f940 con 0x7efedc0b54d0 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0ba6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45710/0 (socket says 172.21.15.53:45710) 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 -- 172.21.15.53:0/3388154141 learned_addr learned my addr 172.21.15.53:0/3388154141 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee1bbc640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee13bb640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc0d0750 0x7efedc0b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 -- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 msgr2=0x7efedc0b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 -- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc0d0750 msgr2=0x7efedc0b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc0d0750 0x7efedc0b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.436+0000 7efee23bd640 1 -- 172.21.15.53:0/3388154141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc0b97b0 con 0x7efedc15f3e0 2024-05-10T10:43:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.437+0000 7efee23bd640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0ba6e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efed400cda0 tx=0x7efed400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.437+0000 7efece7fc640 1 -- 172.21.15.53:0/3388154141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4003b90 con 0x7efedc15f3e0 2024-05-10T10:43:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.437+0000 7efece7fc640 1 -- 172.21.15.53:0/3388154141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4003d30 con 0x7efedc15f3e0 2024-05-10T10:43:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.437+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc0bac20 con 0x7efedc15f3e0 2024-05-10T10:43:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.437+0000 7efece7fc640 1 -- 172.21.15.53:0/3388154141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4004240 con 0x7efedc15f3e0 2024-05-10T10:43:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.437+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc0b8790 con 0x7efedc15f3e0 2024-05-10T10:43:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.438+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc069400 con 0x7efedc15f3e0 2024-05-10T10:43:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.439+0000 7efece7fc640 1 -- 172.21.15.53:0/3388154141 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efed40043e0 con 0x7efedc15f3e0 2024-05-10T10:43:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.439+0000 7efece7fc640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efec00762d0 0x7efec0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.439+0000 7efece7fc640 1 -- 172.21.15.53:0/3388154141 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7efed4098200 con 0x7efedc15f3e0 2024-05-10T10:43:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.439+0000 7efee1bbc640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efec00762d0 0x7efec0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:02.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.440+0000 7efee1bbc640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efec00762d0 0x7efec0078790 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efed0034000 tx=0x7efed002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.443+0000 7efece7fc640 1 -- 172.21.15.53:0/3388154141 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7efed40612a0 con 0x7efedc15f3e0 2024-05-10T10:43:02.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.677+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efedc0bae70 con 0x7efedc15f3e0 2024-05-10T10:43:02.679 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.677+0000 7efece7fc640 1 -- 172.21.15.53:0/3388154141 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5173 (secure 0 0 0) 0x7efed40655a0 con 0x7efedc15f3e0 2024-05-10T10:43:02.679 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:02.679 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:02.310464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"c","rank":0,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.53:6834/38461894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":38461894},{"type":"v1","addr":"172.21.15.53:6834","nonce":38461894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-10T10:43:02.679 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 101 2024-05-10T10:43:02.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efec00762d0 msgr2=0x7efec0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efec00762d0 0x7efec0078790 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efed0034000 tx=0x7efed002e000 comp rx=0 tx=0).stop 2024-05-10T10:43:02.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 msgr2=0x7efedc0ba6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:02.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0ba6e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efed400cda0 tx=0x7efed400a660 comp rx=0 tx=0).stop 2024-05-10T10:43:02.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 shutdown_connections 2024-05-10T10:43:02.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7efec00762d0 0x7efec0078790 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efedc15f3e0 0x7efedc0ba6e0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efedc0d0750 0x7efedc0b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 --2- 172.21.15.53:0/3388154141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efedc0b54d0 0x7efedc0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:02.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 >> 172.21.15.53:0/3388154141 conn(0x7efedc0d4ff0 msgr2=0x7efedc06d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:02.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 shutdown_connections 2024-05-10T10:43:02.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:02.679+0000 7efee3e20640 1 -- 172.21.15.53:0/3388154141 wait complete. 2024-05-10T10:43:03.094 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:42:49.598991+0000', 'modified': '2024-05-10T10:43:02.310464+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 119, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8025': {'gid': 8025, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.53:6834/38461894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 38461894}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 38461894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:43:03.095 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8025:up:replay 2024-05-10T10:43:04.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.210+0000 7fa0819f2640 1 Processor -- start 2024-05-10T10:43:04.212 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.211+0000 7fa0819f2640 1 -- start start 2024-05-10T10:43:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.211+0000 7fa0819f2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.211+0000 7fa0819f2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa07c0d3900 0x7fa07c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.211+0000 7fa0819f2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07c0b30d0 con 0x7fa07c0cfbd0 2024-05-10T10:43:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.211+0000 7fa0819f2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa07c0b4690 con 0x7fa07c0d3900 2024-05-10T10:43:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.211+0000 7fa0819f2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa07c0b33b0 con 0x7fa07c06e5d0 2024-05-10T10:43:04.213 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa07c0d3900 0x7fa07c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07a7fc640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07affd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa07c0cfbd0 0x7fa07c06dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:35668/0 (socket says 172.21.15.53:35668) 2024-05-10T10:43:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07affd640 1 -- 172.21.15.53:0/3594683255 learned_addr learned my addr 172.21.15.53:0/3594683255 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa07c0d3900 0x7fa07c0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:54040/0 (socket says 172.21.15.53:54040) 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa079ffb640 1 -- 172.21.15.53:0/3594683255 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2846334107 0 0) 0x7fa07c0b30d0 con 0x7fa07c0cfbd0 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa079ffb640 1 -- 172.21.15.53:0/3594683255 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa064003520 con 0x7fa07c0cfbd0 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07b7fe640 1 -- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 msgr2=0x7fa07c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07b7fe640 1 -- 172.21.15.53:0/3594683255 >> v1:172.21.15.135:6789/0 conn(0x7fa07c0cfbd0 legacy=0x7fa07c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07b7fe640 1 -- 172.21.15.53:0/3594683255 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c0b3690 con 0x7fa07c0d3900 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.212+0000 7fa07a7fc640 1 --2- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.213+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0d3900 0x7fa07c0b4150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0740097a0 tx=0x7fa07402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29677253e768b60d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.213+0000 7fa079ffb640 1 -- 172.21.15.53:0/3594683255 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074037070 con 0x7fa07c0d3900 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.213+0000 7fa079ffb640 1 -- 172.21.15.53:0/3594683255 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa074032440 con 0x7fa07c0d3900 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.213+0000 7fa079ffb640 1 -- 172.21.15.53:0/3594683255 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074032810 con 0x7fa07c0d3900 2024-05-10T10:43:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 -- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0d3900 msgr2=0x7fa07c0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0d3900 0x7fa07c0b4150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0740097a0 tx=0x7fa07402a180 comp rx=0 tx=0).stop 2024-05-10T10:43:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 -- 172.21.15.53:0/3594683255 shutdown_connections 2024-05-10T10:43:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3594683255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0d3900 0x7fa07c0b4150 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 -- 172.21.15.53:0/3594683255 >> 172.21.15.53:0/3594683255 conn(0x7fa07c0d2d70 msgr2=0x7fa07c0d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 -- 172.21.15.53:0/3594683255 shutdown_connections 2024-05-10T10:43:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.214+0000 7fa0819f2640 1 -- 172.21.15.53:0/3594683255 wait complete. 2024-05-10T10:43:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.215+0000 7fa0819f2640 1 Processor -- start 2024-05-10T10:43:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.215+0000 7fa0819f2640 1 -- start start 2024-05-10T10:43:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa0819f2640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa0819f2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0cfbd0 0x7fa07c14b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa0819f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0d3900 0x7fa07c14d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:04.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa0819f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0cffb0 con 0x7fa07c0d3900 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa0819f2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c0b3c50 con 0x7fa07c0cfbd0 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa0819f2640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fa07c0d0130 con 0x7fa07c06e5d0 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa07b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0d3900 0x7fa07c14d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa07b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0d3900 0x7fa07c14d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49984/0 (socket says 172.21.15.53:49984) 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa07b7fe640 1 -- 172.21.15.53:0/3852861696 learned_addr learned my addr 172.21.15.53:0/3852861696 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa07affd640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.216+0000 7fa07a7fc640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0cfbd0 0x7fa07c14b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa07b7fe640 1 -- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 msgr2=0x7fa07c162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa07b7fe640 1 -- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0cfbd0 msgr2=0x7fa07c14b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0cfbd0 0x7fa07c14b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa07b7fe640 1 -- 172.21.15.53:0/3852861696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c14dca0 con 0x7fa07c0d3900 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa07a7fc640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0cfbd0 0x7fa07c14b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0d3900 0x7fa07c14d760 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa074035040 tx=0x7fa0740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa063fff640 1 -- 172.21.15.53:0/3852861696 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074041020 con 0x7fa07c0d3900 2024-05-10T10:43:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa063fff640 1 -- 172.21.15.53:0/3852861696 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07403d640 con 0x7fa07c0d3900 2024-05-10T10:43:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa063fff640 1 -- 172.21.15.53:0/3852861696 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074037040 con 0x7fa07c0d3900 2024-05-10T10:43:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07c14c6c0 con 0x7fa07c0d3900 2024-05-10T10:43:04.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.217+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07c14cc80 con 0x7fa07c0d3900 2024-05-10T10:43:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.218+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07c069b10 con 0x7fa07c0d3900 2024-05-10T10:43:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.219+0000 7fa063fff640 1 -- 172.21.15.53:0/3852861696 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa074044020 con 0x7fa07c0d3900 2024-05-10T10:43:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.219+0000 7fa063fff640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0680762d0 0x7fa068078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.219+0000 7fa063fff640 1 -- 172.21.15.53:0/3852861696 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7fa0740b7ea0 con 0x7fa07c0d3900 2024-05-10T10:43:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.220+0000 7fa07affd640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0680762d0 0x7fa068078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:04.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.220+0000 7fa07affd640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0680762d0 0x7fa068078790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa06c005c00 tx=0x7fa06c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:04.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.224+0000 7fa063fff640 1 -- 172.21.15.53:0/3852861696 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fa07404b030 con 0x7fa07c0d3900 2024-05-10T10:43:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.453+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa07c1e9c60 con 0x7fa07c0d3900 2024-05-10T10:43:04.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.454+0000 7fa063fff640 1 -- 172.21.15.53:0/3852861696 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5173 (secure 0 0 0) 0x7fa074081630 con 0x7fa07c0d3900 2024-05-10T10:43:04.456 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:04.456 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:03.317552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"c","rank":0,"incarnation":101,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.53:6834/38461894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":38461894},{"type":"v1","addr":"172.21.15.53:6834","nonce":38461894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-10T10:43:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 103 2024-05-10T10:43:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0680762d0 msgr2=0x7fa068078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0680762d0 0x7fa068078790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa06c005c00 tx=0x7fa06c009040 comp rx=0 tx=0).stop 2024-05-10T10:43:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0d3900 msgr2=0x7fa07c14d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0d3900 0x7fa07c14d760 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa074035040 tx=0x7fa0740046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 shutdown_connections 2024-05-10T10:43:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fa0680762d0 0x7fa068078790 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0d3900 0x7fa07c14d760 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa07c0cfbd0 0x7fa07c14b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 --2- 172.21.15.53:0/3852861696 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa07c06e5d0 0x7fa07c162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 >> 172.21.15.53:0/3852861696 conn(0x7fa07c0d2d70 msgr2=0x7fa07c0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 shutdown_connections 2024-05-10T10:43:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:04.456+0000 7fa0819f2640 1 -- 172.21.15.53:0/3852861696 wait complete. 2024-05-10T10:43:04.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:42:49.598991+0000', 'modified': '2024-05-10T10:43:03.317552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 119, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8025': {'gid': 8025, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.53:6834/38461894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 38461894}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 38461894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:43:04.871 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8025:up:rejoin 2024-05-10T10:43:05.846 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:43:05.845+0000 7fcd1b901640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:43:05.872 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:05.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.987+0000 7f1bccf24640 1 Processor -- start 2024-05-10T10:43:05.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.987+0000 7f1bccf24640 1 -- start start 2024-05-10T10:43:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.988+0000 7f1bccf24640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1bc80d0750 0x7f1bc80d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.988+0000 7f1bccf24640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc80d0ff0 con 0x7f1bc815f3e0 2024-05-10T10:43:05.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.988+0000 7f1bccf24640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc80d0d10 con 0x7f1bc80b54d0 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.988+0000 7f1bccf24640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc80b49f0 con 0x7f1bc80d0750 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.988+0000 7f1bc5d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1bc80d0750 0x7f1bc80d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.988+0000 7f1bc659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1bc815f3e0 0x7f1bc80d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51138/0 (socket says 172.21.15.53:51138) 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.988+0000 7f1bc659c640 1 -- 172.21.15.53:0/2595629232 learned_addr learned my addr 172.21.15.53:0/2595629232 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.989+0000 7f1bc5d9b640 1 -- 172.21.15.53:0/2595629232 >> v1:172.21.15.135:6789/0 conn(0x7f1bc80b54d0 legacy=0x7f1bc80b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.989+0000 7f1bc559a640 1 -- 172.21.15.53:0/2595629232 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209078103 0 0) 0x7f1bc80d0ff0 con 0x7f1bc815f3e0 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.989+0000 7f1bc5d9b640 1 -- 172.21.15.53:0/2595629232 >> v1:172.21.15.53:6789/0 conn(0x7f1bc815f3e0 legacy=0x7f1bc80d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:05.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.989+0000 7f1bc5d9b640 1 -- 172.21.15.53:0/2595629232 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80b4cd0 con 0x7f1bc80d0750 2024-05-10T10:43:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.989+0000 7f1bc5d9b640 1 --2- 172.21.15.53:0/2595629232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80d0750 0x7f1bc80d2110 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0097a0 tx=0x7f1bbc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d539ecf4c7034907 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bc559a640 1 -- 172.21.15.53:0/2595629232 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc037070 con 0x7f1bc80d0750 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bc559a640 1 -- 172.21.15.53:0/2595629232 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc032440 con 0x7f1bc80d0750 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bc559a640 1 -- 172.21.15.53:0/2595629232 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc032810 con 0x7f1bc80d0750 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bccf24640 1 -- 172.21.15.53:0/2595629232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80d0750 msgr2=0x7f1bc80d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bccf24640 1 --2- 172.21.15.53:0/2595629232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80d0750 0x7f1bc80d2110 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0097a0 tx=0x7f1bbc02a230 comp rx=0 tx=0).stop 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bccf24640 1 -- 172.21.15.53:0/2595629232 shutdown_connections 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bccf24640 1 --2- 172.21.15.53:0/2595629232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80d0750 0x7f1bc80d2110 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bccf24640 1 -- 172.21.15.53:0/2595629232 >> 172.21.15.53:0/2595629232 conn(0x7f1bc80d4ff0 msgr2=0x7f1bc80d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bccf24640 1 -- 172.21.15.53:0/2595629232 shutdown_connections 2024-05-10T10:43:05.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.990+0000 7f1bccf24640 1 -- 172.21.15.53:0/2595629232 wait complete. 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.992+0000 7f1bccf24640 1 Processor -- start 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.992+0000 7f1bccf24640 1 -- start start 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.992+0000 7f1bccf24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80b54d0 0x7f1bc80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.992+0000 7f1bccf24640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1bc80d0750 0x7f1bc80cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bccf24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bc815f3e0 0x7f1bc81f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bccf24640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc815f7c0 con 0x7f1bc80b54d0 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bccf24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80d3f90 con 0x7f1bc815f3e0 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bccf24640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc815f940 con 0x7f1bc80d0750 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80b54d0 0x7f1bc80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:05.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80b54d0 0x7f1bc80cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50004/0 (socket says 172.21.15.53:50004) 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 -- 172.21.15.53:0/1884182159 learned_addr learned my addr 172.21.15.53:0/1884182159 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc5d9b640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1bc80d0750 0x7f1bc80cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc6d9d640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bc815f3e0 0x7f1bc81f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 -- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1bc80d0750 msgr2=0x7f1bc80cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1bc80d0750 0x7f1bc80cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 -- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bc815f3e0 msgr2=0x7f1bc81f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bc815f3e0 0x7f1bc81f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 -- 172.21.15.53:0/1884182159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80ccbd0 con 0x7f1bc80b54d0 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bc659c640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80b54d0 0x7f1bc80cc380 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c3f0 tx=0x7f1bb800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bb6ffd640 1 -- 172.21.15.53:0/1884182159 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80074e0 con 0x7f1bc80b54d0 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bb6ffd640 1 -- 172.21.15.53:0/1884182159 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb80047e0 con 0x7f1bc80b54d0 2024-05-10T10:43:05.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bb6ffd640 1 -- 172.21.15.53:0/1884182159 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8004c10 con 0x7f1bc80b54d0 2024-05-10T10:43:05.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc80cceb0 con 0x7f1bc80b54d0 2024-05-10T10:43:05.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.993+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc80cd470 con 0x7f1bc80b54d0 2024-05-10T10:43:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.995+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc8073ff0 con 0x7f1bc80b54d0 2024-05-10T10:43:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.995+0000 7f1bb6ffd640 1 -- 172.21.15.53:0/1884182159 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1bb8011070 con 0x7f1bc80b54d0 2024-05-10T10:43:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.996+0000 7f1bb6ffd640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ba80762d0 0x7f1ba8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.996+0000 7f1bb6ffd640 1 -- 172.21.15.53:0/1884182159 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f1bb8097330 con 0x7f1bc80b54d0 2024-05-10T10:43:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.996+0000 7f1bc5d9b640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ba80762d0 0x7f1ba8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:05.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:05.996+0000 7f1bc5d9b640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ba80762d0 0x7f1ba8078790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc034000 tx=0x7f1bbc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.000+0000 7f1bb6ffd640 1 -- 172.21.15.53:0/1884182159 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f1bc8073ff0 con 0x7f1bc80b54d0 2024-05-10T10:43:06.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.233+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bc80d4320 con 0x7f1bc80b54d0 2024-05-10T10:43:06.235 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.234+0000 7f1bb6ffd640 1 -- 172.21.15.53:0/1884182159 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5180 (secure 0 0 0) 0x7f1bb8060b40 con 0x7f1bc80b54d0 2024-05-10T10:43:06.236 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:06.236 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:05.316528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6834/38461894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":38461894},{"type":"v1","addr":"172.21.15.53:6834","nonce":38461894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8025,"qdb_cluster":[8025]},"id":8}]} 2024-05-10T10:43:06.236 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 104 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ba80762d0 msgr2=0x7f1ba8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ba80762d0 0x7f1ba8078790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc034000 tx=0x7f1bbc02e000 comp rx=0 tx=0).stop 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80b54d0 msgr2=0x7f1bc80cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80b54d0 0x7f1bc80cc380 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800c3f0 tx=0x7f1bb800c8b0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 shutdown_connections 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f1ba80762d0 0x7f1ba8078790 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bc815f3e0 0x7f1bc81f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1bc80d0750 0x7f1bc80cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 --2- 172.21.15.53:0/1884182159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bc80b54d0 0x7f1bc80cc380 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 >> 172.21.15.53:0/1884182159 conn(0x7f1bc80d4ff0 msgr2=0x7f1bc80d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 shutdown_connections 2024-05-10T10:43:06.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.236+0000 7f1bccf24640 1 -- 172.21.15.53:0/1884182159 wait complete. 2024-05-10T10:43:06.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:42:49.598991+0000', 'modified': '2024-05-10T10:43:05.316528+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 119, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8025': {'gid': 8025, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.53:6834/38461894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 38461894}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 38461894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8025, 'qdb_cluster': [8025]} 2024-05-10T10:43:06.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:43:06.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T10:43:06.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.759+0000 7f171163e640 1 Processor -- start 2024-05-10T10:43:06.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.759+0000 7f171163e640 1 -- start start 2024-05-10T10:43:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f171163e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f170c0b64e0 0x7f170c16a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f171163e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170c0d2820 con 0x7f170c0d0750 2024-05-10T10:43:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f171163e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170c0d0ff0 con 0x7f170c15f3e0 2024-05-10T10:43:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f171163e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f170c0d0d10 con 0x7f170c0b64e0 2024-05-10T10:43:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f170a7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f170c0d0750 0x7f170c0d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51152/0 (socket says 172.21.15.53:51152) 2024-05-10T10:43:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f170a7fc640 1 -- 172.21.15.53:0/2800461945 learned_addr learned my addr 172.21.15.53:0/2800461945 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:06.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f170b7fe640 1 --2- 172.21.15.53:0/2800461945 >> v2:172.21.15.53:3300/0 conn(0x7f170c0b64e0 0x7f170c16a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f1709ffb640 1 -- 172.21.15.53:0/2800461945 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369090219 0 0) 0x7f170c0d2820 con 0x7f170c0d0750 2024-05-10T10:43:06.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.760+0000 7f1709ffb640 1 -- 172.21.15.53:0/2800461945 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16f8003520 con 0x7f170c0d0750 2024-05-10T10:43:06.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.761+0000 7f170b7fe640 1 -- 172.21.15.53:0/2800461945 >> v1:172.21.15.135:6789/0 conn(0x7f170c15f3e0 legacy=0x7f170c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.761+0000 7f170b7fe640 1 -- 172.21.15.53:0/2800461945 >> v1:172.21.15.53:6789/0 conn(0x7f170c0d0750 legacy=0x7f170c0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.761+0000 7f170b7fe640 1 -- 172.21.15.53:0/2800461945 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c0b49f0 con 0x7f170c0b64e0 2024-05-10T10:43:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.761+0000 7f170b7fe640 1 --2- 172.21.15.53:0/2800461945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c0b64e0 0x7f170c16a060 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f17040097a0 tx=0x7f170402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80c6d4dcf89f0a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.761+0000 7f1709ffb640 1 -- 172.21.15.53:0/2800461945 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704037070 con 0x7f170c0b64e0 2024-05-10T10:43:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.761+0000 7f1709ffb640 1 -- 172.21.15.53:0/2800461945 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704032440 con 0x7f170c0b64e0 2024-05-10T10:43:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.761+0000 7f1709ffb640 1 -- 172.21.15.53:0/2800461945 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704032810 con 0x7f170c0b64e0 2024-05-10T10:43:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.762+0000 7f171163e640 1 -- 172.21.15.53:0/2800461945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c0b64e0 msgr2=0x7f170c16a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.762+0000 7f171163e640 1 --2- 172.21.15.53:0/2800461945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c0b64e0 0x7f170c16a060 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f17040097a0 tx=0x7f170402a1e0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.762+0000 7f171163e640 1 -- 172.21.15.53:0/2800461945 shutdown_connections 2024-05-10T10:43:06.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.762+0000 7f171163e640 1 --2- 172.21.15.53:0/2800461945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c0b64e0 0x7f170c16a060 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.762+0000 7f171163e640 1 -- 172.21.15.53:0/2800461945 >> 172.21.15.53:0/2800461945 conn(0x7f170c0d4ff0 msgr2=0x7f170c0d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.762+0000 7f171163e640 1 -- 172.21.15.53:0/2800461945 shutdown_connections 2024-05-10T10:43:06.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.762+0000 7f171163e640 1 -- 172.21.15.53:0/2800461945 wait complete. 2024-05-10T10:43:06.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.763+0000 7f171163e640 1 Processor -- start 2024-05-10T10:43:06.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.763+0000 7f171163e640 1 -- start start 2024-05-10T10:43:06.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f171163e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0b64e0 0x7f170c159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f171163e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d0750 0x7f170c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f171163e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c15f3e0 0x7f170c0ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f171163e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f170c0d3f90 con 0x7f170c15f3e0 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f171163e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f170c0b4fb0 con 0x7f170c0d0750 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f171163e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f170c15f7c0 con 0x7f170c0b64e0 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c15f3e0 0x7f170c0ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c15f3e0 0x7f170c0ce610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50022/0 (socket says 172.21.15.53:50022) 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f170b7fe640 1 -- 172.21.15.53:0/3726837137 learned_addr learned my addr 172.21.15.53:0/3726837137 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f170affd640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0b64e0 0x7f170c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.764+0000 7f170a7fc640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d0750 0x7f170c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f170b7fe640 1 -- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0b64e0 msgr2=0x7f170c159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f170b7fe640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0b64e0 0x7f170c159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f170b7fe640 1 -- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d0750 msgr2=0x7f170c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f170b7fe640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d0750 0x7f170c0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f170b7fe640 1 -- 172.21.15.53:0/3726837137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c159ba0 con 0x7f170c15f3e0 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f170b7fe640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c15f3e0 0x7f170c0ce610 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f17040098c0 tx=0x7f17040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f17037fe640 1 -- 172.21.15.53:0/3726837137 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704037040 con 0x7f170c15f3e0 2024-05-10T10:43:06.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f17037fe640 1 -- 172.21.15.53:0/3726837137 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704042070 con 0x7f170c15f3e0 2024-05-10T10:43:06.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c0b7e40 con 0x7f170c15f3e0 2024-05-10T10:43:06.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f17037fe640 1 -- 172.21.15.53:0/3726837137 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704030860 con 0x7f170c15f3e0 2024-05-10T10:43:06.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.765+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c0b8400 con 0x7f170c15f3e0 2024-05-10T10:43:06.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.767+0000 7f17037fe640 1 -- 172.21.15.53:0/3726837137 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1704030a00 con 0x7f170c15f3e0 2024-05-10T10:43:06.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.767+0000 7f17037fe640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f16ec076300 0x7f16ec0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.772 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.767+0000 7f17037fe640 1 -- 172.21.15.53:0/3726837137 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f17040b80e0 con 0x7f170c15f3e0 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.767+0000 7f170affd640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f16ec076300 0x7f16ec0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.768+0000 7f1714432740 1 Processor -- start 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.768+0000 7f170affd640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f16ec076300 0x7f16ec0787c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f170c1f8e10 tx=0x7f16fc0094d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.768+0000 7f1714432740 1 -- start start 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846dfe9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f1714432740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfb9bb0 0x55846dfbd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f1714432740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55846df238d0 con 0x55846dfb9bb0 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55846dedcc10 con 0x55846dfe7330 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55846df1f890 con 0x55846dfe6f50 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfb9bb0 0x55846dfbd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfb9bb0 0x55846dfbd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50036/0 (socket says 172.21.15.53:50036) 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 -- 172.21.15.53:0/963659553 learned_addr learned my addr 172.21.15.53:0/963659553 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:06.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170affd640 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846dfe9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170a7fc640 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 -- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 msgr2=0x55846dfe9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846dfe9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 -- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 msgr2=0x55846dfe9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 -- 172.21.15.53:0/963659553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846dee45d0 con 0x55846dfb9bb0 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.769+0000 7f170b7fe640 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfb9bb0 0x55846dfbd3c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f170407d750 tx=0x7f170407d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeaa8a73a27b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1700ff9640 1 -- 172.21.15.53:0/963659553 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704081860 con 0x55846dfb9bb0 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1700ff9640 1 -- 172.21.15.53:0/963659553 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704081160 con 0x55846dfb9bb0 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1700ff9640 1 -- 172.21.15.53:0/963659553 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704085510 con 0x55846dfb9bb0 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 -- 172.21.15.53:0/963659553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfb9bb0 msgr2=0x55846dfbd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfb9bb0 0x55846dfbd3c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f170407d750 tx=0x7f170407d150 comp rx=0 tx=0).stop 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 -- 172.21.15.53:0/963659553 shutdown_connections 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfb9bb0 0x55846dfbd3c0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 --2- 172.21.15.53:0/963659553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846dfe9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 -- 172.21.15.53:0/963659553 >> 172.21.15.53:0/963659553 conn(0x55846dfbbe20 msgr2=0x55846e004760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 -- 172.21.15.53:0/963659553 shutdown_connections 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.770+0000 7f1714432740 1 -- 172.21.15.53:0/963659553 wait complete. 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.771+0000 7f1714432740 1 -- start start 2024-05-10T10:43:06.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.771+0000 7f1714432740 1 Processor -- start 2024-05-10T10:43:06.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846e0093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846e00b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f1714432740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e00bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f1714432740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55846dfbefd0 con 0x55846e0072f0 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55846dee0760 con 0x55846dfe7330 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55846dfbf2b0 con 0x55846dfe6f50 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f1714432740 10 client.0 _ll_register_callbacks cb 0x55846df19cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e00bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e00bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50046/0 (socket says 172.21.15.53:50046) 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 -- 172.21.15.53:0/3643865007 learned_addr learned my addr 172.21.15.53:0/3643865007 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170affd640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846e0093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170a7fc640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846e00b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 -- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 msgr2=0x55846e0093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846e0093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 -- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 msgr2=0x55846e00b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846e00b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 -- 172.21.15.53:0/3643865007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846dc352d0 con 0x55846e0072f0 2024-05-10T10:43:06.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170affd640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846e0093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f170b7fe640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e00bd50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f17040b6480 tx=0x7f17040b6300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697b0e07f5c1dac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.772+0000 7f16e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f16e9ffb640 1 -- 172.21.15.53:0/3643865007 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040b4060 con 0x55846e0072f0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f16e9ffb640 1 -- 172.21.15.53:0/3643865007 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f17040813c0 con 0x55846e0072f0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1704081160 con 0x55846e0072f0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f16e9ffb640 1 -- 172.21.15.53:0/3643865007 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f170409a040 con 0x55846e0072f0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 10 client.8259 fetch_fsmap learned FSMap version 104 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55846dcff6b0 con 0x55846e0072f0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f16e9ffb640 1 -- 172.21.15.53:0/3643865007 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704081160 con 0x55846e0072f0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f16e9ffb640 1 -- 172.21.15.53:0/3643865007 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f170403a070 con 0x55846e0072f0 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 10 client.8259 fetch_fsmap finished waiting for FSMap version 104 2024-05-10T10:43:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 10 client.8259 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 20 client.8259 populate_metadata read hostname 'smithi053' 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846e020000 0x55846e0223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 4 client.8259 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 --> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846dea7760 con 0x55846e020000 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f170affd640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846e020000 0x55846e0223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f16e9ffb640 10 client.8259 ms_handle_connect on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.773+0000 7f170affd640 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846e020000 0x55846e0223f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b6af692e94d901 server_cookie=7d4df7b19e4b783b in_seq=0 out_seq=0 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f16e9ffb640 1 -- 172.21.15.53:0/3643865007 <== mds.0 v2:172.21.15.53:6832/38461894 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55846dea7760 con 0x55846e020000 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f16e9ffb640 10 client.8259 handle_command_reply: tid=0 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 client.8259 shutdown 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 msgr2=0x55846e00bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e00bd50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f17040b6480 tx=0x7f17040b6300 comp rx=0 tx=0).stop 2024-05-10T10:43:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 shutdown_connections 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846e020000 0x55846e0223f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e00bd50 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846e00b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f16e9ffb640 0 client.8259 ms_handle_reset on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 --2- 172.21.15.53:0/3643865007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe6f50 0x55846e0093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 >> 172.21.15.53:0/3643865007 conn(0x55846dfb9bb0 msgr2=0x55846dfba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 shutdown_connections 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.774+0000 7f1714432740 1 -- 172.21.15.53:0/3643865007 wait complete. 2024-05-10T10:43:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.775+0000 7f1714432740 20 client.8259 trim_cache size 0 max 16384 2024-05-10T10:43:06.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.793+0000 7f1714432740 1 Processor -- start 2024-05-10T10:43:06.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.793+0000 7f1714432740 1 -- start start 2024-05-10T10:43:06.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f1714432740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfe6f50 0x55846dfe5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846e0072f0 0x55846e009a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f1714432740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f170c162590 con 0x55846dfe6f50 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f170c1f9970 con 0x55846dfe7330 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f170c161f40 con 0x55846e0072f0 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfe6f50 0x55846dfe5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfe6f50 0x55846dfe5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50062/0 (socket says 172.21.15.53:50062) 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 -- 172.21.15.53:0/2872868122 learned_addr learned my addr 172.21.15.53:0/2872868122 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170a7fc640 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170b7fe640 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846e0072f0 0x55846e009a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 -- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846e0072f0 msgr2=0x55846e009a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846e0072f0 0x55846e009a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 -- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 msgr2=0x55846dfe6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170affd640 1 -- 172.21.15.53:0/2872868122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846dfbb8d0 con 0x55846dfe6f50 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170b7fe640 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846e0072f0 0x55846e009a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.794+0000 7f170a7fc640 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f170affd640 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfe6f50 0x55846dfe5d20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009830 tx=0x7f16fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0c06bbd8ed79ec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f16ebfff640 1 -- 172.21.15.53:0/2872868122 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc024410 con 0x55846dfe6f50 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f16ebfff640 1 -- 172.21.15.53:0/2872868122 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc02c070 con 0x55846dfe6f50 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f16ebfff640 1 -- 172.21.15.53:0/2872868122 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0247b0 con 0x55846dfe6f50 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 -- 172.21.15.53:0/2872868122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfe6f50 msgr2=0x55846dfe5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfe6f50 0x55846dfe5d20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009830 tx=0x7f16fc015000 comp rx=0 tx=0).stop 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 -- 172.21.15.53:0/2872868122 shutdown_connections 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846e0072f0 0x55846e009a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe7330 0x55846dfe6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 --2- 172.21.15.53:0/2872868122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846dfe6f50 0x55846dfe5d20 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 -- 172.21.15.53:0/2872868122 >> 172.21.15.53:0/2872868122 conn(0x55846dfb9bb0 msgr2=0x55846dfed670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 -- 172.21.15.53:0/2872868122 shutdown_connections 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.795+0000 7f1714432740 1 -- 172.21.15.53:0/2872868122 wait complete. 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.796+0000 7f1714432740 1 -- start start 2024-05-10T10:43:06.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.796+0000 7f1714432740 1 Processor -- start 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe6f50 0x55846e0029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f1714432740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe7330 0x55846e01c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f1714432740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e01c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f1714432740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55846dfbefd0 con 0x55846e0072f0 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55846dee0760 con 0x55846dfe6f50 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f1714432740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55846e00c8e0 con 0x55846dfe7330 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f1714432740 10 client.0 _ll_register_callbacks cb 0x7f16f4007db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e01c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e01c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50076/0 (socket says 172.21.15.53:50076) 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 -- 172.21.15.53:0/1317631961 learned_addr learned my addr 172.21.15.53:0/1317631961 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170affd640 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe6f50 0x55846e0029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170a7fc640 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe7330 0x55846e01c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 -- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe7330 msgr2=0x55846e01c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe7330 0x55846e01c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 -- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe6f50 msgr2=0x55846e0029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe6f50 0x55846e0029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 -- 172.21.15.53:0/1317631961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846e002ee0 con 0x55846e0072f0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f170b7fe640 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e01c770 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1704009890 tx=0x7f170407d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199638bcc4757988 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.797+0000 7f16e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f16e9ffb640 1 -- 172.21.15.53:0/1317631961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040b4070 con 0x55846e0072f0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f16e9ffb640 1 -- 172.21.15.53:0/1317631961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f17040805d0 con 0x55846e0072f0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc02c070 con 0x55846e0072f0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f16e9ffb640 1 -- 172.21.15.53:0/1317631961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f1704032440 con 0x55846e0072f0 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 10 client.8265 fetch_fsmap learned FSMap version 104 2024-05-10T10:43:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55846e002ee0 con 0x55846e0072f0 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f16e9ffb640 1 -- 172.21.15.53:0/1317631961 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170408b020 con 0x55846e0072f0 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f16e9ffb640 1 -- 172.21.15.53:0/1317631961 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f170403a040 con 0x55846e0072f0 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 10 client.8265 fetch_fsmap finished waiting for FSMap version 104 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 10 client.8265 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 20 client.8265 populate_metadata read hostname 'smithi053' 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846dfbd430 0x55846dfbd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 4 client.8265 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 --> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] -- command(tid 0: {"prefix": "status"}) -- 0x55846df314e0 con 0x55846dfbd430 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f170affd640 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846dfbd430 0x55846dfbd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f170affd640 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846dfbd430 0x55846dfbd810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a005b6baae2af2be server_cookie=3a3a067486143a33 in_seq=0 out_seq=0 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.798+0000 7f16e9ffb640 10 client.8265 ms_handle_connect on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f16e9ffb640 1 -- 172.21.15.53:0/1317631961 <== mds.0 v2:172.21.15.53:6832/38461894 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55846df314e0 con 0x55846dfbd430 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f16e9ffb640 10 client.8265 handle_command_reply: tid=0 2024-05-10T10:43:06.806 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 client.8265 shutdown 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 msgr2=0x55846e01c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e01c770 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1704009890 tx=0x7f170407d830 comp rx=0 tx=0).stop 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 shutdown_connections 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x55846dfbd430 0x55846dfbd810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55846e0072f0 0x55846e01c770 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55846dfe7330 0x55846e01c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f16e9ffb640 0 client.8265 ms_handle_reset on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 --2- 172.21.15.53:0/1317631961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55846dfe6f50 0x55846e0029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 >> 172.21.15.53:0/1317631961 conn(0x55846dfb9bb0 msgr2=0x55846dfc0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 shutdown_connections 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 1 -- 172.21.15.53:0/1317631961 wait complete. 2024-05-10T10:43:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f1714432740 20 client.8265 trim_cache size 0 max 16384 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 8025, 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 4.4863459460000001, 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 104, 2024-05-10T10:43:06.808 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 119, 2024-05-10T10:43:06.809 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 119, 2024-05-10T10:43:06.809 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 21.923555604000001 2024-05-10T10:43:06.809 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f16ec076300 msgr2=0x7f16ec0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f16ec076300 0x7f16ec0787c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f170c1f8e10 tx=0x7f16fc0094d0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c15f3e0 msgr2=0x7f170c0ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c15f3e0 0x7f170c0ce610 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f17040098c0 tx=0x7f17040046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 shutdown_connections 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f16ec076300 0x7f16ec0787c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f170c15f3e0 0x7f170c0ce610 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d0750 0x7f170c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 --2- 172.21.15.53:0/3726837137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0b64e0 0x7f170c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.799+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 >> 172.21.15.53:0/3726837137 conn(0x7f170c0d4ff0 msgr2=0x7f170c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.800+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 shutdown_connections 2024-05-10T10:43:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.800+0000 7f171163e640 1 -- 172.21.15.53:0/3726837137 wait complete. 2024-05-10T10:43:06.812 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:43:06.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.913+0000 7f7d38fd4640 1 Processor -- start 2024-05-10T10:43:06.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.913+0000 7f7d38fd4640 1 -- start start 2024-05-10T10:43:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.914+0000 7f7d38fd4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d340cf7b0 0x7f7d340cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.914+0000 7f7d38fd4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d3415f650 0x7f7d340d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.914+0000 7f7d38fd4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d340d24d0 con 0x7f7d340b1f60 2024-05-10T10:43:06.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.914+0000 7f7d38fd4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7d340b24b0 con 0x7f7d340cf7b0 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.914+0000 7f7d38fd4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3406dc80 con 0x7f7d3415f650 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d3259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d340cf7b0 0x7f7d340cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d3259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d340cf7b0 0x7f7d340cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50082/0 (socket says 172.21.15.53:50082) 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d3259c640 1 -- 172.21.15.53:0/1767922338 learned_addr learned my addr 172.21.15.53:0/1767922338 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d31d9b640 1 --2- 172.21.15.53:0/1767922338 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d3415f650 0x7f7d340d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d3259c640 1 -- 172.21.15.53:0/1767922338 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d3415f650 msgr2=0x7f7d340d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d3259c640 1 --2- 172.21.15.53:0/1767922338 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d3415f650 0x7f7d340d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d3259c640 1 -- 172.21.15.53:0/1767922338 >> v1:172.21.15.135:6789/0 conn(0x7f7d340b1f60 legacy=0x7f7d340d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.915+0000 7f7d3259c640 1 -- 172.21.15.53:0/1767922338 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d340d00d0 con 0x7f7d340cf7b0 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d3259c640 1 --2- 172.21.15.53:0/1767922338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d340cf7b0 0x7f7d340cfb90 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009e50 tx=0x7f7d2402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffa1673f7cbb23c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d3159a640 1 -- 172.21.15.53:0/1767922338 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24036070 con 0x7f7d340cf7b0 2024-05-10T10:43:06.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d3159a640 1 -- 172.21.15.53:0/1767922338 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d24031600 con 0x7f7d340cf7b0 2024-05-10T10:43:06.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d3159a640 1 -- 172.21.15.53:0/1767922338 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240319d0 con 0x7f7d340cf7b0 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1767922338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d340cf7b0 msgr2=0x7f7d340cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1767922338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d340cf7b0 0x7f7d340cfb90 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009e50 tx=0x7f7d2402a710 comp rx=0 tx=0).stop 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1767922338 shutdown_connections 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1767922338 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d3415f650 0x7f7d340d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1767922338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d340cf7b0 0x7f7d340cfb90 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.916+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1767922338 >> 172.21.15.53:0/1767922338 conn(0x7f7d340d2dd0 msgr2=0x7f7d340d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.917+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1767922338 shutdown_connections 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.917+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1767922338 wait complete. 2024-05-10T10:43:06.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.918+0000 7f7d38fd4640 1 Processor -- start 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.918+0000 7f7d38fd4640 1 -- start start 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d340b1f60 0x7f7d34162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d340cf7b0 0x7f7d341f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3415f650 0x7f7d3415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d340cd5c0 con 0x7f7d3415f650 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3406d750 con 0x7f7d340b1f60 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f7d340cd8a0 con 0x7f7d340cf7b0 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3415f650 0x7f7d3415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3415f650 0x7f7d3415b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50086/0 (socket says 172.21.15.53:50086) 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 -- 172.21.15.53:0/1044120741 learned_addr learned my addr 172.21.15.53:0/1044120741 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d3259c640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d340b1f60 0x7f7d34162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d31d9b640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d340cf7b0 0x7f7d341f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 -- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d340cf7b0 msgr2=0x7f7d341f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d340cf7b0 0x7f7d341f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 -- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d340b1f60 msgr2=0x7f7d34162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d340b1f60 0x7f7d34162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 -- 172.21.15.53:0/1044120741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d341f8bb0 con 0x7f7d3415f650 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d31d9b640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d340cf7b0 0x7f7d341f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d3259c640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d340b1f60 0x7f7d34162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d32d9d640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3415f650 0x7f7d3415b260 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c3a0 tx=0x7f7d2c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d2affd640 1 -- 172.21.15.53:0/1044120741 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c017070 con 0x7f7d3415f650 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d2affd640 1 -- 172.21.15.53:0/1044120741 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0052d0 con 0x7f7d3415f650 2024-05-10T10:43:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d2affd640 1 -- 172.21.15.53:0/1044120741 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c003830 con 0x7f7d3415f650 2024-05-10T10:43:06.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d341f8e90 con 0x7f7d3415f650 2024-05-10T10:43:06.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.919+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3415fb30 con 0x7f7d3415f650 2024-05-10T10:43:06.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.920+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34069400 con 0x7f7d3415f650 2024-05-10T10:43:06.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.921+0000 7f7d2affd640 1 -- 172.21.15.53:0/1044120741 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d2c01a070 con 0x7f7d3415f650 2024-05-10T10:43:06.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.922+0000 7f7d2affd640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d14076300 0x7f7d140787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:06.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.922+0000 7f7d2affd640 1 -- 172.21.15.53:0/1044120741 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f7d2c096a60 con 0x7f7d3415f650 2024-05-10T10:43:06.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.922+0000 7f7d3259c640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d14076300 0x7f7d140787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:06.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.922+0000 7f7d3259c640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d14076300 0x7f7d140787c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d24033000 tx=0x7f7d24009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:06.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:06.926+0000 7f7d2affd640 1 -- 172.21.15.53:0/1044120741 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f7d2c060330 con 0x7f7d3415f650 2024-05-10T10:43:07.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.157+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d34069c90 con 0x7f7d3415f650 2024-05-10T10:43:07.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.158+0000 7f7d2affd640 1 -- 172.21.15.53:0/1044120741 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) ==== 53+0+73 (secure 0 0 0) 0x7f7d2c068320 con 0x7f7d3415f650 2024-05-10T10:43:07.159 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:43:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.159+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d14076300 msgr2=0x7f7d140787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d14076300 0x7f7d140787c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d24033000 tx=0x7f7d24009000 comp rx=0 tx=0).stop 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3415f650 msgr2=0x7f7d3415b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3415f650 0x7f7d3415b260 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c3a0 tx=0x7f7d2c00c860 comp rx=0 tx=0).stop 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 shutdown_connections 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f7d14076300 0x7f7d140787c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3415f650 0x7f7d3415b260 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d340cf7b0 0x7f7d341f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 --2- 172.21.15.53:0/1044120741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d340b1f60 0x7f7d34162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 >> 172.21.15.53:0/1044120741 conn(0x7f7d340d2dd0 msgr2=0x7f7d340b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 shutdown_connections 2024-05-10T10:43:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07.160+0000 7f7d38fd4640 1 -- 172.21.15.53:0/1044120741 wait complete. 2024-05-10T10:43:07.172 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:43:07.172 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:43:07.172 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:43:07.172 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:43:07.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:07.172 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:43:07.173 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:43:07.173 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:43:07.173 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:43:07.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.173 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:43:07.189 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:43:07.189 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:43:07.189 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:07.190 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:43:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:07.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.192 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:43:07.254 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:43:07.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.255 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:43:07.312 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:43:07.312 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:43:07.312 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:43:07.312 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:43:07.312 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:43:07.313 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:43:07.313 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:43:07.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:43:07.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:43:07.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:43:07.313 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:43:07.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:43:07.436 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:07.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.442 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:43:07.459 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:43:07.459 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:43:07.459 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:43:07.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:43:07.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:43:07.525 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:07.581 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:43:07.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:43:07.581 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:07.637 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:43:07.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:07.722 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:07.722 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:43:07.722 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:43:07.730 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:07.731 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:43:07.731 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:43:07.731 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:43:07.731 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:43:07.736 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:43:07.737 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:43:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:43:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:43:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:43:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:43:07.739 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:43:07.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:07.764 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:43:07.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:43:07.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:07.863 INFO:teuthology.orchestra.run.smithi053.stdout:client7417 2024-05-10T10:43:07.864 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:07.864 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:07.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:07.880 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-10T10:43:07.940 INFO:teuthology.orchestra.run.smithi053.stdout:./ 2024-05-10T10:43:07.940 INFO:teuthology.orchestra.run.smithi053.stdout:./parent 2024-05-10T10:43:07.940 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed 2024-05-10T10:43:07.940 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/charlie 2024-05-10T10:43:07.940 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/alpha 2024-05-10T10:43:07.940 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/unflushed 2024-05-10T10:43:07.940 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/unflushed/jfile 2024-05-10T10:43:07.941 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:07.941 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:07.941 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:07.941 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:07.941 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-10T10:43:07.941 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:08.048 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:43:08.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:08.057 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:08.057 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:08.057 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:08.057 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:08.057 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-10T10:43:08.057 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:08.165 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2024-05-10T10:43:08.168 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:08.174 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:08.174 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:08.174 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:08.174 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:08.174 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-10T10:43:08.174 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:08.283 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2024-05-10T10:43:08.285 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:08.292 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:08.292 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:08.293 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:08.293 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:08.293 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-10T10:43:08.293 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:08.400 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628280 2024-05-10T10:43:08.403 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:08.409 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:08.409 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:08.409 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:08.409 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:08.410 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-10T10:43:08.410 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:08.518 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627778 2024-05-10T10:43:08.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:08.527 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:08.527 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:08.527 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:08.527 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:08.527 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-10T10:43:08.527 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:08.635 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627779 2024-05-10T10:43:08.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:08.644 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:08.644 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:08.644 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:08.644 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:08.644 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-10T10:43:08.644 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:08.752 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627780 2024-05-10T10:43:08.755 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:08.761 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:08.761 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:08.818 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:08.818 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:08.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:43:09.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.018+0000 7fbb2f0a0640 1 Processor -- start 2024-05-10T10:43:09.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.018+0000 7fbb2f0a0640 1 -- start start 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2f0a0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbb2815fd20 0x7fbb2806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2f0a0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb280b4c20 0x7fbb28169090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2f0a0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2806c8e0 con 0x7fbb28160100 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2f0a0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2806cb60 con 0x7fbb280b4c20 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2f0a0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2806ce40 con 0x7fbb2815fd20 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2d63d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb280b4c20 0x7fbb28169090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb1ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbb28160100 0x7fbb280b4510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51162/0 (socket says 172.21.15.53:51162) 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb1ffff640 1 -- 172.21.15.53:0/640074779 learned_addr learned my addr 172.21.15.53:0/640074779 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2d63d640 1 -- 172.21.15.53:0/640074779 >> v2:172.21.15.135:3300/0 conn(0x7fbb2815fd20 msgr2=0x7fbb2806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2d63d640 1 --2- 172.21.15.53:0/640074779 >> v2:172.21.15.135:3300/0 conn(0x7fbb2815fd20 0x7fbb2806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2d63d640 1 -- 172.21.15.53:0/640074779 >> v1:172.21.15.53:6789/0 conn(0x7fbb28160100 legacy=0x7fbb280b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.019+0000 7fbb2d63d640 1 -- 172.21.15.53:0/640074779 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2806d120 con 0x7fbb280b4c20 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.020+0000 7fbb2d63d640 1 --2- 172.21.15.53:0/640074779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28169090 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbb28073c50 tx=0x7fbb2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb45e21864e9b876 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.020+0000 7fbb1f7fe640 1 -- 172.21.15.53:0/640074779 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20007740 con 0x7fbb280b4c20 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.020+0000 7fbb1f7fe640 1 -- 172.21.15.53:0/640074779 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb200078e0 con 0x7fbb280b4c20 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.020+0000 7fbb1f7fe640 1 -- 172.21.15.53:0/640074779 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20007cb0 con 0x7fbb280b4c20 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/640074779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 msgr2=0x7fbb28169090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/640074779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28169090 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbb28073c50 tx=0x7fbb2002a680 comp rx=0 tx=0).stop 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/640074779 shutdown_connections 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/640074779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28169090 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/640074779 >> v2:172.21.15.135:3300/0 conn(0x7fbb2815fd20 0x7fbb2806c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/640074779 >> 172.21.15.53:0/640074779 conn(0x7fbb280d1560 msgr2=0x7fbb280d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/640074779 shutdown_connections 2024-05-10T10:43:09.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.021+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/640074779 wait complete. 2024-05-10T10:43:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.022+0000 7fbb2f0a0640 1 Processor -- start 2024-05-10T10:43:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.022+0000 7fbb2f0a0640 1 -- start start 2024-05-10T10:43:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2f0a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2f0a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2815fd20 0x7fbb280b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2f0a0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb28160100 0x7fbb280727d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2f0a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2806dc00 con 0x7fbb280b4c20 2024-05-10T10:43:09.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2f0a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2806d6e0 con 0x7fbb2815fd20 2024-05-10T10:43:09.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2f0a0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbb28160560 con 0x7fbb28160100 2024-05-10T10:43:09.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50108/0 (socket says 172.21.15.53:50108) 2024-05-10T10:43:09.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 -- 172.21.15.53:0/3606699374 learned_addr learned my addr 172.21.15.53:0/3606699374 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:09.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb1ffff640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2815fd20 0x7fbb280b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2d63d640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb28160100 0x7fbb280727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 -- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb28160100 msgr2=0x7fbb280727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb28160100 0x7fbb280727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 -- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2815fd20 msgr2=0x7fbb280b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2815fd20 0x7fbb280b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 -- 172.21.15.53:0/3606699374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb280b1f70 con 0x7fbb280b4c20 2024-05-10T10:43:09.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.023+0000 7fbb2ce3c640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28143080 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400b4a0 tx=0x7fbb2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.024+0000 7fbb1d7fa640 1 -- 172.21.15.53:0/3606699374 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24014070 con 0x7fbb280b4c20 2024-05-10T10:43:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.024+0000 7fbb1d7fa640 1 -- 172.21.15.53:0/3606699374 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb24010610 con 0x7fbb280b4c20 2024-05-10T10:43:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.024+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb280b2250 con 0x7fbb280b4c20 2024-05-10T10:43:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.024+0000 7fbb1d7fa640 1 -- 172.21.15.53:0/3606699374 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24010a00 con 0x7fbb280b4c20 2024-05-10T10:43:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.024+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb280b2810 con 0x7fbb280b4c20 2024-05-10T10:43:09.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.025+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb28069400 con 0x7fbb280b4c20 2024-05-10T10:43:09.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.025+0000 7fbb1d7fa640 1 -- 172.21.15.53:0/3606699374 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbb24017070 con 0x7fbb280b4c20 2024-05-10T10:43:09.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.026+0000 7fbb1d7fa640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb140762d0 0x7fbb14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.026+0000 7fbb1d7fa640 1 -- 172.21.15.53:0/3606699374 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7fbb24097200 con 0x7fbb280b4c20 2024-05-10T10:43:09.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.026+0000 7fbb1ffff640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb140762d0 0x7fbb14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.026+0000 7fbb1ffff640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb140762d0 0x7fbb14078790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbb28068610 tx=0x7fbb18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.030+0000 7fbb1d7fa640 1 -- 172.21.15.53:0/3606699374 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbb24060980 con 0x7fbb280b4c20 2024-05-10T10:43:09.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.233+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb280d12d0 con 0x7fbb280b4c20 2024-05-10T10:43:09.236 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.234+0000 7fbb1d7fa640 1 -- 172.21.15.53:0/3606699374 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v119) ==== 80+0+122 (secure 0 0 0) 0x7fbb24060280 con 0x7fbb280b4c20 2024-05-10T10:43:09.236 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.135:6837/776479349 2024-05-11T10:43:01.305186+0000 2024-05-10T10:43:09.236 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.135:6835/776479349 2024-05-11T10:43:01.305186+0000 2024-05-10T10:43:09.236 INFO:teuthology.orchestra.run.smithi053.stderr:listed 2 entries 2024-05-10T10:43:09.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb140762d0 msgr2=0x7fbb14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb140762d0 0x7fbb14078790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbb28068610 tx=0x7fbb18009040 comp rx=0 tx=0).stop 2024-05-10T10:43:09.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 msgr2=0x7fbb28143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28143080 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400b4a0 tx=0x7fbb2400b960 comp rx=0 tx=0).stop 2024-05-10T10:43:09.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 shutdown_connections 2024-05-10T10:43:09.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbb140762d0 0x7fbb14078790 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb28160100 0x7fbb280727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2815fd20 0x7fbb280b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 --2- 172.21.15.53:0/3606699374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb280b4c20 0x7fbb28143080 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 >> 172.21.15.53:0/3606699374 conn(0x7fbb280d1560 msgr2=0x7fbb2806fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 shutdown_connections 2024-05-10T10:43:09.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.236+0000 7fbb2f0a0640 1 -- 172.21.15.53:0/3606699374 wait complete. 2024-05-10T10:43:09.248 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:43:09.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:09.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:09.278 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:09.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:09.278 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:43:09.334 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:09.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.476+0000 7f837e023640 1 Processor -- start 2024-05-10T10:43:09.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.477+0000 7f837e023640 1 -- start start 2024-05-10T10:43:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.477+0000 7f837e023640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83780d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.477+0000 7f837e023640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83780d3a20 0x7f8378162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.477+0000 7f837e023640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83780d0f90 con 0x7f8378163340 2024-05-10T10:43:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.477+0000 7f837e023640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f83780b1c20 con 0x7f83780d3a20 2024-05-10T10:43:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.477+0000 7f837e023640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83780b1f00 con 0x7f837815f1a0 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f83777fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83780d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f8376ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83780d3a20 0x7f8378162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f83777fe640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83780d34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:57886/0 (socket says 172.21.15.53:57886) 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f83777fe640 1 -- 172.21.15.53:0/459401870 learned_addr learned my addr 172.21.15.53:0/459401870 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f8377fff640 1 --1- 172.21.15.53:0/459401870 >> v1:172.21.15.135:6789/0 conn(0x7f8378163340 0x7f83780cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:35696/0 (socket says 172.21.15.53:35696) 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f83777fe640 1 -- 172.21.15.53:0/459401870 >> v2:172.21.15.135:3300/0 conn(0x7f83780d3a20 msgr2=0x7f8378162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f83777fe640 1 --2- 172.21.15.53:0/459401870 >> v2:172.21.15.135:3300/0 conn(0x7f83780d3a20 0x7f8378162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f83777fe640 1 -- 172.21.15.53:0/459401870 >> v1:172.21.15.135:6789/0 conn(0x7f8378163340 legacy=0x7f83780cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.478+0000 7f83777fe640 1 -- 172.21.15.53:0/459401870 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780b21e0 con 0x7f837815f1a0 2024-05-10T10:43:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.479+0000 7f83777fe640 1 --2- 172.21.15.53:0/459401870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83780d34e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8368009870 tx=0x7f836802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2df3fc9f23099a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.479+0000 7f83767fc640 1 -- 172.21.15.53:0/459401870 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368037070 con 0x7f837815f1a0 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.479+0000 7f83767fc640 1 -- 172.21.15.53:0/459401870 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8368032440 con 0x7f837815f1a0 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.479+0000 7f83767fc640 1 -- 172.21.15.53:0/459401870 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368032810 con 0x7f837815f1a0 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 -- 172.21.15.53:0/459401870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 msgr2=0x7f83780d34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 --2- 172.21.15.53:0/459401870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83780d34e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8368009870 tx=0x7f836802a270 comp rx=0 tx=0).stop 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 -- 172.21.15.53:0/459401870 shutdown_connections 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 --2- 172.21.15.53:0/459401870 >> v2:172.21.15.135:3300/0 conn(0x7f83780d3a20 0x7f8378162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 --2- 172.21.15.53:0/459401870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83780d34e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 -- 172.21.15.53:0/459401870 >> 172.21.15.53:0/459401870 conn(0x7f83780d4f90 msgr2=0x7f83780d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 -- 172.21.15.53:0/459401870 shutdown_connections 2024-05-10T10:43:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.480+0000 7f837e023640 1 -- 172.21.15.53:0/459401870 wait complete. 2024-05-10T10:43:09.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.481+0000 7f837e023640 1 Processor -- start 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.481+0000 7f837e023640 1 -- start start 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f837e023640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83780d3a20 0x7f837815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f837e023640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83781e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f837e023640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8378163340 0x7f83780ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f837e023640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f837815f580 con 0x7f83780d3a20 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f837e023640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83780d4050 con 0x7f8378163340 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f837e023640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f837815f700 con 0x7f837815f1a0 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83780d3a20 0x7f837815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83780d3a20 0x7f837815de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50116/0 (socket says 172.21.15.53:50116) 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 -- 172.21.15.53:0/949826765 learned_addr learned my addr 172.21.15.53:0/949826765 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f8376ffd640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83781e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f8377fff640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8378163340 0x7f83780ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 -- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 msgr2=0x7f83781e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83781e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 -- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8378163340 msgr2=0x7f83780ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8378163340 0x7f83780ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f83777fe640 1 -- 172.21.15.53:0/949826765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83781e8600 con 0x7f83780d3a20 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.482+0000 7f8377fff640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8378163340 0x7f83780ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.483+0000 7f83777fe640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83780d3a20 0x7f837815de20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f8368009870 tx=0x7f83680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.483+0000 7f835ffff640 1 -- 172.21.15.53:0/949826765 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368041070 con 0x7f83780d3a20 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.483+0000 7f835ffff640 1 -- 172.21.15.53:0/949826765 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836803c890 con 0x7f83780d3a20 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.483+0000 7f835ffff640 1 -- 172.21.15.53:0/949826765 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368037040 con 0x7f83780d3a20 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.483+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83781e88a0 con 0x7f83780d3a20 2024-05-10T10:43:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.483+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83781e8e60 con 0x7f83780d3a20 2024-05-10T10:43:09.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.484+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8378069400 con 0x7f83780d3a20 2024-05-10T10:43:09.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.484+0000 7f835ffff640 1 -- 172.21.15.53:0/949826765 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8368044070 con 0x7f83780d3a20 2024-05-10T10:43:09.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.485+0000 7f835ffff640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83640762d0 0x7f8364078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.485+0000 7f835ffff640 1 -- 172.21.15.53:0/949826765 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f83680b8050 con 0x7f83780d3a20 2024-05-10T10:43:09.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.485+0000 7f8376ffd640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83640762d0 0x7f8364078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.488 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.486+0000 7f8376ffd640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83640762d0 0x7f8364078790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f83781f8550 tx=0x7f836c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.490 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.489+0000 7f835ffff640 1 -- 172.21.15.53:0/949826765 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8368081210 con 0x7f83780d3a20 2024-05-10T10:43:09.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.717+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f837815ea30 con 0x7f83780d3a20 2024-05-10T10:43:09.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.717+0000 7f835ffff640 1 -- 172.21.15.53:0/949826765 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5180 (secure 0 0 0) 0x7f8368085510 con 0x7f83780d3a20 2024-05-10T10:43:09.719 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:09.719 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:05.316528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6834/38461894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":38461894},{"type":"v1","addr":"172.21.15.53:6834","nonce":38461894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8025,"qdb_cluster":[8025]},"id":8}]} 2024-05-10T10:43:09.719 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 104 2024-05-10T10:43:09.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83640762d0 msgr2=0x7f8364078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83640762d0 0x7f8364078790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f83781f8550 tx=0x7f836c009190 comp rx=0 tx=0).stop 2024-05-10T10:43:09.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83780d3a20 msgr2=0x7f837815de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83780d3a20 0x7f837815de20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f8368009870 tx=0x7f83680046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 shutdown_connections 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83640762d0 0x7f8364078790 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8378163340 0x7f83780ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f837815f1a0 0x7f83781e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 --2- 172.21.15.53:0/949826765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83780d3a20 0x7f837815de20 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 >> 172.21.15.53:0/949826765 conn(0x7f83780d4f90 msgr2=0x7f837806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.719+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 shutdown_connections 2024-05-10T10:43:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.720+0000 7f837e023640 1 -- 172.21.15.53:0/949826765 wait complete. 2024-05-10T10:43:09.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-05-10T10:43:09.858 INFO:teuthology.orchestra.run.smithi053.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-05-10T10:43:09.858 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-05-10T10:43:09.858 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-10T10:43:09.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 Processor -- start 2024-05-10T10:43:09.903 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 -- start start 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 0x55749cb9edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 0x55749cb7ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749cb84030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55749cb11f20 con 0x55749cb85e10 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55749ca72f50 con 0x55749cb85a30 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e37120f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55749cacccd0 con 0x55749cacbfd0 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e349b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749cb84030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e349b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749cb84030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50132/0 (socket says 172.21.15.53:50132) 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.902+0000 7f2e349b3640 1 -- 172.21.15.53:0/1124811427 learned_addr learned my addr 172.21.15.53:0/1124811427 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e349b3640 1 -- 172.21.15.53:0/1124811427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 msgr2=0x55749cb9edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e349b3640 1 --2- 172.21.15.53:0/1124811427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 0x55749cb9edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e349b3640 1 -- 172.21.15.53:0/1124811427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 msgr2=0x55749cb7ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e349b3640 1 --2- 172.21.15.53:0/1124811427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 0x55749cb7ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e349b3640 1 -- 172.21.15.53:0/1124811427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749cb9fbc0 con 0x55749cb85e10 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e349b3640 1 --2- 172.21.15.53:0/1124811427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749cb84030 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00a4b0 tx=0x7f2e2c00a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2cf95a329c32e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e329af640 1 -- 172.21.15.53:0/1124811427 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c015070 con 0x55749cb85e10 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e329af640 1 -- 172.21.15.53:0/1124811427 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c00e730 con 0x55749cb85e10 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e329af640 1 -- 172.21.15.53:0/1124811427 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00eb20 con 0x55749cb85e10 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 -- 172.21.15.53:0/1124811427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 msgr2=0x55749cb84030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 --2- 172.21.15.53:0/1124811427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749cb84030 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00a4b0 tx=0x7f2e2c00a970 comp rx=0 tx=0).stop 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 -- 172.21.15.53:0/1124811427 shutdown_connections 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 --2- 172.21.15.53:0/1124811427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749cb84030 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 --2- 172.21.15.53:0/1124811427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 0x55749cb7ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 --2- 172.21.15.53:0/1124811427 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 0x55749cb9edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 -- 172.21.15.53:0/1124811427 >> 172.21.15.53:0/1124811427 conn(0x55749cac9ba0 msgr2=0x55749cac8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 -- 172.21.15.53:0/1124811427 shutdown_connections 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.903+0000 7f2e37120f40 1 -- 172.21.15.53:0/1124811427 wait complete. 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.904+0000 7f2e37120f40 1 Processor -- start 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.904+0000 7f2e37120f40 1 -- start start 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e37120f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 0x55749cb9dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e37120f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 0x55749caf62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e37120f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749caf6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e37120f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55749cb84a30 con 0x55749cb85e10 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e37120f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55749cba03f0 con 0x55749cb85a30 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e37120f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55749cb891d0 con 0x55749cacbfd0 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749caf6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749caf6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50146/0 (socket says 172.21.15.53:50146) 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 -- 172.21.15.53:0/2519767484 learned_addr learned my addr 172.21.15.53:0/2519767484 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:09.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e341b2640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 0x55749cb9dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e339b1640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 0x55749caf62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 -- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 msgr2=0x55749cb9dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55749cacbfd0 0x55749cb9dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 -- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 msgr2=0x55749caf62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55749cb85a30 0x55749caf62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 -- 172.21.15.53:0/2519767484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749cb82db0 con 0x55749cb85e10 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e349b3640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749caf6830 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0032c0 tx=0x7f2e2c0032f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.905+0000 7f2e319ad640 1 -- 172.21.15.53:0/2519767484 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c020070 con 0x55749cb85e10 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.906+0000 7f2e319ad640 1 -- 172.21.15.53:0/2519767484 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c01b850 con 0x55749cb85e10 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.906+0000 7f2e319ad640 1 -- 172.21.15.53:0/2519767484 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c015040 con 0x55749cb85e10 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.906+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55749cb7ca30 con 0x55749cb85e10 2024-05-10T10:43:09.911 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.906+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749cb7cff0 con 0x55749cb85e10 2024-05-10T10:43:09.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:43:09.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.907+0000 7f2e319ad640 1 -- 172.21.15.53:0/2519767484 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e2c01bc40 con 0x55749cb85e10 2024-05-10T10:43:09.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.907+0000 7f2e319ad640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e20074cb0 0x7f2e20077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.908+0000 7f2e319ad640 1 -- 172.21.15.53:0/2519767484 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f2e2c0978d0 con 0x55749cb85e10 2024-05-10T10:43:09.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.908+0000 7f2e37120f40 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55749cb7d7f0 0x55749cb7dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.908+0000 7f2e341b2640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e20074cb0 0x7f2e20077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.908+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) -- 0x55749cb7e140 con 0x55749cb7d7f0 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.908+0000 7f2e339b1640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55749cb7d7f0 0x55749cb7dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.908+0000 7f2e339b1640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55749cb7d7f0 0x55749cb7dc00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.908+0000 7f2e341b2640 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e20074cb0 0x7f2e20077170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2e24057000 tx=0x7f2e24051000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e339b1640 1 -- 172.21.15.53:0/2519767484 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f2e28009150 con 0x55749cb7d7f0 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55749cb7d7f0 msgr2=0x55749cb7dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55749cb7d7f0 0x55749cb7dc00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e20074cb0 msgr2=0x7f2e20077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f2e20074cb0 0x7f2e20077170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2e24057000 tx=0x7f2e24051000 comp rx=0 tx=0).stop 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 msgr2=0x55749caf6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 --2- 172.21.15.53:0/2519767484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55749cb85e10 0x55749caf6830 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0032c0 tx=0x7f2e2c0032f0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e349b3640 1 -- 172.21.15.53:0/2519767484 reap_dead start 2024-05-10T10:43:09.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 shutdown_connections 2024-05-10T10:43:09.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 >> 172.21.15.53:0/2519767484 conn(0x55749cac9ba0 msgr2=0x55749cacb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 shutdown_connections 2024-05-10T10:43:09.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.909+0000 7f2e37120f40 1 -- 172.21.15.53:0/2519767484 wait complete. 2024-05-10T10:43:09.916 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-10T10:43:09.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.959+0000 7fb770010f40 1 Processor -- start 2024-05-10T10:43:09.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.959+0000 7fb770010f40 1 -- start start 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.959+0000 7fb770010f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606838c2650 0x5606839896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.959+0000 7fb770010f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x5606838f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.959+0000 7fb770010f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560683992d20 0x5606838f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb770010f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560683868f50 con 0x56068390a0d0 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb770010f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606838c3cb0 con 0x5606838c2650 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb770010f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5606838eef80 con 0x560683992d20 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x5606838f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x5606838f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50158/0 (socket says 172.21.15.53:50158) 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 -- 172.21.15.53:0/3894078647 learned_addr learned my addr 172.21.15.53:0/3894078647 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76db0a640 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606838c2650 0x5606839896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76e30b640 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560683992d20 0x5606838f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 -- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560683992d20 msgr2=0x5606838f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560683992d20 0x5606838f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 -- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606838c2650 msgr2=0x5606839896d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606838c2650 0x5606839896d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 -- 172.21.15.53:0/3894078647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606838ef260 con 0x56068390a0d0 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb76d309640 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x5606838f00a0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb758000c00 tx=0x7fb758051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eef0449c06e6a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb767fff640 1 -- 172.21.15.53:0/3894078647 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758063070 con 0x56068390a0d0 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb767fff640 1 -- 172.21.15.53:0/3894078647 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758055930 con 0x56068390a0d0 2024-05-10T10:43:09.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.960+0000 7fb767fff640 1 -- 172.21.15.53:0/3894078647 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758055d20 con 0x56068390a0d0 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 -- 172.21.15.53:0/3894078647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 msgr2=0x5606838f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x5606838f00a0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb758000c00 tx=0x7fb758051000 comp rx=0 tx=0).stop 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 -- 172.21.15.53:0/3894078647 shutdown_connections 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x560683992d20 0x5606838f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x5606838f00a0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 --2- 172.21.15.53:0/3894078647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606838c2650 0x5606839896d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 -- 172.21.15.53:0/3894078647 >> 172.21.15.53:0/3894078647 conn(0x5606838c0420 msgr2=0x5606838c0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 -- 172.21.15.53:0/3894078647 shutdown_connections 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 -- 172.21.15.53:0/3894078647 wait complete. 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 Processor -- start 2024-05-10T10:43:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.961+0000 7fb770010f40 1 -- start start 2024-05-10T10:43:09.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb770010f40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5606838c2650 0x560683988710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb770010f40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x56068398c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb770010f40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560683992d20 0x56068398c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb770010f40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5606838ef820 con 0x56068390a0d0 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb770010f40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606838ef540 con 0x560683992d20 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb770010f40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x56068399a4b0 con 0x5606838c2650 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb76d309640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x56068398c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb76d309640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x56068398c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50166/0 (socket says 172.21.15.53:50166) 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb76d309640 1 -- 172.21.15.53:0/3070894762 learned_addr learned my addr 172.21.15.53:0/3070894762 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb76e30b640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560683992d20 0x56068398c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb76db0a640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5606838c2650 0x560683988710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb76d309640 1 -- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5606838c2650 msgr2=0x560683988710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.962+0000 7fb76d309640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5606838c2650 0x560683988710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb76d309640 1 -- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560683992d20 msgr2=0x56068398c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb76d309640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560683992d20 0x56068398c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb76d309640 1 -- 172.21.15.53:0/3070894762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560683989c90 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb76d309640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x56068398c2d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb758005570 tx=0x7fb758002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb766ffd640 1 -- 172.21.15.53:0/3070894762 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758076070 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb766ffd640 1 -- 172.21.15.53:0/3070894762 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758068460 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56068398d150 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb766ffd640 1 -- 172.21.15.53:0/3070894762 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75806d070 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.963+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606839899b0 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.964+0000 7fb766ffd640 1 -- 172.21.15.53:0/3070894762 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb758079070 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.965+0000 7fb766ffd640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb750074cb0 0x7fb750077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.965+0000 7fb766ffd640 1 -- 172.21.15.53:0/3070894762 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7fb7580dbc60 con 0x56068390a0d0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.965+0000 7fb76db0a640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb750074cb0 0x7fb750077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.965+0000 7fb770010f40 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x560683988d60 0x560683907440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.965+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) -- 0x560683907980 con 0x560683988d60 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.965+0000 7fb76e30b640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x560683988d60 0x560683907440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.965+0000 7fb76db0a640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb750074cb0 0x7fb750077170 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb768005fd0 tx=0x7fb768005ad0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb76e30b640 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x560683988d60 0x560683907440 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb76e30b640 1 -- 172.21.15.53:0/3070894762 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fb76000d760 con 0x560683988d60 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x560683988d60 msgr2=0x560683907440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x560683988d60 0x560683907440 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb750074cb0 msgr2=0x7fb750077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb750074cb0 0x7fb750077170 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb768005fd0 tx=0x7fb768005ad0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 msgr2=0x56068398c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 --2- 172.21.15.53:0/3070894762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56068390a0d0 0x56068398c2d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb758005570 tx=0x7fb758002fb0 comp rx=0 tx=0).stop 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb76e30b640 1 -- 172.21.15.53:0/3070894762 reap_dead start 2024-05-10T10:43:09.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 shutdown_connections 2024-05-10T10:43:09.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 >> 172.21.15.53:0/3070894762 conn(0x5606838c0420 msgr2=0x5606838c0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:09.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 shutdown_connections 2024-05-10T10:43:09.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:09.966+0000 7fb770010f40 1 -- 172.21.15.53:0/3070894762 wait complete. 2024-05-10T10:43:09.974 INFO:teuthology.orchestra.run.smithi053.stdout:mytag123 2024-05-10T10:43:09.974 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-10T10:43:10.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.017+0000 7f24cad1cf40 1 Processor -- start 2024-05-10T10:43:10.019 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.017+0000 7f24cad1cf40 1 -- start start 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24cad1cf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4002820 0x55cb2de1fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24cad1cf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cb2de36e10 0x55cb2de1f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24cad1cf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 0x55cb2dd92410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24cad1cf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cb2ddacc50 con 0x7f24b4002820 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24cad1cf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cb2dd0af50 con 0x55cb2dd8c7c0 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24cad1cf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55cb2de38d90 con 0x55cb2de36e10 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4002820 0x55cb2de1fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4002820 0x55cb2de1fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50178/0 (socket says 172.21.15.53:50178) 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 -- 172.21.15.53:0/2601105941 learned_addr learned my addr 172.21.15.53:0/2601105941 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c854b640 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 0x55cb2dd92410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7549640 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cb2de36e10 0x55cb2de1f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 -- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cb2de36e10 msgr2=0x55cb2de1f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cb2de36e10 0x55cb2de1f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 -- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 msgr2=0x55cb2dd92410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 0x55cb2dd92410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7d4a640 1 -- 172.21.15.53:0/2601105941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2de3ae70 con 0x7f24b4002820 2024-05-10T10:43:10.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.018+0000 7f24c7549640 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cb2de36e10 0x55cb2de1f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24c7d4a640 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4002820 0x55cb2de1fc10 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f24b80027a0 tx=0x7f24b8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6f688270b0619c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24c6547640 1 -- 172.21.15.53:0/2601105941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8017070 con 0x7f24b4002820 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24c6547640 1 -- 172.21.15.53:0/2601105941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b8002e40 con 0x7f24b4002820 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24c6547640 1 -- 172.21.15.53:0/2601105941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b800fc60 con 0x7f24b4002820 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2601105941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4002820 msgr2=0x55cb2de1fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4002820 0x55cb2de1fc10 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f24b80027a0 tx=0x7f24b8005d80 comp rx=0 tx=0).stop 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2601105941 shutdown_connections 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 0x55cb2dd92410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cb2de36e10 0x55cb2de1f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2601105941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4002820 0x55cb2de1fc10 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2601105941 >> 172.21.15.53:0/2601105941 conn(0x55cb2dd8f880 msgr2=0x55cb2de3b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2601105941 shutdown_connections 2024-05-10T10:43:10.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.019+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2601105941 wait complete. 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.020+0000 7f24cad1cf40 1 Processor -- start 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.020+0000 7f24cad1cf40 1 -- start start 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24cad1cf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 0x55cb2dd89e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24cad1cf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb2de36e10 0x55cb2dd90e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24cad1cf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24b4002820 0x55cb2ded1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24cad1cf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cb2de3b110 con 0x55cb2de36e10 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24cad1cf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cb2de22220 con 0x55cb2dd8c7c0 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24cad1cf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55cb2dd8efd0 con 0x7f24b4002820 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb2de36e10 0x55cb2dd90e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb2de36e10 0x55cb2dd90e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50188/0 (socket says 172.21.15.53:50188) 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 -- 172.21.15.53:0/2269174817 learned_addr learned my addr 172.21.15.53:0/2269174817 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7d4a640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 0x55cb2dd89e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c854b640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24b4002820 0x55cb2ded1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.027 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 -- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24b4002820 msgr2=0x55cb2ded1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24b4002820 0x55cb2ded1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 -- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 msgr2=0x55cb2dd89e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb2dd8c7c0 0x55cb2dd89e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 -- 172.21.15.53:0/2269174817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2dd88df0 con 0x55cb2de36e10 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c7549640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb2de36e10 0x55cb2dd90e70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f24b0026440 tx=0x7f24b0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c5545640 1 -- 172.21.15.53:0/2269174817 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0065070 con 0x55cb2de36e10 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c5545640 1 -- 172.21.15.53:0/2269174817 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b0061650 con 0x55cb2de36e10 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb2dd890d0 con 0x55cb2de36e10 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.021+0000 7f24c5545640 1 -- 172.21.15.53:0/2269174817 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0061a20 con 0x55cb2de36e10 2024-05-10T10:43:10.028 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.022+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2dd89690 con 0x55cb2de36e10 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.023+0000 7f24c5545640 1 -- 172.21.15.53:0/2269174817 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f24b0068070 con 0x55cb2de36e10 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.023+0000 7f24c5545640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24a8074cb0 0x7f24a8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.023+0000 7f24c5545640 1 -- 172.21.15.53:0/2269174817 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f24b00db1e0 con 0x55cb2de36e10 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.024+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55cb2dd913b0 0x55cb2de13b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.024+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) -- 0x55cb2de37340 con 0x55cb2dd913b0 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.024+0000 7f24c7d4a640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24a8074cb0 0x7f24a8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.024+0000 7f24c854b640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55cb2dd913b0 0x55cb2de13b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.031 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.024+0000 7f24c7d4a640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24a8074cb0 0x7f24a8077170 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f24b8005d20 tx=0x7f24b8014000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.024+0000 7f24c854b640 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55cb2dd913b0 0x55cb2de13b30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24c854b640 1 -- 172.21.15.53:0/2269174817 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f24c0009750 con 0x55cb2dd913b0 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55cb2dd913b0 msgr2=0x55cb2de13b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55cb2dd913b0 0x55cb2de13b30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24a8074cb0 msgr2=0x7f24a8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f24a8074cb0 0x7f24a8077170 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f24b8005d20 tx=0x7f24b8014000 comp rx=0 tx=0).stop 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb2de36e10 msgr2=0x55cb2dd90e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.032 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 --2- 172.21.15.53:0/2269174817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb2de36e10 0x55cb2dd90e70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f24b0026440 tx=0x7f24b0000fe0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24c854b640 1 -- 172.21.15.53:0/2269174817 reap_dead start 2024-05-10T10:43:10.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 shutdown_connections 2024-05-10T10:43:10.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 >> 172.21.15.53:0/2269174817 conn(0x55cb2dd8f880 msgr2=0x55cb2dd8e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 shutdown_connections 2024-05-10T10:43:10.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.025+0000 7f24cad1cf40 1 -- 172.21.15.53:0/2269174817 wait complete. 2024-05-10T10:43:10.034 INFO:teuthology.orchestra.run.smithi053.stdout:mytag123 2024-05-10T10:43:10.034 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-05-10T10:43:10.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.078+0000 7fefd09dbf40 1 Processor -- start 2024-05-10T10:43:10.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.078+0000 7fefd09dbf40 1 -- start start 2024-05-10T10:43:10.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefd09dbf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb80030f0 0x55d212288450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefd09dbf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d212304300 0x55d212288f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefd09dbf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2123ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefd09dbf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d2121eaf50 con 0x7fefb80030f0 2024-05-10T10:43:10.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefd09dbf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d2121eef90 con 0x55d2122894b0 2024-05-10T10:43:10.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefd09dbf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d212242320 con 0x55d212304300 2024-05-10T10:43:10.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb80030f0 0x55d212288450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb80030f0 0x55d212288450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50202/0 (socket says 172.21.15.53:50202) 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 -- 172.21.15.53:0/373995491 learned_addr learned my addr 172.21.15.53:0/373995491 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefcecd8640 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2123ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefcdcd6640 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d212304300 0x55d212288f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 -- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d212304300 msgr2=0x55d212288f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d212304300 0x55d212288f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 -- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 msgr2=0x55d2123ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2123ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.079+0000 7fefce4d7640 1 -- 172.21.15.53:0/373995491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d212287c50 con 0x7fefb80030f0 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefcecd8640 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2123ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefce4d7640 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb80030f0 0x55d212288450 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fefc00095e0 tx=0x7fefc0009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b7c87666b4efbc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefcccd4640 1 -- 172.21.15.53:0/373995491 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc000e7e0 con 0x7fefb80030f0 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefcccd4640 1 -- 172.21.15.53:0/373995491 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0013030 con 0x7fefb80030f0 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefcccd4640 1 -- 172.21.15.53:0/373995491 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc000ebc0 con 0x7fefb80030f0 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 -- 172.21.15.53:0/373995491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb80030f0 msgr2=0x55d212288450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb80030f0 0x55d212288450 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fefc00095e0 tx=0x7fefc0009aa0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 -- 172.21.15.53:0/373995491 shutdown_connections 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2123ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55d212304300 0x55d212288f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/373995491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefb80030f0 0x55d212288450 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 -- 172.21.15.53:0/373995491 >> 172.21.15.53:0/373995491 conn(0x55d21226f880 msgr2=0x55d212286f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 -- 172.21.15.53:0/373995491 shutdown_connections 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.080+0000 7fefd09dbf40 1 -- 172.21.15.53:0/373995491 wait complete. 2024-05-10T10:43:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.081+0000 7fefd09dbf40 1 Processor -- start 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.081+0000 7fefd09dbf40 1 -- start start 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefd09dbf40 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2122fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefd09dbf40 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d212304300 0x55d2122fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefd09dbf40 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefb80030f0 0x55d21231b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefd09dbf40 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d2123195d0 con 0x55d212304300 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefd09dbf40 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d212319820 con 0x55d2122894b0 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefd09dbf40 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55d21226fe60 con 0x7fefb80030f0 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d212304300 0x55d2122fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d212304300 0x55d2122fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50218/0 (socket says 172.21.15.53:50218) 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 -- 172.21.15.53:0/4172373297 learned_addr learned my addr 172.21.15.53:0/4172373297 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefce4d7640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2122fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcecd8640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefb80030f0 0x55d21231b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 -- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefb80030f0 msgr2=0x55d21231b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefb80030f0 0x55d21231b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 -- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 msgr2=0x55d2122fd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2122fd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 -- 172.21.15.53:0/4172373297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2122fd2c0 con 0x55d212304300 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefce4d7640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d2122894b0 0x55d2122fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefcdcd6640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d212304300 0x55d2122fc8e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fefc4045d50 tx=0x7fefc4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.082+0000 7fefbf7fe640 1 -- 172.21.15.53:0/4172373297 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4064070 con 0x55d212304300 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.083+0000 7fefbf7fe640 1 -- 172.21.15.53:0/4172373297 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc404e8a0 con 0x55d212304300 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.083+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d212313730 con 0x55d212304300 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.083+0000 7fefbf7fe640 1 -- 172.21.15.53:0/4172373297 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc404ec50 con 0x55d212304300 2024-05-10T10:43:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.083+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d212313cf0 con 0x55d212304300 2024-05-10T10:43:10.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.084+0000 7fefbf7fe640 1 -- 172.21.15.53:0/4172373297 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fefc4067070 con 0x55d212304300 2024-05-10T10:43:10.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.085+0000 7fefbf7fe640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fefb0074ce0 0x7fefb00771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.085+0000 7fefbf7fe640 1 -- 172.21.15.53:0/4172373297 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7fefc40da5a0 con 0x55d212304300 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.085+0000 7fefce4d7640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fefb0074ce0 0x7fefb00771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.085+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55d212245660 0x55d212314720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.085+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119) -- 0x55d212314c60 con 0x55d212245660 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.085+0000 7fefcecd8640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55d212245660 0x55d212314720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.085+0000 7fefcecd8640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55d212245660 0x55d212314720 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefce4d7640 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fefb0074ce0 0x7fefb00771a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009580 tx=0x7fefc001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefcecd8640 1 -- 172.21.15.53:0/4172373297 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fefc8006020 con 0x55d212245660 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55d212245660 msgr2=0x55d212314720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55d212245660 0x55d212314720 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fefb0074ce0 msgr2=0x7fefb00771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fefb0074ce0 0x7fefb00771a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009580 tx=0x7fefc001c040 comp rx=0 tx=0).stop 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d212304300 msgr2=0x55d2122fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 --2- 172.21.15.53:0/4172373297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d212304300 0x55d2122fc8e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fefc4045d50 tx=0x7fefc4003020 comp rx=0 tx=0).stop 2024-05-10T10:43:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefcecd8640 1 -- 172.21.15.53:0/4172373297 reap_dead start 2024-05-10T10:43:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 shutdown_connections 2024-05-10T10:43:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 >> 172.21.15.53:0/4172373297 conn(0x55d21226f880 msgr2=0x55d212288170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 shutdown_connections 2024-05-10T10:43:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.086+0000 7fefd09dbf40 1 -- 172.21.15.53:0/4172373297 wait complete. 2024-05-10T10:43:10.094 INFO:teuthology.orchestra.run.smithi053.stdout:mytag123 2024-05-10T10:43:10.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-10T10:43:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.199+0000 7f24857af640 1 Processor -- start 2024-05-10T10:43:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.199+0000 7f24857af640 1 -- start start 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.199+0000 7f24857af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f248006fe50 0x7f2480072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.199+0000 7f24857af640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f24800b0450 0x7f2480072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.199+0000 7f24857af640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2480073110 con 0x7f2480148bc0 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.199+0000 7f24857af640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f24800affc0 con 0x7f24800b0450 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.199+0000 7f24857af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2480148fa0 con 0x7f248006fe50 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f24800b0450 0x7f2480072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2480148bc0 0x7f248006f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51194/0 (socket says 172.21.15.53:51194) 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247effd640 1 -- 172.21.15.53:0/3382674467 learned_addr learned my addr 172.21.15.53:0/3382674467 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247e7fc640 1 --2- 172.21.15.53:0/3382674467 >> v2:172.21.15.135:3300/0 conn(0x7f248006fe50 0x7f2480072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247f7fe640 1 -- 172.21.15.53:0/3382674467 >> v2:172.21.15.135:3300/0 conn(0x7f248006fe50 msgr2=0x7f2480072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247f7fe640 1 --2- 172.21.15.53:0/3382674467 >> v2:172.21.15.135:3300/0 conn(0x7f248006fe50 0x7f2480072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247f7fe640 1 -- 172.21.15.53:0/3382674467 >> v1:172.21.15.53:6789/0 conn(0x7f2480148bc0 legacy=0x7f248006f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247f7fe640 1 -- 172.21.15.53:0/3382674467 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24800af3b0 con 0x7f24800b0450 2024-05-10T10:43:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.200+0000 7f247dffb640 1 -- 172.21.15.53:0/3382674467 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416822041 0 0) 0x7f2480073110 con 0x7f2480148bc0 2024-05-10T10:43:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f247f7fe640 1 --2- 172.21.15.53:0/3382674467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480072bd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f24800690f0 tx=0x7f247802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a961e3f40e102d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f247dffb640 1 -- 172.21.15.53:0/3382674467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478007740 con 0x7f24800b0450 2024-05-10T10:43:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f247dffb640 1 -- 172.21.15.53:0/3382674467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24780078e0 con 0x7f24800b0450 2024-05-10T10:43:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f247dffb640 1 -- 172.21.15.53:0/3382674467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478007cb0 con 0x7f24800b0450 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 -- 172.21.15.53:0/3382674467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 msgr2=0x7f2480072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 --2- 172.21.15.53:0/3382674467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480072bd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f24800690f0 tx=0x7f247802a5f0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 -- 172.21.15.53:0/3382674467 shutdown_connections 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 --2- 172.21.15.53:0/3382674467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480072bd0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 --2- 172.21.15.53:0/3382674467 >> v2:172.21.15.135:3300/0 conn(0x7f248006fe50 0x7f2480072690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 -- 172.21.15.53:0/3382674467 >> 172.21.15.53:0/3382674467 conn(0x7f24800d1020 msgr2=0x7f24800d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 -- 172.21.15.53:0/3382674467 shutdown_connections 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.201+0000 7f24857af640 1 -- 172.21.15.53:0/3382674467 wait complete. 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.203+0000 7f24857af640 1 Processor -- start 2024-05-10T10:43:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.203+0000 7f24857af640 1 -- start start 2024-05-10T10:43:10.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f248006fe50 0x7f2480146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2480148bc0 0x7f2480155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24800afc50 con 0x7f24800b0450 2024-05-10T10:43:10.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24800af970 con 0x7f2480148bc0 2024-05-10T10:43:10.217 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f24800b3fd0 con 0x7f248006fe50 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50234/0 (socket says 172.21.15.53:50234) 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 -- 172.21.15.53:0/444075460 learned_addr learned my addr 172.21.15.53:0/444075460 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247f7fe640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2480148bc0 0x7f2480155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247effd640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f248006fe50 0x7f2480146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 -- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f248006fe50 msgr2=0x7f2480146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f248006fe50 0x7f2480146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 -- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2480148bc0 msgr2=0x7f2480155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2480148bc0 0x7f2480155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 -- 172.21.15.53:0/444075460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24801451f0 con 0x7f24800b0450 2024-05-10T10:43:10.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247f7fe640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2480148bc0 0x7f2480155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f247e7fc640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480144aa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f247400d800 tx=0x7f247400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24677fe640 1 -- 172.21.15.53:0/444075460 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247400b6c0 con 0x7f24800b0450 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24677fe640 1 -- 172.21.15.53:0/444075460 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2474019070 con 0x7f24800b0450 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24677fe640 1 -- 172.21.15.53:0/444075460 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247400ba80 con 0x7f24800b0450 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24801454d0 con 0x7f24800b0450 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.204+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2480145a90 con 0x7f24800b0450 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.206+0000 7f24677fe640 1 -- 172.21.15.53:0/444075460 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f247400bc20 con 0x7f24800b0450 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.207+0000 7f24677fe640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f246c0762d0 0x7f246c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.207+0000 7f24677fe640 1 -- 172.21.15.53:0/444075460 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f2474097d50 con 0x7f24800b0450 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.207+0000 7f247effd640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f246c0762d0 0x7f246c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.207+0000 7f24885a0740 1 Processor -- start 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.207+0000 7f247effd640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f246c0762d0 0x7f246c078790 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2470000f60 tx=0x7f2470009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.207+0000 7f24885a0740 1 -- start start 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x561330693af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f24885a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c3b40 0x561330698950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f24885a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561330660690 con 0x5613306c3b40 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613305fdf30 con 0x5613306e1160 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5613305b6b20 con 0x5613306c4280 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c3b40 0x561330698950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c3b40 0x561330698950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50238/0 (socket says 172.21.15.53:50238) 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 -- 172.21.15.53:0/2955803608 learned_addr learned my addr 172.21.15.53:0/2955803608 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247effd640 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x561330693af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247e7fc640 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 -- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 msgr2=0x5613306c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 -- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 msgr2=0x561330693af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x561330693af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.208+0000 7f247f7fe640 1 -- 172.21.15.53:0/2955803608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613305aee60 con 0x5613306c3b40 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.209+0000 7f247e7fc640 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.209+0000 7f247f7fe640 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c3b40 0x561330698950 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2478009e50 tx=0x7f247802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36f3eba2b440499 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.209+0000 7f2464ff9640 1 -- 172.21.15.53:0/2955803608 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478004040 con 0x5613306c3b40 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.209+0000 7f2464ff9640 1 -- 172.21.15.53:0/2955803608 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478041070 con 0x5613306c3b40 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.209+0000 7f2464ff9640 1 -- 172.21.15.53:0/2955803608 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24780043e0 con 0x5613306c3b40 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.209+0000 7f24885a0740 1 -- 172.21.15.53:0/2955803608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c3b40 msgr2=0x561330698950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.209+0000 7f24885a0740 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c3b40 0x561330698950 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2478009e50 tx=0x7f247802e000 comp rx=0 tx=0).stop 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 -- 172.21.15.53:0/2955803608 shutdown_connections 2024-05-10T10:43:10.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c3b40 0x561330698950 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 --2- 172.21.15.53:0/2955803608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x561330693af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 -- 172.21.15.53:0/2955803608 >> 172.21.15.53:0/2955803608 conn(0x561330695d60 msgr2=0x5613306de600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 -- 172.21.15.53:0/2955803608 shutdown_connections 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 -- 172.21.15.53:0/2955803608 wait complete. 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 -- start start 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.210+0000 7f24885a0740 1 Processor -- start 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306e0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f24885a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306e1160 0x5613306f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306f2070 0x5613306b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f24885a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5613305ba5e0 con 0x5613306e1160 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613305f9ef0 con 0x5613306f2070 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x561330697f10 con 0x5613306c4280 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f24885a0740 10 client.0 _ll_register_callbacks cb 0x5613305b5320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306e1160 0x5613306f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306e1160 0x5613306f1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50252/0 (socket says 172.21.15.53:50252) 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 -- 172.21.15.53:0/1517503764 learned_addr learned my addr 172.21.15.53:0/1517503764 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247f7fe640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306f2070 0x5613306b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247effd640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 -- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 msgr2=0x5613306e0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.223 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306e0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 -- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306f2070 msgr2=0x5613306b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306f2070 0x5613306b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247e7fc640 1 -- 172.21.15.53:0/1517503764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613303d97d0 con 0x5613306e1160 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247f7fe640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306f2070 0x5613306b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.211+0000 7f247effd640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f247e7fc640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306e1160 0x5613306f1b30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f247405cdb0 tx=0x7f24740604a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854790012562778b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f245dffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f245dffb640 1 -- 172.21.15.53:0/1517503764 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474061430 con 0x5613306e1160 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f245dffb640 1 -- 172.21.15.53:0/1517503764 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f24740672e0 con 0x5613306e1160 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2478041070 con 0x5613306e1160 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f245dffb640 1 -- 172.21.15.53:0/1517503764 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f2474060d30 con 0x5613306e1160 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 10 client.8334 fetch_fsmap learned FSMap version 104 2024-05-10T10:43:10.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x56133030f3f0 con 0x5613306e1160 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f245dffb640 1 -- 172.21.15.53:0/1517503764 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478041070 con 0x5613306e1160 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f245dffb640 1 -- 172.21.15.53:0/1517503764 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f2474093060 con 0x5613306e1160 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 10 client.8334 fetch_fsmap finished waiting for FSMap version 104 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 10 client.8334 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 20 client.8334 populate_metadata read hostname 'smithi053' 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306c4770 0x5613306c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 4 client.8334 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 --> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56133060bb40 con 0x5613306c4770 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.212+0000 7f247effd640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306c4770 0x5613306c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f247effd640 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306c4770 0x5613306c4bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d6a2dfa36526e3 server_cookie=5d983747724af7c5 in_seq=0 out_seq=0 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f245dffb640 10 client.8334 ms_handle_connect on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:10.225 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f245dffb640 1 -- 172.21.15.53:0/1517503764 <== mds.0 v2:172.21.15.53:6832/38461894 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56133060bb40 con 0x5613306c4770 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f245dffb640 10 client.8334 handle_command_reply: tid=0 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 client.8334 shutdown 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306e1160 msgr2=0x5613306f1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306e1160 0x5613306f1b30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f247405cdb0 tx=0x7f24740604a0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 shutdown_connections 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306c4770 0x5613306c4bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306f2070 0x5613306b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306e1160 0x5613306f1b30 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 --2- 172.21.15.53:0/1517503764 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306c4280 0x5613306e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f245dffb640 0 client.8334 ms_handle_reset on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 >> 172.21.15.53:0/1517503764 conn(0x5613306c3b40 msgr2=0x561330693230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 shutdown_connections 2024-05-10T10:43:10.226 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.213+0000 7f24885a0740 1 -- 172.21.15.53:0/1517503764 wait complete. 2024-05-10T10:43:10.227 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.214+0000 7f24885a0740 20 client.8334 trim_cache size 0 max 16384 2024-05-10T10:43:10.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.229+0000 7f24885a0740 1 Processor -- start 2024-05-10T10:43:10.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.229+0000 7f24885a0740 1 -- start start 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f24885a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x561330699b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f24885a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5613306c56a0 con 0x5613306c4280 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2480201b80 con 0x5613306e1160 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f2480200b80 con 0x5613306f2070 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x561330699b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x561330699b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50262/0 (socket says 172.21.15.53:50262) 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 -- 172.21.15.53:0/2138386331 learned_addr learned my addr 172.21.15.53:0/2138386331 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247e7fc640 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247f7fe640 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 -- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 msgr2=0x5613306dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306dcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 -- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 msgr2=0x5613306e1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306e1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 -- 172.21.15.53:0/2138386331 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613302cf0f0 con 0x5613306c4280 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247f7fe640 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f247effd640 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x561330699b80 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x5613306c5300 tx=0x7f2470017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea4456fcb05b9a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f245ffff640 1 -- 172.21.15.53:0/2138386331 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470007660 con 0x5613306c4280 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f245ffff640 1 -- 172.21.15.53:0/2138386331 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247002b070 con 0x5613306c4280 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.230+0000 7f245ffff640 1 -- 172.21.15.53:0/2138386331 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247000e7b0 con 0x5613306c4280 2024-05-10T10:43:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 -- 172.21.15.53:0/2138386331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 msgr2=0x561330699b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x561330699b80 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x5613306c5300 tx=0x7f2470017040 comp rx=0 tx=0).stop 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 -- 172.21.15.53:0/2138386331 shutdown_connections 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 --2- 172.21.15.53:0/2138386331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x561330699b80 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 -- 172.21.15.53:0/2138386331 >> 172.21.15.53:0/2138386331 conn(0x5613306c3b40 msgr2=0x561330694830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 -- 172.21.15.53:0/2138386331 shutdown_connections 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 -- 172.21.15.53:0/2138386331 wait complete. 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 -- start start 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.231+0000 7f24885a0740 1 Processor -- start 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f24885a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x56133075b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f24885a0740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f24885a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f248013fcc0 con 0x5613306c4280 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f248013f670 con 0x5613306e1160 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f24885a0740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x5613306dfe50 con 0x5613306f2070 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f24885a0740 10 client.0 _ll_register_callbacks cb 0x7f24740046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f247effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x56133075b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f247effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x56133075b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50270/0 (socket says 172.21.15.53:50270) 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f247effd640 1 -- 172.21.15.53:0/217968145 learned_addr learned my addr 172.21.15.53:0/217968145 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f247f7fe640 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.232+0000 7f247e7fc640 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f247effd640 1 -- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 msgr2=0x5613306ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f247effd640 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f247effd640 1 -- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 msgr2=0x5613306c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f247effd640 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f247effd640 1 -- 172.21.15.53:0/217968145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561330700fe0 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f247effd640 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x56133075b910 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f24800d1bb0 tx=0x7f2470000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcaffcb9d93c598 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f245dffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f245dffb640 1 -- 172.21.15.53:0/217968145 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470033020 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f245dffb640 1 -- 172.21.15.53:0/217968145 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f247001e040 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f247002b070 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f245dffb640 1 -- 172.21.15.53:0/217968145 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f247003d050 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 10 client.8340 fetch_fsmap learned FSMap version 104 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5613306debe0 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f245dffb640 1 -- 172.21.15.53:0/217968145 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247002b070 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f245dffb640 1 -- 172.21.15.53:0/217968145 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f2470040020 con 0x5613306c4280 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 10 client.8340 fetch_fsmap finished waiting for FSMap version 104 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 10 client.8340 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-10T10:43:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 20 client.8340 populate_metadata read hostname 'smithi053' 2024-05-10T10:43:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306ed240 0x5613306d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 4 client.8340 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T10:43:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.233+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 --> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613306c5bd0 con 0x5613306ed240 2024-05-10T10:43:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.234+0000 7f247e7fc640 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306ed240 0x5613306d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.234+0000 7f247e7fc640 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306ed240 0x5613306d47f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd623297fe6c17a server_cookie=e3c0559f416cd8b6 in_seq=0 out_seq=0 2024-05-10T10:43:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.234+0000 7f245dffb640 10 client.8340 ms_handle_connect on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.242+0000 7f245dffb640 1 -- 172.21.15.53:0/217968145 <== mds.0 v2:172.21.15.53:6832/38461894 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613306c5bd0 con 0x5613306ed240 2024-05-10T10:43:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.242+0000 7f245dffb640 10 client.8340 handle_command_reply: tid=0 2024-05-10T10:43:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 client.8340 shutdown 2024-05-10T10:43:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 msgr2=0x56133075b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x56133075b910 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f24800d1bb0 tx=0x7f2470000ed0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 shutdown_connections 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:6832/38461894,v1:172.21.15.53:6834/38461894] conn(0x5613306ed240 0x5613306d47f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5613306f2070 0x5613306ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f245dffb640 0 client.8340 ms_handle_reset on v2:172.21.15.53:6832/38461894 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613306e1160 0x5613306c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 --2- 172.21.15.53:0/217968145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5613306c4280 0x56133075b910 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 >> 172.21.15.53:0/217968145 conn(0x5613306c3b40 msgr2=0x561330694830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 shutdown_connections 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 1 -- 172.21.15.53:0/217968145 wait complete. 2024-05-10T10:43:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24885a0740 20 client.8340 trim_cache size 0 max 16384 2024-05-10T10:43:10.247 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:10.247 INFO:teuthology.orchestra.run.smithi053.stdout: "message": "", 2024-05-10T10:43:10.247 INFO:teuthology.orchestra.run.smithi053.stdout: "return_code": 0 2024-05-10T10:43:10.247 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f246c0762d0 msgr2=0x7f246c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24857af640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f246c0762d0 0x7f246c078790 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2470000f60 tx=0x7f2470009040 comp rx=0 tx=0).stop 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 msgr2=0x7f2480144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.243+0000 7f24857af640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480144aa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f247400d800 tx=0x7f247400dcc0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 shutdown_connections 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f246c0762d0 0x7f246c078790 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2480148bc0 0x7f2480155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24800b0450 0x7f2480144aa0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 --2- 172.21.15.53:0/444075460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f248006fe50 0x7f2480146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 >> 172.21.15.53:0/444075460 conn(0x7f24800d1020 msgr2=0x7f24800b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 shutdown_connections 2024-05-10T10:43:10.249 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.244+0000 7f24857af640 1 -- 172.21.15.53:0/444075460 wait complete. 2024-05-10T10:43:10.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:43:10.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.361+0000 7fe7570f8640 1 Processor -- start 2024-05-10T10:43:10.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.361+0000 7fe7570f8640 1 -- start start 2024-05-10T10:43:10.365 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe7570f8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe75015f2c0 0x7fe750163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe7570f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500c6de0 0x7fe750073000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe7570f8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7500cdb80 0x7fe7500ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe7570f8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe7500cc3a0 con 0x7fe7500cdb80 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe7570f8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe750073710 con 0x7fe75015f2c0 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe7570f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe7500739f0 con 0x7fe7500c6de0 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7500cdb80 0x7fe7500ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7500cdb80 0x7fe7500ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50284/0 (socket says 172.21.15.53:50284) 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 -- 172.21.15.53:0/428118204 learned_addr learned my addr 172.21.15.53:0/428118204 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe754e94640 1 --2- 172.21.15.53:0/428118204 >> v2:172.21.15.135:3300/0 conn(0x7fe75015f2c0 0x7fe750163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe747fff640 1 --2- 172.21.15.53:0/428118204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500c6de0 0x7fe750073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 -- 172.21.15.53:0/428118204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500c6de0 msgr2=0x7fe750073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.366 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 --2- 172.21.15.53:0/428118204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500c6de0 0x7fe750073000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 -- 172.21.15.53:0/428118204 >> v2:172.21.15.135:3300/0 conn(0x7fe75015f2c0 msgr2=0x7fe750163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 --2- 172.21.15.53:0/428118204 >> v2:172.21.15.135:3300/0 conn(0x7fe75015f2c0 0x7fe750163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe755695640 1 -- 172.21.15.53:0/428118204 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe750073cd0 con 0x7fe7500cdb80 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.362+0000 7fe754e94640 1 --2- 172.21.15.53:0/428118204 >> v2:172.21.15.135:3300/0 conn(0x7fe75015f2c0 0x7fe750163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.363+0000 7fe755695640 1 --2- 172.21.15.53:0/428118204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7500cdb80 0x7fe7500ce030 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe7500743d0 tx=0x7fe74802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51e0430b6d616bb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.363+0000 7fe7477fe640 1 -- 172.21.15.53:0/428118204 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748036070 con 0x7fe7500cdb80 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.363+0000 7fe7477fe640 1 -- 172.21.15.53:0/428118204 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe748031600 con 0x7fe7500cdb80 2024-05-10T10:43:10.367 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.363+0000 7fe7477fe640 1 -- 172.21.15.53:0/428118204 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480319d0 con 0x7fe7500cdb80 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 -- 172.21.15.53:0/428118204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7500cdb80 msgr2=0x7fe7500ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 --2- 172.21.15.53:0/428118204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7500cdb80 0x7fe7500ce030 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe7500743d0 tx=0x7fe74802a610 comp rx=0 tx=0).stop 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 -- 172.21.15.53:0/428118204 shutdown_connections 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 --2- 172.21.15.53:0/428118204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7500cdb80 0x7fe7500ce030 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 --2- 172.21.15.53:0/428118204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500c6de0 0x7fe750073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 --2- 172.21.15.53:0/428118204 >> v2:172.21.15.135:3300/0 conn(0x7fe75015f2c0 0x7fe750163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 -- 172.21.15.53:0/428118204 >> 172.21.15.53:0/428118204 conn(0x7fe7500d1d50 msgr2=0x7fe7500d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 -- 172.21.15.53:0/428118204 shutdown_connections 2024-05-10T10:43:10.368 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.364+0000 7fe7570f8640 1 -- 172.21.15.53:0/428118204 wait complete. 2024-05-10T10:43:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.365+0000 7fe7570f8640 1 Processor -- start 2024-05-10T10:43:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.365+0000 7fe7570f8640 1 -- start start 2024-05-10T10:43:10.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe7570f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7500c6de0 0x7fe7501e8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe7570f8640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500cdb80 0x7fe75015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe7570f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe75015f2c0 0x7fe75014d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe7570f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe75015f820 con 0x7fe75015f2c0 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe7570f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe75015f6a0 con 0x7fe7500c6de0 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe7570f8640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fe7500cf920 con 0x7fe7500cdb80 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe75015f2c0 0x7fe75014d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe75015f2c0 0x7fe75014d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50300/0 (socket says 172.21.15.53:50300) 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 -- 172.21.15.53:0/2061086776 learned_addr learned my addr 172.21.15.53:0/2061086776 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe754e94640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7500c6de0 0x7fe7501e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe747fff640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500cdb80 0x7fe75015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 -- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500cdb80 msgr2=0x7fe75015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500cdb80 0x7fe75015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 -- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7500c6de0 msgr2=0x7fe7501e8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7500c6de0 0x7fe7501e8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.366+0000 7fe755695640 1 -- 172.21.15.53:0/2061086776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7500b4f10 con 0x7fe75015f2c0 2024-05-10T10:43:10.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.367+0000 7fe755695640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe75015f2c0 0x7fe75014d6f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe74802ac00 tx=0x7fe748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.367+0000 7fe7457fa640 1 -- 172.21.15.53:0/2061086776 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74803f020 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.367+0000 7fe7457fa640 1 -- 172.21.15.53:0/2061086776 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe748037040 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.367+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7501e9010 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.367+0000 7fe7457fa640 1 -- 172.21.15.53:0/2061086776 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748036070 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.367+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501e95d0 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.368+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe750069400 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.369+0000 7fe7457fa640 1 -- 172.21.15.53:0/2061086776 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe748042020 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.369+0000 7fe7457fa640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe73c0762d0 0x7fe73c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.369+0000 7fe7457fa640 1 -- 172.21.15.53:0/2061086776 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7fe7480b7b80 con 0x7fe75015f2c0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.370+0000 7fe754e94640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe73c0762d0 0x7fe73c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.370+0000 7fe754e94640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe73c0762d0 0x7fe73c078790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00aa40 tx=0x7fe74c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.374+0000 7fe7457fa640 1 -- 172.21.15.53:0/2061086776 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fe748049030 con 0x7fe75015f2c0 2024-05-10T10:43:10.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.603+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe7500b50e0 con 0x7fe75015f2c0 2024-05-10T10:43:10.605 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:43:10.603+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:43:10.606 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.605+0000 7fe7457fa640 1 -- 172.21.15.53:0/2061086776 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) ==== 161+0+0 (secure 0 0 0) 0x7fe748080d40 con 0x7fe75015f2c0 2024-05-10T10:43:10.606 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:43:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe73c0762d0 msgr2=0x7fe73c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe73c0762d0 0x7fe73c078790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00aa40 tx=0x7fe74c008040 comp rx=0 tx=0).stop 2024-05-10T10:43:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe75015f2c0 msgr2=0x7fe75014d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe75015f2c0 0x7fe75014d6f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe74802ac00 tx=0x7fe748004a40 comp rx=0 tx=0).stop 2024-05-10T10:43:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 shutdown_connections 2024-05-10T10:43:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fe73c0762d0 0x7fe73c078790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe75015f2c0 0x7fe75014d6f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7500cdb80 0x7fe75015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 --2- 172.21.15.53:0/2061086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7500c6de0 0x7fe7501e8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 >> 172.21.15.53:0/2061086776 conn(0x7fe7500d1d50 msgr2=0x7fe7500c71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 shutdown_connections 2024-05-10T10:43:10.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.607+0000 7fe7570f8640 1 -- 172.21.15.53:0/2061086776 wait complete. 2024-05-10T10:43:10.612 INFO:tasks.ceph.mds.c.smithi053.stderr: -1843> 2024-05-10T10:43:05.845+0000 7fcd1b901640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:43:10.624 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-05-10T10:43:10.645 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 -- start start 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 Processor -- start 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76b518c00 0x55e76c2f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76b519800 0x55e76c2f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76b519c00 0x55e76c2f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b6180 con 0x55e76b519c00 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b6300 con 0x55e76b518c00 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e76c2b6480 con 0x55e76b519800 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.667+0000 7ff7b85d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76b519c00 0x55e76c2f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76b519c00 0x55e76c2f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50330/0 (socket says 172.21.15.53:50330) 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 -- 172.21.15.53:0/1748916108 learned_addr learned my addr 172.21.15.53:0/1748916108 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76b518c00 0x55e76c2f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 -- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76b519800 msgr2=0x55e76c2f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76b519800 0x55e76c2f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76b519800 0x55e76c2f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 -- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76b518c00 msgr2=0x55e76c2f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76b518c00 0x55e76c2f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 -- 172.21.15.53:0/1748916108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76b54e780 con 0x55e76b519c00 2024-05-10T10:43:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76b518c00 0x55e76c2f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76b519c00 0x55e76c2f0b00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x55e76c343710 tx=0x55e76c2b9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44c6999d8236cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b55cd640 1 -- 172.21.15.53:0/1748916108 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53e9c0 con 0x55e76b519c00 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b55cd640 1 -- 172.21.15.53:0/1748916108 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e76b544fc0 con 0x55e76b519c00 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b9626280 1 -- 172.21.15.53:0/1748916108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76b54e3c0 con 0x55e76b519c00 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b55cd640 1 -- 172.21.15.53:0/1748916108 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53e680 con 0x55e76b519c00 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b9626280 1 -- 172.21.15.53:0/1748916108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55e76b54f680 con 0x55e76b519c00 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b9626280 4 waiting for MDS map... 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b55cd640 1 -- 172.21.15.53:0/1748916108 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x55e76c3f4200 con 0x55e76b519c00 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b55cd640 1 -- 172.21.15.53:0/1748916108 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x55e76b573180 con 0x55e76b519c00 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.668+0000 7ff7b9626280 4 Got MDS map 105 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.669+0000 7ff7b9626280 10 main: JournalTool::main 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.669+0000 7ff7b9626280 4 main: JournalTool: connecting to RADOS... 2024-05-10T10:43:10.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.669+0000 7ff7b9626280 1 Processor -- start 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.669+0000 7ff7b9626280 1 -- start start 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b9626280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b6600 con 0x55e76c44e400 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b6780 con 0x55e76c44e800 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e76c2b6900 con 0x55e76c44e000 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50334/0 (socket says 172.21.15.53:50334) 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/2143411029 learned_addr learned my addr 172.21.15.53:0/2143411029 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 msgr2=0x55e76c2f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e800 msgr2=0x55e76c2f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/2143411029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76b54f4a0 con 0x55e76c44e400 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x55e76c326750 tx=0x55e76c406ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315de4ca513877f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.670+0000 7ff7b35c9640 1 -- 172.21.15.53:0/2143411029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53e9c0 con 0x55e76c44e400 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b35c9640 1 -- 172.21.15.53:0/2143411029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e76b544c40 con 0x55e76c44e400 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b35c9640 1 -- 172.21.15.53:0/2143411029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53e680 con 0x55e76c44e400 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 -- 172.21.15.53:0/2143411029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e400 msgr2=0x55e76c2f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x55e76c326750 tx=0x55e76c406ea0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 -- 172.21.15.53:0/2143411029 shutdown_connections 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.686 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 --2- 172.21.15.53:0/2143411029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 -- 172.21.15.53:0/2143411029 >> 172.21.15.53:0/2143411029 conn(0x55e76c309c00 msgr2=0x55e76b613000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 -- 172.21.15.53:0/2143411029 shutdown_connections 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.671+0000 7ff7b9626280 1 -- 172.21.15.53:0/2143411029 wait complete. 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 Processor -- start 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 -- start start 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b7c80 con 0x55e76c44e800 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b7b00 con 0x55e76c44e400 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e76c2b7980 con 0x55e76c44e000 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b75d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b75d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50340/0 (socket says 172.21.15.53:50340) 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b75d1640 1 -- 172.21.15.53:0/4025877831 learned_addr learned my addr 172.21.15.53:0/4025877831 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.672+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b75d1640 1 -- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 msgr2=0x55e76c2f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b75d1640 1 -- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e400 msgr2=0x55e76c2f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b75d1640 1 -- 172.21.15.53:0/4025877831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76b54e000 con 0x55e76c44e800 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x55e76c3279b0 tx=0x55e76c406d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b1dc6640 1 -- 172.21.15.53:0/4025877831 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53e9c0 con 0x55e76c44e800 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b1dc6640 1 -- 172.21.15.53:0/4025877831 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e76b544a80 con 0x55e76c44e800 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e76b54e960 con 0x55e76c44e800 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b1dc6640 1 -- 172.21.15.53:0/4025877831 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53e680 con 0x55e76c44e800 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76b54f860 con 0x55e76c44e800 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.673+0000 7ff7b9626280 4 main: JournalTool: resolving pool 17 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.674+0000 7ff7b1dc6640 1 -- 172.21.15.53:0/4025877831 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55e76c45ec00 con 0x55e76c44e800 2024-05-10T10:43:10.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.675+0000 7ff7b1dc6640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55e76c474c00 0x55e76c2f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.675+0000 7ff7b1dc6640 1 -- 172.21.15.53:0/4025877831 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x55e76c45d000 con 0x55e76c44e800 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.675+0000 7ff7b9626280 4 main: JournalTool: creating IoCtx.. 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.675+0000 7ff7b9626280 4 main: Executing for rank 0 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.675+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55e76c474c00 0x55e76c2f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.675+0000 7ff7b9626280 1 -- start start 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55e76c474c00 0x55e76c2f2100 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55e76c326750 tx=0x55e76c406c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b9626280 1 Processor -- start 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44f000 0x55e76c2f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c475000 0x55e76c2f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b9626280 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c475400 0x55e76c2f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b9626280 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b7680 con 0x55e76c44f000 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e76c2b6600 con 0x55e76c475400 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b9626280 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55e76c2b6780 con 0x55e76c475000 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b85d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44f000 0x55e76c2f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b85d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44f000 0x55e76c2f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50346/0 (socket says 172.21.15.53:50346) 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b85d3640 1 -- 172.21.15.53:0/738263138 learned_addr learned my addr 172.21.15.53:0/738263138 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.676+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c475000 0x55e76c2f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b85d3640 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c475000 msgr2=0x55e76c2f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c475000 0x55e76c2f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b85d3640 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c475400 msgr2=0x55e76c2f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c475400 0x55e76c2f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c475400 0x55e76c2f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b85d3640 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76b54fa40 con 0x55e76c44f000 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c475400 0x55e76c2f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44f000 0x55e76c2f2680 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x55e76b84d800 tx=0x55e76c2b88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2855b70785706b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7aedc0640 1 -- 172.21.15.53:0/738263138 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53eb60 con 0x55e76c44f000 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7aedc0640 1 -- 172.21.15.53:0/738263138 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e76b544e00 con 0x55e76c44f000 2024-05-10T10:43:10.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76b54e000 con 0x55e76c44f000 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55e76b54e960 con 0x55e76c44f000 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7b9626280 4 waiting for MDS map... 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.677+0000 7ff7aedc0640 1 -- 172.21.15.53:0/738263138 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e76b53e820 con 0x55e76c44f000 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.678+0000 7ff7aedc0640 1 -- 172.21.15.53:0/738263138 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x55e76c44c800 con 0x55e76c44f000 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.678+0000 7ff7aedc0640 1 -- 172.21.15.53:0/738263138 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x55e76b573500 con 0x55e76c44f000 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.678+0000 7ff7b9626280 4 Got MDS map 105 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.678+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55e76c475800 0x55e76c2f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.678+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76c475c00 con 0x55e76c475800 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.678+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55e76c475800 0x55e76c2f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.678+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55e76c475800 0x55e76c2f3700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=236295212ccfa622 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.680+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/738263138 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v120'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55e76b626000 con 0x55e76c475800 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.680+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55e76b911800 0x55e76b930000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.680+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76b911c00 con 0x55e76b911800 2024-05-10T10:43:10.691 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.680+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55e76b911800 0x55e76b930000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.680+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55e76b911800 0x55e76b930000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=8d7f9accb31bc14c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.692 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.681+0000 7ff7b75d1640 1 -- 172.21.15.53:0/738263138 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v120'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55e76b626500 con 0x55e76b911800 2024-05-10T10:43:10.692 INFO:teuthology.orchestra.run.smithi053.stdout:writing EResetJournal entry 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.681+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76b93c000 con 0x55e76b911800 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55e76b93c400 0x55e76b930580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76b93c800 con 0x55e76b93c400 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e76b93cc00 0x55e76b930b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76b93d000 con 0x55e76b93cc00 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55e76b93d400 0x55e76b931080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76b93d800 con 0x55e76b93d400 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76b93dc00 con 0x55e76b93cc00 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76b942000 con 0x55e76c475800 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55e76b93c400 0x55e76b930580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55e76b93d400 0x55e76b931080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e76b93cc00 0x55e76b930b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b85d3640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55e76b93c400 0x55e76b930580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=ed2f8676e18f88d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.682+0000 7ff7b75d1640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55e76b93d400 0x55e76b931080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=82c8b9eb052b50a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.683+0000 7ff7b75d1640 1 -- 172.21.15.53:0/738263138 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== osd_op_reply(3 200.00000001 [delete] v120'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55e76b626500 con 0x55e76b911800 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.683+0000 7ff7b7dd2640 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e76b93cc00 0x55e76b930b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=970726f151013cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.683+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/738263138 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== osd_op_reply(8 200.00000006 [delete] v120'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55e76b626000 con 0x55e76c475800 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.684+0000 7ff7b85d3640 1 -- 172.21.15.53:0/738263138 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(4 200.00000002 [delete] v120'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55e76b626780 con 0x55e76b93c400 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.684+0000 7ff7ad5bd640 1 -- 172.21.15.53:0/738263138 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55e76c44fc00 con 0x55e76b911800 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.684+0000 7ff7b75d1640 1 -- 172.21.15.53:0/738263138 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(6 200.00000004 [delete] v120'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55e76b626500 con 0x55e76b93d400 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.684+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/738263138 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(5 200.00000003 [delete] v120'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55e76b626000 con 0x55e76b93cc00 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b7dd2640 1 -- 172.21.15.53:0/738263138 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(7 200.00000005 [delete] v120'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55e76b626000 con 0x55e76b93cc00 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b75d1640 1 -- 172.21.15.53:0/738263138 <== osd.3 v2:172.21.15.53:6816/2871564334 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v120'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55e76b626500 con 0x55e76b911800 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-05-10T10:43:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e76b93cc00 msgr2=0x55e76b930b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55e76b93cc00 0x55e76b930b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55e76c475800 msgr2=0x55e76c2f3700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55e76c475800 0x55e76c2f3700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55e76b911800 msgr2=0x55e76b930000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55e76b911800 0x55e76b930000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55e76b93c400 msgr2=0x55e76b930580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55e76b93c400 0x55e76b930580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55e76b93d400 msgr2=0x55e76b931080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55e76b93d400 0x55e76b931080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b75d1640 1 -- 172.21.15.53:0/738263138 reap_dead start 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44f000 msgr2=0x55e76c2f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/738263138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44f000 0x55e76c2f2680 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x55e76b84d800 tx=0x55e76c2b88a0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b75d1640 1 -- 172.21.15.53:0/738263138 reap_dead start 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b75d1640 1 -- 172.21.15.53:0/738263138 reap_dead start 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 shutdown_connections 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 >> 172.21.15.53:0/738263138 conn(0x55e76c44ec00 msgr2=0x55e76b613800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 shutdown_connections 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/738263138 wait complete. 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55e76c474c00 msgr2=0x55e76c2f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55e76c474c00 0x55e76c2f2100 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55e76c326750 tx=0x55e76c406c00 comp rx=0 tx=0).stop 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e800 msgr2=0x55e76c2f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.685+0000 7ff7b9626280 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x55e76c3279b0 tx=0x55e76c406d20 comp rx=0 tx=0).stop 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 shutdown_connections 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55e76c474c00 0x55e76c2f2100 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76c44e800 0x55e76c2f1b80 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76c44e400 0x55e76c2f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/4025877831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76c44e000 0x55e76c2f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 >> 172.21.15.53:0/4025877831 conn(0x55e76c309c00 msgr2=0x55e76b613000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 shutdown_connections 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/4025877831 wait complete. 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/1748916108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76b519c00 msgr2=0x55e76c2f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76b519c00 0x55e76c2f0b00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x55e76c343710 tx=0x55e76c2b9f80 comp rx=0 tx=0).stop 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/1748916108 shutdown_connections 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e76b519c00 0x55e76c2f0b00 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55e76b519800 0x55e76c2f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 --2- 172.21.15.53:0/1748916108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e76b518c00 0x55e76c2f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/1748916108 >> 172.21.15.53:0/1748916108 conn(0x55e76b519000 msgr2=0x55e76b612800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/1748916108 shutdown_connections 2024-05-10T10:43:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.686+0000 7ff7b9626280 1 -- 172.21.15.53:0/1748916108 wait complete. 2024-05-10T10:43:10.701 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.077810, result: writing EResetJournal entry 2024-05-10T10:43:10.702 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-10T10:43:10.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.777+0000 7fa36e322740 1 -- start start 2024-05-10T10:43:10.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.777+0000 7fa36e322740 1 Processor -- start 2024-05-10T10:43:10.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36e322740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef44786c00 0x55ef4554e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36e322740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef44787800 0x55ef4554e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36e322740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef44787c00 0x55ef4554eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36e322740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef4488a180 con 0x55ef44787800 2024-05-10T10:43:10.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36e322740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef4488a300 con 0x55ef44786c00 2024-05-10T10:43:10.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36e322740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ef4488a480 con 0x55ef44787c00 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36b9db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef44787800 0x55ef4554e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36b9db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef44787800 0x55ef4554e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50372/0 (socket says 172.21.15.53:50372) 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36b9db640 1 -- 172.21.15.53:0/2033563701 learned_addr learned my addr 172.21.15.53:0/2033563701 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36b9db640 1 -- 172.21.15.53:0/2033563701 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef44787c00 msgr2=0x55ef4554eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36b9db640 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef44787c00 0x55ef4554eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef44786c00 0x55ef4554e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.778+0000 7fa36b9db640 1 -- 172.21.15.53:0/2033563701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef44786c00 msgr2=0x55ef4554e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36b9db640 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef44786c00 0x55ef4554e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36b9db640 1 -- 172.21.15.53:0/2033563701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef447b3680 con 0x55ef44787800 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36b9db640 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef44787800 0x55ef4554e580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x55ef4559fbc0 tx=0x55ef4552cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae51b07e30f3524 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa3699d7640 1 -- 172.21.15.53:0/2033563701 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef447ac680 con 0x55ef44787800 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa3699d7640 1 -- 172.21.15.53:0/2033563701 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ef447ce540 con 0x55ef44787800 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa3699d7640 1 -- 172.21.15.53:0/2033563701 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef447acb60 con 0x55ef44787800 2024-05-10T10:43:10.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36e322740 1 -- 172.21.15.53:0/2033563701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef447b34a0 con 0x55ef44787800 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36e322740 1 -- 172.21.15.53:0/2033563701 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ef447b3a40 con 0x55ef44787800 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36e322740 4 waiting for MDS map... 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa3699d7640 1 -- 172.21.15.53:0/2033563701 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x55ef4565c600 con 0x55ef44787800 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa3699d7640 1 -- 172.21.15.53:0/2033563701 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x55ef447df180 con 0x55ef44787800 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36e322740 4 Got MDS map 105 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36e322740 4 datascan.main: Using metadata pool output 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.779+0000 7fa36e322740 4 datascan.main: connecting to RADOS... 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 Processor -- start 2024-05-10T10:43:10.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 -- start start 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef4488a600 con 0x55ef45663c00 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef4488a780 con 0x55ef45663400 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.780+0000 7fa36e322740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ef4488a900 con 0x55ef45663800 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50386/0 (socket says 172.21.15.53:50386) 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 -- 172.21.15.53:0/496956177 learned_addr learned my addr 172.21.15.53:0/496956177 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 -- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663800 msgr2=0x55ef4554f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 -- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663400 msgr2=0x55ef4554f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 -- 172.21.15.53:0/496956177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef447b3860 con 0x55ef45663c00 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa36b9db640 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x55ef45679050 tx=0x55ef4552c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e980ee8c7963d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa3679d3640 1 -- 172.21.15.53:0/496956177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef447ac820 con 0x55ef45663c00 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa3679d3640 1 -- 172.21.15.53:0/496956177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ef447cee00 con 0x55ef45663c00 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.781+0000 7fa3679d3640 1 -- 172.21.15.53:0/496956177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef447ac680 con 0x55ef45663c00 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 -- 172.21.15.53:0/496956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663c00 msgr2=0x55ef4554fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x55ef45679050 tx=0x55ef4552c600 comp rx=0 tx=0).stop 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 -- 172.21.15.53:0/496956177 shutdown_connections 2024-05-10T10:43:10.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 --2- 172.21.15.53:0/496956177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 -- 172.21.15.53:0/496956177 >> 172.21.15.53:0/496956177 conn(0x55ef45663000 msgr2=0x55ef45505000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 -- 172.21.15.53:0/496956177 shutdown_connections 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 -- 172.21.15.53:0/496956177 wait complete. 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 Processor -- start 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.782+0000 7fa36e322740 1 -- start start 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36e322740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36e322740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36e322740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36e322740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef4488bb00 con 0x55ef45663800 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36e322740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef4488b980 con 0x55ef45663c00 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36e322740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55ef4488a600 con 0x55ef45663400 2024-05-10T10:43:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50400/0 (socket says 172.21.15.53:50400) 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 learned_addr learned my addr 172.21.15.53:0/1418246422 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36b9db640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663400 msgr2=0x55ef4554f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef45663400 0x55ef4554f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663c00 msgr2=0x55ef4554fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef45663c00 0x55ef4554fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef447b3680 con 0x55ef45663800 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663800 0x55ef4554f600 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x55ef45679ef0 tx=0x55ef4552ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.783+0000 7fa3661d0640 1 -- 172.21.15.53:0/1418246422 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef447ac9c0 con 0x55ef45663800 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.784+0000 7fa3661d0640 1 -- 172.21.15.53:0/1418246422 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ef447cec40 con 0x55ef45663800 2024-05-10T10:43:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.784+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ef447b34a0 con 0x55ef45663800 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.784+0000 7fa3661d0640 1 -- 172.21.15.53:0/1418246422 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef447ac820 con 0x55ef45663800 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.784+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef447b3a40 con 0x55ef45663800 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.784+0000 7fa36e322740 4 datascan.init: resolving metadata pool 17 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa3661d0640 1 -- 172.21.15.53:0/1418246422 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55ef45580c00 con 0x55ef45663800 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa3661d0640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ef4557f000 0x55ef45550100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa3661d0640 1 -- 172.21.15.53:0/1418246422 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x55ef45584600 con 0x55ef45663800 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa36e322740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa36e322740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa36e322740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ef4557f400 0x55ef45550680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557f800 con 0x55ef4557f400 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.785+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ef4557f000 0x55ef45550100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.786+0000 7fa36b9db640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ef4557f400 0x55ef45550680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.786+0000 7fa36b9db640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ef4557f400 0x55ef45550680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.786+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ef4557f000 0x55ef45550100 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55ef4552f170 tx=0x55ef44b2e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.786+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(1 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.786+0000 7fa3671d2640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ef45662800 0x55ef44b40000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.786+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557ec00 con 0x55ef45662800 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.787+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ef45662800 0x55ef44b40000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.787+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ef45662800 0x55ef44b40000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.787+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(2 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.787+0000 7fa3679d3640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ef4557e400 0x55ef44b40580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.787+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557e000 con 0x55ef4557e400 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.788+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ef4557e400 0x55ef44b40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.788+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ef4557e400 0x55ef44b40580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.788+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(3 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef4557e400 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.788+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557e800 con 0x55ef4557f400 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.789+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(4 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.789+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557f800 con 0x55ef4557f400 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.789+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 3 ==== osd_op_reply(5 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.789+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9ec00 con 0x55ef45662800 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.789+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== osd_op_reply(6 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.789+0000 7fa3679d3640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef44a9f800 0x55ef44b40b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.789+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9fc00 con 0x55ef44a9f800 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.790+0000 7fa36b9db640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef44a9f800 0x55ef44b40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.790+0000 7fa36b9db640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef44a9f800 0x55ef44b40b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.790+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(7 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.790+0000 7fa3671d2640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ef44a9f000 0x55ef44b41600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.790+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9f400 con 0x55ef44a9f000 2024-05-10T10:43:10.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.790+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ef44a9f000 0x55ef44b41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.791+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ef44a9f000 0x55ef44b41600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.791+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== osd_op_reply(8 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.791+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9e800 con 0x55ef4557f400 2024-05-10T10:43:10.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.791+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 4 ==== osd_op_reply(9 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.791+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58000 con 0x55ef4557f400 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.791+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 5 ==== osd_op_reply(10 [pgnls start_epoch 120 out=49b] v118'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.791+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9ec00 con 0x55ef4557e400 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.792+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(11 [pgnls start_epoch 120 out=49b] v118'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef4557e400 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.792+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58400 con 0x55ef4557f400 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.792+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 6 ==== osd_op_reply(12 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.792+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9f400 con 0x55ef45662800 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.792+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 3 ==== osd_op_reply(13 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.792+0000 7fa3671d2640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ef44b58800 0x55ef44b41b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.792+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58c00 con 0x55ef44b58800 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.793+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ef44b58800 0x55ef44b41b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.793+0000 7fa36c1dc640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ef44b58800 0x55ef44b41b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.793+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== osd_op_reply(14 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef44b58800 2024-05-10T10:43:10.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.793+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557ec00 con 0x55ef44a9f800 2024-05-10T10:43:10.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.793+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== osd_op_reply(15 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.793+0000 7fa3671d2640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ef4557e800 0x55ef44b42100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.793+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557f800 con 0x55ef4557e800 2024-05-10T10:43:10.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.794+0000 7fa36b9db640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ef4557e800 0x55ef44b42100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.794+0000 7fa36b9db640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ef4557e800 0x55ef44b42100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.794+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(16 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.794+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58000 con 0x55ef4557e800 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.795+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== osd_op_reply(17 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.795+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557ec00 con 0x55ef44a9f000 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.795+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== osd_op_reply(18 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.795+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58400 con 0x55ef44b58800 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.795+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== osd_op_reply(19 [pgnls start_epoch 120 out=81b] v119'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ef45518780 con 0x55ef44b58800 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.795+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557fc00 con 0x55ef44b58800 2024-05-10T10:43:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.796+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.2 v2:172.21.15.53:6800/2235813461 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ef45518780 con 0x55ef44b58800 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.796+0000 7fa36e322740 10 datascan.operator(): handling object 1099511627778.0 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.796+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58c00 con 0x55ef44b58800 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.797+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.2 v2:172.21.15.53:6800/2235813461 4 ==== osd_op_reply(21 10000000002.00000000 [call] v120'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ef45518780 con 0x55ef44b58800 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.797+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58400 con 0x55ef44a9f000 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.797+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 3 ==== osd_op_reply(22 [pgnls start_epoch 120 out=49b] v118'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.797+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59000 con 0x55ef44a9f000 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.798+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 4 ==== osd_op_reply(23 [pgnls start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.798+0000 7fa3671d2640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ef44a9fc00 0x55ef44b42680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.798+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9e800 con 0x55ef44a9fc00 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.798+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ef44a9fc00 0x55ef44b42680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.798+0000 7fa36c9dd640 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ef44a9fc00 0x55ef44b42680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.799+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(24 [pgnls start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.799+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59400 con 0x55ef44a9f000 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.799+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 5 ==== osd_op_reply(25 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.799+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9f400 con 0x55ef4557f400 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.799+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 7 ==== osd_op_reply(26 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.799+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59800 con 0x55ef4557e800 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.799+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 3 ==== osd_op_reply(27 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.800+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9e800 con 0x55ef44b58800 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.800+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.2 v2:172.21.15.53:6800/2235813461 5 ==== osd_op_reply(28 [pgnls start_epoch 120 out=49b] v118'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef44b58800 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.800+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a000 con 0x55ef44a9f000 2024-05-10T10:43:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.800+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 6 ==== osd_op_reply(29 [pgnls start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.800+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58400 con 0x55ef44a9f800 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.800+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 3 ==== osd_op_reply(30 [pgnls start_epoch 120 out=81b] v118'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.800+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59c00 con 0x55ef44a9f800 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.801+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.801+0000 7fa36e322740 10 datascan.operator(): handling object 1099511628279.0 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.801+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9ec00 con 0x55ef44a9f800 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.802+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v120'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.802+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557e000 con 0x55ef4557f400 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.802+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 8 ==== osd_op_reply(33 [pgnls start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.802+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59000 con 0x55ef44a9f000 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.802+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 7 ==== osd_op_reply(34 [pgnls start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.802+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a400 con 0x55ef45662800 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.803+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 4 ==== osd_op_reply(35 [pgnls start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.803+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557ec00 con 0x55ef44a9f000 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.803+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 8 ==== osd_op_reply(36 [pgnls start_epoch 120 out=49b] v118'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.803+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a800 con 0x55ef44a9f000 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.803+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 9 ==== osd_op_reply(37 [pgnls start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.803+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9f400 con 0x55ef45662800 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.804+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 5 ==== osd_op_reply(38 [pgnls start_epoch 120 out=49b] v118'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.804+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6ac00 con 0x55ef44a9f800 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.804+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 6 ==== osd_op_reply(39 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.804+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59400 con 0x55ef44a9fc00 2024-05-10T10:43:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.804+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== osd_op_reply(40 [pgnls start_epoch 120 out=81b] v118'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.804+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55ef45567400 con 0x55ef44a9fc00 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.805+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.805+0000 7fa36e322740 10 datascan.operator(): handling object 1099511627780.1 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.805+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a000 con 0x55ef44a9f800 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.806+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 7 ==== osd_op_reply(42 10000000004.00000000 [call] v120'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.806+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a400 con 0x55ef44a9f000 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.806+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 10 ==== osd_op_reply(43 [pgnls start_epoch 120 out=49b] v118'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.806+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59000 con 0x55ef4557e800 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.807+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 4 ==== osd_op_reply(44 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.807+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6b000 con 0x55ef44a9fc00 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.807+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 4 ==== osd_op_reply(45 [pgnls start_epoch 120 out=81b] v119'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.807+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a800 con 0x55ef44a9fc00 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.807+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.807+0000 7fa36e322740 10 datascan.operator(): handling object 1099511628280.0 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.807+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557ec00 con 0x55ef44a9fc00 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.809+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v120'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.809+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557f800 con 0x55ef4557f400 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.809+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 9 ==== osd_op_reply(48 [pgnls start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.809+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6b400 con 0x55ef44a9f800 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.809+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 8 ==== osd_op_reply(49 [pgnls start_epoch 120 out=81b] v120'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.809+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557e000 con 0x55ef44a9f800 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.810+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.810+0000 7fa36e322740 10 datascan.operator(): handling object 1099511627780.0 2024-05-10T10:43:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.810+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9ec00 con 0x55ef44a9f800 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.811+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 10 ==== osd_op_reply(51 10000000004.00000000 [call] v120'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.811+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a400 con 0x55ef4557e400 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.811+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(52 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef4557e400 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.811+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6bc00 con 0x55ef45662800 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.811+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 6 ==== osd_op_reply(53 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.812+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58000 con 0x55ef44a9fc00 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.812+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 7 ==== osd_op_reply(54 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.812+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b76000 con 0x55ef44a9fc00 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.812+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 8 ==== osd_op_reply(55 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.813+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59800 con 0x55ef4557e400 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.813+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.6 v2:172.21.15.135:6808/1123855098 4 ==== osd_op_reply(56 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef4557e400 2024-05-10T10:43:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.813+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b76400 con 0x55ef4557f400 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.813+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.0 v2:172.21.15.53:6802/1634417257 10 ==== osd_op_reply(57 [pgnls start_epoch 120 out=49b] v118'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557f400 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.813+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58400 con 0x55ef44a9f000 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.3 v2:172.21.15.53:6816/2871564334 11 ==== osd_op_reply(58 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9f000 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b76800 con 0x55ef44a9f800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 11 ==== osd_op_reply(59 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59c00 con 0x55ef44b58800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.2 v2:172.21.15.53:6800/2235813461 6 ==== osd_op_reply(60 [pgnls start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef44b58800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b76c00 con 0x55ef4557e800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 5 ==== osd_op_reply(61 [pgnls start_epoch 120 out=49b] v118'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.814+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a400 con 0x55ef44a9f800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.815+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 12 ==== osd_op_reply(62 [pgnls start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.815+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b77000 con 0x55ef45662800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.815+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 7 ==== osd_op_reply(63 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.815+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6ac00 con 0x55ef4557e800 2024-05-10T10:43:10.828 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.815+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 6 ==== osd_op_reply(64 [pgnls start_epoch 120 out=49b] v118'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.815+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b77400 con 0x55ef44a9f800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.816+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 13 ==== osd_op_reply(65 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.816+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6a000 con 0x55ef44a9fc00 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.816+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 9 ==== osd_op_reply(66 [pgnls start_epoch 120 out=49b] v118'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.816+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b77800 con 0x55ef44b58800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.816+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 <== osd.2 v2:172.21.15.53:6800/2235813461 7 ==== osd_op_reply(67 [pgnls start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518780 con 0x55ef44b58800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.816+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b6b400 con 0x55ef45662800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.817+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 8 ==== osd_op_reply(68 [pgnls start_epoch 120 out=49b] v118'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.817+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b77c00 con 0x55ef45662800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.817+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 9 ==== osd_op_reply(69 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.817+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b58c00 con 0x55ef4557e800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.817+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 7 ==== osd_op_reply(70 [pgnls start_epoch 120 out=49b] v118'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.817+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44a9e800 con 0x55ef45662800 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.818+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.5 v2:172.21.15.135:6824/3244007917 10 ==== osd_op_reply(71 [pgnls start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef45662800 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.818+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59800 con 0x55ef44a9f800 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.818+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.1 v2:172.21.15.53:6824/2705510088 14 ==== osd_op_reply(72 [pgnls start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef44a9f800 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.818+0000 7fa3679d3640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef4557fc00 con 0x55ef4557e800 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36b9db640 1 -- 172.21.15.53:0/1418246422 <== osd.7 v2:172.21.15.135:6809/751247261 8 ==== osd_op_reply(73 [pgnls start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518000 con 0x55ef4557e800 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa3671d2640 1 -- 172.21.15.53:0/1418246422 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55ef44b59c00 con 0x55ef44a9fc00 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36c9dd640 1 -- 172.21.15.53:0/1418246422 <== osd.4 v2:172.21.15.135:6800/3517077655 10 ==== osd_op_reply(74 [pgnls start_epoch 120 out=49b] v118'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ef45518500 con 0x55ef44a9fc00 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ef4557f400 msgr2=0x55ef45550680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55ef4557f400 0x55ef45550680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef44a9f800 msgr2=0x55ef44b40b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55ef44a9f800 0x55ef44b40b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ef44b58800 msgr2=0x55ef44b41b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x55ef44b58800 0x55ef44b41b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ef44a9f000 msgr2=0x55ef44b41600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x55ef44a9f000 0x55ef44b41600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 reap_dead start 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ef44a9fc00 msgr2=0x55ef44b42680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55ef44a9fc00 0x55ef44b42680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ef45662800 msgr2=0x55ef44b40000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55ef45662800 0x55ef44b40000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ef4557e400 msgr2=0x55ef44b40580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55ef4557e400 0x55ef44b40580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ef4557e800 msgr2=0x55ef44b42100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55ef4557e800 0x55ef44b42100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36c1dc640 1 -- 172.21.15.53:0/1418246422 reap_dead start 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ef4557f000 msgr2=0x55ef45550100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ef4557f000 0x55ef45550100 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55ef4552f170 tx=0x55ef44b2e000 comp rx=0 tx=0).stop 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663800 msgr2=0x55ef4554f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663800 0x55ef4554f600 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x55ef45679ef0 tx=0x55ef4552ca20 comp rx=0 tx=0).stop 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 shutdown_connections 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55ef4557f000 0x55ef45550100 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 --2- 172.21.15.53:0/1418246422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef45663800 0x55ef4554f600 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 >> 172.21.15.53:0/1418246422 conn(0x55ef45663000 msgr2=0x55ef45505000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 shutdown_connections 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.819+0000 7fa36e322740 1 -- 172.21.15.53:0/1418246422 wait complete. 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 -- 172.21.15.53:0/2033563701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef44787800 msgr2=0x55ef4554e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef44787800 0x55ef4554e580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x55ef4559fbc0 tx=0x55ef4552cae0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 -- 172.21.15.53:0/2033563701 shutdown_connections 2024-05-10T10:43:10.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55ef44787c00 0x55ef4554eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef44787800 0x55ef4554e580 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 --2- 172.21.15.53:0/2033563701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef44786c00 0x55ef4554e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 -- 172.21.15.53:0/2033563701 >> 172.21.15.53:0/2033563701 conn(0x55ef44787000 msgr2=0x55ef45504800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 -- 172.21.15.53:0/2033563701 shutdown_connections 2024-05-10T10:43:10.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.820+0000 7fa36e322740 1 -- 172.21.15.53:0/2033563701 wait complete. 2024-05-10T10:43:10.833 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.131284, result: 2024-05-10T10:43:10.833 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-05-10T10:43:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.908+0000 7fcfe280d740 1 -- start start 2024-05-10T10:43:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.908+0000 7fcfe280d740 1 Processor -- start 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55800f58ec00 0x558010362000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55800f58f800 0x558010362580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800f58fc00 0x558010362b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe280d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558010380180 con 0x55800f58fc00 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe280d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558010380300 con 0x55800f58ec00 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe280d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558010380480 con 0x55800f58f800 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800f58fc00 0x558010362b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800f58fc00 0x558010362b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50418/0 (socket says 172.21.15.53:50418) 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/1554298603 learned_addr learned my addr 172.21.15.53:0/1554298603 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55800f58f800 0x558010362580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0529640 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55800f58ec00 0x558010362000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55800f58f800 msgr2=0x558010362580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55800f58f800 0x558010362580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55800f58ec00 msgr2=0x558010362000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55800f58ec00 0x558010362000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/1554298603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800f5bb680 con 0x55800f58fc00 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55800f58f800 0x558010362580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.909+0000 7fcfe0529640 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55800f58ec00 0x558010362000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800f58fc00 0x558010362b00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x5580103b1b90 tx=0x558010335f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4746d4bc5957020d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfddd24640 1 -- 172.21.15.53:0/1554298603 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55800f5b49c0 con 0x55800f58fc00 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfddd24640 1 -- 172.21.15.53:0/1554298603 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55800f5d61c0 con 0x55800f58fc00 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfddd24640 1 -- 172.21.15.53:0/1554298603 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55800f5b4680 con 0x55800f58fc00 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe280d740 1 -- 172.21.15.53:0/1554298603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800f5bb4a0 con 0x55800f58fc00 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe280d740 1 -- 172.21.15.53:0/1554298603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55800f5bba40 con 0x55800f58fc00 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe280d740 4 waiting for MDS map... 2024-05-10T10:43:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfddd24640 1 -- 172.21.15.53:0/1554298603 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x558010464200 con 0x55800f58fc00 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfddd24640 1 -- 172.21.15.53:0/1554298603 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x55800f5e7180 con 0x55800f58fc00 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe280d740 4 Got MDS map 105 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe280d740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe280d740 4 datascan.main: Using metadata pool output 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.910+0000 7fcfe280d740 4 datascan.main: connecting to RADOS... 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 Processor -- start 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 -- start start 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 0x558010363b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558010380600 con 0x5580104c8400 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558010380780 con 0x5580104c8000 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.911+0000 7fcfe280d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558010380900 con 0x5580104c8800 2024-05-10T10:43:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50432/0 (socket says 172.21.15.53:50432) 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 -- 172.21.15.53:0/372530662 learned_addr learned my addr 172.21.15.53:0/372530662 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 -- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 msgr2=0x558010363b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 0x558010363b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 -- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 msgr2=0x558010363080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 -- 172.21.15.53:0/372530662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800f5bb860 con 0x5580104c8400 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe0529640 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x558010399170 tx=0x558010476cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a6d3c7d698e6a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfdbd20640 1 -- 172.21.15.53:0/372530662 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55800f5b49c0 con 0x5580104c8400 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfdbd20640 1 -- 172.21.15.53:0/372530662 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55800f5d6c40 con 0x5580104c8400 2024-05-10T10:43:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfdbd20640 1 -- 172.21.15.53:0/372530662 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55800f5b4680 con 0x5580104c8400 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe280d740 1 -- 172.21.15.53:0/372530662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 msgr2=0x558010363600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.912+0000 7fcfe280d740 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x558010399170 tx=0x558010476cc0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 -- 172.21.15.53:0/372530662 shutdown_connections 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 0x558010363b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 --2- 172.21.15.53:0/372530662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 -- 172.21.15.53:0/372530662 >> 172.21.15.53:0/372530662 conn(0x55801037dc00 msgr2=0x55801030d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 -- 172.21.15.53:0/372530662 shutdown_connections 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 -- 172.21.15.53:0/372530662 wait complete. 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 Processor -- start 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.913+0000 7fcfe280d740 1 -- start start 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 0x558010363b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558010381c80 con 0x5580104c8400 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558010381b00 con 0x5580104c8000 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x558010381980 con 0x5580104c8800 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50444/0 (socket says 172.21.15.53:50444) 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 learned_addr learned my addr 172.21.15.53:0/2383733502 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 0x558010363b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 msgr2=0x558010363b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 0x558010363b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 msgr2=0x558010363080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580104c8000 0x558010363080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800f5bb2c0 con 0x5580104c8400 2024-05-10T10:43:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5580104c8800 0x558010363b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe0529640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x558010399e30 tx=0x558010476c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfda51d640 1 -- 172.21.15.53:0/2383733502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55800f5b44e0 con 0x5580104c8400 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfda51d640 1 -- 172.21.15.53:0/2383733502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55800f5d6a80 con 0x5580104c8400 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55800f5bb0e0 con 0x5580104c8400 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfda51d640 1 -- 172.21.15.53:0/2383733502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55800f5b49c0 con 0x5580104c8400 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800f5ba3c0 con 0x5580104c8400 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.914+0000 7fcfe280d740 4 datascan.init: resolving metadata pool 17 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfda51d640 1 -- 172.21.15.53:0/2383733502 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x5580104d4c00 con 0x5580104c8400 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfda51d640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55800f92c000 0x558010364100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfda51d640 1 -- 172.21.15.53:0/2383733502 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x5580104cea00 con 0x5580104c8400 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfe280d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfe280d740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfe280d740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-10T10:43:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x5580104c8c00 0x558010364680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x5580104c9000 con 0x5580104c8c00 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55800f92c000 0x558010364100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.916+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x5580104c8c00 0x558010364680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.917+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x5580104c8c00 0x558010364680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.917+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55800f92c000 0x558010364100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5580103371a0 tx=0x55800f8a8060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.917+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.917+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55800f92d000 0x558010364c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.917+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55800f92d400 con 0x55800f92d000 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.917+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55800f92d000 0x558010364c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.918+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55800f92d000 0x558010364c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.918+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x558010320500 con 0x55800f92d000 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.918+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55800f92d800 0x558010365180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.918+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92dc00 con 0x55800f92d800 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.918+0000 7fcfe0529640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55800f92d800 0x558010365180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.918+0000 7fcfe0529640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55800f92d800 0x558010365180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.919+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.919+0000 7fcfdb51f640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55800f956000 0x55800f95c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.919+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956400 con 0x55800f956000 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.919+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55800f956000 0x55800f95c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.920+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55800f956000 0x55800f95c000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.920+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.920+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956c00 con 0x55800f92d000 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.920+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f92d000 2024-05-10T10:43:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.920+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956800 con 0x55800f92d800 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.920+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.920+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957000 con 0x55800f92d800 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.921+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.921+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957800 con 0x55800f956000 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.921+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.921+0000 7fcfdbd20640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55800f957400 0x55800f95c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.921+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956400 con 0x55800f957400 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.921+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55800f957400 0x55800f95c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.922+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55800f957400 0x55800f95c580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.922+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.922+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957c00 con 0x5580104c8c00 2024-05-10T10:43:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.922+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.922+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x5580104c9000 con 0x55800f92d800 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.922+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.922+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956800 con 0x55800f92d800 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.923+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 120 out=49b] v118'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.923+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92cc00 con 0x55800f92d000 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.923+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 120 out=49b] v118'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f92d000 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.923+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957000 con 0x55800f92d800 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.923+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.923+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957800 con 0x55800f956000 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.923+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfdb51f640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580104c9000 0x55800f95cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92dc00 con 0x5580104c9000 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfe0529640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580104c9000 0x55800f95cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfe0529640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580104c9000 0x55800f95cb00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x5580104c9000 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e000 con 0x55800f957400 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfdb51f640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55800f96e000 0x55800f95d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.924+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92d400 con 0x55800f96e000 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.925+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55800f96e000 0x55800f95d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.925+0000 7fcfdfd28640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55800f96e000 0x55800f95d080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.925+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.925+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e400 con 0x55800f96e000 2024-05-10T10:43:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.926+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.926+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92cc00 con 0x5580104c8c00 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.926+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.926+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 120 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e800 con 0x5580104c9000 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.926+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 120 out=81b] v120'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x558010320780 con 0x5580104c9000 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.926+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e400 con 0x55800f92d800 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.927+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.927+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96ec00 con 0x55800f956000 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.927+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.927+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956c00 con 0x55800f92d000 2024-05-10T10:43:10.959 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.927+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.6 v2:172.21.15.135:6808/1123855098 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f92d000 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.927+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96f000 con 0x55800f92d800 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.927+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.928+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956400 con 0x55800f92d800 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.928+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.928+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e800 con 0x55800f956000 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.928+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.928+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972000 con 0x55800f957400 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.928+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.928+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e400 con 0x5580104c8c00 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972400 con 0x55800f92d800 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96fc00 con 0x55800f92d800 2024-05-10T10:43:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 120 out=49b] v118'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972800 con 0x55800f92d000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.6 v2:172.21.15.135:6808/1123855098 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 120 out=49b] v118'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f92d000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.929+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92dc00 con 0x55800f92d800 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972c00 con 0x55800f956000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956800 con 0x5580104c9000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.2 v2:172.21.15.53:6800/2235813461 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x5580104c9000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973000 con 0x55800f957400 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.930+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957000 con 0x55800f96e000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.931+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.931+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973400 con 0x55800f96e000 2024-05-10T10:43:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.931+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.931+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957000 con 0x5580104c8c00 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.931+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.931+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972c00 con 0x5580104c9000 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.2 v2:172.21.15.53:6800/2235813461 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 120 out=49b] v120'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x5580104c9000 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92dc00 con 0x5580104c8c00 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 120 out=49b] v118'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973400 con 0x5580104c8c00 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfdbd20640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55800f96ec00 0x55800f95d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e800 con 0x55800f96ec00 2024-05-10T10:43:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.932+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55800f96ec00 0x55800f95d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.933+0000 7fcfe0d2a640 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55800f96ec00 0x55800f95d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.933+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f96ec00 2024-05-10T10:43:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.933+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972400 con 0x5580104c8c00 2024-05-10T10:43:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.933+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.933+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e400 con 0x55800f92d800 2024-05-10T10:43:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973800 con 0x55800f96e000 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96f000 con 0x5580104c9000 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.2 v2:172.21.15.53:6800/2235813461 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 120 out=49b] v118'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x5580104c9000 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973c00 con 0x5580104c8c00 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.934+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96fc00 con 0x55800f957400 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 120 out=81b] v120'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 10 datascan.operator(): handling object 100000001f7.0 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55800f96f800 con 0x55800f957400 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-05-10T10:43:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55800f92d400 con 0x55800f92d800 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.935+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55800f92cc00 con 0x5580104c8c00 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 20 datascan.get_frag_of: frag is * 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55800f957c00 con 0x55800f92d800 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-05-10T10:43:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55800f957800 con 0x55800f92d800 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 20 datascan.read_dentry: bravo_head not found in result 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.936+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55800f973400 con 0x55800f92d800 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.937+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v120'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.937+0000 7fcfe280d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.937+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972400 con 0x55800f92d800 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.938+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.938+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972c00 con 0x5580104c8c00 2024-05-10T10:43:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.938+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.938+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956400 con 0x55800f956000 2024-05-10T10:43:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.938+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.938+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973400 con 0x5580104c8c00 2024-05-10T10:43:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.938+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 120 out=49b] v118'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.939+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956800 con 0x5580104c8c00 2024-05-10T10:43:10.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.939+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.939+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972000 con 0x55800f956000 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.939+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 120 out=49b] v118'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.939+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e400 con 0x55800f957400 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.939+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.939+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972800 con 0x55800f96ec00 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.940+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 120 out=49b] v118'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f96ec00 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.940+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96f000 con 0x5580104c8c00 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.940+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 120 out=49b] v118'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.940+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973000 con 0x55800f96e000 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.940+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.940+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e800 con 0x55800f96ec00 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.941+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.4 v2:172.21.15.135:6800/3517077655 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 120 out=49b] v120'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f96ec00 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.941+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957c00 con 0x55800f92d800 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.941+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.941+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96fc00 con 0x55800f957400 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.941+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 120 out=49b] v120'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.941+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957800 con 0x55800f92d000 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.942+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.6 v2:172.21.15.135:6808/1123855098 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f92d000 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.942+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957800 con 0x55800f956000 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.942+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.942+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f92d400 con 0x55800f96ec00 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.942+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.4 v2:172.21.15.135:6800/3517077655 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f96ec00 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.942+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96f800 con 0x55800f96ec00 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.943+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.4 v2:172.21.15.135:6800/3517077655 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f96ec00 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.943+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956400 con 0x55800f92d000 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.943+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.6 v2:172.21.15.135:6808/1123855098 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f92d000 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.943+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96f400 con 0x55800f92d800 2024-05-10T10:43:10.969 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.943+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.0 v2:172.21.15.53:6802/1634417257 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 120 out=49b] v118'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x55800f92d800 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.943+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f956800 con 0x5580104c8c00 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.3 v2:172.21.15.53:6816/2871564334 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x5580104c8c00 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e400 con 0x55800f957400 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f957800 con 0x5580104c9000 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.2 v2:172.21.15.53:6800/2235813461 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 120 out=49b] v118'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x5580104c9000 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96e800 con 0x55800f96e000 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 120 out=49b] v118'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.944+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972c00 con 0x55800f957400 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.945+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 120 out=49b] v118'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.945+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96fc00 con 0x55800f956000 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.945+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.945+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973400 con 0x55800f96e000 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.945+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 120 out=49b] v118'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.945+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972800 con 0x55800f957400 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.946+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.946+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f972000 con 0x55800f96ec00 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.946+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.4 v2:172.21.15.135:6800/3517077655 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 120 out=49b] v118'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f96ec00 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.946+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f984000 con 0x5580104c9000 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.947+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 <== osd.2 v2:172.21.15.53:6800/2235813461 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 120 out=49b] v118'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320780 con 0x5580104c9000 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.947+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f973000 con 0x55800f956000 2024-05-10T10:43:10.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.947+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 120 out=49b] v118'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.947+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f984400 con 0x55800f956000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.947+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.947+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f96f000 con 0x55800f96e000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.947+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 120 out=49b] v118'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.948+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f984800 con 0x55800f956000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.948+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.5 v2:172.21.15.135:6824/3244007917 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 120 out=49b] v118'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f956000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.948+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f988000 con 0x55800f957400 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.948+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.1 v2:172.21.15.53:6824/2705510088 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 120 out=49b] v118'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f957400 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.948+0000 7fcfdb51f640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f984c00 con 0x55800f96e000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.948+0000 7fcfdfd28640 1 -- 172.21.15.53:0/2383733502 <== osd.7 v2:172.21.15.135:6809/751247261 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 120 out=49b] v118'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320000 con 0x55800f96e000 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.948+0000 7fcfdbd20640 1 -- 172.21.15.53:0/2383733502 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 120 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x55800f988400 con 0x55800f96ec00 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe0d2a640 1 -- 172.21.15.53:0/2383733502 <== osd.4 v2:172.21.15.135:6800/3517077655 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 120 out=49b] v118'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558010320500 con 0x55800f96ec00 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55800f92d800 msgr2=0x558010365180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.972 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x55800f92d800 0x558010365180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55800f957400 msgr2=0x55800f95c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x55800f957400 0x55800f95c580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580104c9000 msgr2=0x55800f95cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x5580104c9000 0x55800f95cb00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x5580104c8c00 msgr2=0x558010364680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x5580104c8c00 0x558010364680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55800f96ec00 msgr2=0x55800f95d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x55800f96ec00 0x55800f95d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 reap_dead start 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55800f956000 msgr2=0x55800f95c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x55800f956000 0x55800f95c000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55800f92d000 msgr2=0x558010364c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x55800f92d000 0x558010364c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55800f96e000 msgr2=0x55800f95d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55800f96e000 0x55800f95d080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe0529640 1 -- 172.21.15.53:0/2383733502 reap_dead start 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55800f92c000 msgr2=0x558010364100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55800f92c000 0x558010364100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5580103371a0 tx=0x55800f8a8060 comp rx=0 tx=0).stop 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 msgr2=0x558010363600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x558010399e30 tx=0x558010476c00 comp rx=0 tx=0).stop 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 shutdown_connections 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x55800f96e000 0x55800f95d080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.974 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x55800f92c000 0x558010364100 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/2383733502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580104c8400 0x558010363600 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 >> 172.21.15.53:0/2383733502 conn(0x55801037dc00 msgr2=0x55801030d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 shutdown_connections 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/2383733502 wait complete. 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/1554298603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800f58fc00 msgr2=0x558010362b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800f58fc00 0x558010362b00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x5580103b1b90 tx=0x558010335f20 comp rx=0 tx=0).stop 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.949+0000 7fcfe280d740 1 -- 172.21.15.53:0/1554298603 shutdown_connections 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.950+0000 7fcfe280d740 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55800f58fc00 0x558010362b00 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.950+0000 7fcfe280d740 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55800f58f800 0x558010362580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.950+0000 7fcfe280d740 1 --2- 172.21.15.53:0/1554298603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55800f58ec00 0x558010362000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.950+0000 7fcfe280d740 1 -- 172.21.15.53:0/1554298603 >> 172.21.15.53:0/1554298603 conn(0x55800f58f000 msgr2=0x55801030c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.950+0000 7fcfe280d740 1 -- 172.21.15.53:0/1554298603 shutdown_connections 2024-05-10T10:43:10.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:10.950+0000 7fcfe280d740 1 -- 172.21.15.53:0/1554298603 wait complete. 2024-05-10T10:43:10.976 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.142569, result: 2024-05-10T10:43:10.976 DEBUG:teuthology.orchestra.run.smithi053:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-10T10:43:11.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.035+0000 7fea307dc740 1 -- start start 2024-05-10T10:43:11.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.035+0000 7fea307dc740 1 Processor -- start 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea307dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc800 0x556dae2b5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea307dc740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dad4e9c00 0x556dae2b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea307dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae2bdc00 0x556dae2b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea307dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556dae2d4000 con 0x556dae2bdc00 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea307dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556dae2d4180 con 0x556dae2bc800 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea307dc740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x556dae2d4300 con 0x556dad4e9c00 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae2bdc00 0x556dae2b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae2bdc00 0x556dae2b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50458/0 (socket says 172.21.15.53:50458) 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 -- 172.21.15.53:0/279651818 learned_addr learned my addr 172.21.15.53:0/279651818 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc800 0x556dae2b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2e789640 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dad4e9c00 0x556dae2b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 -- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dad4e9c00 msgr2=0x556dae2b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dad4e9c00 0x556dae2b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 -- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc800 msgr2=0x556dae2b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc800 0x556dae2b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2f78b640 1 -- 172.21.15.53:0/279651818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dad515680 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.036+0000 7fea2e789640 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dad4e9c00 0x556dae2b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea2f78b640 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae2bdc00 0x556dae2b4580 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x556dae319bf0 tx=0x556dae28ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db495a01ec467c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea2c785640 1 -- 172.21.15.53:0/279651818 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556dad50eb60 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea2c785640 1 -- 172.21.15.53:0/279651818 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556dad530c40 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea2c785640 1 -- 172.21.15.53:0/279651818 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556dad50ed00 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea307dc740 1 -- 172.21.15.53:0/279651818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dad5143c0 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea307dc740 1 -- 172.21.15.53:0/279651818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556dad515a40 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea307dc740 4 waiting for MDS map... 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea2c785640 1 -- 172.21.15.53:0/279651818 <== mon.0 v2:172.21.15.53:3300/0 4 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x556dae304400 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea2c785640 1 -- 172.21.15.53:0/279651818 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x556dad541180 con 0x556dae2bdc00 2024-05-10T10:43:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea307dc740 4 Got MDS map 105 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea307dc740 4 datascan.main: Using metadata pool output 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.037+0000 7fea307dc740 4 datascan.main: connecting to RADOS... 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 Processor -- start 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 -- start start 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc000 0x556dae2b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dc400 0x556dae2b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dc800 0x556dae2b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556dae2d4480 con 0x556dae3dc400 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556dae2d4600 con 0x556dae2bc000 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea307dc740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x556dae2d4780 con 0x556dae3dc800 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea2ef8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dc400 0x556dae2b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea2ef8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dc400 0x556dae2b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50466/0 (socket says 172.21.15.53:50466) 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.038+0000 7fea2ef8a640 1 -- 172.21.15.53:0/2827263837 learned_addr learned my addr 172.21.15.53:0/2827263837 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2e789640 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dc800 0x556dae2b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2ef8a640 1 -- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dc800 msgr2=0x556dae2b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dc800 0x556dae2b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2ef8a640 1 -- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc000 msgr2=0x556dae2b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2e789640 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc000 0x556dae2b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc000 0x556dae2b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2ef8a640 1 -- 172.21.15.53:0/2827263837 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dad5152c0 con 0x556dae3dc400 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dc400 0x556dae2b5b80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x556dae3d5230 tx=0x556dae3dec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2607d06cfdeab0d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2a781640 1 -- 172.21.15.53:0/2827263837 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556dad50eb60 con 0x556dae3dc400 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2a781640 1 -- 172.21.15.53:0/2827263837 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556dad531340 con 0x556dae3dc400 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea2a781640 1 -- 172.21.15.53:0/2827263837 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556dad50ed00 con 0x556dae3dc400 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 -- 172.21.15.53:0/2827263837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dc400 msgr2=0x556dae2b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dc400 0x556dae2b5b80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x556dae3d5230 tx=0x556dae3dec00 comp rx=0 tx=0).stop 2024-05-10T10:43:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 -- 172.21.15.53:0/2827263837 shutdown_connections 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dc800 0x556dae2b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dc400 0x556dae2b5b80 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 --2- 172.21.15.53:0/2827263837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc000 0x556dae2b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 -- 172.21.15.53:0/2827263837 >> 172.21.15.53:0/2827263837 conn(0x556dae2bc400 msgr2=0x556dae267800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.039+0000 7fea307dc740 1 -- 172.21.15.53:0/2827263837 shutdown_connections 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.040+0000 7fea307dc740 1 -- 172.21.15.53:0/2827263837 wait complete. 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.040+0000 7fea307dc740 1 Processor -- start 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.040+0000 7fea307dc740 1 -- start start 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea307dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc400 0x556dae2b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea307dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dd400 0x556dae2b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea307dc740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dd000 0x556dae2b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea307dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556dae2d5e00 con 0x556dae3dd400 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea307dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556dae2d5c80 con 0x556dae2bc400 2024-05-10T10:43:11.048 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea307dc740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x556dae2d5b00 con 0x556dae3dd000 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dd400 0x556dae2b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dd400 0x556dae2b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50470/0 (socket says 172.21.15.53:50470) 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 learned_addr learned my addr 172.21.15.53:0/28733099 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc400 0x556dae2b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dd000 0x556dae2b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dd000 msgr2=0x556dae2b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dae3dd000 0x556dae2b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc400 msgr2=0x556dae2b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc400 0x556dae2b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dad515680 con 0x556dae3dd400 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc400 0x556dae2b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dd400 0x556dae2b5b80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x556dae3d5ef0 tx=0x556dae28e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.041+0000 7fea28f7e640 1 -- 172.21.15.53:0/28733099 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556dad50eb60 con 0x556dae3dd400 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.042+0000 7fea28f7e640 1 -- 172.21.15.53:0/28733099 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556dad531500 con 0x556dae3dd400 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.042+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556dad5154a0 con 0x556dae3dd400 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.042+0000 7fea28f7e640 1 -- 172.21.15.53:0/28733099 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556dad50ed00 con 0x556dae3dd400 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.042+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dad514b40 con 0x556dae3dd400 2024-05-10T10:43:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.042+0000 7fea307dc740 4 datascan.init: resolving metadata pool 17 2024-05-10T10:43:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea28f7e640 1 -- 172.21.15.53:0/28733099 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x556dae43ec00 con 0x556dae3dd400 2024-05-10T10:43:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea28f7e640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x556dae451000 0x556dae2b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea28f7e640 1 -- 172.21.15.53:0/28733099 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x556dad5ec000 con 0x556dae3dd400 2024-05-10T10:43:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea307dc740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-10T10:43:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea307dc740 4 datascan.main: resolving metadata pool 17 2024-05-10T10:43:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x556dae3dcc00 0x556dae2b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x556dae451000 0x556dae2b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.043+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dae3dc800 con 0x556dae3dcc00 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.044+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x556dae3dcc00 0x556dae2b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.044+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x556dae451000 0x556dae2b6680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x556dae3d5dd0 tx=0x556dae3dff80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.044+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x556dae3dcc00 0x556dae2b6c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.044+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== osd_op_reply(1 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.044+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x556dae3dc800 0x556dad8a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.044+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dae451c00 con 0x556dae3dc800 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.045+0000 7fea2f78b640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x556dae3dc800 0x556dad8a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.045+0000 7fea2f78b640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x556dae3dc800 0x556dad8a6000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.045+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== osd_op_reply(2 [pgnls start_epoch 120 out=177b] v119'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.045+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dae451c00 con 0x556dae3dcc00 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== osd_op_reply(3 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x556dae451c00 0x556dad8a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa000 con 0x556dae451c00 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x556dae451c00 0x556dad8a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x556dae451c00 0x556dad8a6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== osd_op_reply(4 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa000 con 0x556dae451c00 2024-05-10T10:43:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== osd_op_reply(5 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.046+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa000 con 0x556dae3dc800 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.047+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== osd_op_reply(6 [pgnls start_epoch 120 out=149b] v120'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.047+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa000 con 0x556dae3dcc00 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.047+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 3 ==== osd_op_reply(7 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.047+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x556dad8aa000 0x556dad8a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.047+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa400 con 0x556dad8aa000 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.047+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x556dad8aa000 0x556dad8a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.048+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x556dad8aa000 0x556dad8a6b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.048+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== osd_op_reply(8 [pgnls start_epoch 120 out=73b] v119'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.048+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x556dad8aa400 0x556dad8a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.048+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa800 con 0x556dad8aa400 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.048+0000 7fea2f78b640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x556dad8aa400 0x556dad8a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.048+0000 7fea2f78b640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x556dad8aa400 0x556dad8a7080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.049+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== osd_op_reply(9 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dad8aa400 2024-05-10T10:43:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.049+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa800 con 0x556dae3dc800 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.049+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 3 ==== osd_op_reply(10 [pgnls start_epoch 120 out=81b] v119'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.049+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa800 con 0x556dae3dcc00 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.049+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 4 ==== osd_op_reply(11 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.049+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aa800 con 0x556dae451c00 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 3 ==== osd_op_reply(12 [pgnls start_epoch 120 out=49b] v120'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x556dad8aa800 0x556dad8a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dae450800 con 0x556dad8aa800 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x556dad8aa800 0x556dad8a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x556dad8aa800 0x556dad8a8100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== osd_op_reply(13 [pgnls start_epoch 120 out=77b] v119'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x556dae450800 0x556dad8a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.050+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dae450c00 con 0x556dae450800 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.051+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x556dae450800 0x556dad8a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.051+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x556dae450800 0x556dad8a8680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.051+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== osd_op_reply(14 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.051+0000 7fea2e789640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x556dae450c00 0x556dad8a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.051+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae450c00 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.051+0000 7fea2f78b640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x556dae450c00 0x556dad8a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.052+0000 7fea2f78b640 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x556dae450c00 0x556dad8a7600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.052+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== osd_op_reply(15 [pgnls start_epoch 120 out=73b] v119'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a500 con 0x556dae450c00 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.052+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa000 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.052+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== osd_op_reply(16 [pgnls start_epoch 120 out=49b] v120'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.052+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.052+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 4 ==== osd_op_reply(17 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.052+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.053+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 4 ==== osd_op_reply(18 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.053+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.053+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 5 ==== osd_op_reply(19 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.053+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae450800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.053+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== osd_op_reply(20 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.053+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.054+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== osd_op_reply(21 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.054+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae450800 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.054+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 3 ==== osd_op_reply(22 [pgnls start_epoch 120 out=97b] v119'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.054+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dcc00 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.054+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 5 ==== osd_op_reply(23 [pgnls start_epoch 120 out=74b] v117'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.054+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dcc00 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.055+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 6 ==== osd_op_reply(24 [pgnls start_epoch 120 out=73b] v119'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.055+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa400 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.055+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== osd_op_reply(25 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dad8aa400 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.055+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.055+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 6 ==== osd_op_reply(26 [pgnls start_epoch 120 out=98b] v119'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.055+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.055+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 5 ==== osd_op_reply(27 [pgnls start_epoch 120 out=76b] v120'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.056+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa400 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.056+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.4 v2:172.21.15.135:6800/3517077655 3 ==== osd_op_reply(28 [pgnls start_epoch 120 out=49b] v120'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dad8aa400 2024-05-10T10:43:11.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.056+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.056+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 6 ==== osd_op_reply(29 [pgnls start_epoch 120 out=105b] v120'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.056+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa800 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.056+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 3 ==== osd_op_reply(30 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.056+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa000 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 3 ==== osd_op_reply(31 [pgnls start_epoch 120 out=73b] v120'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa800 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 4 ==== osd_op_reply(32 [pgnls start_epoch 120 out=101b] v120'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dae3dc400 con 0x556dae3dc800 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-10T10:43:11.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.057+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab000 con 0x556dae3dc800 2024-05-10T10:43:11.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-10T10:43:11.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab400 con 0x556dae3dc800 2024-05-10T10:43:11.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-10T10:43:11.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab800 con 0x556dae3dc800 2024-05-10T10:43:11.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.058+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8abc00 con 0x556dae3dc800 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c0000 con 0x556dae3dc800 2024-05-10T10:43:11.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c0400 con 0x556dad8aa000 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c0800 con 0x556dae3dc800 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.059+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c0c00 con 0x556dae450c00 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.060+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae450c00 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.060+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-10T10:43:11.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.060+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c1000 con 0x556dae450800 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c1400 con 0x556dae3dcc00 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c1800 con 0x556dae3dc800 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-10T10:43:11.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-10T10:43:11.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-10T10:43:11.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.061+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c1c00 con 0x556dae451c00 2024-05-10T10:43:11.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-05-10T10:43:11.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c0000 con 0x556dae451c00 2024-05-10T10:43:11.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 20 datascan.scan_links: injected ino detected 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c0800 con 0x556dad8aa800 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.062+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab000 con 0x556dae3dcc00 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.063+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 8 ==== osd_op_reply(48 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.063+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab000 con 0x556dae3dc800 2024-05-10T10:43:11.086 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.063+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 15 ==== osd_op_reply(49 [pgnls start_epoch 120 out=177b] v119'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.063+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab000 con 0x556dae3dcc00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.063+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 9 ==== osd_op_reply(50 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.063+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab000 con 0x556dae451c00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.064+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 9 ==== osd_op_reply(51 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.064+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.064+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 10 ==== osd_op_reply(52 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.064+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.064+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 16 ==== osd_op_reply(53 [pgnls start_epoch 120 out=149b] v120'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.064+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dcc00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 10 ==== osd_op_reply(54 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa000 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 5 ==== osd_op_reply(55 [pgnls start_epoch 120 out=73b] v119'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa400 2024-05-10T10:43:11.087 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.4 v2:172.21.15.135:6800/3517077655 4 ==== osd_op_reply(56 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dad8aa400 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 17 ==== osd_op_reply(57 [pgnls start_epoch 120 out=81b] v119'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.065+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dcc00 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.066+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 11 ==== osd_op_reply(58 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.066+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.066+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 11 ==== osd_op_reply(59 [pgnls start_epoch 120 out=49b] v120'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.066+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa800 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 6 ==== osd_op_reply(60 [pgnls start_epoch 120 out=77b] v119'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae450800 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 5 ==== osd_op_reply(61 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae450c00 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.2 v2:172.21.15.53:6800/2235813461 3 ==== osd_op_reply(62 [pgnls start_epoch 120 out=73b] v119'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a500 con 0x556dae450c00 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa000 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 6 ==== osd_op_reply(63 [pgnls start_epoch 120 out=49b] v120'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.067+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.068+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 12 ==== osd_op_reply(64 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.068+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.068+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 18 ==== osd_op_reply(65 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.068+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.068+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 19 ==== osd_op_reply(66 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.068+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae450800 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.069+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 6 ==== osd_op_reply(67 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.069+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa800 2024-05-10T10:43:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.069+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 7 ==== osd_op_reply(68 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.069+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae450800 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.069+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 7 ==== osd_op_reply(69 [pgnls start_epoch 120 out=97b] v119'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.069+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dcc00 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.070+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 12 ==== osd_op_reply(70 [pgnls start_epoch 120 out=74b] v117'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.070+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dcc00 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.070+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 13 ==== osd_op_reply(71 [pgnls start_epoch 120 out=73b] v119'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.070+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa400 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.071+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.4 v2:172.21.15.135:6800/3517077655 5 ==== osd_op_reply(72 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dad8aa400 2024-05-10T10:43:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.071+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae3dc800 2024-05-10T10:43:11.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.071+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 20 ==== osd_op_reply(73 [pgnls start_epoch 120 out=98b] v119'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.071+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.071+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 13 ==== osd_op_reply(74 [pgnls start_epoch 120 out=76b] v120'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.071+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa400 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.072+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.4 v2:172.21.15.135:6800/3517077655 6 ==== osd_op_reply(75 [pgnls start_epoch 120 out=49b] v120'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a500 con 0x556dad8aa400 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.072+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dae451c00 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.072+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 14 ==== osd_op_reply(76 [pgnls start_epoch 120 out=105b] v120'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.072+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa800 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.073+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 8 ==== osd_op_reply(77 [pgnls start_epoch 120 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.073+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa000 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.073+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 7 ==== osd_op_reply(78 [pgnls start_epoch 120 out=73b] v120'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.073+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 120 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e120) -- 0x556dad8aac00 con 0x556dad8aa800 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.073+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 9 ==== osd_op_reply(79 [pgnls start_epoch 120 out=101b] v120'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.073+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.073+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab400 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab800 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8abc00 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.074+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dae3dc400 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ce000 con 0x556dae3dc800 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ce400 con 0x556dae3dc800 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ce800 con 0x556dad8aa000 2024-05-10T10:43:11.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.1 v2:172.21.15.53:6824/2705510088 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a000 con 0x556dad8aa000 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.075+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8cec00 con 0x556dae3dc800 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8cf000 con 0x556dae450c00 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.2 v2:172.21.15.53:6800/2235813461 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a500 con 0x556dae450c00 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.076+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8cf400 con 0x556dae450800 2024-05-10T10:43:11.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.5 v2:172.21.15.135:6824/3244007917 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a000 con 0x556dae450800 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8cf800 con 0x556dae3dcc00 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8cfc00 con 0x556dae3dc800 2024-05-10T10:43:11.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.077+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ce800 con 0x556dae451c00 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8cf400 con 0x556dae451c00 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 20 datascan.scan_links: injected ino detected 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-10T10:43:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8cf800 con 0x556dad8aa800 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.6 v2:172.21.15.135:6808/1123855098 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x556dae27a780 con 0x556dad8aa800 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-05-10T10:43:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: processing 1 injected_inos 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.078+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dae3dc400 con 0x556dae451c00 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.079+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.079+0000 7fea307dc740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.079+0000 7fea307dc740 20 datascan.scan_links: first is now 2 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.079+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab400 con 0x556dae451c00 2024-05-10T10:43:11.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 <== osd.0 v2:172.21.15.53:6802/1634417257 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v120'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x556dae27a780 con 0x556dae451c00 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea307dc740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea307dc740 10 datascan.scan_links: updating inotable 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8ab800 con 0x556dae3dc800 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea307dc740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea307dc740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.080+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x556dad8abc00 con 0x556dae3dc800 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.082+0000 7fea2f78b640 1 -- 172.21.15.53:0/28733099 <== osd.3 v2:172.21.15.53:6816/2871564334 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v120'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x556dae27a500 con 0x556dae3dc800 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.082+0000 7fea307dc740 10 datascan.scan_links: updating snaptable 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.082+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x556dad8c0400 con 0x556dae3dcc00 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.082+0000 7fea2e789640 1 -- 172.21.15.53:0/28733099 <== osd.7 v2:172.21.15.135:6809/751247261 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x556dae27a000 con 0x556dae3dcc00 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.082+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x556dae451c00 msgr2=0x556dad8a6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x556dae451c00 0x556dad8a6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.111 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x556dad8aa000 msgr2=0x556dad8a6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x556dad8aa000 0x556dad8a6b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x556dae450c00 msgr2=0x556dad8a7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x556dae450c00 0x556dad8a7600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x556dae3dc800 msgr2=0x556dad8a6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x556dae3dc800 0x556dad8a6000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 reap_dead start 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x556dad8aa400 msgr2=0x556dad8a7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x556dad8aa400 0x556dad8a7080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 reap_dead start 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x556dae450800 msgr2=0x556dad8a8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x556dae450800 0x556dad8a8680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x556dad8aa800 msgr2=0x556dad8a8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x556dad8aa800 0x556dad8a8100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x556dae3dcc00 msgr2=0x556dae2b6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x556dae3dcc00 0x556dae2b6c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x556dae451000 msgr2=0x556dae2b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x556dae451000 0x556dae2b6680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x556dae3d5dd0 tx=0x556dae3dff80 comp rx=0 tx=0).stop 2024-05-10T10:43:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dd400 msgr2=0x556dae2b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/28733099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae3dd400 0x556dae2b5b80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x556dae3d5ef0 tx=0x556dae28e9c0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea2ef8a640 1 -- 172.21.15.53:0/28733099 reap_dead start 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 shutdown_connections 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 >> 172.21.15.53:0/28733099 conn(0x556dae2bc000 msgr2=0x556dae267800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 shutdown_connections 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/28733099 wait complete. 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/279651818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae2bdc00 msgr2=0x556dae2b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae2bdc00 0x556dae2b4580 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x556dae319bf0 tx=0x556dae28ea80 comp rx=0 tx=0).stop 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/279651818 shutdown_connections 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556dae2bdc00 0x556dae2b4580 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x556dad4e9c00 0x556dae2b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 --2- 172.21.15.53:0/279651818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556dae2bc800 0x556dae2b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/279651818 >> 172.21.15.53:0/279651818 conn(0x556dad4e9000 msgr2=0x556dae266000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/279651818 shutdown_connections 2024-05-10T10:43:11.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.083+0000 7fea307dc740 1 -- 172.21.15.53:0/279651818 wait complete. 2024-05-10T10:43:11.115 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.138973, result: 2024-05-10T10:43:11.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:11.145 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:11.187 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-05-10T10:43:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2024-05-10T10:43:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-05-10T10:43:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: mon down mkfs grace = 300 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2024-05-10T10:43:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-05-10T10:43:11.197 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = False 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2024-05-10T10:43:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.199 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:11.200 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:11.201 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-05-10T10:43:11.202 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-05-10T10:43:11.203 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug trim objects = true 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.204 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 20 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 20 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.205 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 10 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:11.206 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.207 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:11.208 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: mon down mkfs grace = 300 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:11.209 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = "" 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-05-10T10:43:11.210 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = False 2024-05-10T10:43:11.211 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2024-05-10T10:43:11.212 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = True 2024-05-10T10:43:11.212 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:11.212 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.212 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:11.256 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T10:43:11.298 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-05-10T10:43:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-05-10T10:43:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2024-05-10T10:43:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-05-10T10:43:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-05-10T10:43:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: mon down mkfs grace = 300 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-05-10T10:43:11.307 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-05-10T10:43:11.308 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.309 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.310 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-05-10T10:43:11.310 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-05-10T10:43:11.310 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T10:43:11.310 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-05-10T10:43:11.310 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.310 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-05-10T10:43:11.310 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T10:43:11.311 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.312 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.53,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 543b513e-90c7-47ff-9cdd-f7fa4d978734 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.313 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug trim objects = true 2024-05-10T10:43:11.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-05-10T10:43:11.315 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 20 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 20 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 10 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:11.316 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-05-10T10:43:11.317 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.318 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon down mkfs grace = 300 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-05-10T10:43:11.319 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = "" 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-05-10T10:43:11.320 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = True 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = True 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-05-10T10:43:11.321 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-05-10T10:43:11.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-10T10:43:11.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.467+0000 7fbc69589640 1 Processor -- start 2024-05-10T10:43:11.469 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.467+0000 7fbc69589640 1 -- start start 2024-05-10T10:43:11.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc69589640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc640cffd0 0x7fbc640b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc69589640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cd500 0x7fbc640d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc69589640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6406e720 con 0x7fbc640cf3e0 2024-05-10T10:43:11.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc69589640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbc6406de80 con 0x7fbc640cffd0 2024-05-10T10:43:11.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc69589640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6406e160 con 0x7fbc640cd500 2024-05-10T10:43:11.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc627fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc640cffd0 0x7fbc640b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc627fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc640cffd0 0x7fbc640b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50472/0 (socket says 172.21.15.53:50472) 2024-05-10T10:43:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc627fc640 1 -- 172.21.15.53:0/2956262564 learned_addr learned my addr 172.21.15.53:0/2956262564 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.468+0000 7fbc637fe640 1 --2- 172.21.15.53:0/2956262564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cd500 0x7fbc640d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc627fc640 1 -- 172.21.15.53:0/2956262564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cd500 msgr2=0x7fbc640d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc627fc640 1 --2- 172.21.15.53:0/2956262564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cd500 0x7fbc640d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc627fc640 1 -- 172.21.15.53:0/2956262564 >> v1:172.21.15.135:6789/0 conn(0x7fbc640cf3e0 legacy=0x7fbc640cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc627fc640 1 -- 172.21.15.53:0/2956262564 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6406e440 con 0x7fbc640cffd0 2024-05-10T10:43:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc637fe640 1 --2- 172.21.15.53:0/2956262564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cd500 0x7fbc640d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc627fc640 1 --2- 172.21.15.53:0/2956262564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cffd0 0x7fbc640b3de0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009870 tx=0x7fbc4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e415fa55ad77d94f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc61ffb640 1 -- 172.21.15.53:0/2956262564 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c037070 con 0x7fbc640cffd0 2024-05-10T10:43:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc61ffb640 1 -- 172.21.15.53:0/2956262564 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c032440 con 0x7fbc640cffd0 2024-05-10T10:43:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.469+0000 7fbc61ffb640 1 -- 172.21.15.53:0/2956262564 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c032810 con 0x7fbc640cffd0 2024-05-10T10:43:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 -- 172.21.15.53:0/2956262564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cffd0 msgr2=0x7fbc640b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 --2- 172.21.15.53:0/2956262564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cffd0 0x7fbc640b3de0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009870 tx=0x7fbc4c02a270 comp rx=0 tx=0).stop 2024-05-10T10:43:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 -- 172.21.15.53:0/2956262564 shutdown_connections 2024-05-10T10:43:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 --2- 172.21.15.53:0/2956262564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cd500 0x7fbc640d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 --2- 172.21.15.53:0/2956262564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cffd0 0x7fbc640b3de0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 -- 172.21.15.53:0/2956262564 >> 172.21.15.53:0/2956262564 conn(0x7fbc640d2eb0 msgr2=0x7fbc640d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 -- 172.21.15.53:0/2956262564 shutdown_connections 2024-05-10T10:43:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.470+0000 7fbc69589640 1 -- 172.21.15.53:0/2956262564 wait complete. 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.471+0000 7fbc69589640 1 Processor -- start 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.471+0000 7fbc69589640 1 -- start start 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc69589640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cd500 0x7fbc64157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc69589640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc640cf3e0 0x7fbc640ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc69589640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cffd0 0x7fbc640cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc69589640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbc640d16c0 con 0x7fbc640cd500 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc69589640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc640b4510 con 0x7fbc640cf3e0 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc69589640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fbc640d1840 con 0x7fbc640cffd0 2024-05-10T10:43:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc62ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cd500 0x7fbc64157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc62ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cd500 0x7fbc64157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50484/0 (socket says 172.21.15.53:50484) 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc62ffd640 1 -- 172.21.15.53:0/1879258184 learned_addr learned my addr 172.21.15.53:0/1879258184 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc627fc640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc640cf3e0 0x7fbc640ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.472+0000 7fbc637fe640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cffd0 0x7fbc640cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc62ffd640 1 -- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cffd0 msgr2=0x7fbc640cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc62ffd640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cffd0 0x7fbc640cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc62ffd640 1 -- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc640cf3e0 msgr2=0x7fbc640ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc62ffd640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc640cf3e0 0x7fbc640ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc62ffd640 1 -- 172.21.15.53:0/1879258184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc640cb1c0 con 0x7fbc640cd500 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc62ffd640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cd500 0x7fbc64157400 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c7c0 tx=0x7fbc5400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc5b7fe640 1 -- 172.21.15.53:0/1879258184 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54016020 con 0x7fbc640cd500 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc5b7fe640 1 -- 172.21.15.53:0/1879258184 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54006d50 con 0x7fbc640cd500 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc640cb4a0 con 0x7fbc640cd500 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc5b7fe640 1 -- 172.21.15.53:0/1879258184 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54003bb0 con 0x7fbc640cd500 2024-05-10T10:43:11.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.473+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc640cba60 con 0x7fbc640cd500 2024-05-10T10:43:11.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.474+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6406a220 con 0x7fbc640cd500 2024-05-10T10:43:11.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.475+0000 7fbc5b7fe640 1 -- 172.21.15.53:0/1879258184 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbc5400a850 con 0x7fbc640cd500 2024-05-10T10:43:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.475+0000 7fbc5b7fe640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbc440762d0 0x7fbc44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.475+0000 7fbc5b7fe640 1 -- 172.21.15.53:0/1879258184 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7fbc54097550 con 0x7fbc640cd500 2024-05-10T10:43:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.475+0000 7fbc627fc640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbc440762d0 0x7fbc44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.476+0000 7fbc627fc640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbc440762d0 0x7fbc44078790 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c034000 tx=0x7fbc4c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:11.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.480+0000 7fbc5b7fe640 1 -- 172.21.15.53:0/1879258184 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fbc54020020 con 0x7fbc640cd500 2024-05-10T10:43:11.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:11.713+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbc641626e0 con 0x7fbc640cd500 2024-05-10T10:43:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.610+0000 7fbc5b7fe640 1 -- 172.21.15.53:0/1879258184 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v107) ==== 162+0+0 (secure 0 0 0) 0x7fbc54060d00 con 0x7fbc640cd500 2024-05-10T10:43:12.612 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-10T10:43:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbc440762d0 msgr2=0x7fbc44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbc440762d0 0x7fbc44078790 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c034000 tx=0x7fbc4c02e000 comp rx=0 tx=0).stop 2024-05-10T10:43:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cd500 msgr2=0x7fbc64157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cd500 0x7fbc64157400 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400c7c0 tx=0x7fbc5400cc80 comp rx=0 tx=0).stop 2024-05-10T10:43:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 shutdown_connections 2024-05-10T10:43:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fbc440762d0 0x7fbc44078790 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.615 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc640cffd0 0x7fbc640cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc640cf3e0 0x7fbc640ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 --2- 172.21.15.53:0/1879258184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640cd500 0x7fbc64157400 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 >> 172.21.15.53:0/1879258184 conn(0x7fbc640d2eb0 msgr2=0x7fbc640757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 shutdown_connections 2024-05-10T10:43:12.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.613+0000 7fbc69589640 1 -- 172.21.15.53:0/1879258184 wait complete. 2024-05-10T10:43:12.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:12.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.740+0000 7f08fd546640 1 Processor -- start 2024-05-10T10:43:12.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.740+0000 7f08fd546640 1 -- start start 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.740+0000 7f08fd546640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f80d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.740+0000 7f08fd546640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08f80d5cb0 0x7f08f80d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.740+0000 7f08fd546640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f80b45f0 con 0x7f08f8162b30 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.740+0000 7f08fd546640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f08f80b4f20 con 0x7f08f80d5cb0 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.740+0000 7f08fd546640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f08f80d0fe0 con 0x7f08f8162750 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f67fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08f80d5cb0 0x7f08f80d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f77fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f08f8162b30 0x7f08f80d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42352/0 (socket says 172.21.15.53:42352) 2024-05-10T10:43:12.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f77fe640 1 -- 172.21.15.53:0/3885189442 learned_addr learned my addr 172.21.15.53:0/3885189442 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f6ffd640 1 --2- 172.21.15.53:0/3885189442 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f80d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f67fc640 1 -- 172.21.15.53:0/3885189442 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 msgr2=0x7f08f80d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f67fc640 1 --2- 172.21.15.53:0/3885189442 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f80d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f67fc640 1 -- 172.21.15.53:0/3885189442 >> v1:172.21.15.53:6789/0 conn(0x7f08f8162b30 legacy=0x7f08f80d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f67fc640 1 -- 172.21.15.53:0/3885189442 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f80b48d0 con 0x7f08f80d5cb0 2024-05-10T10:43:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.741+0000 7f08f5ffb640 1 -- 172.21.15.53:0/3885189442 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2204634959 0 0) 0x7f08f80b45f0 con 0x7f08f8162b30 2024-05-10T10:43:12.744 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08f67fc640 1 --2- 172.21.15.53:0/3885189442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f80d5cb0 0x7f08f80d3630 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097a0 tx=0x7f08ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f414fe7acc4e4d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:12.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08f5ffb640 1 -- 172.21.15.53:0/3885189442 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec037070 con 0x7f08f80d5cb0 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08f5ffb640 1 -- 172.21.15.53:0/3885189442 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec02ae50 con 0x7f08f80d5cb0 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08f5ffb640 1 -- 172.21.15.53:0/3885189442 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec032660 con 0x7f08f80d5cb0 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08fd546640 1 -- 172.21.15.53:0/3885189442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f80d5cb0 msgr2=0x7f08f80d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08fd546640 1 --2- 172.21.15.53:0/3885189442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f80d5cb0 0x7f08f80d3630 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097a0 tx=0x7f08ec02a100 comp rx=0 tx=0).stop 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08fd546640 1 -- 172.21.15.53:0/3885189442 shutdown_connections 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08fd546640 1 --2- 172.21.15.53:0/3885189442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f80d5cb0 0x7f08f80d3630 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08fd546640 1 --2- 172.21.15.53:0/3885189442 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f80d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.742+0000 7f08fd546640 1 -- 172.21.15.53:0/3885189442 >> 172.21.15.53:0/3885189442 conn(0x7f08f809e3b0 msgr2=0x7f08f80d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.743+0000 7f08fd546640 1 -- 172.21.15.53:0/3885189442 shutdown_connections 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.743+0000 7f08fd546640 1 -- 172.21.15.53:0/3885189442 wait complete. 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.744+0000 7f08fd546640 1 Processor -- start 2024-05-10T10:43:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.744+0000 7f08fd546640 1 -- start start 2024-05-10T10:43:12.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:12.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08f8162b30 0x7f08f80b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:12.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f8160010 0x7f08f815b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:12.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08f80d0d30 con 0x7f08f8160010 2024-05-10T10:43:12.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08f80d0bb0 con 0x7f08f8162b30 2024-05-10T10:43:12.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f08f80b63c0 con 0x7f08f8162750 2024-05-10T10:43:12.749 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f8160010 0x7f08f815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f8160010 0x7f08f815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52348/0 (socket says 172.21.15.53:52348) 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 -- 172.21.15.53:0/231706177 learned_addr learned my addr 172.21.15.53:0/231706177 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f6ffd640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f67fc640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08f8162b30 0x7f08f80b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 -- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 msgr2=0x7f08f815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f815c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 -- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08f8162b30 msgr2=0x7f08f80b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08f8162b30 0x7f08f80b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 -- 172.21.15.53:0/231706177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f80b7ce0 con 0x7f08f8160010 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08f77fe640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f8160010 0x7f08f815b600 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c700 tx=0x7f08f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08df7fe640 1 -- 172.21.15.53:0/231706177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0018070 con 0x7f08f8160010 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08df7fe640 1 -- 172.21.15.53:0/231706177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f000ce10 con 0x7f08f8160010 2024-05-10T10:43:12.750 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f80b7fc0 con 0x7f08f8160010 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08df7fe640 1 -- 172.21.15.53:0/231706177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f000a860 con 0x7f08f8160010 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.745+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f80b8580 con 0x7f08f8160010 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.746+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f806a220 con 0x7f08f8160010 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.747+0000 7f08df7fe640 1 -- 172.21.15.53:0/231706177 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f08f001b070 con 0x7f08f8160010 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.748+0000 7f08df7fe640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08e40762d0 0x7f08e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.748+0000 7f08df7fe640 1 -- 172.21.15.53:0/231706177 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f08f0096d10 con 0x7f08f8160010 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.748+0000 7f08f6ffd640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08e40762d0 0x7f08e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:12.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.748+0000 7f08f6ffd640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08e40762d0 0x7f08e4078790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f08f8160a80 tx=0x7f08e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:12.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.752+0000 7f08df7fe640 1 -- 172.21.15.53:0/231706177 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f08f00604c0 con 0x7f08f8160010 2024-05-10T10:43:12.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.982+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08f80d1810 con 0x7f08f8160010 2024-05-10T10:43:12.984 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.982+0000 7f08df7fe640 1 -- 172.21.15.53:0/231706177 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5179 (secure 0 0 0) 0x7f08f005fdc0 con 0x7f08f8160010 2024-05-10T10:43:12.984 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:12.984 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:12.615305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8352},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8352":{"gid":8352,"name":"c","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.53:6834/2292645283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2292645283},{"type":"v1","addr":"172.21.15.53:6834","nonce":2292645283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-10T10:43:12.985 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 108 2024-05-10T10:43:12.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08e40762d0 msgr2=0x7f08e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08e40762d0 0x7f08e4078790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f08f8160a80 tx=0x7f08e8008040 comp rx=0 tx=0).stop 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f8160010 msgr2=0x7f08f815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f8160010 0x7f08f815b600 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c700 tx=0x7f08f000cbc0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 shutdown_connections 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f08e40762d0 0x7f08e4078790 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08f8160010 0x7f08f815b600 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08f8162b30 0x7f08f80b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 --2- 172.21.15.53:0/231706177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08f8162750 0x7f08f815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 >> 172.21.15.53:0/231706177 conn(0x7f08f809e3b0 msgr2=0x7f08f815e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 shutdown_connections 2024-05-10T10:43:12.987 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:12.984+0000 7f08fd546640 1 -- 172.21.15.53:0/231706177 wait complete. 2024-05-10T10:43:13.401 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:42:49.598991+0000', 'modified': '2024-05-10T10:43:12.615305+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8352}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8352': {'gid': 8352, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.53:6834/2292645283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 2292645283}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 2292645283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:43:13.401 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8352:up:replay 2024-05-10T10:43:14.402 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:14.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.517+0000 7f29b1119640 1 Processor -- start 2024-05-10T10:43:14.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.517+0000 7f29b1119640 1 -- start start 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29b1119640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f29ac162750 0x7f29ac0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29b1119640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f29ac162b30 0x7f29ac0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29b1119640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29ac0b45f0 con 0x7f29ac0d5cb0 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29b1119640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f29ac0b4f20 con 0x7f29ac162750 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29b1119640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f29ac0d0fe0 con 0x7f29ac162b30 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29aad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f29ac162750 0x7f29ac0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29a3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f29ac0d5cb0 0x7f29ac0d3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42366/0 (socket says 172.21.15.53:42366) 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29a3fff640 1 -- 172.21.15.53:0/3444489032 learned_addr learned my addr 172.21.15.53:0/3444489032 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29ab59e640 1 --2- 172.21.15.53:0/3444489032 >> v2:172.21.15.135:3300/0 conn(0x7f29ac162b30 0x7f29ac0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29aad9d640 1 -- 172.21.15.53:0/3444489032 >> v2:172.21.15.135:3300/0 conn(0x7f29ac162b30 msgr2=0x7f29ac0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29aad9d640 1 --2- 172.21.15.53:0/3444489032 >> v2:172.21.15.135:3300/0 conn(0x7f29ac162b30 0x7f29ac0b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29aad9d640 1 -- 172.21.15.53:0/3444489032 >> v1:172.21.15.53:6789/0 conn(0x7f29ac0d5cb0 legacy=0x7f29ac0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:14.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.518+0000 7f29aad9d640 1 -- 172.21.15.53:0/3444489032 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac0b48d0 con 0x7f29ac162750 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.519+0000 7f29aad9d640 1 --2- 172.21.15.53:0/3444489032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac162750 0x7f29ac0d0670 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0d32c0 tx=0x7f299c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77772b3283cd25d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.519+0000 7f29aa59c640 1 -- 172.21.15.53:0/3444489032 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c037070 con 0x7f29ac162750 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.519+0000 7f29aa59c640 1 -- 172.21.15.53:0/3444489032 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f299c032440 con 0x7f29ac162750 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.519+0000 7f29aa59c640 1 -- 172.21.15.53:0/3444489032 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c02ae60 con 0x7f29ac162750 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 -- 172.21.15.53:0/3444489032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac162750 msgr2=0x7f29ac0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 --2- 172.21.15.53:0/3444489032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac162750 0x7f29ac0d0670 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0d32c0 tx=0x7f299c02a110 comp rx=0 tx=0).stop 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 -- 172.21.15.53:0/3444489032 shutdown_connections 2024-05-10T10:43:14.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 --2- 172.21.15.53:0/3444489032 >> v2:172.21.15.135:3300/0 conn(0x7f29ac162b30 0x7f29ac0b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 --2- 172.21.15.53:0/3444489032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac162750 0x7f29ac0d0670 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 -- 172.21.15.53:0/3444489032 >> 172.21.15.53:0/3444489032 conn(0x7f29ac09e3b0 msgr2=0x7f29ac0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:14.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 -- 172.21.15.53:0/3444489032 shutdown_connections 2024-05-10T10:43:14.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.520+0000 7f29b1119640 1 -- 172.21.15.53:0/3444489032 wait complete. 2024-05-10T10:43:14.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.521+0000 7f29b1119640 1 Processor -- start 2024-05-10T10:43:14.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.521+0000 7f29b1119640 1 -- start start 2024-05-10T10:43:14.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29b1119640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ac162750 0x7f29ac1f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:14.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29b1119640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29ac162b30 0x7f29ac15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:14.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29b1119640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac15cfa0 0x7f29ac15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29b1119640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0d0d30 con 0x7f29ac15cfa0 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29b1119640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0d0bb0 con 0x7f29ac162750 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29b1119640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac0d3d40 con 0x7f29ac162b30 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac15cfa0 0x7f29ac15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac15cfa0 0x7f29ac15b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52368/0 (socket says 172.21.15.53:52368) 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 -- 172.21.15.53:0/1191428550 learned_addr learned my addr 172.21.15.53:0/1191428550 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29aad9d640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ac162750 0x7f29ac1f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29a3fff640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29ac162b30 0x7f29ac15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 -- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29ac162b30 msgr2=0x7f29ac15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29ac162b30 0x7f29ac15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 -- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ac162750 msgr2=0x7f29ac1f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ac162750 0x7f29ac1f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29ab59e640 1 -- 172.21.15.53:0/1191428550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac1f53d0 con 0x7f29ac15cfa0 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.522+0000 7f29a3fff640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29ac162b30 0x7f29ac15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.523+0000 7f29ab59e640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac15cfa0 0x7f29ac15b5a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f29a400cc30 tx=0x7f29a400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.523+0000 7f29a37fe640 1 -- 172.21.15.53:0/1191428550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a400ce40 con 0x7f29ac15cfa0 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.523+0000 7f29a37fe640 1 -- 172.21.15.53:0/1191428550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a4003990 con 0x7f29ac15cfa0 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.523+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac1f56b0 con 0x7f29ac15cfa0 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.523+0000 7f29a37fe640 1 -- 172.21.15.53:0/1191428550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4003d40 con 0x7f29ac15cfa0 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.523+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac1f5c70 con 0x7f29ac15cfa0 2024-05-10T10:43:14.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.524+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ac1f5e70 con 0x7f29ac15cfa0 2024-05-10T10:43:14.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.525+0000 7f29a37fe640 1 -- 172.21.15.53:0/1191428550 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f29a4004000 con 0x7f29ac15cfa0 2024-05-10T10:43:14.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.525+0000 7f29a37fe640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f298c0762d0 0x7f298c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:14.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.525+0000 7f29a37fe640 1 -- 172.21.15.53:0/1191428550 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f29a4097c90 con 0x7f29ac15cfa0 2024-05-10T10:43:14.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.526+0000 7f29aad9d640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f298c0762d0 0x7f298c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:14.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.526+0000 7f29aad9d640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f298c0762d0 0x7f298c078790 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f299c034000 tx=0x7f299c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:14.531 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.530+0000 7f29a37fe640 1 -- 172.21.15.53:0/1191428550 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f29ac1f5e70 con 0x7f29ac15cfa0 2024-05-10T10:43:14.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.764+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29ac06a220 con 0x7f29ac15cfa0 2024-05-10T10:43:14.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.764+0000 7f29a37fe640 1 -- 172.21.15.53:0/1191428550 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5179 (secure 0 0 0) 0x7f29a4014030 con 0x7f29ac15cfa0 2024-05-10T10:43:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:14.766 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:13.619305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8352},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8352":{"gid":8352,"name":"c","rank":0,"incarnation":108,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.53:6834/2292645283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2292645283},{"type":"v1","addr":"172.21.15.53:6834","nonce":2292645283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-10T10:43:14.767 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 110 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f298c0762d0 msgr2=0x7f298c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f298c0762d0 0x7f298c078790 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f299c034000 tx=0x7f299c02e000 comp rx=0 tx=0).stop 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac15cfa0 msgr2=0x7f29ac15b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac15cfa0 0x7f29ac15b5a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f29a400cc30 tx=0x7f29a400a5c0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 shutdown_connections 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f298c0762d0 0x7f298c078790 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29ac15cfa0 0x7f29ac15b5a0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29ac162b30 0x7f29ac15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.769 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 --2- 172.21.15.53:0/1191428550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29ac162750 0x7f29ac1f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:14.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 >> 172.21.15.53:0/1191428550 conn(0x7f29ac09e3b0 msgr2=0x7f29ac06e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:14.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 shutdown_connections 2024-05-10T10:43:14.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:14.767+0000 7f29b1119640 1 -- 172.21.15.53:0/1191428550 wait complete. 2024-05-10T10:43:15.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:42:49.598991+0000', 'modified': '2024-05-10T10:43:13.619305+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8352}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8352': {'gid': 8352, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.53:6834/2292645283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 2292645283}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 2292645283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-10T10:43:15.183 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8352:up:rejoin 2024-05-10T10:43:16.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:16.118 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:16.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:16.298 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.297+0000 7f161152b640 1 Processor -- start 2024-05-10T10:43:16.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.297+0000 7f161152b640 1 -- start start 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f161152b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f160c162750 0x7f160c0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f161152b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162b30 0x7f160c0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f161152b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f160c0b45f0 con 0x7f160c0d5cb0 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f161152b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f160c0b4f20 con 0x7f160c162750 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f161152b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f160c0d0fe0 con 0x7f160c162b30 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f160c162750 0x7f160c0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f160c162750 0x7f160c0d0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52376/0 (socket says 172.21.15.53:52376) 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160affd640 1 -- 172.21.15.53:0/2178773554 learned_addr learned my addr 172.21.15.53:0/2178773554 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160b7fe640 1 --2- 172.21.15.53:0/2178773554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162b30 0x7f160c0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160affd640 1 -- 172.21.15.53:0/2178773554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162b30 msgr2=0x7f160c0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:16.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160affd640 1 --2- 172.21.15.53:0/2178773554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162b30 0x7f160c0b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160affd640 1 -- 172.21.15.53:0/2178773554 >> v1:172.21.15.135:6789/0 conn(0x7f160c0d5cb0 legacy=0x7f160c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:16.301 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.298+0000 7f160affd640 1 -- 172.21.15.53:0/2178773554 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c0b48d0 con 0x7f160c162750 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.299+0000 7f160affd640 1 --2- 172.21.15.53:0/2178773554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162750 0x7f160c0d0670 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009870 tx=0x7f15fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf746cf9eb6e3fd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.299+0000 7f160a7fc640 1 -- 172.21.15.53:0/2178773554 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc037070 con 0x7f160c162750 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.299+0000 7f160a7fc640 1 -- 172.21.15.53:0/2178773554 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc032440 con 0x7f160c162750 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.299+0000 7f160a7fc640 1 -- 172.21.15.53:0/2178773554 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc032810 con 0x7f160c162750 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 -- 172.21.15.53:0/2178773554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162750 msgr2=0x7f160c0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 --2- 172.21.15.53:0/2178773554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162750 0x7f160c0d0670 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009870 tx=0x7f15fc02a180 comp rx=0 tx=0).stop 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 -- 172.21.15.53:0/2178773554 shutdown_connections 2024-05-10T10:43:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 --2- 172.21.15.53:0/2178773554 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162b30 0x7f160c0b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 --2- 172.21.15.53:0/2178773554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162750 0x7f160c0d0670 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 -- 172.21.15.53:0/2178773554 >> 172.21.15.53:0/2178773554 conn(0x7f160c09e3b0 msgr2=0x7f160c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 -- 172.21.15.53:0/2178773554 shutdown_connections 2024-05-10T10:43:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.300+0000 7f161152b640 1 -- 172.21.15.53:0/2178773554 wait complete. 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.301+0000 7f161152b640 1 Processor -- start 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f161152b640 1 -- start start 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f161152b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162750 0x7f160c15b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f161152b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162b30 0x7f160c0b8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f161152b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f160c0cef50 0x7f160c0ba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f161152b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f160c0d0d30 con 0x7f160c162b30 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f161152b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f160c0d0bb0 con 0x7f160c0cef50 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f161152b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f160c0d3d40 con 0x7f160c162750 2024-05-10T10:43:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f16027fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162b30 0x7f160c0b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f16027fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162b30 0x7f160c0b8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52386/0 (socket says 172.21.15.53:52386) 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.302+0000 7f16027fc640 1 -- 172.21.15.53:0/2133249404 learned_addr learned my addr 172.21.15.53:0/2133249404 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f160affd640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162750 0x7f160c15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f160b7fe640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f160c0cef50 0x7f160c0ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f16027fc640 1 -- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162750 msgr2=0x7f160c15b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f16027fc640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162750 0x7f160c15b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f16027fc640 1 -- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f160c0cef50 msgr2=0x7f160c0ba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f16027fc640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f160c0cef50 0x7f160c0ba100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f16027fc640 1 -- 172.21.15.53:0/2133249404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c0cce90 con 0x7f160c162b30 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f16027fc640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162b30 0x7f160c0b8450 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c3f0 tx=0x7f15f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f1603fff640 1 -- 172.21.15.53:0/2133249404 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4007020 con 0x7f160c162b30 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f1603fff640 1 -- 172.21.15.53:0/2133249404 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f40071c0 con 0x7f160c162b30 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f1603fff640 1 -- 172.21.15.53:0/2133249404 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4007590 con 0x7f160c162b30 2024-05-10T10:43:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c0cae80 con 0x7f160c162b30 2024-05-10T10:43:16.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.303+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c0cb440 con 0x7f160c162b30 2024-05-10T10:43:16.307 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.304+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c15b6a0 con 0x7f160c162b30 2024-05-10T10:43:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.305+0000 7f1603fff640 1 -- 172.21.15.53:0/2133249404 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f15f4019020 con 0x7f160c162b30 2024-05-10T10:43:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.305+0000 7f1603fff640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15ec0762d0 0x7f15ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.306+0000 7f1603fff640 1 -- 172.21.15.53:0/2133249404 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f15f4097020 con 0x7f160c162b30 2024-05-10T10:43:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.306+0000 7f160affd640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15ec0762d0 0x7f15ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.306+0000 7f160affd640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15ec0762d0 0x7f15ec078790 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f15fc034000 tx=0x7f15fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.310+0000 7f1603fff640 1 -- 172.21.15.53:0/2133249404 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f15f4060740 con 0x7f160c162b30 2024-05-10T10:43:16.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.538+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f160c0cd060 con 0x7f160c162b30 2024-05-10T10:43:16.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.539+0000 7f1603fff640 1 -- 172.21.15.53:0/2133249404 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5186 (secure 0 0 0) 0x7f15f4060040 con 0x7f160c162b30 2024-05-10T10:43:16.541 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:16.541 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:15.622565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8352},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8352":{"gid":8352,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/2292645283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2292645283},{"type":"v1","addr":"172.21.15.53:6834","nonce":2292645283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8352,"qdb_cluster":[8352]},"id":8}]} 2024-05-10T10:43:16.541 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 111 2024-05-10T10:43:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15ec0762d0 msgr2=0x7f15ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15ec0762d0 0x7f15ec078790 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f15fc034000 tx=0x7f15fc02e000 comp rx=0 tx=0).stop 2024-05-10T10:43:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162b30 msgr2=0x7f160c0b8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162b30 0x7f160c0b8450 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c3f0 tx=0x7f15f400c8b0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 shutdown_connections 2024-05-10T10:43:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f15ec0762d0 0x7f15ec078790 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f160c0cef50 0x7f160c0ba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c162b30 0x7f160c0b8450 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 --2- 172.21.15.53:0/2133249404 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f160c162750 0x7f160c15b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:16.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 >> 172.21.15.53:0/2133249404 conn(0x7f160c09e3b0 msgr2=0x7f160c0d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:16.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 shutdown_connections 2024-05-10T10:43:16.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:16.541+0000 7f161152b640 1 -- 172.21.15.53:0/2133249404 wait complete. 2024-05-10T10:43:16.612 INFO:tasks.ceph.mds.c.smithi053.stderr:2024-05-10T10:43:16.612+0000 7ffa9705a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:43:16.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T10:42:49.598991+0000', 'modified': '2024-05-10T10:43:15.622565+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8352}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8352': {'gid': 8352, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6834/2292645283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 2292645283}, {'type': 'v1', 'addr': '172.21.15.53:6834', 'nonce': 2292645283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8352, 'qdb_cluster': [8352]} 2024-05-10T10:43:16.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T10:43:16.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T10:43:17.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83be7af640 1 Processor -- start 2024-05-10T10:43:17.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83be7af640 1 -- start start 2024-05-10T10:43:17.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83be7af640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f83b815f4e0 0x7f83b806d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83be7af640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83b806e620 0x7f83b80d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83be7af640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b80b0a90 con 0x7f83b815f8c0 2024-05-10T10:43:17.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83be7af640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f83b80b23f0 con 0x7f83b806e620 2024-05-10T10:43:17.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83be7af640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f83b80b4150 con 0x7f83b815f4e0 2024-05-10T10:43:17.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83bcd4c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83b806e620 0x7f83b80d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83bcd4c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83b806e620 0x7f83b80d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52398/0 (socket says 172.21.15.53:52398) 2024-05-10T10:43:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83bcd4c640 1 -- 172.21.15.53:0/3275919170 learned_addr learned my addr 172.21.15.53:0/3275919170 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83b7fff640 1 --2- 172.21.15.53:0/3275919170 >> v2:172.21.15.135:3300/0 conn(0x7f83b815f4e0 0x7f83b806d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83bcd4c640 1 -- 172.21.15.53:0/3275919170 >> v2:172.21.15.135:3300/0 conn(0x7f83b815f4e0 msgr2=0x7f83b806d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/3275919170 >> v2:172.21.15.135:3300/0 conn(0x7f83b815f4e0 0x7f83b806d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83bcd4c640 1 -- 172.21.15.53:0/3275919170 >> v1:172.21.15.135:6789/0 conn(0x7f83b815f8c0 legacy=0x7f83b806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.063+0000 7f83bcd4c640 1 -- 172.21.15.53:0/3275919170 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b80b3540 con 0x7f83b806e620 2024-05-10T10:43:17.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.064+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/3275919170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b80d0cf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f83b8068c10 tx=0x7f83b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c6c34fef676c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.064+0000 7f83b77fe640 1 -- 172.21.15.53:0/3275919170 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0036070 con 0x7f83b806e620 2024-05-10T10:43:17.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.064+0000 7f83b77fe640 1 -- 172.21.15.53:0/3275919170 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b0031600 con 0x7f83b806e620 2024-05-10T10:43:17.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.064+0000 7f83b77fe640 1 -- 172.21.15.53:0/3275919170 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b00319d0 con 0x7f83b806e620 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 -- 172.21.15.53:0/3275919170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 msgr2=0x7f83b80d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 --2- 172.21.15.53:0/3275919170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b80d0cf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f83b8068c10 tx=0x7f83b002a660 comp rx=0 tx=0).stop 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 -- 172.21.15.53:0/3275919170 shutdown_connections 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 --2- 172.21.15.53:0/3275919170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b80d0cf0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 --2- 172.21.15.53:0/3275919170 >> v2:172.21.15.135:3300/0 conn(0x7f83b815f4e0 0x7f83b806d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 -- 172.21.15.53:0/3275919170 >> 172.21.15.53:0/3275919170 conn(0x7f83b80d2df0 msgr2=0x7f83b80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 -- 172.21.15.53:0/3275919170 shutdown_connections 2024-05-10T10:43:17.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.065+0000 7f83be7af640 1 -- 172.21.15.53:0/3275919170 wait complete. 2024-05-10T10:43:17.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.066+0000 7f83be7af640 1 Processor -- start 2024-05-10T10:43:17.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.066+0000 7f83be7af640 1 -- start start 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83be7af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b81f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83be7af640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83b815f4e0 0x7f83b8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83be7af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83b815f8c0 0x7f83b8157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83be7af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80b3de0 con 0x7f83b806e620 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83be7af640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80b3b00 con 0x7f83b815f8c0 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83be7af640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83b80b45e0 con 0x7f83b815f4e0 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b81f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b81f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52406/0 (socket says 172.21.15.53:52406) 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 -- 172.21.15.53:0/3529386320 learned_addr learned my addr 172.21.15.53:0/3529386320 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83b815f8c0 0x7f83b8157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83affff640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83b815f4e0 0x7f83b8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 -- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83b815f4e0 msgr2=0x7f83b8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83b815f4e0 0x7f83b8157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 -- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83b815f8c0 msgr2=0x7f83b8157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83b815f8c0 0x7f83b8157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83b7fff640 1 -- 172.21.15.53:0/3529386320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b8160ed0 con 0x7f83b806e620 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83b815f8c0 0x7f83b8157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.067+0000 7f83affff640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83b815f4e0 0x7f83b8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.068+0000 7f83b7fff640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b81f4b40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f83a800d830 tx=0x7f83a800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.068+0000 7f83b57fa640 1 -- 172.21.15.53:0/3529386320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a800b6c0 con 0x7f83b806e620 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.068+0000 7f83b57fa640 1 -- 172.21.15.53:0/3529386320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a8019070 con 0x7f83b806e620 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.068+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b81611b0 con 0x7f83b806e620 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.068+0000 7f83b57fa640 1 -- 172.21.15.53:0/3529386320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a800ba80 con 0x7f83b806e620 2024-05-10T10:43:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.068+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b8161770 con 0x7f83b806e620 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.069+0000 7f83b57fa640 1 -- 172.21.15.53:0/3529386320 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f83a801e460 con 0x7f83b806e620 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.070+0000 7f83b57fa640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83980762d0 0x7f8398078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.070+0000 7f83b57fa640 1 -- 172.21.15.53:0/3529386320 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f83a8097e40 con 0x7f83b806e620 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.070+0000 7f83affff640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83980762d0 0x7f8398078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 Processor -- start 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 -- start start 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83affff640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83980762d0 0x7f8398078790 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f83b815b5a0 tx=0x7f83a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be103760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.079 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be101330 0x55b2be103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0d3bb0 0x55b2be0d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b2be03d8d0 con 0x55b2be0d3bb0 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2bdff6c10 con 0x55b2be101330 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b2be039890 con 0x55b2be100f50 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83bcd4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0d3bb0 0x55b2be0d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83bcd4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0d3bb0 0x55b2be0d73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52412/0 (socket says 172.21.15.53:52412) 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.071+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1793500050 learned_addr learned my addr 172.21.15.53:0/1793500050 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83affff640 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be101330 0x55b2be103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be103760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 msgr2=0x55b2be103760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be103760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be101330 msgr2=0x55b2be103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be101330 0x55b2be103ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1793500050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2bdffe5d0 con 0x55b2be0d3bb0 2024-05-10T10:43:17.080 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0d3bb0 0x55b2be0d73c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f83b002a6a0 tx=0x7f83b002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b3f169a28f8042 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83adffb640 1 -- 172.21.15.53:0/1793500050 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0036040 con 0x55b2be0d3bb0 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83adffb640 1 -- 172.21.15.53:0/1793500050 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b0042070 con 0x55b2be0d3bb0 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.072+0000 7f83adffb640 1 -- 172.21.15.53:0/1793500050 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b003c870 con 0x55b2be0d3bb0 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 -- 172.21.15.53:0/1793500050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0d3bb0 msgr2=0x55b2be0d73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0d3bb0 0x55b2be0d73c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f83b002a6a0 tx=0x7f83b002e000 comp rx=0 tx=0).stop 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 -- 172.21.15.53:0/1793500050 shutdown_connections 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0d3bb0 0x55b2be0d73c0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be101330 0x55b2be103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1793500050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 -- 172.21.15.53:0/1793500050 >> 172.21.15.53:0/1793500050 conn(0x55b2be0d5e20 msgr2=0x55b2be11e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 -- 172.21.15.53:0/1793500050 shutdown_connections 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 -- 172.21.15.53:0/1793500050 wait complete. 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.073+0000 7f83c15a3740 1 -- start start 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 1 Processor -- start 2024-05-10T10:43:17.081 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be0faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be0fed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b2be0d8fd0 con 0x55b2be0fb100 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2bdffa760 con 0x55b2be100f50 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b2be0d92b0 con 0x55b2be101330 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83c15a3740 10 client.0 _ll_register_callbacks cb 0x55b2be033cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83bcd4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83bcd4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52416/0 (socket says 172.21.15.53:52416) 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.074+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1295848057 learned_addr learned my addr 172.21.15.53:0/1295848057 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be0faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83affff640 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be0fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 msgr2=0x55b2be0fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.082 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be0fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 msgr2=0x55b2be0faa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be0faa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1295848057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2bdd4f2d0 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fb580 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f83b002a6a0 tx=0x7f83b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672e7385988c672c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f8396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f8396ffd640 1 -- 172.21.15.53:0/1295848057 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b004a070 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f8396ffd640 1 -- 172.21.15.53:0/1295848057 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f83b0004450 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b0042070 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f8396ffd640 1 -- 172.21.15.53:0/1295848057 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f83b0057050 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83c15a3740 10 client.8454 fetch_fsmap learned FSMap version 111 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55b2bde196b0 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f8396ffd640 1 -- 172.21.15.53:0/1295848057 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b0042070 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.075+0000 7f8396ffd640 1 -- 172.21.15.53:0/1295848057 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 111) ==== 3494+0+0 (secure 0 0 0) 0x7f83b005a020 con 0x55b2be0fb100 2024-05-10T10:43:17.083 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83c15a3740 10 client.8454 fetch_fsmap finished waiting for FSMap version 111 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83c15a3740 10 client.8454 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83c15a3740 20 client.8454 populate_metadata read hostname 'smithi053' 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be120310 0x55b2be1206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83c15a3740 4 client.8454 mds_command: new command op to 8352 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 --> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2bdfc1760 con 0x55b2be120310 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be120310 0x55b2be1206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be120310 0x55b2be1206f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ceaf5707784d984 server_cookie=ca768c04afee26f7 in_seq=0 out_seq=0 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.076+0000 7f8396ffd640 10 client.8454 ms_handle_connect on v2:172.21.15.53:6832/2292645283 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f8396ffd640 1 -- 172.21.15.53:0/1295848057 <== mds.0 v2:172.21.15.53:6832/2292645283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2bdfc1760 con 0x55b2be120310 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f8396ffd640 10 client.8454 handle_command_reply: tid=0 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 client.8454 shutdown 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 msgr2=0x55b2be0fb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fb580 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f83b002a6a0 tx=0x7f83b0004ab0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.084 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 shutdown_connections 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be120310 0x55b2be1206f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fb580 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be0fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f8396ffd640 0 client.8454 ms_handle_reset on v2:172.21.15.53:6832/2292645283 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1295848057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be0faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 >> 172.21.15.53:0/1295848057 conn(0x55b2be0d3bb0 msgr2=0x55b2be100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 shutdown_connections 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 1 -- 172.21.15.53:0/1295848057 wait complete. 2024-05-10T10:43:17.085 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.077+0000 7f83c15a3740 20 client.8454 trim_cache size 0 max 16384 2024-05-10T10:43:17.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 Processor -- start 2024-05-10T10:43:17.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 -- start start 2024-05-10T10:43:17.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be142630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be11e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8201ee0 con 0x55b2be0fb100 2024-05-10T10:43:17.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8200f50 con 0x55b2be100f50 2024-05-10T10:43:17.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.096+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f83b815acc0 con 0x55b2be101330 2024-05-10T10:43:17.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52426/0 (socket says 172.21.15.53:52426) 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 -- 172.21.15.53:0/1231318989 learned_addr learned my addr 172.21.15.53:0/1231318989 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be11e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83affff640 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 -- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 msgr2=0x55b2be11e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be11e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 -- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 msgr2=0x55b2be142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 -- 172.21.15.53:0/1231318989 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2be0fd050 con 0x55b2be0fb100 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83affff640 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fd800 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f83a80961c0 tx=0x7f83a8060950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4419c69ee017d1c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83ad7fa640 1 -- 172.21.15.53:0/1231318989 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a80652a0 con 0x55b2be0fb100 2024-05-10T10:43:17.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83ad7fa640 1 -- 172.21.15.53:0/1231318989 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a8069cc0 con 0x55b2be0fb100 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.097+0000 7f83ad7fa640 1 -- 172.21.15.53:0/1231318989 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8011ba0 con 0x55b2be0fb100 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 -- 172.21.15.53:0/1231318989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 msgr2=0x55b2be0fd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fd800 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f83a80961c0 tx=0x7f83a8060950 comp rx=0 tx=0).stop 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 -- 172.21.15.53:0/1231318989 shutdown_connections 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be101330 0x55b2be11e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be100f50 0x55b2be142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1231318989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be0fb100 0x55b2be0fd800 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 -- 172.21.15.53:0/1231318989 >> 172.21.15.53:0/1231318989 conn(0x55b2be0d3bb0 msgr2=0x55b2be1268e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 -- 172.21.15.53:0/1231318989 shutdown_connections 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 -- 172.21.15.53:0/1231318989 wait complete. 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 -- start start 2024-05-10T10:43:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.098+0000 7f83c15a3740 1 Processor -- start 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be0fb100 0x55b2be19c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be11b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83c15a3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be101330 0x55b2be11bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83c15a3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b2be0fe650 con 0x55b2be101330 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8200a10 con 0x55b2be0fb100 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83c15a3740 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x55b2be120ed0 con 0x55b2be100f50 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83c15a3740 10 client.0 _ll_register_callbacks cb 0x7f83b0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83bcd4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be101330 0x55b2be11bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83bcd4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be101330 0x55b2be11bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52432/0 (socket says 172.21.15.53:52432) 2024-05-10T10:43:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1750205215 learned_addr learned my addr 172.21.15.53:0/1750205215 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83affff640 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be11b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.099+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be0fb100 0x55b2be19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 msgr2=0x55b2be11b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be11b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be0fb100 msgr2=0x55b2be19c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be0fb100 0x55b2be19c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83bcd4c640 1 -- 172.21.15.53:0/1750205215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2be1319c0 con 0x55b2be101330 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83bcd4c640 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be101330 0x55b2be11bb90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f83b002a6a0 tx=0x7f83b0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5673c20cb9c41e09 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f8396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f8396ffd640 1 -- 172.21.15.53:0/1750205215 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b004a070 con 0x55b2be101330 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f8396ffd640 1 -- 172.21.15.53:0/1750205215 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f83b0004450 con 0x55b2be101330 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a8069cc0 con 0x55b2be101330 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f8396ffd640 1 -- 172.21.15.53:0/1750205215 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f83b0068050 con 0x55b2be101330 2024-05-10T10:43:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 10 client.8460 fetch_fsmap learned FSMap version 111 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55b2be11ecd0 con 0x55b2be101330 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f8396ffd640 1 -- 172.21.15.53:0/1750205215 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a8069cc0 con 0x55b2be101330 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f8396ffd640 1 -- 172.21.15.53:0/1750205215 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 111) ==== 3494+0+0 (secure 0 0 0) 0x7f83b006b020 con 0x55b2be101330 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 10 client.8460 fetch_fsmap finished waiting for FSMap version 111 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 10 client.8460 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 20 client.8460 populate_metadata read hostname 'smithi053' 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be0f7d20 0x55b2be0d7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 4 client.8460 mds_command: new command op to 8352 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 --> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] -- command(tid 0: {"prefix": "status"}) -- 0x55b2be04b4e0 con 0x55b2be0f7d20 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.100+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be0f7d20 0x55b2be0d7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f8396ffd640 10 client.8460 ms_handle_connect on v2:172.21.15.53:6832/2292645283 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83b7fff640 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be0f7d20 0x55b2be0d7230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf4236fa2b603b5 server_cookie=dfd055d04e273e2 in_seq=0 out_seq=0 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f8396ffd640 1 -- 172.21.15.53:0/1750205215 <== mds.0 v2:172.21.15.53:6832/2292645283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55b2be04b4e0 con 0x55b2be0f7d20 2024-05-10T10:43:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f8396ffd640 10 client.8460 handle_command_reply: tid=0 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 client.8460 shutdown 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be101330 msgr2=0x55b2be11bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be101330 0x55b2be11bb90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f83b002a6a0 tx=0x7f83b0009000 comp rx=0 tx=0).stop 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 shutdown_connections 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:6832/2292645283,v1:172.21.15.53:6834/2292645283] conn(0x55b2be0f7d20 0x55b2be0d7230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b2be101330 0x55b2be11bb90 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b2be100f50 0x55b2be11b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f8396ffd640 0 client.8460 ms_handle_reset on v2:172.21.15.53:6832/2292645283 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 --2- 172.21.15.53:0/1750205215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2be0fb100 0x55b2be19c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 >> 172.21.15.53:0/1750205215 conn(0x55b2be0d3bb0 msgr2=0x55b2be0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 shutdown_connections 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 1 -- 172.21.15.53:0/1750205215 wait complete. 2024-05-10T10:43:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.101+0000 7f83c15a3740 20 client.8460 trim_cache size 0 max 16384 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "543b513e-90c7-47ff-9cdd-f7fa4d978734", 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 8352, 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 4.485994657, 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 111, 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 120, 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 120, 2024-05-10T10:43:17.111 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 6.455771887 2024-05-10T10:43:17.112 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83980762d0 msgr2=0x7f8398078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83980762d0 0x7f8398078790 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f83b815b5a0 tx=0x7f83a0009040 comp rx=0 tx=0).stop 2024-05-10T10:43:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 msgr2=0x7f83b81f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b81f4b40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f83a800d830 tx=0x7f83a800dcf0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 shutdown_connections 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f83980762d0 0x7f8398078790 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83b815f8c0 0x7f83b8157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f83b815f4e0 0x7f83b8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 --2- 172.21.15.53:0/3529386320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83b806e620 0x7f83b81f4b40 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 >> 172.21.15.53:0/3529386320 conn(0x7f83b80d2df0 msgr2=0x7f83b80d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 shutdown_connections 2024-05-10T10:43:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.102+0000 7f83be7af640 1 -- 172.21.15.53:0/3529386320 wait complete. 2024-05-10T10:43:17.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T10:43:17.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.216+0000 7fcd87b14640 1 Processor -- start 2024-05-10T10:43:17.218 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.216+0000 7fcd87b14640 1 -- start start 2024-05-10T10:43:17.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.217+0000 7fcd87b14640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd8006ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.219 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.217+0000 7fcd87b14640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8015c840 con 0x7fcd8015feb0 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.217+0000 7fcd87b14640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8006c3f0 con 0x7fcd80072f60 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.217+0000 7fcd87b14640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8006d410 con 0x7fcd8015fad0 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.217+0000 7fcd850af640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcd8015feb0 0x7fcd80072850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42382/0 (socket says 172.21.15.53:42382) 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.217+0000 7fcd850af640 1 -- 172.21.15.53:0/2999559151 learned_addr learned my addr 172.21.15.53:0/2999559151 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.218+0000 7fcd858b0640 1 --2- 172.21.15.53:0/2999559151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd8006ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.218+0000 7fcd848ae640 1 -- 172.21.15.53:0/2999559151 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 119442216 0 0) 0x7fcd8015c840 con 0x7fcd8015feb0 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.218+0000 7fcd848ae640 1 -- 172.21.15.53:0/2999559151 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd68003440 con 0x7fcd8015feb0 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.218+0000 7fcd858b0640 1 -- 172.21.15.53:0/2999559151 >> v1:172.21.15.135:6789/0 conn(0x7fcd80072f60 legacy=0x7fcd8015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.218+0000 7fcd858b0640 1 -- 172.21.15.53:0/2999559151 >> v1:172.21.15.53:6789/0 conn(0x7fcd8015feb0 legacy=0x7fcd80072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.218+0000 7fcd848ae640 1 -- 172.21.15.53:0/2999559151 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778341321 0 0) 0x7fcd8006c3f0 con 0x7fcd80072f60 2024-05-10T10:43:17.220 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.218+0000 7fcd858b0640 1 -- 172.21.15.53:0/2999559151 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8006c6d0 con 0x7fcd8015fad0 2024-05-10T10:43:17.221 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd858b0640 1 --2- 172.21.15.53:0/2999559151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd8006ced0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009dc0 tx=0x7fcd7002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7728469c0dfa011 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd848ae640 1 -- 172.21.15.53:0/2999559151 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70007740 con 0x7fcd8015fad0 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd848ae640 1 -- 172.21.15.53:0/2999559151 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd700078e0 con 0x7fcd8015fad0 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd848ae640 1 -- 172.21.15.53:0/2999559151 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70007c90 con 0x7fcd8015fad0 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd87b14640 1 -- 172.21.15.53:0/2999559151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 msgr2=0x7fcd8006ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2999559151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd8006ced0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009dc0 tx=0x7fcd7002a5d0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd87b14640 1 -- 172.21.15.53:0/2999559151 shutdown_connections 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2999559151 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd8006ced0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.219+0000 7fcd87b14640 1 -- 172.21.15.53:0/2999559151 >> 172.21.15.53:0/2999559151 conn(0x7fcd800d1510 msgr2=0x7fcd800d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.220+0000 7fcd87b14640 1 -- 172.21.15.53:0/2999559151 shutdown_connections 2024-05-10T10:43:17.222 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.220+0000 7fcd87b14640 1 -- 172.21.15.53:0/2999559151 wait complete. 2024-05-10T10:43:17.228 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 Processor -- start 2024-05-10T10:43:17.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 -- start start 2024-05-10T10:43:17.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd80072f60 0x7fcd80145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd800b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd80155850 0x7fcd800b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8015c290 con 0x7fcd80155850 2024-05-10T10:43:17.229 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8015cb20 con 0x7fcd80072f60 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.221+0000 7fcd87b14640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8015c410 con 0x7fcd8015fad0 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd80155850 0x7fcd800b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd80155850 0x7fcd800b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52446/0 (socket says 172.21.15.53:52446) 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 -- 172.21.15.53:0/2482363995 learned_addr learned my addr 172.21.15.53:0/2482363995 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd858b0640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd80072f60 0x7fcd80145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd850af640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd800b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 -- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 msgr2=0x7fcd800b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd800b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 -- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd80072f60 msgr2=0x7fcd80145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd80072f60 0x7fcd80145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 -- 172.21.15.53:0/2482363995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd800b42b0 con 0x7fcd80155850 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd858b0640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd80072f60 0x7fcd80145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:17.230 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd860b1640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd80155850 0x7fcd800b1110 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c410 tx=0x7fcd7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd7e7fc640 1 -- 172.21.15.53:0/2482363995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78017070 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd7e7fc640 1 -- 172.21.15.53:0/2482363995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd780035f0 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd800b6cd0 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd7e7fc640 1 -- 172.21.15.53:0/2482363995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd780039e0 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.222+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd800b4830 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.223+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8006a220 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.224+0000 7fcd7e7fc640 1 -- 172.21.15.53:0/2482363995 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcd7801a070 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.225+0000 7fcd7e7fc640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcd6c0762d0 0x7fcd6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.225+0000 7fcd7e7fc640 1 -- 172.21.15.53:0/2482363995 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7fcd78096bf0 con 0x7fcd80155850 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.225+0000 7fcd858b0640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcd6c0762d0 0x7fcd6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:17.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.225+0000 7fcd858b0640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcd6c0762d0 0x7fcd6c078790 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcd70034000 tx=0x7fcd7002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:17.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.230+0000 7fcd7e7fc640 1 -- 172.21.15.53:0/2482363995 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fcd78060330 con 0x7fcd80155850 2024-05-10T10:43:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.462+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcd8006da70 con 0x7fcd80155850 2024-05-10T10:43:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.462+0000 7fcd7e7fc640 1 -- 172.21.15.53:0/2482363995 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) ==== 53+0+73 (secure 0 0 0) 0x7fcd7805fc30 con 0x7fcd80155850 2024-05-10T10:43:17.464 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcd6c0762d0 msgr2=0x7fcd6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcd6c0762d0 0x7fcd6c078790 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcd70034000 tx=0x7fcd7002f000 comp rx=0 tx=0).stop 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd80155850 msgr2=0x7fcd800b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd80155850 0x7fcd800b1110 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c410 tx=0x7fcd7800c8d0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 shutdown_connections 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fcd6c0762d0 0x7fcd6c078790 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd80155850 0x7fcd800b1110 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd8015fad0 0x7fcd800b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 --2- 172.21.15.53:0/2482363995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd80072f60 0x7fcd80145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 >> 172.21.15.53:0/2482363995 conn(0x7fcd800d1510 msgr2=0x7fcd800d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 shutdown_connections 2024-05-10T10:43:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17.465+0000 7fcd87b14640 1 -- 172.21.15.53:0/2482363995 wait complete. 2024-05-10T10:43:17.477 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T10:43:17.477 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T10:43:17.478 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-05-10T10:43:17.478 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T10:43:17.478 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:17.478 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T10:43:17.478 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T10:43:17.478 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T10:43:17.478 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:43:17.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:17.478 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-05-10T10:43:17.495 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T10:43:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T10:43:17.495 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-05-10T10:43:17.496 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42172sec preferred_lft 42172sec 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::682f:98ff:feb0:7fa1/64 scope link 2024-05-10T10:43:17.497 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 7e:41:b4:9d:14:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::7c41:b4ff:fe9d:14c5/64 scope link 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 9e:d5:8e:ae:d2:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::9cd5:8eff:feae:d27a/64 scope link 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-05-10T10:43:17.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:17.499 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T10:43:17.559 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:43:17.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:17.561 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-05-10T10:43:17.618 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-05-10T10:43:17.618 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T10:43:17.618 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T10:43:17.618 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T10:43:17.618 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T10:43:17.618 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-10T10:43:17.619 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-05-10T10:43:17.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T10:43:17.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T10:43:17.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T10:43:17.619 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-05-10T10:43:17.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-10T10:43:17.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:17.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:17.749 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:43:17.765 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:43:17.765 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:43:17.765 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:43:17.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:17.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-05-10T10:43:17.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:43:17.831 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:17.887 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T10:43:17.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T10:43:17.888 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:17.943 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T10:43:17.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:17.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:18.028 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:18.028 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:43:18.028 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:43:18.037 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:18.037 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:43:18.037 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:43:18.037 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:43:18.037 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.135:3300,172.21.15.135:3301" 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790" 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.53:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2024-05-10T10:43:18.043 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=543b513e-90c7-47ff-9cdd-f7fa4d978734" will pass to kernel 2024-05-10T10:43:18.044 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T10:43:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-05-10T10:43:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T10:43:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T10:43:18.044 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T10:43:18.045 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T10:43:18.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:18.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:18.071 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T10:43:18.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-10T10:43:18.163 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:18.169 INFO:teuthology.orchestra.run.smithi053.stdout:client7537 2024-05-10T10:43:18.169 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:18.169 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:18.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:18.186 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./ 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./parent 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/charlie 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/alpha 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/flushed/bravo 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/unflushed 2024-05-10T10:43:18.246 INFO:teuthology.orchestra.run.smithi053.stdout:./parent/unflushed/jfile 2024-05-10T10:43:18.248 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:18.248 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:18.248 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:18.248 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:18.248 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-10T10:43:18.248 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:18.355 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-05-10T10:43:18.358 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:18.364 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:18.365 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:18.365 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:18.365 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:18.365 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-10T10:43:18.365 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:18.473 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627776 2024-05-10T10:43:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:18.482 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:18.482 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:18.482 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:18.482 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:18.482 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-10T10:43:18.482 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:18.589 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627777 2024-05-10T10:43:18.592 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:18.599 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:18.599 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:18.599 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:18.599 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:18.599 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-10T10:43:18.599 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628280 2024-05-10T10:43:18.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:18.716 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:18.716 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:18.716 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:18.716 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:18.716 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-10T10:43:18.716 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:18.824 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627778 2024-05-10T10:43:18.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:18.833 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:18.833 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:18.833 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:18.834 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:18.834 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-10T10:43:18.834 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:18.941 INFO:teuthology.orchestra.run.smithi053.stdout:1099511628279 2024-05-10T10:43:18.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:18.951 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:18.951 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:18.951 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:18.951 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:18.951 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-10T10:43:18.951 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:19.059 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627779 2024-05-10T10:43:19.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:19.068 DEBUG:teuthology.orchestra.run.smithi053:> stdin-killer -- python3 -c ' 2024-05-10T10:43:19.068 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-05-10T10:43:19.068 DEBUG:teuthology.orchestra.run.smithi053:> import stat 2024-05-10T10:43:19.068 DEBUG:teuthology.orchestra.run.smithi053:> 2024-05-10T10:43:19.068 DEBUG:teuthology.orchestra.run.smithi053:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-10T10:43:19.068 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-05-10T10:43:19.176 INFO:teuthology.orchestra.run.smithi053.stdout:1099511627780 2024-05-10T10:43:19.178 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-10T10:43:19.185 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T10:43:19.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:43:19.272 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T10:43:19.315 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T10:43:19.315 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:43:19.315 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:19.315 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:19.331 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:19.332 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:19.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T10:43:19.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.533+0000 7fc4ba171640 1 Processor -- start 2024-05-10T10:43:19.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.533+0000 7fc4ba171640 1 -- start start 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4ba171640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 0x7fc4b415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4ba171640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b4160190 con 0x7fc4b406c990 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4ba171640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b4160470 con 0x7fc4b415f620 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4ba171640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b406cd70 con 0x7fc4b4070bd0 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4b37fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc4b406c990 0x7fc4b4163660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42384/0 (socket says 172.21.15.53:42384) 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4b37fe640 1 -- 172.21.15.53:0/2606676454 learned_addr learned my addr 172.21.15.53:0/2606676454 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4b3fff640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 0x7fc4b415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.538 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577750584 0 0) 0x7fc4b4160190 con 0x7fc4b406c990 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.534+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc49c003520 con 0x7fc4b406c990 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1491709162 0 0) 0x7fc49c003520 con 0x7fc4b406c990 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 msgr2=0x7fc4b415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 0x7fc4b415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 >> v1:172.21.15.135:6789/0 conn(0x7fc4b415f620 legacy=0x7fc4b40704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b4162e60 con 0x7fc4b406c990 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fc4a4001690 con 0x7fc4b406c990 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 >> v1:172.21.15.53:6789/0 conn(0x7fc4b406c990 legacy=0x7fc4b4163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc49c0039e0 0x7fc49c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 0x7fc49c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc49c004d40 0x7fc49c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.539 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b406cd70 con 0x7fc49c0039e0 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c00b9f0 con 0x7fc49c004d40 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c00bbf0 con 0x7fc4b4070bd0 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b37fe640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc49c0039e0 0x7fc49c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b3fff640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc49c004d40 0x7fc49c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b2ffd640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 0x7fc49c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b37fe640 1 -- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 msgr2=0x7fc49c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b37fe640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 0x7fc49c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b37fe640 1 -- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc49c004d40 msgr2=0x7fc49c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b37fe640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc49c004d40 0x7fc49c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b37fe640 1 -- 172.21.15.53:0/2606676454 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c003520 con 0x7fc49c0039e0 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b2ffd640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4b4070bd0 0x7fc49c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.535+0000 7fc4b3fff640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc49c004d40 0x7fc49c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.536+0000 7fc4b37fe640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc49c0039e0 0x7fc49c003dc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40d1290 tx=0x7fc49c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5bfbd4f0769053 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.536+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4007ce0 con 0x7fc49c0039e0 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.536+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a4005ad0 con 0x7fc49c0039e0 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.536+0000 7fc4b27fc640 1 -- 172.21.15.53:0/2606676454 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4007070 con 0x7fc49c0039e0 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.537+0000 7fc4ba171640 1 -- 172.21.15.53:0/2606676454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc49c0039e0 msgr2=0x7fc49c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.537+0000 7fc4ba171640 1 --2- 172.21.15.53:0/2606676454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc49c0039e0 0x7fc49c003dc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40d1290 tx=0x7fc49c005830 comp rx=0 tx=0).stop 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.537+0000 7fc4b3fff640 1 -- 172.21.15.53:0/2606676454 reap_dead start 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.537+0000 7fc4ba171640 1 -- 172.21.15.53:0/2606676454 shutdown_connections 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.537+0000 7fc4ba171640 1 -- 172.21.15.53:0/2606676454 >> 172.21.15.53:0/2606676454 conn(0x7fc4b40b3980 msgr2=0x7fc4b40b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.537+0000 7fc4ba171640 1 -- 172.21.15.53:0/2606676454 shutdown_connections 2024-05-10T10:43:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.537+0000 7fc4ba171640 1 -- 172.21.15.53:0/2606676454 wait complete. 2024-05-10T10:43:19.544 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.538+0000 7fc4ba171640 1 Processor -- start 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.538+0000 7fc4ba171640 1 -- start start 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4ba171640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc49c0039e0 0x7fc4b4149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4ba171640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b41f3d00 0x7fc4b41f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4ba171640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b41f46a0 0x7fc4b414bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4ba171640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b415fa80 con 0x7fc4b41f3d00 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4ba171640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4073e70 con 0x7fc4b41f46a0 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4ba171640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b415fc00 con 0x7fc49c0039e0 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b41f3d00 0x7fc4b41f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b41f3d00 0x7fc4b41f4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52472/0 (socket says 172.21.15.53:52472) 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 -- 172.21.15.53:0/3470464029 learned_addr learned my addr 172.21.15.53:0/3470464029 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:19.545 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b37fe640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc49c0039e0 0x7fc4b4149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b3fff640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b41f46a0 0x7fc4b414bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 -- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc49c0039e0 msgr2=0x7fc4b4149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc49c0039e0 0x7fc4b4149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 -- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b41f46a0 msgr2=0x7fc4b414bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b41f46a0 0x7fc4b414bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 -- 172.21.15.53:0/3470464029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b40cfbf0 con 0x7fc4b41f3d00 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.539+0000 7fc4b2ffd640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b41f3d00 0x7fc4b41f4160 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800e830 tx=0x7fc4a800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.540+0000 7fc49bfff640 1 -- 172.21.15.53:0/3470464029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80052e0 con 0x7fc4b41f3d00 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.540+0000 7fc49bfff640 1 -- 172.21.15.53:0/3470464029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a8007020 con 0x7fc4b41f3d00 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.540+0000 7fc49bfff640 1 -- 172.21.15.53:0/3470464029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8007210 con 0x7fc4b41f3d00 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.540+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b40cfed0 con 0x7fc4b41f3d00 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.540+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b414fe50 con 0x7fc4b41f3d00 2024-05-10T10:43:19.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.541+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b406a220 con 0x7fc4b41f3d00 2024-05-10T10:43:19.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.541+0000 7fc49bfff640 1 -- 172.21.15.53:0/3470464029 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4a801a020 con 0x7fc4b41f3d00 2024-05-10T10:43:19.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.542+0000 7fc49bfff640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4a00762d0 0x7fc4a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:19.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.542+0000 7fc49bfff640 1 -- 172.21.15.53:0/3470464029 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7fc4a80970e0 con 0x7fc4b41f3d00 2024-05-10T10:43:19.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.542+0000 7fc4b37fe640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4a00762d0 0x7fc4a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:19.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.543+0000 7fc4b37fe640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4a00762d0 0x7fc4a0078790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4032000 tx=0x7fc4a4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:19.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.546+0000 7fc49bfff640 1 -- 172.21.15.53:0/3470464029 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc4a80607f0 con 0x7fc4b41f3d00 2024-05-10T10:43:19.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.750+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4b4073540 con 0x7fc4b41f3d00 2024-05-10T10:43:19.751 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.751+0000 7fc49bfff640 1 -- 172.21.15.53:0/3470464029 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v120) ==== 80+0+240 (secure 0 0 0) 0x7fc4a80600f0 con 0x7fc4b41f3d00 2024-05-10T10:43:19.751 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6834/38461894 2024-05-11T10:43:10.604857+0000 2024-05-10T10:43:19.752 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53:6832/38461894 2024-05-11T10:43:10.604857+0000 2024-05-10T10:43:19.752 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.135:6837/776479349 2024-05-11T10:43:01.305186+0000 2024-05-10T10:43:19.752 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.135:6835/776479349 2024-05-11T10:43:01.305186+0000 2024-05-10T10:43:19.752 INFO:teuthology.orchestra.run.smithi053.stderr:listed 4 entries 2024-05-10T10:43:19.753 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4a00762d0 msgr2=0x7fc4a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4a00762d0 0x7fc4a0078790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4032000 tx=0x7fc4a4009030 comp rx=0 tx=0).stop 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b41f3d00 msgr2=0x7fc4b41f4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b41f3d00 0x7fc4b41f4160 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800e830 tx=0x7fc4a800ecf0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 shutdown_connections 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc4a00762d0 0x7fc4a0078790 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4b41f46a0 0x7fc4b414bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b41f3d00 0x7fc4b41f4160 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 --2- 172.21.15.53:0/3470464029 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc49c0039e0 0x7fc4b4149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 >> 172.21.15.53:0/3470464029 conn(0x7fc4b40b3980 msgr2=0x7fc4b4070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 shutdown_connections 2024-05-10T10:43:19.754 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:19.753+0000 7fc4ba171640 1 -- 172.21.15.53:0/3470464029 wait complete. 2024-05-10T10:43:19.764 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T10:43:19.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:19.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T10:43:19.794 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:19.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:19.794 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T10:43:19.851 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:43:19.851 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:43:19.851 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:19.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:43:19.872 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:43:19.872 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:19.872 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:19.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:43:19.907 INFO:tasks.cephfs.mount:Terminating background process 2024-05-10T10:43:19.908 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:43:19.908 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:19.931 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:43:19.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:20.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.050+0000 7f355cf90640 1 Processor -- start 2024-05-10T10:43:20.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.050+0000 7f355cf90640 1 -- start start 2024-05-10T10:43:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.050+0000 7f355cf90640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f35580d5d30 0x7f355816a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.050+0000 7f355cf90640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580d30b0 con 0x7f355815f3b0 2024-05-10T10:43:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.050+0000 7f355cf90640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580d3390 con 0x7f355815f790 2024-05-10T10:43:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.050+0000 7f355cf90640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f35580c8c80 con 0x7f35580d5d30 2024-05-10T10:43:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f355659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f355815f3b0 0x7f35580d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42390/0 (socket says 172.21.15.53:42390) 2024-05-10T10:43:20.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f355659c640 1 -- 172.21.15.53:0/72657212 learned_addr learned my addr 172.21.15.53:0/72657212 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3556d9d640 1 --2- 172.21.15.53:0/72657212 >> v2:172.21.15.135:3300/0 conn(0x7f35580d5d30 0x7f355816a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137105494 0 0) 0x7f35580d30b0 con 0x7f355815f3b0 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3544003520 con 0x7f355815f3b0 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 81356702 0 0) 0x7f3544003520 con 0x7f355815f3b0 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 >> v2:172.21.15.135:3300/0 conn(0x7f35580d5d30 msgr2=0x7f355816a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 --2- 172.21.15.53:0/72657212 >> v2:172.21.15.135:3300/0 conn(0x7f35580d5d30 0x7f355816a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 >> v1:172.21.15.135:6789/0 conn(0x7f355815f790 legacy=0x7f35580cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580c8f60 con 0x7f355815f3b0 2024-05-10T10:43:20.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f3548001670 con 0x7f355815f3b0 2024-05-10T10:43:20.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 >> v1:172.21.15.53:6789/0 conn(0x7f355815f3b0 legacy=0x7f35580d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35440039b0 0x7f3544003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544004800 0x7f3544004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35440051c0 0x7f354400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35580c8c80 con 0x7f3544004800 2024-05-10T10:43:20.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f354400c060 con 0x7f35440039b0 2024-05-10T10:43:20.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f354400c260 con 0x7f35440051c0 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f354ffff640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544004800 0x7f3544004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f355659c640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35440039b0 0x7f3544003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.051+0000 7f3556d9d640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35440051c0 0x7f354400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f354ffff640 1 -- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35440051c0 msgr2=0x7f354400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f354ffff640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35440051c0 0x7f354400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f354ffff640 1 -- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35440039b0 msgr2=0x7f3544003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f354ffff640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35440039b0 0x7f3544003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f354ffff640 1 -- 172.21.15.53:0/72657212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3544003520 con 0x7f3544004800 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f3556d9d640 1 -- 172.21.15.53:0/72657212 reap_dead start 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f354ffff640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544004800 0x7f3544004c80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3540001720 tx=0x7f3544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f95ef08dae4d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540001370 con 0x7f3544004800 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35400052d0 con 0x7f3544004800 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.052+0000 7f3555d9b640 1 -- 172.21.15.53:0/72657212 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354002abc0 con 0x7f3544004800 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.053+0000 7f355cf90640 1 -- 172.21.15.53:0/72657212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544004800 msgr2=0x7f3544004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.053+0000 7f355cf90640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544004800 0x7f3544004c80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3540001720 tx=0x7f3544005830 comp rx=0 tx=0).stop 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.053+0000 7f355cf90640 1 -- 172.21.15.53:0/72657212 shutdown_connections 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.053+0000 7f355cf90640 1 --2- 172.21.15.53:0/72657212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544004800 0x7f3544004c80 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.053+0000 7f355cf90640 1 -- 172.21.15.53:0/72657212 >> 172.21.15.53:0/72657212 conn(0x7f35580d5130 msgr2=0x7f35580d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.053+0000 7f355cf90640 1 -- 172.21.15.53:0/72657212 shutdown_connections 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.053+0000 7f355cf90640 1 -- 172.21.15.53:0/72657212 wait complete. 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.054+0000 7f355cf90640 1 Processor -- start 2024-05-10T10:43:20.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.054+0000 7f355cf90640 1 -- start start 2024-05-10T10:43:20.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f355cf90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3544004800 0x7f35580ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f355cf90640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35580cc470 0x7f35581f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f355cf90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35580cc8f0 0x7f35581f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f355cf90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35580d3870 con 0x7f35580cc8f0 2024-05-10T10:43:20.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f355cf90640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35580d36f0 con 0x7f3544004800 2024-05-10T10:43:20.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f355cf90640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f35580d39f0 con 0x7f35580cc470 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f3556d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35580cc8f0 0x7f35581f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f3556d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35580cc8f0 0x7f35581f8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52494/0 (socket says 172.21.15.53:52494) 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f3556d9d640 1 -- 172.21.15.53:0/2945245612 learned_addr learned my addr 172.21.15.53:0/2945245612 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f354ffff640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35580cc470 0x7f35581f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.055+0000 7f355659c640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3544004800 0x7f35580ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f3556d9d640 1 -- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35580cc470 msgr2=0x7f35581f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f3556d9d640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35580cc470 0x7f35581f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f3556d9d640 1 -- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3544004800 msgr2=0x7f35580ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f3556d9d640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3544004800 0x7f35580ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f3556d9d640 1 -- 172.21.15.53:0/2945245612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580cf250 con 0x7f35580cc8f0 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f3556d9d640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35580cc8f0 0x7f35581f8ab0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f35580d36a0 tx=0x7f35500115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f354effd640 1 -- 172.21.15.53:0/2945245612 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550011e10 con 0x7f35580cc8f0 2024-05-10T10:43:20.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f354effd640 1 -- 172.21.15.53:0/2945245612 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355001b070 con 0x7f35580cc8f0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f354effd640 1 -- 172.21.15.53:0/2945245612 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550003990 con 0x7f35580cc8f0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35580ce9b0 con 0x7f35580cc8f0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.056+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35580cef70 con 0x7f35580cc8f0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.057+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355806a220 con 0x7f35580cc8f0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.058+0000 7f354effd640 1 -- 172.21.15.53:0/2945245612 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3550005d10 con 0x7f35580cc8f0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.058+0000 7f354effd640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3538076300 0x7f35380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.058+0000 7f354effd640 1 -- 172.21.15.53:0/2945245612 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f3550097a50 con 0x7f35580cc8f0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.059+0000 7f355659c640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3538076300 0x7f35380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.059+0000 7f355659c640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3538076300 0x7f35380787c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3548001050 tx=0x7f35480078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.063+0000 7f354effd640 1 -- 172.21.15.53:0/2945245612 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3550060bb0 con 0x7f35580cc8f0 2024-05-10T10:43:20.293 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.293+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f355815e0e0 con 0x7f35580cc8f0 2024-05-10T10:43:20.294 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.293+0000 7f354effd640 1 -- 172.21.15.53:0/2945245612 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5186 (secure 0 0 0) 0x7f3550064eb0 con 0x7f35580cc8f0 2024-05-10T10:43:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:20.294 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:15.622565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8352},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8352":{"gid":8352,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/2292645283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2292645283},{"type":"v1","addr":"172.21.15.53:6834","nonce":2292645283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8352,"qdb_cluster":[8352]},"id":8}]} 2024-05-10T10:43:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 111 2024-05-10T10:43:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3538076300 msgr2=0x7f35380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3538076300 0x7f35380787c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3548001050 tx=0x7f35480078f0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35580cc8f0 msgr2=0x7f35581f8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35580cc8f0 0x7f35581f8ab0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f35580d36a0 tx=0x7f35500115a0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 shutdown_connections 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3538076300 0x7f35380787c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35580cc8f0 0x7f35581f8ab0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35580cc470 0x7f35581f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 --2- 172.21.15.53:0/2945245612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3544004800 0x7f35580ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 >> 172.21.15.53:0/2945245612 conn(0x7f35580d5130 msgr2=0x7f35580cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 shutdown_connections 2024-05-10T10:43:20.300 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.295+0000 7f355cf90640 1 -- 172.21.15.53:0/2945245612 wait complete. 2024-05-10T10:43:20.307 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:20.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.410+0000 7fc8b5d9d640 1 Processor -- start 2024-05-10T10:43:20.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.410+0000 7fc8b5d9d640 1 -- start start 2024-05-10T10:43:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8b5d9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 0x7fc8b006e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8b5d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc8b00d0230 0x7fc8b00cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8b5d9d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b015fc90 con 0x7fc8b00cfbb0 2024-05-10T10:43:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8b5d9d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b006ee50 con 0x7fc8b00d0230 2024-05-10T10:43:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8b5d9d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b00b8c40 con 0x7fc8b00b8860 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8affff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc8b00cfbb0 0x7fc8b00d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42400/0 (socket says 172.21.15.53:42400) 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8affff640 1 -- 172.21.15.53:0/3850940028 learned_addr learned my addr 172.21.15.53:0/3850940028 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 0x7fc8b006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8aeffd640 1 --2- 172.21.15.53:0/3850940028 >> v2:172.21.15.135:3300/0 conn(0x7fc8b00d0230 0x7fc8b00cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502116085 0 0) 0x7fc8b015fc90 con 0x7fc8b00cfbb0 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc898003520 con 0x7fc8b00cfbb0 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3356986462 0 0) 0x7fc898003520 con 0x7fc8b00cfbb0 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 msgr2=0x7fc8b006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 0x7fc8b006e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 >> v2:172.21.15.135:3300/0 conn(0x7fc8b00d0230 msgr2=0x7fc8b00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 --2- 172.21.15.53:0/3850940028 >> v2:172.21.15.135:3300/0 conn(0x7fc8b00d0230 0x7fc8b00cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b00cd570 con 0x7fc8b00cfbb0 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7fc8a80019f0 con 0x7fc8b00cfbb0 2024-05-10T10:43:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 >> v1:172.21.15.53:6789/0 conn(0x7fc8b00cfbb0 legacy=0x7fc8b00d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8980039b0 0x7fc898003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 0x7fc898004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898004d40 0x7fc8980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b006ee50 con 0x7fc8980039b0 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b00b8c40 con 0x7fc898004d40 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.411+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc89800b9c0 con 0x7fc8b00b8860 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8980039b0 0x7fc898003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8affff640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898004d40 0x7fc8980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8aeffd640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 0x7fc898004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8af7fe640 1 -- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 msgr2=0x7fc898004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8b00b8860 0x7fc898004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8af7fe640 1 -- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898004d40 msgr2=0x7fc8980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc898004d40 0x7fc8980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.417 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8af7fe640 1 -- 172.21.15.53:0/3850940028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc898003520 con 0x7fc8980039b0 2024-05-10T10:43:20.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.412+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8980039b0 0x7fc898003dd0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009ac0 tx=0x7fc898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee58a77b4358212c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0036070 con 0x7fc8980039b0 2024-05-10T10:43:20.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a0031400 con 0x7fc8980039b0 2024-05-10T10:43:20.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8ae7fc640 1 -- 172.21.15.53:0/3850940028 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0009690 con 0x7fc8980039b0 2024-05-10T10:43:20.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/3850940028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8980039b0 msgr2=0x7fc898003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.418 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8b5d9d640 1 --2- 172.21.15.53:0/3850940028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8980039b0 0x7fc898003dd0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009ac0 tx=0x7fc898005830 comp rx=0 tx=0).stop 2024-05-10T10:43:20.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8affff640 1 -- 172.21.15.53:0/3850940028 reap_dead start 2024-05-10T10:43:20.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/3850940028 shutdown_connections 2024-05-10T10:43:20.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/3850940028 >> 172.21.15.53:0/3850940028 conn(0x7fc8b00d2df0 msgr2=0x7fc8b00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:20.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/3850940028 shutdown_connections 2024-05-10T10:43:20.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.413+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/3850940028 wait complete. 2024-05-10T10:43:20.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.414+0000 7fc8b5d9d640 1 Processor -- start 2024-05-10T10:43:20.419 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.415+0000 7fc8b5d9d640 1 -- start start 2024-05-10T10:43:20.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.415+0000 7fc8b5d9d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8980039b0 0x7fc8b0161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.415+0000 7fc8b5d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b015fae0 0x7fc8b0152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.415+0000 7fc8b5d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b00caff0 0x7fc8b00cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.415+0000 7fc8b5d9d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b00cde10 con 0x7fc8b00caff0 2024-05-10T10:43:20.422 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.415+0000 7fc8b5d9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b00cdb30 con 0x7fc8b015fae0 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.415+0000 7fc8b5d9d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b00b5c40 con 0x7fc8980039b0 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b00caff0 0x7fc8b00cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b00caff0 0x7fc8b00cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52508/0 (socket says 172.21.15.53:52508) 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 -- 172.21.15.53:0/1215148697 learned_addr learned my addr 172.21.15.53:0/1215148697 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8aeffd640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b015fae0 0x7fc8b0152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8980039b0 0x7fc8b0161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 -- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8980039b0 msgr2=0x7fc8b0161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8980039b0 0x7fc8b0161140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 -- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b015fae0 msgr2=0x7fc8b0152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b015fae0 0x7fc8b0152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 -- 172.21.15.53:0/1215148697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b01600a0 con 0x7fc8b00caff0 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8affff640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b00caff0 0x7fc8b00cbbb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc8b006ff80 tx=0x7fc8a800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.423 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc897fff640 1 -- 172.21.15.53:0/1215148697 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800ee20 con 0x7fc8b00caff0 2024-05-10T10:43:20.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc897fff640 1 -- 172.21.15.53:0/1215148697 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a8018070 con 0x7fc8b00caff0 2024-05-10T10:43:20.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc897fff640 1 -- 172.21.15.53:0/1215148697 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8013660 con 0x7fc8b00caff0 2024-05-10T10:43:20.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.416+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b0160380 con 0x7fc8b00caff0 2024-05-10T10:43:20.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.417+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b0160940 con 0x7fc8b00caff0 2024-05-10T10:43:20.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.418+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b006a220 con 0x7fc8b00caff0 2024-05-10T10:43:20.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.418+0000 7fc897fff640 1 -- 172.21.15.53:0/1215148697 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc8a8013800 con 0x7fc8b00caff0 2024-05-10T10:43:20.424 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.418+0000 7fc897fff640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc89c0762d0 0x7fc89c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.418+0000 7fc897fff640 1 -- 172.21.15.53:0/1215148697 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7fc8a8097cf0 con 0x7fc8b00caff0 2024-05-10T10:43:20.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.419+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc89c0762d0 0x7fc89c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.419+0000 7fc8af7fe640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc89c0762d0 0x7fc89c078790 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0033000 tx=0x7fc8a0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.425 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.423+0000 7fc897fff640 1 -- 172.21.15.53:0/1215148697 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fc8a809c070 con 0x7fc8b00caff0 2024-05-10T10:43:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.652+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8b00693f0 con 0x7fc8b00caff0 2024-05-10T10:43:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.653+0000 7fc897fff640 1 -- 172.21.15.53:0/1215148697 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5186 (secure 0 0 0) 0x7fc8a8061410 con 0x7fc8b00caff0 2024-05-10T10:43:20.654 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:20.654 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:15.622565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8352},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8352":{"gid":8352,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/2292645283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2292645283},{"type":"v1","addr":"172.21.15.53:6834","nonce":2292645283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8352,"qdb_cluster":[8352]},"id":8}]} 2024-05-10T10:43:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 111 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc89c0762d0 msgr2=0x7fc89c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc89c0762d0 0x7fc89c078790 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0033000 tx=0x7fc8a0008030 comp rx=0 tx=0).stop 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b00caff0 msgr2=0x7fc8b00cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b00caff0 0x7fc8b00cbbb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc8b006ff80 tx=0x7fc8a800e530 comp rx=0 tx=0).stop 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 shutdown_connections 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fc89c0762d0 0x7fc89c078790 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b00caff0 0x7fc8b00cbbb0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b015fae0 0x7fc8b0152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 --2- 172.21.15.53:0/1215148697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8980039b0 0x7fc8b0161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 >> 172.21.15.53:0/1215148697 conn(0x7fc8b00d2df0 msgr2=0x7fc8b00d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 shutdown_connections 2024-05-10T10:43:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.655+0000 7fc8b5d9d640 1 -- 172.21.15.53:0/1215148697 wait complete. 2024-05-10T10:43:20.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:43:20.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.771+0000 7f8ed8591640 1 Processor -- start 2024-05-10T10:43:20.771 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.771+0000 7f8ed8591640 1 -- start start 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed8591640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8ed015ce60 0x7f8ed00cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed8591640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed01611e0 0x7f8ed015d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed8591640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed00c6a60 con 0x7f8ed00c65f0 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed8591640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed00cd090 con 0x7f8ed015ce60 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed8591640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed00cd370 con 0x7f8ed01611e0 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed632d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8ed015ce60 0x7f8ed00cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed6b2e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed01611e0 0x7f8ed015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed5b2c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8ed00c65f0 0x7f8ed0160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:56980/0 (socket says 172.21.15.53:56980) 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed6b2e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed01611e0 0x7f8ed015d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:53872/0 (socket says 172.21.15.53:53872) 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed632d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8ed015ce60 0x7f8ed00cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.53:37984/0 (socket says 172.21.15.53:37984) 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed5b2c640 1 -- 172.21.15.53:0/1072570816 learned_addr learned my addr 172.21.15.53:0/1072570816 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:20.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed632d640 1 -- 172.21.15.53:0/1072570816 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed01611e0 msgr2=0x7f8ed015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:20.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed632d640 1 --2- 172.21.15.53:0/1072570816 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed01611e0 0x7f8ed015d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed632d640 1 -- 172.21.15.53:0/1072570816 >> v1:172.21.15.135:6789/0 conn(0x7f8ed00c65f0 legacy=0x7f8ed0160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed532b640 1 -- 172.21.15.53:0/1072570816 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827425416 0 0) 0x7f8ed00c6a60 con 0x7f8ed00c65f0 2024-05-10T10:43:20.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.772+0000 7f8ed632d640 1 -- 172.21.15.53:0/1072570816 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed00cd650 con 0x7f8ed015ce60 2024-05-10T10:43:20.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.773+0000 7f8ed632d640 1 --2- 172.21.15.53:0/1072570816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed00cb300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0097a0 tx=0x7f8ecc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c21f959e495c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.773+0000 7f8ed532b640 1 -- 172.21.15.53:0/1072570816 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc037070 con 0x7f8ed015ce60 2024-05-10T10:43:20.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.773+0000 7f8ed532b640 1 -- 172.21.15.53:0/1072570816 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc02ae50 con 0x7f8ed015ce60 2024-05-10T10:43:20.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.773+0000 7f8ed532b640 1 -- 172.21.15.53:0/1072570816 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc032660 con 0x7f8ed015ce60 2024-05-10T10:43:20.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 -- 172.21.15.53:0/1072570816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 msgr2=0x7f8ed00cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 --2- 172.21.15.53:0/1072570816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed00cb300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0097a0 tx=0x7f8ecc02a100 comp rx=0 tx=0).stop 2024-05-10T10:43:20.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 -- 172.21.15.53:0/1072570816 shutdown_connections 2024-05-10T10:43:20.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 --2- 172.21.15.53:0/1072570816 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed01611e0 0x7f8ed015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 --2- 172.21.15.53:0/1072570816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed00cb300 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 -- 172.21.15.53:0/1072570816 >> 172.21.15.53:0/1072570816 conn(0x7f8ed00b2900 msgr2=0x7f8ed00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:20.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 -- 172.21.15.53:0/1072570816 shutdown_connections 2024-05-10T10:43:20.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.774+0000 7f8ed8591640 1 -- 172.21.15.53:0/1072570816 wait complete. 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.775+0000 7f8ed8591640 1 Processor -- start 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.775+0000 7f8ed8591640 1 -- start start 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed8591640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed00c65f0 0x7f8ed0152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed8591640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed0073280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed8591640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed01611e0 0x7f8ed00737c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed8591640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00cde50 con 0x7f8ed01611e0 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed8591640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00ccd60 con 0x7f8ed015ce60 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed8591640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed00cdfd0 con 0x7f8ed00c65f0 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed6b2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed01611e0 0x7f8ed00737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed6b2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed01611e0 0x7f8ed00737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52518/0 (socket says 172.21.15.53:52518) 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed6b2e640 1 -- 172.21.15.53:0/3102290278 learned_addr learned my addr 172.21.15.53:0/3102290278 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed5b2c640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed0073280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.776+0000 7f8ed632d640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed00c65f0 0x7f8ed0152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed6b2e640 1 -- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed00c65f0 msgr2=0x7f8ed0152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed6b2e640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed00c65f0 0x7f8ed0152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed6b2e640 1 -- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 msgr2=0x7f8ed0073280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed6b2e640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed0073280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed6b2e640 1 -- 172.21.15.53:0/3102290278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed009c2f0 con 0x7f8ed01611e0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed5b2c640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed0073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed6b2e640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed01611e0 0x7f8ed00737c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800efc0 tx=0x7f8ec800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ec6ffd640 1 -- 172.21.15.53:0/3102290278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8018020 con 0x7f8ed01611e0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ec6ffd640 1 -- 172.21.15.53:0/3102290278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec8014480 con 0x7f8ed01611e0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed009c5d0 con 0x7f8ed01611e0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ec6ffd640 1 -- 172.21.15.53:0/3102290278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8014870 con 0x7f8ed01611e0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.777+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed009bfe0 con 0x7f8ed01611e0 2024-05-10T10:43:20.780 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.778+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed006a220 con 0x7f8ed01611e0 2024-05-10T10:43:20.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.779+0000 7f8ec6ffd640 1 -- 172.21.15.53:0/3102290278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ec8005d10 con 0x7f8ed01611e0 2024-05-10T10:43:20.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.779+0000 7f8ec6ffd640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8eb40762d0 0x7f8eb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:20.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.779+0000 7f8ec6ffd640 1 -- 172.21.15.53:0/3102290278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f8ec80980d0 con 0x7f8ed01611e0 2024-05-10T10:43:20.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.779+0000 7f8ed632d640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8eb40762d0 0x7f8eb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:20.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.780+0000 7f8ed632d640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8eb40762d0 0x7f8eb4078790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc034000 tx=0x7f8ecc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:20.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.783+0000 7f8ec6ffd640 1 -- 172.21.15.53:0/3102290278 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8ec8022070 con 0x7f8ed01611e0 2024-05-10T10:43:20.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.978+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ed00d1a30 con 0x7f8ed01611e0 2024-05-10T10:43:20.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.979+0000 7f8ec6ffd640 1 -- 172.21.15.53:0/3102290278 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+17207 (secure 0 0 0) 0x7f8ec8010040 con 0x7f8ed01611e0 2024-05-10T10:43:20.979 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:20.980 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":120,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:43:10.604881+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:42:47.266983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-10T10:42:48.551069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6834/38461894":"2024-05-11T10:43:10.604857+0000","172.21.15.53:6832/38461894":"2024-05-11T10:43:10.604857+0000","172.21.15.135:6837/776479349":"2024-05-11T10:43:01.305186+0000","172.21.15.135:6835/776479349":"2024-05-11T10:43:01.305186+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8eb40762d0 msgr2=0x7f8eb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8eb40762d0 0x7f8eb4078790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc034000 tx=0x7f8ecc02e000 comp rx=0 tx=0).stop 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed01611e0 msgr2=0x7f8ed00737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed01611e0 0x7f8ed00737c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800efc0 tx=0x7f8ec800c490 comp rx=0 tx=0).stop 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 shutdown_connections 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8eb40762d0 0x7f8eb4078790 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed01611e0 0x7f8ed00737c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed015ce60 0x7f8ed0073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 --2- 172.21.15.53:0/3102290278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed00c65f0 0x7f8ed0152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 >> 172.21.15.53:0/3102290278 conn(0x7f8ed00b2900 msgr2=0x7f8ed00cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 shutdown_connections 2024-05-10T10:43:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:20.981+0000 7f8ed8591640 1 -- 172.21.15.53:0/3102290278 wait complete. 2024-05-10T10:43:20.993 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:43:20.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:21.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.097+0000 7f36ac146640 1 Processor -- start 2024-05-10T10:43:21.098 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.097+0000 7f36ac146640 1 -- start start 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36ac146640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36a40d5cb0 0x7f36a40d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36ac146640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162b30 0x7f36a40d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36ac146640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a40b45f0 con 0x7f36a4162750 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36ac146640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f36a40b4f20 con 0x7f36a40d5cb0 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36ac146640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f36a40d0fe0 con 0x7f36a4162b30 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a96e1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36a40d5cb0 0x7f36a40d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a96e1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36a40d5cb0 0x7f36a40d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52522/0 (socket says 172.21.15.53:52522) 2024-05-10T10:43:21.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a96e1640 1 -- 172.21.15.53:0/3351753413 learned_addr learned my addr 172.21.15.53:0/3351753413 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:21.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36aa6e3640 1 --2- 172.21.15.53:0/3351753413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162b30 0x7f36a40d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a96e1640 1 -- 172.21.15.53:0/3351753413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162b30 msgr2=0x7f36a40d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:21.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a96e1640 1 --2- 172.21.15.53:0/3351753413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162b30 0x7f36a40d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a96e1640 1 -- 172.21.15.53:0/3351753413 >> v1:172.21.15.135:6789/0 conn(0x7f36a4162750 legacy=0x7f36a40d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a8ee0640 1 -- 172.21.15.53:0/3351753413 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557639809 0 0) 0x7f36a40b45f0 con 0x7f36a4162750 2024-05-10T10:43:21.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.098+0000 7f36a96e1640 1 -- 172.21.15.53:0/3351753413 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a40b48d0 con 0x7f36a40d5cb0 2024-05-10T10:43:21.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.099+0000 7f36a96e1640 1 --2- 172.21.15.53:0/3351753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a40d5cb0 0x7f36a40d3630 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3694009870 tx=0x7f369402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19357947a6bb5e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.099+0000 7f36a8ee0640 1 -- 172.21.15.53:0/3351753413 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694037070 con 0x7f36a40d5cb0 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.099+0000 7f36a8ee0640 1 -- 172.21.15.53:0/3351753413 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3694032440 con 0x7f36a40d5cb0 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.099+0000 7f36a8ee0640 1 -- 172.21.15.53:0/3351753413 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694032810 con 0x7f36a40d5cb0 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 -- 172.21.15.53:0/3351753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a40d5cb0 msgr2=0x7f36a40d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 --2- 172.21.15.53:0/3351753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a40d5cb0 0x7f36a40d3630 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3694009870 tx=0x7f369402a1c0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 -- 172.21.15.53:0/3351753413 shutdown_connections 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 --2- 172.21.15.53:0/3351753413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162b30 0x7f36a40d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 --2- 172.21.15.53:0/3351753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a40d5cb0 0x7f36a40d3630 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 -- 172.21.15.53:0/3351753413 >> 172.21.15.53:0/3351753413 conn(0x7f36a409e3b0 msgr2=0x7f36a40d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 -- 172.21.15.53:0/3351753413 shutdown_connections 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.100+0000 7f36ac146640 1 -- 172.21.15.53:0/3351753413 wait complete. 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.101+0000 7f36ac146640 1 Processor -- start 2024-05-10T10:43:21.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.101+0000 7f36ac146640 1 -- start start 2024-05-10T10:43:21.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36ac146640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162750 0x7f36a415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36ac146640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a4162b30 0x7f36a415c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36ac146640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a41558b0 0x7f36a40ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36ac146640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36a40b53b0 con 0x7f36a41558b0 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36ac146640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36a40d0d80 con 0x7f36a4162b30 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36ac146640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f36a40b5530 con 0x7f36a4162750 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a41558b0 0x7f36a40ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a41558b0 0x7f36a40ba7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52528/0 (socket says 172.21.15.53:52528) 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 -- 172.21.15.53:0/31214402 learned_addr learned my addr 172.21.15.53:0/31214402 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36a96e1640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a4162b30 0x7f36a415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36a9ee2640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162750 0x7f36a415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 -- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162750 msgr2=0x7f36a415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162750 0x7f36a415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 -- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a4162b30 msgr2=0x7f36a415c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a4162b30 0x7f36a415c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 -- 172.21.15.53:0/31214402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a415c650 con 0x7f36a41558b0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.102+0000 7f36aa6e3640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a41558b0 0x7f36a40ba7b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f369c00efc0 tx=0x7f369c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.103+0000 7f369a7fc640 1 -- 172.21.15.53:0/31214402 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c018020 con 0x7f36a41558b0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.103+0000 7f369a7fc640 1 -- 172.21.15.53:0/31214402 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369c014480 con 0x7f36a41558b0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.103+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a40b7c70 con 0x7f36a41558b0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.103+0000 7f369a7fc640 1 -- 172.21.15.53:0/31214402 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c014870 con 0x7f36a41558b0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.103+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a40b7f30 con 0x7f36a41558b0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.104+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a40cf470 con 0x7f36a41558b0 2024-05-10T10:43:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.104+0000 7f369a7fc640 1 -- 172.21.15.53:0/31214402 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f369c005d10 con 0x7f36a41558b0 2024-05-10T10:43:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.105+0000 7f369a7fc640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3688076300 0x7f36880787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.105+0000 7f369a7fc640 1 -- 172.21.15.53:0/31214402 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f369c098200 con 0x7f36a41558b0 2024-05-10T10:43:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.105+0000 7f36a9ee2640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3688076300 0x7f36880787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.106+0000 7f36a9ee2640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3688076300 0x7f36880787c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000f60 tx=0x7f36a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.109+0000 7f369a7fc640 1 -- 172.21.15.53:0/31214402 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f36a40cf470 con 0x7f36a41558b0 2024-05-10T10:43:21.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.342+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36a40d1b10 con 0x7f36a41558b0 2024-05-10T10:43:21.343 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.342+0000 7f369a7fc640 1 -- 172.21.15.53:0/31214402 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5186 (secure 0 0 0) 0x7f369c022070 con 0x7f36a41558b0 2024-05-10T10:43:21.343 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:21.343 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:15.622565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8352},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8352":{"gid":8352,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/2292645283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2292645283},{"type":"v1","addr":"172.21.15.53:6834","nonce":2292645283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8352,"qdb_cluster":[8352]},"id":8}]} 2024-05-10T10:43:21.343 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 111 2024-05-10T10:43:21.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3688076300 msgr2=0x7f36880787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3688076300 0x7f36880787c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000f60 tx=0x7f36a0009040 comp rx=0 tx=0).stop 2024-05-10T10:43:21.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a41558b0 msgr2=0x7f36a40ba7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a41558b0 0x7f36a40ba7b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f369c00efc0 tx=0x7f369c00c490 comp rx=0 tx=0).stop 2024-05-10T10:43:21.345 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 shutdown_connections 2024-05-10T10:43:21.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3688076300 0x7f36880787c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36a41558b0 0x7f36a40ba7b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a4162b30 0x7f36a415c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 --2- 172.21.15.53:0/31214402 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a4162750 0x7f36a415dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 >> 172.21.15.53:0/31214402 conn(0x7f36a409e3b0 msgr2=0x7f36a40cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:21.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 shutdown_connections 2024-05-10T10:43:21.346 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.344+0000 7f36ac146640 1 -- 172.21.15.53:0/31214402 wait complete. 2024-05-10T10:43:21.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:21.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.459+0000 7f363d59d640 1 Processor -- start 2024-05-10T10:43:21.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.459+0000 7f363d59d640 1 -- start start 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f363d59d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f363815f4e0 0x7f363806d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f363d59d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36380b0a90 con 0x7f363815f8c0 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f363d59d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36380b3540 con 0x7f363806e620 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f363d59d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f36380b23f0 con 0x7f363815f4e0 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f362e7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f363815f8c0 0x7f363806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42410/0 (socket says 172.21.15.53:42410) 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f362e7fc640 1 -- 172.21.15.53:0/3129900660 learned_addr learned my addr 172.21.15.53:0/3129900660 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f3636ffd640 1 --2- 172.21.15.53:0/3129900660 >> v2:172.21.15.135:3300/0 conn(0x7f363815f4e0 0x7f363806d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400645032 0 0) 0x7f36380b0a90 con 0x7f363815f8c0 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3624003520 con 0x7f363815f8c0 2024-05-10T10:43:21.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2823856477 0 0) 0x7f3624003520 con 0x7f363815f8c0 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 >> v2:172.21.15.135:3300/0 conn(0x7f363815f4e0 msgr2=0x7f363806d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 --2- 172.21.15.53:0/3129900660 >> v2:172.21.15.135:3300/0 conn(0x7f363815f4e0 0x7f363806d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 >> v1:172.21.15.135:6789/0 conn(0x7f363806e620 legacy=0x7f36380d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36380b3820 con 0x7f363815f8c0 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f3620001670 con 0x7f363815f8c0 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 >> v1:172.21.15.53:6789/0 conn(0x7f363815f8c0 legacy=0x7f363806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.460+0000 7f36367fc640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36367fc640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3624004800 0x7f3624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36367fc640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36380b23f0 con 0x7f3624004800 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f362400c160 con 0x7f36240051c0 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f362400c360 con 0x7f36240039b0 2024-05-10T10:43:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f362e7fc640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3624004800 0x7f3624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f3636ffd640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36377fe640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f362e7fc640 1 -- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36240039b0 msgr2=0x7f3624003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f362e7fc640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f362e7fc640 1 -- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36240051c0 msgr2=0x7f362400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f362e7fc640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f362e7fc640 1 -- 172.21.15.53:0/3129900660 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3624003520 con 0x7f3624004800 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36377fe640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36240051c0 0x7f362400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.461+0000 7f36377fe640 1 -- 172.21.15.53:0/3129900660 reap_dead start 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f362e7fc640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3624004800 0x7f3624004c80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3638073620 tx=0x7f3624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28669dcab068e8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620007d30 con 0x7f3624004800 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3620005af0 con 0x7f3624004800 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f36367fc640 1 -- 172.21.15.53:0/3129900660 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620007070 con 0x7f3624004800 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f363d59d640 1 -- 172.21.15.53:0/3129900660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3624004800 msgr2=0x7f3624004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f363d59d640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3624004800 0x7f3624004c80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3638073620 tx=0x7f3624005830 comp rx=0 tx=0).stop 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f363d59d640 1 -- 172.21.15.53:0/3129900660 shutdown_connections 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f363d59d640 1 --2- 172.21.15.53:0/3129900660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3624004800 0x7f3624004c80 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f363d59d640 1 -- 172.21.15.53:0/3129900660 >> 172.21.15.53:0/3129900660 conn(0x7f36380d2df0 msgr2=0x7f36380d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f363d59d640 1 -- 172.21.15.53:0/3129900660 shutdown_connections 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.462+0000 7f363d59d640 1 -- 172.21.15.53:0/3129900660 wait complete. 2024-05-10T10:43:21.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.464+0000 7f363d59d640 1 Processor -- start 2024-05-10T10:43:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.464+0000 7f363d59d640 1 -- start start 2024-05-10T10:43:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.464+0000 7f363d59d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3624004800 0x7f3638150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.464+0000 7f363d59d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3638161730 0x7f363809f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.464+0000 7f363d59d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3638161bb0 0x7f363809f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f363d59d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36380b3de0 con 0x7f3638161bb0 2024-05-10T10:43:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f363d59d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36380b4150 con 0x7f3624004800 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f363d59d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f36380b45e0 con 0x7f3638161730 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3638161bb0 0x7f363809f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3638161bb0 0x7f363809f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52548/0 (socket says 172.21.15.53:52548) 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 -- 172.21.15.53:0/2274770678 learned_addr learned my addr 172.21.15.53:0/2274770678 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f3636ffd640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3624004800 0x7f3638150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 -- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3638161730 msgr2=0x7f363809f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f362e7fc640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3638161730 0x7f363809f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3638161730 0x7f363809f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 -- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3624004800 msgr2=0x7f3638150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3624004800 0x7f3638150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 -- 172.21.15.53:0/2274770678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3638162140 con 0x7f3638161bb0 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f3636ffd640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3624004800 0x7f3638150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f36377fe640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3638161bb0 0x7f363809f970 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3638072b50 tx=0x7f363000e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f362ffff640 1 -- 172.21.15.53:0/2274770678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630019070 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f362ffff640 1 -- 172.21.15.53:0/2274770678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3630003c40 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3638162420 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f362ffff640 1 -- 172.21.15.53:0/2274770678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300050d0 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.465+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3638150670 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.466+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363806a220 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.467+0000 7f362ffff640 1 -- 172.21.15.53:0/2274770678 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f363001c070 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.467+0000 7f362ffff640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36180762d0 0x7f3618078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.468+0000 7f362ffff640 1 -- 172.21.15.53:0/2274770678 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f3630096e90 con 0x7f3638161bb0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.468+0000 7f3636ffd640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36180762d0 0x7f3618078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.468+0000 7f3636ffd640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36180762d0 0x7f3618078790 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3628009b30 tx=0x7f3628008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.472+0000 7f362ffff640 1 -- 172.21.15.53:0/2274770678 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f36300605a0 con 0x7f3638161bb0 2024-05-10T10:43:21.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.704+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36381625f0 con 0x7f3638161bb0 2024-05-10T10:43:21.705 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.704+0000 7f362ffff640 1 -- 172.21.15.53:0/2274770678 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5186 (secure 0 0 0) 0x7f3630098090 con 0x7f3638161bb0 2024-05-10T10:43:21.705 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:21.705 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T10:42:49.598991+0000","modified":"2024-05-10T10:43:15.622565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8352},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8352":{"gid":8352,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6834/2292645283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2292645283},{"type":"v1","addr":"172.21.15.53:6834","nonce":2292645283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8352,"qdb_cluster":[8352]},"id":8}]} 2024-05-10T10:43:21.705 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 111 2024-05-10T10:43:21.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36180762d0 msgr2=0x7f3618078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36180762d0 0x7f3618078790 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3628009b30 tx=0x7f3628008040 comp rx=0 tx=0).stop 2024-05-10T10:43:21.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3638161bb0 msgr2=0x7f363809f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3638161bb0 0x7f363809f970 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3638072b50 tx=0x7f363000e980 comp rx=0 tx=0).stop 2024-05-10T10:43:21.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 shutdown_connections 2024-05-10T10:43:21.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f36180762d0 0x7f3618078790 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3638161bb0 0x7f363809f970 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3638161730 0x7f363809f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 --2- 172.21.15.53:0/2274770678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3624004800 0x7f3638150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 >> 172.21.15.53:0/2274770678 conn(0x7f36380d2df0 msgr2=0x7f36380b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:21.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 shutdown_connections 2024-05-10T10:43:21.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.706+0000 7f363d59d640 1 -- 172.21.15.53:0/2274770678 wait complete. 2024-05-10T10:43:21.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:43:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.821+0000 7fd93d119640 1 Processor -- start 2024-05-10T10:43:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.821+0000 7fd93d119640 1 -- start start 2024-05-10T10:43:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93d119640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380b8860 0x7fd93806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93d119640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9380d0230 0x7fd9380cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93d119640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd9380cfbb0 0x7fd9380d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93d119640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd93806ee50 con 0x7fd9380d0230 2024-05-10T10:43:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93d119640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd93815fc90 con 0x7fd9380cfbb0 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93d119640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd9380b8c40 con 0x7fd9380b8860 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9380d0230 0x7fd9380cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9380d0230 0x7fd9380cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52562/0 (socket says 172.21.15.53:52562) 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 -- 172.21.15.53:0/2815023468 learned_addr learned my addr 172.21.15.53:0/2815023468 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93759e640 1 --2- 172.21.15.53:0/2815023468 >> v2:172.21.15.135:3300/0 conn(0x7fd9380cfbb0 0x7fd9380d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd936d9d640 1 --2- 172.21.15.53:0/2815023468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380b8860 0x7fd93806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 -- 172.21.15.53:0/2815023468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380b8860 msgr2=0x7fd93806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 --2- 172.21.15.53:0/2815023468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380b8860 0x7fd93806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 -- 172.21.15.53:0/2815023468 >> v2:172.21.15.135:3300/0 conn(0x7fd9380cfbb0 msgr2=0x7fd9380d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 --2- 172.21.15.53:0/2815023468 >> v2:172.21.15.135:3300/0 conn(0x7fd9380cfbb0 0x7fd9380d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93659c640 1 -- 172.21.15.53:0/2815023468 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9380cd570 con 0x7fd9380d0230 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.822+0000 7fd93759e640 1 --2- 172.21.15.53:0/2815023468 >> v2:172.21.15.135:3300/0 conn(0x7fd9380cfbb0 0x7fd9380d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.823+0000 7fd93659c640 1 --2- 172.21.15.53:0/2815023468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380d0230 0x7fd9380cf670 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd9380734b0 tx=0x7fd92c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47ebbc186b910c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.823+0000 7fd935d9b640 1 -- 172.21.15.53:0/2815023468 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c037070 con 0x7fd9380d0230 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.823+0000 7fd935d9b640 1 -- 172.21.15.53:0/2815023468 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92c032440 con 0x7fd9380d0230 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.823+0000 7fd935d9b640 1 -- 172.21.15.53:0/2815023468 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c032810 con 0x7fd9380d0230 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.823+0000 7fd93d119640 1 -- 172.21.15.53:0/2815023468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380d0230 msgr2=0x7fd9380cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.823+0000 7fd93d119640 1 --2- 172.21.15.53:0/2815023468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380d0230 0x7fd9380cf670 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd9380734b0 tx=0x7fd92c02a180 comp rx=0 tx=0).stop 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.824+0000 7fd93d119640 1 -- 172.21.15.53:0/2815023468 shutdown_connections 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.824+0000 7fd93d119640 1 --2- 172.21.15.53:0/2815023468 >> v2:172.21.15.135:3300/0 conn(0x7fd9380cfbb0 0x7fd9380d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.824+0000 7fd93d119640 1 --2- 172.21.15.53:0/2815023468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380d0230 0x7fd9380cf670 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.824+0000 7fd93d119640 1 --2- 172.21.15.53:0/2815023468 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380b8860 0x7fd93806e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.824+0000 7fd93d119640 1 -- 172.21.15.53:0/2815023468 >> 172.21.15.53:0/2815023468 conn(0x7fd9380d2df0 msgr2=0x7fd9380d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.824+0000 7fd93d119640 1 -- 172.21.15.53:0/2815023468 shutdown_connections 2024-05-10T10:43:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.824+0000 7fd93d119640 1 -- 172.21.15.53:0/2815023468 wait complete. 2024-05-10T10:43:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.825+0000 7fd93d119640 1 Processor -- start 2024-05-10T10:43:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.825+0000 7fd93d119640 1 -- start start 2024-05-10T10:43:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93d119640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380b8860 0x7fd9380cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93d119640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380cfbb0 0x7fd93815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93d119640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9380d0230 0x7fd93809d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93d119640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd9380cde10 con 0x7fd9380b8860 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93d119640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9380cdb30 con 0x7fd9380d0230 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93d119640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd93806ea60 con 0x7fd9380cfbb0 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380b8860 0x7fd9380cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380b8860 0x7fd9380cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52566/0 (socket says 172.21.15.53:52566) 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 -- 172.21.15.53:0/2644088114 learned_addr learned my addr 172.21.15.53:0/2644088114 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93759e640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9380d0230 0x7fd93809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93659c640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380cfbb0 0x7fd93815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 -- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380cfbb0 msgr2=0x7fd93815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380cfbb0 0x7fd93815ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 -- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9380d0230 msgr2=0x7fd93809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9380d0230 0x7fd93809d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 -- 172.21.15.53:0/2644088114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93815ff00 con 0x7fd9380b8860 2024-05-10T10:43:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93759e640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9380d0230 0x7fd93809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd93659c640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380cfbb0 0x7fd93815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.826+0000 7fd936d9d640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380b8860 0x7fd9380cc260 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c5a0 tx=0x7fd92800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.827+0000 7fd91f7fe640 1 -- 172.21.15.53:0/2644088114 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928015070 con 0x7fd9380b8860 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.827+0000 7fd91f7fe640 1 -- 172.21.15.53:0/2644088114 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd928004220 con 0x7fd9380b8860 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.827+0000 7fd91f7fe640 1 -- 172.21.15.53:0/2644088114 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928003610 con 0x7fd9380b8860 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.827+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9381601e0 con 0x7fd9380b8860 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.827+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93815f320 con 0x7fd9380b8860 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.828+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93806a220 con 0x7fd9380b8860 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.828+0000 7fd91f7fe640 1 -- 172.21.15.53:0/2644088114 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd9280037b0 con 0x7fd9380b8860 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.829+0000 7fd91f7fe640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9240762d0 0x7fd924078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.829+0000 7fd91f7fe640 1 -- 172.21.15.53:0/2644088114 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7fd928097570 con 0x7fd9380b8860 2024-05-10T10:43:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.829+0000 7fd93659c640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9240762d0 0x7fd924078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.830+0000 7fd93659c640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9240762d0 0x7fd924078790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd92c034000 tx=0x7fd92c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:21.833+0000 7fd91f7fe640 1 -- 172.21.15.53:0/2644088114 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd928060c90 con 0x7fd9380b8860 2024-05-10T10:43:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.025+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9380d1e20 con 0x7fd9380b8860 2024-05-10T10:43:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.026+0000 7fd91f7fe640 1 -- 172.21.15.53:0/2644088114 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+17207 (secure 0 0 0) 0x7fd928060590 con 0x7fd9380b8860 2024-05-10T10:43:22.027 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:22.027 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":120,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:43:10.604881+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-10T10:42:47.266983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-10T10:42:48.551069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6834/38461894":"2024-05-11T10:43:10.604857+0000","172.21.15.53:6832/38461894":"2024-05-11T10:43:10.604857+0000","172.21.15.135:6837/776479349":"2024-05-11T10:43:01.305186+0000","172.21.15.135:6835/776479349":"2024-05-11T10:43:01.305186+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9240762d0 msgr2=0x7fd924078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9240762d0 0x7fd924078790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd92c034000 tx=0x7fd92c02e000 comp rx=0 tx=0).stop 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380b8860 msgr2=0x7fd9380cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380b8860 0x7fd9380cc260 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c5a0 tx=0x7fd92800ca60 comp rx=0 tx=0).stop 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 shutdown_connections 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd9240762d0 0x7fd924078790 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9380d0230 0x7fd93809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9380cfbb0 0x7fd93815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 --2- 172.21.15.53:0/2644088114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9380b8860 0x7fd9380cc260 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 >> 172.21.15.53:0/2644088114 conn(0x7fd9380d2df0 msgr2=0x7fd9380d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 shutdown_connections 2024-05-10T10:43:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.028+0000 7fd93d119640 1 -- 172.21.15.53:0/2644088114 wait complete. 2024-05-10T10:43:22.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T10:43:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.144+0000 7f72d022a640 1 Processor -- start 2024-05-10T10:43:22.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.144+0000 7f72d022a640 1 -- start start 2024-05-10T10:43:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72d022a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 0x7f72c815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72d022a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f72c80704c0 0x7f72c8070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72d022a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c8160190 con 0x7f72c806c990 2024-05-10T10:43:22.148 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72d022a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f72c806cd70 con 0x7f72c80704c0 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72d022a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f72c8160470 con 0x7f72c815f620 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72cdfc6640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f72c806c990 0x7f72c8163660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42418/0 (socket says 172.21.15.53:42418) 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72cdfc6640 1 -- 172.21.15.53:0/3775462282 learned_addr learned my addr 172.21.15.53:0/3775462282 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.145+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863044627 0 0) 0x7f72c8160190 con 0x7f72c806c990 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72b0003520 con 0x7f72c806c990 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ce7c7640 1 --2- 172.21.15.53:0/3775462282 >> v2:172.21.15.135:3300/0 conn(0x7f72c80704c0 0x7f72c8070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cd7c5640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 0x7f72c815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2169980520 0 0) 0x7f72b0003520 con 0x7f72c806c990 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 msgr2=0x7f72c815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 0x7f72c815fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 >> v2:172.21.15.135:3300/0 conn(0x7f72c80704c0 msgr2=0x7f72c8070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 --2- 172.21.15.53:0/3775462282 >> v2:172.21.15.135:3300/0 conn(0x7f72c80704c0 0x7f72c8070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c8162e60 con 0x7f72c806c990 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f72c0001670 con 0x7f72c806c990 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 >> v1:172.21.15.53:6789/0 conn(0x7f72c806c990 legacy=0x7f72c8163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00039b0 0x7f72b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 0x7f72b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b0004d40 0x7f72b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f72c806cd70 con 0x7f72b00039b0 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8160470 con 0x7f72b0004d40 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f72b000b9c0 con 0x7f72c815f620 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cdfc6640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00039b0 0x7f72b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cd7c5640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 0x7f72b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72ce7c7640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b0004d40 0x7f72b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cdfc6640 1 -- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 msgr2=0x7f72b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:22.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cdfc6640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c815f620 0x7f72b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cdfc6640 1 -- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b0004d40 msgr2=0x7f72b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cdfc6640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b0004d40 0x7f72b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.146+0000 7f72cdfc6640 1 -- 172.21.15.53:0/3775462282 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b0003520 con 0x7f72b00039b0 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.147+0000 7f72cdfc6640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00039b0 0x7f72b0003dd0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f72c0007fd0 tx=0x7f72b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35120bf3938ef84 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.147+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0007d50 con 0x7f72b00039b0 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.147+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c0005ad0 con 0x7f72b00039b0 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.147+0000 7f72ccfc4640 1 -- 172.21.15.53:0/3775462282 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0007070 con 0x7f72b00039b0 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.147+0000 7f72d022a640 1 -- 172.21.15.53:0/3775462282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00039b0 msgr2=0x7f72b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.147+0000 7f72d022a640 1 --2- 172.21.15.53:0/3775462282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b00039b0 0x7f72b0003dd0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f72c0007fd0 tx=0x7f72b0005830 comp rx=0 tx=0).stop 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.148+0000 7f72ce7c7640 1 -- 172.21.15.53:0/3775462282 reap_dead start 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.148+0000 7f72d022a640 1 -- 172.21.15.53:0/3775462282 shutdown_connections 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.148+0000 7f72d022a640 1 -- 172.21.15.53:0/3775462282 >> 172.21.15.53:0/3775462282 conn(0x7f72c80b3980 msgr2=0x7f72c80b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:22.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.148+0000 7f72d022a640 1 -- 172.21.15.53:0/3775462282 shutdown_connections 2024-05-10T10:43:22.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.148+0000 7f72d022a640 1 -- 172.21.15.53:0/3775462282 wait complete. 2024-05-10T10:43:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.149+0000 7f72d022a640 1 Processor -- start 2024-05-10T10:43:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.149+0000 7f72d022a640 1 -- start start 2024-05-10T10:43:22.154 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72d022a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b00039b0 0x7f72c814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72d022a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c81f5290 0x7f72c81f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72d022a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2000 0x7f72c80b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72d022a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8073e70 con 0x7f72c80b2000 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72d022a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8070e90 con 0x7f72b00039b0 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72d022a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f72c815fa80 con 0x7f72c81f5290 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2000 0x7f72c80b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2000 0x7f72c80b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52588/0 (socket says 172.21.15.53:52588) 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 -- 172.21.15.53:0/3873878470 learned_addr learned my addr 172.21.15.53:0/3873878470 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72cd7c5640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c81f5290 0x7f72c81f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72cdfc6640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b00039b0 0x7f72c814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 -- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c81f5290 msgr2=0x7f72c81f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c81f5290 0x7f72c81f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.155 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 -- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b00039b0 msgr2=0x7f72c814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b00039b0 0x7f72c814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 -- 172.21.15.53:0/3873878470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c80b5e00 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.150+0000 7f72ce7c7640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2000 0x7f72c80b2480 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f72c809ffe0 tx=0x7f72b80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.151+0000 7f72c67fc640 1 -- 172.21.15.53:0/3873878470 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8011e40 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.151+0000 7f72c67fc640 1 -- 172.21.15.53:0/3873878470 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b801b070 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.151+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c80b60e0 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.151+0000 7f72c67fc640 1 -- 172.21.15.53:0/3873878470 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8003990 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.151+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c80b5ab0 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.152+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c806a220 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.152+0000 7f72c67fc640 1 -- 172.21.15.53:0/3873878470 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f72b8005d10 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.153+0000 7f72c67fc640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72b40762d0 0x7f72b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.153+0000 7f72c67fc640 1 -- 172.21.15.53:0/3873878470 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6273+0+0 (secure 0 0 0) 0x7f72b8097ce0 con 0x7f72c80b2000 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.153+0000 7f72cdfc6640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72b40762d0 0x7f72b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.156 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.154+0000 7f72cdfc6640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72b40762d0 0x7f72b4078790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f72c0033000 tx=0x7f72c0009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.157+0000 7f72c67fc640 1 -- 172.21.15.53:0/3873878470 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f72b8025020 con 0x7f72c80b2000 2024-05-10T10:43:22.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.390+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f72c80b36e0 con 0x7f72c80b2000 2024-05-10T10:43:22.391 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-05-10T10:43:22.390+0000 7f39f6f67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T10:43:22.392 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.392+0000 7f72c67fc640 1 -- 172.21.15.53:0/3873878470 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) ==== 161+0+0 (secure 0 0 0) 0x7f72b8061490 con 0x7f72c80b2000 2024-05-10T10:43:22.392 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72b40762d0 msgr2=0x7f72b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72b40762d0 0x7f72b4078790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f72c0033000 tx=0x7f72c0009030 comp rx=0 tx=0).stop 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2000 msgr2=0x7f72c80b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2000 0x7f72c80b2480 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f72c809ffe0 tx=0x7f72b80115a0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 shutdown_connections 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f72b40762d0 0x7f72b4078790 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2000 0x7f72c80b2480 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f72c81f5290 0x7f72c81f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 --2- 172.21.15.53:0/3873878470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72b00039b0 0x7f72c814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 >> 172.21.15.53:0/3873878470 conn(0x7f72c80b3980 msgr2=0x7f72c80b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:22.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 shutdown_connections 2024-05-10T10:43:22.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.395+0000 7f72d022a640 1 -- 172.21.15.53:0/3873878470 wait complete. 2024-05-10T10:43:22.397 INFO:tasks.ceph.mds.c.smithi053.stderr: -1271> 2024-05-10T10:43:16.612+0000 7ffa9705a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T10:43:22.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T10:43:22.425 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2024-05-10T10:43:22.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.516+0000 7f4b45b6a640 1 Processor -- start 2024-05-10T10:43:22.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.516+0000 7f4b45b6a640 1 -- start start 2024-05-10T10:43:22.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b45b6a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4b400cffd0 0x7f4b400b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b45b6a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4006e720 con 0x7f4b400cd500 2024-05-10T10:43:22.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b45b6a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4006de80 con 0x7f4b400cf3e0 2024-05-10T10:43:22.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b45b6a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4006e160 con 0x7f4b400cffd0 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4b400cd500 0x7f4b400d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42424/0 (socket says 172.21.15.53:42424) 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3ffff640 1 -- 172.21.15.53:0/24370305 learned_addr learned my addr 172.21.15.53:0/24370305 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3effd640 1 --2- 172.21.15.53:0/24370305 >> v2:172.21.15.135:3300/0 conn(0x7f4b400cffd0 0x7f4b400b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489808957 0 0) 0x7f4b4006e720 con 0x7f4b400cd500 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b28003520 con 0x7f4b400cd500 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3827221876 0 0) 0x7f4b28003520 con 0x7f4b400cd500 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 >> v2:172.21.15.135:3300/0 conn(0x7f4b400cffd0 msgr2=0x7f4b400b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 --2- 172.21.15.53:0/24370305 >> v2:172.21.15.135:3300/0 conn(0x7f4b400cffd0 0x7f4b400b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 >> v1:172.21.15.135:6789/0 conn(0x7f4b400cf3e0 legacy=0x7f4b400cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4006e440 con 0x7f4b400cd500 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f4b380019f0 con 0x7f4b400cd500 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.517+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 >> v1:172.21.15.53:6789/0 conn(0x7f4b400cd500 legacy=0x7f4b400d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.521 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3e7fc640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b280039b0 0x7f4b28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3e7fc640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3e7fc640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b280051c0 0x7f4b2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4006e160 con 0x7f4b28004800 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2800c060 con 0x7f4b280051c0 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2800c260 con 0x7f4b280039b0 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3effd640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3f7fe640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b280039b0 0x7f4b28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3ffff640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b280051c0 0x7f4b2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3effd640 1 -- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b280039b0 msgr2=0x7f4b28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3effd640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b280039b0 0x7f4b28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3effd640 1 -- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b280051c0 msgr2=0x7f4b2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:22.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3effd640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b280051c0 0x7f4b2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3effd640 1 -- 172.21.15.53:0/24370305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b28003520 con 0x7f4b28004800 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3ffff640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b280051c0 0x7f4b2800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.518+0000 7f4b3ffff640 1 -- 172.21.15.53:0/24370305 reap_dead start 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b3effd640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009ac0 tx=0x7f4b28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf4d2a1268ff3ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34036070 con 0x7f4b28004800 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b34031400 con 0x7f4b28004800 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b3e7fc640 1 -- 172.21.15.53:0/24370305 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34009690 con 0x7f4b28004800 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b45b6a640 1 -- 172.21.15.53:0/24370305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b28004800 msgr2=0x7f4b28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009ac0 tx=0x7f4b28005830 comp rx=0 tx=0).stop 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b45b6a640 1 -- 172.21.15.53:0/24370305 shutdown_connections 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/24370305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b45b6a640 1 -- 172.21.15.53:0/24370305 >> 172.21.15.53:0/24370305 conn(0x7f4b400d2eb0 msgr2=0x7f4b400d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.519+0000 7f4b45b6a640 1 -- 172.21.15.53:0/24370305 shutdown_connections 2024-05-10T10:43:22.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.520+0000 7f4b45b6a640 1 -- 172.21.15.53:0/24370305 wait complete. 2024-05-10T10:43:22.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.521+0000 7f4b45b6a640 1 Processor -- start 2024-05-10T10:43:22.526 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.521+0000 7f4b45b6a640 1 -- start start 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004800 0x7f4b40157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cb300 0x7f4b4009ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b4009ef90 0x7f4b401f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400cf8c0 con 0x7f4b4009ef90 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400b4510 con 0x7f4b28004800 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4b400cfa40 con 0x7f4b400cb300 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b4009ef90 0x7f4b401f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b4009ef90 0x7f4b401f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52632/0 (socket says 172.21.15.53:52632) 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 -- 172.21.15.53:0/3544004263 learned_addr learned my addr 172.21.15.53:0/3544004263 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3effd640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cb300 0x7f4b4009ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3f7fe640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004800 0x7f4b40157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 -- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cb300 msgr2=0x7f4b4009ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cb300 0x7f4b4009ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 -- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004800 msgr2=0x7f4b40157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004800 0x7f4b40157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 -- 172.21.15.53:0/3544004263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b400c9f80 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b3ffff640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b4009ef90 0x7f4b401f53e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4b38001f70 tx=0x7f4b3800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b27fff640 1 -- 172.21.15.53:0/3544004263 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38007a80 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b27fff640 1 -- 172.21.15.53:0/3544004263 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b38018070 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b27fff640 1 -- 172.21.15.53:0/3544004263 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38013420 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b400ca260 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.522+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b400ca820 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.524+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4006a220 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.524+0000 7f4b27fff640 1 -- 172.21.15.53:0/3544004263 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b380135c0 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.525+0000 7f4b27fff640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b2c076300 0x7f4b2c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.525+0000 7f4b27fff640 1 -- 172.21.15.53:0/3544004263 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6359+0+0 (secure 0 0 0) 0x7f4b38097b60 con 0x7f4b4009ef90 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.525+0000 7f4b3f7fe640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b2c076300 0x7f4b2c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:22.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.525+0000 7f4b3f7fe640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b2c076300 0x7f4b2c0787c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:22.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.529+0000 7f4b27fff640 1 -- 172.21.15.53:0/3544004263 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4b380612c0 con 0x7f4b4009ef90 2024-05-10T10:43:22.761 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:22.760+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4b4006e720 con 0x7f4b4009ef90 2024-05-10T10:43:23.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.393+0000 7f4b27fff640 1 -- 172.21.15.53:0/3544004263 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) ==== 104+0+0 (secure 0 0 0) 0x7f4b38060bc0 con 0x7f4b4009ef90 2024-05-10T10:43:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b2c076300 msgr2=0x7f4b2c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b2c076300 0x7f4b2c0787c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b30009040 comp rx=0 tx=0).stop 2024-05-10T10:43:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b4009ef90 msgr2=0x7f4b401f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b4009ef90 0x7f4b401f53e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4b38001f70 tx=0x7f4b3800e530 comp rx=0 tx=0).stop 2024-05-10T10:43:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 shutdown_connections 2024-05-10T10:43:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4b2c076300 0x7f4b2c0787c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b4009ef90 0x7f4b401f53e0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cb300 0x7f4b4009ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 --2- 172.21.15.53:0/3544004263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004800 0x7f4b40157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.396+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 >> 172.21.15.53:0/3544004263 conn(0x7f4b400d2eb0 msgr2=0x7f4b400b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:23.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.397+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 shutdown_connections 2024-05-10T10:43:23.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.397+0000 7f4b45b6a640 1 -- 172.21.15.53:0/3544004263 wait complete. 2024-05-10T10:43:23.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T10:43:23.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.569+0000 7f8e0d989640 1 Processor -- start 2024-05-10T10:43:23.569 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.569+0000 7f8e0d989640 1 -- start start 2024-05-10T10:43:23.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.569+0000 7f8e0d989640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e0815f370 0x7f8e080d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:23.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.569+0000 7f8e0d989640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e081639a0 0x7f8e08070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.569+0000 7f8e0d989640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e080d2510 con 0x7f8e08071520 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.569+0000 7f8e0d989640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8e08070220 con 0x7f8e0815f370 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.569+0000 7f8e0d989640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8e08070500 con 0x7f8e081639a0 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e06ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e0815f370 0x7f8e080d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e077fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8e08071520 0x7f8e08162e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57956/0 (socket says 172.21.15.53:57956) 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e077fe640 1 -- 172.21.15.53:0/1495620916 learned_addr learned my addr 172.21.15.53:0/1495620916 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e067fc640 1 --2- 172.21.15.53:0/1495620916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e081639a0 0x7f8e08070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e06ffd640 1 -- 172.21.15.53:0/1495620916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e081639a0 msgr2=0x7f8e08070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e06ffd640 1 --2- 172.21.15.53:0/1495620916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e081639a0 0x7f8e08070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e06ffd640 1 -- 172.21.15.53:0/1495620916 >> v1:172.21.15.53:6789/0 conn(0x7f8e08071520 legacy=0x7f8e08162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e06ffd640 1 -- 172.21.15.53:0/1495620916 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e080707e0 con 0x7f8e0815f370 2024-05-10T10:43:23.574 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e05ffb640 1 -- 172.21.15.53:0/1495620916 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1168585192 0 0) 0x7f8e080d2510 con 0x7f8e08071520 2024-05-10T10:43:23.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.570+0000 7f8e067fc640 1 --2- 172.21.15.53:0/1495620916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e081639a0 0x7f8e08070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:23.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e06ffd640 1 --2- 172.21.15.53:0/1495620916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e080d1cc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8df80097a0 tx=0x7f8df802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2330c9fc2376fe3d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:23.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e05ffb640 1 -- 172.21.15.53:0/1495620916 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8037070 con 0x7f8e0815f370 2024-05-10T10:43:23.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e05ffb640 1 -- 172.21.15.53:0/1495620916 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df8032440 con 0x7f8e0815f370 2024-05-10T10:43:23.575 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e05ffb640 1 -- 172.21.15.53:0/1495620916 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8032810 con 0x7f8e0815f370 2024-05-10T10:43:23.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e0d989640 1 -- 172.21.15.53:0/1495620916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 msgr2=0x7f8e080d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:23.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e0d989640 1 --2- 172.21.15.53:0/1495620916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e080d1cc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8df80097a0 tx=0x7f8df802a210 comp rx=0 tx=0).stop 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e0d989640 1 -- 172.21.15.53:0/1495620916 shutdown_connections 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e0d989640 1 --2- 172.21.15.53:0/1495620916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e081639a0 0x7f8e08070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e0d989640 1 --2- 172.21.15.53:0/1495620916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e080d1cc0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.571+0000 7f8e0d989640 1 -- 172.21.15.53:0/1495620916 >> 172.21.15.53:0/1495620916 conn(0x7f8e080d2e10 msgr2=0x7f8e080d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.572+0000 7f8e0d989640 1 -- 172.21.15.53:0/1495620916 shutdown_connections 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.572+0000 7f8e0d989640 1 -- 172.21.15.53:0/1495620916 wait complete. 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.573+0000 7f8e0d989640 1 Processor -- start 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.573+0000 7f8e0d989640 1 -- start start 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e0d989640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e081f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:23.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e0d989640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e081639a0 0x7f8e081f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e0d989640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e08153140 0x7f8e0806e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e0d989640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08163510 con 0x7f8e0815f370 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e0d989640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8e080d2200 con 0x7f8e081639a0 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e0d989640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f8e08163690 con 0x7f8e08153140 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e081f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e081f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49742/0 (socket says 172.21.15.53:49742) 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 -- 172.21.15.53:0/4088395127 learned_addr learned my addr 172.21.15.53:0/4088395127 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e077fe640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e08153140 0x7f8e0806e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e067fc640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e081639a0 0x7f8e081f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 -- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e08153140 msgr2=0x7f8e0806e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e08153140 0x7f8e0806e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 -- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e081639a0 msgr2=0x7f8e081f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e081639a0 0x7f8e081f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:23.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 -- 172.21.15.53:0/4088395127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e081f91f0 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e077fe640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e08153140 0x7f8e0806e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e067fc640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e081639a0 0x7f8e081f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e06ffd640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e081f46e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8df8035040 tx=0x7f8df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8deffff640 1 -- 172.21.15.53:0/4088395127 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8037040 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8deffff640 1 -- 172.21.15.53:0/4088395127 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df8030770 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e081f8c30 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.574+0000 7f8deffff640 1 -- 172.21.15.53:0/4088395127 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8030b40 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.575+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0809eb30 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.576+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0806a220 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.576+0000 7f8deffff640 1 -- 172.21.15.53:0/4088395127 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8df8044020 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.577+0000 7f8deffff640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8df40762d0 0x7f8df4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.577+0000 7f8deffff640 1 -- 172.21.15.53:0/4088395127 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6359+0+0 (secure 0 0 0) 0x7f8df80b7de0 con 0x7f8e0815f370 2024-05-10T10:43:23.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.577+0000 7f8e067fc640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8df40762d0 0x7f8df4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:23.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.577+0000 7f8e067fc640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8df40762d0 0x7f8df4078790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8e080c75c0 tx=0x7f8dfc007530 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:23.582 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.581+0000 7f8deffff640 1 -- 172.21.15.53:0/4088395127 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f8df8032e20 con 0x7f8e0815f370 2024-05-10T10:43:23.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:23.774+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8e080b8d90 con 0x7f8e0815f370 2024-05-10T10:43:24.657 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.656+0000 7f8deffff640 1 -- 172.21.15.53:0/4088395127 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7f8df8081540 con 0x7f8e0815f370 2024-05-10T10:43:24.657 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8df40762d0 msgr2=0x7f8df4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8df40762d0 0x7f8df4078790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8e080c75c0 tx=0x7f8dfc007530 comp rx=0 tx=0).stop 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 msgr2=0x7f8e081f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e081f46e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8df8035040 tx=0x7f8df80046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 shutdown_connections 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f8df40762d0 0x7f8df4078790 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e08153140 0x7f8e0806e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e081639a0 0x7f8e081f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 --2- 172.21.15.53:0/4088395127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0815f370 0x7f8e081f46e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 >> 172.21.15.53:0/4088395127 conn(0x7f8e080d2e10 msgr2=0x7f8e080d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 shutdown_connections 2024-05-10T10:43:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.660+0000 7f8e0d989640 1 -- 172.21.15.53:0/4088395127 wait complete. 2024-05-10T10:43:24.675 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T10:43:24.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.790+0000 7f29f6a6b640 1 Processor -- start 2024-05-10T10:43:24.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.790+0000 7f29f6a6b640 1 -- start start 2024-05-10T10:43:24.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f6a6b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 0x7f29f00d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f6a6b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f29f00d3d50 0x7f29f00b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f6a6b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f00d1080 con 0x7f29f00d5ca0 2024-05-10T10:43:24.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f6a6b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f29f00b4e70 con 0x7f29f00d3d50 2024-05-10T10:43:24.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f6a6b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29f00b4820 con 0x7f29f015f490 2024-05-10T10:43:24.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29effff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f29f00d5ca0 0x7f29f00d3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57964/0 (socket says 172.21.15.53:57964) 2024-05-10T10:43:24.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29effff640 1 -- 172.21.15.53:0/3168873146 learned_addr learned my addr 172.21.15.53:0/3168873146 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:24.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159910627 0 0) 0x7f29f00d1080 con 0x7f29f00d5ca0 2024-05-10T10:43:24.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29e0003520 con 0x7f29f00d5ca0 2024-05-10T10:43:24.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f4807640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 0x7f29f00d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f5008640 1 --2- 172.21.15.53:0/3168873146 >> v2:172.21.15.135:3300/0 conn(0x7f29f00d3d50 0x7f29f00b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4224353756 0 0) 0x7f29e0003520 con 0x7f29f00d5ca0 2024-05-10T10:43:24.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 msgr2=0x7f29f00d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 0x7f29f00d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 >> v2:172.21.15.135:3300/0 conn(0x7f29f00d3d50 msgr2=0x7f29f00b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 --2- 172.21.15.53:0/3168873146 >> v2:172.21.15.135:3300/0 conn(0x7f29f00d3d50 0x7f29f00b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00b4b00 con 0x7f29f00d5ca0 2024-05-10T10:43:24.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f29dc001670 con 0x7f29f00d5ca0 2024-05-10T10:43:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 >> v1:172.21.15.53:6789/0 conn(0x7f29f00d5ca0 legacy=0x7f29f00d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29e00039b0 0x7f29e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 0x7f29e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e0004d40 0x7f29e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00b4e70 con 0x7f29e00039b0 2024-05-10T10:43:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00b4820 con 0x7f29e0004d40 2024-05-10T10:43:24.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29e000bb20 con 0x7f29f015f490 2024-05-10T10:43:24.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.791+0000 7f29f4807640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29e00039b0 0x7f29e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29effff640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 0x7f29e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29f5008640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e0004d40 0x7f29e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29f4807640 1 -- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 msgr2=0x7f29e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29f4807640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 0x7f29e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29f4807640 1 -- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e0004d40 msgr2=0x7f29e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:24.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29f4807640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e0004d40 0x7f29e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29f4807640 1 -- 172.21.15.53:0/3168873146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e0003520 con 0x7f29e00039b0 2024-05-10T10:43:24.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29effff640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f015f490 0x7f29e0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:24.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.792+0000 7f29f4807640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29e00039b0 0x7f29e0003dd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f29e400ba30 tx=0x7f29e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebad5888d46d767c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4030070 con 0x7f29e00039b0 2024-05-10T10:43:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e402ade0 con 0x7f29e00039b0 2024-05-10T10:43:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29ef7fe640 1 -- 172.21.15.53:0/3168873146 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e400b600 con 0x7f29e00039b0 2024-05-10T10:43:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29f6a6b640 1 -- 172.21.15.53:0/3168873146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29e00039b0 msgr2=0x7f29e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29f6a6b640 1 --2- 172.21.15.53:0/3168873146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29e00039b0 0x7f29e0003dd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f29e400ba30 tx=0x7f29e0005830 comp rx=0 tx=0).stop 2024-05-10T10:43:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29f5008640 1 -- 172.21.15.53:0/3168873146 reap_dead start 2024-05-10T10:43:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29f6a6b640 1 -- 172.21.15.53:0/3168873146 shutdown_connections 2024-05-10T10:43:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29f6a6b640 1 -- 172.21.15.53:0/3168873146 >> 172.21.15.53:0/3168873146 conn(0x7f29f00d50f0 msgr2=0x7f29f00d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29f6a6b640 1 -- 172.21.15.53:0/3168873146 shutdown_connections 2024-05-10T10:43:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.793+0000 7f29f6a6b640 1 -- 172.21.15.53:0/3168873146 wait complete. 2024-05-10T10:43:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.794+0000 7f29f6a6b640 1 Processor -- start 2024-05-10T10:43:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.795+0000 7f29f6a6b640 1 -- start start 2024-05-10T10:43:24.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.795+0000 7f29f6a6b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e00039b0 0x7f29f0154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.795+0000 7f29f6a6b640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f00cc1c0 0x7f29f00ba190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.795+0000 7f29f6a6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29f00cc620 0x7f29f00ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.795+0000 7f29f6a6b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29f015f9f0 con 0x7f29f00cc620 2024-05-10T10:43:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.795+0000 7f29f6a6b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29f015f870 con 0x7f29e00039b0 2024-05-10T10:43:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.795+0000 7f29f6a6b640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f29f00d2f60 con 0x7f29f00cc1c0 2024-05-10T10:43:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29f00cc620 0x7f29f00ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29f00cc620 0x7f29f00ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49764/0 (socket says 172.21.15.53:49764) 2024-05-10T10:43:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 -- 172.21.15.53:0/4128462896 learned_addr learned my addr 172.21.15.53:0/4128462896 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29effff640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f00cc1c0 0x7f29f00ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f4807640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e00039b0 0x7f29f0154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 -- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f00cc1c0 msgr2=0x7f29f00ba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f00cc1c0 0x7f29f00ba190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 -- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e00039b0 msgr2=0x7f29f0154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e00039b0 0x7f29f0154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 -- 172.21.15.53:0/4128462896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00cd3e0 con 0x7f29f00cc620 2024-05-10T10:43:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f4807640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e00039b0 0x7f29f0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f5008640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29f00cc620 0x7f29f00ba6d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f29f00d15f0 tx=0x7f29e8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29ed7fa640 1 -- 172.21.15.53:0/4128462896 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8007c70 con 0x7f29f00cc620 2024-05-10T10:43:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29ed7fa640 1 -- 172.21.15.53:0/4128462896 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e8019070 con 0x7f29f00cc620 2024-05-10T10:43:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f0160b00 con 0x7f29f00cc620 2024-05-10T10:43:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29ed7fa640 1 -- 172.21.15.53:0/4128462896 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8014650 con 0x7f29f00cc620 2024-05-10T10:43:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.796+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f00cce20 con 0x7f29f00cc620 2024-05-10T10:43:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.797+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f006a220 con 0x7f29f00cc620 2024-05-10T10:43:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.798+0000 7f29ed7fa640 1 -- 172.21.15.53:0/4128462896 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f29e80147f0 con 0x7f29f00cc620 2024-05-10T10:43:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.799+0000 7f29ed7fa640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29d4076300 0x7f29d40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.799+0000 7f29ed7fa640 1 -- 172.21.15.53:0/4128462896 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5921+0+0 (secure 0 0 0) 0x7f29e8097aa0 con 0x7f29f00cc620 2024-05-10T10:43:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.799+0000 7f29f4807640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29d4076300 0x7f29d40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:24.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.799+0000 7f29f4807640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29d4076300 0x7f29d40787c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f29e400ba00 tx=0x7f29e4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:24.813 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.803+0000 7f29ed7fa640 1 -- 172.21.15.53:0/4128462896 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f29e8061320 con 0x7f29f00cc620 2024-05-10T10:43:24.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:24.997+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f29f0160cd0 con 0x7f29f00cc620 2024-05-10T10:43:25.659 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.658+0000 7f29ed7fa640 1 -- 172.21.15.53:0/4128462896 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7f29e8060c20 con 0x7f29f00cc620 2024-05-10T10:43:25.659 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2024-05-10T10:43:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.661+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29d4076300 msgr2=0x7f29d40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.661+0000 7f29f6a6b640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29d4076300 0x7f29d40787c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f29e400ba00 tx=0x7f29e4030000 comp rx=0 tx=0).stop 2024-05-10T10:43:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.661+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29f00cc620 msgr2=0x7f29f00ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.661+0000 7f29f6a6b640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29f00cc620 0x7f29f00ba6d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f29f00d15f0 tx=0x7f29e8009f80 comp rx=0 tx=0).stop 2024-05-10T10:43:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 shutdown_connections 2024-05-10T10:43:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f29d4076300 0x7f29d40787c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29f00cc620 0x7f29f00ba6d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29f00cc1c0 0x7f29f00ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 --2- 172.21.15.53:0/4128462896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29e00039b0 0x7f29f0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 >> 172.21.15.53:0/4128462896 conn(0x7f29f00d50f0 msgr2=0x7f29f00d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 shutdown_connections 2024-05-10T10:43:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.662+0000 7f29f6a6b640 1 -- 172.21.15.53:0/4128462896 wait complete. 2024-05-10T10:43:25.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T10:43:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.791+0000 7f84ed899640 1 Processor -- start 2024-05-10T10:43:25.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.791+0000 7f84ed899640 1 -- start start 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84ed899640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f84e8162ce0 0x7f84e80b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84ed899640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f84e80b4e20 0x7f84e80d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84ed899640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e80d4190 con 0x7f84e8162900 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84ed899640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f84e80cfc40 con 0x7f84e8162ce0 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84ed899640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f84e80d3eb0 con 0x7f84e80b4e20 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84dffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f84e8162ce0 0x7f84e80b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84e6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f84e8162900 0x7f84e80d3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57972/0 (socket says 172.21.15.53:57972) 2024-05-10T10:43:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84e6ffd640 1 -- 172.21.15.53:0/3799203268 learned_addr learned my addr 172.21.15.53:0/3799203268 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84dffff640 1 --2- 172.21.15.53:0/3799203268 >> v2:172.21.15.53:3300/0 conn(0x7f84e8162ce0 0x7f84e80b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49778/0 (socket says 172.21.15.53:49778) 2024-05-10T10:43:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84e77fe640 1 --2- 172.21.15.53:0/3799203268 >> v2:172.21.15.135:3300/0 conn(0x7f84e80b4e20 0x7f84e80d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84dffff640 1 -- 172.21.15.53:0/3799203268 >> v2:172.21.15.135:3300/0 conn(0x7f84e80b4e20 msgr2=0x7f84e80d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84dffff640 1 --2- 172.21.15.53:0/3799203268 >> v2:172.21.15.135:3300/0 conn(0x7f84e80b4e20 0x7f84e80d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84dffff640 1 -- 172.21.15.53:0/3799203268 >> v1:172.21.15.53:6789/0 conn(0x7f84e8162900 legacy=0x7f84e80d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84e67fc640 1 -- 172.21.15.53:0/3799203268 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2250264979 0 0) 0x7f84d8001110 con 0x7f84e8162900 2024-05-10T10:43:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.792+0000 7f84dffff640 1 -- 172.21.15.53:0/3799203268 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e80cfee0 con 0x7f84e8162ce0 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.793+0000 7f84dffff640 1 --2- 172.21.15.53:0/3799203268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e80b48e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f84e808d760 tx=0x7f84d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9814b4578e474f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.793+0000 7f84e67fc640 1 -- 172.21.15.53:0/3799203268 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0037070 con 0x7f84e8162ce0 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.793+0000 7f84e67fc640 1 -- 172.21.15.53:0/3799203268 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d0032440 con 0x7f84e8162ce0 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.793+0000 7f84e67fc640 1 -- 172.21.15.53:0/3799203268 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d002ae60 con 0x7f84e8162ce0 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 -- 172.21.15.53:0/3799203268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 msgr2=0x7f84e80b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 --2- 172.21.15.53:0/3799203268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e80b48e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f84e808d760 tx=0x7f84d002a110 comp rx=0 tx=0).stop 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 -- 172.21.15.53:0/3799203268 shutdown_connections 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 --2- 172.21.15.53:0/3799203268 >> v2:172.21.15.135:3300/0 conn(0x7f84e80b4e20 0x7f84e80d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 --2- 172.21.15.53:0/3799203268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e80b48e0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 -- 172.21.15.53:0/3799203268 >> 172.21.15.53:0/3799203268 conn(0x7f84e809e430 msgr2=0x7f84e80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 -- 172.21.15.53:0/3799203268 shutdown_connections 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.794+0000 7f84ed899640 1 -- 172.21.15.53:0/3799203268 wait complete. 2024-05-10T10:43:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.795+0000 7f84ed899640 1 Processor -- start 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.795+0000 7f84ed899640 1 -- start start 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84ed899640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84e80b4e20 0x7f84e814ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84ed899640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e8162900 0x7f84e80bab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84ed899640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e81504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84ed899640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80b4180 con 0x7f84e8162ce0 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84ed899640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80d3b20 con 0x7f84e8162900 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84ed899640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80b4300 con 0x7f84e80b4e20 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e81504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e81504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49792/0 (socket says 172.21.15.53:49792) 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 -- 172.21.15.53:0/2865385949 learned_addr learned my addr 172.21.15.53:0/2865385949 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84dffff640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e8162900 0x7f84e80bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e6ffd640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84e80b4e20 0x7f84e814ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 -- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84e80b4e20 msgr2=0x7f84e814ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84e80b4e20 0x7f84e814ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 -- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e8162900 msgr2=0x7f84e80bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e8162900 0x7f84e80bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 -- 172.21.15.53:0/2865385949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e814f430 con 0x7f84e8162ce0 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84dffff640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e8162900 0x7f84e80bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.796+0000 7f84e77fe640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e81504d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007e40 tx=0x7f84e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.797+0000 7f84e4ff9640 1 -- 172.21.15.53:0/2865385949 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0014620 con 0x7f84e8162ce0 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.797+0000 7f84e4ff9640 1 -- 172.21.15.53:0/2865385949 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0019070 con 0x7f84e8162ce0 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.797+0000 7f84e4ff9640 1 -- 172.21.15.53:0/2865385949 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0014a20 con 0x7f84e8162ce0 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.797+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e814f710 con 0x7f84e8162ce0 2024-05-10T10:43:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.797+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e814fcd0 con 0x7f84e8162ce0 2024-05-10T10:43:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.798+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e806a220 con 0x7f84e8162ce0 2024-05-10T10:43:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.798+0000 7f84e4ff9640 1 -- 172.21.15.53:0/2865385949 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f84e0014bc0 con 0x7f84e8162ce0 2024-05-10T10:43:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.799+0000 7f84e4ff9640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f84c8076300 0x7f84c80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.799+0000 7f84e4ff9640 1 -- 172.21.15.53:0/2865385949 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f84e0097e30 con 0x7f84e8162ce0 2024-05-10T10:43:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.799+0000 7f84e6ffd640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f84c8076300 0x7f84c80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:25.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.800+0000 7f84e6ffd640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f84c8076300 0x7f84c80787c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000f60 tx=0x7f84d8007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:25.804 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.804+0000 7f84e4ff9640 1 -- 172.21.15.53:0/2865385949 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f84e0061850 con 0x7f84e8162ce0 2024-05-10T10:43:25.996 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.995+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f84e809e840 con 0x7f84e8162ce0 2024-05-10T10:43:25.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:25.998+0000 7f84e4ff9640 1 -- 172.21.15.53:0/2865385949 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f84e0061150 con 0x7f84e8162ce0 2024-05-10T10:43:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f84c8076300 msgr2=0x7f84c80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f84c8076300 0x7f84c80787c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000f60 tx=0x7f84d8007970 comp rx=0 tx=0).stop 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 msgr2=0x7f84e81504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e81504d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007e40 tx=0x7f84e000e4b0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 shutdown_connections 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f84c8076300 0x7f84c80787c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84e8162ce0 0x7f84e81504d0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e8162900 0x7f84e80bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 --2- 172.21.15.53:0/2865385949 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84e80b4e20 0x7f84e814ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 >> 172.21.15.53:0/2865385949 conn(0x7f84e809e430 msgr2=0x7f84e80b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 shutdown_connections 2024-05-10T10:43:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.000+0000 7f84ed899640 1 -- 172.21.15.53:0/2865385949 wait complete. 2024-05-10T10:43:26.012 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-10T10:43:26.115 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.115+0000 7f95da551640 1 Processor -- start 2024-05-10T10:43:26.116 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.115+0000 7f95da551640 1 -- start start 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95da551640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95d40d5d30 0x7f95d416a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95da551640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d40d30b0 con 0x7f95d415f3b0 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95da551640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d40d3390 con 0x7f95d415f790 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95da551640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f95d40c8c80 con 0x7f95d40d5d30 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95d8aee640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95d40d5d30 0x7f95d416a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95d3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f95d415f3b0 0x7f95d40d36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57986/0 (socket says 172.21.15.53:57986) 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95d3fff640 1 -- 172.21.15.53:0/2502954223 learned_addr learned my addr 172.21.15.53:0/2502954223 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:26.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95d8aee640 1 -- 172.21.15.53:0/2502954223 >> v1:172.21.15.135:6789/0 conn(0x7f95d415f790 legacy=0x7f95d40cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95d8aee640 1 -- 172.21.15.53:0/2502954223 >> v1:172.21.15.53:6789/0 conn(0x7f95d415f3b0 legacy=0x7f95d40d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95d2ffd640 1 -- 172.21.15.53:0/2502954223 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819685212 0 0) 0x7f95d40d30b0 con 0x7f95d415f3b0 2024-05-10T10:43:26.118 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.116+0000 7f95d8aee640 1 -- 172.21.15.53:0/2502954223 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d40c8f60 con 0x7f95d40d5d30 2024-05-10T10:43:26.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.117+0000 7f95d8aee640 1 --2- 172.21.15.53:0/2502954223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d416a0c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009d90 tx=0x7f95cc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533ba88d96807d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:26.119 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.117+0000 7f95d2ffd640 1 -- 172.21.15.53:0/2502954223 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007740 con 0x7f95d40d5d30 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.117+0000 7f95d2ffd640 1 -- 172.21.15.53:0/2502954223 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc0078e0 con 0x7f95d40d5d30 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.117+0000 7f95d2ffd640 1 -- 172.21.15.53:0/2502954223 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007cb0 con 0x7f95d40d5d30 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.118+0000 7f95da551640 1 -- 172.21.15.53:0/2502954223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 msgr2=0x7f95d416a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.118+0000 7f95da551640 1 --2- 172.21.15.53:0/2502954223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d416a0c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009d90 tx=0x7f95cc02a6f0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.118+0000 7f95da551640 1 -- 172.21.15.53:0/2502954223 shutdown_connections 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.118+0000 7f95da551640 1 --2- 172.21.15.53:0/2502954223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d416a0c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.118+0000 7f95da551640 1 -- 172.21.15.53:0/2502954223 >> 172.21.15.53:0/2502954223 conn(0x7f95d40d5130 msgr2=0x7f95d40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.118+0000 7f95da551640 1 -- 172.21.15.53:0/2502954223 shutdown_connections 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.118+0000 7f95da551640 1 -- 172.21.15.53:0/2502954223 wait complete. 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.119+0000 7f95da551640 1 Processor -- start 2024-05-10T10:43:26.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.119+0000 7f95da551640 1 -- start start 2024-05-10T10:43:26.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95da551640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:26.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95da551640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d415f3b0 0x7f95d40b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95da551640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95d40b8c70 0x7f95d41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95da551640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f95d40d3f80 con 0x7f95d40d5d30 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95da551640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95d40d3e00 con 0x7f95d415f3b0 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95da551640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f95d40ca770 con 0x7f95d40b8c70 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d414d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49814/0 (socket says 172.21.15.53:49814) 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 -- 172.21.15.53:0/1712995109 learned_addr learned my addr 172.21.15.53:0/1712995109 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d37fe640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d415f3b0 0x7f95d40b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d8aee640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95d40b8c70 0x7f95d41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 -- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95d40b8c70 msgr2=0x7f95d41532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95d40b8c70 0x7f95d41532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 -- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d415f3b0 msgr2=0x7f95d40b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d415f3b0 0x7f95d40b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 -- 172.21.15.53:0/1712995109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d40b9180 con 0x7f95d40d5d30 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d37fe640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d415f3b0 0x7f95d40b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d3fff640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d414d650 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c4d0 tx=0x7f95c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d0ff9640 1 -- 172.21.15.53:0/1712995109 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4018070 con 0x7f95d40d5d30 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d0ff9640 1 -- 172.21.15.53:0/1712995109 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c40074e0 con 0x7f95d40d5d30 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d40b9460 con 0x7f95d40d5d30 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.120+0000 7f95d0ff9640 1 -- 172.21.15.53:0/1712995109 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4004a20 con 0x7f95d40d5d30 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.121+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d4154000 con 0x7f95d40d5d30 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.121+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d406a220 con 0x7f95d40d5d30 2024-05-10T10:43:26.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.122+0000 7f95d0ff9640 1 -- 172.21.15.53:0/1712995109 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f95c401b070 con 0x7f95d40d5d30 2024-05-10T10:43:26.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.123+0000 7f95d0ff9640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95c0076300 0x7f95c00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:26.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.123+0000 7f95d0ff9640 1 -- 172.21.15.53:0/1712995109 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f95c40967c0 con 0x7f95d40d5d30 2024-05-10T10:43:26.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.123+0000 7f95d37fe640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95c0076300 0x7f95c00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:26.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.123+0000 7f95d37fe640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95c0076300 0x7f95c00787c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f95d40d0040 tx=0x7f95c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:26.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.127+0000 7f95d0ff9640 1 -- 172.21.15.53:0/1712995109 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f95c4060270 con 0x7f95d40d5d30 2024-05-10T10:43:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.400+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f95d40ba750 con 0x7f95d40d5d30 2024-05-10T10:43:26.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.664+0000 7f95d0ff9640 1 -- 172.21.15.53:0/1712995109 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v453) ==== 144+0+0 (secure 0 0 0) 0x7f95c405fb70 con 0x7f95d40d5d30 2024-05-10T10:43:26.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95c0076300 msgr2=0x7f95c00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95c0076300 0x7f95c00787c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f95d40d0040 tx=0x7f95c8009040 comp rx=0 tx=0).stop 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 msgr2=0x7f95d414d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d414d650 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c4d0 tx=0x7f95c400c990 comp rx=0 tx=0).stop 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 shutdown_connections 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f95c0076300 0x7f95c00787c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f95d40b8c70 0x7f95d41532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d415f3b0 0x7f95d40b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 --2- 172.21.15.53:0/1712995109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95d40d5d30 0x7f95d414d650 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 >> 172.21.15.53:0/1712995109 conn(0x7f95d40d5130 msgr2=0x7f95d415e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 shutdown_connections 2024-05-10T10:43:26.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:26.667+0000 7f95da551640 1 -- 172.21.15.53:0/1712995109 wait complete. 2024-05-10T10:43:26.679 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-10T10:43:26.679 INFO:tasks.cephfs_test_runner: 2024-05-10T10:43:26.679 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-10T10:43:26.680 INFO:tasks.cephfs_test_runner:Ran 7 tests in 562.663s 2024-05-10T10:43:26.680 INFO:tasks.cephfs_test_runner: 2024-05-10T10:43:26.680 INFO:tasks.cephfs_test_runner:OK 2024-05-10T10:43:26.680 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-10T10:43:26.691 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-10T10:43:26.699 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-10T10:43:26.699 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:26.699 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:26.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:43:26.717 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-10T10:43:26.717 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:26.775 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17859 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T10:43:26.775 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:43:26.775 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:26.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:43:26.794 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-05-10T10:43:26.794 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T10:43:26.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056323,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16847 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248824k,nr_inodes=812206,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497656k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T10:43:26.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:26.853 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:43:26.869 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T10:43:26.869 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T10:43:26.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:26.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T10:43:26.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:26.937 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T10:43:27.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-05-10T10:43:27.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.124 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-05-10T10:43:27.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-05-10T10:43:27.210 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-05-10T10:43:27.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:43:27.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.212 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-05-10T10:43:27.231 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T10:43:27.231 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T10:43:27.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.232 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T10:43:27.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.301 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T10:43:27.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.370 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-05-10T10:43:27.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.474 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-05-10T10:43:27.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T10:43:27.491 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-05-10T10:43:27.559 INFO:teuthology.orchestra.run.smithi135.stderr:Cannot find device "ceph-brx" 2024-05-10T10:43:27.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:43:27.560 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-10T10:43:27.570 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T10:43:27.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:43:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.680+0000 7f8934237640 1 Processor -- start 2024-05-10T10:43:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.680+0000 7f8934237640 1 -- start start 2024-05-10T10:43:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f8934237640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f892c0c6c80 0x7f892c0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f8934237640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c06f1c0 con 0x7f892c0cee40 2024-05-10T10:43:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f8934237640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c06f4a0 con 0x7f892c1598c0 2024-05-10T10:43:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f8934237640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f892c06f780 con 0x7f892c0c6c80 2024-05-10T10:43:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f89327d4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f892c0cee40 0x7f892c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57994/0 (socket says 172.21.15.53:57994) 2024-05-10T10:43:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f89327d4640 1 -- 172.21.15.53:0/1013539110 learned_addr learned my addr 172.21.15.53:0/1013539110 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f89317d2640 1 --2- 172.21.15.53:0/1013539110 >> v2:172.21.15.135:3300/0 conn(0x7f892c0c6c80 0x7f892c0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f8930fd1640 1 -- 172.21.15.53:0/1013539110 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855137273 0 0) 0x7f892c06f1c0 con 0x7f892c0cee40 2024-05-10T10:43:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.681+0000 7f8930fd1640 1 -- 172.21.15.53:0/1013539110 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8918003520 con 0x7f892c0cee40 2024-05-10T10:43:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.682+0000 7f89317d2640 1 -- 172.21.15.53:0/1013539110 >> v1:172.21.15.135:6789/0 conn(0x7f892c1598c0 legacy=0x7f892c06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.682+0000 7f89317d2640 1 -- 172.21.15.53:0/1013539110 >> v1:172.21.15.53:6789/0 conn(0x7f892c0cee40 legacy=0x7f892c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.682+0000 7f89317d2640 1 -- 172.21.15.53:0/1013539110 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c06fa60 con 0x7f892c0c6c80 2024-05-10T10:43:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.682+0000 7f89317d2640 1 --2- 172.21.15.53:0/1013539110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0c6c80 0x7f892c0c7110 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f891c0097a0 tx=0x7f891c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb3655858ba63d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.682+0000 7f8930fd1640 1 -- 172.21.15.53:0/1013539110 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c037070 con 0x7f892c0c6c80 2024-05-10T10:43:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8930fd1640 1 -- 172.21.15.53:0/1013539110 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c032440 con 0x7f892c0c6c80 2024-05-10T10:43:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8930fd1640 1 -- 172.21.15.53:0/1013539110 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c032810 con 0x7f892c0c6c80 2024-05-10T10:43:27.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8934237640 1 -- 172.21.15.53:0/1013539110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0c6c80 msgr2=0x7f892c0c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:27.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8934237640 1 --2- 172.21.15.53:0/1013539110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0c6c80 0x7f892c0c7110 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f891c0097a0 tx=0x7f891c02a230 comp rx=0 tx=0).stop 2024-05-10T10:43:27.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8934237640 1 -- 172.21.15.53:0/1013539110 shutdown_connections 2024-05-10T10:43:27.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8934237640 1 --2- 172.21.15.53:0/1013539110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0c6c80 0x7f892c0c7110 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:27.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8934237640 1 -- 172.21.15.53:0/1013539110 >> 172.21.15.53:0/1013539110 conn(0x7f892c0b2900 msgr2=0x7f892c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:27.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8934237640 1 -- 172.21.15.53:0/1013539110 shutdown_connections 2024-05-10T10:43:27.685 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.683+0000 7f8934237640 1 -- 172.21.15.53:0/1013539110 wait complete. 2024-05-10T10:43:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.684+0000 7f8934237640 1 Processor -- start 2024-05-10T10:43:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.684+0000 7f8934237640 1 -- start start 2024-05-10T10:43:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f8934237640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f892c0c6c80 0x7f892c073060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f8934237640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0cee40 0x7f892c1559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f8934237640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f892c1598c0 0x7f892c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f8934237640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f892c06ee10 con 0x7f892c1598c0 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f8934237640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f892c15c6e0 con 0x7f892c0cee40 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f8934237640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f892c073cb0 con 0x7f892c0c6c80 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f89327d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f892c1598c0 0x7f892c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f89327d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f892c1598c0 0x7f892c15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49830/0 (socket says 172.21.15.53:49830) 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f89327d4640 1 -- 172.21.15.53:0/2652540214 learned_addr learned my addr 172.21.15.53:0/2652540214 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f8931fd3640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f892c0c6c80 0x7f892c073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.685+0000 7f89317d2640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0cee40 0x7f892c1559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89327d4640 1 -- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f892c0c6c80 msgr2=0x7f892c073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89327d4640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f892c0c6c80 0x7f892c073060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89327d4640 1 -- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0cee40 msgr2=0x7f892c1559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89327d4640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0cee40 0x7f892c1559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89327d4640 1 -- 172.21.15.53:0/2652540214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c15b410 con 0x7f892c1598c0 2024-05-10T10:43:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89327d4640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f892c1598c0 0x7f892c15e4e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f892400cb40 tx=0x7f892400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89227fc640 1 -- 172.21.15.53:0/2652540214 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924017070 con 0x7f892c1598c0 2024-05-10T10:43:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89227fc640 1 -- 172.21.15.53:0/2652540214 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892400aca0 con 0x7f892c1598c0 2024-05-10T10:43:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c0736f0 con 0x7f892c1598c0 2024-05-10T10:43:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f89227fc640 1 -- 172.21.15.53:0/2652540214 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924012620 con 0x7f892c1598c0 2024-05-10T10:43:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.686+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c15b6b0 con 0x7f892c1598c0 2024-05-10T10:43:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.687+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c06a220 con 0x7f892c1598c0 2024-05-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.687+0000 7f89227fc640 1 -- 172.21.15.53:0/2652540214 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f892401a070 con 0x7f892c1598c0 2024-05-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.688+0000 7f89227fc640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f89100762d0 0x7f8910078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.689+0000 7f89227fc640 1 -- 172.21.15.53:0/2652540214 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f8924097240 con 0x7f892c1598c0 2024-05-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.689+0000 7f8931fd3640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f89100762d0 0x7f8910078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.689+0000 7f8931fd3640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f89100762d0 0x7f8910078790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f89280016e0 tx=0x7f8928009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.693+0000 7f89227fc640 1 -- 172.21.15.53:0/2652540214 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f89240606a0 con 0x7f892c1598c0 2024-05-10T10:43:27.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.878+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f892c06e4e0 con 0x7f89100762d0 2024-05-10T10:43:27.880 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.879+0000 7f89227fc640 1 -- 172.21.15.53:0/2652540214 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13399 (secure 0 0 0) 0x7f892c06e4e0 con 0x7f89100762d0 2024-05-10T10:43:27.880 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:27.880 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:43:27.882 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f89100762d0 msgr2=0x7f8910078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f89100762d0 0x7f8910078790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f89280016e0 tx=0x7f8928009040 comp rx=0 tx=0).stop 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f892c1598c0 msgr2=0x7f892c15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f892c1598c0 0x7f892c15e4e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f892400cb40 tx=0x7f892400a060 comp rx=0 tx=0).stop 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 shutdown_connections 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f89100762d0 0x7f8910078790 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f892c1598c0 0x7f892c15e4e0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f892c0cee40 0x7f892c1559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.881+0000 7f8934237640 1 --2- 172.21.15.53:0/2652540214 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f892c0c6c80 0x7f892c073060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.882+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 >> 172.21.15.53:0/2652540214 conn(0x7f892c0b2900 msgr2=0x7f892c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.882+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 shutdown_connections 2024-05-10T10:43:27.884 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.882+0000 7f8934237640 1 -- 172.21.15.53:0/2652540214 wait complete. 2024-05-10T10:43:27.893 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-05-10T10:43:26.304521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278772,"kb_used_data":29348,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695948,"statfs":{"total":773094113280,"available":772808650752,"internally_reserved":0,"allocated":30052352,"data_stored":23567735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-256,"num_write":-172,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001624"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":316,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-10T10:43:22.395626+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:43:22.395626+0000","last_peered":"2024-05-10T10:43:22.395626+0000","last_clean":"2024-05-10T10:43:22.395626+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:43:22.395626+0000","last_undegraded":"2024-05-10T10:43:22.395626+0000","last_fullsized":"2024-05-10T10:43:22.395626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2121728,"data_stored":1295616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":5560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":5693440,"data_stored":4892706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":5576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":5709824,"data_stored":4898929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":5576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":5709824,"data_stored":4894965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1511424,"data_stored":700683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":704470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":5564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":5697536,"data_stored":4892151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2097152,"data_stored":1288215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:43:27.894 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:43:27.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.997+0000 7f31ccf89640 1 Processor -- start 2024-05-10T10:43:27.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.997+0000 7f31ccf89640 1 -- start start 2024-05-10T10:43:27.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31ccf89640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31c815f3b0 0x7f31c80d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31ccf89640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f790 0x7f31c80d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31ccf89640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31c80d5d30 0x7f31c80c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31ccf89640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f31c80d30b0 con 0x7f31c80d5d30 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31ccf89640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f31c80d3390 con 0x7f31c815f3b0 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31ccf89640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31c80c8c80 con 0x7f31c815f790 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31c6d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31c80d5d30 0x7f31c80c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31c6d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31c80d5d30 0x7f31c80c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49846/0 (socket says 172.21.15.53:49846) 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.998+0000 7f31c6d9d640 1 -- 172.21.15.53:0/793905133 learned_addr learned my addr 172.21.15.53:0/793905133 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c5d9b640 1 --2- 172.21.15.53:0/793905133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f790 0x7f31c80d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c659c640 1 --2- 172.21.15.53:0/793905133 >> v2:172.21.15.135:3300/0 conn(0x7f31c815f3b0 0x7f31c80d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c6d9d640 1 -- 172.21.15.53:0/793905133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f790 msgr2=0x7f31c80d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c6d9d640 1 --2- 172.21.15.53:0/793905133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f790 0x7f31c80d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c6d9d640 1 -- 172.21.15.53:0/793905133 >> v2:172.21.15.135:3300/0 conn(0x7f31c815f3b0 msgr2=0x7f31c80d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c6d9d640 1 --2- 172.21.15.53:0/793905133 >> v2:172.21.15.135:3300/0 conn(0x7f31c815f3b0 0x7f31c80d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c6d9d640 1 -- 172.21.15.53:0/793905133 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c80c8f60 con 0x7f31c80d5d30 2024-05-10T10:43:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c659c640 1 --2- 172.21.15.53:0/793905133 >> v2:172.21.15.135:3300/0 conn(0x7f31c815f3b0 0x7f31c80d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:27.999+0000 7f31c6d9d640 1 --2- 172.21.15.53:0/793905133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c80c9760 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f31c808e820 tx=0x7f31b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a0ddec9a27cd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31c559a640 1 -- 172.21.15.53:0/793905133 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8037070 con 0x7f31c80d5d30 2024-05-10T10:43:28.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31c559a640 1 -- 172.21.15.53:0/793905133 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b8032440 con 0x7f31c80d5d30 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31c559a640 1 -- 172.21.15.53:0/793905133 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8032810 con 0x7f31c80d5d30 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 -- 172.21.15.53:0/793905133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 msgr2=0x7f31c80c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 --2- 172.21.15.53:0/793905133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c80c9760 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f31c808e820 tx=0x7f31b802a1e0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 -- 172.21.15.53:0/793905133 shutdown_connections 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 --2- 172.21.15.53:0/793905133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c80c9760 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 --2- 172.21.15.53:0/793905133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f790 0x7f31c80d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 --2- 172.21.15.53:0/793905133 >> v2:172.21.15.135:3300/0 conn(0x7f31c815f3b0 0x7f31c80d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 -- 172.21.15.53:0/793905133 >> 172.21.15.53:0/793905133 conn(0x7f31c80d5130 msgr2=0x7f31c80d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 -- 172.21.15.53:0/793905133 shutdown_connections 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.000+0000 7f31ccf89640 1 -- 172.21.15.53:0/793905133 wait complete. 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 Processor -- start 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 -- start start 2024-05-10T10:43:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c8154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f3b0 0x7f31c8155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31c8155db0 0x7f31c8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80c9e20 con 0x7f31c80d5d30 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80c9ca0 con 0x7f31c8155db0 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31ccf89640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31c80c9fa0 con 0x7f31c815f3b0 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31c659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c8154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31c659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c8154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49860/0 (socket says 172.21.15.53:49860) 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.002+0000 7f31c659c640 1 -- 172.21.15.53:0/26430819 learned_addr learned my addr 172.21.15.53:0/26430819 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c5d9b640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f3b0 0x7f31c8155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c6d9d640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31c8155db0 0x7f31c8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c659c640 1 -- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f3b0 msgr2=0x7f31c8155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c659c640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f3b0 0x7f31c8155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c659c640 1 -- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31c8155db0 msgr2=0x7f31c8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c659c640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31c8155db0 0x7f31c8160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c659c640 1 -- 172.21.15.53:0/26430819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c80cd880 con 0x7f31c80d5d30 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c5d9b640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f3b0 0x7f31c8155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c6d9d640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31c8155db0 0x7f31c8160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31c659c640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c8154530 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f31c0007ef0 tx=0x7f31c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31beffd640 1 -- 172.21.15.53:0/26430819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c001a070 con 0x7f31c80d5d30 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31beffd640 1 -- 172.21.15.53:0/26430819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c0003b90 con 0x7f31c80d5d30 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c8156360 con 0x7f31c80d5d30 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31beffd640 1 -- 172.21.15.53:0/26430819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0004030 con 0x7f31c80d5d30 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.003+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c80cde00 con 0x7f31c80d5d30 2024-05-10T10:43:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.004+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c806a220 con 0x7f31c80d5d30 2024-05-10T10:43:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.005+0000 7f31beffd640 1 -- 172.21.15.53:0/26430819 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f31c001d070 con 0x7f31c80d5d30 2024-05-10T10:43:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.005+0000 7f31beffd640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31a8076300 0x7f31a80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.005+0000 7f31beffd640 1 -- 172.21.15.53:0/26430819 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f31c00976e0 con 0x7f31c80d5d30 2024-05-10T10:43:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.005+0000 7f31c5d9b640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31a8076300 0x7f31a80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.006+0000 7f31c5d9b640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31a8076300 0x7f31a80787c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f31c80cd670 tx=0x7f31b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.010+0000 7f31beffd640 1 -- 172.21.15.53:0/26430819 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f31c0010070 con 0x7f31c80d5d30 2024-05-10T10:43:28.196 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.196+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31c80baf10 con 0x7f31a8076300 2024-05-10T10:43:28.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.197+0000 7f31beffd640 1 -- 172.21.15.53:0/26430819 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13399 (secure 0 0 0) 0x7f31c80baf10 con 0x7f31a8076300 2024-05-10T10:43:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:28.198 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31a8076300 msgr2=0x7f31a80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31a8076300 0x7f31a80787c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f31c80cd670 tx=0x7f31b0008040 comp rx=0 tx=0).stop 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 msgr2=0x7f31c8154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c8154530 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f31c0007ef0 tx=0x7f31c000d4a0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 shutdown_connections 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31a8076300 0x7f31a80787c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31c8155db0 0x7f31c8160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31c815f3b0 0x7f31c8155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 --2- 172.21.15.53:0/26430819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31c80d5d30 0x7f31c8154530 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 >> 172.21.15.53:0/26430819 conn(0x7f31c80d5130 msgr2=0x7f31c80d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:28.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 shutdown_connections 2024-05-10T10:43:28.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.199+0000 7f31ccf89640 1 -- 172.21.15.53:0/26430819 wait complete. 2024-05-10T10:43:28.211 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-05-10T10:43:26.304521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278772,"kb_used_data":29348,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695948,"statfs":{"total":773094113280,"available":772808650752,"internally_reserved":0,"allocated":30052352,"data_stored":23567735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-256,"num_write":-172,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001624"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":316,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-05-10T10:43:22.395626+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:43:22.395626+0000","last_peered":"2024-05-10T10:43:22.395626+0000","last_clean":"2024-05-10T10:43:22.395626+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:43:22.395626+0000","last_undegraded":"2024-05-10T10:43:22.395626+0000","last_fullsized":"2024-05-10T10:43:22.395626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2121728,"data_stored":1295616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":5560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":5693440,"data_stored":4892706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":5576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":5709824,"data_stored":4898929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":5576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":5709824,"data_stored":4894965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1511424,"data_stored":700683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":704470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":5564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":5697536,"data_stored":4892151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2097152,"data_stored":1288215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:43:28.211 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T10:43:28.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:43:28.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 Processor -- start 2024-05-10T10:43:28.316 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 -- start start 2024-05-10T10:43:28.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c06d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd90c15f8c0 0x7fd90c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd90c06e450 0x7fd90c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd90c0b23f0 con 0x7fd90c15f8c0 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd90c0b4150 con 0x7fd90c06e450 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.316+0000 7fd912558640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c06e8c0 con 0x7fd90c15f4e0 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd90c15f8c0 0x7fd90c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd90c15f8c0 0x7fd90c06df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49862/0 (socket says 172.21.15.53:49862) 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 -- 172.21.15.53:0/1560802237 learned_addr learned my addr 172.21.15.53:0/1560802237 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90bfff640 1 --2- 172.21.15.53:0/1560802237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c06d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd910af5640 1 --2- 172.21.15.53:0/1560802237 >> v2:172.21.15.135:3300/0 conn(0x7fd90c06e450 0x7fd90c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 -- 172.21.15.53:0/1560802237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 msgr2=0x7fd90c06d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 --2- 172.21.15.53:0/1560802237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c06d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 -- 172.21.15.53:0/1560802237 >> v2:172.21.15.135:3300/0 conn(0x7fd90c06e450 msgr2=0x7fd90c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 --2- 172.21.15.53:0/1560802237 >> v2:172.21.15.135:3300/0 conn(0x7fd90c06e450 0x7fd90c0afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd90b7fe640 1 -- 172.21.15.53:0/1560802237 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c0b3540 con 0x7fd90c15f8c0 2024-05-10T10:43:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.317+0000 7fd910af5640 1 --2- 172.21.15.53:0/1560802237 >> v2:172.21.15.135:3300/0 conn(0x7fd90c06e450 0x7fd90c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd90b7fe640 1 --2- 172.21.15.53:0/1560802237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c06df10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd90c073990 tx=0x7fd90002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08513303f4c71d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd90affd640 1 -- 172.21.15.53:0/1560802237 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900037070 con 0x7fd90c15f8c0 2024-05-10T10:43:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd90affd640 1 -- 172.21.15.53:0/1560802237 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900032440 con 0x7fd90c15f8c0 2024-05-10T10:43:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd90affd640 1 -- 172.21.15.53:0/1560802237 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900032810 con 0x7fd90c15f8c0 2024-05-10T10:43:28.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 -- 172.21.15.53:0/1560802237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 msgr2=0x7fd90c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 --2- 172.21.15.53:0/1560802237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c06df10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd90c073990 tx=0x7fd90002a140 comp rx=0 tx=0).stop 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 -- 172.21.15.53:0/1560802237 shutdown_connections 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 --2- 172.21.15.53:0/1560802237 >> v2:172.21.15.135:3300/0 conn(0x7fd90c06e450 0x7fd90c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 --2- 172.21.15.53:0/1560802237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c06df10 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 --2- 172.21.15.53:0/1560802237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c06d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 -- 172.21.15.53:0/1560802237 >> 172.21.15.53:0/1560802237 conn(0x7fd90c0d2df0 msgr2=0x7fd90c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 -- 172.21.15.53:0/1560802237 shutdown_connections 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.318+0000 7fd912558640 1 -- 172.21.15.53:0/1560802237 wait complete. 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.320+0000 7fd912558640 1 Processor -- start 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.320+0000 7fd912558640 1 -- start start 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.320+0000 7fd912558640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c06e450 0x7fd90c14e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.320+0000 7fd912558640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.320+0000 7fd912558640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd912558640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c0b3de0 con 0x7fd90c15f8c0 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd912558640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c0b3b00 con 0x7fd90c06e450 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd912558640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c0b45e0 con 0x7fd90c15f4e0 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c0c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49876/0 (socket says 172.21.15.53:49876) 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 -- 172.21.15.53:0/1272682784 learned_addr learned my addr 172.21.15.53:0/1272682784 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd90b7fe640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd90bfff640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c06e450 0x7fd90c14e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 -- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 msgr2=0x7fd90c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c0c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 -- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c06e450 msgr2=0x7fd90c14e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c06e450 0x7fd90c14e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 -- 172.21.15.53:0/1272682784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c0c8a00 con 0x7fd90c15f8c0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd90bfff640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c06e450 0x7fd90c14e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd910af5640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c0c83d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd904007db0 tx=0x7fd90400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd908ff9640 1 -- 172.21.15.53:0/1272682784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90401a070 con 0x7fd90c15f8c0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd908ff9640 1 -- 172.21.15.53:0/1272682784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd904003990 con 0x7fd90c15f8c0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd908ff9640 1 -- 172.21.15.53:0/1272682784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904003d60 con 0x7fd90c15f8c0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c0c8ce0 con 0x7fd90c15f8c0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.321+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c0c92a0 con 0x7fd90c15f8c0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.322+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c06a220 con 0x7fd90c15f8c0 2024-05-10T10:43:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.323+0000 7fd908ff9640 1 -- 172.21.15.53:0/1272682784 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd90401d070 con 0x7fd90c15f8c0 2024-05-10T10:43:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.324+0000 7fd908ff9640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd8f80762d0 0x7fd8f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.324+0000 7fd908ff9640 1 -- 172.21.15.53:0/1272682784 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7fd904097740 con 0x7fd90c15f8c0 2024-05-10T10:43:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.324+0000 7fd90bfff640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd8f80762d0 0x7fd8f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.324+0000 7fd90bfff640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd8f80762d0 0x7fd8f8078790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc006fd0 tx=0x7fd8fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.328 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.328+0000 7fd908ff9640 1 -- 172.21.15.53:0/1272682784 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fd904010070 con 0x7fd90c15f8c0 2024-05-10T10:43:28.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.518+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd90c15eb70 con 0x7fd8f80762d0 2024-05-10T10:43:28.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.519+0000 7fd908ff9640 1 -- 172.21.15.53:0/1272682784 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13502 (secure 0 0 0) 0x7fd90c15eb70 con 0x7fd8f80762d0 2024-05-10T10:43:28.520 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:28.520 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:43:28.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd8f80762d0 msgr2=0x7fd8f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd8f80762d0 0x7fd8f8078790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc006fd0 tx=0x7fd8fc008040 comp rx=0 tx=0).stop 2024-05-10T10:43:28.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 msgr2=0x7fd90c0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.522 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c0c83d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd904007db0 tx=0x7fd90400d4a0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 shutdown_connections 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fd8f80762d0 0x7fd8f8078790 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd90c15f8c0 0x7fd90c0c83d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd90c15f4e0 0x7fd90c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 --2- 172.21.15.53:0/1272682784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd90c06e450 0x7fd90c14e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 >> 172.21.15.53:0/1272682784 conn(0x7fd90c0d2df0 msgr2=0x7fd90c0d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 shutdown_connections 2024-05-10T10:43:28.523 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.521+0000 7fd912558640 1 -- 172.21.15.53:0/1272682784 wait complete. 2024-05-10T10:43:28.533 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-05-10T10:43:28.304919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266820,"kb_used_data":13192,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707900,"statfs":{"total":773094113280,"available":772820889600,"internally_reserved":0,"allocated":13508608,"data_stored":6918024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-256,"num_write":-172,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001643"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":320,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-10T10:43:25.663135+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:43:25.663135+0000","last_peered":"2024-05-10T10:43:25.663135+0000","last_clean":"2024-05-10T10:43:25.663135+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:43:25.663135+0000","last_undegraded":"2024-05-10T10:43:25.663135+0000","last_fullsized":"2024-05-10T10:43:25.663135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2134016,"data_stored":1307529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37432,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334408,"statfs":{"total":96636764160,"available":96598433792,"internally_reserved":0,"allocated":2134016,"data_stored":1307529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:43:28.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T10:43:28.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.637+0000 7f5bd93b6640 1 Processor -- start 2024-05-10T10:43:28.637 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.637+0000 7f5bd93b6640 1 -- start start 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.637+0000 7f5bd93b6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5bd4162750 0x7f5bd40d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.637+0000 7f5bd93b6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5bd40d5cb0 0x7f5bd40d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.637+0000 7f5bd93b6640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd40b45f0 con 0x7f5bd4162b30 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.637+0000 7f5bd93b6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd40b4f20 con 0x7f5bd40d5cb0 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.637+0000 7f5bd93b6640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd40d0fe0 con 0x7f5bd4162750 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5bd40d5cb0 0x7f5bd40d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5bd40d5cb0 0x7f5bd40d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49892/0 (socket says 172.21.15.53:49892) 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 -- 172.21.15.53:0/3662976341 learned_addr learned my addr 172.21.15.53:0/3662976341 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 -- 172.21.15.53:0/3662976341 >> v2:172.21.15.135:3300/0 conn(0x7f5bd4162750 msgr2=0x7f5bd40d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 --2- 172.21.15.53:0/3662976341 >> v2:172.21.15.135:3300/0 conn(0x7f5bd4162750 0x7f5bd40d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 -- 172.21.15.53:0/3662976341 >> v1:172.21.15.135:6789/0 conn(0x7f5bd4162b30 legacy=0x7f5bd40d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 -- 172.21.15.53:0/3662976341 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd40b48d0 con 0x7f5bd40d5cb0 2024-05-10T10:43:28.640 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.638+0000 7f5bd27fc640 1 --2- 172.21.15.53:0/3662976341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40d5cb0 0x7f5bd40d3630 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b7c40 tx=0x7f5bbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce3e57f7850f4a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd1ffb640 1 -- 172.21.15.53:0/3662976341 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc037070 con 0x7f5bd40d5cb0 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd1ffb640 1 -- 172.21.15.53:0/3662976341 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc032440 con 0x7f5bd40d5cb0 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd1ffb640 1 -- 172.21.15.53:0/3662976341 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc032810 con 0x7f5bd40d5cb0 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 -- 172.21.15.53:0/3662976341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40d5cb0 msgr2=0x7f5bd40d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/3662976341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40d5cb0 0x7f5bd40d3630 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b7c40 tx=0x7f5bbc02a1a0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 -- 172.21.15.53:0/3662976341 shutdown_connections 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/3662976341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40d5cb0 0x7f5bd40d3630 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/3662976341 >> v2:172.21.15.135:3300/0 conn(0x7f5bd4162750 0x7f5bd40d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 -- 172.21.15.53:0/3662976341 >> 172.21.15.53:0/3662976341 conn(0x7f5bd409e3b0 msgr2=0x7f5bd40d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 -- 172.21.15.53:0/3662976341 shutdown_connections 2024-05-10T10:43:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.639+0000 7f5bd93b6640 1 -- 172.21.15.53:0/3662976341 wait complete. 2024-05-10T10:43:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 Processor -- start 2024-05-10T10:43:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 -- start start 2024-05-10T10:43:28.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bd4162750 0x7f5bd40cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5bd4162b30 0x7f5bd415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40ceb50 0x7f5bd40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd40d0d30 con 0x7f5bd40ceb50 2024-05-10T10:43:28.644 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd40d0bb0 con 0x7f5bd4162750 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.641+0000 7f5bd93b6640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd40b63c0 con 0x7f5bd4162b30 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40ceb50 0x7f5bd40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40ceb50 0x7f5bd40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49904/0 (socket says 172.21.15.53:49904) 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 -- 172.21.15.53:0/19482 learned_addr learned my addr 172.21.15.53:0/19482 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd2ffd640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bd4162750 0x7f5bd40cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd27fc640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5bd4162b30 0x7f5bd415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 -- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5bd4162b30 msgr2=0x7f5bd415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5bd4162b30 0x7f5bd415b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 -- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bd4162750 msgr2=0x7f5bd40cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bd4162750 0x7f5bd40cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 -- 172.21.15.53:0/19482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd415b780 con 0x7f5bd40ceb50 2024-05-10T10:43:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd37fe640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40ceb50 0x7f5bd40cc160 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00b990 tx=0x7f5bcc00bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bcb7fe640 1 -- 172.21.15.53:0/19482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc019070 con 0x7f5bd40ceb50 2024-05-10T10:43:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bcb7fe640 1 -- 172.21.15.53:0/19482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc00caa0 con 0x7f5bd40ceb50 2024-05-10T10:43:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd40cc6a0 con 0x7f5bd40ceb50 2024-05-10T10:43:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bcb7fe640 1 -- 172.21.15.53:0/19482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc014440 con 0x7f5bd40ceb50 2024-05-10T10:43:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.642+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd415a760 con 0x7f5bd40ceb50 2024-05-10T10:43:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.643+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd406a220 con 0x7f5bd40ceb50 2024-05-10T10:43:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.644+0000 7f5bcb7fe640 1 -- 172.21.15.53:0/19482 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5bcc01c070 con 0x7f5bd40ceb50 2024-05-10T10:43:28.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.644+0000 7f5bcb7fe640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5bb40762d0 0x7f5bb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:28.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.645+0000 7f5bcb7fe640 1 -- 172.21.15.53:0/19482 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f5bcc097130 con 0x7f5bd40ceb50 2024-05-10T10:43:28.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.645+0000 7f5bd2ffd640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5bb40762d0 0x7f5bb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:28.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.645+0000 7f5bd2ffd640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5bb40762d0 0x7f5bb4078790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009990 tx=0x7f5bc4009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:28.649 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.649+0000 7f5bcb7fe640 1 -- 172.21.15.53:0/19482 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f5bcc060510 con 0x7f5bd40ceb50 2024-05-10T10:43:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.842+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bd4161b70 con 0x7f5bd40ceb50 2024-05-10T10:43:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.843+0000 7f5bcb7fe640 1 -- 172.21.15.53:0/19482 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+13559 (secure 0 0 0) 0x7f5bcc064810 con 0x7f5bd40ceb50 2024-05-10T10:43:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:28.844 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":123,"fsid":"543b513e-90c7-47ff-9cdd-f7fa4d978734","created":"2024-05-10T10:33:17.760086+0000","modified":"2024-05-10T10:43:25.656911+0000","last_up_change":"2024-05-10T10:33:27.436094+0000","last_in_change":"2024-05-10T10:33:19.895364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T10:33:28.171977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48a726a-57e3-4859-87d0-db3e2bf78036","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6804","nonce":1634417257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6808","nonce":1634417257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6815","nonce":1634417257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1634417257},{"type":"v1","addr":"172.21.15.53:6813","nonce":1634417257}]},"public_addr":"172.21.15.53:6804/1634417257","cluster_addr":"172.21.15.53:6808/1634417257","heartbeat_back_addr":"172.21.15.53:6815/1634417257","heartbeat_front_addr":"172.21.15.53:6813/1634417257","state":["exists","up"]},{"osd":1,"uuid":"59347929-0d1d-4f6b-b017-74aeb6cdea40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6825","nonce":2705510088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6827","nonce":2705510088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6831","nonce":2705510088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2705510088},{"type":"v1","addr":"172.21.15.53:6829","nonce":2705510088}]},"public_addr":"172.21.15.53:6825/2705510088","cluster_addr":"172.21.15.53:6827/2705510088","heartbeat_back_addr":"172.21.15.53:6831/2705510088","heartbeat_front_addr":"172.21.15.53:6829/2705510088","state":["exists","up"]},{"osd":2,"uuid":"66b03867-514f-45f6-bb2b-312e9595f5c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6801","nonce":2235813461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6803","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6805","nonce":2235813461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6811","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6812","nonce":2235813461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6807","nonce":2235813461},{"type":"v1","addr":"172.21.15.53:6809","nonce":2235813461}]},"public_addr":"172.21.15.53:6801/2235813461","cluster_addr":"172.21.15.53:6805/2235813461","heartbeat_back_addr":"172.21.15.53:6812/2235813461","heartbeat_front_addr":"172.21.15.53:6809/2235813461","state":["exists","up"]},{"osd":3,"uuid":"72678bd2-e32b-4271-9545-28b19dce7a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6817","nonce":2871564334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6819","nonce":2871564334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6823","nonce":2871564334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2871564334},{"type":"v1","addr":"172.21.15.53:6821","nonce":2871564334}]},"public_addr":"172.21.15.53:6817/2871564334","cluster_addr":"172.21.15.53:6819/2871564334","heartbeat_back_addr":"172.21.15.53:6823/2871564334","heartbeat_front_addr":"172.21.15.53:6821/2871564334","state":["exists","up"]},{"osd":4,"uuid":"d9463593-fb16-49dd-8f5d-c1324a27b3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6801","nonce":3517077655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6803","nonce":3517077655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6807","nonce":3517077655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3517077655},{"type":"v1","addr":"172.21.15.135:6805","nonce":3517077655}]},"public_addr":"172.21.15.135:6801/3517077655","cluster_addr":"172.21.15.135:6803/3517077655","heartbeat_back_addr":"172.21.15.135:6807/3517077655","heartbeat_front_addr":"172.21.15.135:6805/3517077655","state":["exists","up"]},{"osd":5,"uuid":"2c3f1892-fe80-41af-8bb4-cd35fec9c32d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6825","nonce":3244007917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6827","nonce":3244007917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6831","nonce":3244007917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3244007917},{"type":"v1","addr":"172.21.15.135:6829","nonce":3244007917}]},"public_addr":"172.21.15.135:6825/3244007917","cluster_addr":"172.21.15.135:6827/3244007917","heartbeat_back_addr":"172.21.15.135:6831/3244007917","heartbeat_front_addr":"172.21.15.135:6829/3244007917","state":["exists","up"]},{"osd":6,"uuid":"128a5eac-c0c4-4f21-8531-d626e5edf1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6810","nonce":1123855098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6814","nonce":1123855098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6822","nonce":1123855098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1123855098},{"type":"v1","addr":"172.21.15.135:6818","nonce":1123855098}]},"public_addr":"172.21.15.135:6810/1123855098","cluster_addr":"172.21.15.135:6814/1123855098","heartbeat_back_addr":"172.21.15.135:6822/1123855098","heartbeat_front_addr":"172.21.15.135:6818/1123855098","state":["exists","up"]},{"osd":7,"uuid":"aa88b628-2dcd-4305-81c5-b932a094c0cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6811","nonce":751247261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6813","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6815","nonce":751247261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6823","nonce":751247261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":751247261},{"type":"v1","addr":"172.21.15.135:6819","nonce":751247261}]},"public_addr":"172.21.15.135:6811/751247261","cluster_addr":"172.21.15.135:6815/751247261","heartbeat_back_addr":"172.21.15.135:6823/751247261","heartbeat_front_addr":"172.21.15.135:6819/751247261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.121796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:24.998458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.046775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.112181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.066802+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.017871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T10:33:25.047252+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6832/2292645283":"2024-05-11T10:43:22.391150+0000","172.21.15.53:6834/2292645283":"2024-05-11T10:43:22.391150+0000","172.21.15.53:6834/38461894":"2024-05-11T10:43:10.604857+0000","172.21.15.53:6832/38461894":"2024-05-11T10:43:10.604857+0000","172.21.15.135:6837/776479349":"2024-05-11T10:43:01.305186+0000","172.21.15.135:6835/776479349":"2024-05-11T10:43:01.305186+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5bb40762d0 msgr2=0x7f5bb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5bb40762d0 0x7f5bb4078790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009990 tx=0x7f5bc4009110 comp rx=0 tx=0).stop 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40ceb50 msgr2=0x7f5bd40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40ceb50 0x7f5bd40cc160 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00b990 tx=0x7f5bcc00bd90 comp rx=0 tx=0).stop 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 shutdown_connections 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f5bb40762d0 0x7f5bb4078790 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd40ceb50 0x7f5bd40cc160 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5bd4162b30 0x7f5bd415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 --2- 172.21.15.53:0/19482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bd4162750 0x7f5bd40cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 >> 172.21.15.53:0/19482 conn(0x7f5bd409e3b0 msgr2=0x7f5bd40b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 shutdown_connections 2024-05-10T10:43:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:28.845+0000 7f5bd93b6640 1 -- 172.21.15.53:0/19482 wait complete. 2024-05-10T10:43:29.858 INFO:tasks.ceph:Scrubbing osd.0 2024-05-10T10:43:29.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:29.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.979+0000 7f4d5abf0640 1 Processor -- start 2024-05-10T10:43:29.980 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.979+0000 7f4d5abf0640 1 -- start start 2024-05-10T10:43:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.980+0000 7f4d5abf0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d5415d2d0 0x7f4d540b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.980+0000 7f4d5abf0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d540d3920 0x7f4d540ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.980+0000 7f4d5abf0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4d540cff10 0x7f4d540ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.980+0000 7f4d5abf0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540d0d60 con 0x7f4d5415d2d0 2024-05-10T10:43:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.980+0000 7f4d5abf0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540d1040 con 0x7f4d540cff10 2024-05-10T10:43:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.980+0000 7f4d5abf0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540cdd60 con 0x7f4d540d3920 2024-05-10T10:43:29.982 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d5415d2d0 0x7f4d540b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d5415d2d0 0x7f4d540b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49920/0 (socket says 172.21.15.53:49920) 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 -- 172.21.15.53:0/1655051785 learned_addr learned my addr 172.21.15.53:0/1655051785 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d53fff640 1 --2- 172.21.15.53:0/1655051785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d540d3920 0x7f4d540ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5918d640 1 --2- 172.21.15.53:0/1655051785 >> v2:172.21.15.135:3300/0 conn(0x7f4d540cff10 0x7f4d540ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 -- 172.21.15.53:0/1655051785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d540d3920 msgr2=0x7f4d540ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 --2- 172.21.15.53:0/1655051785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d540d3920 0x7f4d540ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 -- 172.21.15.53:0/1655051785 >> v2:172.21.15.135:3300/0 conn(0x7f4d540cff10 msgr2=0x7f4d540ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 --2- 172.21.15.53:0/1655051785 >> v2:172.21.15.135:3300/0 conn(0x7f4d540cff10 0x7f4d540ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.981+0000 7f4d5898c640 1 -- 172.21.15.53:0/1655051785 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d540ce040 con 0x7f4d5415d2d0 2024-05-10T10:43:29.983 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.982+0000 7f4d5898c640 1 --2- 172.21.15.53:0/1655051785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d5415d2d0 0x7f4d540b2760 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4d54069170 tx=0x7f4d4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889c1ce15fea155b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.982+0000 7f4d537fe640 1 -- 172.21.15.53:0/1655051785 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037070 con 0x7f4d5415d2d0 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.982+0000 7f4d537fe640 1 -- 172.21.15.53:0/1655051785 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d48032440 con 0x7f4d5415d2d0 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.982+0000 7f4d537fe640 1 -- 172.21.15.53:0/1655051785 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48032810 con 0x7f4d5415d2d0 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1655051785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d5415d2d0 msgr2=0x7f4d540b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 --2- 172.21.15.53:0/1655051785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d5415d2d0 0x7f4d540b2760 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4d54069170 tx=0x7f4d4802a1e0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1655051785 shutdown_connections 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 --2- 172.21.15.53:0/1655051785 >> v2:172.21.15.135:3300/0 conn(0x7f4d540cff10 0x7f4d540ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 --2- 172.21.15.53:0/1655051785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d540d3920 0x7f4d540ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 --2- 172.21.15.53:0/1655051785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d5415d2d0 0x7f4d540b2760 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.985 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1655051785 >> 172.21.15.53:0/1655051785 conn(0x7f4d540d2e00 msgr2=0x7f4d540d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1655051785 shutdown_connections 2024-05-10T10:43:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.983+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1655051785 wait complete. 2024-05-10T10:43:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.984+0000 7f4d5abf0640 1 Processor -- start 2024-05-10T10:43:29.986 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.984+0000 7f4d5abf0640 1 -- start start 2024-05-10T10:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d5abf0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d540cff10 0x7f4d54152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d5abf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d540d3920 0x7f4d5415f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d5abf0640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d5415d2d0 0x7f4d5415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d5abf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540cda10 con 0x7f4d540d3920 2024-05-10T10:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d5abf0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5415d6b0 con 0x7f4d540cff10 2024-05-10T10:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d5abf0640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540b2ca0 con 0x7f4d5415d2d0 2024-05-10T10:43:29.988 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d540d3920 0x7f4d5415f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d540d3920 0x7f4d5415f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49936/0 (socket says 172.21.15.53:49936) 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.985+0000 7f4d53fff640 1 -- 172.21.15.53:0/1547815870 learned_addr learned my addr 172.21.15.53:0/1547815870 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d5918d640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d5415d2d0 0x7f4d5415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d5898c640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d540cff10 0x7f4d54152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d53fff640 1 -- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d5415d2d0 msgr2=0x7f4d5415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d53fff640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4d5415d2d0 0x7f4d5415f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d53fff640 1 -- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d540cff10 msgr2=0x7f4d54152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d53fff640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d540cff10 0x7f4d54152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d53fff640 1 -- 172.21.15.53:0/1547815870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d541f94a0 con 0x7f4d540d3920 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d53fff640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d540d3920 0x7f4d5415f160 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000d800 tx=0x7f4d4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000b6c0 con 0x7f4d540d3920 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d40019070 con 0x7f4d540d3920 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d540c5af0 con 0x7f4d540d3920 2024-05-10T10:43:29.989 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000bac0 con 0x7f4d540d3920 2024-05-10T10:43:29.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.986+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541f89c0 con 0x7f4d540d3920 2024-05-10T10:43:29.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.987+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d540693f0 con 0x7f4d540d3920 2024-05-10T10:43:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.988+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4d4000bc60 con 0x7f4d540d3920 2024-05-10T10:43:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.989+0000 7f4d517fa640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d380762d0 0x7f4d38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.989+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f4d400984b0 con 0x7f4d540d3920 2024-05-10T10:43:29.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.989+0000 7f4d5898c640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d380762d0 0x7f4d38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.989+0000 7f4d517fa640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7f4d3807c8b0 0x7f4d3807ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.989+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d40001d30 con 0x7f4d3807c8b0 2024-05-10T10:43:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.989+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f4d40098860 con 0x7f4d540d3920 2024-05-10T10:43:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.990+0000 7f4d5918d640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7f4d3807c8b0 0x7f4d3807ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.990+0000 7f4d5898c640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d380762d0 0x7f4d38078790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4d48034000 tx=0x7f4d4802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.990+0000 7f4d5918d640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7f4d3807c8b0 0x7f4d3807ecf0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:29.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:29.991+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== osd.0 v2:172.21.15.53:6802/1634417257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d40001d30 con 0x7f4d3807c8b0 2024-05-10T10:43:30.004 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.004+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 --> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4d541f90d0 con 0x7f4d3807c8b0 2024-05-10T10:43:30.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.004+0000 7f4d517fa640 1 -- 172.21.15.53:0/1547815870 <== osd.0 v2:172.21.15.53:6802/1634417257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4d541f90d0 con 0x7f4d3807c8b0 2024-05-10T10:43:30.005 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:30.005 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:30.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.004+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7f4d3807c8b0 msgr2=0x7f4d3807ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.004+0000 7f4d5abf0640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:6802/1634417257,v1:172.21.15.53:6804/1634417257] conn(0x7f4d3807c8b0 0x7f4d3807ecf0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d380762d0 msgr2=0x7f4d38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f4d380762d0 0x7f4d38078790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4d48034000 tx=0x7f4d4802e000 comp rx=0 tx=0).stop 2024-05-10T10:43:30.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d540d3920 msgr2=0x7f4d5415f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 --2- 172.21.15.53:0/1547815870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d540d3920 0x7f4d5415f160 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000d800 tx=0x7f4d4000dcc0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5918d640 1 -- 172.21.15.53:0/1547815870 reap_dead start 2024-05-10T10:43:30.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 shutdown_connections 2024-05-10T10:43:30.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 >> 172.21.15.53:0/1547815870 conn(0x7f4d540d2e00 msgr2=0x7f4d5415c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 shutdown_connections 2024-05-10T10:43:30.007 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.005+0000 7f4d5abf0640 1 -- 172.21.15.53:0/1547815870 wait complete. 2024-05-10T10:43:30.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-10T10:43:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.122+0000 7f17af4f1640 1 Processor -- start 2024-05-10T10:43:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.122+0000 7f17af4f1640 1 -- start start 2024-05-10T10:43:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17af4f1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a815f3b0 0x7f17a80d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17af4f1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17a815f790 0x7f17a80d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17af4f1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a80d30b0 con 0x7f17a80d5d30 2024-05-10T10:43:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17af4f1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f17a80d3390 con 0x7f17a815f790 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17af4f1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f17a80c8c80 con 0x7f17a815f3b0 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17aca8c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17a815f790 0x7f17a80d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17ada8e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f17a80d5d30 0x7f17a80c9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58010/0 (socket says 172.21.15.53:58010) 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17ada8e640 1 -- 172.21.15.53:0/398919084 learned_addr learned my addr 172.21.15.53:0/398919084 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17ad28d640 1 --2- 172.21.15.53:0/398919084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a815f3b0 0x7f17a80d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17aca8c640 1 -- 172.21.15.53:0/398919084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a815f3b0 msgr2=0x7f17a80d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17aca8c640 1 --2- 172.21.15.53:0/398919084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a815f3b0 0x7f17a80d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17aca8c640 1 -- 172.21.15.53:0/398919084 >> v1:172.21.15.53:6789/0 conn(0x7f17a80d5d30 legacy=0x7f17a80c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f17aca8c640 1 -- 172.21.15.53:0/398919084 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a80c8f60 con 0x7f17a815f790 2024-05-10T10:43:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.123+0000 7f179ffff640 1 -- 172.21.15.53:0/398919084 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034357004 0 0) 0x7f17a80d30b0 con 0x7f17a80d5d30 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.124+0000 7f17aca8c640 1 --2- 172.21.15.53:0/398919084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a815f790 0x7f17a80d3c30 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f17980097a0 tx=0x7f179802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8fd47efb522150 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.124+0000 7f179ffff640 1 -- 172.21.15.53:0/398919084 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798037070 con 0x7f17a815f790 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.124+0000 7f179ffff640 1 -- 172.21.15.53:0/398919084 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1798032440 con 0x7f17a815f790 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.124+0000 7f179ffff640 1 -- 172.21.15.53:0/398919084 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798032810 con 0x7f17a815f790 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 -- 172.21.15.53:0/398919084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a815f790 msgr2=0x7f17a80d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 --2- 172.21.15.53:0/398919084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a815f790 0x7f17a80d3c30 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f17980097a0 tx=0x7f179802a180 comp rx=0 tx=0).stop 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 -- 172.21.15.53:0/398919084 shutdown_connections 2024-05-10T10:43:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 --2- 172.21.15.53:0/398919084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a815f790 0x7f17a80d3c30 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 --2- 172.21.15.53:0/398919084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a815f3b0 0x7f17a80d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 -- 172.21.15.53:0/398919084 >> 172.21.15.53:0/398919084 conn(0x7f17a80d5130 msgr2=0x7f17a80d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 -- 172.21.15.53:0/398919084 shutdown_connections 2024-05-10T10:43:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.125+0000 7f17af4f1640 1 -- 172.21.15.53:0/398919084 wait complete. 2024-05-10T10:43:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.126+0000 7f17af4f1640 1 Processor -- start 2024-05-10T10:43:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17af4f1640 1 -- start start 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17af4f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a80d5d30 0x7f17a81546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17af4f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17a815f3b0 0x7f17a8154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17af4f1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a8161b90 0x7f17a8162040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17af4f1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80d0e60 con 0x7f17a80d5d30 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17af4f1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80d0ce0 con 0x7f17a815f3b0 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17af4f1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f17a80d0fe0 con 0x7f17a8161b90 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17ad28d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a80d5d30 0x7f17a81546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17ad28d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a80d5d30 0x7f17a81546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49950/0 (socket says 172.21.15.53:49950) 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.127+0000 7f17ad28d640 1 -- 172.21.15.53:0/2518350509 learned_addr learned my addr 172.21.15.53:0/2518350509 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17ada8e640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a8161b90 0x7f17a8162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17aca8c640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17a815f3b0 0x7f17a8154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17ad28d640 1 -- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a8161b90 msgr2=0x7f17a8162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17ad28d640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a8161b90 0x7f17a8162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17ad28d640 1 -- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17a815f3b0 msgr2=0x7f17a8154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17ad28d640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17a815f3b0 0x7f17a8154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17ad28d640 1 -- 172.21.15.53:0/2518350509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a8154c00 con 0x7f17a80d5d30 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17ad28d640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a80d5d30 0x7f17a81546c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f17a400e9e0 tx=0x7f17a400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f179dffb640 1 -- 172.21.15.53:0/2518350509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4017070 con 0x7f17a80d5d30 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f179dffb640 1 -- 172.21.15.53:0/2518350509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a400c970 con 0x7f17a80d5d30 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f179dffb640 1 -- 172.21.15.53:0/2518350509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a400cd60 con 0x7f17a80d5d30 2024-05-10T10:43:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a8158500 con 0x7f17a80d5d30 2024-05-10T10:43:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.128+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a8158ac0 con 0x7f17a80d5d30 2024-05-10T10:43:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.129+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a806a220 con 0x7f17a80d5d30 2024-05-10T10:43:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.130+0000 7f179dffb640 1 -- 172.21.15.53:0/2518350509 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f17a401a070 con 0x7f17a80d5d30 2024-05-10T10:43:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.130+0000 7f179dffb640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f178c076300 0x7f178c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.131+0000 7f179dffb640 1 -- 172.21.15.53:0/2518350509 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f17a4097780 con 0x7f17a80d5d30 2024-05-10T10:43:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.131+0000 7f17aca8c640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f178c076300 0x7f178c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.131+0000 7f17aca8c640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f178c076300 0x7f178c0787c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1798034000 tx=0x7f179802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.135+0000 7f179dffb640 1 -- 172.21.15.53:0/2518350509 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f17a8154c00 con 0x7f17a80d5d30 2024-05-10T10:43:30.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.329+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f17a80c9ee0 con 0x7f178c076300 2024-05-10T10:43:30.330 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.329+0000 7f179dffb640 1 -- 172.21.15.53:0/2518350509 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f17a80c9ee0 con 0x7f178c076300 2024-05-10T10:43:30.330 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 0 to deep-scrub 2024-05-10T10:43:30.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f178c076300 msgr2=0x7f178c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f178c076300 0x7f178c0787c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1798034000 tx=0x7f179802e000 comp rx=0 tx=0).stop 2024-05-10T10:43:30.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a80d5d30 msgr2=0x7f17a81546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a80d5d30 0x7f17a81546c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f17a400e9e0 tx=0x7f17a400eea0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.332 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 shutdown_connections 2024-05-10T10:43:30.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f178c076300 0x7f178c0787c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17a8161b90 0x7f17a8162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17a815f3b0 0x7f17a8154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 --2- 172.21.15.53:0/2518350509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a80d5d30 0x7f17a81546c0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 >> 172.21.15.53:0/2518350509 conn(0x7f17a80d5130 msgr2=0x7f17a80d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 shutdown_connections 2024-05-10T10:43:30.333 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.331+0000 7f17af4f1640 1 -- 172.21.15.53:0/2518350509 wait complete. 2024-05-10T10:43:30.344 INFO:tasks.ceph:Scrubbing osd.1 2024-05-10T10:43:30.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:30.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.452+0000 7f31da3da640 1 Processor -- start 2024-05-10T10:43:30.452 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.452+0000 7f31da3da640 1 -- start start 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31da3da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31d415f490 0x7f31d40d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31da3da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31d40d5ca0 0x7f31d40d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31da3da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d40d3b80 0x7f31d40cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31da3da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f31d40b4e70 con 0x7f31d40d5ca0 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31da3da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f31d40d1080 con 0x7f31d415f490 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31da3da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31d40b4820 con 0x7f31d40d3b80 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31d40d5ca0 0x7f31d40d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31d40d5ca0 0x7f31d40d3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49966/0 (socket says 172.21.15.53:49966) 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 -- 172.21.15.53:0/350600317 learned_addr learned my addr 172.21.15.53:0/350600317 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31d8977640 1 --2- 172.21.15.53:0/350600317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d40d3b80 0x7f31d40cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31d3fff640 1 --2- 172.21.15.53:0/350600317 >> v2:172.21.15.135:3300/0 conn(0x7f31d415f490 0x7f31d40d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 -- 172.21.15.53:0/350600317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d40d3b80 msgr2=0x7f31d40cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 --2- 172.21.15.53:0/350600317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d40d3b80 0x7f31d40cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 -- 172.21.15.53:0/350600317 >> v2:172.21.15.135:3300/0 conn(0x7f31d415f490 msgr2=0x7f31d40d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 --2- 172.21.15.53:0/350600317 >> v2:172.21.15.135:3300/0 conn(0x7f31d415f490 0x7f31d40d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31cbfff640 1 -- 172.21.15.53:0/350600317 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d40b4b00 con 0x7f31d40d5ca0 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31d8977640 1 --2- 172.21.15.53:0/350600317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d40d3b80 0x7f31d40cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.453+0000 7f31d3fff640 1 --2- 172.21.15.53:0/350600317 >> v2:172.21.15.135:3300/0 conn(0x7f31d415f490 0x7f31d40d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.454+0000 7f31cbfff640 1 --2- 172.21.15.53:0/350600317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40d3640 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f31d408d410 tx=0x7f31bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492dbe980fc23199 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.454+0000 7f31d37fe640 1 -- 172.21.15.53:0/350600317 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc037070 con 0x7f31d40d5ca0 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.454+0000 7f31d37fe640 1 -- 172.21.15.53:0/350600317 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31bc032440 con 0x7f31d40d5ca0 2024-05-10T10:43:30.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.454+0000 7f31d37fe640 1 -- 172.21.15.53:0/350600317 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc032810 con 0x7f31d40d5ca0 2024-05-10T10:43:30.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 -- 172.21.15.53:0/350600317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 msgr2=0x7f31d40d3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 --2- 172.21.15.53:0/350600317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40d3640 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f31d408d410 tx=0x7f31bc02a140 comp rx=0 tx=0).stop 2024-05-10T10:43:30.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 -- 172.21.15.53:0/350600317 shutdown_connections 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 --2- 172.21.15.53:0/350600317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d40d3b80 0x7f31d40cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 --2- 172.21.15.53:0/350600317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40d3640 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 --2- 172.21.15.53:0/350600317 >> v2:172.21.15.135:3300/0 conn(0x7f31d415f490 0x7f31d40d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 -- 172.21.15.53:0/350600317 >> 172.21.15.53:0/350600317 conn(0x7f31d40d50f0 msgr2=0x7f31d40d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 -- 172.21.15.53:0/350600317 shutdown_connections 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.455+0000 7f31da3da640 1 -- 172.21.15.53:0/350600317 wait complete. 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.456+0000 7f31da3da640 1 Processor -- start 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.456+0000 7f31da3da640 1 -- start start 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31da3da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d40d3b80 0x7f31d40c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31da3da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31da3da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d415f490 0x7f31d41573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31da3da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31d40d0350 con 0x7f31d40d5ca0 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31da3da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31d40d01d0 con 0x7f31d40d3b80 2024-05-10T10:43:30.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31da3da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f31d40d04d0 con 0x7f31d415f490 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49980/0 (socket says 172.21.15.53:49980) 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 -- 172.21.15.53:0/2050490085 learned_addr learned my addr 172.21.15.53:0/2050490085 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31d3fff640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d40d3b80 0x7f31d40c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31d8977640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d415f490 0x7f31d41573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 -- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d415f490 msgr2=0x7f31d41573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d415f490 0x7f31d41573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 -- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d40d3b80 msgr2=0x7f31d40c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d40d3b80 0x7f31d40c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31cbfff640 1 -- 172.21.15.53:0/2050490085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d40c9cc0 con 0x7f31d40d5ca0 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31d3fff640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31d40d3b80 0x7f31d40c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.457+0000 7f31d8977640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31d415f490 0x7f31d41573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:30.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.458+0000 7f31cbfff640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40cb5a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f31bc02a180 tx=0x7f31bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.458+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc037040 con 0x7f31d40d5ca0 2024-05-10T10:43:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.458+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31bc042070 con 0x7f31d40d5ca0 2024-05-10T10:43:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.458+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc030bb0 con 0x7f31d40d5ca0 2024-05-10T10:43:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.458+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d40c9fa0 con 0x7f31d40d5ca0 2024-05-10T10:43:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.458+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d40b6d10 con 0x7f31d40d5ca0 2024-05-10T10:43:30.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.459+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31d406a220 con 0x7f31d40d5ca0 2024-05-10T10:43:30.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.459+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f31bc047460 con 0x7f31d40d5ca0 2024-05-10T10:43:30.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.460+0000 7f31d17fa640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31b4076300 0x7f31b40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.460+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f31bc0b8100 con 0x7f31d40d5ca0 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.460+0000 7f31d17fa640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f31b407c8e0 0x7f31b407ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.460+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31bc040e80 con 0x7f31b407c8e0 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.460+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f31bc0bf050 con 0x7f31d40d5ca0 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.460+0000 7f31d3fff640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31b4076300 0x7f31b40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.460+0000 7f31d8977640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f31b407c8e0 0x7f31b407ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.461+0000 7f31d8977640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f31b407c8e0 0x7f31b407ed20 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.461+0000 7f31d3fff640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31b4076300 0x7f31b40787c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006fd0 tx=0x7f31c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.461+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== osd.1 v2:172.21.15.53:6824/2705510088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31bc040e80 con 0x7f31b407c8e0 2024-05-10T10:43:30.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 --> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f31d40b2320 con 0x7f31b407c8e0 2024-05-10T10:43:30.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31d17fa640 1 -- 172.21.15.53:0/2050490085 <== osd.1 v2:172.21.15.53:6824/2705510088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f31d40b2320 con 0x7f31b407c8e0 2024-05-10T10:43:30.476 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:30.476 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:30.476 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f31b407c8e0 msgr2=0x7f31b407ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:6824/2705510088,v1:172.21.15.53:6825/2705510088] conn(0x7f31b407c8e0 0x7f31b407ed20 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31b4076300 msgr2=0x7f31b40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f31b4076300 0x7f31b40787c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006fd0 tx=0x7f31c4008040 comp rx=0 tx=0).stop 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 msgr2=0x7f31d40cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 --2- 172.21.15.53:0/2050490085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31d40d5ca0 0x7f31d40cb5a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f31bc02a180 tx=0x7f31bc0046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31d8977640 1 -- 172.21.15.53:0/2050490085 reap_dead start 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 shutdown_connections 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 >> 172.21.15.53:0/2050490085 conn(0x7f31d40d50f0 msgr2=0x7f31d406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 shutdown_connections 2024-05-10T10:43:30.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.475+0000 7f31da3da640 1 -- 172.21.15.53:0/2050490085 wait complete. 2024-05-10T10:43:30.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-10T10:43:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.593+0000 7f411f190640 1 Processor -- start 2024-05-10T10:43:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.593+0000 7f411f190640 1 -- start start 2024-05-10T10:43:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.593+0000 7f411f190640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f41180d0d40 0x7f41181467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.593+0000 7f411f190640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41180afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411f190640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41180714f0 con 0x7f4118070c10 2024-05-10T10:43:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411f190640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f41180717d0 con 0x7f41180d0d40 2024-05-10T10:43:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411f190640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41180af230 con 0x7f411806c9d0 2024-05-10T10:43:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411d72d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41180afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411d72d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41180afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:49724/0 (socket says 172.21.15.53:49724) 2024-05-10T10:43:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411d72d640 1 -- 172.21.15.53:0/2823516637 learned_addr learned my addr 172.21.15.53:0/2823516637 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411cf2c640 1 --2- 172.21.15.53:0/2823516637 >> v2:172.21.15.135:3300/0 conn(0x7f41180d0d40 0x7f41181467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411cf2c640 1 -- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 msgr2=0x7f41180afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411cf2c640 1 --2- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41180afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411cf2c640 1 -- 172.21.15.53:0/2823516637 >> v1:172.21.15.135:6789/0 conn(0x7f4118070c10 legacy=0x7f411806c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411cf2c640 1 -- 172.21.15.53:0/2823516637 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41180af510 con 0x7f41180d0d40 2024-05-10T10:43:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.594+0000 7f411d72d640 1 --2- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41180afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.595+0000 7f411cf2c640 1 --2- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 0x7f41181467e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f410c0097a0 tx=0x7f410c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4511d84018d28ff2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.595+0000 7f41177fe640 1 -- 172.21.15.53:0/2823516637 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c037070 con 0x7f41180d0d40 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.595+0000 7f41177fe640 1 -- 172.21.15.53:0/2823516637 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f410c02ae50 con 0x7f41180d0d40 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.595+0000 7f41177fe640 1 -- 172.21.15.53:0/2823516637 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c032660 con 0x7f41180d0d40 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 -- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 msgr2=0x7f41181467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 --2- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 0x7f41181467e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f410c0097a0 tx=0x7f410c02a100 comp rx=0 tx=0).stop 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 -- 172.21.15.53:0/2823516637 shutdown_connections 2024-05-10T10:43:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 --2- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41180afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 --2- 172.21.15.53:0/2823516637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 0x7f41181467e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 -- 172.21.15.53:0/2823516637 >> 172.21.15.53:0/2823516637 conn(0x7f41180d1d70 msgr2=0x7f41180d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 -- 172.21.15.53:0/2823516637 shutdown_connections 2024-05-10T10:43:30.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.596+0000 7f411f190640 1 -- 172.21.15.53:0/2823516637 wait complete. 2024-05-10T10:43:30.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.597+0000 7f411f190640 1 Processor -- start 2024-05-10T10:43:30.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.597+0000 7f411f190640 1 -- start start 2024-05-10T10:43:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f411f190640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41181e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f411f190640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4118070c10 0x7f41181e4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f411f190640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 0x7f41181378a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f411f190640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41180aff70 con 0x7f4118070c10 2024-05-10T10:43:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f411f190640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41180712b0 con 0x7f41180d0d40 2024-05-10T10:43:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f411f190640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f41180b00f0 con 0x7f411806c9d0 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f4117fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4118070c10 0x7f41181e4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f4117fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4118070c10 0x7f41181e4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49986/0 (socket says 172.21.15.53:49986) 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f4117fff640 1 -- 172.21.15.53:0/3676945205 learned_addr learned my addr 172.21.15.53:0/3676945205 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.598+0000 7f411d72d640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 0x7f41181378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f411cf2c640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41181e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f4117fff640 1 -- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 msgr2=0x7f41181e6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f4117fff640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41181e6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f4117fff640 1 -- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 msgr2=0x7f41181378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f4117fff640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 0x7f41181378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f4117fff640 1 -- 172.21.15.53:0/3676945205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41181e4c80 con 0x7f4118070c10 2024-05-10T10:43:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f4117fff640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4118070c10 0x7f41181e4740 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f410400c400 tx=0x7f410400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f41157fa640 1 -- 172.21.15.53:0/3676945205 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41040052d0 con 0x7f4118070c10 2024-05-10T10:43:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f41157fa640 1 -- 172.21.15.53:0/3676945205 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4104005490 con 0x7f4118070c10 2024-05-10T10:43:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41181e4f00 con 0x7f4118070c10 2024-05-10T10:43:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f41157fa640 1 -- 172.21.15.53:0/3676945205 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104007260 con 0x7f4118070c10 2024-05-10T10:43:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.599+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181e54c0 con 0x7f4118070c10 2024-05-10T10:43:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.600+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41180693f0 con 0x7f4118070c10 2024-05-10T10:43:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.601+0000 7f41157fa640 1 -- 172.21.15.53:0/3676945205 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4104007400 con 0x7f4118070c10 2024-05-10T10:43:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.602+0000 7f41157fa640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f40fc0762d0 0x7f40fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.602+0000 7f41157fa640 1 -- 172.21.15.53:0/3676945205 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f41040969c0 con 0x7f4118070c10 2024-05-10T10:43:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.602+0000 7f411cf2c640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f40fc0762d0 0x7f40fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.603+0000 7f411cf2c640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f40fc0762d0 0x7f40fc078790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.606+0000 7f41157fa640 1 -- 172.21.15.53:0/3676945205 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f4104098e20 con 0x7f4118070c10 2024-05-10T10:43:30.798 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.798+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f41180b1f70 con 0x7f40fc0762d0 2024-05-10T10:43:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.798+0000 7f41157fa640 1 -- 172.21.15.53:0/3676945205 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f41180b1f70 con 0x7f40fc0762d0 2024-05-10T10:43:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 1 to deep-scrub 2024-05-10T10:43:30.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.800+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f40fc0762d0 msgr2=0x7f40fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.801 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.800+0000 7f411f190640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f40fc0762d0 0x7f40fc078790 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c02e000 comp rx=0 tx=0).stop 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.800+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4118070c10 msgr2=0x7f41181e4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.800+0000 7f411f190640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4118070c10 0x7f41181e4740 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f410400c400 tx=0x7f410400c8c0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 shutdown_connections 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f40fc0762d0 0x7f40fc078790 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41180d0d40 0x7f41181378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4118070c10 0x7f41181e4740 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 --2- 172.21.15.53:0/3676945205 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f411806c9d0 0x7f41181e6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 >> 172.21.15.53:0/3676945205 conn(0x7f41180d1d70 msgr2=0x7f411806f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 shutdown_connections 2024-05-10T10:43:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.801+0000 7f411f190640 1 -- 172.21.15.53:0/3676945205 wait complete. 2024-05-10T10:43:30.812 INFO:tasks.ceph:Scrubbing osd.2 2024-05-10T10:43:30.812 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:30.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.919+0000 7f2613989640 1 Processor -- start 2024-05-10T10:43:30.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.920+0000 7f2613989640 1 -- start start 2024-05-10T10:43:30.921 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.920+0000 7f2613989640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f260c15f490 0x7f260c0d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.920+0000 7f2613989640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c0d5ca0 0x7f260c0d3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.920+0000 7f2613989640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260c0d1080 con 0x7f260c0d3b80 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.920+0000 7f2613989640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f260c0b4e70 con 0x7f260c15f490 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.920+0000 7f2613989640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f260c0b4820 con 0x7f260c0d5ca0 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.920+0000 7f2611725640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f260c15f490 0x7f260c0d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.921+0000 7f2611725640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f260c15f490 0x7f260c0d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49994/0 (socket says 172.21.15.53:49994) 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.921+0000 7f2611725640 1 -- 172.21.15.53:0/3076919409 learned_addr learned my addr 172.21.15.53:0/3076919409 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.921+0000 7f2611725640 1 -- 172.21.15.53:0/3076919409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c0d5ca0 msgr2=0x7f260c0d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.921+0000 7f2611725640 1 --2- 172.21.15.53:0/3076919409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c0d5ca0 0x7f260c0d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.921+0000 7f2611725640 1 -- 172.21.15.53:0/3076919409 >> v1:172.21.15.135:6789/0 conn(0x7f260c0d3b80 legacy=0x7f260c0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.921+0000 7f2611725640 1 -- 172.21.15.53:0/3076919409 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c0b4b00 con 0x7f260c15f490 2024-05-10T10:43:30.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.921+0000 7f2611725640 1 --2- 172.21.15.53:0/3076919409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c15f490 0x7f260c0d2f60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f2608009870 tx=0x7f260802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dddbca2f9b9619 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f25fffff640 1 -- 172.21.15.53:0/3076919409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608037070 con 0x7f260c15f490 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f25fffff640 1 -- 172.21.15.53:0/3076919409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2608032440 con 0x7f260c15f490 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f25fffff640 1 -- 172.21.15.53:0/3076919409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608032810 con 0x7f260c15f490 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f2613989640 1 -- 172.21.15.53:0/3076919409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c15f490 msgr2=0x7f260c0d2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f2613989640 1 --2- 172.21.15.53:0/3076919409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c15f490 0x7f260c0d2f60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f2608009870 tx=0x7f260802a180 comp rx=0 tx=0).stop 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f2613989640 1 -- 172.21.15.53:0/3076919409 shutdown_connections 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f2613989640 1 --2- 172.21.15.53:0/3076919409 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c0d5ca0 0x7f260c0d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f2613989640 1 --2- 172.21.15.53:0/3076919409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c15f490 0x7f260c0d2f60 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f2613989640 1 -- 172.21.15.53:0/3076919409 >> 172.21.15.53:0/3076919409 conn(0x7f260c0d50f0 msgr2=0x7f260c0d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.922+0000 7f2613989640 1 -- 172.21.15.53:0/3076919409 shutdown_connections 2024-05-10T10:43:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.923+0000 7f2613989640 1 -- 172.21.15.53:0/3076919409 wait complete. 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 Processor -- start 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 -- start start 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c0d3b80 0x7f260c0ba410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c0d5ca0 0x7f260c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c15f490 0x7f260c0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f260c0b6310 con 0x7f260c0d5ca0 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f260c15fa80 con 0x7f260c0d3b80 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.924+0000 7f2613989640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f260c0b6490 con 0x7f260c15f490 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c0d5ca0 0x7f260c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c0d5ca0 0x7f260c0cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49998/0 (socket says 172.21.15.53:49998) 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 -- 172.21.15.53:0/4036426498 learned_addr learned my addr 172.21.15.53:0/4036426498 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2611725640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c0d3b80 0x7f260c0ba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2611f26640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c15f490 0x7f260c0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 -- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c15f490 msgr2=0x7f260c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f260c15f490 0x7f260c0cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 -- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c0d3b80 msgr2=0x7f260c0ba410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c0d3b80 0x7f260c0ba410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 -- 172.21.15.53:0/4036426498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c1f51d0 con 0x7f260c0d5ca0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2610f24640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c0d5ca0 0x7f260c0cab90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f260000b480 tx=0x7f260000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600014070 con 0x7f260c0d5ca0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2600010870 con 0x7f260c0d5ca0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c1f5490 con 0x7f260c0d5ca0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.925+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600010c60 con 0x7f260c0d5ca0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.926+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c1f5a50 con 0x7f260c0d5ca0 2024-05-10T10:43:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.926+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f260c1f5e30 con 0x7f260c0d5ca0 2024-05-10T10:43:30.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.927+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2600017070 con 0x7f260c0d5ca0 2024-05-10T10:43:30.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.928+0000 7f25fdffb640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25f00762d0 0x7f25f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.928+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f26000978f0 con 0x7f260c0d5ca0 2024-05-10T10:43:30.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.928+0000 7f25fdffb640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f25f007c8b0 0x7f25f007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:30.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.928+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25f007f3c0 con 0x7f25f007c8b0 2024-05-10T10:43:30.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.928+0000 7f2611725640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25f00762d0 0x7f25f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.928+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f2600097ca0 con 0x7f260c0d5ca0 2024-05-10T10:43:30.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.928+0000 7f2611f26640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f25f007c8b0 0x7f25f007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:30.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.929+0000 7f2611f26640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f25f007c8b0 0x7f25f007ecf0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.929+0000 7f2611725640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25f00762d0 0x7f25f0078790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2608034000 tx=0x7f260802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:30.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.929+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== osd.2 v2:172.21.15.53:6800/2235813461 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25f007f3c0 con 0x7f25f007c8b0 2024-05-10T10:43:30.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 --> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f260c0b7520 con 0x7f25f007c8b0 2024-05-10T10:43:30.943 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f25fdffb640 1 -- 172.21.15.53:0/4036426498 <== osd.2 v2:172.21.15.53:6800/2235813461 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f260c0b7520 con 0x7f25f007c8b0 2024-05-10T10:43:30.944 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:30.944 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f25f007c8b0 msgr2=0x7f25f007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:6800/2235813461,v1:172.21.15.53:6801/2235813461] conn(0x7f25f007c8b0 0x7f25f007ecf0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25f00762d0 msgr2=0x7f25f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f25f00762d0 0x7f25f0078790 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2608034000 tx=0x7f260802e000 comp rx=0 tx=0).stop 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c0d5ca0 msgr2=0x7f260c0cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 --2- 172.21.15.53:0/4036426498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c0d5ca0 0x7f260c0cab90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f260000b480 tx=0x7f260000b940 comp rx=0 tx=0).stop 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2611f26640 1 -- 172.21.15.53:0/4036426498 reap_dead start 2024-05-10T10:43:30.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 shutdown_connections 2024-05-10T10:43:30.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 >> 172.21.15.53:0/4036426498 conn(0x7f260c0d50f0 msgr2=0x7f260c0b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:30.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 shutdown_connections 2024-05-10T10:43:30.946 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:30.943+0000 7f2613989640 1 -- 172.21.15.53:0/4036426498 wait complete. 2024-05-10T10:43:30.954 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-10T10:43:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.059+0000 7f972356a640 1 Processor -- start 2024-05-10T10:43:31.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.059+0000 7f972356a640 1 -- start start 2024-05-10T10:43:31.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f972356a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f972356a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f971c15f790 0x7f971c0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f972356a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f971c0d5d30 0x7f971c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f972356a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f971c0d30b0 con 0x7f971c15f790 2024-05-10T10:43:31.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f972356a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f971c0d3390 con 0x7f971c0d5d30 2024-05-10T10:43:31.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f972356a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f971c0c8c80 con 0x7f971c15f3b0 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f9720b05640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f971c15f790 0x7f971c0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f9720b05640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f971c15f790 0x7f971c0d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50002/0 (socket says 172.21.15.53:50002) 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.060+0000 7f9720b05640 1 -- 172.21.15.53:0/2978760749 learned_addr learned my addr 172.21.15.53:0/2978760749 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9721b07640 1 --2- 172.21.15.53:0/2978760749 >> v2:172.21.15.135:3300/0 conn(0x7f971c0d5d30 0x7f971c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9721306640 1 --2- 172.21.15.53:0/2978760749 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9720b05640 1 -- 172.21.15.53:0/2978760749 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 msgr2=0x7f971c0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9720b05640 1 --2- 172.21.15.53:0/2978760749 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9720b05640 1 -- 172.21.15.53:0/2978760749 >> v2:172.21.15.135:3300/0 conn(0x7f971c0d5d30 msgr2=0x7f971c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9720b05640 1 --2- 172.21.15.53:0/2978760749 >> v2:172.21.15.135:3300/0 conn(0x7f971c0d5d30 0x7f971c0c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9720b05640 1 -- 172.21.15.53:0/2978760749 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c0c8f60 con 0x7f971c15f790 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.061+0000 7f9720b05640 1 --2- 172.21.15.53:0/2978760749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15f790 0x7f971c0d3c30 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f971c0d36a0 tx=0x7f971002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e898cdbd6e4d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f970ffff640 1 -- 172.21.15.53:0/2978760749 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710037070 con 0x7f971c15f790 2024-05-10T10:43:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f970ffff640 1 -- 172.21.15.53:0/2978760749 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9710032440 con 0x7f971c15f790 2024-05-10T10:43:31.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f970ffff640 1 -- 172.21.15.53:0/2978760749 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710032810 con 0x7f971c15f790 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 -- 172.21.15.53:0/2978760749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15f790 msgr2=0x7f971c0d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 --2- 172.21.15.53:0/2978760749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15f790 0x7f971c0d3c30 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f971c0d36a0 tx=0x7f971002a1e0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 -- 172.21.15.53:0/2978760749 shutdown_connections 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 --2- 172.21.15.53:0/2978760749 >> v2:172.21.15.135:3300/0 conn(0x7f971c0d5d30 0x7f971c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 --2- 172.21.15.53:0/2978760749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15f790 0x7f971c0d3c30 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 --2- 172.21.15.53:0/2978760749 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c0d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 -- 172.21.15.53:0/2978760749 >> 172.21.15.53:0/2978760749 conn(0x7f971c0d5130 msgr2=0x7f971c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 -- 172.21.15.53:0/2978760749 shutdown_connections 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.062+0000 7f972356a640 1 -- 172.21.15.53:0/2978760749 wait complete. 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.063+0000 7f972356a640 1 Processor -- start 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f972356a640 1 -- start start 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f972356a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c0d5d30 0x7f971c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f972356a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c1f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f972356a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15b180 0x7f971c158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f972356a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0c9e20 con 0x7f971c15b180 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f972356a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f971c0c9ca0 con 0x7f971c0d5d30 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f972356a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f971c0c9fa0 con 0x7f971c15f3b0 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f9721b07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15b180 0x7f971c158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f9721b07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15b180 0x7f971c158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50018/0 (socket says 172.21.15.53:50018) 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f9721b07640 1 -- 172.21.15.53:0/4249554150 learned_addr learned my addr 172.21.15.53:0/4249554150 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.064+0000 7f9721306640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c0d5d30 0x7f971c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f9720b05640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c1f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f9721b07640 1 -- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 msgr2=0x7f971c1f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f9721b07640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c1f8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f9721b07640 1 -- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c0d5d30 msgr2=0x7f971c159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f9721b07640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c0d5d30 0x7f971c159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f9721b07640 1 -- 172.21.15.53:0/4249554150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c1f9240 con 0x7f971c15b180 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f9721b07640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15b180 0x7f971c158db0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f9714007e60 tx=0x7f971400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f970dffb640 1 -- 172.21.15.53:0/4249554150 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714014420 con 0x7f971c15b180 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f970dffb640 1 -- 172.21.15.53:0/4249554150 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9714019070 con 0x7f971c15b180 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f970dffb640 1 -- 172.21.15.53:0/4249554150 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714014800 con 0x7f971c15b180 2024-05-10T10:43:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c1f89a0 con 0x7f971c15b180 2024-05-10T10:43:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.065+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c1f8f60 con 0x7f971c15b180 2024-05-10T10:43:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.066+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c06a220 con 0x7f971c15b180 2024-05-10T10:43:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.067+0000 7f970dffb640 1 -- 172.21.15.53:0/4249554150 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f97140149a0 con 0x7f971c15b180 2024-05-10T10:43:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.068+0000 7f970dffb640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9700076300 0x7f97000787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.068+0000 7f970dffb640 1 -- 172.21.15.53:0/4249554150 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f9714097ac0 con 0x7f971c15b180 2024-05-10T10:43:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.068+0000 7f9721306640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9700076300 0x7f97000787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.069+0000 7f9721306640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9700076300 0x7f97000787c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9718006fd0 tx=0x7f9718008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.072+0000 7f970dffb640 1 -- 172.21.15.53:0/4249554150 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f9714060f20 con 0x7f971c15b180 2024-05-10T10:43:31.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.263+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f971c0d3f80 con 0x7f9700076300 2024-05-10T10:43:31.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.264+0000 7f970dffb640 1 -- 172.21.15.53:0/4249554150 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f971c0d3f80 con 0x7f9700076300 2024-05-10T10:43:31.264 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 2 to deep-scrub 2024-05-10T10:43:31.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9700076300 msgr2=0x7f97000787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9700076300 0x7f97000787c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9718006fd0 tx=0x7f9718008040 comp rx=0 tx=0).stop 2024-05-10T10:43:31.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15b180 msgr2=0x7f971c158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15b180 0x7f971c158db0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f9714007e60 tx=0x7f971400e4b0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 shutdown_connections 2024-05-10T10:43:31.267 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f9700076300 0x7f97000787c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15b180 0x7f971c158db0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f971c15f3b0 0x7f971c1f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 --2- 172.21.15.53:0/4249554150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f971c0d5d30 0x7f971c159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 >> 172.21.15.53:0/4249554150 conn(0x7f971c0d5130 msgr2=0x7f971c075930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 shutdown_connections 2024-05-10T10:43:31.268 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.266+0000 7f972356a640 1 -- 172.21.15.53:0/4249554150 wait complete. 2024-05-10T10:43:31.277 INFO:tasks.ceph:Scrubbing osd.3 2024-05-10T10:43:31.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:31.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.385+0000 7f33b4ada640 1 Processor -- start 2024-05-10T10:43:31.386 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.385+0000 7f33b4ada640 1 -- start start 2024-05-10T10:43:31.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.385+0000 7f33b4ada640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 0x7f33b00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33b4ada640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f33b00b5f30 0x7f33b00d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.388 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33b4ada640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f33b00d1aa0 0x7f33b00d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33b4ada640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f33b00d10d0 con 0x7f33b00d1aa0 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33b4ada640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f33b00d0830 con 0x7f33b00b5f30 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33b4ada640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f33b00d0b10 con 0x7f33b0162990 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f33b00d1aa0 0x7f33b00d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f33b00d1aa0 0x7f33b00d1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50020/0 (socket says 172.21.15.53:50020) 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 -- 172.21.15.53:0/2488258892 learned_addr learned my addr 172.21.15.53:0/2488258892 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33ae59c640 1 --2- 172.21.15.53:0/2488258892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 0x7f33b00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33a5d9b640 1 --2- 172.21.15.53:0/2488258892 >> v2:172.21.15.135:3300/0 conn(0x7f33b00b5f30 0x7f33b00d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 -- 172.21.15.53:0/2488258892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 msgr2=0x7f33b00b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 --2- 172.21.15.53:0/2488258892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 0x7f33b00b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 -- 172.21.15.53:0/2488258892 >> v2:172.21.15.135:3300/0 conn(0x7f33b00b5f30 msgr2=0x7f33b00d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 --2- 172.21.15.53:0/2488258892 >> v2:172.21.15.135:3300/0 conn(0x7f33b00b5f30 0x7f33b00d59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.389 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.386+0000 7f33aed9d640 1 -- 172.21.15.53:0/2488258892 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b00d0df0 con 0x7f33b00d1aa0 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.387+0000 7f33aed9d640 1 --2- 172.21.15.53:0/2488258892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00d1aa0 0x7f33b00d1f30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009e50 tx=0x7f33a802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415423ff70bf2e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.387+0000 7f33add9b640 1 -- 172.21.15.53:0/2488258892 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8036070 con 0x7f33b00d1aa0 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.387+0000 7f33add9b640 1 -- 172.21.15.53:0/2488258892 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a8031600 con 0x7f33b00d1aa0 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.387+0000 7f33add9b640 1 -- 172.21.15.53:0/2488258892 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80319d0 con 0x7f33b00d1aa0 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.387+0000 7f33b4ada640 1 -- 172.21.15.53:0/2488258892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00d1aa0 msgr2=0x7f33b00d1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 --2- 172.21.15.53:0/2488258892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00d1aa0 0x7f33b00d1f30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009e50 tx=0x7f33a802a6c0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 -- 172.21.15.53:0/2488258892 shutdown_connections 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 --2- 172.21.15.53:0/2488258892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00d1aa0 0x7f33b00d1f30 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 --2- 172.21.15.53:0/2488258892 >> v2:172.21.15.135:3300/0 conn(0x7f33b00b5f30 0x7f33b00d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 --2- 172.21.15.53:0/2488258892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 0x7f33b00b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 -- 172.21.15.53:0/2488258892 >> 172.21.15.53:0/2488258892 conn(0x7f33b009e3f0 msgr2=0x7f33b00d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 -- 172.21.15.53:0/2488258892 shutdown_connections 2024-05-10T10:43:31.391 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.388+0000 7f33b4ada640 1 -- 172.21.15.53:0/2488258892 wait complete. 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.389+0000 7f33b4ada640 1 Processor -- start 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.389+0000 7f33b4ada640 1 -- start start 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33b4ada640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00b5f30 0x7f33b00ba810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33b4ada640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b00d1aa0 0x7f33b01f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33b4ada640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 0x7f33b01f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33b4ada640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00d5ef0 con 0x7f33b00b5f30 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33b4ada640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33b0162d70 con 0x7f33b00d1aa0 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33b4ada640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f33b01630a0 con 0x7f33b0162990 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00b5f30 0x7f33b00ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00b5f30 0x7f33b00ba810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50034/0 (socket says 172.21.15.53:50034) 2024-05-10T10:43:31.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 -- 172.21.15.53:0/988350477 learned_addr learned my addr 172.21.15.53:0/988350477 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33a5d9b640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b00d1aa0 0x7f33b01f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33aed9d640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 0x7f33b01f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 -- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 msgr2=0x7f33b01f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33b0162990 0x7f33b01f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 -- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b00d1aa0 msgr2=0x7f33b01f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b00d1aa0 0x7f33b01f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.390+0000 7f33ae59c640 1 -- 172.21.15.53:0/988350477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b014f090 con 0x7f33b00b5f30 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.391+0000 7f33ae59c640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00b5f30 0x7f33b00ba810 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e870 tx=0x7f33a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.391+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0017070 con 0x7f33b00b5f30 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.391+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a000c8e0 con 0x7f33b00b5f30 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.391+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a000ccd0 con 0x7f33b00b5f30 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.391+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b014f370 con 0x7f33b00b5f30 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.391+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b014f930 con 0x7f33b00b5f30 2024-05-10T10:43:31.395 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.392+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33b00bad50 con 0x7f33b00b5f30 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.392+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f33a001a070 con 0x7f33b00b5f30 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.393+0000 7f33a77fe640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33900762d0 0x7f3390078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.393+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f33a0097830 con 0x7f33b00b5f30 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.393+0000 7f33a77fe640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f339007c8b0 0x7f339007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.393+0000 7f33a5d9b640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33900762d0 0x7f3390078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.393+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f339007f3c0 con 0x7f339007c8b0 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.393+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f33a009e050 con 0x7f33b00b5f30 2024-05-10T10:43:31.396 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.393+0000 7f33aed9d640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f339007c8b0 0x7f339007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.394+0000 7f33a5d9b640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33900762d0 0x7f3390078790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f33b00697c0 tx=0x7f3398008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.394+0000 7f33aed9d640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f339007c8b0 0x7f339007ecf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.394+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== osd.3 v2:172.21.15.53:6816/2871564334 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f339007f3c0 con 0x7f339007c8b0 2024-05-10T10:43:31.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 --> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f33b014fd80 con 0x7f339007c8b0 2024-05-10T10:43:31.408 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33a77fe640 1 -- 172.21.15.53:0/988350477 <== osd.3 v2:172.21.15.53:6816/2871564334 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f33b014fd80 con 0x7f339007c8b0 2024-05-10T10:43:31.408 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:31.409 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:31.409 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f339007c8b0 msgr2=0x7f339007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:6816/2871564334,v1:172.21.15.53:6817/2871564334] conn(0x7f339007c8b0 0x7f339007ecf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33900762d0 msgr2=0x7f3390078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f33900762d0 0x7f3390078790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f33b00697c0 tx=0x7f3398008040 comp rx=0 tx=0).stop 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00b5f30 msgr2=0x7f33b00ba810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 --2- 172.21.15.53:0/988350477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33b00b5f30 0x7f33b00ba810 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e870 tx=0x7f33a000ed30 comp rx=0 tx=0).stop 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33aed9d640 1 -- 172.21.15.53:0/988350477 reap_dead start 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 shutdown_connections 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 >> 172.21.15.53:0/988350477 conn(0x7f33b009e3f0 msgr2=0x7f33b006e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 shutdown_connections 2024-05-10T10:43:31.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.408+0000 7f33b4ada640 1 -- 172.21.15.53:0/988350477 wait complete. 2024-05-10T10:43:31.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-10T10:43:31.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.524+0000 7fb89acda640 1 Processor -- start 2024-05-10T10:43:31.525 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.524+0000 7fb89acda640 1 -- start start 2024-05-10T10:43:31.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb89acda640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb89415f4e0 0x7fb89406d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.527 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb89acda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89415f8c0 0x7fb89406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb89acda640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb89406e450 0x7fb8940afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb89acda640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb8940b23f0 con 0x7fb89415f4e0 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb89acda640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb8940b4150 con 0x7fb89406e450 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb89acda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb89406e8c0 con 0x7fb89415f8c0 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb89415f4e0 0x7fb89406d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb89415f4e0 0x7fb89406d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50038/0 (socket says 172.21.15.53:50038) 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 -- 172.21.15.53:0/1276154958 learned_addr learned my addr 172.21.15.53:0/1276154958 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb899277640 1 --2- 172.21.15.53:0/1276154958 >> v2:172.21.15.135:3300/0 conn(0x7fb89406e450 0x7fb8940afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb893fff640 1 --2- 172.21.15.53:0/1276154958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89415f8c0 0x7fb89406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 -- 172.21.15.53:0/1276154958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89415f8c0 msgr2=0x7fb89406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 --2- 172.21.15.53:0/1276154958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89415f8c0 0x7fb89406df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 -- 172.21.15.53:0/1276154958 >> v2:172.21.15.135:3300/0 conn(0x7fb89406e450 msgr2=0x7fb8940afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 --2- 172.21.15.53:0/1276154958 >> v2:172.21.15.135:3300/0 conn(0x7fb89406e450 0x7fb8940afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.525+0000 7fb898a76640 1 -- 172.21.15.53:0/1276154958 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8940b3540 con 0x7fb89415f4e0 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.526+0000 7fb898a76640 1 --2- 172.21.15.53:0/1276154958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb89406d9d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb894073990 tx=0x7fb88802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c34d4c1b02731da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.526+0000 7fb8937fe640 1 -- 172.21.15.53:0/1276154958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888037070 con 0x7fb89415f4e0 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.526+0000 7fb8937fe640 1 -- 172.21.15.53:0/1276154958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb888032440 con 0x7fb89415f4e0 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.526+0000 7fb8937fe640 1 -- 172.21.15.53:0/1276154958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888032810 con 0x7fb89415f4e0 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 -- 172.21.15.53:0/1276154958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 msgr2=0x7fb89406d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 --2- 172.21.15.53:0/1276154958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb89406d9d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb894073990 tx=0x7fb88802a140 comp rx=0 tx=0).stop 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 -- 172.21.15.53:0/1276154958 shutdown_connections 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 --2- 172.21.15.53:0/1276154958 >> v2:172.21.15.135:3300/0 conn(0x7fb89406e450 0x7fb8940afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 --2- 172.21.15.53:0/1276154958 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89415f8c0 0x7fb89406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 --2- 172.21.15.53:0/1276154958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb89406d9d0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 -- 172.21.15.53:0/1276154958 >> 172.21.15.53:0/1276154958 conn(0x7fb8940d2df0 msgr2=0x7fb8940d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 -- 172.21.15.53:0/1276154958 shutdown_connections 2024-05-10T10:43:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.527+0000 7fb89acda640 1 -- 172.21.15.53:0/1276154958 wait complete. 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.528+0000 7fb89acda640 1 Processor -- start 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.528+0000 7fb89acda640 1 -- start start 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb89acda640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89406e450 0x7fb894159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb89acda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb8940c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb89acda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb89415f8c0 0x7fb894152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb89acda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb8940b3de0 con 0x7fb89415f4e0 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb89acda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8940b3b00 con 0x7fb89415f8c0 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb89acda640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb8940b45e0 con 0x7fb89406e450 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb8940c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb8940c96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50042/0 (socket says 172.21.15.53:50042) 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 -- 172.21.15.53:0/4029321996 learned_addr learned my addr 172.21.15.53:0/4029321996 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb898a76640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89406e450 0x7fb894159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 -- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89406e450 msgr2=0x7fb894159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89406e450 0x7fb894159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 -- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb89415f8c0 msgr2=0x7fb894152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb899277640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb89415f8c0 0x7fb894152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb89415f8c0 0x7fb894152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.529+0000 7fb893fff640 1 -- 172.21.15.53:0/4029321996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8940c9c10 con 0x7fb89415f4e0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.530+0000 7fb893fff640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb8940c96d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb88000d7d0 tx=0x7fb88000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.530+0000 7fb8917fa640 1 -- 172.21.15.53:0/4029321996 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000b6c0 con 0x7fb89415f4e0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.530+0000 7fb8917fa640 1 -- 172.21.15.53:0/4029321996 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb880019070 con 0x7fb89415f4e0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.530+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb894159610 con 0x7fb89415f4e0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.530+0000 7fb8917fa640 1 -- 172.21.15.53:0/4029321996 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000bac0 con 0x7fb89415f4e0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.530+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8941532e0 con 0x7fb89415f4e0 2024-05-10T10:43:31.533 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.531+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb894152df0 con 0x7fb89415f4e0 2024-05-10T10:43:31.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.531+0000 7fb8917fa640 1 -- 172.21.15.53:0/4029321996 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb88000bc60 con 0x7fb89415f4e0 2024-05-10T10:43:31.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.532+0000 7fb8917fa640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb8780762d0 0x7fb878078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.532+0000 7fb8917fa640 1 -- 172.21.15.53:0/4029321996 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7fb8800979e0 con 0x7fb89415f4e0 2024-05-10T10:43:31.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.532+0000 7fb898a76640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb8780762d0 0x7fb878078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.533+0000 7fb898a76640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb8780762d0 0x7fb878078790 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb888034000 tx=0x7fb88802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.536+0000 7fb8917fa640 1 -- 172.21.15.53:0/4029321996 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb8800613f0 con 0x7fb89415f4e0 2024-05-10T10:43:31.729 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.728+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb894071f60 con 0x7fb8780762d0 2024-05-10T10:43:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.729+0000 7fb8917fa640 1 -- 172.21.15.53:0/4029321996 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb894071f60 con 0x7fb8780762d0 2024-05-10T10:43:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 3 to deep-scrub 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb8780762d0 msgr2=0x7fb878078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb8780762d0 0x7fb878078790 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb888034000 tx=0x7fb88802e000 comp rx=0 tx=0).stop 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 msgr2=0x7fb8940c96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb8940c96d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb88000d7d0 tx=0x7fb88000dc90 comp rx=0 tx=0).stop 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 shutdown_connections 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb8780762d0 0x7fb878078790 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb89415f8c0 0x7fb894152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb89415f4e0 0x7fb8940c96d0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 --2- 172.21.15.53:0/4029321996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb89406e450 0x7fb894159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 >> 172.21.15.53:0/4029321996 conn(0x7fb8940d2df0 msgr2=0x7fb894071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 shutdown_connections 2024-05-10T10:43:31.733 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.731+0000 7fb89acda640 1 -- 172.21.15.53:0/4029321996 wait complete. 2024-05-10T10:43:31.743 INFO:tasks.ceph:Scrubbing osd.4 2024-05-10T10:43:31.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:31.850 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.850+0000 7f0f18693640 1 Processor -- start 2024-05-10T10:43:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.850+0000 7f0f18693640 1 -- start start 2024-05-10T10:43:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f18693640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0f100cc800 0x7f0f1016a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f18693640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f100cf720 con 0x7f0f10070a40 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f18693640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f100cfa00 con 0x7f0f100d1c50 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f18693640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0f100d0330 con 0x7f0f100cc800 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1642f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0f10070a40 0x7f0f100d0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58012/0 (socket says 172.21.15.53:58012) 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1642f640 1 -- 172.21.15.53:0/2248369881 learned_addr learned my addr 172.21.15.53:0/2248369881 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f16c30640 1 --2- 172.21.15.53:0/2248369881 >> v2:172.21.15.135:3300/0 conn(0x7f0f100cc800 0x7f0f1016a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834989078 0 0) 0x7f0f100cf720 con 0x7f0f10070a40 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0efc003520 con 0x7f0f10070a40 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2697203626 0 0) 0x7f0efc003520 con 0x7f0f10070a40 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 >> v2:172.21.15.135:3300/0 conn(0x7f0f100cc800 msgr2=0x7f0f1016a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 --2- 172.21.15.53:0/2248369881 >> v2:172.21.15.135:3300/0 conn(0x7f0f100cc800 0x7f0f1016a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 >> v1:172.21.15.135:6789/0 conn(0x7f0f100d1c50 legacy=0x7f0f10073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f100cfce0 con 0x7f0f10070a40 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7f0f0c001670 con 0x7f0f10070a40 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 >> v1:172.21.15.53:6789/0 conn(0x7f0f10070a40 legacy=0x7f0f100d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.851+0000 7f0f1542d640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc0039b0 0x7f0efc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f1542d640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0efc004800 0x7f0efc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f1542d640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0efc0051c0 0x7f0efc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100d0330 con 0x7f0efc004800 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc00c060 con 0x7f0efc0051c0 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc00c260 con 0x7f0efc0039b0 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f15c2e640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0efc004800 0x7f0efc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f1642f640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc0039b0 0x7f0efc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f16c30640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0efc0051c0 0x7f0efc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f15c2e640 1 -- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc0039b0 msgr2=0x7f0efc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f15c2e640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc0039b0 0x7f0efc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f15c2e640 1 -- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0efc0051c0 msgr2=0x7f0efc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f15c2e640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0efc0051c0 0x7f0efc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f15c2e640 1 -- 172.21.15.53:0/2248369881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc003520 con 0x7f0efc004800 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.852+0000 7f0f16c30640 1 -- 172.21.15.53:0/2248369881 reap_dead start 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f15c2e640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0efc004800 0x7f0efc004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0f00001720 tx=0x7f0efc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac46ca6c06240fa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00001370 con 0x7f0efc004800 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f000052d0 con 0x7f0efc004800 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f1542d640 1 -- 172.21.15.53:0/2248369881 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0002ab90 con 0x7f0efc004800 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f18693640 1 -- 172.21.15.53:0/2248369881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0efc004800 msgr2=0x7f0efc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f18693640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0efc004800 0x7f0efc004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0f00001720 tx=0x7f0efc005830 comp rx=0 tx=0).stop 2024-05-10T10:43:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f18693640 1 -- 172.21.15.53:0/2248369881 shutdown_connections 2024-05-10T10:43:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f18693640 1 --2- 172.21.15.53:0/2248369881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0efc004800 0x7f0efc004c80 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.853+0000 7f0f18693640 1 -- 172.21.15.53:0/2248369881 >> 172.21.15.53:0/2248369881 conn(0x7f0f100d10a0 msgr2=0x7f0f100d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.854+0000 7f0f18693640 1 -- 172.21.15.53:0/2248369881 shutdown_connections 2024-05-10T10:43:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.854+0000 7f0f18693640 1 -- 172.21.15.53:0/2248369881 wait complete. 2024-05-10T10:43:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.855+0000 7f0f18693640 1 Processor -- start 2024-05-10T10:43:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.855+0000 7f0f18693640 1 -- start start 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f18693640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc004800 0x7f0f100b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f18693640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1015af60 0x7f0f100ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f18693640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f100ca6c0 0x7f0f100a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f18693640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100d08c0 con 0x7f0f1015af60 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f18693640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100d0740 con 0x7f0f100ca6c0 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f18693640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0f100d0a40 con 0x7f0efc004800 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1015af60 0x7f0f100ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1015af60 0x7f0f100ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50066/0 (socket says 172.21.15.53:50066) 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 -- 172.21.15.53:0/1794014007 learned_addr learned my addr 172.21.15.53:0/1794014007 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.863 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f16c30640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f100ca6c0 0x7f0f100a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f1642f640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc004800 0x7f0f100b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 -- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc004800 msgr2=0x7f0f100b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc004800 0x7f0f100b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 -- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f100ca6c0 msgr2=0x7f0f100a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f100ca6c0 0x7f0f100a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 -- 172.21.15.53:0/1794014007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1015b430 con 0x7f0f1015af60 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f16c30640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f100ca6c0 0x7f0f100a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f1642f640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0efc004800 0x7f0f100b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f15c2e640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1015af60 0x7f0f100ca180 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0f00000c00 tx=0x7f0f00001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00037040 con 0x7f0f1015af60 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f00005480 con 0x7f0f1015af60 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f100b6760 con 0x7f0f1015af60 2024-05-10T10:43:31.864 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.856+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00007a30 con 0x7f0f1015af60 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.857+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f100b6d20 con 0x7f0f1015af60 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.857+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f1006a220 con 0x7f0f1015af60 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.858+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0f00044020 con 0x7f0f1015af60 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.858+0000 7f0f06ffd640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ef4076300 0x7f0ef40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f0f000b72a0 con 0x7f0f1015af60 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f06ffd640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7f0ef407c8e0 0x7f0ef407ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f00004e90 con 0x7f0ef407c8e0 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f0f000be050 con 0x7f0f1015af60 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f1642f640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ef4076300 0x7f0ef40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f16c30640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7f0ef407c8e0 0x7f0ef407ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f1642f640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ef4076300 0x7f0ef40787c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0f10069aa0 tx=0x7f0f0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.859+0000 7f0f16c30640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7f0ef407c8e0 0x7f0ef407ed20 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.860+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== osd.4 v2:172.21.15.135:6800/3517077655 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f00004e90 con 0x7f0ef407c8e0 2024-05-10T10:43:31.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.874+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 --> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0f100caaa0 con 0x7f0ef407c8e0 2024-05-10T10:43:31.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.874+0000 7f0f06ffd640 1 -- 172.21.15.53:0/1794014007 <== osd.4 v2:172.21.15.135:6800/3517077655 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0f100caaa0 con 0x7f0ef407c8e0 2024-05-10T10:43:31.875 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:31.875 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:31.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7f0ef407c8e0 msgr2=0x7f0ef407ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6800/3517077655,v1:172.21.15.135:6801/3517077655] conn(0x7f0ef407c8e0 0x7f0ef407ed20 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ef4076300 msgr2=0x7f0ef40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ef4076300 0x7f0ef40787c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0f10069aa0 tx=0x7f0f0c009040 comp rx=0 tx=0).stop 2024-05-10T10:43:31.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1015af60 msgr2=0x7f0f100ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 --2- 172.21.15.53:0/1794014007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f1015af60 0x7f0f100ca180 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0f00000c00 tx=0x7f0f00001120 comp rx=0 tx=0).stop 2024-05-10T10:43:31.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f16c30640 1 -- 172.21.15.53:0/1794014007 reap_dead start 2024-05-10T10:43:31.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 shutdown_connections 2024-05-10T10:43:31.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 >> 172.21.15.53:0/1794014007 conn(0x7f0f100d10a0 msgr2=0x7f0f1006f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 shutdown_connections 2024-05-10T10:43:31.877 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.875+0000 7f0f18693640 1 -- 172.21.15.53:0/1794014007 wait complete. 2024-05-10T10:43:31.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-10T10:43:31.990 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.990+0000 7f80f539d640 1 Processor -- start 2024-05-10T10:43:31.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.990+0000 7f80f539d640 1 -- start start 2024-05-10T10:43:31.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80f539d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80f00c6c80 0x7f80f00c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.991 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80f539d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f006f1c0 con 0x7f80f01598c0 2024-05-10T10:43:31.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80f539d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f006f4a0 con 0x7f80f00cee40 2024-05-10T10:43:31.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80f539d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f80f006f780 con 0x7f80f00c6c80 2024-05-10T10:43:31.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80ee7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80f00c6c80 0x7f80f00c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80eeffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f80f01598c0 0x7f80f006e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58014/0 (socket says 172.21.15.53:58014) 2024-05-10T10:43:31.992 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80eeffd640 1 -- 172.21.15.53:0/1439434112 learned_addr learned my addr 172.21.15.53:0/1439434112 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:31.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80ee7fc640 1 -- 172.21.15.53:0/1439434112 >> v1:172.21.15.135:6789/0 conn(0x7f80f00cee40 legacy=0x7f80f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80ee7fc640 1 -- 172.21.15.53:0/1439434112 >> v1:172.21.15.53:6789/0 conn(0x7f80f01598c0 legacy=0x7f80f006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.991+0000 7f80ee7fc640 1 -- 172.21.15.53:0/1439434112 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f006fa60 con 0x7f80f00c6c80 2024-05-10T10:43:31.993 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.992+0000 7f80ee7fc640 1 --2- 172.21.15.53:0/1439434112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f00c6c80 0x7f80f00c7110 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f80d80097a0 tx=0x7f80d802c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6daf9a649180fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.992+0000 7f80edffb640 1 -- 172.21.15.53:0/1439434112 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8038070 con 0x7f80f00c6c80 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.992+0000 7f80edffb640 1 -- 172.21.15.53:0/1439434112 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d8031710 con 0x7f80f00c6c80 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.992+0000 7f80edffb640 1 -- 172.21.15.53:0/1439434112 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8031ae0 con 0x7f80f00c6c80 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.993+0000 7f80f539d640 1 -- 172.21.15.53:0/1439434112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f00c6c80 msgr2=0x7f80f00c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.993+0000 7f80f539d640 1 --2- 172.21.15.53:0/1439434112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f00c6c80 0x7f80f00c7110 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f80d80097a0 tx=0x7f80d802c440 comp rx=0 tx=0).stop 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.993+0000 7f80f539d640 1 -- 172.21.15.53:0/1439434112 shutdown_connections 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.993+0000 7f80f539d640 1 --2- 172.21.15.53:0/1439434112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f00c6c80 0x7f80f00c7110 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.993+0000 7f80f539d640 1 -- 172.21.15.53:0/1439434112 >> 172.21.15.53:0/1439434112 conn(0x7f80f00b2900 msgr2=0x7f80f00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.993+0000 7f80f539d640 1 -- 172.21.15.53:0/1439434112 shutdown_connections 2024-05-10T10:43:31.994 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.993+0000 7f80f539d640 1 -- 172.21.15.53:0/1439434112 wait complete. 2024-05-10T10:43:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.994+0000 7f80f539d640 1 Processor -- start 2024-05-10T10:43:31.995 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.994+0000 7f80f539d640 1 -- start start 2024-05-10T10:43:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80f539d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f00c6c80 0x7f80f0073e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80f539d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f00cee40 0x7f80f01f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80f539d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f01598c0 0x7f80f00af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80f539d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f80f006ee10 con 0x7f80f01598c0 2024-05-10T10:43:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80f539d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80f015c6e0 con 0x7f80f00cee40 2024-05-10T10:43:31.997 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80f539d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f80f01f5750 con 0x7f80f00c6c80 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f01598c0 0x7f80f00af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f01598c0 0x7f80f00af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50084/0 (socket says 172.21.15.53:50084) 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80ef7fe640 1 -- 172.21.15.53:0/3146215806 learned_addr learned my addr 172.21.15.53:0/3146215806 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80ee7fc640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f00cee40 0x7f80f01f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.995+0000 7f80eeffd640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f00c6c80 0x7f80f0073e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80ef7fe640 1 -- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f00c6c80 msgr2=0x7f80f0073e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80ef7fe640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f00c6c80 0x7f80f0073e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80ef7fe640 1 -- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f00cee40 msgr2=0x7f80f01f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80ef7fe640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f00cee40 0x7f80f01f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80ef7fe640 1 -- 172.21.15.53:0/3146215806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f01f5a20 con 0x7f80f01598c0 2024-05-10T10:43:31.998 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80ef7fe640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f01598c0 0x7f80f00af010 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f80e8001eb0 tx=0x7f80e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:31.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80e77fe640 1 -- 172.21.15.53:0/3146215806 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8017070 con 0x7f80f01598c0 2024-05-10T10:43:31.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80e77fe640 1 -- 172.21.15.53:0/3146215806 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e80052d0 con 0x7f80f01598c0 2024-05-10T10:43:31.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f01f5d00 con 0x7f80f01598c0 2024-05-10T10:43:31.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80e77fe640 1 -- 172.21.15.53:0/3146215806 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8003810 con 0x7f80f01598c0 2024-05-10T10:43:31.999 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.996+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f00af550 con 0x7f80f01598c0 2024-05-10T10:43:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.997+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f006a220 con 0x7f80f01598c0 2024-05-10T10:43:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.998+0000 7f80e77fe640 1 -- 172.21.15.53:0/3146215806 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f80e801a070 con 0x7f80f01598c0 2024-05-10T10:43:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.998+0000 7f80e77fe640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80d00762d0 0x7f80d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.998+0000 7f80e77fe640 1 -- 172.21.15.53:0/3146215806 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f80e8096840 con 0x7f80f01598c0 2024-05-10T10:43:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.998+0000 7f80eeffd640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80d00762d0 0x7f80d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:31.999+0000 7f80eeffd640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80d00762d0 0x7f80d0078790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005c00 tx=0x7f80e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.002+0000 7f80e77fe640 1 -- 172.21.15.53:0/3146215806 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f80e80602f0 con 0x7f80f01598c0 2024-05-10T10:43:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.196+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f80f01f6220 con 0x7f80d00762d0 2024-05-10T10:43:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.197+0000 7f80e77fe640 1 -- 172.21.15.53:0/3146215806 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f80e0001950 con 0x7f80d00762d0 2024-05-10T10:43:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 4 to deep-scrub 2024-05-10T10:43:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80d00762d0 msgr2=0x7f80d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80d00762d0 0x7f80d0078790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005c00 tx=0x7f80e0009040 comp rx=0 tx=0).stop 2024-05-10T10:43:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f01598c0 msgr2=0x7f80f00af010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f01598c0 0x7f80f00af010 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f80e8001eb0 tx=0x7f80e800c890 comp rx=0 tx=0).stop 2024-05-10T10:43:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 shutdown_connections 2024-05-10T10:43:32.200 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80d00762d0 0x7f80d0078790 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80f01598c0 0x7f80f00af010 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f00cee40 0x7f80f01f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 --2- 172.21.15.53:0/3146215806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f00c6c80 0x7f80f0073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 >> 172.21.15.53:0/3146215806 conn(0x7f80f00b2900 msgr2=0x7f80f00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.199+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 shutdown_connections 2024-05-10T10:43:32.201 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.200+0000 7f80f539d640 1 -- 172.21.15.53:0/3146215806 wait complete. 2024-05-10T10:43:32.211 INFO:tasks.ceph:Scrubbing osd.5 2024-05-10T10:43:32.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.319+0000 7f0394b0a640 1 Processor -- start 2024-05-10T10:43:32.319 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.319+0000 7f0394b0a640 1 -- start start 2024-05-10T10:43:32.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.319+0000 7f0394b0a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03900d5ca0 0x7f03900d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.319+0000 7f0394b0a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03900d1080 con 0x7f039015f490 2024-05-10T10:43:32.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.319+0000 7f0394b0a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03900b4820 con 0x7f03900d3d50 2024-05-10T10:43:32.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.319+0000 7f0394b0a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f03900b4e70 con 0x7f03900d5ca0 2024-05-10T10:43:32.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.320+0000 7f038ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03900d5ca0 0x7f03900d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.321 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.320+0000 7f038e59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f039015f490 0x7f03900d3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58030/0 (socket says 172.21.15.53:58030) 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.320+0000 7f038e59c640 1 -- 172.21.15.53:0/2275864413 learned_addr learned my addr 172.21.15.53:0/2275864413 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.320+0000 7f038ed9d640 1 -- 172.21.15.53:0/2275864413 >> v1:172.21.15.135:6789/0 conn(0x7f03900d3d50 legacy=0x7f03900b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.320+0000 7f038ed9d640 1 -- 172.21.15.53:0/2275864413 >> v1:172.21.15.53:6789/0 conn(0x7f039015f490 legacy=0x7f03900d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.320+0000 7f038dd9b640 1 -- 172.21.15.53:0/2275864413 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283084639 0 0) 0x7f03900d1080 con 0x7f039015f490 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.320+0000 7f038ed9d640 1 -- 172.21.15.53:0/2275864413 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03900b4b00 con 0x7f03900d5ca0 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f038ed9d640 1 --2- 172.21.15.53:0/2275864413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d5ca0 0x7f03900d2f60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0388009d90 tx=0x7f038802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e951044de81993 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f038dd9b640 1 -- 172.21.15.53:0/2275864413 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388007740 con 0x7f03900d5ca0 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f038dd9b640 1 -- 172.21.15.53:0/2275864413 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03880078e0 con 0x7f03900d5ca0 2024-05-10T10:43:32.322 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f038dd9b640 1 -- 172.21.15.53:0/2275864413 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388007cb0 con 0x7f03900d5ca0 2024-05-10T10:43:32.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f0394b0a640 1 -- 172.21.15.53:0/2275864413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d5ca0 msgr2=0x7f03900d2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f0394b0a640 1 --2- 172.21.15.53:0/2275864413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d5ca0 0x7f03900d2f60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0388009d90 tx=0x7f038802a660 comp rx=0 tx=0).stop 2024-05-10T10:43:32.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f0394b0a640 1 -- 172.21.15.53:0/2275864413 shutdown_connections 2024-05-10T10:43:32.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f0394b0a640 1 --2- 172.21.15.53:0/2275864413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d5ca0 0x7f03900d2f60 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f0394b0a640 1 -- 172.21.15.53:0/2275864413 >> 172.21.15.53:0/2275864413 conn(0x7f03900d50f0 msgr2=0x7f03900d5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f0394b0a640 1 -- 172.21.15.53:0/2275864413 shutdown_connections 2024-05-10T10:43:32.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.321+0000 7f0394b0a640 1 -- 172.21.15.53:0/2275864413 wait complete. 2024-05-10T10:43:32.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.323+0000 7f0394b0a640 1 Processor -- start 2024-05-10T10:43:32.324 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.323+0000 7f0394b0a640 1 -- start start 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d3d50 0x7f0390155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03900d5ca0 0x7f039015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f039015f490 0x7f03900c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f039015f9f0 con 0x7f03900d3d50 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f039015f870 con 0x7f039015f490 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f03900b5a10 con 0x7f03900d5ca0 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d3d50 0x7f0390155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d3d50 0x7f0390155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50098/0 (socket says 172.21.15.53:50098) 2024-05-10T10:43:32.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 -- 172.21.15.53:0/3674356918 learned_addr learned my addr 172.21.15.53:0/3674356918 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038ed9d640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f039015f490 0x7f03900c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0387fff640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03900d5ca0 0x7f039015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 -- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03900d5ca0 msgr2=0x7f039015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f03900d5ca0 0x7f039015bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 -- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f039015f490 msgr2=0x7f03900c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f039015f490 0x7f03900c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 -- 172.21.15.53:0/3674356918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03901e6bc0 con 0x7f03900d3d50 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f038e59c640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d3d50 0x7f0390155f20 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f038000c420 tx=0x7f038000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03800074e0 con 0x7f03900d3d50 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03800047e0 con 0x7f03900d3d50 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380004c10 con 0x7f03900d3d50 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03901f4bf0 con 0x7f03900d3d50 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.324+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039015c980 con 0x7f03900d3d50 2024-05-10T10:43:32.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.325+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f039006a220 con 0x7f03900d3d50 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.326+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0380011070 con 0x7f03900d3d50 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.326+0000 7f0386ffd640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03700762d0 0x7f0370078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f0380097110 con 0x7f03900d3d50 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f0387fff640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03700762d0 0x7f0370078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f0386ffd640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7f037007c8b0 0x7f037007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f037007f3c0 con 0x7f037007c8b0 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f038009e050 con 0x7f03900d3d50 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f038ed9d640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7f037007c8b0 0x7f037007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f0387fff640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03700762d0 0x7f0370078790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0390156460 tx=0x7f0378009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.327+0000 7f038ed9d640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7f037007c8b0 0x7f037007ecf0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.328+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== osd.5 v2:172.21.15.135:6824/3244007917 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f037007f3c0 con 0x7f037007c8b0 2024-05-10T10:43:32.341 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.341+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 --> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f039015ce10 con 0x7f037007c8b0 2024-05-10T10:43:32.342 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0386ffd640 1 -- 172.21.15.53:0/3674356918 <== osd.5 v2:172.21.15.135:6824/3244007917 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f039015ce10 con 0x7f037007c8b0 2024-05-10T10:43:32.343 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:32.343 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:32.343 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7f037007c8b0 msgr2=0x7f037007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6824/3244007917,v1:172.21.15.135:6825/3244007917] conn(0x7f037007c8b0 0x7f037007ecf0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03700762d0 msgr2=0x7f0370078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f03700762d0 0x7f0370078790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0390156460 tx=0x7f0378009040 comp rx=0 tx=0).stop 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d3d50 msgr2=0x7f0390155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 --2- 172.21.15.53:0/3674356918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03900d3d50 0x7f0390155f20 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f038000c420 tx=0x7f038000c8e0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f038ed9d640 1 -- 172.21.15.53:0/3674356918 reap_dead start 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 shutdown_connections 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.342+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 >> 172.21.15.53:0/3674356918 conn(0x7f03900d50f0 msgr2=0x7f03900b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.343+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 shutdown_connections 2024-05-10T10:43:32.344 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.343+0000 7f0394b0a640 1 -- 172.21.15.53:0/3674356918 wait complete. 2024-05-10T10:43:32.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-10T10:43:32.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.457+0000 7f61366f1640 1 Processor -- start 2024-05-10T10:43:32.458 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.457+0000 7f61366f1640 1 -- start start 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.458+0000 7f61366f1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f613015f4e0 0x7f613006d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.458+0000 7f61366f1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61300b0a90 con 0x7f613015f8c0 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.458+0000 7f61366f1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61300b3540 con 0x7f613006e620 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.458+0000 7f61366f1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f61300b23f0 con 0x7f613015f4e0 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.458+0000 7f612ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f613015f4e0 0x7f613006d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.458+0000 7f612f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f613015f8c0 0x7f613006df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58042/0 (socket says 172.21.15.53:58042) 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.458+0000 7f612f7fe640 1 -- 172.21.15.53:0/1951382167 learned_addr learned my addr 172.21.15.53:0/1951382167 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.459+0000 7f612ffff640 1 -- 172.21.15.53:0/1951382167 >> v1:172.21.15.135:6789/0 conn(0x7f613006e620 legacy=0x7f61300d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:32.460 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.459+0000 7f612ffff640 1 -- 172.21.15.53:0/1951382167 >> v1:172.21.15.53:6789/0 conn(0x7f613015f8c0 legacy=0x7f613006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.459+0000 7f612effd640 1 -- 172.21.15.53:0/1951382167 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2186546150 0 0) 0x7f61300b0a90 con 0x7f613015f8c0 2024-05-10T10:43:32.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.459+0000 7f612ffff640 1 -- 172.21.15.53:0/1951382167 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61300b3820 con 0x7f613015f4e0 2024-05-10T10:43:32.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.459+0000 7f612ffff640 1 --2- 172.21.15.53:0/1951382167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f613006d9d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f61200097a0 tx=0x7f612002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678283c53529985d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.459+0000 7f612effd640 1 -- 172.21.15.53:0/1951382167 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120037070 con 0x7f613015f4e0 2024-05-10T10:43:32.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f612effd640 1 -- 172.21.15.53:0/1951382167 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612002ae50 con 0x7f613015f4e0 2024-05-10T10:43:32.461 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f612effd640 1 -- 172.21.15.53:0/1951382167 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120032660 con 0x7f613015f4e0 2024-05-10T10:43:32.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f61366f1640 1 -- 172.21.15.53:0/1951382167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 msgr2=0x7f613006d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.464 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f61366f1640 1 --2- 172.21.15.53:0/1951382167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f613006d9d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f61200097a0 tx=0x7f612002a140 comp rx=0 tx=0).stop 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f61366f1640 1 -- 172.21.15.53:0/1951382167 shutdown_connections 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f61366f1640 1 --2- 172.21.15.53:0/1951382167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f613006d9d0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f61366f1640 1 -- 172.21.15.53:0/1951382167 >> 172.21.15.53:0/1951382167 conn(0x7f61300d2df0 msgr2=0x7f61300d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f61366f1640 1 -- 172.21.15.53:0/1951382167 shutdown_connections 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.460+0000 7f61366f1640 1 -- 172.21.15.53:0/1951382167 wait complete. 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.461+0000 7f61366f1640 1 Processor -- start 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.461+0000 7f61366f1640 1 -- start start 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f61366f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613006e620 0x7f6130158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f61366f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f61301f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f61366f1640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f613015f8c0 0x7f61301f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f61366f1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61300b3de0 con 0x7f613015f4e0 2024-05-10T10:43:32.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f61366f1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f61300b4150 con 0x7f613006e620 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f61366f1640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f61300b45e0 con 0x7f613015f8c0 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f61301f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f61301f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50110/0 (socket says 172.21.15.53:50110) 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.462+0000 7f612f7fe640 1 -- 172.21.15.53:0/1048069093 learned_addr learned my addr 172.21.15.53:0/1048069093 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612ffff640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613006e620 0x7f6130158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f6134c8e640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f613015f8c0 0x7f61301f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612f7fe640 1 -- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f613015f8c0 msgr2=0x7f61301f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612f7fe640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f613015f8c0 0x7f61301f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612f7fe640 1 -- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613006e620 msgr2=0x7f6130158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612f7fe640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613006e620 0x7f6130158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612f7fe640 1 -- 172.21.15.53:0/1048069093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6130159200 con 0x7f613015f4e0 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612ffff640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613006e620 0x7f6130158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612f7fe640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f61301f5940 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f612400c4a0 tx=0x7f612400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612cff9640 1 -- 172.21.15.53:0/1048069093 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124018070 con 0x7f613015f4e0 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612cff9640 1 -- 172.21.15.53:0/1048069093 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61240074e0 con 0x7f613015f4e0 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61301594e0 con 0x7f613015f4e0 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f612cff9640 1 -- 172.21.15.53:0/1048069093 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124004a20 con 0x7f613015f4e0 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.463+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6130159aa0 con 0x7f613015f4e0 2024-05-10T10:43:32.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.464+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613006a220 con 0x7f613015f4e0 2024-05-10T10:43:32.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.465+0000 7f612cff9640 1 -- 172.21.15.53:0/1048069093 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f612401b070 con 0x7f613015f4e0 2024-05-10T10:43:32.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.465+0000 7f612cff9640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f611c0762d0 0x7f611c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.465+0000 7f612cff9640 1 -- 172.21.15.53:0/1048069093 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f6124096820 con 0x7f613015f4e0 2024-05-10T10:43:32.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.465+0000 7f612ffff640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f611c0762d0 0x7f611c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.466+0000 7f612ffff640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f611c0762d0 0x7f611c078790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6120034000 tx=0x7f612002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.470 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.469+0000 7f612cff9640 1 -- 172.21.15.53:0/1048069093 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f6124060240 con 0x7f613015f4e0 2024-05-10T10:43:32.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.663+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f613015e970 con 0x7f611c0762d0 2024-05-10T10:43:32.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.663+0000 7f612cff9640 1 -- 172.21.15.53:0/1048069093 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f613015e970 con 0x7f611c0762d0 2024-05-10T10:43:32.664 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 5 to deep-scrub 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f611c0762d0 msgr2=0x7f611c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f611c0762d0 0x7f611c078790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6120034000 tx=0x7f612002e000 comp rx=0 tx=0).stop 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 msgr2=0x7f61301f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f61301f5940 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f612400c4a0 tx=0x7f612400c960 comp rx=0 tx=0).stop 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 shutdown_connections 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f611c0762d0 0x7f611c078790 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f613015f8c0 0x7f61301f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f613015f4e0 0x7f61301f5940 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 --2- 172.21.15.53:0/1048069093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f613006e620 0x7f6130158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 >> 172.21.15.53:0/1048069093 conn(0x7f61300d2df0 msgr2=0x7f6130071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 shutdown_connections 2024-05-10T10:43:32.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.666+0000 7f61366f1640 1 -- 172.21.15.53:0/1048069093 wait complete. 2024-05-10T10:43:32.678 INFO:tasks.ceph:Scrubbing osd.6 2024-05-10T10:43:32.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:32.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.785+0000 7f432146d640 1 Processor -- start 2024-05-10T10:43:32.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.785+0000 7f432146d640 1 -- start start 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f432146d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f431c15f6e0 0x7f431c0b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f432146d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f431c0d2df0 0x7f431c0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f432146d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431c0d5d80 con 0x7f431c0d35f0 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f432146d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f431c0d5b30 con 0x7f431c0d2df0 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f432146d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f431c0b1c60 con 0x7f431c15f6e0 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f431c0d2df0 0x7f431c0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f431c0d2df0 0x7f431c0b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59656/0 (socket says 172.21.15.53:59656) 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431b7fe640 1 -- 172.21.15.53:0/31353203 learned_addr learned my addr 172.21.15.53:0/31353203 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431a7fc640 1 --2- 172.21.15.53:0/31353203 >> v2:172.21.15.135:3300/0 conn(0x7f431c15f6e0 0x7f431c0b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431b7fe640 1 -- 172.21.15.53:0/31353203 >> v2:172.21.15.135:3300/0 conn(0x7f431c15f6e0 msgr2=0x7f431c0b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431b7fe640 1 --2- 172.21.15.53:0/31353203 >> v2:172.21.15.135:3300/0 conn(0x7f431c15f6e0 0x7f431c0b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431b7fe640 1 -- 172.21.15.53:0/31353203 >> v1:172.21.15.135:6789/0 conn(0x7f431c0d35f0 legacy=0x7f431c0d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.786+0000 7f431b7fe640 1 -- 172.21.15.53:0/31353203 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c0b1eb0 con 0x7f431c0d2df0 2024-05-10T10:43:32.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.787+0000 7f431b7fe640 1 --2- 172.21.15.53:0/31353203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0b2bf0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f431c08d850 tx=0x7f431402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dead94c0b14f37b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.787+0000 7f4319ffb640 1 -- 172.21.15.53:0/31353203 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314036070 con 0x7f431c0d2df0 2024-05-10T10:43:32.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.787+0000 7f4319ffb640 1 -- 172.21.15.53:0/31353203 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4314031600 con 0x7f431c0d2df0 2024-05-10T10:43:32.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.787+0000 7f4319ffb640 1 -- 172.21.15.53:0/31353203 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140319d0 con 0x7f431c0d2df0 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 -- 172.21.15.53:0/31353203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 msgr2=0x7f431c0b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 --2- 172.21.15.53:0/31353203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0b2bf0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f431c08d850 tx=0x7f431402a610 comp rx=0 tx=0).stop 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 -- 172.21.15.53:0/31353203 shutdown_connections 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 --2- 172.21.15.53:0/31353203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0b2bf0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 --2- 172.21.15.53:0/31353203 >> v2:172.21.15.135:3300/0 conn(0x7f431c15f6e0 0x7f431c0b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 -- 172.21.15.53:0/31353203 >> 172.21.15.53:0/31353203 conn(0x7f431c0d5010 msgr2=0x7f431c0d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 -- 172.21.15.53:0/31353203 shutdown_connections 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.788+0000 7f432146d640 1 -- 172.21.15.53:0/31353203 wait complete. 2024-05-10T10:43:32.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.789+0000 7f432146d640 1 Processor -- start 2024-05-10T10:43:32.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.789+0000 7f432146d640 1 -- start start 2024-05-10T10:43:32.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f432146d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0ba700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f432146d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f431c0d35f0 0x7f431c0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f432146d640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f431c151390 0x7f431c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f432146d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0b3fa0 con 0x7f431c0d2df0 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f432146d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0d2410 con 0x7f431c0d35f0 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f432146d640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f431c0b4120 con 0x7f431c151390 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0ba700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59658/0 (socket says 172.21.15.53:59658) 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431affd640 1 -- 172.21.15.53:0/3218236314 learned_addr learned my addr 172.21.15.53:0/3218236314 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431a7fc640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f431c0d35f0 0x7f431c0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431b7fe640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f431c151390 0x7f431c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431affd640 1 -- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f431c151390 msgr2=0x7f431c151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431affd640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f431c151390 0x7f431c151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.793 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.790+0000 7f431affd640 1 -- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f431c0d35f0 msgr2=0x7f431c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f431affd640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f431c0d35f0 0x7f431c0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f431affd640 1 -- 172.21.15.53:0/3218236314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c0cdf30 con 0x7f431c0d2df0 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f431affd640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0ba700 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c400 tx=0x7f430c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c0052d0 con 0x7f431c0d2df0 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c005490 con 0x7f431c0d2df0 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c0cce90 con 0x7f431c0d2df0 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c007260 con 0x7f431c0d2df0 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.791+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c0cd450 con 0x7f431c0d2df0 2024-05-10T10:43:32.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.792+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f431c06a220 con 0x7f431c0d2df0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f430c007400 con 0x7f431c0d2df0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f43037fe640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43080762d0 0x7f4308078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f430c097170 con 0x7f431c0d2df0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f43037fe640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7f430807c8b0 0x7f430807ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f430c00ee50 con 0x7f430807c8b0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f430c097600 con 0x7f431c0d2df0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f431a7fc640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43080762d0 0x7f4308078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.793+0000 7f431b7fe640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7f430807c8b0 0x7f430807ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.794+0000 7f431a7fc640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43080762d0 0x7f4308078790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f431c0ba460 tx=0x7f4310009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.794+0000 7f431b7fe640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7f430807c8b0 0x7f430807ecf0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.795+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== osd.6 v2:172.21.15.135:6808/1123855098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f430c00ee50 con 0x7f430807c8b0 2024-05-10T10:43:32.808 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.808+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 --> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f431c0d33d0 con 0x7f430807c8b0 2024-05-10T10:43:32.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.808+0000 7f43037fe640 1 -- 172.21.15.53:0/3218236314 <== osd.6 v2:172.21.15.135:6808/1123855098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f431c0d33d0 con 0x7f430807c8b0 2024-05-10T10:43:32.809 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:32.809 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:32.809 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:32.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7f430807c8b0 msgr2=0x7f430807ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6808/1123855098,v1:172.21.15.135:6810/1123855098] conn(0x7f430807c8b0 0x7f430807ecf0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43080762d0 msgr2=0x7f4308078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.810 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f43080762d0 0x7f4308078790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f431c0ba460 tx=0x7f4310009190 comp rx=0 tx=0).stop 2024-05-10T10:43:32.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 msgr2=0x7f431c0ba700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 --2- 172.21.15.53:0/3218236314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f431c0d2df0 0x7f431c0ba700 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c400 tx=0x7f430c00c8c0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f431b7fe640 1 -- 172.21.15.53:0/3218236314 reap_dead start 2024-05-10T10:43:32.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 shutdown_connections 2024-05-10T10:43:32.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 >> 172.21.15.53:0/3218236314 conn(0x7f431c0d5010 msgr2=0x7f431c06e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 shutdown_connections 2024-05-10T10:43:32.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.809+0000 7f432146d640 1 -- 172.21.15.53:0/3218236314 wait complete. 2024-05-10T10:43:32.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-10T10:43:32.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.924+0000 7ff798a69640 1 Processor -- start 2024-05-10T10:43:32.925 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.924+0000 7ff798a69640 1 -- start start 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.925+0000 7ff798a69640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 0x7ff7940b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.925+0000 7ff798a69640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7940d59d0 0x7ff7940d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.925+0000 7ff798a69640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7940d2d20 con 0x7ff7940b47a0 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.925+0000 7ff798a69640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff794162d20 con 0x7ff7940d59d0 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.925+0000 7ff798a69640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff7940d3000 con 0x7ff794162940 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.925+0000 7ff792d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff7940b47a0 0x7ff794162f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53766/0 (socket says 172.21.15.53:53766) 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.925+0000 7ff792d9d640 1 -- 172.21.15.53:0/1466498505 learned_addr learned my addr 172.21.15.53:0/1466498505 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79259c640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 0x7ff7940b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff791d9b640 1 --2- 172.21.15.53:0/1466498505 >> v2:172.21.15.135:3300/0 conn(0x7ff7940d59d0 0x7ff7940d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367107510 0 0) 0x7ff7940d2d20 con 0x7ff7940b47a0 2024-05-10T10:43:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff780003520 con 0x7ff7940b47a0 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3657881532 0 0) 0x7ff780003520 con 0x7ff7940b47a0 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 msgr2=0x7ff7940b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 0x7ff7940b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 >> v2:172.21.15.135:3300/0 conn(0x7ff7940d59d0 msgr2=0x7ff7940d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 --2- 172.21.15.53:0/1466498505 >> v2:172.21.15.135:3300/0 conn(0x7ff7940d59d0 0x7ff7940d3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7940d32e0 con 0x7ff7940b47a0 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1652744222 0 0) 0x7ff7840019f0 con 0x7ff7940b47a0 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 >> v1:172.21.15.53:6789/0 conn(0x7ff7940b47a0 legacy=0x7ff794162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7800039b0 0x7ff780003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff780004800 0x7ff780004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 0x7ff7800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.930 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7940d3000 con 0x7ff780004800 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff78000b9c0 con 0x7ff7800039b0 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff78000bbc0 con 0x7ff794162940 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff791d9b640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff780004800 0x7ff780004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff79259c640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7800039b0 0x7ff780003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff792d9d640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 0x7ff7800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff791d9b640 1 -- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 msgr2=0x7ff7800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff791d9b640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 0x7ff7800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff791d9b640 1 -- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7800039b0 msgr2=0x7ff780003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.926+0000 7ff791d9b640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7800039b0 0x7ff780003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.927+0000 7ff791d9b640 1 -- 172.21.15.53:0/1466498505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff780003520 con 0x7ff780004800 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.927+0000 7ff792d9d640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff794162940 0x7ff7800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.927+0000 7ff791d9b640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff780004800 0x7ff780004c80 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009ac0 tx=0x7ff780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd9943ed7c26daa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.927+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c036070 con 0x7ff780004800 2024-05-10T10:43:32.931 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.927+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77c031400 con 0x7ff780004800 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.927+0000 7ff79159a640 1 -- 172.21.15.53:0/1466498505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c009690 con 0x7ff780004800 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.928+0000 7ff798a69640 1 -- 172.21.15.53:0/1466498505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff780004800 msgr2=0x7ff780004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.928+0000 7ff798a69640 1 --2- 172.21.15.53:0/1466498505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff780004800 0x7ff780004c80 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009ac0 tx=0x7ff780005830 comp rx=0 tx=0).stop 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.928+0000 7ff792d9d640 1 -- 172.21.15.53:0/1466498505 reap_dead start 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.928+0000 7ff798a69640 1 -- 172.21.15.53:0/1466498505 shutdown_connections 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.928+0000 7ff798a69640 1 -- 172.21.15.53:0/1466498505 >> 172.21.15.53:0/1466498505 conn(0x7ff79409e380 msgr2=0x7ff7940d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.928+0000 7ff798a69640 1 -- 172.21.15.53:0/1466498505 shutdown_connections 2024-05-10T10:43:32.932 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.928+0000 7ff798a69640 1 -- 172.21.15.53:0/1466498505 wait complete. 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.929+0000 7ff798a69640 1 Processor -- start 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff798a69640 1 -- start start 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff798a69640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff780004800 0x7ff7940cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff798a69640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7940b7c90 0x7ff7941f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff798a69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff794152c00 0x7ff7940b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff798a69640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7940d2390 con 0x7ff794152c00 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff798a69640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7940cfc50 con 0x7ff7940b7c90 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff798a69640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7ff7940d2510 con 0x7ff780004800 2024-05-10T10:43:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff792d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff794152c00 0x7ff7940b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff792d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff794152c00 0x7ff7940b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59684/0 (socket says 172.21.15.53:59684) 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.930+0000 7ff792d9d640 1 -- 172.21.15.53:0/1842216503 learned_addr learned my addr 172.21.15.53:0/1842216503 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff79259c640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff780004800 0x7ff7940cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff791d9b640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7940b7c90 0x7ff7941f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff792d9d640 1 -- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff780004800 msgr2=0x7ff7940cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff792d9d640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff780004800 0x7ff7940cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff792d9d640 1 -- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7940b7c90 msgr2=0x7ff7941f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff792d9d640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7940b7c90 0x7ff7941f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff792d9d640 1 -- 172.21.15.53:0/1842216503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7940c8cf0 con 0x7ff794152c00 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff791d9b640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7940b7c90 0x7ff7941f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff792d9d640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff794152c00 0x7ff7940b3410 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff79408d970 tx=0x7ff784007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff78affd640 1 -- 172.21.15.53:0/1842216503 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78400eb40 con 0x7ff794152c00 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff78affd640 1 -- 172.21.15.53:0/1842216503 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff784018070 con 0x7ff794152c00 2024-05-10T10:43:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7940c8f90 con 0x7ff794152c00 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff78affd640 1 -- 172.21.15.53:0/1842216503 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784013420 con 0x7ff794152c00 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.931+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941530b0 con 0x7ff794152c00 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.932+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7940d4460 con 0x7ff794152c00 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.933+0000 7ff78affd640 1 -- 172.21.15.53:0/1842216503 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff7840135c0 con 0x7ff794152c00 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.933+0000 7ff78affd640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff774076300 0x7ff7740787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.933+0000 7ff78affd640 1 -- 172.21.15.53:0/1842216503 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7ff784097880 con 0x7ff794152c00 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.933+0000 7ff79259c640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff774076300 0x7ff7740787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:32.937 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.934+0000 7ff79259c640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff774076300 0x7ff7740787c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009a20 tx=0x7ff78c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:32.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:32.938+0000 7ff78affd640 1 -- 172.21.15.53:0/1842216503 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7ff7940d4460 con 0x7ff794152c00 2024-05-10T10:43:33.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.129+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ff79406a220 con 0x7ff774076300 2024-05-10T10:43:33.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.130+0000 7ff78affd640 1 -- 172.21.15.53:0/1842216503 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff79406a220 con 0x7ff774076300 2024-05-10T10:43:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 6 to deep-scrub 2024-05-10T10:43:33.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff774076300 msgr2=0x7ff7740787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff774076300 0x7ff7740787c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009a20 tx=0x7ff78c008040 comp rx=0 tx=0).stop 2024-05-10T10:43:33.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff794152c00 msgr2=0x7ff7940b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff794152c00 0x7ff7940b3410 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff79408d970 tx=0x7ff784007bc0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 shutdown_connections 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7ff774076300 0x7ff7740787c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff794152c00 0x7ff7940b3410 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7940b7c90 0x7ff7941f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 --2- 172.21.15.53:0/1842216503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff780004800 0x7ff7940cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 >> 172.21.15.53:0/1842216503 conn(0x7ff79409e380 msgr2=0x7ff7940d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 shutdown_connections 2024-05-10T10:43:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.132+0000 7ff798a69640 1 -- 172.21.15.53:0/1842216503 wait complete. 2024-05-10T10:43:33.145 INFO:tasks.ceph:Scrubbing osd.7 2024-05-10T10:43:33.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-10T10:43:33.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.252+0000 7f0ed54da640 1 Processor -- start 2024-05-10T10:43:33.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.252+0000 7f0ed54da640 1 -- start start 2024-05-10T10:43:33.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ed54da640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0ed01639a0 0x7f0ed0070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ed54da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ed0071520 0x7f0ed0162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ed54da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed00d2510 con 0x7f0ed015f370 2024-05-10T10:43:33.254 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ed54da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed0070220 con 0x7f0ed0071520 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ed54da640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed0070500 con 0x7f0ed01639a0 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ecf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ed0071520 0x7f0ed0162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0eceffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0ed015f370 0x7f0ed00d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53772/0 (socket says 172.21.15.53:53772) 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0eceffd640 1 -- 172.21.15.53:0/2354059455 learned_addr learned my addr 172.21.15.53:0/2354059455 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ece7fc640 1 --2- 172.21.15.53:0/2354059455 >> v2:172.21.15.135:3300/0 conn(0x7f0ed01639a0 0x7f0ed0070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ecf7fe640 1 -- 172.21.15.53:0/2354059455 >> v2:172.21.15.135:3300/0 conn(0x7f0ed01639a0 msgr2=0x7f0ed0070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ecf7fe640 1 --2- 172.21.15.53:0/2354059455 >> v2:172.21.15.135:3300/0 conn(0x7f0ed01639a0 0x7f0ed0070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ecf7fe640 1 -- 172.21.15.53:0/2354059455 >> v1:172.21.15.53:6789/0 conn(0x7f0ed015f370 legacy=0x7f0ed00d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ecf7fe640 1 -- 172.21.15.53:0/2354059455 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed00707e0 con 0x7f0ed0071520 2024-05-10T10:43:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.253+0000 7f0ecdffb640 1 -- 172.21.15.53:0/2354059455 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965646026 0 0) 0x7f0ed00d2510 con 0x7f0ed015f370 2024-05-10T10:43:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.254+0000 7f0ecf7fe640 1 --2- 172.21.15.53:0/2354059455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0071520 0x7f0ed0162e00 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00c6910 tx=0x7f0ec802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef18036666d516c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.254+0000 7f0ecdffb640 1 -- 172.21.15.53:0/2354059455 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8007740 con 0x7f0ed0071520 2024-05-10T10:43:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.254+0000 7f0ecdffb640 1 -- 172.21.15.53:0/2354059455 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec80078e0 con 0x7f0ed0071520 2024-05-10T10:43:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ecdffb640 1 -- 172.21.15.53:0/2354059455 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8007cb0 con 0x7f0ed0071520 2024-05-10T10:43:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 -- 172.21.15.53:0/2354059455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0071520 msgr2=0x7f0ed0162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 --2- 172.21.15.53:0/2354059455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0071520 0x7f0ed0162e00 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00c6910 tx=0x7f0ec802a710 comp rx=0 tx=0).stop 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 -- 172.21.15.53:0/2354059455 shutdown_connections 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 --2- 172.21.15.53:0/2354059455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0071520 0x7f0ed0162e00 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 --2- 172.21.15.53:0/2354059455 >> v2:172.21.15.135:3300/0 conn(0x7f0ed01639a0 0x7f0ed0070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 -- 172.21.15.53:0/2354059455 >> 172.21.15.53:0/2354059455 conn(0x7f0ed00d2e10 msgr2=0x7f0ed00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 -- 172.21.15.53:0/2354059455 shutdown_connections 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.255+0000 7f0ed54da640 1 -- 172.21.15.53:0/2354059455 wait complete. 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.256+0000 7f0ed54da640 1 Processor -- start 2024-05-10T10:43:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.256+0000 7f0ed54da640 1 -- start start 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0ed54da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed015f370 0x7f0ed00b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0ed54da640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ed01639a0 0x7f0ed014f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0ed54da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ed009fb10 0x7f0ed01eae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0ed54da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01634c0 con 0x7f0ed015f370 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0ed54da640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0163340 con 0x7f0ed009fb10 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0ed54da640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed0163640 con 0x7f0ed01639a0 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0eceffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed015f370 0x7f0ed00b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0eceffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed015f370 0x7f0ed00b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59704/0 (socket says 172.21.15.53:59704) 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.257+0000 7f0eceffd640 1 -- 172.21.15.53:0/967073487 learned_addr learned my addr 172.21.15.53:0/967073487 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eceffd640 1 -- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ed01639a0 msgr2=0x7f0ed014f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eceffd640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0ed01639a0 0x7f0ed014f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eceffd640 1 -- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ed009fb10 msgr2=0x7f0ed01eae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0ecf7fe640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ed009fb10 0x7f0ed01eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eceffd640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ed009fb10 0x7f0ed01eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eceffd640 1 -- 172.21.15.53:0/967073487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed014f5f0 con 0x7f0ed015f370 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eceffd640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed015f370 0x7f0ed00b1560 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c500 tx=0x7f0ec000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0018070 con 0x7f0ed015f370 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec0007560 con 0x7f0ed015f370 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed009ffa0 con 0x7f0ed015f370 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0004a20 con 0x7f0ed015f370 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.258+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed00a0560 con 0x7f0ed015f370 2024-05-10T10:43:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.259+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ed006a220 con 0x7f0ed015f370 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.260+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0ec001b070 con 0x7f0ed015f370 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.260+0000 7f0eb77fe640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ebc076300 0x7f0ebc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.260+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f0ec0096e00 con 0x7f0ed015f370 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.260+0000 7f0eb77fe640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f0ebc07c8e0 0x7f0ebc07ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.260+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ebc07f3f0 con 0x7f0ebc07c8e0 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.260+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f0ec0097290 con 0x7f0ed015f370 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.260+0000 7f0ece7fc640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ebc076300 0x7f0ebc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.261+0000 7f0ecf7fe640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f0ebc07c8e0 0x7f0ebc07ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.261+0000 7f0ece7fc640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ebc076300 0x7f0ebc0787c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0162160 tx=0x7f0ec40095a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.261+0000 7f0ecf7fe640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f0ebc07c8e0 0x7f0ebc07ed20 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.265 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.262+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== osd.7 v2:172.21.15.135:6809/751247261 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ebc07f3f0 con 0x7f0ebc07c8e0 2024-05-10T10:43:33.275 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.275+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 --> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0ed00b8d90 con 0x7f0ebc07c8e0 2024-05-10T10:43:33.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.275+0000 7f0eb77fe640 1 -- 172.21.15.53:0/967073487 <== osd.7 v2:172.21.15.135:6809/751247261 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0ed00b8d90 con 0x7f0ebc07c8e0 2024-05-10T10:43:33.276 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-10T10:43:33.276 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T10:43:33.276 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-10T10:43:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f0ebc07c8e0 msgr2=0x7f0ebc07ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6809/751247261,v1:172.21.15.135:6811/751247261] conn(0x7f0ebc07c8e0 0x7f0ebc07ed20 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ebc076300 msgr2=0x7f0ebc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0ebc076300 0x7f0ebc0787c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0162160 tx=0x7f0ec40095a0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed015f370 msgr2=0x7f0ed00b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 --2- 172.21.15.53:0/967073487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed015f370 0x7f0ed00b1560 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c500 tx=0x7f0ec000c9c0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ecf7fe640 1 -- 172.21.15.53:0/967073487 reap_dead start 2024-05-10T10:43:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 shutdown_connections 2024-05-10T10:43:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 >> 172.21.15.53:0/967073487 conn(0x7f0ed00d2e10 msgr2=0x7f0ed00c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 shutdown_connections 2024-05-10T10:43:33.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.276+0000 7f0ed54da640 1 -- 172.21.15.53:0/967073487 wait complete. 2024-05-10T10:43:33.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-10T10:43:33.393 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.393+0000 7f3b1d79e640 1 Processor -- start 2024-05-10T10:43:33.394 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.393+0000 7f3b1d79e640 1 -- start start 2024-05-10T10:43:33.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.393+0000 7f3b1d79e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b1815f190 0x7f3b180cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.393+0000 7f3b1d79e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b1815fc80 0x7f3b18070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.397 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.393+0000 7f3b1d79e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b180d12f0 con 0x7f3b180d0e10 2024-05-10T10:43:33.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.393+0000 7f3b1d79e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3b180d1f00 con 0x7f3b1815fc80 2024-05-10T10:43:33.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.393+0000 7f3b1d79e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3b180d15d0 con 0x7f3b1815f190 2024-05-10T10:43:33.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b0ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3b180d0e10 0x7f3b1815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53780/0 (socket says 172.21.15.53:53780) 2024-05-10T10:43:33.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b1cf9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b1815fc80 0x7f3b18070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b0ffff640 1 -- 172.21.15.53:0/3897557615 learned_addr learned my addr 172.21.15.53:0/3897557615 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:33.398 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b17fff640 1 --2- 172.21.15.53:0/3897557615 >> v2:172.21.15.135:3300/0 conn(0x7f3b1815f190 0x7f3b180cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b1cf9d640 1 -- 172.21.15.53:0/3897557615 >> v2:172.21.15.135:3300/0 conn(0x7f3b1815f190 msgr2=0x7f3b180cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:33.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b1cf9d640 1 --2- 172.21.15.53:0/3897557615 >> v2:172.21.15.135:3300/0 conn(0x7f3b1815f190 0x7f3b180cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b1cf9d640 1 -- 172.21.15.53:0/3897557615 >> v1:172.21.15.53:6789/0 conn(0x7f3b180d0e10 legacy=0x7f3b1815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b1cf9d640 1 -- 172.21.15.53:0/3897557615 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b180d18b0 con 0x7f3b1815fc80 2024-05-10T10:43:33.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b177fe640 1 -- 172.21.15.53:0/3897557615 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022464951 0 0) 0x7f3b180d12f0 con 0x7f3b180d0e10 2024-05-10T10:43:33.399 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.394+0000 7f3b17fff640 1 --2- 172.21.15.53:0/3897557615 >> v2:172.21.15.135:3300/0 conn(0x7f3b1815f190 0x7f3b180cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1cf9d640 1 --2- 172.21.15.53:0/3897557615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b18070060 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3b18073200 tx=0x7f3b1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f067b2e08e865f20 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b177fe640 1 -- 172.21.15.53:0/3897557615 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10007740 con 0x7f3b1815fc80 2024-05-10T10:43:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b177fe640 1 -- 172.21.15.53:0/3897557615 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b100078e0 con 0x7f3b1815fc80 2024-05-10T10:43:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b177fe640 1 -- 172.21.15.53:0/3897557615 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10007cb0 con 0x7f3b1815fc80 2024-05-10T10:43:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 -- 172.21.15.53:0/3897557615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 msgr2=0x7f3b18070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/3897557615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b18070060 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3b18073200 tx=0x7f3b1002a680 comp rx=0 tx=0).stop 2024-05-10T10:43:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 -- 172.21.15.53:0/3897557615 shutdown_connections 2024-05-10T10:43:33.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/3897557615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b18070060 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/3897557615 >> v2:172.21.15.135:3300/0 conn(0x7f3b1815f190 0x7f3b180cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 -- 172.21.15.53:0/3897557615 >> 172.21.15.53:0/3897557615 conn(0x7f3b180d2ca0 msgr2=0x7f3b180d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 -- 172.21.15.53:0/3897557615 shutdown_connections 2024-05-10T10:43:33.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.395+0000 7f3b1d79e640 1 -- 172.21.15.53:0/3897557615 wait complete. 2024-05-10T10:43:33.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.397+0000 7f3b1d79e640 1 Processor -- start 2024-05-10T10:43:33.401 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.397+0000 7f3b1d79e640 1 -- start start 2024-05-10T10:43:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b180d0e10 0x7f3b181e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b1815f190 0x7f3b18157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b180c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b180d05f0 con 0x7f3b1815fc80 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b180d0470 con 0x7f3b1815f190 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180705a0 con 0x7f3b180d0e10 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b180c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b180c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59722/0 (socket says 172.21.15.53:59722) 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 -- 172.21.15.53:0/882481987 learned_addr learned my addr 172.21.15.53:0/882481987 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b0ffff640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b1815f190 0x7f3b18157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b17fff640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b180d0e10 0x7f3b181e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 -- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b180d0e10 msgr2=0x7f3b181e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b180d0e10 0x7f3b181e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 -- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b1815f190 msgr2=0x7f3b18157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b1815f190 0x7f3b18157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 -- 172.21.15.53:0/882481987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b181e8030 con 0x7f3b1815fc80 2024-05-10T10:43:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1cf9d640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b180c9d70 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3b1002ac70 tx=0x7f3b10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b157fa640 1 -- 172.21.15.53:0/882481987 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10040070 con 0x7f3b1815fc80 2024-05-10T10:43:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b157fa640 1 -- 172.21.15.53:0/882481987 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b10031c90 con 0x7f3b1815fc80 2024-05-10T10:43:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b180b68e0 con 0x7f3b1815fc80 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b157fa640 1 -- 172.21.15.53:0/882481987 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10045680 con 0x7f3b1815fc80 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.398+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b181e7010 con 0x7f3b1815fc80 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.399+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b180693f0 con 0x7f3b1815fc80 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.400+0000 7f3b157fa640 1 -- 172.21.15.53:0/882481987 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b10039040 con 0x7f3b1815fc80 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.401+0000 7f3b157fa640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af8076300 0x7f3af80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.401+0000 7f3b157fa640 1 -- 172.21.15.53:0/882481987 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f3b100b8eb0 con 0x7f3b1815fc80 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.401+0000 7f3b17fff640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af8076300 0x7f3af80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.401+0000 7f3b17fff640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af8076300 0x7f3af80787c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3b08008850 tx=0x7f3b08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.405+0000 7f3b157fa640 1 -- 172.21.15.53:0/882481987 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f3b10082990 con 0x7f3b1815fc80 2024-05-10T10:43:33.595 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.595+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f3b1806cfb0 con 0x7f3af8076300 2024-05-10T10:43:33.596 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.595+0000 7f3b157fa640 1 -- 172.21.15.53:0/882481987 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3b1806cfb0 con 0x7f3af8076300 2024-05-10T10:43:33.596 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 7 to deep-scrub 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af8076300 msgr2=0x7f3af80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af8076300 0x7f3af80787c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3b08008850 tx=0x7f3b08008040 comp rx=0 tx=0).stop 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 msgr2=0x7f3b180c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b180c9d70 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3b1002ac70 tx=0x7f3b10004a40 comp rx=0 tx=0).stop 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 shutdown_connections 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f3af8076300 0x7f3af80787c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1815fc80 0x7f3b180c9d70 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b1815f190 0x7f3b18157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 --2- 172.21.15.53:0/882481987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b180d0e10 0x7f3b181e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 >> 172.21.15.53:0/882481987 conn(0x7f3b180d2ca0 msgr2=0x7f3b180d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 shutdown_connections 2024-05-10T10:43:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.597+0000 7f3b1d79e640 1 -- 172.21.15.53:0/882481987 wait complete. 2024-05-10T10:43:33.611 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:43:33.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.714+0000 7f0f35914640 1 Processor -- start 2024-05-10T10:43:33.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.714+0000 7f0f35914640 1 -- start start 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f35914640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5520 0x7f0f300d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f35914640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0f300b9180 0x7f0f300d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f35914640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f300d41f0 con 0x7f0f300b5900 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f35914640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0f300d5c60 con 0x7f0f300b9180 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f35914640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0f300d5eb0 con 0x7f0f300b5520 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f2f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0f300b9180 0x7f0f300d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f2f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0f300b9180 0x7f0f300d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59734/0 (socket says 172.21.15.53:59734) 2024-05-10T10:43:33.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.715+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/3171016699 learned_addr learned my addr 172.21.15.53:0/3171016699 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.716+0000 7f0f2effd640 1 --2- 172.21.15.53:0/3171016699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5520 0x7f0f300d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.716+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/3171016699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5520 msgr2=0x7f0f300d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.716+0000 7f0f2f7fe640 1 --2- 172.21.15.53:0/3171016699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5520 0x7f0f300d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.716+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/3171016699 >> v1:172.21.15.135:6789/0 conn(0x7f0f300b5900 legacy=0x7f0f300d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.716+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/3171016699 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f300d0a30 con 0x7f0f300b9180 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.716+0000 7f0f2f7fe640 1 --2- 172.21.15.53:0/3171016699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300d1de0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb22f7eadd23673 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f2dffb640 1 -- 172.21.15.53:0/3171016699 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037070 con 0x7f0f300b9180 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f2dffb640 1 -- 172.21.15.53:0/3171016699 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f20032440 con 0x7f0f300b9180 2024-05-10T10:43:33.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f2dffb640 1 -- 172.21.15.53:0/3171016699 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20032810 con 0x7f0f300b9180 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 -- 172.21.15.53:0/3171016699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 msgr2=0x7f0f300d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 --2- 172.21.15.53:0/3171016699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300d1de0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a270 comp rx=0 tx=0).stop 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 -- 172.21.15.53:0/3171016699 shutdown_connections 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 --2- 172.21.15.53:0/3171016699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300d1de0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 --2- 172.21.15.53:0/3171016699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5520 0x7f0f300d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 -- 172.21.15.53:0/3171016699 >> 172.21.15.53:0/3171016699 conn(0x7f0f300d5170 msgr2=0x7f0f300d5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 -- 172.21.15.53:0/3171016699 shutdown_connections 2024-05-10T10:43:33.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.717+0000 7f0f35914640 1 -- 172.21.15.53:0/3171016699 wait complete. 2024-05-10T10:43:33.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.718+0000 7f0f35914640 1 Processor -- start 2024-05-10T10:43:33.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f35914640 1 -- start start 2024-05-10T10:43:33.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f35914640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f300b5520 0x7f0f3014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f35914640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5900 0x7f0f300b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f35914640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f35914640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300b6460 con 0x7f0f300b9180 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f35914640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300b62e0 con 0x7f0f300b5520 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f35914640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f0f300b65e0 con 0x7f0f300b5900 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f2f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f2f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59740/0 (socket says 172.21.15.53:59740) 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.719+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/1122226819 learned_addr learned my addr 172.21.15.53:0/1122226819 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2effd640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f300b5520 0x7f0f3014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2e7fc640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5900 0x7f0f300b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5900 msgr2=0x7f0f300b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2f7fe640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5900 0x7f0f300b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f300b5520 msgr2=0x7f0f3014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2f7fe640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f300b5520 0x7f0f3014ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2f7fe640 1 -- 172.21.15.53:0/1122226819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f300a05e0 con 0x7f0f300b9180 2024-05-10T10:43:33.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f2f7fe640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300b3330 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0f2002a2b0 tx=0x7f0f200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.725 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f27fff640 1 -- 172.21.15.53:0/1122226819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20041070 con 0x7f0f300b9180 2024-05-10T10:43:33.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f27fff640 1 -- 172.21.15.53:0/1122226819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2003c870 con 0x7f0f300b9180 2024-05-10T10:43:33.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f300cf270 con 0x7f0f300b9180 2024-05-10T10:43:33.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f27fff640 1 -- 172.21.15.53:0/1122226819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037040 con 0x7f0f300b9180 2024-05-10T10:43:33.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.720+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3014f940 con 0x7f0f300b9180 2024-05-10T10:43:33.726 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.721+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f300693f0 con 0x7f0f300b9180 2024-05-10T10:43:33.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.722+0000 7f0f27fff640 1 -- 172.21.15.53:0/1122226819 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0f20044070 con 0x7f0f300b9180 2024-05-10T10:43:33.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.722+0000 7f0f27fff640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f100762d0 0x7f0f10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:33.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.722+0000 7f0f27fff640 1 -- 172.21.15.53:0/1122226819 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f0f200b7ef0 con 0x7f0f300b9180 2024-05-10T10:43:33.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.722+0000 7f0f2effd640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f100762d0 0x7f0f10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:33.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.723+0000 7f0f2effd640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f100762d0 0x7f0f10078790 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009a60 tx=0x7f0f28003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:33.728 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.726+0000 7f0f27fff640 1 -- 172.21.15.53:0/1122226819 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f0f300a05e0 con 0x7f0f300b9180 2024-05-10T10:43:33.913 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.912+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f3006ce20 con 0x7f0f100762d0 2024-05-10T10:43:33.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.913+0000 7f0f27fff640 1 -- 172.21.15.53:0/1122226819 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13502 (secure 0 0 0) 0x7f0f3006ce20 con 0x7f0f100762d0 2024-05-10T10:43:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:33.915 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:43:33.916 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f100762d0 msgr2=0x7f0f10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f100762d0 0x7f0f10078790 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009a60 tx=0x7f0f28003990 comp rx=0 tx=0).stop 2024-05-10T10:43:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 msgr2=0x7f0f300b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300b3330 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0f2002a2b0 tx=0x7f0f200046c0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 shutdown_connections 2024-05-10T10:43:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f0f100762d0 0x7f0f10078790 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f300b9180 0x7f0f300b3330 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f300b5900 0x7f0f300b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 --2- 172.21.15.53:0/1122226819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f300b5520 0x7f0f3014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:33.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.915+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 >> 172.21.15.53:0/1122226819 conn(0x7f0f300d5170 msgr2=0x7f0f300b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:33.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.916+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 shutdown_connections 2024-05-10T10:43:33.918 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:33.916+0000 7f0f35914640 1 -- 172.21.15.53:0/1122226819 wait complete. 2024-05-10T10:43:33.927 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-05-10T10:43:32.305771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266820,"kb_used_data":13192,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707900,"statfs":{"total":773094113280,"available":772820889600,"internally_reserved":0,"allocated":13508608,"data_stored":6918024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392252,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-178,"num_write_kb":-8305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002048"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":320,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-10T10:43:25.663135+0000","last_change":"2024-05-10T10:33:29.463508+0000","last_active":"2024-05-10T10:43:25.663135+0000","last_peered":"2024-05-10T10:43:25.663135+0000","last_clean":"2024-05-10T10:43:25.663135+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:43:25.663135+0000","last_undegraded":"2024-05-10T10:43:25.663135+0000","last_fullsized":"2024-05-10T10:43:25.663135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T10:33:28.444294+0000","last_clean_scrub_stamp":"2024-05-10T10:33:28.444294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:12.043983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2134016,"data_stored":1307529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37432,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334408,"statfs":{"total":96636764160,"available":96598433792,"internally_reserved":0,"allocated":2134016,"data_stored":1307529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:43:33.932 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-10T10:33:28.444294+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=10, tm_hour=10, tm_min=33, tm_sec=28, tm_wday=4, tm_yday=131, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=10, tm_hour=10, tm_min=43, tm_sec=28, tm_wday=4, tm_yday=131, tm_isdst=0) 2024-05-10T10:43:33.932 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-10T10:43:46.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:46.299 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:53.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T10:43:54.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.049+0000 7fb3dd209640 1 Processor -- start 2024-05-10T10:43:54.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.049+0000 7fb3dd209640 1 -- start start 2024-05-10T10:43:54.051 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3dd209640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d815f620 0x7fb3d815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3dd209640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb3d80704c0 0x7fb3d8070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3dd209640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d8160190 con 0x7fb3d806c990 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3dd209640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d806cd70 con 0x7fb3d80704c0 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3dd209640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d8160470 con 0x7fb3d815f620 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3d759e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb3d80704c0 0x7fb3d8070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3ce59c640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d815f620 0x7fb3d815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3d6d9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb3d806c990 0x7fb3d8163660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36676/0 (socket says 172.21.15.53:36676) 2024-05-10T10:43:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.050+0000 7fb3d6d9d640 1 -- 172.21.15.53:0/3711571826 learned_addr learned my addr 172.21.15.53:0/3711571826 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-05-10T10:43:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.051+0000 7fb3ce59c640 1 -- 172.21.15.53:0/3711571826 >> v2:172.21.15.135:3300/0 conn(0x7fb3d80704c0 msgr2=0x7fb3d8070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.051+0000 7fb3ce59c640 1 --2- 172.21.15.53:0/3711571826 >> v2:172.21.15.135:3300/0 conn(0x7fb3d80704c0 0x7fb3d8070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.051+0000 7fb3ce59c640 1 -- 172.21.15.53:0/3711571826 >> v1:172.21.15.135:6789/0 conn(0x7fb3d806c990 legacy=0x7fb3d8163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.051+0000 7fb3ce59c640 1 -- 172.21.15.53:0/3711571826 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d8162e60 con 0x7fb3d815f620 2024-05-10T10:43:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.051+0000 7fb3d759e640 1 --2- 172.21.15.53:0/3711571826 >> v2:172.21.15.135:3300/0 conn(0x7fb3d80704c0 0x7fb3d8070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.051+0000 7fb3ce59c640 1 --2- 172.21.15.53:0/3711571826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d815f620 0x7fb3d815fa80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00097a0 tx=0x7fb3c002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff3668769d3ae27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3d659c640 1 -- 172.21.15.53:0/3711571826 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0037070 con 0x7fb3d815f620 2024-05-10T10:43:54.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3d659c640 1 -- 172.21.15.53:0/3711571826 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0032440 con 0x7fb3d815f620 2024-05-10T10:43:54.054 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3d659c640 1 -- 172.21.15.53:0/3711571826 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0032810 con 0x7fb3d815f620 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 -- 172.21.15.53:0/3711571826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d815f620 msgr2=0x7fb3d815fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 --2- 172.21.15.53:0/3711571826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d815f620 0x7fb3d815fa80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00097a0 tx=0x7fb3c002a180 comp rx=0 tx=0).stop 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 -- 172.21.15.53:0/3711571826 shutdown_connections 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 --2- 172.21.15.53:0/3711571826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d815f620 0x7fb3d815fa80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 --2- 172.21.15.53:0/3711571826 >> v2:172.21.15.135:3300/0 conn(0x7fb3d80704c0 0x7fb3d8070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 -- 172.21.15.53:0/3711571826 >> 172.21.15.53:0/3711571826 conn(0x7fb3d80b3980 msgr2=0x7fb3d80b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 -- 172.21.15.53:0/3711571826 shutdown_connections 2024-05-10T10:43:54.055 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.052+0000 7fb3dd209640 1 -- 172.21.15.53:0/3711571826 wait complete. 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.054+0000 7fb3dd209640 1 Processor -- start 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.054+0000 7fb3dd209640 1 -- start start 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3dd209640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d806c990 0x7fb3d815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3dd209640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d80704c0 0x7fb3d81f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3dd209640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d815f620 0x7fb3d815e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3dd209640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8073e70 con 0x7fb3d815f620 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3dd209640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8070e90 con 0x7fb3d80704c0 2024-05-10T10:43:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3dd209640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d80cf920 con 0x7fb3d806c990 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d815f620 0x7fb3d815e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d815f620 0x7fb3d815e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39114/0 (socket says 172.21.15.53:39114) 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 -- 172.21.15.53:0/2459943705 learned_addr learned my addr 172.21.15.53:0/2459943705 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3ce59c640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d80704c0 0x7fb3d81f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d6d9d640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d806c990 0x7fb3d815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 -- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d806c990 msgr2=0x7fb3d815c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d806c990 0x7fb3d815c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 -- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d80704c0 msgr2=0x7fb3d81f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d80704c0 0x7fb3d81f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 -- 172.21.15.53:0/2459943705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d81f89d0 con 0x7fb3d815f620 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3ce59c640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d80704c0 0x7fb3d81f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3d759e640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d815f620 0x7fb3d815e8e0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000efc0 tx=0x7fb3d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3cffff640 1 -- 172.21.15.53:0/2459943705 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0018030 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.055+0000 7fb3cffff640 1 -- 172.21.15.53:0/2459943705 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d000cde0 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.056+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d815ee20 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.056+0000 7fb3cffff640 1 -- 172.21.15.53:0/2459943705 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0003bb0 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.056+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81f8c70 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.057+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d80b7a70 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.057+0000 7fb3cffff640 1 -- 172.21.15.53:0/2459943705 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb3d0005d10 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.058+0000 7fb3cffff640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3b80762d0 0x7fb3b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.058+0000 7fb3cffff640 1 -- 172.21.15.53:0/2459943705 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7fb3d0098020 con 0x7fb3d815f620 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.058+0000 7fb3d6d9d640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3b80762d0 0x7fb3b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.059+0000 7fb3d6d9d640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3b80762d0 0x7fb3b8078790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8000f60 tx=0x7fb3c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.062+0000 7fb3cffff640 1 -- 172.21.15.53:0/2459943705 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7fb3d80b7a70 con 0x7fb3d815f620 2024-05-10T10:43:54.252 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.251+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 --> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3d806a220 con 0x7fb3b80762d0 2024-05-10T10:43:54.253 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.252+0000 7fb3cffff640 1 -- 172.21.15.53:0/2459943705 <== mgr.4103 v2:172.21.15.135:6832/1385963166 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7fb3d806a220 con 0x7fb3b80762d0 2024-05-10T10:43:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:54.254 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3b80762d0 msgr2=0x7fb3b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3b80762d0 0x7fb3b8078790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8000f60 tx=0x7fb3c8009040 comp rx=0 tx=0).stop 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d815f620 msgr2=0x7fb3d815e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d815f620 0x7fb3d815e8e0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000efc0 tx=0x7fb3d0009f90 comp rx=0 tx=0).stop 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 shutdown_connections 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7fb3b80762d0 0x7fb3b8078790 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d815f620 0x7fb3d815e8e0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d80704c0 0x7fb3d81f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 --2- 172.21.15.53:0/2459943705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3d806c990 0x7fb3d815c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 >> 172.21.15.53:0/2459943705 conn(0x7fb3d80b3980 msgr2=0x7fb3d80b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 shutdown_connections 2024-05-10T10:43:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.254+0000 7fb3dd209640 1 -- 172.21.15.53:0/2459943705 wait complete. 2024-05-10T10:43:54.267 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-05-10T10:43:52.309433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266800,"kb_used_data":13192,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707920,"statfs":{"total":773094113280,"available":772820910080,"internally_reserved":0,"allocated":13508608,"data_stored":6918024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002085"},"pg_stats":[{"pgid":"1.0","version":"123'74","reported_seq":332,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-10T10:43:34.149762+0000","last_change":"2024-05-10T10:43:34.149820+0000","last_active":"2024-05-10T10:43:34.149762+0000","last_peered":"2024-05-10T10:43:34.149762+0000","last_clean":"2024-05-10T10:43:34.149762+0000","last_became_active":"2024-05-10T10:33:29.463232+0000","last_became_peered":"2024-05-10T10:33:29.463232+0000","last_unstale":"2024-05-10T10:43:34.149762+0000","last_undegraded":"2024-05-10T10:43:34.149762+0000","last_fullsized":"2024-05-10T10:43:34.149762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"123'74","last_scrub_stamp":"2024-05-10T10:43:34.149643+0000","last_deep_scrub":"123'74","last_deep_scrub_stamp":"2024-05-10T10:43:34.149643+0000","last_clean_scrub_stamp":"2024-05-10T10:43:34.149643+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:42:08.369328+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2134016,"data_stored":1307529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":717161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2134016,"data_stored":1307529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T10:43:54.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-10T10:43:54.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.369+0000 7f60c1620640 1 Processor -- start 2024-05-10T10:43:54.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60c1620640 1 -- start start 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60c1620640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60bc15f410 0x7f60bc0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60c1620640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f60bc0d0f10 0x7f60bc0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60c1620640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc0b54d0 0x7f60bc0d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60c1620640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc0d2800 con 0x7f60bc15f410 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60c1620640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc0d2ae0 con 0x7f60bc0d0f10 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60c1620640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0d01d0 con 0x7f60bc0b54d0 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.370+0000 7f60baffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60bc15f410 0x7f60bc0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60bc15f410 0x7f60bc0d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39126/0 (socket says 172.21.15.53:39126) 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 -- 172.21.15.53:0/1208822497 learned_addr learned my addr 172.21.15.53:0/1208822497 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 -- 172.21.15.53:0/1208822497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc0b54d0 msgr2=0x7f60bc0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 --2- 172.21.15.53:0/1208822497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc0b54d0 0x7f60bc0d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60ba7fc640 1 --2- 172.21.15.53:0/1208822497 >> v2:172.21.15.135:3300/0 conn(0x7f60bc0d0f10 0x7f60bc0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 -- 172.21.15.53:0/1208822497 >> v2:172.21.15.135:3300/0 conn(0x7f60bc0d0f10 msgr2=0x7f60bc0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 --2- 172.21.15.53:0/1208822497 >> v2:172.21.15.135:3300/0 conn(0x7f60bc0d0f10 0x7f60bc0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 -- 172.21.15.53:0/1208822497 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc0d04b0 con 0x7f60bc15f410 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60ba7fc640 1 --2- 172.21.15.53:0/1208822497 >> v2:172.21.15.135:3300/0 conn(0x7f60bc0d0f10 0x7f60bc0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.371+0000 7f60baffd640 1 --2- 172.21.15.53:0/1208822497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc15f410 0x7f60bc0d09d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0b7ee0 tx=0x7f60b402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b506bef3c7acd569 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60b9ffb640 1 -- 172.21.15.53:0/1208822497 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4036070 con 0x7f60bc15f410 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60b9ffb640 1 -- 172.21.15.53:0/1208822497 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b4031600 con 0x7f60bc15f410 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60b9ffb640 1 -- 172.21.15.53:0/1208822497 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b40319d0 con 0x7f60bc15f410 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 -- 172.21.15.53:0/1208822497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc15f410 msgr2=0x7f60bc0d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 --2- 172.21.15.53:0/1208822497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc15f410 0x7f60bc0d09d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0b7ee0 tx=0x7f60b402a610 comp rx=0 tx=0).stop 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 -- 172.21.15.53:0/1208822497 shutdown_connections 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 --2- 172.21.15.53:0/1208822497 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc0b54d0 0x7f60bc0d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 --2- 172.21.15.53:0/1208822497 >> v2:172.21.15.135:3300/0 conn(0x7f60bc0d0f10 0x7f60bc0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 --2- 172.21.15.53:0/1208822497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc15f410 0x7f60bc0d09d0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 -- 172.21.15.53:0/1208822497 >> 172.21.15.53:0/1208822497 conn(0x7f60bc08ed30 msgr2=0x7f60bc09e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:54.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 -- 172.21.15.53:0/1208822497 shutdown_connections 2024-05-10T10:43:54.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.372+0000 7f60c1620640 1 -- 172.21.15.53:0/1208822497 wait complete. 2024-05-10T10:43:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 Processor -- start 2024-05-10T10:43:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 -- start start 2024-05-10T10:43:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60bc0b54d0 0x7f60bc15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc0d0f10 0x7f60bc161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc15f410 0x7f60bc162110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0cffb0 con 0x7f60bc0d0f10 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0cfe30 con 0x7f60bc0b54d0 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.374+0000 7f60c1620640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0d3c10 con 0x7f60bc15f410 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc0d0f10 0x7f60bc161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc0d0f10 0x7f60bc161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39130/0 (socket says 172.21.15.53:39130) 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 -- 172.21.15.53:0/1741778774 learned_addr learned my addr 172.21.15.53:0/1741778774 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60baffd640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60bc0b54d0 0x7f60bc15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60bb7fe640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc15f410 0x7f60bc162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 -- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc15f410 msgr2=0x7f60bc162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc15f410 0x7f60bc162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 -- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60bc0b54d0 msgr2=0x7f60bc15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60bc0b54d0 0x7f60bc15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 -- 172.21.15.53:0/1741778774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc0c9d20 con 0x7f60bc0d0f10 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60baffd640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60bc0b54d0 0x7f60bc15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60bb7fe640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc15f410 0x7f60bc162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60ba7fc640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc0d0f10 0x7f60bc161bd0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f60a400c820 tx=0x7f60a400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4015070 con 0x7f60bc0d0f10 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a4007a30 con 0x7f60bc0d0f10 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4007e00 con 0x7f60bc0d0f10 2024-05-10T10:43:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc0ca000 con 0x7f60bc0d0f10 2024-05-10T10:43:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.375+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc0ca5c0 con 0x7f60bc0d0f10 2024-05-10T10:43:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.376+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc06a220 con 0x7f60bc0d0f10 2024-05-10T10:43:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.377+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f60a4018070 con 0x7f60bc0d0f10 2024-05-10T10:43:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.377+0000 7f60b37fe640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f609c0762d0 0x7f609c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.377+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f60a40974c0 con 0x7f60bc0d0f10 2024-05-10T10:43:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.378+0000 7f60baffd640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f609c0762d0 0x7f609c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.378+0000 7f60baffd640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f609c0762d0 0x7f609c078790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f60b4033000 tx=0x7f60b4009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.382+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f60a4060ed0 con 0x7f60bc0d0f10 2024-05-10T10:43:54.573 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.572+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f60bc0d6090 con 0x7f60bc0d0f10 2024-05-10T10:43:54.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.575+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f60a40607d0 con 0x7f60bc0d0f10 2024-05-10T10:43:54.577 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.576+0000 7f60b37fe640 1 -- 172.21.15.53:0/1741778774 <== mon.0 v2:172.21.15.53:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f60a40602e0 con 0x7f60bc0d0f10 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f609c0762d0 msgr2=0x7f609c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f609c0762d0 0x7f609c078790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f60b4033000 tx=0x7f60b4009000 comp rx=0 tx=0).stop 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc0d0f10 msgr2=0x7f60bc161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc0d0f10 0x7f60bc161bd0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f60a400c820 tx=0x7f60a400cc20 comp rx=0 tx=0).stop 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 shutdown_connections 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f609c0762d0 0x7f609c078790 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f60bc15f410 0x7f60bc162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60bc0d0f10 0x7f60bc161bd0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 --2- 172.21.15.53:0/1741778774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60bc0b54d0 0x7f60bc15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 >> 172.21.15.53:0/1741778774 conn(0x7f60bc08ed30 msgr2=0x7f60bc0d4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:54.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 shutdown_connections 2024-05-10T10:43:54.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.578+0000 7f60c1620640 1 -- 172.21.15.53:0/1741778774 wait complete. 2024-05-10T10:43:54.590 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T10:43:54.591 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T10:43:54.694 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.693+0000 7f8053072640 1 Processor -- start 2024-05-10T10:43:54.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.693+0000 7f8053072640 1 -- start start 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f8053072640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f804c15b6d0 0x7f804c0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f8053072640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c15f830 0x7f804c06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f8053072640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804c0c5d10 con 0x7f804c0cf170 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f8053072640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f804c0cc380 con 0x7f804c15b6d0 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f8053072640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f804c0c5ff0 con 0x7f804c15f830 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f805160f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c15f830 0x7f804c06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f8050e0e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f804c15b6d0 0x7f804c0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f805160f640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c15f830 0x7f804c06dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.53:57374/0 (socket says 172.21.15.53:57374) 2024-05-10T10:43:54.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f804bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f804c0cf170 0x7f804c15f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.53:36680/0 (socket says 172.21.15.53:36680) 2024-05-10T10:43:54.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.694+0000 7f805160f640 1 -- 172.21.15.53:0/60900968 learned_addr learned my addr 172.21.15.53:0/60900968 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:54.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.695+0000 7f8050e0e640 1 -- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c15f830 msgr2=0x7f804c06dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T10:43:54.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.695+0000 7f8050e0e640 1 --2- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c15f830 0x7f804c06dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.695+0000 7f8050e0e640 1 -- 172.21.15.53:0/60900968 >> v1:172.21.15.135:6789/0 conn(0x7f804c0cf170 legacy=0x7f804c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.695+0000 7f8050e0e640 1 -- 172.21.15.53:0/60900968 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c0c62d0 con 0x7f804c15b6d0 2024-05-10T10:43:54.697 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.695+0000 7f805160f640 1 --2- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c15f830 0x7f804c06dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T10:43:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.695+0000 7f8050e0e640 1 --2- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 0x7f804c0c6db0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f804000a940 tx=0x7f804002b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9c2564b845eb8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.695+0000 7f804b7fe640 1 -- 172.21.15.53:0/60900968 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804000b040 con 0x7f804c15b6d0 2024-05-10T10:43:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f804b7fe640 1 -- 172.21.15.53:0/60900968 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8040030510 con 0x7f804c15b6d0 2024-05-10T10:43:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f804b7fe640 1 -- 172.21.15.53:0/60900968 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040030900 con 0x7f804c15b6d0 2024-05-10T10:43:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 -- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 msgr2=0x7f804c0c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 --2- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 0x7f804c0c6db0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f804000a940 tx=0x7f804002b5e0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 -- 172.21.15.53:0/60900968 shutdown_connections 2024-05-10T10:43:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 --2- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c15f830 0x7f804c06dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 --2- 172.21.15.53:0/60900968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 0x7f804c0c6db0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 -- 172.21.15.53:0/60900968 >> 172.21.15.53:0/60900968 conn(0x7f804c0b2900 msgr2=0x7f804c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 -- 172.21.15.53:0/60900968 shutdown_connections 2024-05-10T10:43:54.699 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.696+0000 7f8053072640 1 -- 172.21.15.53:0/60900968 wait complete. 2024-05-10T10:43:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.697+0000 7f8053072640 1 Processor -- start 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.697+0000 7f8053072640 1 -- start start 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f8053072640 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c0cf170 0x7f804c151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f8053072640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 0x7f804c09dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f8053072640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f804c15f830 0x7f804c09e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f8053072640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0cd170 con 0x7f804c15f830 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f8053072640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0c6890 con 0x7f804c15b6d0 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f8053072640 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 -- 0x7f804c0cd2f0 con 0x7f804c0cf170 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f805160f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f804c15f830 0x7f804c09e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f805160f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f804c15f830 0x7f804c09e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39138/0 (socket says 172.21.15.53:39138) 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.698+0000 7f805160f640 1 -- 172.21.15.53:0/3703894382 learned_addr learned my addr 172.21.15.53:0/3703894382 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f8050e0e640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c0cf170 0x7f804c151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f804bfff640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 0x7f804c09dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f805160f640 1 -- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c0cf170 msgr2=0x7f804c151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f805160f640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c0cf170 0x7f804c151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f805160f640 1 -- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 msgr2=0x7f804c09dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f805160f640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 0x7f804c09dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f805160f640 1 -- 172.21.15.53:0/3703894382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c09ce60 con 0x7f804c15f830 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f805160f640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f804c15f830 0x7f804c09e400 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8044009e70 tx=0x7f804400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f80497fa640 1 -- 172.21.15.53:0/3703894382 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804400cbe0 con 0x7f804c15f830 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f80497fa640 1 -- 172.21.15.53:0/3703894382 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f804400cd80 con 0x7f804c15f830 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c09bdc0 con 0x7f804c15f830 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f80497fa640 1 -- 172.21.15.53:0/3703894382 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044003bd0 con 0x7f804c15f830 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.699+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c09c380 con 0x7f804c15f830 2024-05-10T10:43:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.700+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c0b05c0 con 0x7f804c15f830 2024-05-10T10:43:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.701+0000 7f80497fa640 1 -- 172.21.15.53:0/3703894382 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f804401b030 con 0x7f804c15f830 2024-05-10T10:43:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.701+0000 7f80497fa640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80300762d0 0x7f8030078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T10:43:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.701+0000 7f80497fa640 1 -- 172.21.15.53:0/3703894382 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5507+0+0 (secure 0 0 0) 0x7f80440978b0 con 0x7f804c15f830 2024-05-10T10:43:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.701+0000 7f8050e0e640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80300762d0 0x7f8030078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T10:43:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.702+0000 7f8050e0e640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80300762d0 0x7f8030078790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8040035000 tx=0x7f804000c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T10:43:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.705+0000 7f80497fa640 1 -- 172.21.15.53:0/3703894382 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190715 (secure 0 0 0) 0x7f804c0b05c0 con 0x7f804c15f830 2024-05-10T10:43:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.936+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f804c0c7b40 con 0x7f804c15f830 2024-05-10T10:43:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.937+0000 7f80497fa640 1 -- 172.21.15.53:0/3703894382 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+3866 (secure 0 0 0) 0x7f8044022020 con 0x7f804c15f830 2024-05-10T10:43:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-10T10:43:54.939 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":113,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/991761934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":991761934},{"type":"v1","addr":"172.21.15.135:6836","nonce":991761934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/595654521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":595654521},{"type":"v1","addr":"172.21.15.53:6835","nonce":595654521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/789478261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":789478261},{"type":"v1","addr":"172.21.15.135:6837","nonce":789478261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/994883705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":994883705},{"type":"v1","addr":"172.21.15.53:6834","nonce":994883705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113}],"filesystems":[]} 2024-05-10T10:43:54.939 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 113 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80300762d0 msgr2=0x7f8030078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80300762d0 0x7f8030078790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8040035000 tx=0x7f804000c040 comp rx=0 tx=0).stop 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f804c15f830 msgr2=0x7f804c09e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f804c15f830 0x7f804c09e400 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8044009e70 tx=0x7f804400c490 comp rx=0 tx=0).stop 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 shutdown_connections 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:6832/1385963166,v1:172.21.15.135:6833/1385963166] conn(0x7f80300762d0 0x7f8030078790 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f804c15f830 0x7f804c09e400 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f804c15b6d0 0x7f804c09dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 --2- 172.21.15.53:0/3703894382 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f804c0cf170 0x7f804c151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 >> 172.21.15.53:0/3703894382 conn(0x7f804c0b2900 msgr2=0x7f804c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T10:43:54.941 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 shutdown_connections 2024-05-10T10:43:54.942 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-10T10:43:54.939+0000 7f8053072640 1 -- 172.21.15.53:0/3703894382 wait complete. 2024-05-10T10:43:54.952 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-10T10:43:54.952 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-10T10:43:54.952 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T10:43:54.952 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:54.962 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T10:43:54.962 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T10:43:54.962 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.011 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T10:43:55.011 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T10:43:55.012 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.023 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T10:43:55.023 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T10:43:55.023 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.075 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T10:43:55.075 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-10T10:43:55.075 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-10T10:43:55.075 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.104 INFO:tasks.ceph.osd.0:Stopped 2024-05-10T10:43:55.104 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-10T10:43:55.104 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.171 INFO:tasks.ceph.osd.1:Stopped 2024-05-10T10:43:55.171 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-10T10:43:55.171 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.238 INFO:tasks.ceph.osd.2:Stopped 2024-05-10T10:43:55.238 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-10T10:43:55.238 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.303 INFO:tasks.ceph.osd.3:Stopped 2024-05-10T10:43:55.304 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-10T10:43:55.304 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.344 INFO:tasks.ceph.osd.4:Stopped 2024-05-10T10:43:55.344 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-10T10:43:55.344 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.426 INFO:tasks.ceph.osd.5:Stopped 2024-05-10T10:43:55.426 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-10T10:43:55.426 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.472 INFO:tasks.ceph.osd.6:Stopped 2024-05-10T10:43:55.472 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-10T10:43:55.472 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.511 INFO:tasks.ceph.osd.7:Stopped 2024-05-10T10:43:55.512 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-10T10:43:55.512 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-10T10:43:55.512 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.537 INFO:tasks.ceph.mgr.y:Stopped 2024-05-10T10:43:55.537 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-10T10:43:55.537 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.582 INFO:tasks.ceph.mgr.x:Stopped 2024-05-10T10:43:55.582 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-10T10:43:55.583 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-10T10:43:55.583 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.594 INFO:tasks.ceph.mon.a:Stopped 2024-05-10T10:43:55.595 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-10T10:43:55.595 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.635 INFO:tasks.ceph.mon.b:Stopped 2024-05-10T10:43:55.636 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-10T10:43:55.636 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T10:43:55.689 INFO:tasks.ceph.mon.c:Stopped 2024-05-10T10:43:55.689 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-10T10:43:55.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-10T10:43:55.724 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:37:37.905558+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-05-10T10:43:55.724 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-05-10T10:43:55.724 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-10T10:43:55.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-10T10:43:55.874 INFO:teuthology.orchestra.run.smithi053.stdout:2024-05-10T10:39:18.036714+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-10T10:43:55.875 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:55.875 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-10T10:43:56.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:56.074 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-10T10:43:56.158 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:56.158 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-10T10:43:56.223 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:43:56.223 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-10T10:43:56.281 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:43:56.281 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-10T10:43:56.451 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:43:56.451 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-10T10:43:56.540 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:43:56.540 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-10T10:43:56.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:43:56.599 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-10T10:43:56.656 INFO:tasks.ceph:Archiving mon data... 2024-05-10T10:43:56.657 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/data/mon.a.tgz 2024-05-10T10:43:56.657 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-05-10T10:43:56.674 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.kZw7E3Ph3d 2024-05-10T10:43:56.674 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.kZw7E3Ph3d 2024-05-10T10:43:56.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.kZw7E3Ph3d 2024-05-10T10:43:56.921 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.kZw7E3Ph3d is 449KB 2024-05-10T10:43:57.022 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.kZw7E3Ph3d 2024-05-10T10:43:57.037 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/data/mon.b.tgz 2024-05-10T10:43:57.038 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-05-10T10:43:57.056 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.5HtMktgc5H 2024-05-10T10:43:57.056 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.5HtMktgc5H 2024-05-10T10:43:57.256 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.5HtMktgc5H 2024-05-10T10:43:57.338 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.5HtMktgc5H is 468KB 2024-05-10T10:43:57.435 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.5HtMktgc5H 2024-05-10T10:43:57.450 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/data/mon.c.tgz 2024-05-10T10:43:57.450 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-05-10T10:43:57.504 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.xtkqrIXT7S 2024-05-10T10:43:57.504 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xtkqrIXT7S 2024-05-10T10:43:57.657 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.xtkqrIXT7S 2024-05-10T10:43:57.740 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.xtkqrIXT7S is 468KB 2024-05-10T10:43:57.839 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.xtkqrIXT7S 2024-05-10T10:43:57.854 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-10T10:43:57.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-10T10:43:57.857 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-10T10:43:57.943 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-10T10:43:57.973 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-10T10:43:57.973 INFO:tasks.ceph:Archiving crash dumps... 2024-05-10T10:43:57.975 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/remote/smithi053/crash 2024-05-10T10:43:57.976 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-10T10:43:58.013 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/remote/smithi135/crash 2024-05-10T10:43:58.014 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-10T10:43:58.038 INFO:tasks.ceph:Shutting down logrotate 2024-05-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:58.058 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:58.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:58.173 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-10T10:43:58.197 INFO:tasks.ceph:Compressing logs... 2024-05-10T10:43:58.197 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T10:43:58.216 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-10T10:43:58.241 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70503.log 2024-05-10T10:43:58.290 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35946.log 2024-05-10T10:43:58.290 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42317.log 2024-05-10T10:43:58.290 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-10T10:43:58.290 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.35946.log: /var/log/ceph/ceph-client.3.42317.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35946.log.gz 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42317.log.gz 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35986.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.35986.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-osd.7.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35986.log.gz 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35775.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.35775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41921.log 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr: 88.6%/var/log/ceph/ceph-osd.4.log: -- replaced with /var/log/ceph/ceph-client.admin.35775.log.gz 2024-05-10T10:43:58.291 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35866.log 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.1.41921.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr: -- replaced with /var/log/ceph/ceph-client.1.41921.log.gz 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gzgzip 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.35866.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr: 94.0% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35866.log.gz 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42321.log 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35819.log 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.3.42321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42321.log.gz 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35906.log 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.35819.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35819.log.gz 2024-05-10T10:43:58.292 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35735.log 2024-05-10T10:43:58.293 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.35906.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35906.log.gz 2024-05-10T10:43:58.293 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41916.log 2024-05-10T10:43:58.293 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.35735.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35735.log.gz 2024-05-10T10:43:58.293 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-10T10:43:58.293 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.1.41916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41916.log.gz 2024-05-10T10:43:58.293 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-10T10:43:58.293 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-05-10T10:43:58.300 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.35695.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-10T10:43:58.301 INFO:teuthology.orchestra.run.smithi135.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-05-10T10:43:58.361 INFO:teuthology.orchestra.run.smithi135.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-10T10:43:58.406 INFO:teuthology.orchestra.run.smithi135.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-10T10:43:58.495 INFO:teuthology.orchestra.run.smithi135.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53670.log 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.70503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53412.log 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70503.log.gz 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63546.log 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55072.log 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53670.log.gz 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-05-10T10:43:58.571 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63546.log.gz 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53412.log.gz 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46468.log 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60855.log 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46512.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55072.log.gz 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67643.log 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46468.log.gz 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67643.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60855.log.gz 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57937.log 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67643.log.gz 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43147.log 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.57937.log: gzip 88.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz -- 2024-05-10T10:43:58.572 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.35771.log 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57937.log.gz 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62137.log 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43147.log.gz 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70207.log 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62137.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35771.log.gz 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65174.log 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59874.log 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70207.log.gz 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.65174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65174.log.gz 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.62137.log.gz 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70066.log 2024-05-10T10:43:58.573 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59874.log.gz 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62442.log 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66745.log 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70066.log.gz 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68510.log 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66745.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62442.log.gz 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61508.log 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66745.log.gz 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69308.log 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68510.log.gz 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66547.log 2024-05-10T10:43:58.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61508.log: /var/log/ceph/ceph-client.admin.69308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73716.log 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69308.log.gz 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54574.log 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66547.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61508.log.gz 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73716.log: gzip -5 --verbose 89.2% -- -- replaced with /var/log/ceph/ceph-client.admin.66547.log.gz 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.57707.log 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54574.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73716.log.gz 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70947.log 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54574.log.gz 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52507.log 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57707.log.gz 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45124.log 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62052.log 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.70947.log.gz 2024-05-10T10:43:58.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55114.log 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45124.log.gz 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52507.log.gz 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68762.log 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55114.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62052.log.gz 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69821.log 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55114.log.gz 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52213.log 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68762.log.gz 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.69821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48088.log 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52213.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69821.log.gz 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60963.log 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52213.log.gz 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-05-10T10:43:58.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60963.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48088.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63817.log 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60963.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71129.log 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46680.log: /var/log/ceph/ceph-client.admin.63817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53736.log 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63817.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49945.log 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69451.log 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.71129.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49945.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53736.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42816.log 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49945.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.69451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65178.log 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42816.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69451.log.gz 2024-05-10T10:43:58.577 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.65178.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42816.log.gz 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48849.log 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65178.log.gz 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55622.log 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48849.log: -- /var/log/ceph/ceph-client.admin.44599.log 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44466.log 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48849.log.gz 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55622.log.gz 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.44599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52765.log 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44599.log.gz 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.44466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52255.log 2024-05-10T10:43:58.578 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44466.log.gz 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.52765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60216.log 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52765.log.gz 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63588.log 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52255.log.gz 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.60216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51519.log 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60216.log.gz 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48536.log 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51519.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63588.log.gz 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46766.log 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48536.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51519.log.gz 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43348.log 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48536.log.gz 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48578.log 2024-05-10T10:43:58.579 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46766.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67349.log 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43348.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59227.log 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48578.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46991.log 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67349.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64370.log 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59227.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46179.log 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46991.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64414.log 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46179.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.64370.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46179.log.gz 2024-05-10T10:43:58.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52032.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64414.log.gz 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43486.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52032.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70753.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52032.log.gz 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66787.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57232.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70753.log.gz 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61574.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66787.log.gz 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61574.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57232.log.gz 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73858.log 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61574.log.gz 2024-05-10T10:43:58.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42550.log 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59061.log 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73858.log.gz 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48723.log 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42550.log.gz 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58549.log 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59061.log.gz 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61400.log 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48723.log.gz 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.58549.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50437.log 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71195.log 2024-05-10T10:43:58.582 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58549.log.gz 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42812.log 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% 86.2% -- replaced with /var/log/ceph/ceph-client.admin.61400.log.gz -- replaced with /var/log/ceph/ceph-client.admin.50437.log.gz 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44461.log 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63775.log 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42812.log.gz 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.44461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43487.log 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44461.log.gz 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72276.log 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.71195.log.gz 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43487.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63775.log.gz 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43936.log 2024-05-10T10:43:58.583 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43487.log.gz 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72276.log.gz 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52894.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43936.log.gz 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47033.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58981.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52894.log.gz 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63336.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55730.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58981.log.gz 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63336.log.gz 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.55730.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63901.log 2024-05-10T10:43:58.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46810.log 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55730.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63901.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40928.log 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63901.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60488.log 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40928.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46810.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64836.log 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40928.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67895.log 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60488.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60024.log 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67895.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.64836.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73464.log 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67895.log.gz 2024-05-10T10:43:58.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43488.log 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60024.log.gz 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57382.log 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43488.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.73464.log.gz -5 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73032.log 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43488.log.gz 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54051.log 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68468.log 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57382.log.gz 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73032.log.gz 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72360.log 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54051.log.gz 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54322.log 2024-05-10T10:43:58.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68468.log: /var/log/ceph/ceph-client.admin.72360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67109.log 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68468.log.gz 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54322.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72360.log.gz 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51603.log 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54322.log.gz 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57274.log 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67109.log.gz 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51603.log: /var/log/ceph/ceph-client.admin.52937.log 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43483.log 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51603.log.gz 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70161.log 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57274.log.gz 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43483.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52937.log.gz 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61155.log 2024-05-10T10:43:58.587 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43483.log.gz 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72906.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70161.log.gz 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.61155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61155.log.gz 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51477.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57340.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51477.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72906.log.gz 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48350.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51477.log.gz 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64920.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57340.log.gz 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51417.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54669.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48350.log.gz 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51417.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-10T10:43:58.588 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51417.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64920.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55412.log 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.audit.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54669.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52573.log 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48638.log 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55412.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65460.log 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48638.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52573.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49711.log 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48638.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.65460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42592.log 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65460.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49711.log.gz 2024-05-10T10:43:58.589 INFO:teuthology.orchestra.run.smithi053.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.42592.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63985.log 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50936.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42592.log.gz 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51188.log 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63985.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53628.log 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63985.log.gz 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61250.log 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51188.log.gz 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58489.log 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61250.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53628.log.gz 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62820.log 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64710.log 2024-05-10T10:43:58.590 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58489.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61250.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66184.log 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47285.log 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64710.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62820.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56262.log 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56100.log 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47285.log: /var/log/ceph/ceph-client.admin.56262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73075.log 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66184.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51062.log 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.56262.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47285.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56100.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51778.log 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73075.log.gz 2024-05-10T10:43:58.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50524.log 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51062.log.gz 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67391.log 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50524.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51778.log.gz 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59271.log 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58844.log 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67391.log.gz 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50524.log.gz 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59271.log.gz 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42814.log 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58844.log.gz 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66356.log 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-05-10T10:43:58.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52339.log 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66356.log: gzip -5 --verbose 87.9% -- -- replaced with /var/log/ceph/ceph-client.admin.42814.log.gz /var/log/ceph/ceph-client.admin.56432.log 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49150.log 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.66356.log.gz 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52339.log.gz 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65321.log 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56432.log.gz 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43978.log 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49150.log.gz 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.65321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56177.log 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43978.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65321.log.gz 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2024-05-10T10:43:58.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73167.log 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56177.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43978.log.gz 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48308.log 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54364.log 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73167.log.gz 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66872.log 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48308.log.gz 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45732.log 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66872.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54364.log.gz 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67043.log 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.66872.log.gz 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55021.log 2024-05-10T10:43:58.594 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45732.log.gz 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68384.log 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73117.log 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55021.log.gz 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68384.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67043.log.gz 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41352.log 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58676.log 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68384.log.gz 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73117.log: /var/log/ceph/ceph-client.admin.41352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64536.log 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73117.log.gz 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58676.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41352.log.gz 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58676.log.gz 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50978.log 2024-05-10T10:43:58.595 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64536.log.gz 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51020.log 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46945.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50978.log.gz 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73422.log 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55454.log 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51020.log: /var/log/ceph/ceph-client.admin.73422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69643.log 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51020.log.gz 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55454.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73422.log.gz 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43152.log 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66657.log 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55454.log.gz 2024-05-10T10:43:58.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.69643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69643.log.gz 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51698.log 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66657.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43152.log.gz 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49316.log 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66657.log.gz 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45858.log 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51698.log: /var/log/ceph/ceph-client.admin.49316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52083.log 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51698.log.gz 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45858.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49316.log.gz 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67685.log 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68070.log 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6%/var/log/ceph/ceph-client.admin.52083.log: -- replaced with /var/log/ceph/ceph-client.admin.45858.log.gz 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67685.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52083.log.gz 2024-05-10T10:43:58.597 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71852.log 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67685.log.gz 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54093.log 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68070.log: /var/log/ceph/ceph-client.0.71852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45816.log 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71852.log.gz 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68070.log.gz 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59958.log 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45816.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54093.log.gz 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56478.log 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45487.log 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45816.log.gz 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59958.log: /var/log/ceph/ceph-client.admin.56478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50133.log 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56478.log.gz 2024-05-10T10:43:58.598 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72402.log 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59958.log.gz 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.50133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46861.log 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50133.log.gz 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45487.log.gz 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46224.log 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46861.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72402.log.gz 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55580.log 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57941.log 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9%/var/log/ceph/ceph-client.admin.46224.log: -- replaced with /var/log/ceph/ceph-client.admin.46861.log.gz 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55580.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46224.log.gz 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60573.log 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55580.log.gz 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67517.log 2024-05-10T10:43:58.599 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.57941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57941.log.gz 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62972.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67517.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60573.log.gz 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70315.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67517.log.gz 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55496.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50674.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70315.log: /var/log/ceph/ceph-client.admin.55496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59315.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr: 90.8% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70315.log.gz -- replaced with /var/log/ceph/ceph-client.admin.62972.log.gz 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55496.log.gz 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48891.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71450.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50674.log.gz 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59315.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-10T10:43:58.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48891.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59315.log.gz 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48891.log.gz 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71450.log.gz 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41268.log 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62354.log 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45250.log: /var/log/ceph/ceph-client.admin.41268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53370.log 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41268.log.gz 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54448.log 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73380.log 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62354.log.gz/var/log/ceph/ceph-client.admin.53370.log: 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73818.log 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53370.log.gz 2024-05-10T10:43:58.601 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.73380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54448.log.gz 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62114.log 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73818.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73380.log.gz 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65398.log 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73818.log.gz 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70984.log 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62114.log.gz 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.65398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50362.log 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65398.log.gz 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59664.log 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62886.log 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50362.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.70984.log.gz 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55328.log 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50362.log.gz 2024-05-10T10:43:58.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62886.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59664.log.gz 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67990.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55328.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62886.log.gz 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66699.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55328.log.gz 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60550.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67990.log: /var/log/ceph/ceph-client.admin.66699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50309.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67990.log.gz 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60550.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66699.log.gz 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43894.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60550.log.gz 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62239.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43894.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50309.log.gz 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64326.log 2024-05-10T10:43:58.603 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43894.log.gz 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42808.log 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64326.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49360.log 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64326.log.gz 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42808.log.gz 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.62239.log.gz 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62742.log 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64668.log 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49360.log.gz 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.62742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62742.log.gz 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63733.log 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2024-05-10T10:43:58.604 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64668.log.gz 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63378.log 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63733.log.gz 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63039.log 89.3% 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gz 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73590.log 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63378.log.gz 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63039.log.gz 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58802.log 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73590.log: /var/log/ceph/ceph-client.admin.45166.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73590.log.gz 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54280.log 2024-05-10T10:43:58.605 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68342.log 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58802.log: gzip -5/var/log/ceph/ceph-client.admin.54280.log: --verbose -- /var/log/ceph/ceph-client.admin.58303.log 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68342.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54280.log.gz 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58802.log.gz 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68342.log.gz 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52171.log 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58303.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52171.log: /var/log/ceph/ceph-client.admin.67811.log 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52171.log.gz 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58303.log.gz 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68678.log 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67811.log.gz 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68678.log: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-10T10:43:58.606 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67132.log 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68678.log.gz 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53262.log 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57166.log 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53262.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67132.log.gz 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53262.log.gz 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71664.log 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60628.log 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64284.log 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57166.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71664.log.gz 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57448.log 2024-05-10T10:43:58.607 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61466.log 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57448.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64284.log.gz 91.4% -- replaced with /var/log/ceph/ceph-client.admin.60628.log.gz 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40884.log 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.57448.log.gz/var/log/ceph/ceph-client.admin.61466.log: 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71856.log 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61466.log.gz 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48392.log 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40884.log: /var/log/ceph/ceph-client.0.71856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54009.log 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71856.log.gz 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48392.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40884.log.gz 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.48008.log 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48392.log.gz 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61682.log 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54009.log.gz 2024-05-10T10:43:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58592.log 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48008.log: /var/log/ceph/ceph-client.admin.61682.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48008.log.gz 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59622.log 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61682.log.gz 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44907.log 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58592.log: /var/log/ceph/ceph-client.admin.59622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68804.log 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58592.log.gz 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.2.44907.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59622.log.gz 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44907.log.gz 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63420.log 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59832.log 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68804.log.gz 2024-05-10T10:43:58.609 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63420.log.gz 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49753.log 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68720.log 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59832.log.gz 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72631.log 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49753.log.gz 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71279.log 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68720.log.gz 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51859.log 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72631.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68298.log 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71279.log: /var/log/ceph/ceph-client.admin.51859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41888.log 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68298.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51859.log.gz 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68298.log.gz -5 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54237.log 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46426.log 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.71279.log.gz 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49539.log 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54237.log.gz 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49274.log 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.41888.log.gz 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46426.log.gz 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52129.log 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66959.log 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49539.log.gz 2024-05-10T10:43:58.611 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49274.log.gz 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53196.log 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52129.log: /var/log/ceph/ceph-client.admin.66959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48162.log 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52129.log.gz 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53196.log: gzip -5 --verbose -- 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66959.log.gz 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.53831.log 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54177.log 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr: 91.1% 90.4% -- replaced with /var/log/ceph/ceph-client.admin.48162.log.gz 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53196.log.gz 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64878.log 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67559.log 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54177.log.gz 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42809.log 2024-05-10T10:43:58.612 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64878.log.gz 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73548.log 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67559.log: /var/log/ceph/ceph-client.admin.42809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54890.log 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73548.log: 88.0% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67559.log.gz 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42809.log.gz 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73548.log.gz 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52381.log 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64626.log 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54890.log: /var/log/ceph/ceph-client.admin.52381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64626.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52381.log.gz 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54890.log.gz 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64626.log.gz 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40842.log 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54490.log 2024-05-10T10:43:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41184.log: /var/log/ceph/ceph-client.admin.40842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72141.log 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54490.log: 89.4% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40842.log.gz 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54490.log.gz 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45984.log 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66272.log 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72141.log: /var/log/ceph/ceph-client.admin.45984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60897.log 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66272.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45984.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72141.log.gz 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48199.log 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41460.log 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66272.log.gz 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60897.log: /var/log/ceph/ceph-client.admin.48199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65731.log 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41460.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48199.log.gz 2024-05-10T10:43:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65483.log 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41460.log.gz 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60897.log.gz 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.65731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65731.log.gz 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65993.log 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66071.log 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.65483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54830.log 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.65993.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65993.log.gz/var/log/ceph/ceph-client.admin.66071.log: 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65483.log.gz 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66071.log.gz 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41310.log 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54830.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53046.log 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54830.log.gz 2024-05-10T10:43:58.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.a.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41310.log.gz 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42804.log 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52423.log 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42804.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53046.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42804.log.gz 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42634.log 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52465.log 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52423.log.gz 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56390.log 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52465.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42634.log.gz 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52465.log.gz 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45655.log 2024-05-10T10:43:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55286.log 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56390.log.gz 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68552.log 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55286.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45655.log.gz 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55286.log.gz 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48494.log 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73506.log 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68552.log.gz 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44903.log 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73506.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48494.log.gz 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73506.log.gz 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69931.log 2024-05-10T10:43:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.2.44903.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71556.log 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.2.44903.log.gz 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.69931.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53562.log 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69931.log.gz 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72546.log 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71556.log.gz 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73295.log 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72546.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72546.log.gz 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53562.log.gz 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55244.log 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57749.log 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73295.log.gz 2024-05-10T10:43:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72444.log 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55244.log.gz 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57749.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57749.log.gz 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42382.log 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48933.log 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.72444.log.gz 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42382.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42382.log.gz 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58219.log 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70010.log 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48933.log.gz 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73900.log 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70010.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58219.log.gz 2024-05-10T10:43:58.619 INFO:teuthology.orchestra.run.smithi053.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.70010.log.gz 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51291.log 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49627.log 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73900.log: /var/log/ceph/ceph-client.admin.51291.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73900.log.gz 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51291.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71598.log 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69058.log 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71598.log: /var/log/ceph/ceph-client.admin.49627.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49627.log.gz 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43481.log 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71598.log.gz 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.69058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67264.log 2024-05-10T10:43:58.620 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0%/var/log/ceph/ceph-client.admin.43481.log: -- replaced with /var/log/ceph/ceph-client.0.69058.log.gz 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43481.log.gz 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58447.log 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60791.log 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67264.log.gz 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69639.log 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60791.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58447.log.gz 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60791.log.gz 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56939.log 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71237.log 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.69639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69639.log.gz 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48125.log 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56939.log.gz 2024-05-10T10:43:58.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67475.log 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48125.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.48125.log.gz 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr: 92.0%/var/log/ceph/ceph-client.admin.67475.log: -- replaced with /var/log/ceph/ceph-client.admin.71237.log.gz 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67475.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51901.log 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47868.log 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59706.log 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47868.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51901.log.gz 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64986.log 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47868.log.gz 2024-05-10T10:43:58.622 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41056.log 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59706.log: /var/log/ceph/ceph-client.admin.64986.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64986.log.gz 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59706.log.gz 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42340.log 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51943.log 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41056.log.gz 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60411.log 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51943.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42340.log.gz 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51943.log.gz 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51988.log 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59538.log 2024-05-10T10:43:58.623 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60411.log: /var/log/ceph/ceph-client.admin.51988.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60411.log.gz 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51988.log.gz 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59538.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59538.log.gz 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70873.log 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52761.log 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55370.log 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.52761.log: 0.0% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70873.log.gz 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.0.52761.log.gz 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45208.log 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71396.log 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55370.log: /var/log/ceph/ceph-client.admin.45208.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55370.log.gz 2024-05-10T10:43:58.624 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45208.log.gzgzip 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63147.log 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64579.log 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63147.log: /var/log/ceph/ceph-client.admin.71396.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63147.log.gz 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71396.log.gz 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70910.log 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72589.log 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64579.log.gz 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41394.log 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.70910.log.gz 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56520.log 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72589.log.gz 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65539.log 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56520.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41394.log.gz 2024-05-10T10:43:58.625 INFO:teuthology.orchestra.run.smithi053.stderr: 91.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56336.log 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56520.log.gz 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67306.log 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.65539.log: /var/log/ceph/ceph-client.admin.56336.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65539.log.gz 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67727.log 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56336.log.gz 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54932.log 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67306.log: /var/log/ceph/ceph-client.admin.67727.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67306.log.gz 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67727.log.gz 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57599.log 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72234.log 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54932.log: /var/log/ceph/ceph-client.admin.57599.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54932.log.gz 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57599.log.gz 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53112.log 2024-05-10T10:43:58.626 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72318.log 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53112.log: /var/log/ceph/ceph-client.admin.72234.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72234.log.gz 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43150.log 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53112.log.gz 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43768.log 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43150.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72318.log.gz 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43150.log.gz 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73338.log 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43768.log: -- /var/log/ceph/ceph-client.admin.46110.log 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43768.log.gz/var/log/ceph/ceph-client.admin.73338.log: 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68254.log 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73338.log.gz/var/log/ceph/ceph-client.admin.46110.log: 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2024-05-10T10:43:58.627 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46110.log.gz 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73674.log 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68254.log: /var/log/ceph/ceph-client.admin.43726.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68254.log.gz 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46068.log 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45774.log 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46068.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73674.log.gz 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46068.log.gz 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49795.log 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45774.log: -- /var/log/ceph/ceph-client.0.47543.log 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45774.log.gz 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72673.log 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.47543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47543.log.gz 2024-05-10T10:43:58.628 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49795.log.gz 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61834.log 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53478.log 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72673.log: /var/log/ceph/ceph-client.admin.61834.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72673.log.gz 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61834.log.gz 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64752.log 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53478.log: /var/log/ceph/ceph-client.admin.43141.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53478.log.gz 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42815.log 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66505.log 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64752.log: /var/log/ceph/ceph-client.admin.42815.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64752.log.gz 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42815.log.gz 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46722.log 2024-05-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70119.log 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66505.log: /var/log/ceph/ceph-client.admin.46722.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.66505.log.gz 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46722.log.gz 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48975.log 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70119.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56239.log 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70119.log.gz/var/log/ceph/ceph-client.admin.48975.log: 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49232.log 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48975.log.gz 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56774.log 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49232.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56239.log.gz 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49232.log.gz 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.56774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56774.log.gz 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42466.log 2024-05-10T10:43:58.630 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55664.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42466.log: /var/log/ceph/ceph-client.admin.44142.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42466.log.gz 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58886.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55664.log: -- /var/log/ceph/ceph-client.admin.45900.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58886.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55664.log.gz 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58886.log.gz 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42214.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68170.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45900.log.gz 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57641.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68170.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42214.log.gz 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68170.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47946.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63859.log 2024-05-10T10:43:58.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57641.log: /var/log/ceph/ceph-client.admin.47946.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47946.log.gz 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57641.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67601.log 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42424.log 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67601.log: /var/log/ceph/ceph-client.admin.63859.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63859.log.gz 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67601.log.gz 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56586.log 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42424.log.gz 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.56586.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56586.log.gz 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46026.log 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66603.log 2024-05-10T10:43:58.632 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47159.log: /var/log/ceph/ceph-client.admin.46026.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46026.log.gz 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66185.log 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66603.log: /var/log/ceph/ceph-client.admin.45334.log: 88.1% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66603.log.gz 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69385.log 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40678.log 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66185.log: /var/log/ceph/ceph-client.admin.69385.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66185.log.gz 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54532.log 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40678.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69385.log.gz 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40678.log.gz 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71492.log 2024-05-10T10:43:58.633 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57082.log 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54532.log.gz 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57082.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71492.log.gz 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59185.log 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr: 90.7%/var/log/ceph/ceph-client.admin.46304.log: -- replaced with /var/log/ceph/ceph-client.admin.57082.log.gz 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67769.log 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73209.log 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59185.log.gz 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55538.log 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73209.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67769.log.gz 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73209.log.gz 2024-05-10T10:43:58.634 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49497.log 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63462.log 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55538.log: /var/log/ceph/ceph-client.admin.49497.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55538.log.gz 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49497.log.gz 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56778.log 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42298.log 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.56778.log: /var/log/ceph/ceph-client.admin.63462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56778.log.gz 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63462.log.gz 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62197.log 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42298.log.gz 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62484.log 2024-05-10T10:43:58.635 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62197.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43480.log 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62484.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.62197.log.gz 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62484.log.gz 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49585.log 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49879.log 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43480.log: /var/log/ceph/ceph-client.admin.49585.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43480.log.gz 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49585.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.60212.log 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48765.log 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.60212.log: /var/log/ceph/ceph-client.admin.49879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60212.log.gz 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45942.log 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48765.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49879.log.gz 2024-05-10T10:43:58.636 INFO:teuthology.orchestra.run.smithi053.stderr:gzip 88.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.67853.log 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48765.log.gz 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64160.log 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45942.log: /var/log/ceph/ceph-client.admin.67853.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45942.log.gz 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72990.log 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67853.log.gz 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47539.log 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72990.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64160.log.gz 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72990.log.gz 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58345.log 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63691.log 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.47539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47539.log.gz 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35487.log 2024-05-10T10:43:58.637 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58345.log.gz 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59143.log 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.35487.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63691.log.gz 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62738.log 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35487.log.gz 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70507.log 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.62738.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59143.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.0.62738.log.gz 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.70507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70507.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64794.log 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55156.log 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46384.log: /var/log/ceph/ceph-client.admin.64794.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2024-05-10T10:43:58.638 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64794.log.gz 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51375.log 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55922.log 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55156.log.gz 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61876.log 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.55922.log: 88.7% 0.0% -- replaced with /var/log/ceph/ceph-client.0.55922.log.gz 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50266.log 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51104.log 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50266.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61876.log.gz 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50266.log.gz 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43144.log 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51104.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51104.log.gz 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61151.log 2024-05-10T10:43:58.639 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43143.log 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43144.log: /var/log/ceph/ceph-client.0.61151.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43144.log.gz 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61151.log.gz 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41226.log 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50436.log 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43143.log: /var/log/ceph/ceph-client.admin.41226.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43143.log.gz 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41226.log.gz 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59790.log 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72486.log 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59790.log: /var/log/ceph/ceph-client.admin.50436.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59790.log.gz 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29470.log 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50436.log.gz 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72486.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72486.log.gz 2024-05-10T10:43:58.640 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41100.log 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48807.log 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.tmp-client.admin.29470.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29470.log.gz 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59408.log 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48807.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41100.log.gz 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48807.log.gz 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40800.log 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71106.log 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59408.log: /var/log/ceph/ceph-client.admin.40800.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59408.log.gz 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40800.log.gz 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72715.log 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54977.log 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72715.log: /var/log/ceph/ceph-client.admin.71106.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71106.log.gz 2024-05-10T10:43:58.641 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72715.log.gz 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58760.log 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68636.log 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54977.log: /var/log/ceph/ceph-client.admin.58760.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54977.log.gz 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58760.log.gz 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54135.log 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63648.log 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68636.log.gz 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50717.log 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63648.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54135.log.gz 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63648.log.gz 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63242.log 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70830.log 2024-05-10T10:43:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63242.log: /var/log/ceph/ceph-client.admin.50717.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63242.log.gz 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50717.log.gz 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47351.log 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70830.log.gz 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62396.log 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47351.log.gz 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62550.log 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz/var/log/ceph/ceph-client.admin.62396.log: 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73253.log 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62550.log: -- replaced with /var/log/ceph/ceph-client.admin.62396.log.gz 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49404.log 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62550.log.gz 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59450.log 2024-05-10T10:43:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49404.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73253.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49404.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53925.log 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49837.log 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59450.log: /var/log/ceph/ceph-client.admin.53925.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59450.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68594.log 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53925.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60695.log 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49837.log: /var/log/ceph/ceph-client.admin.68594.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49837.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45040.log 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68594.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58718.log 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60695.log: /var/log/ceph/ceph-client.admin.45040.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45040.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60695.log.gz 2024-05-10T10:43:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64242.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59496.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58718.log.gz 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57016.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59496.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64242.log.gz 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59496.log.gz 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59580.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52990.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59580.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57016.log.gz 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59580.log.gz 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61616.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59916.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52990.log.gz 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57533.log 2024-05-10T10:43:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59916.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59916.log.gz 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61616.log.gz 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62300.log 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.57533.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.62300.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57533.log.gz 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62300.log.gz 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46638.log 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59748.log 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46638.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46638.log.gz 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72948.log 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr: 92.3%/var/log/ceph/ceph-client.admin.59748.log: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59748.log.gz 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54749.log 2024-05-10T10:43:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.72948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63504.log 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72948.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58261.log 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63504.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54749.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63504.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68212.log 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.40972.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58261.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45082.log 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55918.log 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68212.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52297.log 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2%/var/log/ceph/ceph-client.0.55918.log: -- replaced with /var/log/ceph/ceph-client.admin.45082.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55918.log.gz 2024-05-10T10:43:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73758.log 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.52297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68426.log 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52297.log.gz 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61316.log 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68426.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73758.log.gz 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42508.log 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68426.log.gz 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71040.log 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42508.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42508.log.gz 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61316.log.gz 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61974.log 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.71040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45292.log 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61974.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71040.log.gz 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61974.log.gz 2024-05-10T10:43:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49669.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42256.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45292.log.gz 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.49669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49669.log.gz 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.42079.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42256.log.gz 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63081.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gz 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41014.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.47117.log: /var/log/ceph/ceph-client.admin.63081.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58017.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.41014.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63081.log.gz 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41014.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58405.log 2024-05-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65727.log 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.58017.log.gz 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64080.log 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.65727.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58405.log.gz 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65727.log.gz 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45565.log 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58126.log 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64080.log.gz 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.45565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63943.log 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7%/var/log/ceph/ceph-client.admin.58126.log: -- replaced with /var/log/ceph/ceph-client.admin.45565.log.gz 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42137.log 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58126.log.gz 2024-05-10T10:43:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64494.log 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.63943.log: /var/log/ceph/ceph-client.admin.42137.log: 89.3% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63943.log.gz 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42137.log.gz 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69062.log 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.64494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66445.log 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64494.log.gz 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.69062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69062.log.gz 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48051.log 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48681.log 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48051.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.66445.log.gz 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.48051.log.gz 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68870.log 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53304.log 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48681.log.gz 2024-05-10T10:43:58.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.68870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66871.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53304.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68870.log.gz 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53304.log.gz 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.66871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66422.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43684.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66871.log.gz 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53967.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.53967.log: /var/log/ceph/ceph-client.admin.66422.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66422.log.gz 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53967.log.gz 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51333.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.50842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46903.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70249.log 2024-05-10T10:43:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.46903.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51333.log.gz 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60749.log 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46903.log.gz 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.70249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55202.log 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.60749.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60749.log.gz 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70249.log.gz 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50137.log 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.55202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51561.log 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55202.log.gz 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.50137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50137.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67433.log 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.51561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43151.log 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51561.log.gz 2024-05-10T10:43:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.67433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.43151.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67433.log.gz 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43151.log.gz 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54406.log 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43810.log: -- /var/log/ceph/ceph-client.admin.58634.log 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.54406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73632.log 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.58634.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54406.log.gz 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43484.log 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58634.log.gz 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35854.log 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.73632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48452.log 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0%/var/log/ceph/ceph-client.admin.43484.log: /var/log/ceph/ceph-client.admin.35854.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43484.log.gz 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73632.log.gz 2024-05-10T10:43:58.654 INFO:teuthology.orchestra.run.smithi053.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35854.log.gz 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59366.log 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.48452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61768.log 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48452.log.gz 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.59366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43852.log 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.61768.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59366.log.gz 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi053.stderr: 90.7%/var/log/ceph/ceph-client.admin.43852.log: -- replaced with /var/log/ceph/ceph-client.admin.61768.log.gz 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43852.log.gz 2024-05-10T10:43:58.655 INFO:teuthology.orchestra.run.smithi135.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-10T10:43:58.729 INFO:teuthology.orchestra.run.smithi053.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-10T10:43:59.221 INFO:teuthology.orchestra.run.smithi053.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-10T10:43:59.447 INFO:teuthology.orchestra.run.smithi135.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-10T10:43:59.517 INFO:teuthology.orchestra.run.smithi135.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-10T10:43:59.530 INFO:teuthology.orchestra.run.smithi053.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-10T10:43:59.578 INFO:teuthology.orchestra.run.smithi135.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-10T10:43:59.609 INFO:teuthology.orchestra.run.smithi135.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-10T10:43:59.612 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-05-10T10:43:59.612 INFO:teuthology.orchestra.run.smithi135.stderr:real 0m1.360s 2024-05-10T10:43:59.612 INFO:teuthology.orchestra.run.smithi135.stderr:user 0m5.719s 2024-05-10T10:43:59.612 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m0.209s 2024-05-10T10:43:59.718 INFO:teuthology.orchestra.run.smithi053.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-10T10:43:59.760 INFO:teuthology.orchestra.run.smithi053.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-10T10:43:59.890 INFO:teuthology.orchestra.run.smithi053.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-10T10:43:59.892 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-10T10:43:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m1.661s 2024-05-10T10:43:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m7.063s 2024-05-10T10:43:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.584s 2024-05-10T10:43:59.892 INFO:tasks.ceph:Archiving logs... 2024-05-10T10:43:59.892 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/remote/smithi053/log 2024-05-10T10:43:59.894 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T10:44:01.319 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/remote/smithi135/log 2024-05-10T10:44:01.320 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T10:44:02.199 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T10:44:02.208 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T10:44:02.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T10:44:02.211 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T10:44:02.283 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T10:44:02.293 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T10:44:02.293 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:44:02.296 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T10:44:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:44:02.312 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-05-10T10:44:02.316 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:44:02.316 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-10T10:44:02.316 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +899us[ +913us] +/- 57ms 2024-05-10T10:44:02.316 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1590us[-1590us] +/- 68ms 2024-05-10T10:44:02.316 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +596us[ +596us] +/- 74ms 2024-05-10T10:44:02.317 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:44:02.318 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T10:44:02.318 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-05-10T10:44:02.318 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +884us[ +890us] +/- 57ms 2024-05-10T10:44:02.318 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 -1619us[-1612us] +/- 68ms 2024-05-10T10:44:02.318 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +559us[ +559us] +/- 74ms 2024-05-10T10:44:02.318 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T10:44:02.319 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T10:44:02.328 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T10:44:02.329 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T10:44:02.338 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T10:44:02.448 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T10:44:02.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:44:02.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:44:02.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-10T10:44:02.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:44:02.576 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T10:44:02.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T10:44:02.596 INFO:teuthology.task.internal:Duration was 1058.348714 seconds 2024-05-10T10:44:02.596 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T10:44:02.608 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T10:44:02.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T10:44:02.611 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T10:44:02.651 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:44:02.656 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T10:44:02.913 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T10:44:02.913 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-05-10T10:44:02.913 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T10:44:02.975 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-05-10T10:44:02.975 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T10:44:02.999 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T10:44:02.999 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T10:44:03.018 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T10:44:03.066 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T10:44:03.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T10:44:03.086 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T10:44:03.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T10:44:03.224 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T10:44:03.224 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T10:44:03.250 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T10:44:03.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T10:44:03.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:44:03.293 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:44:03.314 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-05-10T10:44:03.338 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-05-10T10:44:03.352 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:44:03.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:44:03.384 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T10:44:03.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T10:44:03.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T10:44:03.421 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T10:44:03.422 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/remote/smithi053 2024-05-10T10:44:03.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T10:44:03.462 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909/remote/smithi135 2024-05-10T10:44:03.462 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T10:44:03.495 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T10:44:03.496 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T10:44:03.498 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T10:44:03.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T10:44:03.555 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T10:44:03.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T10:44:03.566 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T10:44:03.566 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T10:44:03.568 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T10:44:03.581 INFO:teuthology.orchestra.run.smithi053.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 10:44 /home/ubuntu/cephtest 2024-05-10T10:44:03.600 INFO:teuthology.orchestra.run.smithi135.stdout: 265907 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 10:44 /home/ubuntu/cephtest 2024-05-10T10:44:03.602 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T10:44:03.613 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T10:44:03.634 INFO:teuthology.nuke:Checking targets against current locks 2024-05-10T10:44:03.661 DEBUG:teuthology.nuke:shortname: smithi053 2024-05-10T10:44:03.662 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:44:03.684 DEBUG:teuthology.nuke:shortname: smithi135 2024-05-10T10:44:03.684 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T10:44:03.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:18:22.699854', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK170pW49litN+5EbfmOAYDBcgpt4N1Jca05xmku3ldzuDjm5Yj2KxZb/DnlrBojCr2amhUs1bsPFiBxvhTOFfI='} 2024-05-10T10:44:03.711 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2024-05-10_07:35:43-fs-wip-63896-from-wip-rishabh-testing-20240501.193033-distro-default-smithi/7700909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 10:18:22.698109', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPLvx9cv2KhJqM1O6dcHDgOYHlOVLnnfhQIv2H3uj3O9SvjWai9Cj26e45Cofe6oBhrPFOKX32Bn3RR8OcmV5Js='} 2024-05-10T10:44:03.738 INFO:teuthology.orchestra.console.smithi053:Power off 2024-05-10T10:44:03.738 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T10:44:03.762 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-05-10T10:44:03.763 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:44:03.780 INFO:teuthology.orchestra.console.smithi135:Power off 2024-05-10T10:44:03.780 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-10T10:44:03.790 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-05-10T10:44:03.804 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off 2024-05-10T10:44:03.804 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:44:03.828 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2024-05-10T10:44:07.791 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:44:07.829 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:44:07.903 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-05-10T10:44:07.939 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2024-05-10T10:44:11.905 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:44:11.940 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:44:12.015 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-05-10T10:44:12.049 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2024-05-10T10:44:12.117 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-05-10T10:44:16.050 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-10T10:44:16.160 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off 2024-05-10T10:44:16.260 INFO:teuthology.orchestra.console.smithi135:Power off completed 2024-05-10T10:44:16.367 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1058.3487141132355 failure_reason: '"2024-05-10T10:39:18.036714+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_dparmar@teuthology success: false 2024-05-10T10:44:16.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T10:44:16.451 INFO:teuthology.run:FAIL